Vehicle-road cooperation method based on pedestrian space-time behavior stack
Patent Information
- Application Number
- CN202510696130.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-28
- Publication Date
- 2025-09-19
Smart Images
Figure CN120670879A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving computing, and in particular to a vehicle-road collaboration method based on a pedestrian spatiotemporal behavior stack. Background Art
[0002] With the advancement of science and technology, the deep integration of new energy technologies and intelligent connected vehicles has driven breakthroughs in autonomous driving technology. Amidst the rapid industry evolution, the industry is accelerating road validation of higher-level autonomous driving technologies. Building a safe operational system for intelligent vehicles has become a key industry issue. This is particularly challenging for perception systems in complex intersections and specific scenarios such as pullovers. Multi-target tracking and occlusion handling are key, potentially requiring multi-sensor fusion and algorithm optimization. Furthermore, pedestrian intention prediction is highly uncertain, especially with sudden behavioral changes, requiring high-precision models and real-time data updates. Furthermore, the unstructured interactions between autonomous vehicles and pedestrians require a combination of game theory and machine learning to model these dynamic interactions. Complex scene understanding, such as at intersections without traffic lights, requires understanding traffic rules and social norms, which involves multimodal data fusion and vehicle-to-everything (V2X) communication. Furthermore, the real-time and safety of decision-making and planning are crucial. How can we quickly generate safe paths in dynamic environments while balancing safety and efficiency? Furthermore, extreme weather and lighting conditions can affect sensor performance, necessitating redundant design and data augmentation. Summary of the Invention
[0003] In view of this, the present invention provides a vehicle-road collaboration method based on the pedestrian spatiotemporal behavior stack to solve the technical problem that the existing technology cannot ensure safe and reasonable interaction between autonomous driving and pedestrians, without causing safety accidents such as collisions, while ensuring efficient autonomous driving.
[0004] The present invention provides a vehicle-road collaboration method based on a pedestrian spatiotemporal behavior stack, which is applied to an autonomous driving vehicle. The method comprises: step 1, using a road test device to collect pedestrian motion trajectories in different time periods and weather conditions in the same scene, and combining the physical grid in the scene to extract motion spatiotemporal features in partitions to form a feature data set; step 2, based on the data set, using multi-layer graph convolution to update node features to obtain a high-order spatiotemporal feature matrix; step 3, using a spatiotemporal distance metric and a clustering method, combining the feature similarity and spatiotemporal proximity of the high-order spatiotemporal feature matrix to obtain a pedestrian spatiotemporal behavior stack S = {(L k ,{T k},C k )}, where L k is the behavior label, {T k} is the trajectory set, C kis the cluster center feature; in step 4, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the receiving vehicle, obtains the pedestrian behavior label and trajectory set in the pedestrian spatiotemporal behavior stack corresponding to the focus area, and sends it to the vehicle, thereby enhancing the vehicle's risk identification and handling capabilities in complex scenarios.
[0005] Furthermore, the data set D={(t i ,x i ,y i ,C i ,W i ,T i )},t i Corresponding to the time of data collection, x i 、y i Corresponding position, C i Physical grid partition, W i Corresponding weather information, T i Represents the joint points of the pedestrian skeleton.
[0006] Furthermore, the time periods include morning, noon, and evening; the weather includes sunny, cloudy, and rainy; the scenes include intersections, T-junctions, diverging points, merging points, and pull-over points in specific scenarios; and the pedestrian spatiotemporal behavior stack includes behavior labels and corresponding trajectory sets.
[0007] Furthermore, the step 2 includes: step 21, according to the formula , calculate the adjacency matrix weight, where d ij represents the Euclidean distance between nodes i and j, Δt ij represents the time difference, σ d , σ t is the Gaussian kernel parameter; step 22, use multi-layer graph convolution , update the node features, where p(x,i,j) represents the sampling function at the spatial position X of a single frame in the dataset D, and w(i,j) is the adjacency matrix weight; Step 23, based on the updated node features and the adjacency matrix weight, obtain the high-order spatiotemporal feature matrix , where N is the number of nodes and F is the feature dimension.
[0008] Furthermore, the Gaussian parameter σ d =5.0 m,σ t =2.0 s.
[0009] Furthermore, the space-time distance metric is a space-time distance function , where α is the weight ratio of feature to time, d feat is the maximum distance in feature space, t max is the maximum time window, i and j are nodes, H is the updated node feature, D st (pi ,p j ) is the value of the space-time distance function at point (i, j).
[0010] Furthermore, the weight ratio of the feature to time is α=0.6, and the maximum time window t max =30 s.
[0011] Furthermore, the clustering method is: when D st (p i ,p j )<ε and TPts≥MinPts, all points that meet the conditions are merged into the same cluster, where D st (p i ,p j ) is the spatiotemporal distance function value at point (i, j), ε is the threshold, TPts is the number of points in the domain, and MinPts is the minimum number of points.
[0012] Furthermore, the minimum number of points MinPts=5.
[0013] Furthermore, step 4 includes: step 41, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the vehicle; step 42, screening the actual detected pedestrian positions within the focus area; step 43, matching the physical grid area corresponding to the focus area; step 44, obtaining the corresponding pedestrian spatiotemporal behavior stack behavior label and trajectory set, and sending them to the vehicle.
[0014] This invention provides a vehicle-road collaboration method based on a pedestrian spatiotemporal behavior stack. This technical solution adopts a new safety paradigm of "vehicle-road" collaboration. Through the deep integration of roadside perception equipment and autonomous driving, roadside sensors are used to collect pedestrian trajectories and cluster them to generate a spatiotemporal stack (including behavior labels, physical partitions, and trajectory sets). Combined with road-tested pedestrian detection and autonomous driving expected trajectory matching areas of interest, relevant behavior data is obtained and sent to the vehicle, improving the risk identification ability in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a flow chart of a vehicle-road collaboration method based on a pedestrian spatiotemporal behavior stack provided by the present invention; Figure 2 This is a flow chart of a method for obtaining a high-order spatiotemporal feature matrix provided by the present invention; Figure 3 This is a flow chart of a method provided by the present invention for sending pedestrian behavior labels and trajectory sets in a pedestrian spatiotemporal behavior stack corresponding to a focus area to a vehicle. DETAILED DESCRIPTION
[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0017] Example 1: The present invention provides a vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack, which is applied to autonomous driving vehicles, such as Figure 1 As shown, the method includes the following steps.
[0018] Step 1: Use road test equipment to collect pedestrian movement trajectories in different time periods and weather conditions under the same scene. Combined with the physical grid in the scene, the temporal and spatial characteristics of movement are extracted in different partitions to form a feature dataset. Step 2: Based on the dataset, multi-layer graph convolution is used to update node features to obtain a high-order spatiotemporal feature matrix; Step 3: Using the spatiotemporal distance metric and clustering method, combined with the feature similarity and spatiotemporal proximity of the high-order spatiotemporal feature matrix, we can obtain the spatiotemporal behavior stack S = {(L k ,{T k},C k )}, where L k is the behavior label, {T k} is the trajectory set, C k is the cluster center feature; In step 4, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the receiving vehicle, obtains the pedestrian behavior label and trajectory set in the pedestrian spatiotemporal behavior stack corresponding to the focus area, and sends it to the vehicle, thereby enhancing the vehicle's risk identification and handling capabilities in complex scenarios.
[0019] The present invention provides a vehicle-road collaboration method based on the spatiotemporal behavior stack of pedestrians. Through roadside equipment, the movement trajectories of pedestrians in different time periods of morning, noon and evening at intersections and merging and diverging traffic scenarios are collected to form a feature data set. Then, a clustering method is used to obtain the spatiotemporal behavior stack of pedestrians in this type of scenario. The spatiotemporal behavior stack of pedestrians contains behavior labels, physical grid partitions and corresponding trajectory sets. In actual scenarios, the road test equipment generates a focus area based on the received expected trajectory of autonomous driving, and then compares the actual detected pedestrian positions in the area with the physical grid partitions corresponding to the focus area to obtain the pedestrian behavior labels and trajectory sets of interest in the spatiotemporal behavior stack, and sends them to the autonomous driving vehicle, thereby enhancing the risk identification and handling capabilities in complex scenarios.
[0020] Example 2: The present invention provides a vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack, which is applied to autonomous driving vehicles, such as Figure 1 As shown, the method includes the following steps.
[0021] Step 1: Use road test equipment to collect pedestrian movement trajectories in different time periods and weather conditions under the same scene. Combined with the physical grid in the scene, the temporal and spatial characteristics of movement are extracted in different partitions to form a feature dataset. The drive test equipment includes a drive test sensor, a power supply, a processor, etc. The technical solution mainly collects data through the drive test sensor. The data set D = {(t i ,x i ,y i ,C i ,W i ,T i )},t i Corresponding to the time of data collection, x i 、y i Corresponding position, C i Physical grid partition, W i Corresponding weather information, T i Represents the joints of a pedestrian skeleton. The time periods include morning, noon, and evening; the weather includes sunny, cloudy, and rainy; the scenes include intersections, T-junctions, diverging points, converging points, and specific pull-over points. The pedestrian spatiotemporal behavior stack includes behavior labels and corresponding trajectory sets, as shown in Table 1.
[0022]
[0023] Table 1 Dataset description Step 2: Based on the dataset, multi-layer graph convolution is used to update node features to obtain a high-order spatiotemporal feature matrix; like Figure 2 As shown, the step 2 includes: Step 21, according to the formula , calculate the adjacency matrix weight, where d ij represents the Euclidean distance between nodes i and j, Δt ij represents the time difference, σ d , σ t is the Gaussian kernel parameter; The Gaussian parameter is used to control the spatial and temporal attenuation and is usually set to σ d =5.0 m,σ t =2.0 s.
[0024] Step 22: Use multi-layer graph convolution , update the node features, where p(x,i,j) represents the sampling function at the spatial position X of a single frame in the dataset D, and w(i,j) is the adjacency matrix weight; The sampling function p(x,i,j) is used to evaluate whether the information aggregated from the neighborhood nodes is within the partitioned neighborhood range. If so, it outputs 1, otherwise it outputs 0; Step 23: Based on the updated node features and adjacency matrix weights, a high-order spatiotemporal feature matrix is obtained. , where N is the number of nodes and F is the feature dimension.
[0025] Step 3: Using the spatiotemporal distance metric and clustering method, combined with the feature similarity and spatiotemporal proximity of the high-order spatiotemporal feature matrix, we can obtain the spatiotemporal behavior stack S = {(L k ,{T k},C k )}, where L k is the behavior label, {T k} is the trajectory set, C k is the cluster center feature; In step 4, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the receiving vehicle, obtains the pedestrian behavior label and trajectory set in the pedestrian spatiotemporal behavior stack corresponding to the focus area, and sends it to the vehicle, thereby enhancing the vehicle's risk identification and handling capabilities in complex scenarios.
[0026] The present invention provides a vehicle-road collaboration method based on the spatiotemporal behavior stack of pedestrians. Through roadside equipment, the movement trajectories of pedestrians in different time periods of morning, noon and evening at intersections and merging and diverging traffic scenarios are collected to form a feature data set. Then, a clustering method is used to obtain the spatiotemporal behavior stack of pedestrians in this type of scenario. The spatiotemporal behavior stack of pedestrians contains behavior labels, physical grid partitions and corresponding trajectory sets. In actual scenarios, the road test equipment generates a focus area based on the received expected trajectory of autonomous driving, and then compares the actual detected pedestrian positions in the area with the physical grid partitions corresponding to the focus area to obtain the pedestrian behavior labels and trajectory sets of interest in the spatiotemporal behavior stack, and sends them to the autonomous driving vehicle, thereby enhancing the risk identification and handling capabilities in complex scenarios.
[0027] Example 3: The present invention provides a vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack, which is applied to autonomous driving vehicles, such as Figure 1 As shown, the method includes the following steps.
[0028] Step 1: Use road test equipment to collect pedestrian movement trajectories in different time periods and weather conditions under the same scene. Combined with the physical grid in the scene, the temporal and spatial characteristics of movement are extracted in different partitions to form a feature dataset. Step 2: Based on the dataset, multi-layer graph convolution is used to update node features to obtain a high-order spatiotemporal feature matrix; Step 3: Using the spatiotemporal distance metric and clustering method, combined with the feature similarity and spatiotemporal proximity of the high-order spatiotemporal feature matrix, we can obtain the spatiotemporal behavior stack S = {(L k ,{T k},C k )}, where L k is the behavior label, {T k} is the trajectory set, C k is the cluster center feature; The spatiotemporal distance metric is a spatiotemporal distance function , where α is the weight ratio of feature to time, d feat is the maximum distance in feature space, t max is the maximum time window, i and j are nodes, H is the updated node feature, D st (p i ,p j ) is the value of the space-time distance function at point (i, j). The weight ratio of the feature to time is α=0.6, and the maximum time window t max =30 s. The clustering method is: when D st (p i ,p j )<ε and TPts≥MinPts, all points that meet the conditions are merged into the same cluster, where D st (p i ,p j ) is the spatiotemporal distance function value at point (i, j), ε is the threshold, TPts is the number of points in the domain, and MinPts is the minimum number of points. The minimum number of points MinPts=5. This step uses the spatiotemporal features H extracted by graph convolution to generate behavior labels and trajectory sets, providing a priori knowledge base for subsequent real-time matching. In addition, the pedestrian behavior label L k As shown in Table 2.
[0029]
[0030] Table 2 Pedestrian behavior label classification In step 4, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the receiving vehicle, obtains the pedestrian behavior label and trajectory set in the pedestrian spatiotemporal behavior stack corresponding to the focus area, and sends it to the vehicle, thereby enhancing the vehicle's risk identification and handling capabilities in complex scenarios.
[0031] As we know from the above, the pedestrian spatiotemporal behavior stack S includes the cluster center feature C k , and C k Also associated with the physical grid partition C in the scene i, so we can use the cluster center feature C k , the area of interest is mapped to the physical grid area, such as Figure 3 As shown, step 4 includes: Step 41: The road test equipment generates a region of interest based on the expected autonomous driving trajectory sent by the vehicle. Step 42, screening the actual detected pedestrian positions in the area of interest; Step 43, matching the physical grid area corresponding to the area of interest; Step 44: Obtain the pedestrian label trajectory set in the corresponding pedestrian spatiotemporal behavior stack and send it to the vehicle.
[0032] The present invention provides a vehicle-road collaboration method based on the spatiotemporal behavior stack of pedestrians. Through roadside equipment, the movement trajectories of pedestrians in different time periods of morning, noon and evening at intersections and merging and diverging traffic scenarios are collected to form a feature data set. Then, a clustering method is used to obtain the spatiotemporal behavior stack of pedestrians in this type of scenario. The spatiotemporal behavior stack of pedestrians contains behavior labels, physical grid partitions and corresponding trajectory sets. In actual scenarios, the road test equipment generates a focus area based on the received expected trajectory of autonomous driving, and then compares the actual detected pedestrian positions in the area with the physical grid partitions corresponding to the focus area to obtain the pedestrian behavior labels and trajectory sets of interest in the spatiotemporal behavior stack, and sends them to the autonomous driving vehicle, thereby enhancing the risk identification and handling capabilities in complex scenarios.
[0033] In summary, an embodiment of the present invention provides a vehicle-road collaboration method based on the pedestrian spatiotemporal behavior stack. This technical solution obtains the pedestrian's real-time behavior label and motion trajectory by querying the pedestrian's spatiotemporal behavior stack, thereby improving processing efficiency and data accuracy and validity, and sending it to the autonomous driving vehicle through the V2X device, thereby expanding the perception range of the single-vehicle autonomous driving and assisting it in obtaining accurate and effective prior information, thereby improving the safety and intelligence of the single-vehicle autonomous driving.
[0034] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A vehicle-road collaboration method based on a pedestrian spatiotemporal behavior stack, applied to an autonomous vehicle, characterized by: The method comprises: Step 1: Use road test equipment to collect pedestrian movement trajectories in different time periods and weather conditions under the same scene. Combined with the physical grid of the scene, the temporal and spatial characteristics of the movement are extracted in different partitions to form a feature dataset. Step 2: Based on the dataset, multi-layer graph convolution is used to update node features to obtain a high-order spatiotemporal feature matrix; Step 3: Using the spatiotemporal distance metric and clustering method, combined with the feature similarity and spatiotemporal proximity of the high-order spatiotemporal feature matrix, we can obtain the spatiotemporal behavior stack S = {(L k ,{T k },C k )}, where L k is the behavior label, {T k } is the trajectory set, C k is the cluster center feature; In step 4, the road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the receiving vehicle, obtains the pedestrian behavior label and trajectory set in the pedestrian spatiotemporal behavior stack corresponding to the focus area, and sends it to the vehicle, thereby enhancing the vehicle's risk identification and handling capabilities in complex scenarios.
2. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 1, characterized in that: The data set D={(t i ,x i ,y i ,C i ,W i ,T i )},t i Corresponding to the time of data collection, x i 、y i Corresponding position, C i Physical grid partition, W i Corresponding weather information, T i Represents the joint points of the pedestrian skeleton.
3. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 1, characterized in that: The time periods include morning, noon, and evening; the weather includes sunny, cloudy, and rainy; the scenes include intersections, T-junctions, diverging points, merging points, and pull-over points in specific scenarios; and the pedestrian spatiotemporal behavior stack includes behavior labels and corresponding trajectory sets.
4. The vehicle-road collaboration method based on the pedestrian spatiotemporal behavior stack according to claim 1, characterized in that: The step 2 includes: Step 21, according to the formula , calculate the adjacency matrix weight, where d ij represents the Euclidean distance between nodes i and j, Δt ij represents the time difference, σ d , σ t is the Gaussian kernel parameter; Step 22: Use multi-layer graph convolution , update the node features, where p(x,i,j) represents the sampling function at the spatial position X of a single frame in the dataset D, and w(i,j) is the adjacency matrix weight; Step 23: Based on the updated node features and adjacency matrix weights, a high-order spatiotemporal feature matrix is obtained. , where N is the number of nodes and F is the feature dimension.
5. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 4, characterized in that: The Gaussian parameter σ d =5.0 m,σ t =2.0 s.
6. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 1, characterized in that: The spatiotemporal distance metric is a spatiotemporal distance function , where α is the weight ratio of feature to time, d feat is the maximum distance in feature space, t max is the maximum time window, i and j are nodes, H is the updated node feature, D st (p i ,p j ) is the value of the space-time distance function at point (i, j).
7. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 6, characterized in that: The weight ratio of the feature to time is α=0.6, and the maximum time window t max =30 s.
8. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 1, characterized in that: The clustering method is: when D st (p i ,p j )<ε and TPts≥MinPts, all points that meet the conditions are merged into the same cluster, where D st (p i ,p j ) is the spatiotemporal distance function value at point (i, j), ε is the threshold, TPts is the number of points in the domain, and MinPts is the minimum number of points.
9. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 8, characterized in that: The minimum number of points MinPts=5.
10. The vehicle-road collaboration method based on pedestrian spatiotemporal behavior stack according to claim 2, characterized in that: The step 4 comprises: Step 41: The road test equipment generates a focus area based on the expected autonomous driving trajectory sent by the vehicle; Step 42, screening the actual detected pedestrian positions in the area of interest; Step 43, matching the physical grid area corresponding to the area of interest; Step 44: Obtain the corresponding pedestrian behavior label and trajectory set in the spatiotemporal behavior stack and send them to the vehicle.