A method and apparatus for trajectory extraction and annotation based on roadside data.

Abstract

This invention relates to a method and apparatus for trajectory extraction and annotation based on roadside data. The method includes: receiving a roadside sensing dataset; performing abnormal trajectory cleaning, stationary trajectory smoothing, abnormal turning trajectory repair, and heading angle correction on the roadside sensing dataset; adding lane features, segmenting and identifying trajectory behavior types, and segmenting behavior fragment trajectories for all trajectories in the roadside sensing dataset, and constructing an annotation record set based on all fragment trajectories; identifying the motion game relationship between every two target trajectories in the annotation record set to obtain a game relationship matrix; and generating a multi-objective game scenario based on the game relationship matrix and the annotation record set. This invention enables the construction of numerous complex game scenarios based on real roadside sensing data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a processing method and apparatus for trajectory extraction and annotation based on roadside data. Background Technology

[0002] The optimization path of autonomous driving systems' models / algorithms is highly dependent on typical traffic scenarios, especially complex game-theoretic scenarios with strong interactive features and high realism. However, due to factors such as high cost of vehicle-side data collection, limited monitoring range, and low frequency of such scenarios, system developers generally adopt a hybrid approach of "small sample real data + rule-driven generation" to construct training datasets for models / algorithms. While this method compensates for the lack of data to some extent, it still has significant shortcomings in terms of scenario diversity, interaction complexity, and realism, making it difficult to support the generalization and robust performance of autonomous driving systems in long-tail scenarios.

[0003] With the development and improvement of intelligent traffic management networks, a large amount of roadside perception data has been accumulated on the management side, such as target movement trajectory perception data for different time periods, different road sections / intersections, and different traffic participants. If complex game scenarios can be constructed based on real roadside perception data, it will significantly improve the generalization ability and safety verification effect of models / algorithms in long-tail scenarios. How to construct complex game scenarios based on real roadside perception data is the technical problem that this invention aims to solve. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for trajectory extraction and annotation based on roadside data. Upon receiving a roadside perception dataset, this invention first performs abnormal trajectory cleaning, static trajectory smoothing, abnormal turning trajectory repair, and heading angle smoothing correction. Then, it adds lane features to all trajectories in the dataset, identifies trajectory behavior types, and segments behavior fragment trajectories, creating corresponding annotation record sets based on all fragment trajectories. Next, it identifies the motion game relationship between every two target trajectories in the annotation record set to obtain a game relationship matrix W. Finally, it generates a multi-objective game scenario set based on the game relationship matrix W and the annotation record set. This invention enables the construction of numerous complex game scenarios based on real roadside perception data, which helps improve the generalization ability and safety verification effect of models / algorithms in long-tail scenarios.

[0005] To achieve the above objectives, a first aspect of the present invention provides a method for trajectory extraction and annotation based on roadside data, the method comprising:

[0006] The system receives a roadside perception dataset; the roadside perception dataset corresponds to a road segment or intersection scene in a real road network; the roadside perception dataset consists of a high-precision map of the scene and roadside perception data of multiple traffic participants; the high-precision map of the scene is a high-precision map of the road in the current traffic scene; the roadside perception data corresponds one-to-one with traffic participants in the current traffic scene; the roadside perception data includes participant identifier, participant type, participant geometry, and motion trajectory; the participant type includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the participant geometry includes length, width, and height; the motion trajectory is formed by sorting multiple first trajectory points; the first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; the first position coordinates are latitude and longitude coordinates;

[0007] The roadside sensing dataset is subjected to abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction.

[0008] Lane feature addition, trajectory behavior type segmentation and identification, and behavior segment trajectory segmentation are performed on all trajectories in the roadside perception dataset. Annotated records are then created based on all trajectory segments to obtain a corresponding annotation record set. The annotation record set includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of labeled records; the labeled record X i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i The target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the target shape e i Including length, width, and height; the target trajectory r i It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marker, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time and end time; the trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping;

[0009] For every two target trajectories r in the labeled record set iThe game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set; the game relationship matrix W consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The rows or columns of the game relationship matrix W are related to the target trajectory r. i One-to-one correspondence; the matrix unit w i,j The unit value is used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; the matrix unit w i,j The unit value is 0 or 1, where 0 indicates that there is no game-theoretic relationship between the i-th and j-th objectives, and 1 indicates that there is a game-theoretic relationship between the i-th and j-th objectives. The multi-objective game scenario set consists of one or more multi-objective game scenarios. The multi-objective game scenario includes a local scene map and a multi-objective record set. The multi-objective record set consists of multiple labeled records X. i composition.

[0010] Preferably, the abnormal trajectory cleaning, stationary trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction processing of the roadside perception dataset specifically includes:

[0011] Step 21: Identify abnormal trajectories for each motion trajectory in the roadside sensing dataset, and when the current trajectory is confirmed to be an abnormal trajectory, perform data cleaning processing on the corresponding roadside sensing data. Specifically:

[0012] Each motion trajectory in the roadside sensing dataset is taken as the current trajectory; the total length of the current trajectory is identified; the Manhattan distance between the first and last first trajectory points of the current trajectory is calculated to obtain the corresponding origin-end Manhattan distance; the ratio of the total trajectory length to the origin-end Manhattan distance is taken as the corresponding path complexity; the origin-end Manhattan distance and the path complexity are identified; if the origin-end Manhattan distance is lower than a preset Manhattan distance threshold, or the path complexity is higher than a preset complexity threshold, the current trajectory is identified as an abnormal trajectory; if the origin-end Manhattan distance is higher than the Manhattan distance threshold and the path complexity is lower than the complexity threshold, the current trajectory is identified as a normal trajectory; and when the current trajectory is confirmed to be an abnormal trajectory, the roadside sensing data corresponding to the current trajectory is deleted from the roadside sensing dataset.

[0013] Step 22 involves identifying stationary trajectory segments in each motion trajectory of the cleaned roadside perception dataset and smoothing the identified stationary segments, specifically as follows:

[0014] Each motion trajectory in the roadside perception dataset is taken as the current trajectory; and a trajectory segment on the current trajectory consisting of multiple consecutive first trajectory points whose first speed is lower than a preset stationary speed threshold is recorded as the corresponding stationary trajectory segment; when the total number of stationary trajectory segments is greater than 0, each stationary trajectory segment is taken as the corresponding current stationary segment; and based on a preset first smoothing filtering method, the first position coordinates, first speed, first acceleration, and first heading angle of all first trajectory points in the current stationary segment are smoothed and filtered; wherein, the first smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering.

[0015] Step 23 involves identifying abnormal trajectory segments in each motion trajectory of the roadside perception dataset after static segment smoothing filtering and performing trajectory repair processing on each abnormal trajectory segment, specifically as follows:

[0016] Each motion trajectory in the roadside perception dataset is taken as the current trajectory; the trajectory curvature corresponding to each first trajectory point on the current trajectory is calculated to obtain the corresponding first trajectory curvature; each pair of adjacent first trajectory points in the current trajectory is formed into a trajectory point pair; the heading angle difference, velocity difference, and trajectory curvature difference of each trajectory point pair are calculated, and the calculation results are taken as the corresponding direction change rate, velocity change rate, and curvature change rate; and the trajectory point pairs whose direction change rate does not meet the preset normal direction change rate range, or whose velocity change rate does not meet the preset normal velocity change rate range, or whose curvature change rate does not meet the preset normal curvature change rate range are recorded as anomalies. The system identifies constant point pairs and records trajectory segments composed of multiple consecutive abnormal point pairs as the corresponding abnormal turnaround trajectory segments. When the total number of abnormal turnaround trajectory segments is greater than 0, each abnormal turnaround trajectory segment is taken as the corresponding current turnaround segment. Based on a preset second smoothing filtering method, the first position coordinates, first velocity, first acceleration, and first heading angle of all first trajectory points in the current turnaround segment are smoothed and filtered. During the processing, the first position coordinates of the filtered output are constrained based on the lane boundary information provided by the high-precision map of the scene. The second smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering.

[0017] Step 24: Perform heading angle smoothing correction processing on each motion trajectory of the roadside perception dataset after completing the smoothing filtering of abnormal turnaround trajectory segments, specifically as follows:

[0018] Each motion trajectory in the roadside perception dataset is taken as the current trajectory; and based on a preset third smoothing filtering method, the first heading angle of all first trajectory points of the current trajectory is smoothed and filtered; wherein, the third smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering.

[0019] Preferably, the step of adding lane features, segmenting and identifying trajectory behavior types, and segmenting behavior fragment trajectories for all trajectories in the roadside perception dataset, and creating a corresponding labeled record set based on all segment trajectories, specifically includes:

[0020] Step 31: Take each of the roadside sensing data in the roadside sensing dataset as the current sensing data; take the motion trajectory of the current sensing data as the current motion trajectory; add lane features to the current trajectory to obtain the corresponding first trajectory; and perform trajectory behavior type segmentation recognition and behavior segment trajectory segmentation processing on the first trajectory to obtain the corresponding first segment trajectory set.

[0021] Wherein, the first trajectory is composed of a plurality of second trajectory points; the first trajectory corresponds one-to-one with the motion trajectory; the second trajectory points of the first trajectory correspond one-to-one with the first trajectory points of the corresponding motion trajectory;

[0022] The first segment trajectory set consists of multiple first segment trajectories; the first segment trajectory consists of multiple second trajectory points; each first segment trajectory corresponds to a first trajectory behavior type; the first trajectory behavior type includes stopping, starting, going straight, turning, U-turn, continuous lane changing, emergency braking, rapid acceleration, and emergency stopping.

[0023] Step 32: Merge all the obtained first segment trajectory sets to obtain the corresponding segment trajectory set; and create the corresponding annotation record set based on the segment trajectory set;

[0024] The collection of segment trajectories consists of multiple first segment trajectories.

[0025] Furthermore, the step of adding lane features to the current trajectory to obtain the corresponding first trajectory specifically includes:

[0026] Each of the first trajectory points of the current motion trajectory is taken as the corresponding current trajectory point; and the first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle of the current trajectory point are taken as the corresponding current timestamp, current position coordinates, current velocity, current acceleration, and current heading angle; the current position coordinates are converted into map coordinates of the scene high-precision map; and the scene high-precision map is queried based on the converted current position coordinates to obtain the lane identifier of the lane where the current coordinates are located as the corresponding current lane identifier; and the current timestamp, the current lane identifier, the current position coordinates, the current velocity, the current acceleration, and the current heading angle are taken as a group of corresponding second timestamps, first lane identifiers, second position coordinates, second velocity, second acceleration, and second heading angles to form a corresponding second trajectory point; and all the second trajectory points corresponding to the current motion trajectory are sequentially sorted to form a corresponding first trajectory.

[0027] Furthermore, the step of performing trajectory behavior type segmentation and behavior segment trajectory segmentation on the first trajectory to obtain the corresponding first segment trajectory set specifically includes:

[0028] Based on preset multi-class trajectory segment recognition rules, the first trajectory is subjected to multi-class trajectory segment recognition to obtain multiple corresponding first segment trajectories, and the corresponding first trajectory behavior type is set based on the rule segment type corresponding to each first segment trajectory; and the obtained first segment trajectories form a corresponding first segment trajectory set;

[0029] The multiple trajectory segment recognition rules include parking trajectory segment recognition rules, starting trajectory segment recognition rules, straight-going trajectory segment recognition rules, turning trajectory segment recognition rules, U-turn trajectory segment recognition rules, continuous lane change trajectory segment recognition rules, emergency braking trajectory segment recognition rules, rapid acceleration trajectory segment recognition rules, and emergency stop trajectory segment recognition rules; the rule segment types corresponding to each type of trajectory segment recognition rule are respectively parking, starting, straight-going, turning, U-turn, continuous lane change, emergency braking, rapid acceleration, or emergency stop.

[0030] Furthermore, the step of creating the corresponding labeled record set based on the collection of fragment trajectories specifically includes:

[0031] The total number of segment trajectories in the aforementioned segment trajectory set is counted, and the statistical result is used as the corresponding total number N of labeled records. X ;

[0032] Each of the first segment trajectories in the fragment trajectory set is taken as the current segment trajectory; and the participant identifier, participant type, and participant geometry of the roadside sensing data corresponding to the current segment trajectory are taken as a set of corresponding target identifiers Id. i The target type c i The target shape e i The current segment trajectory is used as a corresponding target trajectory r. i ; and by the target trajectory r i The first and last second timestamps, as a set of corresponding start and end times, form a corresponding trajectory time period t over time. i ; and the first trajectory behavior type corresponding to the current segment trajectory is used as a corresponding trajectory behavior type s. i ; and the target identifier Id corresponding to the current segment trajectory. i The target type c i The target shape e i The target trajectory r i The trajectory time period t i The trajectory behavior type s i Form a corresponding labeled record X i ;

[0033] And the N obtained this time X The labeled record X i This forms the corresponding set of labeled records.

[0034] Preferably, the step of adding every two target trajectories r to the labeled record set... i The game relationship is identified to obtain the corresponding game relationship matrix W, which specifically includes:

[0035] Step 71, for the N of the labeled record set X The target trajectory r i By combining pairs of elements, we can obtain the corresponding N. X (N X -1) / 2 sets of trajectory pairs;

[0036] Step 72: Take the first set of trajectory pairs as the current trajectory pair;

[0037] Step 73, combine the two target trajectories r of the current trajectory pair. i Let these be the corresponding trajectories A and B; and let the two traffic participants corresponding to trajectories A and B be the corresponding targets A and B.

[0038] Step 74: Identify whether the two target identifiers corresponding to trajectories A and B are the same; if yes, set the corresponding first game coefficient to 0 and proceed to step 79; if no, proceed to step 75.

[0039] Step 75: Identify whether there is an intersection between the two time periods of trajectories A and B; if yes, take the intersection time period of the two time periods as the current cross time period and proceed to step 76; if no, set the corresponding first game coefficient to 0 and proceed to step 79.

[0040] Step 76: Identify whether a collision event has occurred between trajectories A and B during the current crossover period; if yes, set the corresponding first game coefficient to 1 and proceed to step 79; if no, proceed to step 77.

[0041] Step 77: Calculate the nearest trajectory distance between trajectories A and B in the current crossover time period; and identify whether the nearest trajectory distance is less than a preset trajectory spacing threshold; if yes, set the corresponding first game coefficient to 1 and proceed to step 79; if no, proceed to step 78.

[0042] Step 78: Record a subsequent time period of a specified duration after the intersection time period as the current derived time period; extract and / or predict the target motion trajectory of target A in the current derived time period based on trajectory A to obtain the corresponding derived trajectory A; extract and / or predict the target motion trajectory of target B in the current derived time period based on trajectory B to obtain the corresponding derived trajectory B; estimate the corresponding TTC data based on derived trajectories A and B according to a preset collision time TTC evaluation mechanism; estimate the corresponding PET data based on derived trajectories A and B according to a preset post-collision time PET evaluation mechanism; and estimate the corresponding PET data based on derived trajectories A and B according to a preset minimum... The safety distance MSS assessment mechanism estimates the corresponding MSS data based on derived trajectories A and B; and identifies the TTC data, PET data, and MSS data; if the TTC data is less than a preset TTC threshold, or the PET data is less than a preset PET threshold, or the MSS data is less than a preset MSS threshold, then the corresponding first game coefficient is set to 1; if the TTC data is greater than or equal to the TTC threshold, and the PET data is greater than or equal to the PET threshold, and the MSS data is greater than or equal to the MSS threshold, then the corresponding first game coefficient is set to 0.

[0043] Step 79: Identify whether the current trajectory pair is the last set of trajectory pairs; if not, take the next set of trajectory pairs as the new current trajectory pair and return to step 73; if yes, set a shape of N. X ×N XThe zero-matrix is ​​used as the initial game relationship matrix W, and the matrix element values ​​of the game relationship matrix W are reset based on the first game coefficients corresponding to all the trajectories.

[0044] Preferably, the step of generating and saving a multi-objective game scenario set based on the game relationship matrix W and the labeled record set specifically includes:

[0045] The game relationship matrix W is traversed row by row in one round; and during this round of traversal, the current traversed row is taken as the corresponding current row; and the target trajectory r corresponding to the current row is... j As the current target trajectory; and set the matrix unit w with each unit value of 1 in the current row as 1. i,j The corresponding target trajectory r i As the associated target trajectory; and when the total number of the associated target trajectories is greater than 0, the current target trajectory and the annotation record X corresponding to all the associated target trajectories are used. i The system constructs a corresponding multi-objective record set, identifies the trajectory of the current target and the trajectory of each associated target in the high-precision map of the scene, merges all identified trajectory regions to obtain the corresponding current merged region, extracts the local high-precision map corresponding to the current merged region in the high-precision map of the scene as the corresponding local scene map, and combines the local scene map obtained this time with the multi-objective record set to form a corresponding multi-objective game scene; and at the end of this round of traversal, the system combines all the obtained multi-objective game scenes to form the corresponding multi-objective game scene set and saves it.

[0046] A second aspect of the present invention provides an apparatus for implementing the processing method for trajectory extraction and annotation based on roadside data as described in the first aspect above. The apparatus includes: a data receiving module, a data preprocessing module, a trajectory extraction and annotation module, and a scene construction module.

[0047] The data receiving module is used to receive roadside perception datasets; the roadside perception datasets correspond to road segments or intersections in a real road network; the roadside perception datasets consist of a high-precision map of the scene and roadside perception data of multiple traffic participants; the high-precision map of the scene is a high-precision map of the road in the current traffic scene; the roadside perception data corresponds one-to-one with traffic participants in the current traffic scene; the roadside perception data includes participant identifiers, participant types, participant geometry, and movement trajectories; the participant types include pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the participant geometry includes length, width, and height; the movement trajectory is formed by sorting multiple first trajectory points; the first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; the first position coordinates are latitude and longitude coordinates;

[0048] The data preprocessing module is used to perform abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction on the roadside sensing dataset.

[0049] The trajectory extraction and annotation module is used to add lane features, segment and identify trajectory behavior types, and process behavior segment trajectories into all trajectories in the roadside perception dataset. It then creates an annotation record set based on all trajectory segments. The annotation record set includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of labeled records; the labeled record X i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i The target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the target shape e i Including length, width, and height; the target trajectory r i It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marker, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time and end time; the trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping;

[0050] The scene construction module is used to process every two target trajectories r in the labeled record set.i The game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set; the game relationship matrix W consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The rows or columns of the game relationship matrix W are related to the target trajectory r. i One-to-one correspondence; the matrix unit w i,j The unit value is used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; the matrix unit w i,j The unit value is 0 or 1, where 0 indicates that there is no game-theoretic relationship between the i-th and j-th objectives, and 1 indicates that there is a game-theoretic relationship between the i-th and j-th objectives. The multi-objective game scenario set consists of one or more multi-objective game scenarios. The multi-objective game scenario includes a local scene map and a multi-objective record set. The multi-objective record set consists of multiple labeled records X. i composition.

[0051] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0052] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0053] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0055] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for trajectory extraction and annotation based on roadside data. As described above, after receiving a roadside perception dataset, this invention first performs abnormal trajectory cleaning, static trajectory smoothing, abnormal turning trajectory repair, and heading angle smoothing correction. Then, it adds lane features to all trajectories in the dataset, identifies trajectory behavior types, and segments behavior fragment trajectories, creating corresponding annotation record sets based on all fragment trajectories. Next, it identifies the motion game relationship between every two target trajectories in the annotation record set to obtain a game relationship matrix W. Finally, it generates a multi-objective game scenario set based on the game relationship matrix W and the annotation record set. This invention allows the construction of numerous complex game scenarios based on real roadside perception data, helping to improve the generalization ability and safety verification effect of models / algorithms in long-tail scenarios. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of a trajectory extraction and annotation method based on roadside data provided in Embodiment 1 of the present invention;

[0057] Figure 2 This is a schematic diagram of the roadside sensing dataset provided in Embodiment 1 of the present invention;

[0058] Figure 3 This is a data diagram of the labeled record set provided in Embodiment 1 of the present invention;

[0059] Figure 4 This is a schematic diagram of the game relationship matrix W provided in Embodiment 1 of the present invention;

[0060] Figure 5 This is a module structure diagram of a processing device for trajectory extraction and annotation based on roadside data provided in Embodiment 2 of the present invention;

[0061] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0063] Embodiment 1 of the present invention provides a method for trajectory extraction and annotation based on roadside data, such as... Figure 1The schematic diagram of a trajectory extraction and annotation method based on roadside data provided in Embodiment 1 of the present invention includes the following main steps:

[0064] Step 1: Receive the roadside sensing dataset.

[0065] Here, the roadside perception dataset in this embodiment of the invention corresponds to a road segment or intersection scenario of a real road network.

[0066] like Figure 2 The data diagram shown is for the roadside perception dataset provided in Embodiment 1 of the present invention. The roadside perception dataset of the present invention consists of a high-precision map of the scene and roadside perception data of multiple traffic participants.

[0067] The scene high-precision map is a high-precision road map of the current traffic scene.

[0068] Roadside perception data corresponds one-to-one with traffic participants in the current traffic scenario. Roadside perception data includes participant identifiers, participant types, participant geometry, and movement trajectories. The participant identifier is a unique identifier for each current traffic participant. Participant types include pedestrians, various types of motor vehicles, and various types of non-motorized vehicles. Participant geometry includes length, width, and height. The movement trajectory is composed of multiple first trajectory points; each first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; it should be noted that the first position coordinates are latitude and longitude coordinates.

[0069] Step 2 involves performing abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction on the roadside perception dataset.

[0070] The current step 2 specifically includes:

[0071] Step 21: Identify abnormal trajectories for each motion trajectory in the roadside perception dataset, and perform data cleaning on the corresponding roadside perception data when the current trajectory is confirmed to be an abnormal trajectory.

[0072] Specifically, this includes: step 211, taking each motion trajectory of the roadside perception dataset as the current trajectory;

[0073] Step 212, and identify the total length of the current trajectory;

[0074] Step 213, and calculate the Manhattan distance between the first and last points of the current trajectory to obtain the corresponding start and end Manhattan distance;

[0075] Step 214, and use the ratio of the total trajectory length to the Manhattan distance between the start and end points as the corresponding path complexity;

[0076] Step 215, and identify the Manhattan distance between the origin and destination and the path complexity; if the Manhattan distance between the origin and destination is lower than the preset Manhattan distance threshold, or the path complexity is higher than the preset complexity threshold, the current trajectory is identified as an abnormal trajectory; if the Manhattan distance between the origin and destination is higher than the Manhattan distance threshold and the path complexity is lower than the complexity threshold, the current trajectory is identified as a non-abnormal trajectory.

[0077] Here, the Manhattan distance threshold and complexity threshold in this embodiment of the invention are two pre-set threshold parameters;

[0078] Step 216, and when it is confirmed that the current trajectory is an abnormal trajectory, delete the roadside sensing data corresponding to the current trajectory from the roadside sensing dataset;

[0079] Step 22: Identify the stationary trajectory segments in each motion trajectory of the cleaned roadside perception dataset and smooth the identified stationary segments.

[0080] Specifically, this includes: step 221, taking each motion trajectory of the roadside perception dataset as the current trajectory;

[0081] Step 222, and record the trajectory segment on the current trajectory composed of multiple consecutive first trajectory points with first velocities lower than the preset static velocity threshold as the corresponding static trajectory segment;

[0082] Here, the stationary speed threshold in this embodiment of the invention is a pre-set threshold parameter;

[0083] Step 223: When the total number of static trajectory segments is greater than 0, each static trajectory segment is taken as the corresponding current static segment; and based on the preset first smoothing filtering method, the first position coordinates, first velocity, first acceleration and first heading angle of all first trajectory points of the current static segment are smoothed and filtered.

[0084] Here, the first smoothing filtering method in this embodiment of the invention includes moving average filtering, median filtering, and Kalman filtering;

[0085] Step 23: Identify abnormal return trajectory segments for each motion trajectory in the roadside perception dataset after completing static segment smoothing filtering, and perform trajectory repair processing on each abnormal return trajectory segment.

[0086] Specifically, this includes: step 231, taking each motion trajectory of the roadside perception dataset as the current trajectory;

[0087] Step 232, and calculate the corresponding first trajectory curvature for each first trajectory point on the current trajectory;

[0088] Step 233, and form a trajectory point pair by each pair of two adjacent first trajectory points in the current trajectory;

[0089] Step 234, and calculate the difference in heading angle, velocity, and trajectory curvature for each pair of trajectory points, and use the calculation results as the corresponding rate of change of direction, rate of change of velocity, and rate of change of curvature;

[0090] Step 235, and record the trajectory point pairs where the rate of change of direction does not meet the preset normal rate of change of direction, or the rate of change of velocity does not meet the preset normal rate of change of velocity, or the rate of change of curvature does not meet the preset normal rate of change of curvature as abnormal point pairs.

[0091] Here, the normal direction rate of change range, normal velocity rate of change range, and normal curvature rate of change range in this embodiment of the invention are three preset numerical ranges;

[0092] Step 236, and record the trajectory segment composed of multiple consecutive pairs of abnormal points as the corresponding abnormal reversal trajectory segment;

[0093] Step 237: When the total number of abnormal turnaround trajectory segments is greater than 0, each abnormal turnaround trajectory segment is taken as the corresponding current turnaround segment; and based on the preset second smoothing filtering method, the first position coordinates, first velocity, first acceleration and first heading angle of all first trajectory points of the current turnaround segment are smoothed and filtered, and the first position coordinates of the filtered output are constrained based on the lane boundary information provided by the high-precision map of the scene during the processing.

[0094] Here, the second smoothing filtering method in this embodiment of the invention includes moving average filtering, median filtering, and Kalman filtering;

[0095] Step 24: Perform heading angle smoothing correction processing on each motion trajectory of the roadside perception dataset that has completed the smoothing filtering of abnormal turnaround trajectory segments;

[0096] Specifically, this includes: taking each motion trajectory of the roadside perception dataset as the current trajectory; and performing smoothing filtering on the first heading angle of all first trajectory points of the current trajectory based on a preset third smoothing filtering method;

[0097] Here, the third smoothing filtering method in this embodiment of the invention includes moving average filtering, median filtering, and Kalman filtering.

[0098] Step 3: Add lane features, segment and identify trajectory behavior types, and segment behavior fragment trajectories for all trajectories in the roadside perception dataset, and create corresponding annotation record sets based on the annotation records of all fragment trajectories.

[0099] Here, as Figure 3As shown in the data diagram of the annotation record set provided in Embodiment 1 of the present invention, the annotation record set of the present invention includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of annotated records; X annotated records. i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i ;Target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the various types of motor vehicles include at least trucks, lorries, construction vehicles, passenger cars, buses, school buses, and motorcycles; the various types of non-motor vehicles include at least ordinary bicycles, pedicabs, electric bicycles, and electric tricycles; target shape e i Including length, width, and height; target trajectory r i It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marking, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time, end time; trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping.

[0100] The current step 3 specifically includes:

[0101] Step 31: Take each roadside sensing data in the roadside sensing dataset as the current sensing data; take the motion trajectory of the current sensing data as the current motion trajectory; add lane features to the current trajectory to obtain the corresponding first trajectory; and perform trajectory behavior type segmentation and behavior segment trajectory segmentation processing on the first trajectory to obtain the corresponding first segment trajectory set.

[0102] Specifically, this includes: step 311, taking each roadside sensing data in the roadside sensing dataset as the current sensing data; and taking the motion trajectory of the current sensing data as the current motion trajectory;

[0103] Step 312, and add lane features to the current trajectory to obtain the corresponding first trajectory;

[0104] Here, the first trajectory in this embodiment of the invention is composed of a plurality of second trajectory points; the first trajectory corresponds one-to-one with the motion trajectory; the second trajectory points of the first trajectory correspond one-to-one with the first trajectory points of the corresponding motion trajectory;

[0105] Specifically, this includes: step 3121, taking each of the first trajectory points of the current motion trajectory as the corresponding current trajectory point;

[0106] Step 3122, and use the first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle of the current trajectory point as the corresponding current timestamp, current position coordinates, current velocity, current acceleration, and current heading angle;

[0107] Step 3123, and convert the current location coordinates into map coordinates of the scene's high-precision map;

[0108] Step 3124: Based on the converted current location coordinates, query the scene high-precision map to obtain the lane identifier of the lane where the current coordinates are located as the corresponding current lane identifier;

[0109] Step 3125, and the current timestamp, current lane identifier, current position coordinates, current speed, current acceleration, and current heading angle are used as a set of corresponding second timestamps, first lane identifiers, second position coordinates, second speed, second acceleration, and second heading angle to form a corresponding second trajectory point;

[0110] Step 3126, and then sort all the second trajectory points corresponding to the current motion trajectory in order to form a corresponding first trajectory;

[0111] Step 313, and perform trajectory behavior type segmentation and behavior segment trajectory segmentation on the first trajectory to obtain the corresponding first segment trajectory set;

[0112] Specifically, this includes: performing multi-class trajectory segment recognition on the first trajectory based on preset multi-class trajectory segment recognition rules to obtain multiple corresponding first segment trajectories, setting the corresponding first trajectory behavior type based on the rule segment type corresponding to each first segment trajectory, and forming a corresponding first segment trajectory set from all the obtained first segment trajectories;

[0113] Here, the multiple trajectory segment recognition rules in this embodiment of the invention include parking trajectory segment recognition rules, starting trajectory segment recognition rules, straight-going trajectory segment recognition rules, turning trajectory segment recognition rules, U-turn trajectory segment recognition rules, continuous lane change trajectory segment recognition rules, emergency braking trajectory segment recognition rules, rapid acceleration trajectory segment recognition rules, and emergency stop trajectory segment recognition rules; each type of trajectory segment recognition rule can be customized based on actual application needs; the rule segment types corresponding to each type of trajectory segment recognition rule are respectively parking, starting, straight-going, turning, U-turn, continuous lane change, emergency braking, rapid acceleration, or emergency stop;

[0114] The first segment trajectory set in this embodiment of the invention consists of multiple first segment trajectories; each first segment trajectory consists of multiple second trajectory points; each first segment trajectory corresponds to a first trajectory behavior type; the first trajectory behavior type includes stopping, starting, going straight, turning, U-turn, continuous lane changing, emergency braking, rapid acceleration, and emergency stopping.

[0115] Step 32: Merge all the obtained first segment trajectory sets to obtain the corresponding segment trajectory set;

[0116] Here, the fragment trajectory set in this embodiment of the invention consists of multiple first fragment trajectories;

[0117] Step 33, and create the corresponding annotation record set based on the fragment trajectory set;

[0118] Specifically, this includes: Step 331, counting the total number of fragment trajectories in the fragment trajectory set and using the statistical result as the corresponding total number of labeled records N. X ;

[0119] Step 332: Take each first segment trajectory of the segment trajectory set as the current segment trajectory; and take the participant identifier, participant type, and participant geometry of the roadside sensing data corresponding to the current segment trajectory as a set of corresponding target identifiers Id. i Target type c i Target shape e i The current segment trajectory is used as a corresponding target trajectory r. i ; and by the target trajectory r i The first and last second timestamps, as a set of corresponding start and end times, form a corresponding trajectory time period t over time. i And the first trajectory behavior type corresponding to the current segment trajectory is taken as a corresponding trajectory behavior type s. i And determined by the target identifier Id corresponding to the current segment trajectory. i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i Form a corresponding labeled record X i ;

[0120] Step 333, and the N obtained this time X X records of annotations i Form the corresponding set of labeled records.

[0121] Step 4, for every two target trajectories r in the labeled record set iThe game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set.

[0122] Here, as Figure 4 As shown in the schematic diagram of the game relationship matrix W provided in Embodiment 1 of the present invention, the game relationship matrix W of the present invention consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The columns or rows of the game relationship matrix W and the target trajectory r i j corresponds one-to-one with; matrix element w i,j The unit values ​​are used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; matrix unit w i,j The unit value is 0 or 1. A value of 0 indicates that there is no dynamic game relationship between the i-th and j-th objectives, while a value of 1 indicates that there is a dynamic game relationship between the i-th and j-th objectives.

[0123] The multi-objective game scenario set of this invention consists of one or more multi-objective game scenarios; wherein, a multi-objective game scenario includes a local scenario map and a multi-objective record set; the multi-objective record set consists of multiple labeled records X. i composition.

[0124] The current step 4 specifically includes:

[0125] Step 41, for every two target trajectories r in the labeled record set i The corresponding game relationship matrix W is obtained by identifying the game relationship in motion;

[0126] Specifically, this includes: Step 411, processing the N records in the labeled record set. X Target trajectory r i By combining pairs of elements, we can obtain the corresponding N. X (N X -1) / 2 sets of trajectory pairs;

[0127] Step 412: Take the first set of trajectory pairs as the current trajectory pair;

[0128] Step 413, combine the two target trajectories r of the current trajectory pair. i Let these be the corresponding trajectories A and B; and let the two traffic participants corresponding to trajectories A and B be the corresponding targets A and B.

[0129] Step 414: Identify whether the two target identifiers corresponding to trajectories A and B are the same; if yes, set the corresponding first game coefficient to 0 and proceed to step 419; if no, proceed to step 415.

[0130] Step 415: Identify whether there is an intersection between the two time periods of trajectories A and B; if yes, take the intersection time period of the two time periods as the current cross time period and proceed to step 416; if no, set the corresponding first game coefficient to 0 and proceed to step 419.

[0131] Step 416: Identify whether a collision event has occurred between trajectories A and B during the current crossover period; if yes, set the corresponding first game coefficient to 1 and proceed to step 419; if no, proceed to step 417.

[0132] Step 417: Calculate the nearest trajectory distance between trajectories A and B in the current crossover time period; and identify whether the nearest trajectory distance is less than the preset trajectory spacing threshold; if yes, set the corresponding first game coefficient to 1 and proceed to step 419; if no, proceed to step 418.

[0133] Here, the trajectory spacing threshold in this embodiment of the invention is a pre-set threshold parameter;

[0134] Step 418: The subsequent period of a specified duration after the intersection period is recorded as the current derived period; the target motion trajectory of target A in the current derived period is extracted and / or predicted based on trajectory A to obtain the corresponding derived trajectory A; the target motion trajectory of target B in the current derived period is extracted and / or predicted based on trajectory B to obtain the corresponding derived trajectory B; the corresponding TTC data is estimated based on derived trajectories A and B according to the preset collision time TTC evaluation mechanism; the corresponding PET data is estimated based on derived trajectories A and B according to the preset post-collision time PET evaluation mechanism; the corresponding MSS data is estimated based on derived trajectories A and B according to the preset minimum safe distance MSS evaluation mechanism; the TTC data, PET data, and MSS data are identified; if the TTC data is less than the preset TTC threshold, or the PET data is less than the preset PET threshold, or the MSS data is less than the preset MSS threshold, then the corresponding first game coefficient is set to 1; if the TTC data is greater than or equal to the TTC threshold, and the PET data is greater than or equal to the PET threshold, and the MSS data is greater than or equal to the MSS threshold, then the corresponding first game coefficient is set to 0.

[0135] Here, the duration is specified as a pre-set time length parameter;

[0136] The collision time (TTC) assessment mechanism, post-encroachment time (PET) assessment mechanism, and minimum safety spacing (MSS) assessment mechanism of this invention are three pre-implemented numerical assessment / estimation rules based on industry standards or user-defined rules.

[0137] In this embodiment of the invention, the TTC threshold, PET threshold, and MSS threshold are three pre-set threshold parameters;

[0138] Step 419: Identify whether the current trajectory pair is the last set of trajectory pairs; if not, take the next set of trajectory pairs as the new current trajectory pair and return to step 413; if yes, set a shape of N. X ×N X The zero matrix is ​​used as the initial game relationship matrix W, and the matrix unit values ​​of the game relationship matrix W are reset based on the first game coefficients corresponding to all trajectories.

[0139] Step 42, and generate and save a multi-objective game scenario set based on the game relationship matrix W and the labeled record set;

[0140] Specifically, this includes: performing a row-by-row traversal of the game relationship matrix W; during this traversal, taking the current traversed row as the corresponding current row; and setting the target trajectory r corresponding to the current row as... j This is used as the current target trajectory; and the matrix units w with values ​​of 1 in the current row are... i,j The corresponding target trajectory r i As the associated target trajectory; and when the total number of associated target trajectories is greater than 0, the annotation record X corresponding to the current target trajectory and all associated target trajectories is used. i A corresponding multi-objective record set is formed, and the trajectory of the current target and the trajectory of each associated target in the high-precision map of the scene are identified. All identified trajectory regions are merged to obtain the corresponding current merged region. The local high-precision map corresponding to the current merged region in the high-precision map of the scene is extracted as the corresponding local scene map. The local scene map obtained this time and the multi-objective record set are combined to form a corresponding multi-objective game scenario. At the end of this round of traversal, all the obtained multi-objective game scenarios are combined to form the corresponding multi-objective game scenario set and saved.

[0141] Figure 5This is a module structure diagram of a processing device for trajectory extraction and annotation based on roadside data provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 5 As shown, the processing device for trajectory extraction and annotation based on roadside data provided in Embodiment 2 of the present invention includes: a data receiving module 201, a data preprocessing module 202, a trajectory extraction and annotation module 203, and a scene construction module 204.

[0142] The data receiving module 201 is used to receive roadside perception datasets; the roadside perception dataset corresponds to a road segment or intersection scene of a real road network; the roadside perception dataset consists of a high-precision map of the scene and roadside perception data of multiple traffic participants; the high-precision map of the scene is a high-precision map of the road in the current traffic scene; the roadside perception data corresponds one-to-one with the traffic participants in the current traffic scene; the roadside perception data includes participant identifier, participant type, participant geometry, and motion trajectory; the participant type includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the participant geometry includes length, width, and height; the motion trajectory is composed of multiple first trajectory points sorted together; the first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; the first position coordinates are latitude and longitude coordinates.

[0143] The data preprocessing module 202 is used to perform abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction on the roadside perception dataset.

[0144] The trajectory extraction and annotation module 203 is used to add lane features, segment and identify trajectory behavior types, and process behavior segment trajectories for all trajectories in the roadside perception dataset. Based on all trajectory segments, it creates an annotation record set; the annotation record set includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of annotated records; X annotated records. i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i ;Target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motorized vehicles; target shape e i Including length, width, and height; target trajectory ri It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marking, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time, end time; trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping.

[0145] Scene construction module 204 is used to process every two target trajectories r in the labeled record set. i The game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set; the game relationship matrix W consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The rows or columns of the game relationship matrix W and the target trajectory r i One-to-one correspondence; matrix unit w i,j The unit values ​​are used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; matrix unit w i,j The unit value is 0 or 1, where 0 indicates that there is no game-theoretic relationship between the i-th and j-th objectives, and 1 indicates that there is a game-theoretic relationship between the i-th and j-th objectives. A multi-objective game scenario set consists of one or more multi-objective game scenarios. Each multi-objective game scenario includes a local scene map and a multi-objective record set. The multi-objective record set consists of multiple labeled records X. i composition.

[0146] The present invention provides a processing device for trajectory extraction and annotation based on roadside data, which can execute the method steps in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0147] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. For example, the data receiving module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0148] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0149] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0150] Figure 6 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 6 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0151] exist Figure 6The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0152] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0153] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0154] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for trajectory extraction and annotation based on roadside data. As described above, after receiving a roadside perception dataset, this invention first performs abnormal trajectory cleaning, static trajectory smoothing, abnormal turning trajectory repair, and heading angle smoothing correction. Then, it adds lane features, identifies trajectory behavior types, and segments behavior fragments of all trajectories in the dataset, creating corresponding annotation record sets based on all fragmented trajectories. Next, it identifies the motion game relationship between every two target trajectories in the annotation record set to obtain a game relationship matrix W. Finally, it generates a multi-objective game scenario set based on the game relationship matrix W and the annotation record set. This invention allows the construction of numerous complex game scenarios based on real roadside perception data, helping to improve the generalization ability and safety verification effect of models / algorithms in long-tail scenarios.

[0155] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0156] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for trajectory extraction and annotation based on roadside data, characterized in that, The method includes: The system receives a roadside perception dataset; the roadside perception dataset corresponds to a road segment or intersection scene in a real road network; the roadside perception dataset consists of a high-precision map of the scene and roadside perception data of multiple traffic participants; the high-precision map of the scene is a high-precision map of the road in the current traffic scene; the roadside perception data corresponds one-to-one with traffic participants in the current traffic scene; the roadside perception data includes participant identifier, participant type, participant geometry, and motion trajectory; the participant type includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the participant geometry includes length, width, and height; the motion trajectory is formed by sorting multiple first trajectory points; the first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; the first position coordinates are latitude and longitude coordinates; The roadside sensing dataset is subjected to abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction. Lane feature addition, trajectory behavior type segmentation and identification, and behavior segment trajectory segmentation are performed on all trajectories in the roadside perception dataset. Annotated records are then created based on all trajectory segments to obtain a corresponding annotation record set. The annotation record set includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of labeled records; the labeled record X i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i The target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the target shape e i Including length, width, and height; the target trajectory r i It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marker, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time and end time; the trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping; For every two target trajectories r in the labeled record set i The game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set; the game relationship matrix W consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The rows or columns of the game relationship matrix W are related to the target trajectory r. i One-to-one correspondence; the matrix unit w i,j The unit value is used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; the matrix unit w i,j The unit value is 0 or 1, where 0 indicates that there is no game-theoretic relationship between the i-th and j-th objectives, and 1 indicates that there is a game-theoretic relationship between the i-th and j-th objectives. The multi-objective game scenario set consists of one or more multi-objective game scenarios. The multi-objective game scenario includes a local scene map and a multi-objective record set. The multi-objective record set consists of multiple labeled records X. i composition.

2. The method for trajectory extraction and annotation based on roadside data according to claim 1, characterized in that, The process of performing abnormal trajectory cleaning, stationary trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction on the roadside perception dataset specifically includes: Step 21: Identify abnormal trajectories for each motion trajectory in the roadside sensing dataset, and when the current trajectory is confirmed to be an abnormal trajectory, perform data cleaning processing on the corresponding roadside sensing data. Specifically: Each motion trajectory in the roadside sensing dataset is taken as the current trajectory; the total length of the current trajectory is identified; the Manhattan distance between the first and last first trajectory points of the current trajectory is calculated to obtain the corresponding origin-end Manhattan distance; the ratio of the total trajectory length to the origin-end Manhattan distance is taken as the corresponding path complexity; the origin-end Manhattan distance and the path complexity are identified; if the origin-end Manhattan distance is lower than a preset Manhattan distance threshold, or the path complexity is higher than a preset complexity threshold, the current trajectory is identified as an abnormal trajectory; if the origin-end Manhattan distance is higher than the Manhattan distance threshold and the path complexity is lower than the complexity threshold, the current trajectory is identified as a normal trajectory; and when the current trajectory is confirmed to be an abnormal trajectory, the roadside sensing data corresponding to the current trajectory is deleted from the roadside sensing dataset. Step 22 involves identifying stationary trajectory segments in each motion trajectory of the cleaned roadside perception dataset and smoothing the identified stationary segments, specifically as follows: Each motion trajectory in the roadside perception dataset is taken as the current trajectory; and a trajectory segment on the current trajectory consisting of multiple consecutive first trajectory points whose first speed is lower than a preset stationary speed threshold is recorded as the corresponding stationary trajectory segment; when the total number of stationary trajectory segments is greater than 0, each stationary trajectory segment is taken as the corresponding current stationary segment; and based on a preset first smoothing filtering method, the first position coordinates, first speed, first acceleration, and first heading angle of all first trajectory points in the current stationary segment are smoothed and filtered; wherein, the first smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering. Step 23 involves identifying abnormal trajectory segments in each motion trajectory of the roadside perception dataset after static segment smoothing filtering and performing trajectory repair processing on each abnormal trajectory segment, specifically as follows: Each motion trajectory in the roadside perception dataset is taken as the current trajectory; the trajectory curvature corresponding to each first trajectory point on the current trajectory is calculated to obtain the corresponding first trajectory curvature; each pair of adjacent first trajectory points in the current trajectory is formed into a trajectory point pair; the heading angle difference, velocity difference, and trajectory curvature difference of each trajectory point pair are calculated, and the calculation results are taken as the corresponding direction change rate, velocity change rate, and curvature change rate; and the trajectory point pairs whose direction change rate does not meet the preset normal direction change rate range, or whose velocity change rate does not meet the preset normal velocity change rate range, or whose curvature change rate does not meet the preset normal curvature change rate range are recorded as anomalies. The system identifies constant point pairs and records trajectory segments composed of multiple consecutive abnormal point pairs as the corresponding abnormal turnaround trajectory segments. When the total number of abnormal turnaround trajectory segments is greater than 0, each abnormal turnaround trajectory segment is taken as the corresponding current turnaround segment. Based on a preset second smoothing filtering method, the first position coordinates, first velocity, first acceleration, and first heading angle of all first trajectory points in the current turnaround segment are smoothed and filtered. During the processing, the first position coordinates of the filtered output are constrained based on the lane boundary information provided by the high-precision map of the scene. The second smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering. Step 24: Perform heading angle smoothing correction processing on each motion trajectory of the roadside perception dataset after completing the smoothing filtering of abnormal turnaround trajectory segments, specifically as follows: Each motion trajectory in the roadside perception dataset is taken as the current trajectory; and based on a preset third smoothing filtering method, the first heading angle of all first trajectory points of the current trajectory is smoothed and filtered; wherein, the third smoothing filtering method includes moving average filtering, median filtering, and Kalman filtering.

3. The method for trajectory extraction and annotation based on roadside data according to claim 1, characterized in that, The process of adding lane features, segmenting and identifying trajectory behavior types, and segmenting behavior fragment trajectories for all trajectories in the roadside perception dataset, and creating a corresponding labeled record set based on all fragment trajectories, specifically includes: Step 31: Take each of the roadside sensing data in the roadside sensing dataset as the current sensing data; take the motion trajectory of the current sensing data as the current motion trajectory; add lane features to the current motion trajectory to obtain the corresponding first trajectory; and perform trajectory behavior type segmentation recognition and behavior segment trajectory segmentation processing on the first trajectory to obtain the corresponding first segment trajectory set. Wherein, the first trajectory is composed of a plurality of second trajectory points; the first trajectory corresponds one-to-one with the motion trajectory; the second trajectory points of the first trajectory correspond one-to-one with the first trajectory points of the corresponding motion trajectory; The first segment trajectory set consists of multiple first segment trajectories; the first segment trajectory consists of multiple second trajectory points; each first segment trajectory corresponds to a first trajectory behavior type; the first trajectory behavior type includes stopping, starting, going straight, turning, U-turn, continuous lane changing, emergency braking, rapid acceleration, and emergency stopping. Step 32: Merge all the obtained first segment trajectory sets to obtain the corresponding segment trajectory set; and create the corresponding annotation record set based on the segment trajectory set; The collection of segment trajectories consists of multiple first segment trajectories.

4. The method for trajectory extraction and annotation based on roadside data according to claim 3, characterized in that, The step of adding lane features to the current trajectory to obtain the corresponding first trajectory specifically includes: Each of the first trajectory points of the current motion trajectory is taken as the corresponding current trajectory point; and the first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle of the current trajectory point are taken as the corresponding current timestamp, current position coordinates, current velocity, current acceleration, and current heading angle; the current position coordinates are converted into map coordinates of the scene high-precision map; and the scene high-precision map is queried based on the converted current position coordinates to obtain the lane identifier of the lane where the current coordinates are located as the corresponding current lane identifier; and the current timestamp, the current lane identifier, the current position coordinates, the current velocity, the current acceleration, and the current heading angle are taken as a group of corresponding second timestamps, first lane identifiers, second position coordinates, second velocity, second acceleration, and second heading angles to form a corresponding second trajectory point; and all the second trajectory points corresponding to the current motion trajectory are sequentially sorted to form a corresponding first trajectory.

5. The method for trajectory extraction and annotation based on roadside data according to claim 3, characterized in that, The step of performing trajectory behavior type segmentation and behavior segment trajectory segmentation on the first trajectory to obtain the corresponding first segment trajectory set specifically includes: Based on preset multi-class trajectory segment recognition rules, the first trajectory is subjected to multi-class trajectory segment recognition to obtain multiple corresponding first segment trajectories, and the corresponding first trajectory behavior type is set based on the rule segment type corresponding to each first segment trajectory; and the obtained first segment trajectories form a corresponding first segment trajectory set; The multiple trajectory segment recognition rules include parking trajectory segment recognition rules, starting trajectory segment recognition rules, straight-going trajectory segment recognition rules, turning trajectory segment recognition rules, U-turn trajectory segment recognition rules, continuous lane change trajectory segment recognition rules, emergency braking trajectory segment recognition rules, rapid acceleration trajectory segment recognition rules, and emergency stop trajectory segment recognition rules; the rule segment types corresponding to each type of trajectory segment recognition rule are respectively parking, starting, straight-going, turning, U-turn, continuous lane change, emergency braking, rapid acceleration, or emergency stop.

6. The method for trajectory extraction and annotation based on roadside data according to claim 3, characterized in that, The process of creating the corresponding annotation record set based on the collection of fragment trajectories specifically includes: The total number of segment trajectories in the aforementioned segment trajectory set is counted, and the statistical result is used as the total number N of the corresponding labeled records. X ; Each of the first segment trajectories in the fragment trajectory set is taken as the current segment trajectory; and the participant identifier, participant type, and participant geometry of the roadside sensing data corresponding to the current segment trajectory are taken as a set of corresponding target identifiers Id. i The target type c i The target shape e i The current segment trajectory is used as a corresponding target trajectory r. i ; and by the target trajectory r i The first and last second timestamps, as a set of corresponding start and end times, constitute a corresponding trajectory time period t. i ; and the first trajectory behavior type corresponding to the current segment trajectory is used as a corresponding trajectory behavior type s. i ; and the target identifier Id corresponding to the current segment trajectory. i The target type c i The target shape e i The target trajectory r i The trajectory time period t i The trajectory behavior type s i Form a corresponding labeled record X i ; And the N obtained this time X The labeled record X i This forms the corresponding set of labeled records.

7. The method for trajectory extraction and annotation based on roadside data according to claim 1, characterized in that, The target trajectory r in the labeled record set is... i The game relationship is identified to obtain the corresponding game relationship matrix W, which specifically includes: Step 71, for the N of the labeled record set X The target trajectory r i By combining pairs of elements, we can obtain the corresponding N. X (N X -1) / 2 sets of trajectory pairs; Step 72: Take the first set of trajectory pairs as the current trajectory pair; Step 73, combine the two target trajectories r of the current trajectory pair. i Let these be the corresponding trajectories A and B; and let the two traffic participants corresponding to trajectories A and B be the corresponding targets A and B. Step 74: Identify whether the two target identifiers corresponding to trajectories A and B are the same; if yes, set the corresponding first game coefficient to 0 and proceed to step 79; if no, proceed to step 75. Step 75: Identify whether there is an intersection between the two time periods of trajectories A and B; if yes, take the intersection time period of the two time periods as the current cross time period and proceed to step 76; if no, set the corresponding first game coefficient to 0 and proceed to step 79. Step 76: Identify whether a collision event has occurred between trajectories A and B during the current crossover period; if yes, set the corresponding first game coefficient to 1 and proceed to step 79; if no, proceed to step 77. Step 77: Calculate the nearest trajectory distance between trajectories A and B in the current crossover time period; and identify whether the nearest trajectory distance is less than a preset trajectory spacing threshold; if yes, set the corresponding first game coefficient to 1 and proceed to step 79; if no, proceed to step 78. Step 78: Record a subsequent time period of a specified duration after the intersection time period as the current derived time period; extract and / or predict the target motion trajectory of target A in the current derived time period based on trajectory A to obtain the corresponding derived trajectory A; extract and / or predict the target motion trajectory of target B in the current derived time period based on trajectory B to obtain the corresponding derived trajectory B; estimate the corresponding TTC data based on derived trajectories A and B according to a preset collision time TTC evaluation mechanism; estimate the corresponding PET data based on derived trajectories A and B according to a preset post-collision time PET evaluation mechanism; and estimate the corresponding PET data based on derived trajectories A and B according to a preset minimum... The safety distance MSS assessment mechanism estimates the corresponding MSS data based on derived trajectories A and B; and identifies the TTC data, PET data, and MSS data; if the TTC data is less than a preset TTC threshold, or the PET data is less than a preset PET threshold, or the MSS data is less than a preset MSS threshold, then the corresponding first game coefficient is set to 1; if the TTC data is greater than or equal to the TTC threshold, and the PET data is greater than or equal to the PET threshold, and the MSS data is greater than or equal to the MSS threshold, then the corresponding first game coefficient is set to 0. Step 79: Identify whether the current trajectory pair is the last set of trajectory pairs; if not, take the next set of trajectory pairs as the new current trajectory pair and return to step 73; if yes, set a shape of N. X ×N X The zero-matrix is ​​used as the initial game relationship matrix W, and the matrix element values ​​of the game relationship matrix W are reset based on the first game coefficients corresponding to all the trajectories.

8. The method for trajectory extraction and annotation based on roadside data according to claim 1, characterized in that, The step of generating and saving a multi-objective game scenario set based on the game relationship matrix W and the labeled record set specifically includes: The game relationship matrix W is traversed row by row in one round; and during this round of traversal, the current traversed row is taken as the corresponding current row; and the target trajectory r corresponding to the current row is... j As the current target trajectory; and set the matrix unit w with each unit value of 1 in the current row as 1. i,j The corresponding target trajectory r i As the associated target trajectory; and when the total number of the associated target trajectories is greater than 0, the current target trajectory and the annotation record X corresponding to all the associated target trajectories are used. i The system constructs a corresponding multi-objective record set, identifies the trajectory of the current target and the trajectory of each associated target in the high-precision map of the scene, merges all identified trajectory regions to obtain the corresponding current merged region, extracts the local high-precision map corresponding to the current merged region in the high-precision map of the scene as the corresponding local scene map, and combines the local scene map obtained this time with the multi-objective record set to form a corresponding multi-objective game scene; and at the end of this round of traversal, the system combines all the obtained multi-objective game scenes to form the corresponding multi-objective game scene set and saves it.

9. An apparatus for performing the processing method for trajectory extraction and annotation based on roadside data as described in any one of claims 1-8, characterized in that, The device includes: a data receiving module, a data preprocessing module, a trajectory extraction and annotation module, and a scene construction module; The data receiving module is used to receive roadside perception datasets; the roadside perception datasets correspond to road segments or intersections in a real road network; the roadside perception datasets consist of a high-precision map of the scene and roadside perception data of multiple traffic participants; the high-precision map of the scene is a high-precision map of the road in the current traffic scene; the roadside perception data corresponds one-to-one with traffic participants in the current traffic scene; the roadside perception data includes participant identifiers, participant types, participant geometry, and movement trajectories; the participant types include pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the participant geometry includes length, width, and height; the movement trajectory is formed by sorting multiple first trajectory points; the first trajectory point includes a first timestamp, first position coordinates, first velocity, first acceleration, and first heading angle; the first position coordinates are latitude and longitude coordinates; The data preprocessing module is used to perform abnormal trajectory cleaning, static trajectory smoothing, abnormal turnaround trajectory repair, and heading angle smoothing correction on the roadside sensing dataset. The trajectory extraction and annotation module is used to add lane features, segment and identify trajectory behavior types, and process behavior segment trajectories into all trajectories in the roadside perception dataset. It then creates an annotation record set based on all trajectory segments. The annotation record set includes multiple annotation records X. i 1 ≤ index i ≤ N X N X The total number of labeled records; the labeled record X i Including target identifier Id i Target type c i Target shape e i Target trajectory r i Trajectory time period t i Trajectory Behavior Types i The target type c i Includes pedestrians, various types of motor vehicles, and various types of non-motor vehicles; the target shape e i Including length, width, and height; the target trajectory r i It is composed of multiple second trajectory points sorted together; the second trajectory points include a second timestamp, a first lane marker, a second position coordinate, a second velocity, a second acceleration, and a second heading angle; the trajectory time period t i Includes start time and end time; the trajectory behavior type s i This includes stopping, starting, going straight, turning, U-turn, continuous lane changes, sudden braking, sudden acceleration, and sudden stopping; The scene construction module is used to process every two target trajectories r in the labeled record set. i The game relationship is identified to obtain the corresponding game relationship matrix W; and a multi-objective game scenario set is generated and saved based on the game relationship matrix W and the labeled record set; the game relationship matrix W consists of N X ×N X matrix unit w i,j Composition, 1≤indexj≤N X The rows or columns of the game relationship matrix W are related to the target trajectory r. i One-to-one correspondence; the matrix unit w i,j The unit value is used to reflect the motion game relationship between the i-th and j-th targets corresponding to the trajectories of the i-th and j-th targets; the matrix unit w i,j The unit value is 0 or 1, where 0 indicates that there is no game-theoretic relationship between the i-th and j-th objectives, and 1 indicates that there is a game-theoretic relationship between the i-th and j-th objectives. The multi-objective game scenario set consists of one or more multi-objective game scenarios. The multi-objective game scenario includes a local scene map and a multi-objective record set. The multi-objective record set consists of multiple labeled records X. i composition.

10. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the processing method for trajectory extraction and annotation based on roadside data as described in any one of claims 1-8; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the processing method for trajectory extraction and annotation based on roadside data as described in any one of claims 1-8.

Images