Multi-stage track association method and system
By using a multi-stage trajectory association method, high-precision complete target trajectories are generated through classification, filtering, and linear interpolation. This solves the problems of high computational cost and low accuracy in traditional methods, and improves the efficiency and accuracy of trajectory association.
Patent Information
- Application Number
- CN202511034509.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-11-14
AI Technical Summary
Traditional methods involve large computational loads in trajectory association, have low trajectory generation accuracy, and struggle to distinguish the movement trajectories of the same vehicles in dense traffic, leading to misidentification and long computation time.
A multi-stage trajectory association method is adopted, which involves preliminary trajectory classification, relative relationship combination, linear interpolation and data frame completion to filter out missed time domains, select the closest candidate trajectory for data frame completion, and generate a high-precision complete target trajectory.
It improves the accuracy and efficiency of trajectory association, especially in overtaking scenarios, effectively solving the problem of missed detection and reducing the amount of computation and time consumption.
Smart Images
Figure CN120953312A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving perception technology, specifically to a multi-stage trajectory association method and system. Background Technology
[0002] Due to the issues of asynchronous sensors, perspective switching, and occlusion causing trajectory breakage, the target trajectory obtained by perception fusion is not unified. It is necessary to find a method to link the trajectories of the same target throughout its entire lifecycle, which will facilitate subsequent algorithm analysis and improvement. Practical application scenarios include situations where the target vehicle overtakes the vehicle, and the vehicle overtakes the target vehicle.
[0003] Traditional methods rely on single-frame data, using the target's distance and velocity to determine if the target is the same in consecutive frames, thus matching the target's motion trajectory. Because this relies on the target's instantaneous distance and velocity, any jumps in distance or velocity can lead to incorrect association results.
[0004] Traditional methods require determining the correlation between data in each frame. The large amount of data leads to a significant increase in computation, resulting in long processing times and low efficiency.
[0005] Traditional methods determine the correlation between consecutive frames using Euclidean distance, with a fixed threshold. This makes it difficult to distinguish the movement trajectories of the same vehicle when traffic is heavy.
[0006] Traditional methods lack the ability to determine whether the target is in front of or behind the vehicle, resulting in a misidentified tracking trajectory. Summary of the Invention
[0007] This application provides a multi-stage trajectory association method and system, which can solve the technical problems of large computational load in trajectory generation and low accuracy of complete trajectory due to missing data frames in the prior art.
[0008] In a first aspect, embodiments of this application provide a multi-stage trajectory association method, the multi-stage trajectory association method comprising: Multiple preliminary trajectories of the target vehicle are obtained through fusion data processing; the preliminary trajectories are classified to obtain multiple starting trajectories, process trajectories, and ending trajectories; the relative relationships between the trajectories are combined to obtain multiple preliminary complete trajectories, each of which includes a starting trajectory, a process trajectory, and an ending trajectory with a one-to-one correspondence. After merging all data frames of the preliminary complete trajectory, the system searches for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than a preset third threshold to obtain the missed detection time domain. The system performs linear interpolation on the lateral and longitudinal distances of the missed detection time domain in the preliminary complete trajectory to obtain the ideal trajectory. The system selects the trajectory that is closest to the ideal trajectory from the unassociated preliminary trajectory as the candidate trajectory, and completes the data frames of the missed detection time domain in the ideal trajectory based on the candidate trajectory to obtain the target complete trajectory.
[0009] In conjunction with the first aspect, in one implementation, obtaining multiple preliminary trajectories of the target vehicle based on fused data processing specifically includes the following steps: The system acquires fused data of the target vehicle detected by multiple source sensors of the vehicle. The fused data includes the fusion ID of the data frame, timestamp, location coordinates, and confidence score. The fusion ID is generated by a unique identifier of the multiple sensors. The confidence score is generated by the confidence score of the multiple sensors. Multiple preliminary trajectories are generated based on the fused data. Each preliminary trajectory has the same fusion ID and the time interval between frames is no greater than a first threshold. The preliminary trajectories are screened, and those that do not conform to the target scenario or have a confidence level below the second threshold are removed.
[0010] In conjunction with the first aspect, in one implementation, classifying the preliminary trajectory to obtain multiple starting trajectories, ending trajectories, and process trajectories specifically includes the following steps: The initial trajectory is defined as follows: the relative speed between the target vehicle and the vehicle is greater than a first speed threshold; the relative longitudinal distance conforms to a first distance interval; the extreme difference of the relative longitudinal distance is greater than a first extreme difference threshold; the relative lateral distance conforms to a second distance interval; and the lane where the trajectory endpoint is located is adjacent to the lane where the vehicle is located. The first speed threshold is greater than 0; the lower limit of the first distance interval is negative and the upper limit is 0; the first extreme difference threshold is greater than 0; the lower limit of the second distance interval is negative and the upper limit is positive. The initial trajectory of the target vehicle and the vehicle being driven is defined as the trajectory of the target vehicle being driven by a vehicle whose relative speed is greater than the second speed threshold, whose relative longitudinal distance is within the second distance interval, and whose extreme difference in relative longitudinal distance is greater than the second extreme difference threshold. The second speed threshold is greater than 0. The lower limit of the second distance interval is 0 and the upper limit is a positive number. The second extreme difference threshold is greater than 0. The preliminary trajectory, other than the initial trajectory and the ending trajectory, is defined as the process trajectory.
[0011] In conjunction with the first aspect, in one embodiment, the relative relationship includes one or any combination of time relationship, lane position relationship, relative speed relationship, relative longitudinal distance relationship, and / or relative lateral distance relationship; In conjunction with the first aspect, in one implementation, the missed detection time domain is a time domain where the inter-frame time interval is greater than a fourth threshold, a time domain where the confidence of all data frames at the same time is lower than a fifth threshold, and a time domain where the absolute value of the relative lateral distance of all data frames at the same time is less than a sixth threshold.
[0012] In conjunction with the first aspect, in one implementation, the candidate trajectory is the trajectory that is closest to the time interval of the missed detection time domain, the relative lateral distance interval, and the relative longitudinal distance interval of the ideal trajectory.
[0013] In conjunction with the first aspect, in one implementation, the linear interpolation of the horizontal and vertical distances in the time domain of the missed detections in the preliminary complete trajectory to obtain the ideal trajectory specifically includes the following steps: Linear interpolation is performed on the horizontal and vertical distances of the missed time domains in the preliminary complete trajectory to obtain the set of true value points; The set of truth points is used as the ideal trajectory.
[0014] In conjunction with the first aspect, in one implementation, selecting the trajectory closest to the ideal trajectory from the unassociated preliminary trajectories as a candidate trajectory specifically includes the following steps: Based on the time interval, relative horizontal distance interval, and relative vertical distance interval of the missed detection time domain, multiple undetermined trajectories are extracted from the initial trajectory that has not been associated, and data frames that conform to the missed detection time domain are extracted from each undetermined trajectory as a set of undetermined points. Calculate the absolute values of the relative lateral distance difference and the relative longitudinal distance difference between each point in the set of undetermined points and the set of true points. The undetermined trajectories whose sum of the absolute values of the relative lateral distance difference and the relative longitudinal distance difference is less than the score threshold are selected as candidate trajectories.
[0015] In conjunction with the first aspect, in one implementation, the step of completing the missing time-domain data frames in the ideal trajectory based on the candidate trajectory to obtain the complete target trajectory specifically includes the following steps: If the number of data frames in the candidate trajectory that match the missed detection time domain is less than the number of data frames in the ideal trajectory that match the missed detection time domain, then select the trajectory closest to the ideal trajectory from the unassociated preliminary trajectory as a candidate trajectory again, until no new candidate trajectory can be matched; otherwise, perform data frame completion on the missed detection time domain of the ideal trajectory based on the candidate trajectory to obtain the target complete trajectory.
[0016] Secondly, embodiments of this application provide a multi-stage trajectory association system, the multi-stage trajectory association system comprising: The trajectory processing module is used to obtain multiple preliminary trajectories of the target vehicle based on the fused data; classify the preliminary trajectories to obtain multiple starting trajectories, process trajectories, and ending trajectories; and perform trajectory association by combining the relative relationships between the trajectories to obtain multiple preliminary complete trajectories. Each preliminary complete trajectory includes a starting trajectory, process trajectory, and ending trajectory with a one-to-one correspondence. The trajectory completion module is used to merge all data frames of the preliminary complete trajectory, search for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than a preset third threshold to obtain the missed detection time domain; perform linear interpolation on the lateral and longitudinal distances of the missed detection time domain in the preliminary complete trajectory to obtain the ideal trajectory; select the trajectory closest to the ideal trajectory from the unassociated preliminary trajectory as the candidate trajectory, and complete the data frames of the missed detection time domain in the ideal trajectory according to the candidate trajectory to obtain the target complete trajectory.
[0017] The beneficial effects of the technical solutions provided in this application include: When performing multi-stage trajectory association, by filtering out the missed time domains of the preliminary complete trajectory, the time domain data frames are extracted from the closest candidate trajectory to complete the trajectory, thus obtaining a high-precision complete target trajectory. This solves the problem of inconsistent multi-sensor trajectories, especially for the missed detection problem in overtaking scenarios. Attached Figure Description
[0018] Figure 1 This is a flowchart illustrating an embodiment of the multi-stage trajectory association method of this application; Figure 2 This is a flowchart illustrating a specific embodiment of the multi-stage trajectory association method of this application; Figure 3 This is a flowchart illustrating the connection between the beginning and end of a specific embodiment of this application; Figure 4 This is a flowchart illustrating process associations in a specific embodiment of this application; Figure 5 This is a flowchart illustrating the secondary association process in a specific embodiment of this application; Figure 6 This is a schematic diagram of the functional modules of an embodiment of the multi-stage trajectory association system of this application. Detailed Implementation
[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0021] Firstly, embodiments of this application provide a multi-stage trajectory association method.
[0022] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-stage trajectory association method of this application. Figure 1 As shown, multi-stage trajectory association methods include: Step S1: Obtain multiple preliminary trajectories of the target vehicle based on the fused data processing.
[0023] Step S2: Classify the preliminary trajectory to obtain multiple starting trajectories, process trajectories, and ending trajectories.
[0024] Step S3: Combine the relative relationships between the trajectories to perform trajectory association, and obtain multiple preliminary complete trajectories. Each preliminary complete trajectory includes a starting trajectory, a process trajectory, and a ending trajectory with a one-to-one correspondence. Step S4: After merging all data frames of the preliminary complete trajectory, search for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than the preset third threshold to obtain the missed detection time domain.
[0025] Step S5: Perform linear interpolation on the horizontal and vertical distances of the missed time domain in the preliminary complete trajectory to obtain the ideal trajectory.
[0026] Step S6: Select the trajectory closest to the ideal trajectory from the unassociated preliminary trajectories as the candidate trajectory, and use the candidate trajectory to complete the missing time-domain data frames in the ideal trajectory to obtain the complete target trajectory. "Closest to each other" means that the two curves have the highest degree of overlap in the same coordinate system or that the coefficients of the two curves are most consistent within the error range.
[0027] In this embodiment, when performing multi-stage trajectory association, the missed time domain of the preliminary complete trajectory is filtered out, and the time domain data frames are extracted from the closest candidate trajectory to complete the trajectory, thereby obtaining a high-precision target complete trajectory. This solves the problem of inconsistent multi-sensor trajectories, especially the problem of missed detection in overtaking scenarios.
[0028] In one specific embodiment, refer to Figure 2 Taking the scenario of a target vehicle overtaking another vehicle as an example, the missed detection scenarios during the overtaking process are extracted. The implementation steps are divided into the following steps: trajectory generation, trajectory filtering, trajectory classification, trajectory association, missed detection identification, and trajectory visualization. Among them, trajectory association is the main descriptive point of multi-stage trajectory matching, which includes the following: first and last association, process association, and secondary association.
[0029] Furthermore, in one embodiment, step S1 specifically includes the following: The system acquires fused data of the target vehicle detected by multiple source sensors of the vehicle itself. This fused data includes the fusion ID of the data frame, timestamp, location coordinates, and confidence score. The fusion ID is generated from the unique identifiers of the multiple sensors. The confidence score is generated from the multi-sensor confidence scores.
[0030] Based on the above fused data, multiple preliminary trajectories are generated. Each preliminary trajectory has the same fusion ID and the time interval between frames is no greater than the first threshold.
[0031] The preliminary trajectories are then filtered out, and those that do not conform to the target scenario or have a confidence level below the second threshold are removed.
[0032] In this embodiment, text data (using a CSV file as an example) parsed from the collected data is analyzed. By analyzing the fused data (taking visual and millimeter-wave radar as fusion sources, the same applies to other fusion sources), different trajectories are distinguished according to the fused ID (the fusion ID is related to sensor type, target ID, and category; due to the presence of multiple sensors, the fusion ID will change, resulting in different trajectories). Taking the post-fusion method upon which this invention is based as an example, the fusion frequency is 10Hz, and the fusion sensors are 5V5R (5 visual sensors, including forward, left-front, right-front, left-rear, and right-rear; 5 millimeter-wave radar sensors, including forward, left-front, right-front, left-rear, and right-rear). The target's position coordinates are located at the center point of the vehicle's front end touching the ground; forward is the positive X direction, left is the positive Y direction, and upward is the positive Z direction. Only scenarios where targets in adjacent lanes overtake the vehicle are extracted. The number and type of sensors are not limited.
[0033] During trajectory generation, trajectories are grouped and distinguished by fusion ID and inter-frame time interval Dt. Trajectories with Dt less than 0.185s are considered continuous frames, those between 0.185s and 1s are still considered to be in the same group, but there are broken frames. Trajectories with Dt greater than 1s are assigned to other groups, resulting in different trajectories.
[0034] During trajectory filtering, the monotonicity of the target's longitudinal distance is used to determine whether the target vehicle is moving backward (the criterion is that if the target has more than 2 frames, the target vehicle does not pass in front of it, and the longitudinal distance monotonically decreases, then the target vehicle is considered to be moving backward), and trajectories of the target vehicle moving backward are removed. Trajectories with all data frames having low confidence are also removed (e.g., confidence less than 80 indicates a low-confidence target). Additionally, lanes are numbered according to the following rules: Assuming a lane width of 3.75m, the current lane is 1, the right adjacent lane is 2, the left adjacent lane is 3, the right adjacent lane is 4, and the left adjacent lane is 5. Lateral changes are coded as follows: right lane change is 1, left lane change is 2, rightward deviation is 3, and leftward deviation is 4.
[0035] After trajectory generation and trajectory filtering, multiple preliminary trajectories are obtained.
[0036] Furthermore, in one embodiment, step S2 specifically includes the following: The initial trajectory is defined as follows: the relative speed between the target vehicle and the current vehicle is greater than a first speed threshold; the relative longitudinal distance conforms to a first distance interval; the extreme difference of the relative longitudinal distance is greater than a first extreme difference threshold; the relative lateral distance conforms to a second distance interval; and the lane where the trajectory's endpoint is located is adjacent to the current vehicle's lane. The first speed threshold is greater than 0. The lower limit of the first distance interval is negative, and the upper limit is 0. The first extreme difference threshold is greater than 0. The lower limit of the second distance interval is negative, and the upper limit is positive.
[0037] The initial trajectory where the relative speed between the target vehicle and the vehicle exceeds a second speed threshold, the relative longitudinal distance conforms to a second distance interval, and the extreme difference of the relative longitudinal distance exceeds a second extreme difference threshold is defined as the cutoff trajectory. The second speed threshold is greater than 0. The lower limit of the second distance interval is 0, and the upper limit is a positive number. The second extreme difference threshold is greater than 0.
[0038] The preliminary trajectory, other than the initial trajectory and the ending trajectory, is defined as the process trajectory.
[0039] In this embodiment, when classifying the preliminary trajectories, the starting trajectory and the ending trajectory are mainly distinguished. After distinguishing the starting trajectory and the ending trajectory, the remaining preliminary trajectories are process trajectories.
[0040] The conditions for determining the starting trajectory are: the average relative speed of the target is greater than 0.1 m / s, the maximum longitudinal distance of the target is between -20 m and 0 m, the extreme difference of the longitudinal distance of the target is greater than 20 m, the average lateral distance of the target is between -5.6 m and 5.6 m, and the last digit of the reverse lane code is 2 or 3.
[0041] The criteria for determining the cutoff trajectory are: the average relative velocity of the target is greater than 0.1 m / s, the maximum longitudinal distance of the target is between 0 m and 45 m, and the extreme difference of the longitudinal distance of the target is greater than 20 m.
[0042] Furthermore, in one embodiment, the aforementioned relative relationships include one or any combination of time relationships, lane position relationships, relative speed relationships, relative longitudinal distance relationships, and / or relative lateral distance relationships.
[0043] In this embodiment, when performing trajectory association, the starting trajectory and the ending trajectory are first associated through the beginning and end association, then the process trajectory is associated with the paired starting trajectory and the ending trajectory, and finally a secondary association is performed. The secondary association is used to filter and complete the missed time points of the preliminary complete trajectory in order to obtain a more accurate target complete trajectory.
[0044] In one specific embodiment, refer to Figure 3 The first step is head-to-tail correlation, which involves matching the starting and ending trajectories obtained in the previous steps. Each starting trajectory is iterated through, and the best-matching trajectory is found among the ending trajectories. This process is divided into initial screening and fine screening. Initial screening includes: the starting time of the ending trajectory is greater than the ending time of the starting trajectory; the ending and starting trajectories are in the same lane direction; and the ending trajectory cannot be completely located in adjacent lanes. Fine screening requires calculating the matching score between the starting trajectory and the candidate ending trajectories selected in the initial screening. The calculation parameters include: The maximum longitudinal distance of the initial trajectory X is D_start_lon_max.
[0045] The average relative velocity V_start_last_n_ave of the last N1 frames of the initial trajectory X.
[0046] The end timestamp of the starting trajectory X is T_start_end_ts.
[0047] The average lateral distance YD_start_last_n_ave of the last N1 frames of the initial trajectory X.
[0048] The minimum longitudinal distance D_end_lon_min of the candidate trajectory Y.
[0049] The average relative velocity V_end_first_n_ave for the N2 frames preceding the candidate trajectory Y.
[0050] The start timestamp of candidate trajectory Y is T_end_start_ts.
[0051] The average lateral distance YD_end_first_n_ave for the N2 frames preceding the candidate trajectory Y.
[0052] The calculation formula is shown in formula (1) below: (1).
[0053] Arrange the matching scores in ascending order, remove the trajectories with matching scores greater than M1, and select the unused cutoff trajectory with the smallest matching score as the optimal matching trajectory for the starting trajectory.
[0054] Reference Figure 4 Secondly, there is process correlation. First, each start-end trajectory matching pair is traversed, and then the process trajectory is initially screened and then finely screened.
[0055] The initial screening includes timestamp filtering, vertical distance filtering, horizontal distance filtering, and unused filtering.
[0056] The input parameters for timestamp filtering are as follows: The maximum longitudinal distance of the initial trajectory X is D_start_lon_max.
[0057] The average relative velocity V_start_last_n_ave of the last N1 frames of the initial trajectory X.
[0058] The end timestamp of the starting trajectory X is T_start_end_ts.
[0059] The minimum longitudinal distance D_end_lon_min of the candidate trajectory Y.
[0060] The average relative velocity V_end_first_n_ave for the N2 frames preceding the candidate trajectory Y.
[0061] The start timestamp of candidate trajectory Y is T_end_start_ts.
[0062] The maximum longitudinal distance threshold for the initial trajectory X is D_START_LON_MAX_THRES (preferably -25).
[0063] The minimum longitudinal distance threshold D_END_LON_MIN_THRES for the cutoff trajectory Y (preferably 40).
[0064] The fault tolerance threshold for longitudinal distance is D_LON_FAULT_TOLERANT (preferred 5).
[0065] The start timestamp of the process trajectory Z is T_start_ts.
[0066] The calculation formula for screening is as follows (2): (2).
[0067] in, .
[0068] Filtering criteria are .
[0069] The input parameters for vertical distance filtering are as follows: The maximum longitudinal distance of the initial trajectory X is D_start_lon_max.
[0070] The minimum longitudinal distance D_end_lon_min of the candidate trajectory Y.
[0071] The maximum longitudinal distance threshold for the initial trajectory is D_START_LON_MAX_THRES (preferably -25).
[0072] The minimum longitudinal distance threshold for the cutoff trajectory is D_END_LON_MIN_THRES (preferably 40).
[0073] The fault tolerance threshold for longitudinal distance is D_LON_FAULT_TOLERANT (preferred 5).
[0074] The minimum longitudinal distance of the process trajectory Z is D_lon_min.
[0075] The maximum longitudinal distance D_lon_max of the process trajectory Z.
[0076] The calculation formula for screening is as follows (3): (3).
[0077] Filtering criteria are .
[0078] The horizontal distance filter input parameters are as follows: The reverse-time lane code L_start_lat_track_last for the initial trajectory X.
[0079] The sequential lane code L_lat_track_first for the process trajectory Z.
[0080] The calculation formula for screening is as follows (4): (4).
[0081] Filtering criteria are .
[0082] The following input parameters were not used: U_used is the usage status of the process trajectory Z.
[0083] Filtering criteria are .
[0084] The fine-tuning of process correlation requires matching the starting and ending trajectories with the process trajectory separately, and then taking a weighted average of the results to obtain the final score. The input parameters are as follows: The maximum longitudinal distance of the initial trajectory X is D_start_lon_max.
[0085] The average relative velocity V_start_last_n_ave of the last N1 frames of the initial trajectory X.
[0086] The end timestamp of the starting trajectory X is T_start_end_ts.
[0087] The maximum longitudinal distance D_process_lon_max of the process trajectory Z1.
[0088] The average relative velocity V_process_last_n_ave of the last N1 frames of the process trajectory Z1.
[0089] The end timestamp of process trajectory Z1 is T_process_end_ts.
[0090] Cutoff trajectory and process trajectory Z2 (process trajectory Z2 is the trajectory whose end time is less than the start time of the cutoff trajectory).
[0091] The minimum longitudinal distance of the trajectory Y is D_end_lon_min.
[0092] The average relative velocity V_end_first_n_ave for the N2 frames preceding the trajectory Y.
[0093] The start timestamp of trajectory Y is T_end_start_ts.
[0094] The minimum longitudinal distance of process trajectory Z2 is D_process_lon_min.
[0095] The average relative velocity V_process_first_n_ave of the first N2 frames of the process trajectory Z2.
[0096] The start timestamp of process trajectory Z2 is T_process_start_ts.
[0097] The start and end trajectory time difference matching percentage START_TS_DIFF_RATIO (preferably 0.8).
[0098] The calculation formula for screening is as follows (5): (5).
[0099] Arrange the matching scores of the above calculated trajectories in ascending order, and filter the trajectories using the following rule: .
[0100] If the process trajectory obtained using rule 1 is empty, then use rule 2 as follows for filtering: .
[0101] The currently used filtering thresholds are score_factor (preferred 1.5) and shift_ratio_thres (preferred 3.5), which can be adjusted according to data and scenario requirements.
[0102] Reference Figure 5Finally, there is a secondary association, which needs to be associated with the missed detection situation. First, each preliminary complete trajectory chain is traversed to search for the time point of the target missed detection. If there is a missed detection, the missed detection trajectory is matched again until there is no need to check for missed detection again.
[0103] Furthermore, in one embodiment, the above-mentioned linear interpolation of the horizontal and vertical distances in the time domain of the missed detections in the preliminary complete trajectory to obtain the ideal trajectory specifically includes the following steps: Linear interpolation is performed on the horizontal and vertical distances of the missed time domain in the preliminary complete trajectory to obtain the set of true value points.
[0104] The set of truth points is used as the ideal trajectory.
[0105] Furthermore, in one embodiment, selecting the trajectory closest to the ideal trajectory from the aforementioned unassociated preliminary trajectories as a candidate trajectory specifically includes the following steps: Based on the time interval, relative horizontal distance interval, and relative vertical distance interval of the missed detection time domain, multiple undetermined trajectories are extracted from the initial trajectory that has not been associated, and data frames that conform to the missed detection time domain are extracted from each undetermined trajectory as a set of undetermined points.
[0106] Calculate the absolute values of the relative lateral distance difference and the relative longitudinal distance difference between each point in the set of undetermined points and the set of true points. The undetermined trajectory whose sum of the average values of the relative lateral distance difference and the relative longitudinal distance difference is less than the score threshold (preferably 18.1) is selected as the candidate trajectory.
[0107] In this embodiment, when searching for the time point when the target was missed, the data frames of each trajectory are merged according to the trajectory number contained in the target.
[0108] Frame break search: The time domain where the time difference between two frames is greater than T1 (preferably 0.185s) is the time domain that was missed.
[0109] Low confidence search: The time domain in which the confidence of all data frames at the same time is less than C (preferably 80) is the time domain of missed detection.
[0110] Lateral deviation search: The time domain in which the minimum absolute value of the lateral distance of all data frames at the same time is greater than S1 (preferably 5.6) is the time domain in which the detection is missed.
[0111] Based on the time domain of the missed detection, the data DF of the missed detection information is obtained.
[0112] Traverse the undetected time domain with the same target number, perform time linear interpolation on the horizontal and vertical distances of the undetected time domain to obtain the true value point set Truth, and initially screen out the process trajectories that may be missed through conditions such as the time interval, vertical distance interval, horizontal distance interval, and usage of the undetected time domain. Screen the data in the original point set through timestamps to obtain the point set Tracks that needs further analysis. Group Tracks by id, calculate the average of the absolute values of the point-to-point differences in the horizontal and vertical distances between Tracks and Truth, add the above horizontal and vertical values to obtain score, and select the pending trajectories with score < raw_points_thres (18.1) as candidate trajectories.
[0113] Search for the undetected time points of the target for undetected inspection again according to the above steps to obtain the data DF1 of the undetected information.
[0114] Furthermore, in an embodiment, the above-mentioned data frame of the undetected time domain in the ideal trajectory is complemented according to the candidate trajectory to obtain the target complete trajectory, which specifically includes the following steps: Judge whether the number of data frames that meet the undetected time domain in the candidate trajectory is less than the number of data frames that meet the undetected time domain in the ideal trajectory. If so, select the trajectory closest to the ideal trajectory from the unassociated preliminary trajectories again as the candidate trajectory until no new candidate trajectory can be matched. If not, complement the data frame of the undetected time domain in the ideal trajectory according to the candidate trajectory to obtain the target complete trajectory.
[0115] In this embodiment, it is judged whether it is necessary to detect the undetected again. Compare the data sizes of DF and DF1. If DF1 < DF, loop the above two steps again. If DF1 and DF have the same data, terminate the undetected inspection of this target number and traverse the next target number.
[0116] In a preferred embodiment, the finally matched target trajectory is visualized. The abscissa is the direction of vehicle travel, from left to right in the picture is from back to front of the actual road condition, the ordinate is the vehicle width, from bottom to top in the picture is from right to left of the actual road condition, the center of the front of the target vehicle is located at the coordinate origin, the ordinate is marked according to the width of one lane, the lane width is drawn according to a length of 3.75m, the abscissa is marked at intervals of 10m, different trajectories of the same target are represented by point sets with large color differences, and different trajectories are sorted and numbered according to the time of trajectory appearance. The naming rule of the trajectory label is: <sequence number>- <id>-<Number of frames>-<Number of low confidence scores>-<Number of broken frames>-<Matching score>.
[0117] In one specific embodiment, on a highway, a vehicle is traveling at a certain speed in the center of the same lane. A vehicle is rapidly approaching from the left or right rear lane. During the overtaking process of the target vehicle, there are instances of missed detection (missed detections are categorized into three types: frame breaks, low confidence levels, and lateral deviation). The goal is to identify all the time points of these missed detections. Using the method described in this patent, a 100% accuracy rate for missing detection extraction can be achieved.
[0118] In summary, the phased trajectory matching and association method proposed in this invention matches step by step, gradually narrowing the search range, with controllable accuracy and high efficiency. In particular, the method combines initial screening and fine screening in the closing and process association stages, and the subsequent use of secondary association increases the reliability of the results.
[0119] Secondly, embodiments of this application also provide a multi-stage trajectory association system.
[0120] In one embodiment, reference is made to Figure 6 , Figure 6 This is a functional module diagram of an embodiment of the multi-stage trajectory association system of this application. Figure 6 As shown, the multi-stage trajectory association system includes: The trajectory processing module 1 is used to obtain multiple preliminary trajectories of the target vehicle based on the fused data. The preliminary trajectories are classified into multiple starting trajectories, process trajectories, and ending trajectories. Trajectories are correlated by combining the relative relationships between them to obtain multiple preliminary complete trajectories. Each preliminary complete trajectory includes a starting trajectory, a process trajectory, and an ending trajectory with a one-to-one correspondence. The trajectory completion module 2 merges all data frames of the preliminary complete trajectory, searches for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than a preset third threshold to obtain the missed detection time domain. Linear interpolation is performed on the lateral and longitudinal distances of the missed detection time domain in the preliminary complete trajectory to obtain the ideal trajectory. The trajectory closest to the ideal trajectory is selected from the unassociated preliminary trajectories as a candidate trajectory, and data frames in the missed detection time domain of the ideal trajectory are completed based on the candidate trajectory to obtain the target complete trajectory.
[0121] The functions of each module in the above-mentioned multi-stage trajectory association system correspond to the steps in the above-mentioned multi-stage trajectory association method embodiment, and their functions and implementation processes will not be described in detail here.
[0122] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0123] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0124] In the description of the embodiments in this application, terms such as "exemplary," "for example," or "for instance" are used to indicate that they are examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.
[0125] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0126] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0128] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.< / id>
Claims
1. A multi-stage trajectory association method, characterized in that, The multi-stage trajectory association method includes: Multiple preliminary trajectories of the target vehicle are obtained through fusion data processing; the preliminary trajectories are classified to obtain multiple starting trajectories, process trajectories, and ending trajectories; the relative relationships between the trajectories are combined to obtain multiple preliminary complete trajectories, each of which includes a starting trajectory, a process trajectory, and an ending trajectory with a one-to-one correspondence. After merging all data frames of the preliminary complete trajectory, the system searches for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than a preset third threshold to obtain the missed detection time domain. The system performs linear interpolation on the lateral and longitudinal distances of the missed detection time domain in the preliminary complete trajectory to obtain the ideal trajectory. The system selects the trajectory that is closest to the ideal trajectory from the unassociated preliminary trajectory as the candidate trajectory, and completes the data frames of the missed detection time domain in the ideal trajectory based on the candidate trajectory to obtain the target complete trajectory.
2. The multi-stage trajectory association method as described in claim 1, characterized in that, The process of obtaining multiple preliminary trajectories of the target vehicle based on fused data processing specifically includes the following steps: The system acquires fused data of the target vehicle detected by multiple source sensors of the vehicle. The fused data includes the fusion ID of the data frame, timestamp, location coordinates, and confidence score. The fusion ID is generated by a unique identifier of the multiple sensors. The confidence score is generated by the confidence score of the multiple sensors. Multiple preliminary trajectories are generated based on the fused data. Each preliminary trajectory has the same fusion ID and the time interval between frames is no greater than a first threshold. The preliminary trajectories are screened, and those that do not conform to the target scenario or have a confidence level below the second threshold are removed.
3. The multi-stage trajectory association method as described in claim 1, characterized in that, The process of classifying the preliminary trajectory to obtain multiple initial trajectories, ending trajectories, and process trajectories includes the following steps: The initial trajectory is defined as follows: the relative speed between the target vehicle and the vehicle is greater than a first speed threshold; the relative longitudinal distance conforms to a first distance interval; the extreme difference of the relative longitudinal distance is greater than a first extreme difference threshold; the relative lateral distance conforms to a second distance interval; and the lane where the trajectory endpoint is located is the adjacent lane of the vehicle's lane. The first speed threshold is greater than 0; the lower limit of the first distance interval is negative and the upper limit is 0. The first extreme value difference threshold is greater than 0; the lower limit of the second distance interval is negative, and the upper limit is positive. The initial trajectory of the target vehicle and the vehicle being driven by a vehicle whose relative speed is greater than the second speed threshold, whose relative longitudinal distance is within the second distance range, and whose extreme difference in relative longitudinal distance is greater than the second extreme difference threshold is defined as the cutoff trajectory. The second speed threshold is greater than 0; the lower limit of the second distance interval is 0, and the upper limit is a positive number; The second extreme value difference threshold is greater than 0; The preliminary trajectory, other than the initial trajectory and the ending trajectory, is defined as the process trajectory.
4. The multi-stage trajectory association method as described in claim 1, characterized in that, The relative relationships include one or any combination of time relationships, lane position relationships, relative speed relationships, relative longitudinal distance relationships, and / or relative lateral distance relationships.
5. The multi-stage trajectory association method as described in claim 1, characterized in that, The missed detection time domain includes the time domain where the inter-frame time interval is greater than the fourth threshold, the time domain where the confidence of all data frames at the same time is lower than the fifth threshold, and the time domain where the absolute value of the relative lateral distance of all data frames at the same time is less than the sixth threshold.
6. The multi-stage trajectory association method as described in claim 1, characterized in that, The candidate trajectory is the trajectory that is closest to the time interval, relative horizontal distance interval, and relative vertical distance interval of the missed detection time domain in the ideal trajectory.
7. The multi-stage trajectory association method as described in claim 1, characterized in that, The process of linearly interpolating the horizontal and vertical distances of the missed detections in the preliminary complete trajectory to obtain the ideal trajectory includes the following steps: Linear interpolation is performed on the horizontal and vertical distances of the missed time domains in the preliminary complete trajectory to obtain the set of true value points; The set of truth points is used as the ideal trajectory.
8. The multi-stage trajectory association method as described in claim 7, characterized in that, The process of selecting the trajectory that is closest to the ideal trajectory from the initial trajectories that have never been associated with the first trajectory as the candidate trajectory includes the following steps: Based on the time interval, relative horizontal distance interval, and relative vertical distance interval of the missed detection time domain, multiple undetermined trajectories are extracted from the initial trajectory that has not been associated, and data frames that conform to the missed detection time domain are extracted from each undetermined trajectory as a set of undetermined points. Calculate the absolute values of the relative lateral distance difference and the relative longitudinal distance difference between each point in the set of undetermined points and the set of true points. The undetermined trajectories whose sum of the absolute values of the relative lateral distance difference and the relative longitudinal distance difference is less than the score threshold are selected as candidate trajectories.
9. The multi-stage trajectory association method as described in claim 7, characterized in that, The step of completing the missing time-domain data frames in the ideal trajectory based on the candidate trajectory to obtain the complete target trajectory specifically includes the following steps: Determine whether the number of data frames in the candidate trajectory that meet the missed detection time domain is less than the number of data frames in the ideal trajectory that meet the missed detection time domain. If so, select the trajectory that is closest to the ideal trajectory from the unassociated preliminary trajectory again as a candidate trajectory until no new candidate trajectory can be matched. If not, then the missing time domain data frames in the ideal trajectory are completed based on the candidate trajectory to obtain the complete target trajectory.
10. A multi-stage trajectory association system, characterized in that, The multi-stage trajectory association system includes: The trajectory processing module is used to obtain multiple preliminary trajectories of the target vehicle based on the fused data; classify the preliminary trajectories to obtain multiple starting trajectories, process trajectories, and ending trajectories; and perform trajectory association by combining the relative relationships between the trajectories to obtain multiple preliminary complete trajectories. Each preliminary complete trajectory includes a starting trajectory, process trajectory, and ending trajectory with a one-to-one correspondence. The trajectory completion module is used to merge all data frames of the preliminary complete trajectory, search for broken frames, low-confidence frames, and frames whose relative lateral position distance is greater than a preset third threshold to obtain the missed detection time domain; perform linear interpolation on the lateral and longitudinal distances of the missed detection time domain in the preliminary complete trajectory to obtain the ideal trajectory; select the trajectory closest to the ideal trajectory from the unassociated preliminary trajectory as the candidate trajectory, and complete the data frames of the missed detection time domain in the ideal trajectory according to the candidate trajectory to obtain the target complete trajectory.