Millimeter wave radar global vehicle trajectory splicing method based on trajectory similarity

Abstract

The invention relates to a millimeter-wave radar global vehicle track splicing method based on track similarity, which comprises the following steps of: sensing a moving vehicle on a traffic road by utilizing millimeter-wave radars arranged on the traffic road at certain intervals, and acquiring vehicle track data detected by the millimeter-wave radars; and according to the vehicle type similarity, the distance similarity, the speed similarity and the behavior similarity, vehicle tracks of the two radar monitoring blind areas and the overlapping area are matched, the matching degree of the two target vehicle tracks is judged, and finally, tracks between frame windows are spliced according to the matching degree to obtain a final target track. The method solves the problem of discontinuous cross-device multi-target trajectory tracking, solves the problem of poor splicing effect of a trajectory splicing algorithm based on a linear prediction model, and has the advantages of being independent of moving target feature information, high in precision, good in adaptability and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation perception, and particularly relates to a method for stitching millimeter-wave radar global vehicle trajectories based on trajectory similarity. Background Art

[0002] In recent years, with the rapid development of the economy and society, as well as the progress and cost reduction of roadside sensor technology, vehicle trajectory perception technology has witnessed remarkable development. Currently, the vehicle trajectory recognition methods widely used at home and abroad are mainly camera-based technologies. However, this method has certain limitations: on the one hand, cameras are not very sensitive to changes in vehicle speed, which makes it difficult to effectively stitch the vehicle trajectories captured by cameras in different sections based on the motion characteristics of the vehicle; on the other hand, for vehicles in the distance, the recognition accuracy of cameras is relatively low, and it is impossible to accurately track and identify vehicles in real time based on vehicle appearance features, resulting in the difficulty of effectively connecting vehicle trajectories based on appearance information between different sections. Therefore, the continuity of vehicle trajectories between different devices is affected.

[0003] Long-distance continuous vehicle trajectory data is crucial for analyzing vehicle operating states, monitoring illegal driving behaviors, studying driving habits, and promoting intelligent transportation management. In view of this, there is an urgent need to develop a new method that can use roadside sensors to achieve seamless stitching of vehicle trajectories. With the gradual decline of the hardware cost of millimeter-wave radars, high-precision millimeter-wave radars originally used in the military field have started to open up to the civilian market and occupy an important position in automotive collision avoidance systems. According to data from IHSMarkit, the combined occupancy rate of millimeter-wave / microwave radars and cameras in the automotive collision avoidance sensor market has reached 70%. Millimeter-wave radars are famous for their short wavelengths, wide frequency bands (i.e., large frequency ranges), and strong penetration capabilities. These characteristics endow millimeter-wave radars with unique advantages and show great application potential in the field of traffic monitoring. By using the data collected by millimeter-wave radars, vehicle trajectory information can be collected and analyzed more effectively, providing new possibilities for solving the above problems. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method for stitching millimeter-wave radar global vehicle trajectories based on trajectory similarity, which makes full use of the data returned by millimeter-wave radars and uses the reflection area and kinematic characteristics of target objects that can be obtained by millimeter-wave radar devices to match and stitch the vehicle trajectories sensed by different millimeter-wave radar devices, so as to obtain long-range continuous vehicle trajectories with high accuracy and good adaptability.

[0005] A method for stitching millimeter-wave radar global vehicle trajectories based on trajectory similarity includes the following steps:

[0006] 1) Use millimeter-wave radars installed on traffic roads to sense the moving vehicles on the traffic roads and obtain the vehicle trajectory data detected by the millimeter-wave radars. The vehicle trajectory data detected by the millimeter-wave radars includes vehicle ID, timestamp, radial coordinates of the vehicle relative to the radar, tangential coordinates of the vehicle relative to the radar, radial component of the vehicle speed, tangential component of the vehicle speed, and radar reflection area;

[0007] 2) Unify the coordinate system for all vehicle trajectory points according to the position of the millimeter-wave radar;

[0008] 3) Judge the interrupted vehicle trajectories and determine whether there is spatio-temporal overlap between two vehicle trajectories;

[0009] 4) For the data without spatio-temporal overlap, use the GAF-GNN-CNN-LSTM algorithm to predict the vehicle trajectories in the overlapping area;

[0010] 5) Use the tsfresh algorithm to perform dimensionality reduction processing on all vehicle trajectories, and reduce the time-series vehicle trajectories to a one-dimensional array N;

[0011] 6) Normalize all the one-dimensional arrays N to obtain a new one-dimensional array N 1 ;

[0012] 7) Calculate the FastDTW distance of all vehicle trajectories in the overlapping area;

[0013] 8) Calculate the trajectory matching index based on vehicle type similarity, distance proximity, speed proximity, and behavior similarity;

[0014] 9) Match and splice the two trajectories with the highest matching index.

[0015] Further, in step 4), when using the GAF-GNN-CNN-LSTM algorithm to predict the vehicle trajectories in the overlapping area, the specific steps are as follows:

[0016] a) Extract the vehicle trajectory speed time-series data, abscissa time-series data, and ordinate time-series data respectively, and use the Gram angle field to two-dimensionalize the time-series trajectory data respectively to construct a Gram angle field two-dimensional matrix M;

[0017] b) Based on the two-dimensional matrix M, construct a graph neural network to generate a graph G;

[0018] c) Use the module partitioning algorithm in the tsia graph neural network analysis tool to perform community partitioning on the obtained graph neural network to obtain different neuron groups and vehicle speed mapping relationships;

[0019] d) Map the different neuron group labels to the corresponding sub-time series;

[0020] e) Input the neuron group label and the corresponding sub-time series into the CNN-LSTM model for time series prediction;

[0021] f) Obtain the vehicle time series trajectory in the overlapping area, including the vehicle trajectory speed time series data, the abscissa time series data, and the ordinate time series data.

[0022] Further, in step 5), the tsfresh algorithm is used to perform dimensionality reduction on all vehicle trajectories, and the time series vehicle trajectories are reduced to a one-dimensional array N. The specific steps are as follows:

[0023] a) Extract the trajectory data of a 150m section from a distance of 200m to a distance of 50m at the coordinate point where the millimeter-wave radar is located as the time series trajectory dataset, where the trajectory time series feature parameters include the time series data of the horizontal and vertical speeds;

[0024] b) Based on the above time series speed data, take the derivative of the time series data of the horizontal and vertical speeds with respect to time to obtain the time series data of the horizontal and vertical accelerations;

[0025] c) Based on the above time series acceleration data, take the derivative of the time series data of the horizontal and vertical accelerations with respect to time to obtain the time series data of the horizontal and vertical jerk;

[0026] d) Extract the vehicle time series data, including: the time series data of the horizontal and vertical positions, the horizontal and vertical speeds, the horizontal and vertical accelerations, and the horizontal and vertical jerk. Use tsfresh as the time series trajectory feature extraction tool to extract the vehicle time series trajectory feature values;

[0027] e) Use the constructed large dataset to select relevant features. Tsfresh will perform a hypothesis test on each feature to check whether it is relevant to the given target, and then use the extract_relevant_features function in tsfresh to perform extraction, selection, and filtering simultaneously, and finally leave enough relevant time series trajectory features, reducing the time series vehicle trajectories to a one-dimensional array N.

[0028] Further, in step 7), for the vehicle trajectories in the overlapping area, use the vehicle trajectories in the overlapping area from a distance of 200m to a distance of 250m from the millimeter-wave radar, perform interpolation processing at a distance of 1m to obtain the rasterized vehicle trajectory points, and calculate the FastDTW distance D by pairwise matching of the rasterized vehicle trajectory points.

[0029] Further, in step 8), calculate the trajectory matching index based on the vehicle type similarity, distance proximity, speed proximity, and behavior similarity. The specific steps are as follows:

[0030] a) Calculate the vehicle type similarity X of the two vehicle trajectories based on the obtained radar reflection area. The calculation method is as follows:

[0031]

[0032] b) Calculate the FastDTW distance D between all pairs of trajectories in the overlapping area based on the FastDTW distance D calculated in claim 4. i , and take the maximum value D of the FastDTW distance in the overlapping area. max , calculate the distance similarity DL, and the calculation method is as follows:

[0033]

[0034] c) Calculate the speed value S at a distance of 250 m from the millimeter-wave radar based on the predicted time-series speed data in the overlapping area in claim 2. i and the starting value S of the downstream time-series speed j , and calculate the speed similarity SL, and the calculation method is as follows:

[0035]

[0036] d) Calculate the FastDTW distance E between all pairs of one-dimensional arrays of time-series vehicle trajectories based on the one-dimensional array N of time-series vehicle trajectories calculated in claim 3. i , and take the maximum value E of the FastDTW distance of all one-dimensional arrays. max , calculate the behavior similarity XL, and the calculation method is as follows:

[0037]

[0038] e) Calculate the trajectory matching index PL, and the calculation method is as follows:

[0039]

[0040] The vehicle trajectory splicing method based on millimeter-wave radar data provided by the present invention includes at least the following compared with the prior art

[0041] Beneficial effects:

[0042] 1) The data used by the method of the present invention to detect vehicle trajectories is the data collected by the roadside fixed millimeter-wave radar detection equipment, and the real-time radar data is adopted, which has the characteristics of high detection accuracy and fast detection speed, filling the blank in the field of trajectory matching and splicing in the field of using millimeter-wave radar to collect vehicle trajectories; during the trajectory splicing process, corresponding splicing algorithms are respectively constructed for vehicle trajectories with overlapping areas and vehicle trajectories without overlapping areas, which can solve the problem of vehicle trajectory interruption caused by radar blind spots or signal loss;

[0043] 2) The present invention adopts a trajectory prediction method. For vehicle trajectory data without spatio-temporal overlap, a two-dimensionalization method of time-series data is used to supplement the time-series data subsequence tags, improve the prediction accuracy of time-series data, and complete the vehicle trajectory in the spatio-temporal overlap area, thereby realizing the judgment of the distance similarity of vehicle trajectories without spatio-temporal overlap, and solving the problems of discontinuous target tracking trajectories sensed by a single radar and discontinuous target tracking trajectories sensed by multiple radars.

[0044] 3) The present invention constructs four-dimensional vehicle trajectory matching indexes of vehicle type similarity, distance similarity, speed similarity, and behavior similarity respectively, and finally constructs a vehicle trajectory matching index to achieve high-precision matching of vehicle trajectories in the blind areas and overlapping areas monitored by two radars. It fully considers the kinematic characteristics of vehicles and greatly reduces the splicing failure rate. Only by using the data obtained by the millimeter-wave radar can the continuous trajectory of the vehicle be accurately obtained, without relying on high-precision GPS installed on the vehicle, with low cost and higher adaptability. Description of the Drawings

[0045] Figure 1 It is a schematic flow chart of the millimeter-wave radar full-domain vehicle trajectory splicing method based on trajectory similarity in the embodiment.

[0046] Figure 2 It is a schematic diagram of the millimeter-wave radar sensing range and overlapping area of the vehicle trajectory detection method based on millimeter-wave radar data in the embodiment. Detailed Embodiment

[0047] The following further details a millimeter-wave radar full-domain vehicle trajectory splicing method based on trajectory similarity of the present invention in conjunction with the drawings and specific embodiments:

[0048] Embodiment:

[0049] A millimeter-wave radar full-domain vehicle trajectory splicing method based on trajectory similarity first uses millimeter-wave radars installed on traffic roads to sense moving vehicles on the traffic roads and obtain vehicle trajectory data and vehicle radar reflection data detected by the millimeter-wave radars.

[0050] By installing the millimeter-wave radar on a pole at a certain height and tilting it appropriately, the detection and perception of the position of objects within a certain distance range can be realized. In this embodiment, the millimeter-wave radar sensing range is as Figure 2 shown. By setting crossbars at a certain height on both sides of the road and installing the millimeter-wave radar in the center of the crossbar, the position, speed and other information of objects including vehicles on the lane can be detected. For a three-lane road, when the pole height is set to 8m, the millimeter-wave radar can obtain a detection range with a length of 250m and a width exceeding the overall road width. At this time, the layout spacing of the two millimeter-wave radars is 200 meters, and the overlapping area is 50 meters.

[0051] The fields of the vehicle trajectory data detected by the millimeter-wave radar include: vehicle ID, timestamp, radial coordinate of the vehicle relative to the radar, tangential coordinate of the vehicle relative to the radar, radial component of the vehicle speed, and tangential component of the vehicle speed.

[0052] Unify the coordinate system for all vehicle trajectory points according to the position of the millimeter-wave radar.

[0053] Judge the interrupted vehicle trajectories to determine whether there is spatio-temporal overlap between the two vehicle trajectories;

[0054] For the data without spatio-temporal overlap, use the GAF-GNN-CNN-LSTM algorithm to predict the vehicle trajectories in the overlapping area. Specifically:

[0055] 1) Extract the time series data of vehicle trajectory speed, abscissa time series data, and ordinate time series data respectively. Use the Gramian Angular Field (GAF) to two-dimensionize the time series trajectory data and construct the Gramian Angular Field two-dimensional matrix M;

[0056] 2) Based on the two-dimensional matrix M, construct a Graph Neural Network (GNN) to generate graph G;

[0057] 3) Use the module partitioning algorithm in the tsia graph neural network analysis tool developed by the Massachusetts Institute of Technology to perform community partitioning on the obtained graph neural network to obtain different neuron groups and vehicle speed mapping relationships;

[0058] 4) Map the different neuron group labels to the corresponding sub-time series;

[0059] 5) Use the neuron group labels and the corresponding sub-time series as inputs and input them into the CNN-LSTM model for time series prediction;

[0060] 6) Obtain the vehicle time series trajectories in the overlapping area, including the time series data of vehicle trajectory speed, abscissa time series data, and ordinate time series data;

[0061] Use the tsfresh algorithm to perform dimensionality reduction on all vehicle trajectories, and reduce the time series vehicle trajectories to a one-dimensional array N. Specifically:

[0062] 1) Extract the trajectory data of a 150m section from 200m to 50m away from the coordinate point where the millimeter-wave radar is located as the time series trajectory data set, and the trajectory time series feature parameters include the time series data of the horizontal and vertical speeds;

[0063] 2) Based on the above time series speed data, take the derivative of the time series data of the horizontal and vertical speeds with respect to time to obtain the time series data of the horizontal and vertical accelerations;

[0064] 3) Based on the above time-series acceleration data, take the time derivative of the time-series data of the longitudinal and lateral accelerations to obtain the time-series data of the longitudinal and lateral jerk.

[0065] 4) Extract the vehicle time-series data, including: the time-series data of the longitudinal and lateral positions, longitudinal and lateral speeds, longitudinal and lateral accelerations, and longitudinal and lateral jerks. Use tsfresh as the time-series trajectory feature extraction tool to extract the vehicle time-series trajectory feature values.

[0066] 5) Use the constructed large dataset to select relevant features. Tsfresh will perform a hypothesis test on each feature to check whether it is relevant to the given target, and then use the extract_relevant_features() function in tsfresh to perform extraction, selection, and filtering simultaneously. Finally, leave enough relevant time-series trajectory features and reduce the time-series vehicle trajectory to a one-dimensional array N.

[0067] Normalize all the one-dimensional arrays N to obtain a new one-dimensional array N1.

[0068] For the vehicle trajectories in the overlapping area, use the vehicle trajectories in the overlapping area from 200m to 250m away from the millimeter-wave radar. Perform interpolation processing at a distance of 1m to obtain rasterized vehicle trajectory points. And calculate the FastDTW distance D for pairwise matching of the rasterized vehicle trajectory points.

[0069] Calculate the trajectory matching index based on vehicle type similarity, distance proximity, speed proximity, and behavior similarity. Specifically:

[0070] 1) Calculate the vehicle type similarity X between two vehicle trajectories based on the obtained radar cross-sectional area. The calculation method is as follows:

[0071]

[0072] 2) Calculate the FastDTW distance D between all pairs of trajectories in the overlapping area based on the FastDTW distance D calculated in claim 4 i , take the maximum value D of the FastDTW distance in this overlapping area max , and calculate the distance similarity DL. The calculation method is as follows:

[0073]

[0074] 3) Calculate the speed value S at 250m away from the millimeter-wave radar based on the predicted time-series speed data in the overlapping area in claim 2 i and the speed similarity SL with the starting value S of the downstream time-series speed j . The calculation method is as follows:

[0075]

[0076] 4) Calculate the FastDTW distance E between every two of all the one-dimensional arrays of the sequential vehicle trajectories N calculated according to Claim 3. i , and take the maximum value E of the FastDTW distances of all the one-dimensional arrays. max , calculate the behavior similarity XL, and the calculation method is as follows:

[0077]

[0078] 5) Calculate the trajectory matching index PL, and the calculation method is as follows:

[0079]

[0080] Through the above steps, match the trajectories in the overlapping area between two radar devices, and use a unified coordinate system to describe the positions and speeds of the two trajectories for the trajectory coordinates after matching, so as to splice them into a complete trajectory described by a unified coordinate system.

[0081] After the above steps, output the spliced continuous vehicle trajectory data. Continuously match and splice the trajectories of adjacent two radars, and continuously repeat the above method process, so as to continuously adjust and output the complete trajectory spliced when the same vehicle passes through the monitoring areas of different radars.

[0082] The millimeter-wave radar full-domain vehicle trajectory splicing method based on trajectory similarity provided by the present invention is used to solve the technical problems of multi-target trajectory tracking and cross-device splicing when millimeter-wave radar devices lack mobile target feature information. Use millimeter-wave radars arranged at a certain interval on the traffic road to sense the moving vehicles on the traffic road and obtain the vehicle trajectory data detected by the millimeter-wave radars; according to the vehicle type similarity, distance proximity, speed proximity, and behavior similarity, match the vehicle trajectories in the blind areas and overlapping areas of the two radars, judge the matching degree of the two target vehicle trajectories, and finally splice the trajectories between the frame windows according to the matching degree to obtain the target final trajectory. The present invention solves the problem of discontinuous multi-target trajectory tracking across devices, solves the problem of poor splicing effect of the trajectory splicing algorithm based on the linear prediction model, and has the advantages of not relying on mobile target feature information, high accuracy, and good adaptability.

[0083] The above has described the embodiments of the present invention in detail with reference to the examples, but the present invention is not limited to the above examples. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes made without departing from the purpose of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A millimeter-wave radar global vehicle trajectory splicing method based on trajectory similarity, characterized in that: The following steps are involved: 1) Using the millimeter-wave radar installed on the traffic road to sense the vehicles moving on the traffic road, and obtaining the vehicle trajectory data detected by the millimeter-wave radar, the vehicle trajectory data detected by the millimeter-wave radar includes vehicle ID, timestamp, radial coordinates of the vehicle relative to the radar, tangential coordinates of the vehicle relative to the radar, radial component of vehicle speed, tangential component of vehicle speed, and radar reflection area; 2) Based on the location of the millimeter-wave radar, unify the coordinate system for all vehicle trajectory points; 3) Determine whether the interrupted vehicle trajectories overlap in time and space; 4) For data without time-space overlap, the GAF-GNN-CNN-LSTM algorithm is used to predict vehicle trajectories in the overlapping area; 5) Use the tsfresh algorithm to reduce the dimension of all vehicle trajectories and reduce the time series vehicle trajectories into a one-dimensional array N; 6) Normalize all one-dimensional arrays N to obtain a new one-dimensional array N 1 ; 7) Calculate the FastDTW distance of all vehicle trajectories in the overlapping area; 8) Calculate the trajectory matching index based on vehicle model similarity, distance similarity, speed similarity, and behavior similarity; 9) Match and concatenate the two trajectories with the highest matching index.

2. The method for merging vehicle trajectories based on millimeter wave radar global trajectory similarity according to claim 1 is characterized in that: In step 4), the GAF-GNN-CNN-LSTM algorithm is used to predict the vehicle trajectory in the overlapping area. The specific steps are as follows: a) respectively extracting vehicle trajectory speed time series data, abscissa time series data and ordinate time series data, respectively using Gram angle field to two-dimensionalize the time series trajectory data, and constructing a Gram angle field two-dimensional matrix M; b) Based on the two-dimensional matrix M, a graph neural network is constructed to generate a graph G; c) Use the module partitioning algorithm in the tsia graph neural network analysis tool to perform community partitioning on the obtained graph neural network to obtain different neuron groups and vehicle speed mapping relationships; d) Mapping different neuron group labels to corresponding sub-time series; e) Taking the neuron group labels and the corresponding sub-time series as input, they are fed into the CNN-LSTM model for time series prediction; f) obtaining the vehicle time series trajectory in the overlapping area, including vehicle trajectory speed time series data, abscissa time series data and ordinate time series data.

3. The method for merging vehicle trajectories based on millimeter wave radar global trajectory similarity according to claim 1, characterized in that: In step 5), the tsfresh algorithm is used to reduce the dimension of all vehicle trajectories, and the time series vehicle trajectories are reduced to a one-dimensional array N. The specific steps are as follows: a) Extract the trajectory data of the 150m road section from 200m to 50m at the coordinate point where the millimeter-wave radar is located as the time series trajectory data set, where the trajectory time series feature parameters include the time series data of the lateral and longitudinal speeds; b) based on the above time series velocity data, the time series data of the transverse and longitudinal velocities are differentiated with respect to time to obtain the time series data of the transverse and longitudinal accelerations; c) based on the above time series acceleration data, the time series data of the lateral and longitudinal accelerations are derived with respect to time to obtain the time series data of the lateral and longitudinal accelerations; d) Extracting vehicle time series data, including: time series data of transverse and longitudinal position, transverse and longitudinal velocity, transverse and longitudinal acceleration, and transverse and longitudinal jerk, and using tsfresh as a time series trajectory feature extraction tool to extract vehicle time series trajectory feature values; e) Use the constructed large dataset to select relevant features. tsfresh will perform hypothesis testing on each feature to check whether it is relevant to the given target, and then use the extract_relevant_features function in tsfresh to perform extraction, selection and filtering at the same time, and finally leave enough relevant time series trajectory features to reduce the dimension of the time series vehicle trajectory into a one-dimensional array N.

4. The method for merging vehicle trajectories of a millimeter-wave radar based on trajectory similarity according to claim 1, characterized in that: In step 7), for the vehicle trajectory in the overlapping area, the vehicle trajectory in the overlapping area from 200m to 250m from the millimeter-wave radar is interpolated at a distance of 1m to obtain rasterized vehicle trajectory points, and the rasterized vehicle trajectory points are matched pairwise to calculate the FastDTW distance D.

5. The method for merging vehicle trajectories of a millimeter-wave radar based on trajectory similarity according to claim 1, characterized in that: In step 8), the trajectory matching index is calculated based on vehicle model similarity, distance similarity, speed similarity, and behavior similarity. The specific steps are as follows: a) Based on the obtained radar reflection area, calculate the vehicle model similarity X of the two vehicle trajectories. The calculation method is as follows: b) Based on the FastDTW distance D calculated in claim 4, calculate the FastDTW distance D between all the tracks in the overlapping area i , take the maximum FastDTW distance D of the overlapping area max , calculate the distance similarity DL, the calculation method is as follows: c) Calculate the velocity value S at a distance of 250m from the millimeter wave radar based on the time series velocity data of the overlapping area predicted in claim 2 i and the downstream timing speed starting value S j The speed similarity SL is calculated as follows: d) Based on the one-dimensional array N of time-series vehicle trajectories calculated in claim 3, calculate the FastDTW distance E between all one-dimensional arrays of trajectories i , take the maximum value E of the FastDTW distance of all one-dimensional arrays max , calculate the behavior similarity XL, the calculation method is as follows: e) Calculate the trajectory matching index PL, which is calculated as follows:

Images