A global vehicle trajectory splicing method based on trajectory similarity of millimeter wave radar

Abstract

The application relates to a kind of global vehicle trajectory splicing methods based on trajectory similarity of millimeter wave radar, utilize the millimeter wave radar with certain interval arrangement on traffic road, perceive the vehicle moving on traffic road, obtain the vehicle trajectory data detected by millimeter wave radar;According to the similarity of vehicle type, the similarity of distance, the similarity of speed, the similarity of behavior, the vehicle trajectory of two radar monitoring blind area and overlapping area is matched, the matching degree of two target vehicle trajectories is judged, finally the trajectory between frame windows is spliced according to the matching degree to obtain the final target trajectory.The application solves the discontinuity problem of multi-target trajectory tracking across devices, solves the poor splicing effect problem of trajectory splicing algorithm according to linear prediction model, and has the advantages of not depending on mobile target feature information, high precision, good adaptability and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation sensing technology, specifically to a method for stitching vehicle trajectories across the entire range using millimeter-wave radar based on trajectory similarity. Background Technology

[0002] In recent years, with the rapid development of the economy and society, and the advancement and cost reduction of roadside sensor technology, vehicle trajectory perception technology has seen significant development. Currently, the vehicle trajectory recognition methods widely used both domestically and internationally are mainly based on camera technology. However, this method has certain limitations: on the one hand, cameras are not very sensitive to changes in vehicle speed, making it difficult to effectively stitch together vehicle trajectories captured by cameras on different road sections based on vehicle motion characteristics; on the other hand, for vehicles at a distance, the camera's recognition accuracy is low, making it impossible to accurately track and identify vehicles in real time based on their appearance features, resulting in difficulties in effectively connecting vehicle trajectories based on appearance information across different road sections. Therefore, the continuity of vehicle trajectories across different devices is affected.

[0003] Long-range, continuous vehicle trajectory data is crucial for analyzing vehicle operating status, monitoring illegal driving behavior, studying driving habits, and advancing intelligent traffic management. Therefore, there is an urgent need to develop a new method that can seamlessly stitch vehicle trajectories using roadside sensors. With the gradual decrease in the hardware cost of millimeter-wave radar, high-precision millimeter-wave radar, originally used in the military field, is beginning to open up the civilian market and is occupying an important position in automotive collision avoidance systems. According to IHS Markit, millimeter-wave / microwave radar combined with cameras accounts for 70% of the automotive collision avoidance sensor market. Millimeter-wave radar is known for its short wavelength, wide bandwidth (i.e., a large frequency range), and strong penetration capabilities. These characteristics give millimeter-wave radar unique advantages, making it demonstrate great application potential in traffic monitoring. By using data collected by millimeter-wave radar, vehicle trajectory information can be collected and analyzed more effectively, providing new possibilities for solving the aforementioned problems. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity. This method fully utilizes the data returned by millimeter-wave radar and leverages the reflective area obtainable by the millimeter-wave radar equipment and the kinematic characteristics of the target object to match and stitch vehicle trajectories perceived by different millimeter-wave radar equipment, thereby obtaining long-range continuous vehicle trajectories with high accuracy and good adaptability.

[0005] A method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity includes the following steps: 1) Using millimeter-wave radar installed on traffic roads, sense vehicles moving on traffic roads and obtain vehicle trajectory data detected by millimeter-wave radar. The vehicle trajectory data detected by millimeter-wave radar includes vehicle ID, timestamp, radial coordinates of vehicle relative to radar, tangential coordinates of vehicle relative to radar, radial component of vehicle speed, tangential component of vehicle speed, and radar reflective area. 2) Based on the location of the millimeter-wave radar, a unified coordinate system is established for all vehicle trajectory points; 3) Determine whether there is any spatiotemporal overlap between the interrupted vehicle trajectories; 4) For data without spatiotemporal overlap, the GAF-GNN-CNN-LSTM algorithm is used to predict vehicle trajectories in overlapping areas; 5) The tsfresh algorithm is used to reduce the dimensionality of all vehicle trajectories, reducing the time-series vehicle trajectories to a one-dimensional array N; 6) Normalize all one-dimensional arrays N to obtain a new one-dimensional array N. 1 ; 7) Calculate the FastDTW distance of all vehicle trajectories in the overlapping area; 8) Calculate the trajectory matching index based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity; 9) Match and splice the two trajectories with the highest matching index.

[0006] Furthermore, in step 4), the GAF-GNN-CNN-LSTM algorithm is used to predict the vehicle trajectories in the overlapping areas. The specific steps are as follows: a) Extract vehicle trajectory speed time series data, horizontal axis time series data and vertical axis time series data respectively, and use Gram angle field to convert the time series trajectory data into two dimensions to construct Gram angle field two-dimensional matrix M; b) Based on the two-dimensional matrix M, construct a graph neural network to generate a graph G; c) The obtained graph neural network is divided into communities using the module partitioning algorithm in the TSIA graph neural network analysis tool to obtain different groups of neurons and vehicle speed mapping relationships; d) Map the labels of different neuron groups to their corresponding sub-time series; e) Input the neuron group labels and corresponding sub-time series as inputs into the CNN-LSTM model for time series prediction; f) Obtain the vehicle time-series trajectory in the overlapping area, including vehicle trajectory speed time-series data, horizontal axis time-series data, and vertical axis time-series data.

[0007] Furthermore, in step 5), the tsfresh algorithm is used to reduce the dimensionality of all vehicle trajectories, transforming the time-series vehicle trajectories into a one-dimensional array N. The specific steps are as follows: a) Extract trajectory data from a 150m section of road at a distance of 200m to 50m from the coordinate point of the millimeter-wave radar as a time-series trajectory dataset, where the trajectory time-series feature parameters include time-series data of lateral and longitudinal velocities. b) Based on the above time-series velocity data, the time-series data of lateral and longitudinal velocities are differentiated with respect to time to obtain the time-series data of lateral and longitudinal acceleration. c) Based on the above time-series acceleration data, the time-series data of lateral and longitudinal accelerations are differentiated with respect to time to obtain the time-series data of lateral and longitudinal accelerations; d) Extract vehicle time-series data, including: lateral and longitudinal positions, lateral and longitudinal velocities, lateral and longitudinal accelerations, and lateral and longitudinal jerk data. Use tsfresh as the time-series trajectory feature extraction tool to extract vehicle time-series trajectory feature values. e) Using the constructed large dataset to select relevant features, tsfresh will perform hypothesis testing on each feature to check whether it is relevant to the given target. Then, the extract_relevant_features function in tsfresh will be used to perform extraction, selection and filtering simultaneously, and finally leave enough relevant time-series trajectory features to reduce the time-series vehicle trajectory to a one-dimensional array N.

[0008] Furthermore, in step 7), the vehicle trajectories in the overlapping areas are interpolated at a distance of 1m from the vehicle trajectories in the overlapping areas from 200m to 250m from the millimeter-wave radar to obtain rasterized vehicle trajectory points. The FastDTW distance D is then calculated by pairwise matching of the rasterized vehicle trajectory points.

[0009] Furthermore, in step 8), the trajectory matching index is calculated based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity. The specific steps are as follows: a) Based on the obtained radar cross-section, calculate the vehicle type similarity X between the two vehicle trajectories, as follows: , b) Based on the calculated FastDTW distance D, calculate the FastDTW distance D between each pair of trajectories in the overlapping region. i Take the maximum FastDTW distance D in the overlapping region. max The distance similarity (DL) is calculated as follows: ; c) Based on the predicted temporal velocity data of the overlapping area, calculate the velocity value S at a distance of 250m from the millimeter-wave radar. i With downstream timing velocity initial value S j The velocity similarity SL is calculated as follows: ; d) Based on the calculated one-dimensional array N of time-series vehicle trajectories, calculate the FastDTW distance E between all pairs of the one-dimensional arrays of trajectories. i Take the maximum distance E from all one-dimensional arrays FastDTW max The behavioral similarity score XL is calculated as follows: ; e) Calculate the trajectory matching index PL, as follows: .

[0010] The vehicle trajectory stitching method based on millimeter-wave radar data provided by this invention has at least the following advantages compared to existing technologies: 1) The data used by the method of this invention to detect vehicle trajectories is collected by roadside fixed millimeter-wave radar detection equipment. It uses real-time radar data, which has the characteristics of high detection accuracy and fast detection speed, filling the gap in the field of trajectory matching and stitching in the field of vehicle trajectory collection using millimeter-wave radar. In the trajectory stitching process, corresponding stitching algorithms are constructed for vehicle trajectories with overlapping areas and vehicle trajectories without overlapping areas, which can solve the problem of vehicle trajectory discontinuity caused by radar blind spots or signal loss. 2) This invention uses trajectory prediction to supplement time series data subsequence labels by using a two-dimensional time series data method for vehicle trajectory data that does not overlap in time and space, thereby improving the prediction accuracy of time series data and completing vehicle trajectory in the area of ​​overlapping time and space. This enables the judgment of the distance similarity of vehicle trajectories that do not overlap in time and space, and solves the problems of discontinuous target tracking trajectory perceived by a single radar and discontinuous target tracking trajectory perceived by multiple radars. 3) This invention constructs four dimensions of vehicle trajectory matching indicators: vehicle model similarity, distance similarity, speed similarity, and behavior similarity. Finally, it builds a vehicle trajectory matching index, achieving high-precision matching of vehicle trajectories in two radar monitoring blind spots and overlapping areas. It fully considers vehicle kinematic characteristics, significantly reducing the stitching failure rate. It accurately obtains continuous vehicle trajectories based solely on data acquired from millimeter-wave radar, without relying on high-precision GPS devices on the vehicle, resulting in lower costs and higher adaptability. Attached Figure Description

[0011] Figure 1This is a schematic diagram of the process for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity in the embodiment.

[0012] Figure 2 This is a schematic diagram of the millimeter-wave radar sensing range and overlapping area of ​​the vehicle trajectory detection method based on millimeter-wave radar data in the embodiment. Detailed Implementation

[0013] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity, in further detail: Example

[0014] A method for stitching vehicle trajectories across the entire domain using millimeter-wave radar based on trajectory similarity first utilizes millimeter-wave radar installed on traffic roads to sense moving vehicles on the roads and acquire vehicle trajectory data and vehicle radar reflection data detected by the millimeter-wave radar.

[0015] Millimeter-wave radar, by being mounted on a pole at a certain height and tilted appropriately, can detect and sense the position of objects within a certain distance range. In this embodiment, the sensing range of the millimeter-wave radar is as follows: Figure 2 As shown, millimeter-wave radar is installed in the center of horizontal bars at a certain height on both sides of the road to detect information such as the position and speed of objects in the lane, including vehicles. For a three-lane road, when the pole height is 8m, the millimeter-wave radar can detect a length of 250m and a width exceeding the overall road width. In this case, the spacing between two millimeter-wave radars is 200 meters, and the overlap area is 50 meters.

[0016] The fields of vehicle trajectory data detected by millimeter-wave radar include: vehicle ID, timestamp, radial coordinates of the vehicle relative to the radar, tangential coordinates of the vehicle relative to the radar, radial component of vehicle speed, and tangential component of vehicle speed.

[0017] A unified coordinate system is established for all vehicle trajectory points based on the location of the millimeter-wave radar.

[0018] The interrupted vehicle trajectory is judged to determine whether there is a spatiotemporal overlap between the two vehicle trajectories; For data without spatiotemporal overlap, the GAF-GNN-CNN-LSTM algorithm is used to predict vehicle trajectories in overlapping regions. Specifically: 1) Extract vehicle trajectory speed time series data, horizontal axis time series data and vertical axis time series data respectively, and use Gram angle field (GAF) to convert the time series trajectory data into two dimensions to construct a Gram angle field two-dimensional matrix M; 2) Based on the two-dimensional matrix M, construct a graph neural network (GNN) to generate graph G; 3) The obtained graph neural network is divided into communities using the module partitioning algorithm in the tsia graph neural network analysis tool developed by MIT, resulting in different groups of neurons and vehicle speed mapping relationships; 4) Map the labels of different neuron groups to their corresponding sub-time series; 5) Input the neuron group labels and their corresponding sub-time series into the CNN-LSTM model for time series prediction; 6) Obtain the vehicle time-series trajectory in the overlapping area, including vehicle trajectory speed time-series data, horizontal axis time-series data, and vertical axis time-series data; The tsfresh algorithm is used to reduce the dimensionality of all vehicle trajectories, transforming the time-series vehicle trajectories into a one-dimensional array N. Specifically: 1) Extract trajectory data from a 150m section of road at a distance of 200m to 50m from the coordinate point of the millimeter-wave radar as a time-series trajectory dataset. The trajectory time-series feature parameters include time-series data of lateral and longitudinal velocities. 2) Based on the above time-series velocity data, the time-series data of lateral and longitudinal velocities are differentiated with respect to time to obtain the time-series data of lateral and longitudinal acceleration. 3) Based on the above time-series acceleration data, the time-series data of lateral and longitudinal accelerations are differentiated with respect to time to obtain the time-series data of lateral and longitudinal accelerations; 4) Extract vehicle time-series data, including: lateral and longitudinal positions, lateral and longitudinal velocities, lateral and longitudinal accelerations, and lateral and longitudinal jerk data. Use tsfresh as the time-series trajectory feature extraction tool to extract vehicle time-series trajectory feature values. 5) Using the constructed large dataset to select relevant features, tsfresh will perform hypothesis testing on each feature to check whether it is relevant to the given target. Then, the extract_relevant_features() function in tsfresh will be used to perform extraction, selection and filtering simultaneously, and finally leave enough relevant time-series trajectory features to reduce the time-series vehicle trajectory to a one-dimensional array N. Normalize all one-dimensional arrays N to obtain a new one-dimensional array N1; For vehicle trajectories in overlapping areas, the vehicle trajectories in the overlapping areas from 200m to 250m away from the millimeter-wave radar are interpolated at 1m intervals to obtain rasterized vehicle trajectory points. Then, pairwise matching is performed on the rasterized vehicle trajectory points to calculate the FastDTW distance D. The trajectory matching index is calculated based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity. Specifically: 1) Based on the obtained radar cross-section, calculate the vehicle type similarity X of the two vehicle trajectories. The calculation method is as follows: ; 2) Based on the calculated FastDTW distance D, calculate the FastDTW distance D between each pair of all trajectories in the overlapping region. i Take the maximum FastDTW distance D in the overlapping region. max The distance similarity (DL) is calculated as follows: ; 3) Based on the predicted time-series velocity data of the overlapping area, calculate the velocity value S at a distance of 250m from the millimeter-wave radar. i With downstream timing velocity initial value S j The velocity similarity SL is calculated as follows: ; 4) Based on the calculated one-dimensional array N of time-series vehicle trajectories, calculate the FastDTW distance E between each pair of all trajectory one-dimensional arrays. i Take the maximum distance E from all one-dimensional arrays FastDTW max The behavioral similarity score XL is calculated as follows: ; 5) Calculate the trajectory matching index PL, as follows: .

[0019] Through the above steps, the trajectories of the overlapping areas between the two radar devices are matched, and the position and velocity of the two trajectories are described using a unified coordinate system, thus stitching them together into a complete trajectory described by a unified coordinate system.

[0020] After the above steps, the stitched continuous trajectory data of the vehicle is output. The trajectories of adjacent radars are continuously matched and stitched together, and this process is repeated continuously to adjust and output the complete trajectory stitched together when the same vehicle passes through different radar monitoring areas.

[0021] This invention provides a millimeter-wave radar full-domain vehicle trajectory stitching method based on trajectory similarity, addressing the technical challenges of multi-target trajectory tracking and cross-device stitching in millimeter-wave radar equipment when lacking moving target feature information. Utilizing millimeter-wave radars deployed at regular intervals along traffic roads, the method senses moving vehicles and acquires vehicle trajectory data detected by the millimeter-wave radar. Based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity, the method matches vehicle trajectories in two radar blind zones and overlapping areas, determining the degree of matching between the two target vehicle trajectories. Finally, the trajectories between frame windows are stitched together according to the degree of matching to obtain the final target trajectory. This invention solves the problem of discontinuous multi-target trajectory tracking across devices and addresses the poor stitching effect of trajectory stitching algorithms based on linear prediction models. It also boasts advantages such as independence from moving target feature information, high accuracy, and good adaptability.

[0022] The embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments. Various changes that can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity, characterized in that, Includes the following steps: 1) Using millimeter-wave radar installed on traffic roads, sense vehicles moving on traffic roads and obtain vehicle trajectory data detected by millimeter-wave radar. The vehicle trajectory data detected by millimeter-wave radar includes vehicle ID, timestamp, radial coordinates of vehicle relative to radar, tangential coordinates of vehicle relative to radar, radial component of vehicle speed, tangential component of vehicle speed, and radar reflective area. 2) Based on the location of the millimeter-wave radar, a unified coordinate system is established for all vehicle trajectory points; 3) Determine whether there is any spatiotemporal overlap between the interrupted vehicle trajectories; 4) For data without spatiotemporal overlap, the GAF-GNN-CNN-LSTM algorithm is used to predict vehicle trajectories in overlapping areas; 5) The tsfresh algorithm is used to reduce the dimensionality of all vehicle trajectories, reducing the time-series vehicle trajectories to a one-dimensional array N; 6) Normalize all one-dimensional arrays N to obtain a new one-dimensional array N. 1 ; 7) Calculate the FastDTW distance of all vehicle trajectories in the overlapping area; 8) Calculate the trajectory matching index based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity; 9) Match and splice the two trajectories with the highest matching index; In step 4), the GAF-GNN-CNN-LSTM algorithm is used to predict the vehicle trajectory in the overlapping area. The specific steps are as follows: a) Extract vehicle trajectory speed time series data, horizontal axis time series data and vertical axis time series data respectively, and use Gram angle field to convert the time series trajectory data into two dimensions to construct a Gram angle field two-dimensional matrix M; b) Based on the two-dimensional matrix M, construct a graph neural network to generate a graph G; c) The obtained graph neural network is divided into communities using the module partitioning algorithm in the TSIA graph neural network analysis tool to obtain different groups of neurons and vehicle speed mapping relationships; d) Map the labels of different neuron groups to their corresponding sub-time series; e) Input the neuron group labels and corresponding sub-time series as inputs into the CNN-LSTM model for time series prediction; f) Obtain the vehicle time-series trajectory in the overlapping area, including vehicle trajectory speed time-series data, horizontal axis time-series data, and vertical axis time-series data.

2. The method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity as described in claim 1, characterized in that, In step 5), the tsfresh algorithm is used to reduce the dimensionality of all vehicle trajectories, transforming the time-series vehicle trajectories into a one-dimensional array N. The specific steps are as follows: a) Extract trajectory data from a 150m section of road at a distance of 200m to 50m from the coordinate point of the millimeter-wave radar as a time-series trajectory dataset, where the trajectory time-series feature parameters include time-series data of lateral and longitudinal velocities. b) Based on the above time-series velocity data, the time-series data of lateral and longitudinal velocities are differentiated with respect to time to obtain the time-series data of lateral and longitudinal acceleration. c) Based on the above time-series acceleration data, the time-series data of lateral and longitudinal accelerations are differentiated with respect to time to obtain the time-series data of lateral and longitudinal accelerations; d) Extract vehicle time-series data, including: lateral and longitudinal positions, lateral and longitudinal velocities, lateral and longitudinal accelerations, and lateral and longitudinal jerk data. Use tsfresh as the time-series trajectory feature extraction tool to extract vehicle time-series trajectory feature values. e) Using the constructed large dataset to select relevant features, tsfresh will perform hypothesis testing on each feature to check whether it is relevant to the given target. Then, the extract_relevant_features function in tsfresh will be used to perform extraction, selection and filtering simultaneously, and finally leave enough relevant time-series trajectory features to reduce the time-series vehicle trajectory to a one-dimensional array N.

3. The method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity according to claim 2, characterized in that, In step 7), the vehicle trajectories in the overlapping areas are interpolated at a distance of 1m from the vehicle trajectories in the overlapping areas from 200m to 250m from the millimeter-wave radar to obtain rasterized vehicle trajectory points. The FastDTW distance D is then calculated by pairwise matching of the rasterized vehicle trajectory points.

4. The method for stitching vehicle trajectories across the entire range of millimeter-wave radar based on trajectory similarity according to claim 3, characterized in that, In step 8), the trajectory matching index is calculated based on vehicle type similarity, distance similarity, speed similarity, and behavior similarity. The specific steps are as follows: a) Based on the obtained radar cross-section, calculate the vehicle type similarity X between the two vehicle trajectories, as follows: ； b) Based on the calculated FastDTW distance D, calculate the FastDTW distance D between each pair of trajectories in the overlapping region. i Take the maximum FastDTW distance D in the overlapping region. max The distance similarity (DL) is calculated as follows: ； c) Based on the predicted time-series velocity data of the overlapping area, calculate the velocity value S at a distance of 250m from the millimeter-wave radar. i With downstream timing velocity initial value S j The velocity similarity SL is calculated as follows: ； d) Based on the calculated one-dimensional array N of time-series vehicle trajectories, calculate the FastDTW distance E between each pair of all trajectory one-dimensional arrays. i Take the maximum distance E from all one-dimensional arrays FastDTW max The behavioral similarity score XL is calculated as follows: ； e) Calculate the trajectory matching index PL, as follows: 。

Images