A global vehicle trajectory construction method under incomplete perception data

Abstract

The application relates to a global vehicle trajectory construction method under incomplete perception data, which comprises the following steps: a vehicle cross-domain trajectory extraction model is constructed, vehicle running traffic characteristics are combined, and vehicle trajectories under multiple cameras are tracked and extracted; preliminary camera space coordinate conversion is carried out by using camera calibration, and vehicle trajectory data obtained by a radar is smoothed; space-time features are extracted from vehicle trajectory data, a space-time synchronization optimization model is constructed according to the space-time features, multi-sensor space-time coordinates are unified, and camera and radar data are fused; a trajectory completion evaluation index is established, space-time interaction features of vehicle trajectories are fused, a vehicle trajectory completion model is constructed, and missing trajectories are completed and repaired. Compared with the prior art, the application fuses multi-sensor data to complement each other, considers space-time features of vehicle trajectories, and the proposed trajectory completion method can effectively improve the completeness and reliability of vehicle trajectories.

Description

Technical Field

[0001] This invention relates to the field of traffic information analysis and processing technology, and in particular to a method for constructing full-domain vehicle trajectories under incomplete perception data. Background Technology

[0002] Smart highways have become an inevitable trend in the technological evolution of next-generation highway systems. The numerous roadside intelligent sensors deployed on smart highways, used to acquire vehicle trajectory data, form a crucial foundation for intelligent traffic flow management systems and vehicle-road cooperative autonomous driving. Acquiring full-domain trajectories is of great significance for achieving vehicle-road cooperation, as it can broaden the capabilities of individual vehicles, improve traffic efficiency, and ensure traffic safety. Roadside perception units, thanks to their unique deployment locations and edge computing capabilities, typically provide vehicles with a perception range beyond line of sight; however, due to variations in their perception accuracy and reference coordinates, and susceptibility to interference from the traffic environment, the acquired vehicle trajectories may contain gaps. With the development and commercial application of intelligent traffic flow management systems and autonomous vehicles, the completion and reconstruction of missing trajectories will become increasingly important.

[0003] Commonly used sensors in roadside perception systems include high-definition cameras, millimeter-wave radar, and lidar. These sensors are used to extract vehicle trajectories, and the completeness and accuracy of trajectory extraction are affected by sensor accuracy, sensor deployment location, and real-time traffic conditions. For these sensors, the inconsistency between spatiotemporal coordinate systems and the inevitable mutual occlusion of vehicles during operation significantly impact the effectiveness of vehicle identification, tracking, and trajectory extraction. For vehicle-road cooperative systems, complete global vehicle trajectories are crucial for the safe and efficient operation of vehicles. Therefore, completing missing vehicle trajectories is essential, and efficient trajectory completion and reconstruction schemes can effectively compensate for the deficiencies of roadside sensors.

[0004] Previous research on methods for constructing complete vehicle trajectories across highways typically relies on single-type sensors such as cameras or millimeter-wave radar for data acquisition. However, each single sensor suffers from accuracy issues. For example, millimeter-wave radar often experiences anchor point drift, causing positional jitter in the perpendicular lane direction, while cameras have lower accuracy and are greatly affected by the environment. Furthermore, when performing data completion, most models treat vehicle trajectories as time series, ignoring spatial interactions between vehicles. Therefore, based on these two points, current research methods struggle to obtain highly reliable complete vehicle trajectories. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for constructing a full-domain vehicle trajectory under incomplete sensing data. By fusing sensing data from multiple sensors and considering the temporal characteristics of vehicles and the spatial characteristics between vehicles, missing vehicle trajectories can be completed, which can effectively improve the integrity and reliability of the full-domain vehicle trajectory.

[0006] The objective of this invention can be achieved through the following technical solution: a method for constructing a full-domain vehicle trajectory based on incomplete perception data, comprising the following steps:

[0007] S1. Construct a vehicle cross-domain trajectory extraction model, and combine vehicle operation traffic characteristics to track and extract vehicle trajectories under multiple cameras;

[0008] S2. Perform preliminary camera spatial coordinate transformation using camera calibration, and smooth the vehicle trajectory data acquired by radar.

[0009] S3. Extract spatiotemporal features from vehicle trajectory data, construct a spatiotemporal synchronization optimization model based on this, unify the spatiotemporal coordinates of multiple sensors, and fuse camera and radar data.

[0010] S4. Establish trajectory completion evaluation indicators, integrate the spatiotemporal interaction characteristics of vehicle trajectories, construct a vehicle trajectory completion model, and complete and repair missing trajectories.

[0011] Furthermore, step S1 specifically includes the following steps:

[0012] S11. Construct a vehicle tracking model using a single camera, extract the vehicle trajectory, use the YOLOv5 detection model to obtain the vehicle bounding box information (x, y, w, h) in the current video frame, and use Kalman filtering to predict the vehicle position in the next frame. The vehicle position coordinates are calculated as follows:

[0013]

[0014] y′=y

[0015] Where (x′, y′) are the vehicle position coordinates of the current video frame, (x, y) are the coordinates of the lower left corner of the vehicle frame, and (w, h) are the width and height of the vehicle frame;

[0016] Next, the predicted vehicle information is correlated with the detected vehicle information. A cost matrix is ​​constructed using the Mahalanobis distance between the vehicle appearance information and the vehicle position. The Hungarian algorithm is then used to obtain the vehicle tracking scheme. The calculation process is as follows:

[0017] d (1) (i, j) = (d j -y i )T S -1 (d j -y i )

[0018]

[0019] c i，j =λd (1) (i,j)+(1-λ)d (2) (i, j)

[0020] Where i is the predicted vehicle detection box number, j is the current detection box number, d and y are the detected vehicle distribution and predicted vehicle distribution, respectively, S is the covariance matrix between the two distributions, λ is the weighting factor, and d (1) (i, j) is the Mahalanobis distance of the vehicle position, d (2) (i, j) is the minimum cosine distance of the detection box, c i，j This is the overall cost matrix;

[0021] The video roadside perception data is then input into the vehicle tracking model under a single camera to obtain the vehicle's trajectory.

[0022] S12. Construct a cross-camera feature extraction model to obtain vehicle shape and static traffic operation features. First, mark the lane area range in the video data, and obtain the vehicle running lane information based on vehicle position information and boundary judgment. Then, stitch together the vehicle shape features extracted from the video detection box to form vehicle features.

[0023] S13. Construct a vehicle cross-domain trajectory extraction model, match vehicles, calculate the cosine similarity of the features of two vehicles based on the vehicle feature information extracted by the cross-camera feature extraction model, perform feature matching on the data, and complete the trajectory tracking of cross-domain vehicles.

[0024] Furthermore, step S2 specifically includes the following steps:

[0025] S21. Based on the camera calibration method, perform spatial coordinate transformation on the trajectory data acquired by the camera. The trajectory pixel coordinates under the camera are represented as (u, v, 1). T The initial world coordinates are (U, V, 1). T Z is the scaling factor, and H is the homology matrix. The calculation formula is as follows:

[0026]

[0027] S22. Perform data splitting on the converted camera data and polynomial smoothing on the radar sensing data.

[0028] Furthermore, step S3 specifically includes the following steps:

[0029] S31. Extract the spatial and temporal features corresponding to cameras and radar from the vehicle trajectory data under multiple sensors, and perform vehicle matching;

[0030] S32. Construct a multi-sensor spatiotemporal synchronization optimization model to unify the spatiotemporal coordinates of cameras and radars;

[0031] S33. Weighted fusion of multi-sensor data after unifying spatiotemporal coordinates is performed to obtain fused vehicle trajectory data.

[0032] Furthermore, the specific process of step S31 is as follows:

[0033] Considering the temporal and spatial differences between multiple sensors, a virtual detection line is determined at the same value on the y-axis in each of their respective world coordinate systems, and the following equation is constructed:

[0034]

[0035] ΔY=Y radar_act -Y video_act

[0036] Where, Δt q The time interval between the vehicle passing through the two detection lines is given by ΔT, which represents the time coordinate difference between the two sensors. Let ΔY be the average speed of the vehicle, and ΔY be the Y-coordinate of the detection lines of the two sensors. radar_act Y video_act Differences, e q It represents an error term that follows a normal distribution with a mean of 0;

[0037] Based on the timestamp t of vehicle q passing the inspection line q Construct the driving information for each vehicle and represent it as a one-dimensional vector [t] of length N+1. q-N -t q , ..., t q-1 -t q , t q , t q+1 -t q , ..., t q+N -t q ];

[0038] Construct the distance matrix accordingly. The Kuhn-Munkres algorithm was used for vehicle matching.

[0039] Furthermore, the specific process of step S32 is as follows:

[0040] The objective function for constructing the spatiotemporal synchronization optimization model is:

[0041] The objective function for constructing the spatiotemporal synchronization optimization model is:

[0042]

[0043] Determine the equality constraints of the optimization problem:

[0044]

[0045] By adjusting the difference between the detection lines of the two sensors, i.e., ΔY, the objective function is minimized, thereby obtaining the spatial and temporal difference between the two sensors.

[0046] Furthermore, the specific process of step S33 is as follows: based on the vehicle trajectory data under multiple sensors after vehicle and spatiotemporal matching, determine the corresponding weighting coefficient according to the perception accuracy of each sensor, and perform weighted fusion of the vehicle trajectory data of each sensor.

[0047] Furthermore, step S4 specifically includes the following steps:

[0048] S41. Based on the set vehicle trajectory completion target, identify vehicles with missing trajectories, extract data from the weighted fused vehicle trajectory data, and divide it into training set, validation set and test set according to the proportion.

[0049] S42. Use the accuracy of the completed trajectory as the model optimization objective and evaluation index;

[0050] S43. Construct a target vehicle trajectory completion model, and train and test the model using the training set, validation set, and test set;

[0051] S44. Input the actual weighted and fused vehicle trajectory data into the trained vehicle trajectory completion model, and output the completed vehicle trajectory data.

[0052] Further, the specific process of step S42 is as follows: The model completion effect is defined using the average displacement error. Within all considered locations, the average Euclidean distance (L2) difference between each completed location and each true location is calculated, which is expressed as:

[0053]

[0054] in, This represents the position of vehicle q at time t, given the trajectories at both ends. Below, the model completion output The missing segment of the trajectory is represented by real, known data, denoted as [missing data]. Used to evaluate the accuracy of trajectory completion.

[0055] Furthermore, the specific process of step S43 is as follows: using a dual-stream mechanism, the two missing trajectory segments before and after are input into two processing streams respectively. They are encoded separately in the encoder, merged and input together into the decoding layer. Finally, the output of the decoder is the complete trajectory.

[0056] The encoder mainly consists of two LSTM modules and a graph attention module. The LSTM modules are used to capture the historical motion state of a single target, extract the temporal features of the trajectory sequence, and focus on the motion pattern of each car. The encoding result of the LSTM is denoted as:

[0057]

[0058] In the formula, To The embedding encoding result, W m These are the weights of the LSTM;

[0059] After bidirectional LSTM encoding, the graph attention module continues to focus on the interactions between sequences, treating all vehicles as nodes in the graph and assigning different importance to different nodes, thereby aggregating neighbor information. Therefore, the graph attention module receives the initial encoding result from the bidirectional LSTM input. The output is:

[0060]

[0061] In the formula, Let be the graph attention coefficients for node pair (q, p) at time t, and W be the dimension mapping matrix. Let σ represent the set of neighboring nodes of node q, and let σ be the LeaklyReLU activation function.

[0062] The graph attention module output is followed by a bidirectional LSTM layer to model the temporal correlation of trajectory interactions. The result is then used... This indicates that, accordingly, among them To The embedding encoding result, W′ m These are the weights of the bidirectional LSTM;

[0063] After the above steps are completed, node q has the following characteristics at time point t: The former is the result of the first LSTM encoding, and the latter is the result of the former after further graph neural network and LSTM encoding. The two are concatenated in the last layer and then fed into the decoder.

[0064] The decoder uses bidirectional LSTM for the corresponding decoding, i.e.

[0065]

[0066] In the formula, To The embedding encoding result, W″ m This represents the LSTM weights;

[0067] When the decoding results at both ends overlap, it indicates that the trajectory completion of the middle segment has been completed.

[0068] Compared with existing technologies, this invention takes into account the advantages and disadvantages of the sensing effects of various roadside sensors. Therefore, it matches and fuses the sensing data of various sensors to obtain a more complete vehicle trajectory across the entire area. Then, it analyzes the real-time operation characteristics of traffic. By establishing a vehicle trajectory completion model that considers the spatiotemporal characteristics between vehicles, it can complete the vehicle trajectory and effectively improve the integrity and reliability of the vehicle trajectory across the entire area.

[0069] This invention combines existing trajectory data from various types of sensors (cameras and radar) to establish a multi-sensor vehicle matching and spatiotemporal synchronization optimization model. Based on this model, a vehicle trajectory reconstruction method is established that aims to minimize the average displacement error and integrates the temporal and spatial characteristics of vehicle trajectory. This method can meet the needs of completing and reconstructing vehicle trajectory perception data in different traffic operation scenarios. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0071] Figure 2 A schematic diagram showing the locations of roadside sensors;

[0072] Figure 3 A schematic diagram of a vehicle cross-domain trajectory extraction model considering static traffic operation scenarios;

[0073] Figure 4 This is a schematic diagram of data space coordinate transformation based on camera calibration;

[0074] Figure 5 A schematic diagram of the spatiotemporal matching optimization model;

[0075] Figure 6 Complete the diagram of the vehicle trajectory;

[0076] Figure 7 This is a road scene map in the embodiment;

[0077] Figure 8 The image shows the vehicle trajectory completion result in the example. Detailed Implementation

[0078] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0079] Example

[0080] like Figure 1 As shown, a method for constructing the entire vehicle trajectory under incomplete perception data includes the following steps:

[0081] S1. Construct a vehicle cross-domain trajectory extraction model, and combine vehicle operation traffic characteristics to track and extract vehicle trajectories under multiple cameras;

[0082] S2. Perform preliminary camera spatial coordinate transformation using camera calibration, and smooth the vehicle trajectory data acquired by radar.

[0083] S3. Extract spatiotemporal features from vehicle trajectory data, construct a spatiotemporal synchronization optimization model based on this, unify the spatiotemporal coordinates of multiple sensors, and fuse camera and radar data.

[0084] S4. Establish trajectory completion evaluation indicators, integrate the spatiotemporal interaction characteristics of vehicle trajectories, construct a vehicle trajectory completion model, and complete and repair missing trajectories.

[0085] Among them, the vehicle cross-domain trajectory extraction model can identify and track the trajectory of the same vehicle under different sensors; the roadside perception unit (i.e., multiple sensors, including cameras and radar) data includes but is not limited to sampling time, target positioning coordinates, target speed, etc.; the spatiotemporal synchronization optimization model can effectively achieve spatiotemporal coordinate matching of multiple sensors with low computational cost; and the vehicle trajectory completion model can complete the missing vehicle trajectory.

[0086] This embodiment uses a section of the Donghai Bridge as a case study scenario. The roadside sensor deployment locations in this embodiment are as follows: Figure 2 As shown, the road scene is as follows Figure 7 As shown, the vehicle trajectory is reconstructed by fusing camera-sensing trajectory data and radar-sensing trajectory data, and integrating the temporal and spatial interaction features of the trajectory.

[0087] This embodiment applies the above technical solution, and its main process is as follows:

[0088] Step 1: Construct a vehicle cross-domain trajectory extraction model, and combine vehicle operation traffic characteristics to track and extract vehicle trajectories under multiple cameras;

[0089] Step 1.1: Construct a vehicle tracking model using a single camera and extract the vehicle trajectory. Utilize the YOLOv5 detection model to obtain the vehicle bounding box information (x, y, w, h) in the current video frame. Then, use Kalman filtering to predict the vehicle position in the next frame. The vehicle position coordinates are calculated as follows:

[0090]

[0091] y′=y

[0092] Where (x, y) are the coordinates of the lower left corner of the vehicle frame, and (w, h) are the width and height of the vehicle frame.

[0093] Then, the predicted vehicle information is correlated with the detected vehicle information. A cost matrix is ​​constructed using the Mahalanobis distance between the vehicle appearance information and the vehicle position. Finally, the Hungarian algorithm is used to obtain the vehicle tracking scheme. The calculation process is as follows:

[0094] d (1) (i, j) = (d j -y i ) T S -1 (d j -y i )

[0095]

[0096] c i，j =λd (1) (i,j)+(1-λ)d (2) (i, j)

[0097] Where i represents the predicted vehicle detection box number, j represents the current detection box number, d and y represent the detected vehicle distribution and the predicted vehicle distribution, S represents the covariance matrix between the two distributions, and λ is a weighting factor. (1) (i, j) represents the Mahalanobis distance of the vehicle's position, d (2) (i, j) represents the minimum cosine distance of the detection box, c i，j This represents the overall cost matrix.

[0098] The video roadside perception data is then input into the model to obtain the vehicle's trajectory.

[0099] Step 1.2: Construct a cross-camera feature extraction model to obtain vehicle shape and static traffic operation features. First, the lane area range in the video data is defined, and the vehicle's operating lane information is obtained using boundary determination based on the vehicle's position information. Then, the vehicle shape features extracted from the video detection boxes are stitched together to form the vehicle features.

[0100] Step 1.3: Construct a vehicle cross-domain trajectory extraction model (e.g., Figure 3 As shown, vehicle matching is performed. Based on the vehicle feature information extracted by the two camera sensors, the cosine similarity of the features of the two vehicles is calculated to perform feature matching and complete the trajectory tracking of cross-domain vehicles.

[0101] Step 2: Perform preliminary camera spatial coordinate transformation using camera calibration (e.g., ...) Figure 4 (as shown), and smooth the vehicle trajectory data acquired by the radar;

[0102] Step 2.1: Perform spatial coordinate transformation on the trajectory data acquired by the camera based on the camera calibration method. The trajectory pixel coordinates under the camera are represented as (u, v, 1). T The initial world coordinates are (U, V, 1). T Z is the scaling factor, and H is the homology matrix. The calculation process is as follows:

[0103]

[0104] Step 2.2: Perform data splitting on the converted camera data and polynomial smoothing on the radar sensing data.

[0105] Step 3: Extract spatiotemporal features from trajectory data, and construct a spatiotemporal synchronization optimization model based on these features (e.g., ...). Figure 5 As shown), this is used to unify the spatiotemporal coordinates of multiple sensors and perform vehicle matching;

[0106] Step 3.1: Considering the temporal and spatial differences between multiple sensors, determine a virtual detection line at the same value on the y-axis in their respective world coordinate systems, and construct the following equation:

[0107]

[0108] ΔY=Y radar_act -Y video_act

[0109] Where, Δt q The time interval between the vehicle passing through the two detection lines is given by ΔT, which represents the time coordinate difference between the two sensors. Let ΔY be the average speed of the vehicle, and ΔY be the Y-coordinate of the detection lines of the two sensors. radar_act Y video_act Differences, e q It represents an error term that follows a normal distribution with a mean of 0.

[0110] Based on the timestamp t of vehicle q passing the inspection line q Construct the driving information for each vehicle and represent it as a one-dimensional vector [t] of length N+1. q-N -t q , ..., t q-1 -t q , t q , t q+1 -t q , ..., t q+N -t q ];

[0111] Construct the distance matrix accordingly. The Kuhn-Munkres algorithm was used for vehicle matching.

[0112] Step 3.2: Based on the vehicle running time characteristics within the sensors, construct the objective function of the spatiotemporal synchronization optimization model:

[0113]

[0114] The equality constraints of the optimization problem are shown below:

[0115]

[0116] By adjusting the difference between the detection lines of the two sensors, i.e., ΔY, the objective function is minimized, thereby obtaining the spatial and temporal difference between the two sensors.

[0117] Step 3.3: Analyze the vehicle trajectory data under multiple sensors after spatiotemporal matching. Determine the weighting parameters based on the perception accuracy of each sensor and perform weighted fusion of the vehicle trajectory.

[0118] Step 4: Using the fused vehicle trajectory data as a foundation, and with the minimum average displacement error as the objective function, fuse vehicle temporal features and spatial interaction features to complete the missing vehicle trajectories (e.g., ...). Figure 6 (As shown).

[0119] Step 4.1: Based on the requirements of the vehicle trajectory completion target, identify the vehicles with missing trajectories and divide the data into training set, validation set, and test set proportions.

[0120] Step 4.2: Define the model completion effect using the average displacement error: Calculate the average Euclidean distance (L2) difference between each completed location and each true location across all considered locations, expressed as:

[0121]

[0122] in, This represents the position of vehicle q at time t, given the trajectories at both ends. Below, the model completion output The missing segment of the trajectory is represented by real, known data, denoted as [missing data]. Used to evaluate the accuracy of trajectory completion.

[0123] Step 4.3: Using a dual-stream mechanism, the two missing trajectory segments are input into two separate processing streams. They are encoded separately in the encoder, merged, and then input together into the decoding layer. The final output of the decoder is the complete trajectory.

[0124] The encoder mainly consists of two LSTM modules and a graph attention module. The LSTM is used to capture the historical motion state of a single target, extracting temporal features of the trajectory sequence, and focusing on the motion pattern of each car itself. The encoding result of the LSTM can then be denoted as:

[0125]

[0126] In the formula, To The embedding encoding result, where Wm is the weight of the LSTM.

[0127] After bidirectional LSTM encoding, the graph attention module continues to focus on the interactions between sequences, treating all vehicles as nodes in the graph and assigning different importance to different nodes, thereby aggregating neighbor information. Therefore, the graph attention module receives the initial encoding result from the bidirectional LSTM input. The output is:

[0128]

[0129] In the formula, Let be the graph attention coefficients for node pair (q, p) at time t, and W be the dimension mapping matrix. Let σ represent the set of neighboring nodes of node q, and let σ be the LeaklyReLU activation function.

[0130] The graph attention module output is followed by a bidirectional LSTM layer to model the temporal correlation of trajectory interactions. The result is then used... This indicates that, accordingly, among them To The embedding encoding result, W′ m These are the weights of the bidirectional LSTM;

[0131] After the above steps are completed, node q has the following characteristics at time point t: The former is the result of the first LSTM encoding, and the latter is the result of the former after further graph neural network and LSTM encoding. The two are concatenated in the last layer and then fed into the decoder.

[0132] The decoder uses bidirectional LSTM for the corresponding decoding, i.e.

[0133]

[0134] In the formula, To The embedding encoding result, W″ m This represents the weights of the LSTM.

[0135] When the decoding results at both ends are found to overlap (with a difference of less than 0.01m), it is determined that the trajectory completion of the middle segment has been completed.

[0136] Step 4.4: Input the actual fused trajectory data into the vehicle trajectory completion model, and the model outputs the completed vehicle trajectory.

[0137] This embodiment sets an average displacement error index to measure the performance of the trajectory completion model. A higher value indicates a better solution. The method of this invention is compared with traditional methods, and the trajectory completion results are compared... Figure 8 As shown, the completion scheme proposed in this invention has significant advantages.

[0138] In summary, this invention addresses the perception trajectory data of roadside sensing units by proposing a method for constructing full-domain vehicle trajectories under incomplete sensing data. It involves building a cross-domain vehicle trajectory extraction model, combining vehicle operation traffic characteristics, to track and extract vehicle trajectories from multiple cameras; using camera calibration for preliminary camera spatial coordinate transformation, smoothing the vehicle trajectory data acquired by radar, and extracting spatiotemporal features from the multi-sensor trajectory data to construct a spatiotemporal synchronization optimization model for unifying and fusing the spatiotemporal coordinates of multiple sensors; using the fused vehicle trajectory data as a foundation, and with the minimum average displacement error as the objective function, fusing vehicle temporal features and spatial interaction features to complete missing vehicle trajectories, effectively improving the completeness and reliability of the full-domain perception trajectory.

Claims

1. A global vehicle trajectory construction method under incomplete perception data, characterized in that, Comprise the following steps: S1, construct a vehicle cross-domain trajectory extraction model, combine vehicle running traffic characteristics, track and extract vehicle trajectory under multiple cameras; S2, use camera calibration for preliminary camera space coordinate conversion, and smooth the vehicle trajectory data obtained by radar; S3, extract the space-time features from the vehicle trajectory data, and construct a space-time synchronization optimization model accordingly, unify the multi-sensor space-time coordinates, and fuse the camera and radar data; S4, establish a trajectory completion evaluation index, fuse the space-time interaction features of the vehicle trajectory, construct a vehicle trajectory completion model, and complete and repair the missing trajectory; Step S1 specifically includes the following steps: S11, construct a single camera vehicle tracking model, extract the vehicle trajectory, use the yolov5 detection model to obtain the vehicle frame information under the current video frame And use kalman filter to predict the next frame vehicle position, the vehicle position coordinates are calculated as follows: , , wherein, is the vehicle position coordinate of the current video frame, is the left bottom corner coordinate of the vehicle frame, is the width and height of the vehicle frame; Then the predicted part of the vehicle information is associated with the detected vehicle information, a cost matrix is constructed by the Mahalanobis distance of the vehicle appearance information distance and the vehicle position, and the Hungarian algorithm is used to obtain the vehicle tracking scheme, and the calculation process is as follows: , , , wherein, is a predicted vehicle bounding box sequence number, is a current bounding box sequence number, D, y are a detected vehicle distribution and a predicted vehicle distribution, respectively, S is a covariance matrix between the two distributions, is a weighting factor, is a Mahalanobis distance of a vehicle position, is a minimum cosine distance of a bounding box, is a total cost matrix; Then the video roadside perception data is input into a single camera vehicle tracking model to obtain the vehicle motion trajectory; S12, construct a cross-camera feature extraction model to obtain the vehicle shape and static traffic running characteristics, first calibrate the lane area range in the video data, and then use the boundary judgment to obtain the vehicle running lane information according to the vehicle position information, and then splice the vehicle shape features extracted by the video detection frame to form the vehicle features; S13, construct a vehicle cross-domain trajectory extraction model to match the vehicles, calculate the cosine similarity of the two vehicle features according to the vehicle feature information extracted by the cross-camera feature extraction model, and complete the feature matching of the data to complete the trajectory tracking of the cross-domain vehicles. 2.The global vehicle trajectory construction method under incomplete perception data according to claim 1, wherein, The step S2 specifically includes the following steps: S21, based on the camera calibration method, the trajectory data obtained by the camera is converted into spatial coordinates, and the trajectory pixel coordinates under the camera are represented as , the initial tuning world coordinates are , is a proportional factor, is a homogeneity matrix, and the calculation formula is as follows: , S22, split the converted camera data, and perform polynomial smoothing on the radar perception data. 3.The global vehicle trajectory construction method under incomplete perception data according to claim 1, wherein, The step S3 specifically includes the following steps: S31, extract the space and time features corresponding to the camera and radar from the vehicle trajectory data under multiple sensors, and perform vehicle matching; S32, construct a multi-sensor space-time synchronization optimization model to unify the space-time coordinates of the camera and radar; S33, weight and fuse the multi-sensor data after the space-time coordinates are unified to obtain the fused vehicle trajectory data. 4.The global vehicle trajectory construction method under incomplete perception data according to claim 3, wherein, The specific process of step S31 is: Considering the time and space differences between multiple sensors, a virtual detection line is determined on the same value of the y-axis in their respective world coordinate systems, and the following equation is constructed: , , wherein, is the time interval between the two detection lines for the vehicle, is the difference in time coordinates of the two sensors, is the average speed of the vehicle, is the difference in detection line coordinates of the two sensors , , represents an error term that is subject to a normal distribution with mean 0. According to the vehicle q By detecting the timestamp of the line , the driving information of each vehicle is constructed and represented as a one-dimensional vector of length N+1 ; A distance matrix is constructed accordingly and vehicle matching is performed using the Kuhn-Munkres algorithm. 5.The global vehicle trajectory construction method under incomplete perception data according to claim 4, wherein, The specific process of step S32 is: Construct the objective function of the space-time synchronization optimization model: Construct the objective function of the space-time synchronization optimization model: , Determine the equality constraints of the optimization problem: , By adjusting the difference between the two sensor detection lines, that is making the objective function minimum, thus obtaining the space and time difference between the two sensors. 6.The global vehicle trajectory construction method under incomplete perception data according to claim 5, wherein, The specific process of step S33 is: based on the vehicle, space-time matched vehicle trajectory data under multiple sensors, according to the perception accuracy of each sensor, determine the corresponding weighting coefficient, and weight and fuse the vehicle trajectory data of each sensor.

7. The global vehicle trajectory construction method under imperfect perception data according to claim 1, wherein, The step S4 specifically includes the following steps: S41, according to the set vehicle trajectory completion target, determine the missing trajectory vehicle, extract data from the weighted fused vehicle trajectory data, and divide it into training set, validation set and test set according to the proportion; S42, taking the trajectory accuracy as the model optimization target and evaluation index; S43, constructing a target vehicle trajectory completion model, and training and testing the model by using the training set, the verification set and the test set; S44, inputting the actual weighted fused vehicle trajectory data into the trained vehicle trajectory completion model, and outputting the completed vehicle trajectory data. 8.The global vehicle trajectory construction method under incomplete perception data according to claim 7, wherein, The specific process of step S42 is to define the model completion effect by using the average displacement error, and to calculate the average Euclidean distance difference between each completed position and each true value position, i.e. the L2 average distance between the completed value and the true value, in all considered positions, which is expressed as: , wherein, representing a vehicle q At time position, given two end trajectories , the model completion output wherein the intermediate missing segment trajectory is a real known data, denoted as , for evaluating the completion trajectory accuracy. 9.The global vehicle trajectory construction method under incomplete perception data according to claim 8, wherein, The specific process of step S43 is to use a double-flow mechanism to input the two segments of trajectories before and after the missing into two processing flows, respectively, to encode them in the encoder, to input them together into the decoding layer after merging, and to output the complete trajectory from the final decoder. Wherein, the main components of the encoder are two LSTM modules and a graph attention module, the LSTM module is used to capture the historical motion state of a single target, to extract the time sequence features of each trajectory sequence, and to focus on the motion pattern of each car itself, i.e. the encoding result of the LSTM is recorded as: , In the formula, is the embedding encoding result of is the weight of the LSTM.​ After the bidirectional LSTM encoding, the graph attention module continues to pay attention to the interaction between sequences, takes all vehicles as nodes on the graph, assigns different importance to different nodes, aggregates neighbor information, and therefore, the graph attention module receives the preliminary encoding results of the input of the bidirectional LSTM , and the output is: , wherein is a pair of nodes at time , the graph attention coefficient, is a dimension mapping matrix, denotes a set of neighbor nodes of node , and is a LeaklyReLU activation function. After the graph attention module outputs, a layer of bidirectional LSTM is used to model the time correlation of trajectory interaction, and the result is represented as respectively, wherein is the embedding encoding result of , and is the weight of the bidirectional LSTM. After the above steps are completed, the node At time point t, the characteristics are: , The former is the first LSTM encoding result, and the latter is the result after the former is encoded by a graph neural network and an LSTM. The two are spliced at the last layer and sent to the decoder. The decoder uses bidirectional LSTM for corresponding decoding, i.e. , In the formula, For the embedding encoding result of , represents the LSTM weight; When it is judged that the decoding results at both ends coincide, it indicates that the trajectory completion of the middle segment has been completed.

Images