A city road network vehicle trajectory identification and completion method based on multi-modal data

Abstract

The application discloses a kind of based on multimodal data fusion's urban road network vehicle track identification completion method, the method includes: obtaining urban road network topological information, automatic vehicle identification record and monitoring vehicle record, and candidate track node and vehicle image set are generated in combination with topological relationship, time and distance constraint;Build multimodal track identification model including visual feature coding module, space-time state coding module and feature fusion module, vehicle visual similarity is extracted respectively, current track space-time hidden state is generated, and the output candidate track matching score is fused;According to candidate track matching score, vehicle track is gradually expanded until meeting track termination condition, realize vehicle missing track completion and generate complete driving track.The application can effectively fuse vehicle visual feature and space-time state information, reduce the mis-matching problem caused by similar vehicles in complex urban road network environment, improve the accuracy and robustness of vehicle track identification.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation systems and computer vision, and in particular to a method for vehicle trajectory recognition and completion in urban road networks based on multimodal data. Background Technology

[0002] Vehicle trajectory data is fundamental to core applications in intelligent transportation systems, such as traffic monitoring, demand estimation, and signal control. In recent years, the dense deployment of surveillance cameras at urban intersections has accumulated massive amounts of visual data, providing a potential source for obtaining vehicle trajectories from a full sample of the urban road network. Existing cross-camera vehicle trajectory extraction methods primarily rely on vehicle appearance similarity to achieve identification and cross-intersection tracking, thereby extracting continuous trajectories. However, these vision-matching-based methods have inherent limitations: on the one hand, they are easily affected by environmental factors such as changes in lighting and differences in shooting angles; on the other hand, the large number of similar vehicles of the same model, color, and shape in urban road networks further reduces vehicle tracking accuracy, making single-vision tracking in complex urban environments extremely challenging.

[0003] To reduce the uncertainty of visual data, existing methods have attempted to incorporate identity data such as GPS and Automatic Vehicle Identification (AVI). This type of data contains precise spatiotemporal records of vehicle movement and clear identity information, which can compensate for the inherent shortcomings of visual data—being "comprehensive but uncertain"—and improve the reliability of vehicle tracking. However, existing identity data-based methods have significant shortcomings: on the one hand, they rely too heavily on identity data, but are constrained by privacy protection requirements, sparse sensor deployment, and recognition errors, and can only cover a small portion of vehicle samples; on the other hand, for achieving high-quality, long-distance, complete trajectory recognition of all vehicles, existing methods still have room for improvement in the joint modeling of visual features and spatiotemporal constraints.

[0004] To overcome these problems, existing technologies attempt to introduce spatiotemporal information to improve the accuracy of cross-camera vehicle tracking, narrowing down the candidate trajectory range through constraints such as vehicle speed, time intervals, or spatial distances. However, these rule-based cross-camera visual learning models still have shortcomings in jointly modeling the relationship between visual features and spatiotemporal constraints, and their ability to capture continuous vehicle trajectories in complex urban road networks is limited. Ultimately, this restricts the robustness and accuracy of trajectory recognition, making it difficult to adapt to the complex needs of real-world urban traffic scenarios. Currently, there is still a lack of a cross-camera vehicle trajectory recognition completion method that can integrate visual features and spatiotemporal constraint information, and achieve a unified framework for synergistic effects, to meet the high-quality requirements of intelligent transportation systems for vehicle trajectory data and achieve accurate recognition of complete and continuous vehicle trajectories in urban road networks. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a method for vehicle trajectory recognition and completion in urban road networks that can effectively utilize multimodal data and is based on joint modeling of visual features and spatiotemporal constraints.

[0006] Technical solution:

[0007] To achieve the above objectives, this invention provides a method for vehicle trajectory recognition and completion in urban road networks based on multimodal data fusion, comprising the following steps:

[0008] S1. Collect urban area road network topology information, AVI (Automatic Vehicle Identification) vehicle passage records, and monitored vehicle records. Associate the AVI vehicle passage records based on vehicle identification information and sort the road network nodes passed by the same vehicle in chronological order to generate the initial vehicle trajectory for that vehicle. The AVI vehicle passage records include vehicle license plate number, passed nodes, and timestamp. The monitored vehicle records include vehicle image, passed nodes, and timestamp. The initial vehicle trajectory includes a sequence of vehicle passed nodes and a corresponding timestamp sequence.

[0009] S2. Locate missing segments in the vehicle trajectory based on the time interval between adjacent trajectory nodes and the node connectivity in the initial vehicle trajectory; generate a set of candidate trajectory nodes corresponding to the missing segments based on the road network topology, vehicle travel distance constraints, and time constraints, and obtain a set of candidate vehicle images corresponding to the candidate nodes to provide candidate samples for subsequent multimodal trajectory recognition.

[0010] S3. Construct a multimodal trajectory recognition model, which includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module;

[0011] The system includes a visual feature encoding module for encoding vehicle images to generate vehicle appearance feature vectors and a visual similarity module for generating visual similarity based on the feature distance between the target vehicle appearance features and the candidate vehicle appearance features; a spatiotemporal state encoding module for encoding the road network distance information and travel time interval information between corresponding nodes in the vehicle images and updating the state by combining the historical trajectory hidden state to generate the current trajectory spatiotemporal hidden state, thereby representing the spatiotemporal dependencies in the continuous driving process of the vehicle; and a feature fusion module for jointly representing the visual similarity and the current trajectory spatiotemporal hidden state to obtain fused features and calculating the comprehensive matching score of the candidate trajectory based on the fused features.

[0012] The optimal candidate node is selected based on the candidate trajectory matching score, the vehicle trajectory is gradually expanded, and the trajectory status is continuously updated in combination with the road network topology constraints until the trajectory termination condition is met, so as to realize the completion of the missing vehicle trajectory and generate the complete driving trajectory of the target vehicle.

[0013] Furthermore, the visual feature encoding module and the spatiotemporal state encoding module are connected in parallel to the feature fusion module; the visual feature encoding module uses a pre-trained visual neural network as the basic image feature extraction unit to encode the vehicle images, generate vehicle visual feature vectors, and calculate the visual similarity between vehicles through feature distance.

[0014] Furthermore, the spatiotemporal state encoding module includes an input normalization layer, a feature concatenation layer, a low-dimensional mapping layer, a temporal encoding layer, and a hidden state output layer. The input normalization layer is used to normalize the road network distance information between vehicle nodes and the vehicle travel time interval information. The feature concatenation layer is used to concatenate the normalized distance features with the time features to generate a spatiotemporal feature vector. The low-dimensional mapping layer is used to map the high-dimensional spatiotemporal feature vector into a low-dimensional structured spatiotemporal vector. The temporal encoding layer is used to update the trajectory state by combining the historical trajectory hidden state with the current spatiotemporal features. The hidden state output layer is used to output the updated trajectory spatiotemporal hidden state.

[0015] As a preferred option, the temporal coding layer uses a gated recurrent unit (GRU) network to update the trajectory state. By dynamically adjusting the fusion weight between the hidden state of the historical trajectory and the current spatiotemporal features through the update gate and the reset gate, the historical driving patterns of the vehicle and the current spatiotemporal change features are preserved.

[0016] Furthermore, the feature fusion module includes a feature splicing layer, a feature adjustment layer, and a matching score calculation layer; the feature splicing layer is used to fuse visual similarity with the spatiotemporal hidden state of the current trajectory to obtain fused features; the feature adjustment layer is used to perform nonlinear mapping on the fused features; and the matching score calculation layer is used to output the comprehensive matching score of the candidate trajectory.

[0017] Furthermore, during model training, a multi-class cross-entropy loss function is used to supervise the learning of candidate trajectory matching scores, and the model parameters are updated through iterative optimization.

[0018] Furthermore, the trajectory expansion and completion process adopts a sequential node search strategy, selecting the optimal candidate node in turn based on the candidate trajectory matching score, and continuously updating the trajectory status in combination with vehicle travel time constraints and road network topology constraints until the complete trajectory recovery of the target vehicle is completed.

[0019] Based on the above scheme, this invention generates initial vehicle trajectories using AVI vehicle passing records and identifies missing trajectory segments by combining visual features and spatiotemporal state information, thereby achieving continuous association and complete recovery of vehicle trajectories in urban road network environments. By fusing visual similarity and road network topology constraints, it effectively reduces the mismatch problem caused by visually highly similar vehicles. By introducing historical trajectory hidden states to model the continuous motion law of vehicles, it enhances the physical continuity and spatiotemporal rationality in the trajectory reasoning process, thereby improving the accuracy and robustness of vehicle trajectory recognition in complex urban road network environments.

[0020] This invention also provides a vehicle trajectory recognition system for urban road networks based on multimodal data fusion, comprising:

[0021] The data acquisition module is used to acquire urban road network topology information, AVI vehicle passage records, and monitored vehicle records. The AVI vehicle passage records include vehicle license plate numbers, node numbers, and timestamps, while the monitored vehicle records include vehicle images, node numbers, and timestamps.

[0022] The trajectory generation module is used to generate the initial trajectory of the target vehicle by associating the vehicle passing nodes according to the vehicle identification and time sequence based on the AVI vehicle passing records.

[0023] The missing segment localization and candidate generation module is used to identify missing trajectory segments based on the initial trajectory, and generate a set of candidate nodes and a set of corresponding candidate vehicle images by combining road network topology, distance constraints and time constraints.

[0024] The multimodal trajectory recognition model includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module, wherein:

[0025] The visual feature encoding module is used to extract the visual features of the target vehicle and candidate vehicles and calculate their visual similarity.

[0026] The spatiotemporal state coding module is used to generate the current trajectory's spatiotemporal hidden state based on road network distance information, time interval information, and historical trajectory status.

[0027] The feature fusion module is used to fuse visual similarity with spatiotemporal hidden states and output trajectory matching scores.

[0028] The trajectory expansion and completion module is used to select the optimal candidate node based on the candidate trajectory matching score, gradually expand the vehicle trajectory until the termination condition is met, and generate the complete driving trajectory of the target vehicle.

[0029] Beneficial Effects: This invention improves the continuity and completeness of vehicle trajectory recognition in urban road network environments by introducing a multimodal fusion method that integrates AVI vehicle passage records and surveillance vehicle records. It utilizes AVI vehicle passage records to generate initial vehicle trajectories and combines them with surveillance vehicle records to identify missing trajectory segments. By fusing vehicle visual features with road network topology constraints, it effectively reduces mismatches caused by visually highly similar vehicles, improving the accuracy of vehicle matching and tracking in complex traffic scenarios. The construction of a spatiotemporal state coding module jointly models the historical trajectory state and current spatiotemporal features of vehicles, enhancing the spatiotemporal continuity and rationality of the trajectory reasoning process. Furthermore, by combining road network topology constraints with vehicle travel time constraints for trajectory association, it improves the stability and robustness of vehicle trajectory recognition in complex urban road network environments. Experimental results show that this method achieves a trajectory recognition accuracy of 72.06% on regional road networks, significantly outperforming single visual matching and traditional deep learning sequence models. Attached Figure Description

[0030] Figure 1 A schematic diagram of the overall architecture of the proposed trajectory recognition and completion method is shown below;

[0031] Figure 2 A visual example of the vehicle trajectory obtained using this method. Detailed Implementation

[0032] The technical solution of the present invention will be described in detail below with reference to the embodiments and accompanying drawings.

[0033] like Figure 1 As shown, this invention provides a method for vehicle trajectory recognition and completion in urban road networks based on multimodal data. This method comprehensively utilizes AVI vehicle passing records, surveillance vehicle records, and urban road network topology information to correlate and identify the full sample vehicle trajectories in the urban road network environment, thereby obtaining continuous vehicle trajectories in complex traffic scenarios. The specific implementation steps are as follows:

[0034] First, the urban road network topology information, AVI vehicle passage records, and monitored vehicle records are acquired. The urban road network topology information includes intersection node IDs, road connectivity relationships between nodes, and road traffic directions; the AVI vehicle passage records include vehicle license plates, passed nodes, and timestamps; and the monitored vehicle records include vehicle images, passed nodes, and timestamps collected from monitoring equipment deployed at urban intersections.

[0035] The urban road network in this embodiment includes multiple urban intersections equipped with monitoring devices. Each intersection monitoring device can collect vehicle images as vehicles pass through the corresponding intersection to ensure the integrity of vehicle visual information during the vehicle trajectory recognition process.

[0036] Based on AVI vehicle passage records, the nodes passed by vehicles are associated according to their license plate numbers and time sequence to generate an initial vehicle trajectory. The initial vehicle trajectory includes a sequence of vehicle passing nodes and a corresponding timestamp sequence.

[0037] After the initial vehicle trajectory is generated, abnormal interruptions in the vehicle trajectory are detected based on the time interval between adjacent trajectory nodes and the node connectivity. When there is an abnormal time span, node jump, or trajectory continuity interruption between adjacent AVI nodes, it is determined that there is a missing trajectory segment at the corresponding location.

[0038] Subsequently, reachable nodes are searched based on the road network topology, and nodes that do not meet the continuous driving condition are filtered out by combining the historical maximum passage time constraint and travel distance constraint of AVI vehicle records, generating a candidate node set and a candidate vehicle set for each node; the candidate vehicle set comes from the monitored vehicle records. Specifically, taking the starting node of the missing trajectory segment of the target vehicle as the starting node, candidate nodes that the vehicle may reach in the next moment are searched based on the road connectivity relationship, and candidate vehicle images that do not meet the continuous driving pattern of the vehicle are filtered out according to the theoretical driving speed constraint and time window constraint, so as to reduce the size of candidate nodes and candidate vehicles in the subsequent trajectory recognition process.

[0039] Subsequently, a trajectory recognition model is constructed. The multimodal trajectory recognition model includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module. The visual feature encoding module and the spatiotemporal state encoding module are connected in parallel to the feature fusion module to complete the multimodal feature joint inference in the vehicle trajectory recognition process.

[0040] Specifically, the visual feature encoding module uses the pre-trained visual feature extraction network CLIPReID model as the basic image feature extraction unit to extract features from vehicle images and generate vehicle appearance feature vectors. These vehicle appearance feature vectors are used to represent vehicle model, color, body structure, and local textures, among other vehicle appearance information.

[0041] For the target vehicle image and the candidate vehicle image, the visual feature encoding module uses the CLIPReID model to extract the corresponding vehicle appearance feature vectors, and calculates the feature distance through the similarity calculation layer to generate vehicle visual similarity, which is used to characterize the degree of visual similarity between the candidate vehicle and the target vehicle.

[0042] The spatiotemporal state encoding module is used to encode and recursively update the spatiotemporal motion information in the vehicle trajectory. The spatiotemporal state encoding module includes an input normalization layer, a feature concatenation layer, a low-dimensional mapping layer, a temporal encoding layer, and a hidden state output layer.

[0043] The input normalization layer is used to normalize the distance information between vehicle nodes and the vehicle passage time interval information to eliminate the influence between different units; the feature concatenation layer is used to concatenate the normalized distance features and time features to generate a spatiotemporal feature vector to realize the joint representation of vehicle spatial information and time information.

[0044] The low-dimensional mapping layer is used to map high-dimensional spatiotemporal feature vectors into low-dimensional structured spatiotemporal vectors, thereby reducing the computational complexity of the model and preserving key spatiotemporal features during vehicle operation.

[0045] The temporal coding layer employs a gated recurrent unit (GRU) network to update the trajectory state. Specifically, the historical trajectory spatiotemporal hidden state and the current spatiotemporal features are input into the GRU network. The fusion weights between the historical trajectory spatiotemporal hidden state and the current spatiotemporal features are dynamically adjusted through update gates and reset gates, thereby generating an updated trajectory spatiotemporal hidden state to preserve the historical motion patterns and current spatiotemporal change features during the vehicle's continuous driving process.

[0046] The hidden state output layer is used to standardize the updated trajectory spatiotemporal hidden state to improve the stability and robustness of the spatiotemporal feature representation.

[0047] The feature fusion module is used to fuse vehicle visual similarity with the spatiotemporal hidden state of the trajectory and output candidate trajectory matching scores. The feature fusion module includes a feature splicing layer, a feature adjustment layer, and a matching score calculation layer.

[0048] The feature splicing layer is used to splice and fuse the vehicle visual similarity with the current trajectory spatiotemporal hidden state to generate a fused feature vector; the feature adjustment layer is used to perform nonlinear mapping and dimension adjustment on the fused features; and the matching score calculation layer is used to output the trajectory matching score between the candidate trajectory and the target vehicle.

[0049] The specific structure of each module in this embodiment is shown in Table 1 below.

[0050] Table 1. Backbone Network Structure Diagram

[0051]

[0052] The visual feature encoding module uses a CLIPReID network as the basic visual encoder to extract features from the input vehicle images. The input vehicle images are uniformly scaled to 256×128 pixels, and the output is a 512-dimensional vehicle appearance feature vector. For the target vehicle and candidate vehicles, the corresponding 512-dimensional visual feature vectors are extracted respectively. The visual similarity calculation layer receives the visual feature vectors of the target vehicle and candidate vehicles, and calculates the negative Euclidean distance through the features to obtain a 1-dimensional visual similarity.

[0053] The spatiotemporal state coding module adopts a serial structure of normalization, mapping, and temporal coding. The input data includes two spatiotemporal features: the distance between vehicle nodes and the time interval between vehicle passages. The input normalization layer first normalizes the distance and time to eliminate dimensional differences; then, a two-layer fully connected network maps the 2D spatiotemporal features into a 128-dimensional structured spatiotemporal vector. The first fully connected layer outputs 256-dimensional features, and the second fully connected layer outputs 128-dimensional features, employing the ReLU activation function to enhance feature representation.

[0054] The temporal coding layer employs a GRU network to recursively update the trajectory state. The GRU hidden state dimension is set to 128 dimensions, with the initial hidden state set as an all-zero vector. Current spatiotemporal features and historical hidden states are input into the GRU unit. The fusion ratio of historical trajectory information and current spatiotemporal information is dynamically adjusted through update and reset gates, outputting a 128-dimensional updated trajectory spatiotemporal hidden state. The hidden state output layer further performs L2 normalization on the spatiotemporal hidden state to improve the stability and robustness of the feature representation. The GRU update formula is as follows:

[0055]

[0056] in, It is a standardized spatiotemporal feature vector. It is a hidden state of history.

[0057] The feature fusion module employs a structure of concatenation, adjustment, and scoring. First, 1D visual similarity features are concatenated with a 128D spatiotemporal hidden state vector to form a 129D fused feature vector. Then, a fully connected layer maps the fused features to the 128D feature space. Finally, a multilayer perceptron consisting of two fully connected layers outputs a 1D matching score, representing the probability of trajectory association between the candidate vehicle and the target vehicle. A higher matching score indicates a greater likelihood that the candidate vehicle belongs to the target vehicle's trajectory. The formula for the feature fusion module is as follows:

[0058]

[0059] in, It is the sigmoid activation function. This represents the adjustment coefficient for visual similarity features. This is a visual similarity feature. In this example... The calculation is performed using adaptive weights, and the formula is as follows:

[0060]

[0061] In this embodiment, the visual feature encoding module outputs the vehicle visual feature vector, and the spatiotemporal state encoding module outputs the trajectory spatiotemporal hidden state vector; the feature fusion module jointly fuses the vehicle visual similarity and the trajectory spatiotemporal hidden state to generate a fused feature representation; the feature adjustment layer is used to perform nonlinear mapping and feature adjustment on the fused feature representation; the matching score calculation layer further outputs the trajectory matching score between the candidate trajectory and the target vehicle.

[0062] To train the trajectory recognition model, a training set from the sample set is used. In this embodiment, the training set and the test set are divided in a 7:3 ratio, but other ratios can be used depending on the specific application scenario.

[0063] During model training, a multi-class cross-entropy loss function is used for supervised learning of candidate trajectory matching scores, and model parameters are updated through iterative optimization. The specific loss function used in model training is as follows:

[0064]

[0065] in, This represents the total number of mini-batch training samples; Indicates the number of candidate vehicles; Indicates the first The comprehensive matching score of each candidate trajectory vehicle, which is generated by the trajectory reconstruction network described in this invention; This indicates the true value of the vehicle.

[0066] After the model is trained, the initial trajectory of the vehicle to be identified is generated based on the AVI vehicle passing record; a set of candidate trajectory nodes is generated according to the current node of the vehicle and the road network topology; visual features of candidate vehicle images are extracted, and candidate trajectory matching scores are generated by combining the spatiotemporal hidden state of the vehicle's historical trajectory, the distance information between vehicle nodes, and the vehicle's travel time information.

[0067] When the candidate trajectory matching score is higher than the preset threshold, the corresponding candidate node is added to the current vehicle trajectory, and the updated trajectory node is used as the starting point for trajectory expansion at the next moment. At the same time, the candidate trajectory is filtered and updated in combination with vehicle travel time constraints and road network topology constraints until the trajectory termination condition is met, thereby completing the identification of missing trajectory segments of the vehicle and finally generating the complete driving trajectory of the target vehicle.

[0068] Compared with existing technologies, this invention integrates AVI vehicle passing records and monitoring vehicle records, uses AVI vehicle passing records to generate initial vehicle trajectories, and combines vehicle visual features and trajectory spatiotemporal state information to identify missing trajectory segments, thereby achieving continuous association and complete recovery of vehicle trajectories in urban road network environments.

[0069] Meanwhile, by fusing vehicle visual features with road network topology constraint information, this invention can effectively reduce the mismatch problem caused by visually highly similar vehicles; by introducing historical trajectory spatiotemporal hidden states to model the continuous motion law of vehicles, it enhances the physical continuity and spatiotemporal rationality in the trajectory reasoning process, thereby improving the stability and robustness of the vehicle trajectory recognition process in complex urban traffic scenarios.

[0070] The urban road network vehicle trajectory recognition system based on multimodal data of the present invention is characterized by comprising:

[0071] The data acquisition module is used to acquire urban road network topology information, AVI vehicle passage records, and monitored vehicle records. The AVI vehicle passage records include vehicle license plate numbers, node numbers, and timestamps, while the monitored vehicle records include vehicle images, node numbers, and timestamps.

[0072] The trajectory generation module is used to generate the initial trajectory of the target vehicle by associating the vehicle passing nodes according to the vehicle identification and time sequence based on the AVI vehicle passing records.

[0073] The missing segment localization and candidate generation module is used to identify missing trajectory segments based on the initial trajectory, and generate a set of candidate nodes and a set of corresponding candidate vehicle images by combining road network topology, distance constraints and time constraints.

[0074] The multimodal trajectory recognition model includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module, wherein:

[0075] The visual feature encoding module is used to extract the visual features of the target vehicle and candidate vehicles and calculate their visual similarity.

[0076] The spatiotemporal state coding module is used to generate the current trajectory's spatiotemporal hidden state based on road network distance information, time interval information, and historical trajectory status.

[0077] The feature fusion module is used to fuse visual similarity with spatiotemporal hidden states and output trajectory matching scores.

[0078] The trajectory expansion and completion module is used to select the optimal candidate node based on the candidate trajectory matching score, gradually expand the vehicle trajectory until the termination condition is met, and generate the complete driving trajectory of the target vehicle.

[0079] Figure 2 The results of using this method for trajectory recognition in different scenarios are shown. As can be seen from the figure, this method can accurately track the trajectories of vehicles on the road network.

[0080] To better verify the beneficial effects of this invention, the overall model constructed by this invention will be compared with existing neural network models in terms of performance below:

[0081] The detection performance of the model proposed in this invention was analyzed compared with existing multimodal vehicle trajectory recognition models, and the results are as follows:

[0082] Table 2 Model Performance Comparison

[0083]

[0084] As shown in Table 2, the method provided by this invention has the highest accuracy and is a good trajectory recognition and completion method based on multimodal data fusion.

[0085] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A method for vehicle trajectory recognition and completion in urban road networks based on multimodal data, characterized in that, include: S1. Obtain urban road network topology information, monitor vehicle records and AVI vehicle passage records; The AVI vehicle passage records are associated with the vehicle identification information, and the road network nodes passed by the same vehicle are sorted in chronological order to generate the initial trajectory of the target vehicle. The initial trajectory includes the sequence of nodes passed by the vehicle and the corresponding timestamp sequence. S2. Locate missing segments in the vehicle trajectory based on the time interval between adjacent trajectory nodes and the node connectivity in the initial vehicle trajectory; generate a set of candidate trajectory nodes and a set of corresponding candidate vehicle images by combining road network topology, distance constraints and time constraints. S3. Construct a multimodal trajectory recognition model, which includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module. The visual feature encoding module and the spatiotemporal state encoding module are connected in parallel to the feature fusion module; wherein: The visual feature encoding module is used to encode the visual features of the target vehicle image and the candidate vehicle image, and to calculate the visual similarity between the target vehicle and the candidate vehicle. The spatiotemporal state coding module is used to generate the current trajectory spatiotemporal hidden state based on the road network distance information between vehicle nodes, the travel time interval information, and the historical trajectory spatiotemporal hidden state. The feature fusion module is used to fuse the visual similarity with the current trajectory spatiotemporal hidden state and output the candidate trajectory matching score; S4. Select the optimal candidate node based on the candidate trajectory matching score, gradually expand the vehicle trajectory, and continuously update the candidate node set in combination with road network topology constraints until the trajectory termination condition is met, so as to realize the missing vehicle trajectory completion and generate the complete driving trajectory of the target vehicle.

2. The method according to claim 1, characterized in that: The visual feature encoding module uses a pre-trained visual feature extraction network to extract features from vehicle images, generate vehicle visual feature vectors, and calculate the visual similarity between the target vehicle and the candidate vehicles.

3. The method according to claim 1, characterized in that: The spatiotemporal state encoding module includes an input normalization layer, a feature concatenation layer, a low-dimensional mapping layer, a temporal encoding layer, and a hidden state output layer, wherein: The input normalization layer is used to normalize the road network distance between vehicle nodes and the vehicle travel time interval; The feature concatenation layer is used to concatenate the normalized distance features with the time features to generate a spatiotemporal feature vector; The low-dimensional mapping layer is used to map spatiotemporal feature vectors into structured spatiotemporal representations; The temporal coding layer is used to update the current trajectory state by combining the spatiotemporal hidden states of historical trajectories; The hidden state output layer is used to output the updated spatiotemporal hidden state of the current trajectory.

4. The method according to claim 3, characterized in that: The temporal coding layer adopts a gated recurrent unit (GRU) network, which dynamically adjusts the fusion weight between the historical trajectory hidden state and the current spatiotemporal features through update gates and reset gates, so as to preserve the vehicle's historical driving patterns and current spatiotemporal change features and obtain the current trajectory spatiotemporal hidden state.

5. The method according to claim 1, characterized in that: The feature fusion module includes a feature concatenation layer, a feature adjustment layer, and a matching score calculation layer, wherein: The feature stitching layer is used to fuse vehicle visual similarity with the spatiotemporal hidden state of trajectory to obtain fused features; The feature adjustment layer is used to perform non-linear mapping on the fused features; The matching score calculation layer is used to output the candidate trajectory matching score.

6. The method according to claim 1, characterized in that: During model training, a multi-class cross-entropy loss function is used to supervise the learning of candidate trajectory matching scores, and the model parameters are continuously updated through iterative optimization.

7. The method according to claim 1, characterized in that: The trajectory expansion and completion process adopts a sequential node search strategy. The optimal candidate node is selected sequentially based on the candidate trajectory matching score, and the trajectory status is continuously updated in combination with vehicle travel time constraints and road network topology constraints until the complete trajectory of the target vehicle is restored.

8. A vehicle trajectory recognition system for urban road networks based on multimodal data, characterized in that, include: The data acquisition module is used to acquire urban road network topology information, AVI vehicle passage records, and monitored vehicle records. The AVI vehicle passage records include vehicle license plate numbers, node numbers, and timestamps, while the monitored vehicle records include vehicle images, node numbers, and timestamps. The trajectory generation module is used to generate the initial trajectory of the target vehicle by associating the vehicle passing nodes with vehicle identification and time sequence based on the AVI vehicle passing records. The missing segment localization and candidate generation module is used to identify missing trajectory segments based on the initial trajectory, and generate a set of candidate nodes and a set of corresponding candidate vehicle images by combining road network topology, distance constraints and time constraints. The multimodal trajectory recognition model includes a visual feature encoding module, a spatiotemporal state encoding module, and a feature fusion module, wherein: The visual feature encoding module is used to extract the visual features of the target vehicle and candidate vehicles and calculate their visual similarity. The spatiotemporal state coding module is used to generate the current trajectory's spatiotemporal hidden state based on road network distance information, time interval information, and historical trajectory status. The feature fusion module is used to fuse visual similarity with spatiotemporal hidden states and output trajectory matching scores. The trajectory expansion and completion module is used to select the optimal candidate node based on the candidate trajectory matching score, gradually expand the vehicle trajectory until the termination condition is met, and generate the complete driving trajectory of the target vehicle.

Images