Vehicle traffic data in-transit algorithm based on 3D continuous trajectory technology

Abstract

The invention provides a vehicle traffic data in-transit algorithm based on a 3D continuous trajectory technology. According to the invention, multi-dimensional space-time appearance joint features (including appearance similarity, license plate association degree, spatial scale difference and space-time relationship) are constructed by using vehicle appearance features, latitude and longitude data (spatial position), starting and ending time and license plate recognition information; the tracks with similar vehicle appearances, same license plates, continuous time and continuous spatial positions are merged as far as possible, and the probability of merging the tracks with different vehicle appearances, large license plate differences, long time intervals or long spatial position distances is small, so that mismatching is reduced; according to the invention, the method can combine the tracks which cross cameras, come from the same vehicle and are basically continuous in time and space, forms a complete and accurate vehicle driving path, and effectively improves the precision and robustness of vehicle track recognition in a complex traffic scene.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an in-transit algorithm for vehicle traffic data based on 3D continuous trajectory technology. Background Art

[0002] Traffic violations (such as illegal U-turns and running red lights) are common on urban roads, and using cameras to capture offending vehicles has become an important means of addressing them. However, due to factors such as obstruction and long shooting distances, license plate information on offending vehicles is often difficult to clearly identify, necessitating trajectory backtracking to locate the vehicle's recognizable position. Due to the limited monitoring range of a single camera, it is difficult to independently track the entire urban road scene.

[0003] Therefore, cross-camera vehicle recognition technology has emerged. This technology system is divided into two stages: first, target detection and local trajectory generation are completed using a single camera, and then the discrete trajectories collected by multiple cameras are correlated and fused to form a complete vehicle driving path.

[0004] Currently, multi-target detection algorithms in single-camera environments have become increasingly sophisticated, but cross-device trajectory association technology still faces significant challenges. Summary of the Invention

[0005] This application provides a method and device for in-transit vehicle traffic data based on 3D continuous trajectory technology, and provides a cross-camera vehicle recognition method that effectively improves the accuracy and robustness of vehicle trajectory recognition in complex traffic scenarios. The technical solution is as follows.

[0006] In a first aspect, a method for in-transit vehicle traffic data based on 3D continuous trajectory technology is provided, the method comprising:

[0007] Performing vehicle track recognition on the video stream captured by each camera among the multiple cameras to obtain detection frame parameters of each track in the video stream captured by each camera, vehicle appearance features of each track, and license plate recognition information of each track, wherein the detection frame parameters of each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera, the vehicle appearance features of each track are used to characterize the vehicle appearance features of the vehicle, and the license plate recognition information of each track is used to characterize the license plate number of the vehicle in the corresponding track;

[0008] Based on the calibrated parameters of each camera, the detection frame parameters of each track corresponding to each camera are converted from the camera coordinate system to the world coordinate system to obtain the longitude and latitude data of each track, where the longitude and latitude data of each track includes the position coordinates, width and height of the vehicle in the world coordinate system;

[0009] Based on the vehicle appearance features of each trajectory, a candidate matching set is constructed, wherein the candidate matching set includes a plurality of candidate trajectory pairs, each candidate trajectory pair includes a trajectory and other trajectories whose similarity with the vehicle appearance features of the one trajectory satisfies a constraint condition;

[0010] Based on the vehicle appearance features of each trajectory, the latitude and longitude data of each trajectory, the license plate recognition information of each trajectory, and the start and end times of each trajectory in the video stream, feature extraction is performed on each candidate track pair in the candidate matching set to obtain a spatiotemporal appearance joint feature of each candidate track pair. The spatiotemporal appearance joint feature of each candidate track pair includes the vehicle appearance similarity, license plate association, spatial scale feature, and spatiotemporal relationship feature of the corresponding candidate track pair. The spatial scale feature is used to characterize the physical space occupation difference of the vehicles in the corresponding candidate track pair, and the spatiotemporal relationship feature is used to characterize the temporal and spatial differences of the corresponding candidate track pair.

[0011] Based on the spatiotemporal appearance joint features of each candidate track pair, classify each candidate track pair using a classification model to obtain a classification result for each candidate track pair, wherein the classification model is used to predict the probability that the corresponding candidate track pairs originate from the same vehicle;

[0012] Based on the classification result of each candidate trajectory pair, the trajectories in the trajectory pair originating from the same vehicle are merged to obtain the vehicle driving path.

[0013] In some embodiments, the feature extraction for each candidate track pair in the candidate matching set is performed based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end times of each track in the video stream to obtain a spatiotemporal appearance joint feature of each candidate track pair, including:

[0014] Determining the cosine similarity between the vehicle appearance features of the two trajectories in each candidate trajectory pair based on the vehicle appearance features of each trajectory, and obtaining the vehicle appearance similarity of each candidate trajectory pair;

[0015] Based on the license plate recognition information of each trajectory, a longest common substring algorithm is used to determine the length of the longest common substring of the two trajectories in each candidate trajectory pair, and obtain the license plate association degree of each candidate trajectory pair;

[0016] Based on the latitude and longitude data of each trajectory, determining the projected area of ​​the two vehicles in each candidate trajectory pair in the world coordinate system and the difference between the projected areas, and obtaining the spatial scale feature of each candidate trajectory pair;

[0017] Based on the latitude and longitude data of each trajectory and the start and end time of each trajectory in the video stream, the temporal relationship and spatial relationship of the two trajectories in each candidate trajectory pair are analyzed to obtain the spatiotemporal relationship characteristics of each candidate trajectory pair;

[0018] The vehicle appearance similarity of each candidate trajectory pair, the license plate association degree of each candidate trajectory pair, the spatial scale feature of each candidate trajectory pair, and the spatiotemporal relationship feature of each candidate trajectory pair belonging to the same candidate trajectory pair are spliced ​​to obtain the spatiotemporal appearance joint feature of each candidate trajectory pair.

[0019] In some embodiments, the multiple candidate trajectory pairs include at least one time interval trajectory pair, the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments, and the spatiotemporal relationship features of each time interval trajectory pair include time interval, spatial distance, kinematic parameters, and directional consistency. Based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream, the temporal and spatial relationships of the two trajectories in each candidate trajectory pair are analyzed to obtain the spatiotemporal relationship features of each candidate trajectory pair, including:

[0020] The time interval of each time interval type trajectory pair is obtained by subtracting the start time of the latter trajectory from the end time of the former trajectory in each time interval type trajectory pair;

[0021] Determine the Euclidean distance between the longitude and latitude data of the two tracks in each time interval track pair to obtain the spatial distance of each time interval track pair;

[0022] The kinematic parameters of each time interval trajectory pair are obtained by subtracting the instantaneous velocity of the latter trajectory at the start time from the instantaneous velocity of the preceding trajectory at the end time.

[0023] Based on the position coordinates of the vehicle in each trajectory in each time interval trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time, the motion direction vector of each trajectory is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time interval trajectory pair is determined to obtain the direction consistency of each time interval trajectory pair.

[0024] In some embodiments, the plurality of candidate trajectory pairs include at least one temporally overlapping trajectory pair, wherein the start and end time periods of the two trajectories in each temporally overlapping trajectory pair overlap, and the spatiotemporal relationship features of each temporally overlapping trajectory pair include overlapping duration, spatial proximity, speed synchronization, spatial overlap, and directional stability. The temporal and spatial relationships of the two trajectories in each candidate trajectory pair are analyzed based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream, to obtain the spatiotemporal relationship features of each candidate trajectory pair, including:

[0025] Determine the length of the overlapping time segment between the start and end time periods of the two trajectories in each time-coinciding trajectory pair, and obtain the overlapping duration of each time-coinciding trajectory pair;

[0026] Determine the Euclidean distance between the starting moments of the vehicles in the two trajectories of each temporally overlapping trajectory pair in the overlapping time segment, the Euclidean distance between the middle moments in the overlapping time segment, and the Euclidean distance between the ending moments in the overlapping time segment, and obtain the spatial proximity of each temporally overlapping trajectory pair;

[0027] Determine the difference between the instantaneous speeds of the vehicles in the two trajectories in each time-coinciding trajectory pair at the start time of the overlapping time segment, the difference between the instantaneous speeds at the middle time of the overlapping time segment, and the difference between the instantaneous speeds at the end time of the overlapping time segment to obtain the speed synchronization degree of each time-coinciding trajectory pair;

[0028] Determine the IOU of the projection areas of the vehicles in the two trajectories of each time-coincident trajectory pair, and obtain the spatial overlap of each time-coincident trajectory pair, wherein the spatial overlap includes the IOU of the projection areas at the starting moment of the overlapping time segment, the IOU of the projection areas at the middle moment of the overlapping time segment, and the IOU of the projection areas at the ending moment of the overlapping time segment;

[0029] Based on the position coordinates of the vehicle in each trajectory of each time-coinciding trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time, the motion direction vector of each trajectory in the overlapping time period is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time-coinciding trajectory pair in the overlapping time period is determined to obtain the directional stability of each time-coinciding trajectory pair.

[0030] In some implementations, the candidate matching set includes k candidate track pairs corresponding to the first track, and constructing the candidate matching set based on the vehicle appearance features of each track includes:

[0031] Building a cosine similarity index database based on the vehicle appearance features of each trajectory;

[0032] For the first trajectory, an approximate nearest neighbor search artificial neural network (ANN) is used to filter the trajectories with the top k cosine similarities to the first trajectory from the cosine similarity index database to obtain k second trajectories. The first trajectory is then paired with each of the k second trajectories to form a candidate trajectory pair, resulting in k candidate trajectory pairs corresponding to the first trajectory.

[0033] In some embodiments, before classifying each candidate trajectory pair using a classification model based on the spatiotemporal appearance joint features of each candidate trajectory pair to obtain a classification result for each candidate trajectory pair, the method further includes:

[0034] Acquire a sample set, the sample set comprising a plurality of sample trajectory pairs, the plurality of sample trajectory pairs comprising a plurality of first-category sample trajectory pairs and a plurality of second-category sample trajectory pairs, each of the plurality of first-category sample trajectory pairs originating from the same vehicle, each of the plurality of first-category sample trajectory pairs including a positive sample label, each of the plurality of second-category sample trajectory pairs originating from a different vehicle, each of the plurality of second-category sample trajectory pairs including a negative sample label;

[0035] Based on the vehicle appearance features of each sample trajectory pair, the latitude and longitude data of each sample trajectory pair, the license plate recognition information of each sample trajectory pair, and the start and end time of each sample trajectory pair in the video stream, feature extraction is performed on each sample trajectory pair in the sample set to obtain a spatiotemporal appearance joint feature of each sample trajectory pair;

[0036] An initial classification model is trained based on the spatiotemporal appearance joint features of each sample trajectory pair and a label of each sample trajectory pair to obtain the classification model.

[0037] In some embodiments, the calibrated parameters of each camera include an intrinsic parameter matrix and an extrinsic parameter matrix of each camera, and based on the calibrated parameters of each camera, converting the detection box parameters of each trajectory from the camera coordinate system to the world coordinate system to obtain the longitude and latitude data of each trajectory includes:

[0038] The detection frame parameters of each track are multiplied by the intrinsic parameter matrix and the extrinsic parameter matrix of the corresponding camera to obtain the longitude and latitude data of each track.

[0039] In some embodiments, after constructing a candidate matching set based on the vehicle appearance features of each trajectory, the method further includes at least one of the following:

[0040] For any track pair in the candidate matching set, compare the license plate recognition information of the two tracks in the track pair to obtain the license plate character difference between the two tracks in the track pair; if the license plate character difference exceeds the character difference threshold, delete the track pair from the candidate matching set; and / or,

[0041] For any trajectory pair in the candidate matching set, if the time gap between the two trajectories in the trajectory pair exceeds a time gap threshold, the trajectory pair is deleted from the candidate matching set; and / or,

[0042] For each time interval trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time interval trajectory pair in the world coordinate system exceeds a first distance threshold, the time interval trajectory pair is deleted from the candidate matching set, and the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments; and / or,

[0043] For each temporally overlapping trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the temporally overlapping trajectory pair in the world coordinate system during the overlapping time segment exceeds a second distance threshold, the temporally overlapping trajectory pair is deleted from the candidate matching set. The start and end time segments of the two trajectories in each temporally overlapping trajectory pair have overlapping time segments.

[0044] In some embodiments, before constructing a candidate matching set based on the vehicle appearance features of each trajectory, the method further includes:

[0045] Based on the license plate recognition information of each track, tracks with the same license plate number are merged.

[0046] In a second aspect, a vehicle traffic data in-transit device based on 3D continuous trajectory technology is provided, the device comprising:

[0047] an identification module for identifying vehicle tracks in a video stream captured by each of a plurality of cameras, and obtaining detection frame parameters for each track in the video stream captured by each camera, vehicle appearance features for each track, and license plate recognition information for each track, wherein the detection frame parameters for each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera, the vehicle appearance features for each track are used to characterize the vehicle appearance features, and the license plate recognition information for each track is used to characterize the license plate number of the vehicle in the corresponding track;

[0048] A coordinate system conversion module, configured to convert the detection frame parameters of each track corresponding to each camera from the camera coordinate system to the world coordinate system based on the calibrated parameters of each camera, thereby obtaining the longitude and latitude data of each track, wherein the longitude and latitude data of each track includes the position coordinates, width, and height of the vehicle in the world coordinate system;

[0049] a matching set construction module, configured to construct a candidate matching set based on the vehicle appearance features of each trajectory, wherein the candidate matching set includes a plurality of candidate trajectory pairs, each candidate trajectory pair including a trajectory and other trajectories whose similarity to the vehicle appearance features of the one trajectory satisfies a constraint condition;

[0050] a feature extraction module configured to extract features from each candidate track pair in the candidate matching set based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end times of each track in the video stream, to obtain a spatiotemporal appearance joint feature for each candidate track pair. The spatiotemporal appearance joint feature of each candidate track pair includes the vehicle appearance similarity, license plate association, spatial scale features, and spatiotemporal relationship features of the corresponding candidate track pairs. The spatial scale features are used to characterize the physical space occupancy differences of the vehicles in the corresponding candidate track pairs, and the spatiotemporal relationship features are used to characterize the temporal and spatial differences of the corresponding candidate track pairs.

[0051] a classification module, configured to classify each candidate track pair based on the spatiotemporal appearance joint features of each candidate track pair using a classification model to obtain a classification result for each candidate track pair, wherein the classification model is configured to predict the probability that the corresponding candidate track pairs originate from the same vehicle;

[0052] The merging module is configured to merge the trajectories in the trajectory pairs originating from the same vehicle based on the classification results of each candidate trajectory pair to obtain a vehicle driving path.

[0053] In some embodiments, the feature extraction module is configured to determine, based on the vehicle appearance features of each trajectory, the cosine similarity between the vehicle appearance features of the two trajectories in each candidate trajectory pair, to obtain the vehicle appearance similarity of each candidate trajectory pair; determine, based on the license plate recognition information of each trajectory, the length of the longest common substring of the two trajectories in each candidate trajectory pair using a longest common substring algorithm, to obtain the license plate association of each candidate trajectory pair; determine, based on the latitude and longitude data of each trajectory, the projected area of ​​the two vehicles in each candidate trajectory pair in the world coordinate system and the difference between the projected areas, to obtain the spatial scale feature of each candidate trajectory pair; analyze, based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream, the temporal and spatial relationships between the two trajectories in each candidate trajectory pair, to obtain the spatiotemporal relationship feature of each candidate trajectory pair; and concatenate the features belonging to the same candidate trajectory pair from the vehicle appearance similarity of each candidate trajectory pair, the license plate association of each candidate trajectory pair, the spatial scale feature of each candidate trajectory pair, and the spatiotemporal relationship features of each candidate trajectory pair to obtain the spatiotemporal appearance joint feature of each candidate trajectory pair.

[0054] In some embodiments, the multiple candidate trajectory pairs include at least one time interval trajectory pair, the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments, and the spatiotemporal relationship features of each time interval trajectory pair include time interval, spatial distance, kinematic parameters, and directional consistency. The feature extraction module is used to subtract the start time of the later trajectory from the end time of the earlier trajectory in each time interval trajectory pair to obtain the time interval of each time interval trajectory pair; determine the Euclidean distance between the longitude and latitude data of the two trajectories in each time interval trajectory pair to obtain the spatial distance of each time interval trajectory pair; subtract the instantaneous speed of the later trajectory at the start time from the instantaneous speed of the earlier trajectory at the end time of the two trajectories in each time interval trajectory pair to obtain the kinematic parameters of each time interval trajectory pair; determine the motion direction vector of each trajectory based on the position coordinates of the vehicle in each trajectory in each time interval trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time; and determine the cosine similarity between the motion direction vectors of the two trajectories in each time interval trajectory pair to obtain the directional consistency of each time interval trajectory pair.

[0055] In some embodiments, the multiple candidate trajectory pairs include at least one time-overlapping trajectory pair, and there is an overlapping time segment between the start and end time periods of the two trajectories in each time-overlapping trajectory pair. The spatiotemporal relationship features of each time-overlapping trajectory pair include overlapping duration, spatial proximity, speed synchronization, spatial overlap, and directional stability. The feature extraction module is used to determine the length of the overlapping time segment between the start and end time periods of the two trajectories in each time-overlapping trajectory pair to obtain the overlapping duration of each time-overlapping trajectory pair; determine the Euclidean distance between the starting moments of the vehicles in the two trajectories in each time-overlapping trajectory pair in the overlapping time segment, the Euclidean distance between the middle moments in the overlapping time segment, and the Euclidean distance between the ending moments in the overlapping time segment to obtain the spatial proximity of each time-overlapping trajectory pair; determine the difference between the instantaneous speeds of the vehicles in the two trajectories in each time-overlapping trajectory pair in the overlapping time segment, the difference between the instantaneous speeds of the vehicles in the two trajectories in the overlapping time segment, and the difference between the instantaneous speeds of the vehicles in the two trajectories in the overlapping time segment. The speed synchronization degree of each time-coinciding trajectory pair is obtained by calculating the difference between the instantaneous speeds at the middle moments of the segments and the difference between the instantaneous speeds at the end moments of the overlapping time segments; the intersection-overlap ratio (IOU) of the projection areas of the vehicles in the two trajectories in each time-coinciding trajectory pair is determined to obtain the spatial overlap of each time-coinciding trajectory pair, and the spatial overlap includes the intersection-overlap ratio of the projection area at the starting moment of the overlapping time segment, the intersection-overlap ratio of the projection area at the middle moment of the overlapping time segment, and the intersection-overlap ratio of the projection area at the end moment of the overlapping time segment; based on the position coordinates of the vehicle in each trajectory in each time-coinciding trajectory pair in the world coordinate system at the starting moment and the position coordinates of the vehicle in the world coordinate system at the end moment, the motion direction vector of each trajectory in the overlapping time segment is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time-coinciding trajectory pair in the overlapping time segment is determined to obtain the directional stability of each time-coinciding trajectory pair.

[0056] In some embodiments, the candidate matching set includes k candidate trajectory pairs corresponding to the first trajectory, and the matching set construction module is configured to construct a cosine similarity index database based on the vehicle appearance features of each trajectory; for the first trajectory, using an approximate nearest neighbor search artificial neural network (ANN), screen the trajectories with the top k cosine similarities to the first trajectory from the cosine similarity index database to obtain k second trajectories, and form a candidate trajectory pair with each of the k second trajectories to obtain k candidate trajectory pairs corresponding to the first trajectory.

[0057] In some embodiments, the apparatus further comprises:

[0058] an acquisition module, configured to acquire a sample set, the sample set comprising a plurality of sample trajectory pairs, the plurality of sample trajectory pairs comprising a plurality of first-category sample trajectory pairs and a plurality of second-category sample trajectory pairs, each of the plurality of first-category sample trajectory pairs originating from the same vehicle, each of the plurality of first-category sample trajectory pairs including a positive sample label, each of the plurality of second-category sample trajectory pairs originating from a different vehicle, each of the plurality of second-category sample trajectory pairs including a negative sample label;

[0059] The feature extraction module is further configured to extract features from each sample trajectory pair in the sample set based on the vehicle appearance features of each sample trajectory pair, the latitude and longitude data of each sample trajectory pair, the license plate recognition information of each sample trajectory pair, and the start and end times of each sample trajectory pair in the video stream, to obtain a spatiotemporal appearance joint feature of each sample trajectory pair;

[0060] The apparatus further includes a training module configured to train an initial classification model based on the spatiotemporal appearance joint features of each sample trajectory pair and a label of each sample trajectory pair to obtain the classification model.

[0061] In some embodiments, the calibrated parameters of each camera include an intrinsic parameter matrix and an extrinsic parameter matrix of each camera, and the coordinate system conversion module is used to multiply the detection frame parameters of each trajectory with the intrinsic parameter matrix and the extrinsic parameter matrix of the corresponding camera to obtain the longitude and latitude data of each trajectory.

[0062] In some embodiments, the device further comprises at least one of the following:

[0063] A deletion module is configured to compare, for any trajectory pair in a candidate matching set, the license plate recognition information of the two trajectories in the trajectory pair to obtain a degree of difference in the license plate characters of the two trajectories in the trajectory pair; if the degree of difference in the license plate characters exceeds a character difference threshold, delete the trajectory pair from the candidate matching set; and / or, for any trajectory pair in the candidate matching set, if the time gap between the two trajectories in the trajectory pair exceeds a time gap threshold, delete the trajectory pair from the candidate matching set; and / or, for each time interval trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time interval trajectory pair in the world coordinate system exceeds a first distance threshold, delete the time interval trajectory pair from the candidate matching set, and the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments; and / or, for each time overlap trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time overlap trajectory pair in the world coordinate system in the overlapping time segment exceeds a second distance threshold, delete the time overlap trajectory pair from the candidate matching set, and the start and end time periods of the two trajectories in each time overlap trajectory pair have overlapping time segments.

[0064] In some implementations, the merging module is further configured to merge tracks with the same license plate number based on the license plate recognition information of each track.

[0065] In a third aspect, a computer device is provided, comprising: a processor, the processor being coupled to a memory, the memory storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the processor, so that the computer device implements the method described in the first aspect or any optional method of the first aspect.

[0066] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and when the instruction is executed on a computer, the computer executes the method described in the first aspect or any optional method of the first aspect.

[0067] In a fifth aspect, a computer program product is provided, which includes one or more computer program instructions. When the computer program instructions are loaded and run by a computer, the computer executes the method described in the first aspect or any optional method of the first aspect.

[0068] In a sixth aspect, a chip is provided, which includes a programmable logic circuit and / or program instructions, and when the chip is running, it is used to implement the method described in the first aspect or any optional method of the first aspect.

[0069] It can be seen that the embodiments of the present application have the following beneficial effects:

[0070] The method provided in this embodiment utilizes vehicle appearance features, latitude and longitude data (spatial location), start and end times, and license plate recognition information to construct a multi-dimensional spatiotemporal appearance joint feature (including appearance similarity, license plate correlation, spatial scale difference, and spatiotemporal relationship). This allows trajectories with similar vehicle appearance, identical license plates, and continuous temporal and spatial positions to be merged as much as possible, while trajectories with different vehicle appearance, widely different license plates, and long time intervals or long spatial distances are less likely to be merged, thereby reducing mismatches. It is possible to merge trajectories originating from the same vehicle across cameras that are essentially continuous in time and space to form a complete and relatively accurate vehicle driving path, effectively improving the accuracy and robustness of vehicle trajectory recognition in complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a flow chart of a vehicle traffic data in-transit method based on 3D continuous trajectory technology provided by an embodiment of the present application;

[0072] Figure 2 This is a schematic structural diagram of a vehicle traffic data in-transit device based on 3D continuous trajectory technology provided by an embodiment of the present application;

[0073] Figure 3 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0075] The embodiments of the present application provide a cross-camera vehicle recognition method. By detecting independent vehicle trajectories from a single camera, converting the world coordinate system, constructing candidate trajectory pairs based on vehicle appearance features, extracting joint spatiotemporal appearance features, and realizing cross-device trajectory fusion through a classification model, the method enhances trajectory correlation determination through feature engineering, and can accurately splice the vehicle's global path in complex scenarios.

[0076] The cross-camera vehicle tracking task can be deconstructed into the problem of global data association of vehicle trajectories captured by a single camera in a cross-camera scenario. This problem can be further abstracted into a classification decision model through mathematical modeling. For example, please refer to the attached Figure 1 , attached Figure 1 This is a flow chart of a vehicle traffic data en route method based on 3D continuous trajectory technology provided by an embodiment of the present application. Figure 1 The method shown includes the following steps S110 to S180.

[0077] Step S110 , performing vehicle track recognition on the video stream captured by each camera among the multiple cameras, and obtaining detection frame parameters of each track in the video stream captured by each camera, vehicle appearance features of each track, and license plate recognition information of each track.

[0078] The detection frame parameters of each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera. The process of obtaining the detection frame parameters through vehicle track recognition can be implemented based on the vehicle track recognition algorithm of a monocular camera. For example, the ByteTrack algorithm is used to implement vehicle track tracking of a monocular camera. For example, within a unit time window (such as 1 minute), the video stream captured by the i-th camera can identify m i valid trajectories, the total number of trajectories identified by the video streams shot by n cameras can be expressed as Specifically, the j-th track identified by the i-th camera can be represented as , where k represents the number of frames in the video stream where the trajectory is successfully identified, and t i :det i = [x,y,w,h] is the value at t i The detection frame parameters at each moment include the position coordinates, width, and height of the target vehicle in the camera coordinate system.

[0079] The vehicle appearance features of each trajectory are used to characterize the vehicle's appearance. The vehicle appearance features are, for example, vehicle re-identification feature vectors. The cosine similarity between different vehicle re-identification feature vectors can directly reflect vehicle appearance similarity. The vehicle appearance features are, for example, 128-dimensional feature vectors. In some embodiments, the vehicle appearance features are extracted using a vehicle re-identification model. For example, an image containing a vehicle in a video stream captured by a camera is input into the vehicle re-identification model, and the vehicle re-identification model extracts a vehicle re-identification feature vector from the image containing the vehicle.

[0080] The license plate recognition information of each track is used to identify the license plate number of the vehicle in the corresponding track. This provides a reliable basis for vehicle identity authentication.

[0081] Step S110 can also be referred to as the single-camera vehicle trajectory tracking process. Because step S110 extracts not only vehicle appearance features but also detection frame parameters and license plate recognition information, it simultaneously considers multiple dimensions of vehicle features, including appearance, license plate, spatial position, and height geometry. This enhances trajectory feature representation and helps improve the accuracy of subsequent determinations of whether pairs of trajectories originate from the same vehicle.

[0082] In step S120, based on the calibrated parameters of each camera, the detection frame parameters of each track corresponding to each camera are converted from the camera coordinate system to the world coordinate system to obtain the longitude and latitude data of each track. The longitude and latitude data of each track includes the position coordinates, width, and height of the vehicle in the world coordinate system.

[0083] The world coordinate system is also called the geographic coordinate system. The process of converting from the camera coordinate system to the world coordinate system can also be called a projection transformation process. Considering that in a monocular camera system, although the detection frame coordinates can be used to analyze the spatial correlation between homologous tracks (tracks in a video stream captured by a single camera), it is impossible to establish the spatial relationship of vehicles across camera perspectives. In view of this, the embodiments of the present application adopt a unified world coordinate system mapping mechanism. Based on the calibrated camera parameters, the detection frame parameters of each track are converted into longitude and latitude data in the world coordinate system.

[0084] In some embodiments of coordinate transformation, the calibrated parameters of each camera include its intrinsic and extrinsic parameter matrices. Each of the n cameras is rigorously calibrated to obtain its intrinsic and extrinsic parameter matrices. The detection box parameters for each track are multiplied by the intrinsic and extrinsic parameter matrices of the corresponding camera to obtain the longitude and latitude data for each track. Specifically, the pixel position coordinates of the vehicle in the track captured by a camera × the intrinsic parameter matrix of that camera × the extrinsic parameter matrix of that camera = the longitude and latitude coordinates of the vehicle.

[0085] Through step S120, accurate mapping from image space (camera coordinates) to geographic space (world coordinates) is achieved, and the coordinates of the detection frame parameters of each trajectory captured by each camera are uniformly mapped to the world coordinate system, thereby compensating for the impact of different camera coordinate systems. Accurate conversion from pixels (camera coordinates) to geographic coordinates (world coordinates) is achieved, providing a unified benchmark for subsequent trajectory association across cameras.

[0086] Step S130 : Based on the license plate recognition information of each track, tracks with the same license plate number are merged.

[0087] Given that there is a strict one-to-one mapping relationship between license plates and vehicle identities, if the license plate recognition information of two trajectories is the same, it can be considered that the two trajectories come from the same vehicle. In this case, the two trajectories are merged, thereby reducing the computational complexity of subsequent feature extraction and further trajectory merging through the classification model.

[0088] Step S140 : constructing a candidate matching set based on the vehicle appearance features of each trajectory.

[0089] The candidate matching set includes multiple candidate track pairs. Each candidate track pair consists of a track and another track whose similarity to the vehicle appearance features of the first track satisfies a constraint. Taking the first track as an example, the candidate matching set includes k candidate track pairs corresponding to the first track. Each of the k candidate track pairs consists of the first track itself and a second track whose similarity to the vehicle appearance features of the first track satisfies a constraint.

[0090] Regarding the method for constructing a candidate matching set, for example, the vehicle appearance features of the first trajectory are compared with the vehicle appearance features of the second trajectory to obtain the cosine similarity between the vehicle appearance features of the first trajectory and the vehicle appearance features of the second trajectory. If the cosine similarity between the vehicle appearance features of the first trajectory and the vehicle appearance features of the second trajectory satisfies the constraint conditions, the first trajectory and the second trajectory are combined into a candidate trajectory pair. For example, if the cosine similarity between the vehicle appearance features of the first trajectory and the vehicle appearance features of the second trajectory exceeds a similarity threshold, or if the cosine similarity between the vehicle appearance features of the first trajectory and the vehicle appearance features of the second trajectory ranks in the top k, the first trajectory and the second trajectory are combined into a candidate trajectory pair.

[0091] In some implementations, given the rapid growth of trajectory data generated per unit time as multi-camera systems scale, adopting a fully connected comparison strategy (computational complexity O(n²)) would result in severe system performance bottlenecks. Therefore, in the retrieval mechanism provided in the embodiments of the present application, a cosine similarity index database is constructed based on the vehicle appearance features of each trajectory. For a first trajectory, an approximate nearest neighbor search (ANN) is used to filter the cosine similarity index database for the top k trajectories with respect to the first trajectory, obtaining k second trajectories. The first trajectory is then paired with each of the k second trajectories to form a candidate trajectory pair, resulting in k candidate trajectory pairs corresponding to the first trajectory. Here, k is a positive integer, such as 100.

[0092] Directly calculating the cosine similarity of all trajectory pairs would have a computational complexity of O(n²) for a total number of N trajectories. However, by constructing a cosine similarity index database and using an ANN to quickly screen candidate trajectory pairs, only the local data related to the query vector needs to be calculated, reducing the search complexity to O(NlogN) or even lower, significantly reducing computational complexity.

[0093] Step S150 , filtering the trajectory pairs in the candidate matching set based on the license plate, time information and spatial information.

[0094] To further optimize computational efficiency, this embodiment provides the following three-level refined filtering mechanism after constructing the candidate matching set.

[0095] Filtering mechanism 1. License plate filtering mechanism

[0096] For example, for any track pair in the candidate matching set, the license plate recognition information of the two tracks in the track pair is compared to obtain the license plate character difference between the two tracks in the track pair. If the license plate character difference exceeds the character difference threshold, the two tracks in the track pair are determined to be non-homologous vehicles, and the track pair is deleted from the candidate matching set. If the license plate character difference is less than or equal to the character difference threshold, the track pair is retained from the candidate matching set.

[0097] This approach reduces the risk of license plate number detection errors, such as misidentifying one or two characters in a track's license plate number. This can lead to individual character differences between two tracks with the same license plate. By setting a character difference threshold and removing a track pair from the candidate match set if the difference between the two tracks' license plate characters exceeds the threshold, this approach reduces the risk of false license plate detection while ensuring effective filtering, compared to removing a track pair if the two tracks' license plate recognition results differ.

[0098] Regarding the method of determining the difference between the license plate characters of the two trajectories, for example, the length of the longest common substring of the license plates of the two trajectories is determined, and the difference between the license plate number length and the length of the longest common substring is determined as the difference between the license plate characters.

[0099] Filtering Mechanism 2: Space-Time Continuity Filtering Mechanism

[0100] For any trajectory pair in the candidate matching set, if the time gap between the two trajectories in the trajectory pair exceeds the time gap threshold, the trajectory pair is deleted from the candidate matching set.

[0101] To determine the time gap between two tracks, for example, we can subtract the start time of the later track from the end time of the earlier track to obtain the time gap between the two tracks. For example, if Track 1 starts at 1 second and ends at 3 seconds, and Track 2 starts at 5 seconds and ends at 6 seconds, then the time gap between the two tracks is Track 2's start time (5 seconds) - Track 1's end time (3 seconds) = 2 seconds. For another example, if Track 3 starts at 1 second and ends at 3 seconds, and Track 4 starts at 3 seconds and ends at 5 seconds, then the time gap between the two tracks is Track 4's start time (3 seconds) - Track 3's end time (3 seconds) = 0 seconds.

[0102] For example, considering that the average vehicle speed on urban roads is approximately 13 meters per second, this embodiment sets the time gap threshold to 3 seconds. This means that trajectory pairs with time gaps exceeding 3 seconds are filtered out. Since long time gaps significantly increase the uncertainty of trajectory matching and disrupt the spatiotemporal continuity of vehicle motion, by removing trajectory pairs with long time gaps, we avoid mistakenly merging trajectory pairs with excessively large time gaps, thereby improving the accuracy of the global trajectory.

[0103] Filtering Mechanism 3: Spatial Proximity Filtering Mechanism

[0104] There are two ways to process the spatial proximity filtering mechanism.

[0105] The first processing method is: for each time interval trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time interval trajectory pair in the world coordinate system exceeds a first distance threshold (for example, 30 meters), the trajectories in the time interval trajectory pair are determined to originate from different vehicles, and the time interval trajectory pair is deleted from the candidate matching set. The start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments.

[0106] The second processing method: For each temporally overlapping trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the world coordinate system during the overlapping time segment exceeds a second distance threshold (e.g., 10 meters), the temporally overlapping trajectory pair is removed from the candidate matching set. The start and end time segments of the two trajectories in each temporally overlapping trajectory pair overlap. This processing method is designed to accommodate multi-camera calibration errors and temporal synchronization deviations. The filtering operation is only performed when the spatial distance of the trajectories (the projected distance in the world coordinate system) significantly exceeds a physically reasonable range (>10 meters). This threshold balances system robustness and matching accuracy, ensuring efficient trajectory association performance in complex surveillance scenarios.

[0107] The three-level filtering mechanisms shown in Filter Mechanisms 1 through 3 above can be implemented using one, two, or all three of these mechanisms. By combining license plate difference thresholds, spatiotemporal continuity constraints, and spatial distance verification, noisy track pairs can be effectively filtered, improving system robustness.

[0108] In step S160 , based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end time of each track in the video stream, feature extraction is performed on each candidate track pair in the candidate matching set to obtain the spatiotemporal appearance joint features of each candidate track pair.

[0109] Joint spatiotemporal appearance features refer to features that include vehicle appearance similarity, license plate similarity, temporal similarity, and spatial similarity. For example, the joint spatiotemporal appearance features for each candidate trajectory pair include vehicle appearance similarity, license plate association, spatial scale features, and spatiotemporal relationship features. Spatial scale features characterize the differences in the physical space occupied by vehicles in the candidate trajectory pair, while spatiotemporal relationship features characterize the temporal and spatial differences between the candidate trajectory pairs.

[0110] In some embodiments, the feature extraction process includes the following steps.

[0111] Step a: Based on the vehicle appearance features of each track, determine the cosine similarity (reid_sim) between the vehicle appearance features of the two tracks in each candidate track pair to obtain the vehicle appearance similarity of each candidate track pair.

[0112] For example, for track A and track B in a candidate track pair, the cosine similarity between the vehicle appearance features of track A and track B is determined and used as the vehicle appearance similarity for the candidate track pair. A greater cosine similarity indicates that the vehicle appearances of track A and track B are closer. By using vehicle appearance similarity as a basis for determining whether different tracks originate from the same vehicle, it is easier to identify similar-looking vehicles as the same vehicle. This makes it easier to associate and merge different tracks containing similar-looking vehicles, improving the accuracy of track merging.

[0113] Step b: Based on the license plate recognition information of each trajectory, the longest common substring algorithm is used to determine the length of the longest common substring (plate_lcs) of the two trajectories in each candidate trajectory pair, and the license plate association degree of each candidate trajectory pair is obtained.

[0114] For example, for track A and track B in a candidate track pair, determine the length of the longest common substring between the license plate recognition information (e.g., license plate number) of track A and the license plate recognition information (e.g., license plate number) of track B. This length is used as the license plate association degree for the candidate track pair. Furthermore, if a track in a candidate track pair does not have a license plate, the license plate association degree for the candidate track pair is set to zero.

[0115] By introducing license plate correlation as a basis for determining whether different trajectories originate from the same vehicle, vehicles with similar license plates are more easily identified as the same vehicle. Trajectories containing vehicles with similar license plates are also more easily associated and merged, improving the accuracy of trajectory merging.

[0116] Step c: Based on the latitude and longitude data of each trajectory, determine the projected area of ​​the two vehicles in each candidate trajectory pair in the world coordinate system and the difference between the projected areas to obtain the spatial scale characteristics of each candidate trajectory pair.

[0117] For example, for track A and track B in a candidate track pair, since track A's latitude and longitude data includes the vehicle's position, width, and height in the world coordinate system, the projected area of ​​the vehicle in track A in the world coordinate system (car1_area) can be determined based on track A's latitude and longitude data. Similarly, since track B's latitude and longitude data includes the vehicle's position, width, and height in the world coordinate system, the projected area of ​​the vehicle in track B in the world coordinate system (car2_area) can be determined based on track B's latitude and longitude data. The projected area of ​​the vehicle in track A in the world coordinate system is compared with the projected area of ​​the vehicle in track B in the world coordinate system to obtain the difference between the projected areas of track A and track B. The projected area of ​​the vehicle in track A in the world coordinate system, the projected area of ​​the vehicle in track B in the world coordinate system, and the difference between the projected areas of track A and track B are used as the spatial scale features of the candidate track pair. In other embodiments, the difference between the projected areas of trajectory A and trajectory B is used as the spatial scale feature of the candidate trajectory pair, and the projected area of ​​the vehicle in trajectory A in the world coordinate system and the projected area of ​​the vehicle in trajectory B in the world coordinate system are omitted, thereby reducing the data volume of the feature value.

[0118] By introducing spatial scale features as a basis for determining whether different trajectories originate from the same vehicle, vehicles with similar spatial scales (similar projected areas in the world coordinate system, for example, both vehicles in a trajectory pair are trucks or both are cars) are more easily identified as the same vehicle. Different trajectories containing vehicles of similar spatial scales are more easily associated and merged, while vehicles of different spatial scales are easier to distinguish (for example, one vehicle in a trajectory pair is a truck and the other is a car), improving the accuracy of trajectory merging.

[0119] Step d: Based on the latitude and longitude data of each trajectory and the start and end time of each trajectory in the video stream, the temporal and spatial relationships between the two trajectories in each candidate trajectory pair are analyzed to obtain the spatiotemporal relationship features of each candidate trajectory pair.

[0120] In some embodiments, based on the cross-dimensional characteristics of the time dimension, candidate trajectory pairs are divided into two categories: time overlap type and time interval type, and feature expression frameworks are tailored for the two types of candidate trajectory pairs. In a time interval type trajectory pair, there is no overlapping time segment between the start and end time periods of the two trajectories. For example, a time interval type trajectory pair includes trajectory 1 and trajectory 2. The start time of trajectory 1 is 1 second, the end time of trajectory 1 is 3 seconds, the start time of trajectory 2 is 5 seconds, and the end time of trajectory 2 is 6 seconds. In a time overlap type trajectory pair, there is an overlapping time segment between the start and end time periods of the two trajectories. For example, a time overlap type trajectory pair includes trajectory 3 and trajectory 4. The start time of trajectory 3 is 1 second, the end time of trajectory 3 is 3 seconds, the start time of trajectory 4 is 2 seconds, and the end time of trajectory 4 is 4 seconds. In this case, there is an overlapping time segment (2 seconds to 3 seconds) between trajectory 3 and trajectory 4.

[0121] For the feature extraction method of time interval type trajectory pairs, for example, the spatiotemporal relationship features of the time interval type trajectory pairs are the spatiotemporal gap feature group. The spatiotemporal relationship features of the time interval type trajectory pairs include time interval, spatial distance, kinematic parameters and direction consistency. The following examples illustrate the methods of obtaining these features.

[0122] (1) Time interval (t_gap)

[0123] For example, the time interval for each time interval pair is calculated by subtracting the start time of the later track from the end time of the earlier track. Optionally, the time interval is in seconds. For example, if a time interval pair includes Track 1 and Track 2, and Track 1 starts at 1 second and ends at 3 seconds, and Track 2 starts at 5 seconds and ends at 6 seconds, then the time interval is Track 2's start time (5 seconds) - Track 1's end time (3 seconds) = 2 seconds.

[0124] Since the time interval is introduced as the basis for determining whether different trajectories originate from the same vehicle, trajectories with close time intervals are more likely to be merged later, thereby improving the accuracy of trajectory merging.

[0125] (2) Spatial distance (dist_gap)

[0126] For example, the Euclidean distance between the longitude and latitude data of the two tracks in each time interval track pair is determined to obtain the spatial distance of each time interval track pair. For example, a time interval track pair includes track 1 and track 2. Based on the vehicle's position coordinates in the world coordinate system from the longitude and latitude data of track 1 and the vehicle's position coordinates in the world coordinate system from the longitude and latitude data of track 2, the spatial distance between the vehicle in track 1 and the vehicle in track 2 is obtained. By using spatial distance as a basis for determining whether different tracks originate from the same vehicle, two tracks of vehicles that are spatially close are more likely to be subsequently merged, thereby improving the accuracy of track merging.

[0127] (3) Kinematic parameters

[0128] For example, the kinematic parameters of each time interval type trajectory pair are obtained by subtracting the instantaneous velocity of the latter trajectory at the start time from the instantaneous velocity of the former trajectory at the end time of the two trajectories in each time interval type trajectory pair.

[0129] For example, consider a time-interval trajectory pair consisting of trajectory 1 and trajectory 2. Trajectory 2 starts later than trajectory 1. The difference between the instantaneous velocity v2 at the start of trajectory 2 and the instantaneous velocity v1 at the end of trajectory 1 is calculated. The absolute value of the speed difference (v_diff), along with the instantaneous velocity v2 at the start of trajectory 2 and the instantaneous velocity v1 at the end of trajectory 1, is used as the kinematic parameters for the time-interval trajectory pair. By including kinematic parameters as a basis for determining whether different trajectories originate from the same vehicle, two trajectories with similar velocities within a short period of time are more likely to be merged later, improving the accuracy of trajectory merging.

[0130] (4) Directional consistency (cosθ_gap)

[0131] For example, based on the vehicle's position coordinates in the world coordinate system at the start time and the vehicle's position coordinates in the world coordinate system at the end time of each trajectory in each time interval trajectory pair, the motion direction vector of each trajectory is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time interval trajectory pair is determined to obtain the directional consistency of each time interval trajectory pair. For example, the greater the cosine similarity between the motion direction vectors of the two trajectories in a time interval trajectory pair, the greater the directional consistency of the time interval trajectory pair. If the cosine similarity between the motion direction vectors of the two trajectories in a time interval trajectory pair is -1, it indicates that the time interval trajectory pair has opposite directions. If the cosine similarity between the motion direction vectors of the two trajectories in a time interval trajectory pair is 1, it indicates that the time interval trajectory pair has the same direction. Considering that the motion direction of the same vehicle is consistent over a short period of time, the introduction of directional consistency as a basis for determining whether different trajectories originate from the same vehicle increases the probability that two trajectories with approximately similar motion directions over a short period of time will be subsequently merged, thereby improving the accuracy of trajectory merging.

[0132] For example, the spatiotemporal relationship features of temporally overlapping trajectory pairs are a spatiotemporal overlap feature set. The spatiotemporal relationship features of each temporally overlapping trajectory pair include overlap duration, spatial proximity, speed synchronization, spatial overlap, and directional stability. The following examples illustrate how to obtain these features.

[0133] (1) Overlap duration (t_overlap)

[0134] The overlap duration can also be understood as the length of the intersection between the start and end times of two trajectories. For example, the length of the overlapping time segment between the start and end time periods of the two trajectories in each time-overlapping trajectory pair is determined to obtain the overlap duration of each time-overlapping trajectory pair. For example, the time-overlapping trajectory pair includes trajectory 3 and trajectory 4. The start time of trajectory 3 is 1 second, the end time of trajectory 3 is 3 seconds, the start time of trajectory 4 is 2 seconds, and the end time of trajectory 4 is 4 seconds. Then, there is an overlapping time segment between trajectory 3 and trajectory 4 (2 seconds to 3 seconds), and the overlap duration is 3 seconds - 2 seconds = 1 second.

[0135] (2) Spatial proximity

[0136] The spatial proximity of each time-coincident trajectory pair is obtained by determining the Euclidean distance between the starting moments of the vehicles in the two trajectories in the overlapping time segment, the Euclidean distance between the middle moments in the overlapping time segment, and the Euclidean distance between the ending moments in the overlapping time segment.

[0137] For example, at the three time nodes of the start, middle, and end of the overlapping time segment, the Euclidean distances (dist_start, dist_middle, dist_end) between the position coordinates of the vehicles in the two trajectories in the world coordinate system are calculated respectively, and the three Euclidean distances are used as spatial proximity.

[0138] For example, a time-overlapping trajectory pair includes trajectory 3 and trajectory 4. The start time of trajectory 3 is 1 second and the end time of trajectory 3 is 4 seconds. The start time of trajectory 4 is 2 seconds and the end time of trajectory 4 is 5 seconds. Then there is an overlapping time segment (2 seconds to 4 seconds) between trajectory 3 and trajectory 4. The start time, middle time and end time of the overlapping time segment are 2 seconds, 3 seconds and 4 seconds respectively. Then, the Euclidean distance 1 between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 2 seconds, the Euclidean distance 2 between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 3 seconds, and the Euclidean distance 3 between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 4 seconds are determined, and the Euclidean distance 1, Euclidean distance 2 and Euclidean distance 3 are used as spatial proximity.

[0139] Considering that the distances between vehicles at different times are usually far apart and the error is large, the error can be reduced by comparing the Euclidean distances between different vehicles in the same trajectory pair at the same time of the overlapping time segment.

[0140] (3) Speed ​​synchronization

[0141] For example, the difference between the instantaneous speeds of the vehicles in the two trajectories in each time-overlapping trajectory pair at the starting moment of the overlapping time period segment, the difference between the instantaneous speeds at the middle moment of the overlapping time period segment, and the difference between the instantaneous speeds at the ending moment of the overlapping time period segment are determined to obtain the speed synchronization of each time-overlapping trajectory pair.

[0142] For example, a time-overlapping trajectory pair includes trajectory 3 and trajectory 4. The start time of trajectory 3 is 1 second and the end time of trajectory 3 is 4 seconds. The start time of trajectory 4 is 2 seconds and the end time of trajectory 4 is 5 seconds. There is an overlapping time segment (2 seconds to 4 seconds) between trajectory 3 and trajectory 4. The start time, middle time, and end time of the overlapping time segment are 2 seconds, 3 seconds, and 4 seconds, respectively. Then, the absolute value of the instantaneous speed difference 1 (v_diff_start) between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 2 seconds, the absolute value of the instantaneous speed difference 2 (v_diff_middle) between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 3 seconds, and the absolute value of the instantaneous speed difference 3 (v_diff_end) between the vehicle in trajectory 3 and the vehicle in trajectory 4 at 4 seconds are determined. The absolute values ​​of the instantaneous speed difference 1, the absolute values ​​of the instantaneous speed difference 2, and the absolute values ​​of the instantaneous speed difference 3 are used as the speed synchronization degree.

[0143] (4) Spatial overlap

[0144] Specifically, the IOU of the projection areas of the vehicles in the two trajectories of each time-overlapping trajectory pair is determined to obtain the spatial overlap of each time-overlapping trajectory pair. The spatial overlap includes the IOU of the projection area at the starting moment of the overlapping time period segment, the IOU of the projection area at the middle moment of the overlapping time period segment, and the IOU of the projection area at the ending moment of the overlapping time period segment.

[0145] For example, a time-overlapping trajectory pair includes trajectory 3 and trajectory 4. The start time of trajectory 3 is 1 second and the end time of trajectory 3 is 4 seconds. The start time of trajectory 4 is 2 seconds and the end time of trajectory 4 is 5 seconds. There is an overlapping time segment (2 seconds to 4 seconds) between trajectory 3 and trajectory 4. The start time, middle time and end time of the overlapping time segment are 2 seconds, 3 seconds and 4 seconds respectively. Then, the intersection-in-union ratio 1 (iou_start) between the projection area of ​​the vehicle in trajectory 3 at 2 seconds and the projection area of ​​the vehicle in trajectory 4 at 2 seconds, the intersection-in-union ratio 2 (iou_mid) between the projection area of ​​the vehicle in trajectory 3 at 3 seconds and the projection area of ​​the vehicle in trajectory 4 at 3 seconds, and the intersection-in-union ratio 3 (iou_end) between the projection area of ​​the vehicle in trajectory 3 at 4 seconds and the projection area of ​​the vehicle in trajectory 4 at 4 seconds are determined, and the intersection-in-union ratio 1, intersection-in-union ratio 2 and intersection-in-union ratio 3 are used as the spatial overlap degree.

[0146] Considering that a larger IoU between the projection areas of two vehicles in the two trajectories indicates closer vehicle positions in the two trajectories, and a greater probability that the two trajectories originate from the same vehicle, we introduce spatial overlap (the IoU at multiple moments) as a basis for determining whether different trajectories originate from the same vehicle. This increases the probability that two trajectories with larger IoU (closer positions) will be subsequently merged, thereby improving the accuracy of trajectory merging.

[0147] (4) Directional stability (cosθ_overlap)

[0148] Based on the position coordinates of the vehicle in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time of each trajectory in each time-coinciding trajectory pair, the motion direction vector of each trajectory in the overlapping time period is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time-coinciding trajectory pair in the overlapping time period is determined to obtain the directional stability of each time-coinciding trajectory pair.

[0149] Through the above-mentioned method of extracting joint spatiotemporal appearance features, an innovative spatiotemporal trajectory feature system is constructed to implement refined classification and feature modeling of candidate trajectory pairs.

[0150] In one possible implementation, the spatiotemporal appearance joint feature is a 22-dimensional feature vector, where the features for each dimension are [reid_sim, plate_lcs, car1_area, car2_area, area_diff, t_gap, dist_gap, v1, v2, v_diff, cosθ_gap, t_overlap, dist_start, dist_middle, dist_end, v_diff_start, v_diff_middle, v_diff_end, iou_start, iou_mid, iou_end, cosθ_overlap]. The temporal overlap features of time-interval trajectory pairs are automatically padded with zeros (t_gap and dist_gap are padded with zeros), and the time interval features of time-overlap trajectory pairs are automatically padded with zeros, forming an adaptive feature representation mechanism. This design preserves key discriminative information while unifying the dimensionality of the feature vector through zero-padding, providing a standardized input interface for subsequent intelligent matching algorithms.

[0151] In step e, the vehicle appearance similarity, license plate association, spatial scale, and spatiotemporal relationship features of each candidate track pair are combined to obtain a joint spatiotemporal appearance feature for each candidate track pair. This feature extraction method introduces a joint spatiotemporal appearance feature system. By combining vehicle appearance similarity (the similarity between re-identification features), license plate association, spatial scale, and spatiotemporal relationship features as a basis for determining whether to merge different tracks, the system improves the ability to discriminate between track associations.

[0152] In step S170 , each candidate track pair is classified using a classification model based on the spatiotemporal appearance joint features of each candidate track pair to obtain a classification result for each candidate track pair. The classification model is used to predict the probability that the corresponding candidate track pairs originate from the same vehicle.

[0153] The process of classification using a classification model can also be called trajectory inference, which is the process of determining whether different trajectories in a candidate trajectory pair originate from the same vehicle. For example, the joint spatiotemporal appearance features of each candidate trajectory pair in the candidate matching set are input into the classification model. The classification model then performs classification based on these joint spatiotemporal appearance features and outputs a confidence score, which ranges from 0 to 1. The confidence score is then compared with the classification threshold τ. When the confidence score exceeds the classification threshold τ, the trajectory pair is determined to belong to the same vehicle, and the different trajectories in the trajectory pair are merged. When the confidence score is less than or equal to the classification threshold τ, the trajectory pair is determined not to belong to the same vehicle, and the different trajectories in the trajectory pair are not merged.

[0154] Regarding the training process of the classification model, in some embodiments, supervised learning is used to perform model training through the following steps (1) to (3), thereby constructing a high-precision classification model.

[0155] Step (1) Obtain a sample set.

[0156] The sample set includes a plurality of sample trajectory pairs, each of which includes a plurality of first-category sample trajectory pairs and a plurality of second-category sample trajectory pairs. Each of the plurality of first-category sample trajectory pairs originates from the same vehicle. Each of the plurality of first-category sample trajectory pairs includes a positive sample label. Each of the plurality of second-category sample trajectory pairs originates from a different vehicle, and each of the plurality of second-category sample trajectory pairs includes a negative sample label.

[0157] For example, 10,000 sets of data were randomly sampled from a large number of candidate trajectory pairs and manually labeled to form a sample set. The labeling system uses a binary classification standard: if the trajectory pairs originate from the same vehicle, they are labeled as "positive samples"; otherwise, they are labeled as "negative samples."

[0158] Step (2) extracts features from each sample trajectory pair in the sample set based on the vehicle appearance features of each sample trajectory pair, the latitude and longitude data of each sample trajectory pair, the license plate recognition information of each sample trajectory pair, and the start and end time of each sample trajectory pair in the video stream, and obtains the spatiotemporal appearance joint features of each sample trajectory pair.

[0159] The feature extraction process for the sample trajectory pair is similar to the feature extraction process for the candidate trajectory pair described above. Please refer to the description of the feature extraction process for the candidate trajectory pair above.

[0160] Step (3) trains the initial classification model based on the spatiotemporal appearance joint features of each sample trajectory pair and the label of each sample trajectory pair to obtain a classification model.

[0161] For example, using a multidimensional feature vector (joint spatiotemporal appearance features) as the input matrix and binary annotation results (positive or negative sample labels) as the expected output, an initial classification model is trained to learn trajectory association patterns. This training method allows for the mapping from feature space to classification decisions through supervised learning.

[0162] Optionally, the classification model is a decision tree. Since decision trees are faster to train and require less sample data, the time required for model training is saved. Alternatively, the classification model is a neural network model.

[0163] In step S180 , based on the classification results of each candidate trajectory pair, the trajectories in the trajectory pairs originating from the same vehicle are merged to obtain the vehicle driving path.

[0164] For example, the classification results of all candidate trajectory pairs are traversed and constructed into an association matrix. Based on the association matrix, trajectory nodes originating from the same vehicle are clustered into the same subgraph. Each node in the subgraph represents a trajectory. Each subgraph represents the complete spatiotemporal trajectory of a vehicle. This process enables intelligent reconstruction of the global vehicle journey from local trajectory fragments.

[0165] Figure 2 : is a schematic structural diagram of a vehicle traffic data en route device 200 based on 3D continuous trajectory technology provided in an embodiment of the present application, the device 200 comprising:

[0166] Identification module 210, configured to perform vehicle track identification on the video stream captured by each camera of the plurality of cameras, and obtain detection frame parameters for each track in the video stream captured by each camera, vehicle appearance features of each track, and license plate recognition information for each track, wherein the detection frame parameters for each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera, the vehicle appearance features of each track are used to characterize the vehicle appearance features, and the license plate recognition information for each track is used to characterize the license plate number of the vehicle in the corresponding track;

[0167] A coordinate system conversion module 220 is configured to convert the detection frame parameters of each track corresponding to each camera from the camera coordinate system to the world coordinate system based on the calibrated parameters of each camera, thereby obtaining the longitude and latitude data of each track, wherein the longitude and latitude data of each track includes the position coordinates, width, and height of the vehicle in the world coordinate system;

[0168] a matching set construction module 230 for constructing a candidate matching set based on the vehicle appearance features of each trajectory, wherein the candidate matching set includes a plurality of candidate trajectory pairs, each candidate trajectory pair including a trajectory and other trajectories whose similarity to the vehicle appearance features of the trajectory satisfies a constraint;

[0169] A feature extraction module 240 is configured to extract features from each candidate track pair in the candidate matching set based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end times of each track in the video stream, to obtain a spatiotemporal appearance joint feature for each candidate track pair. The spatiotemporal appearance joint feature of each candidate track pair includes the vehicle appearance similarity, license plate association, spatial scale features, and spatiotemporal relationship features of the corresponding candidate track pairs. The spatial scale features are used to characterize the physical spatial occupancy differences of the vehicles in the corresponding candidate track pairs, and the spatiotemporal relationship features are used to characterize the temporal and spatial differences of the corresponding candidate track pairs.

[0170] a classification module 250 for classifying each candidate track pair based on the spatiotemporal appearance joint features of each candidate track pair using a classification model to obtain a classification result for each candidate track pair, wherein the classification model is used to predict the probability that the corresponding candidate track pairs originate from the same vehicle;

[0171] The merging module 260 is configured to merge the trajectories in the trajectory pairs originating from the same vehicle based on the classification results of each candidate trajectory pair to obtain a vehicle driving path.

[0172] In some embodiments, the feature extraction module 240 is configured to determine, based on the vehicle appearance features of each trajectory, the cosine similarity between the vehicle appearance features of the two trajectories in each candidate trajectory pair, to obtain the vehicle appearance similarity of each candidate trajectory pair; determine, based on the license plate recognition information of each trajectory, the length of the longest common substring of the two trajectories in each candidate trajectory pair using a longest common substring algorithm, to obtain the license plate association of each candidate trajectory pair; determine, based on the latitude and longitude data of each trajectory, the projected area of ​​the two vehicles in each candidate trajectory pair in the world coordinate system and the difference between the projected areas, to obtain the spatial scale feature of each candidate trajectory pair; analyze, based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream, the temporal and spatial relationships between the two trajectories in each candidate trajectory pair, to obtain the spatiotemporal relationship feature of each candidate trajectory pair; and concatenate the features belonging to the same candidate trajectory pair from the vehicle appearance similarity of each candidate trajectory pair, the license plate association of each candidate trajectory pair, the spatial scale feature of each candidate trajectory pair, and the spatiotemporal relationship features of each candidate trajectory pair to obtain the spatiotemporal appearance joint feature of each candidate trajectory pair.

[0173] In some embodiments, the multiple candidate trajectory pairs include at least one time interval trajectory pair, the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments, and the spatiotemporal relationship features of each time interval trajectory pair include time interval, spatial distance, kinematic parameters, and directional consistency. The feature extraction module 240 is configured to subtract the start time of the later trajectory from the end time of the earlier trajectory in each time interval trajectory pair to obtain the time interval of each time interval trajectory pair; determine the Euclidean distance between the longitude and latitude data of the two trajectories in each time interval trajectory pair to obtain the spatial distance of each time interval trajectory pair; subtract the instantaneous speed of the later trajectory at the start time from the instantaneous speed of the earlier trajectory at the end time of the two trajectories in each time interval trajectory pair to obtain the kinematic parameters of each time interval trajectory pair; determine the motion direction vector of each trajectory based on the position coordinates of the vehicle in each trajectory in each time interval trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time; and determine the cosine similarity between the motion direction vectors of the two trajectories in each time interval trajectory pair to obtain the directional consistency of each time interval trajectory pair.

[0174] In some embodiments, the multiple candidate trajectory pairs include at least one time-overlapping trajectory pair, and there is an overlapping time segment between the start and end time periods of the two trajectories in each time-overlapping trajectory pair. The spatiotemporal relationship features of each time-overlapping trajectory pair include overlapping duration, spatial proximity, speed synchronization, spatial overlap, and directional stability. The feature extraction module 240 is used to determine the length of the overlapping time segment between the start and end time periods of the two trajectories in each time-overlapping trajectory pair to obtain the overlapping duration of each time-overlapping trajectory pair; determine the Euclidean distance between the starting moments of the vehicles in the two trajectories in each time-overlapping trajectory pair in the overlapping time segment, the Euclidean distance between the middle moments in the overlapping time segment, and the Euclidean distance between the ending moments in the overlapping time segment to obtain the spatial proximity of each time-overlapping trajectory pair; determine the difference between the instantaneous speeds of the vehicles in the two trajectories in each time-overlapping trajectory pair in the overlapping time segment, the difference between the instantaneous speeds of the vehicles in the two trajectories in the overlapping time segment, and the difference between the instantaneous speeds of the vehicles in the two trajectories in the overlapping time segment. The speed synchronization degree of each time-coinciding trajectory pair is obtained by the difference between the instantaneous speeds at the middle moments in the segments and the difference between the instantaneous speeds at the end moments in the overlapping time segments; the intersection-overlapping ratio (IOU) of the projection areas of the vehicles in the two trajectories in each time-coinciding trajectory pair is determined to obtain the spatial overlap degree of each time-coinciding trajectory pair, and the spatial overlap degree includes the intersection-overlapping ratio of the projection area at the starting moment in the overlapping time segment, the intersection-overlapping ratio of the projection area at the middle moment in the overlapping time segment, and the intersection-overlapping ratio of the projection area at the end moment in the overlapping time segment; based on the position coordinates of the vehicle in each trajectory in each time-coinciding trajectory pair in the world coordinate system at the starting moment and the position coordinates of the vehicle in the world coordinate system at the end moment, the motion direction vector of each trajectory in the overlapping time segment is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time-coinciding trajectory pair in the overlapping time segment is determined to obtain the directional stability of each time-coinciding trajectory pair.

[0175] In some embodiments, the candidate matching set includes k candidate trajectory pairs corresponding to the first trajectory. The matching set construction module 230 is configured to construct a cosine similarity index database based on the vehicle appearance features of each trajectory. For the first trajectory, an approximate nearest neighbor search ANN is used to filter the trajectories with the top k cosine similarities to the first trajectory from the cosine similarity index database to obtain k second trajectories. The first trajectory is then combined with each of the k second trajectories to form a candidate trajectory pair, thereby obtaining k candidate trajectory pairs corresponding to the first trajectory.

[0176] In some embodiments, the apparatus 200 further comprises:

[0177] an acquisition module, configured to acquire a sample set, the sample set comprising a plurality of sample trajectory pairs, the plurality of sample trajectory pairs comprising a plurality of first-category sample trajectory pairs and a plurality of second-category sample trajectory pairs, each of the plurality of first-category sample trajectory pairs originating from the same vehicle, each of the plurality of first-category sample trajectory pairs including a positive sample label, each of the plurality of second-category sample trajectory pairs originating from a different vehicle, each of the plurality of second-category sample trajectory pairs including a negative sample label;

[0178] The feature extraction module 240 is further configured to extract features from each sample trajectory pair in the sample set based on the vehicle appearance features of each sample trajectory pair, the latitude and longitude data of each sample trajectory pair, the license plate recognition information of each sample trajectory pair, and the start and end time of each sample trajectory pair in the video stream, to obtain a spatiotemporal appearance joint feature of each sample trajectory pair;

[0179] The apparatus 200 further includes a training module configured to train an initial classification model based on the spatiotemporal appearance joint features of each sample trajectory pair and the label of each sample trajectory pair to obtain the classification model.

[0180] In some embodiments, the calibrated parameters of each camera include an intrinsic parameter matrix and an extrinsic parameter matrix of each camera, and the coordinate system conversion module 220 is used to multiply the detection frame parameters of each trajectory with the intrinsic parameter matrix and the extrinsic parameter matrix of the corresponding camera to obtain the longitude and latitude data of each trajectory.

[0181] In some embodiments, the apparatus 200 further comprises at least one of the following:

[0182] A deletion module is configured to compare, for any trajectory pair in a candidate matching set, the license plate recognition information of the two trajectories in the trajectory pair to obtain a degree of difference in the license plate characters of the two trajectories in the trajectory pair; if the degree of difference in the license plate characters exceeds a character difference threshold, delete the trajectory pair from the candidate matching set; and / or, for any trajectory pair in the candidate matching set, if the time gap between the two trajectories in the trajectory pair exceeds a time gap threshold, delete the trajectory pair from the candidate matching set; and / or, for each time interval trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time interval trajectory pair in the world coordinate system exceeds a first distance threshold, delete the time interval trajectory pair from the candidate matching set, and the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments; and / or, for each time overlap trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time overlap trajectory pair in the world coordinate system in the overlapping time segment exceeds a second distance threshold, delete the time overlap trajectory pair from the candidate matching set, and the start and end time periods of the two trajectories in each time overlap trajectory pair have overlapping time segments.

[0183] In some implementations, the merging module 260 is further configured to merge tracks with the same license plate number based on the license plate recognition information of each track.

[0184] Figure 3 The computer device 600 includes a processor 601 coupled to a memory 602. The memory 602 stores at least one computer program instruction. The at least one computer program instruction is loaded and executed by the processor 601 to enable the computer device 600 to implement Figure 1 The method provided in the embodiment.

[0185] In some embodiments, a computer-readable storage medium is further provided, wherein the storage medium stores at least one instruction, which, when executed on a computer, causes the computer to execute the above-mentioned Figure 1 The method provided in the embodiment.

[0186] In some embodiments, a computer program product is further provided. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed by a computer, the computer performs the above Figure 1 The method provided in the embodiment.

[0187] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0188] A refers to B, which means that A is the same as B or A is a simple variant of B.

[0189] The terms "first" and "second" in the description and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects, and should not be interpreted as indicating or implying relative importance. For example, the terms "first track" and "second track" are used to distinguish different tracks, not to describe a specific order of tracks, and should not be interpreted as implying that the first track is more important than the second track.

[0190] The information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the trajectories involved in this application were all obtained with full authorization.

[0191] In the embodiments of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more. For example, a plurality of trajectories means two or more trajectories.

[0192] The above embodiments can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0193] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle traffic data in-transit method based on 3D continuous trajectory technology, characterized in that: The method comprises: Performing vehicle track recognition on the video stream captured by each camera among the multiple cameras to obtain detection frame parameters of each track in the video stream captured by each camera, vehicle appearance features of each track, and license plate recognition information of each track, wherein the detection frame parameters of each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera, the vehicle appearance features of each track are used to characterize the vehicle appearance features of the vehicle, and the license plate recognition information of each track is used to characterize the license plate number of the vehicle in the corresponding track; Based on the calibrated parameters of each camera, the detection frame parameters of each track corresponding to each camera are converted from the camera coordinate system to the world coordinate system to obtain the longitude and latitude data of each track, where the longitude and latitude data of each track includes the position coordinates, width and height of the vehicle in the world coordinate system; Based on the vehicle appearance features of each trajectory, a candidate matching set is constructed, wherein the candidate matching set includes a plurality of candidate trajectory pairs, each candidate trajectory pair includes a trajectory and other trajectories whose similarity with the vehicle appearance features of the one trajectory satisfies a constraint condition; Based on the vehicle appearance features of each trajectory, the latitude and longitude data of each trajectory, the license plate recognition information of each trajectory, and the start and end times of each trajectory in the video stream, feature extraction is performed on each candidate track pair in the candidate matching set to obtain a spatiotemporal appearance joint feature of each candidate track pair. The spatiotemporal appearance joint feature of each candidate track pair includes the vehicle appearance similarity, license plate association, spatial scale feature, and spatiotemporal relationship feature of the corresponding candidate track pair. The spatial scale feature is used to characterize the physical space occupation difference of the vehicles in the corresponding candidate track pair, and the spatiotemporal relationship feature is used to characterize the temporal and spatial differences of the corresponding candidate track pair. Based on the spatiotemporal appearance joint features of each candidate track pair, classify each candidate track pair using a classification model to obtain a classification result for each candidate track pair, wherein the classification model is used to predict the probability that the corresponding candidate track pairs originate from the same vehicle; Based on the classification result of each candidate trajectory pair, the trajectories in the trajectory pair originating from the same vehicle are merged to obtain the vehicle driving path.

2. The method according to claim 1, characterized in that The method further comprises extracting features of each candidate track pair in the candidate matching set based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end times of each track in the video stream, to obtain a spatiotemporal appearance joint feature of each candidate track pair, including: Determining the cosine similarity between the vehicle appearance features of the two trajectories in each candidate trajectory pair based on the vehicle appearance features of each trajectory, and obtaining the vehicle appearance similarity of each candidate trajectory pair; Based on the license plate recognition information of each trajectory, a longest common substring algorithm is used to determine the length of the longest common substring of the two trajectories in each candidate trajectory pair, and obtain the license plate association degree of each candidate trajectory pair; Based on the latitude and longitude data of each trajectory, determining the projected area of ​​the two vehicles in each candidate trajectory pair in the world coordinate system and the difference between the projected areas, and obtaining the spatial scale feature of each candidate trajectory pair; Based on the latitude and longitude data of each trajectory and the start and end time of each trajectory in the video stream, the temporal relationship and spatial relationship of the two trajectories in each candidate trajectory pair are analyzed to obtain the spatiotemporal relationship characteristics of each candidate trajectory pair; The vehicle appearance similarity of each candidate trajectory pair, the license plate association degree of each candidate trajectory pair, the spatial scale feature of each candidate trajectory pair, and the spatiotemporal relationship feature of each candidate trajectory pair belonging to the same candidate trajectory pair are spliced ​​to obtain the spatiotemporal appearance joint feature of each candidate trajectory pair.

3. The method according to claim 2, characterized in that The multiple candidate trajectory pairs include at least one time interval trajectory pair, the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments, the spatiotemporal relationship features of each time interval trajectory pair include time interval, spatial distance, kinematic parameters, and directional consistency, and the temporal and spatial relationships of the two trajectories in each candidate trajectory pair are analyzed based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream to obtain the spatiotemporal relationship features of each candidate trajectory pair, including: The time interval of each time interval type trajectory pair is obtained by subtracting the start time of the latter trajectory from the end time of the former trajectory in each time interval type trajectory pair; Determine the Euclidean distance between the longitude and latitude data of the two tracks in each time interval track pair to obtain the spatial distance of each time interval track pair; The kinematic parameters of each time interval trajectory pair are obtained by subtracting the instantaneous velocity of the latter trajectory at the start time from the instantaneous velocity of the preceding trajectory at the end time. Based on the position coordinates of the vehicle in each trajectory in each time interval trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time, the motion direction vector of each trajectory is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time interval trajectory pair is determined to obtain the direction consistency of each time interval trajectory pair.

4. The method according to claim 2, characterized in that The multiple candidate trajectory pairs include at least one temporally overlapping trajectory pair, wherein the start and end time periods of the two trajectories in each temporally overlapping trajectory pair have overlapping time segments, and the spatiotemporal relationship features of each temporally overlapping trajectory pair include overlapping duration, spatial proximity, speed synchronization, spatial overlap, and directional stability. Based on the latitude and longitude data of each trajectory and the start and end times of each trajectory in the video stream, the temporal and spatial relationships of the two trajectories in each candidate trajectory pair are analyzed to obtain the spatiotemporal relationship features of each candidate trajectory pair, including: Determine the length of the overlapping time segment between the start and end time periods of the two trajectories in each time-coinciding trajectory pair, and obtain the overlapping duration of each time-coinciding trajectory pair; Determine the Euclidean distance between the starting moments of the vehicles in the two trajectories of each temporally overlapping trajectory pair in the overlapping time segment, the Euclidean distance between the middle moments in the overlapping time segment, and the Euclidean distance between the ending moments in the overlapping time segment, and obtain the spatial proximity of each temporally overlapping trajectory pair; Determine the difference between the instantaneous speeds of the vehicles in the two trajectories in each time-coinciding trajectory pair at the start time of the overlapping time segment, the difference between the instantaneous speeds at the middle time of the overlapping time segment, and the difference between the instantaneous speeds at the end time of the overlapping time segment to obtain the speed synchronization degree of each time-coinciding trajectory pair; Determine the IOU of the projection areas of the vehicles in the two trajectories of each time-coincident trajectory pair, and obtain the spatial overlap of each time-coincident trajectory pair, wherein the spatial overlap includes the IOU of the projection areas at the starting moment of the overlapping time segment, the IOU of the projection areas at the middle moment of the overlapping time segment, and the IOU of the projection areas at the ending moment of the overlapping time segment; Based on the position coordinates of the vehicle in each trajectory of each time-coinciding trajectory pair in the world coordinate system at the start time and the position coordinates of the vehicle in the world coordinate system at the end time, the motion direction vector of each trajectory in the overlapping time period is determined; the cosine similarity between the motion direction vectors of the two trajectories in each time-coinciding trajectory pair in the overlapping time period is determined to obtain the directional stability of each time-coinciding trajectory pair.

5. The method according to claim 1, characterized in that The candidate matching set includes k candidate track pairs corresponding to the first track, and constructing the candidate matching set based on the vehicle appearance feature of each track includes: Building a cosine similarity index database based on the vehicle appearance features of each trajectory; For the first trajectory, an approximate nearest neighbor search artificial neural network (ANN) is used to filter the trajectories with the top k cosine similarities to the first trajectory from the cosine similarity index database to obtain k second trajectories. The first trajectory is then paired with each of the k second trajectories to form a candidate trajectory pair, resulting in k candidate trajectory pairs corresponding to the first trajectory.

6. The method according to claim 1, characterized in that Before classifying each candidate trajectory pair using a classification model based on the spatiotemporal appearance joint features of each candidate trajectory pair to obtain a classification result for each candidate trajectory pair, the method further includes: Acquire a sample set, the sample set comprising a plurality of sample trajectory pairs, the plurality of sample trajectory pairs comprising a plurality of first-category sample trajectory pairs and a plurality of second-category sample trajectory pairs, each of the plurality of first-category sample trajectory pairs originating from the same vehicle, each of the plurality of first-category sample trajectory pairs including a positive sample label, each of the plurality of second-category sample trajectory pairs originating from a different vehicle, each of the plurality of second-category sample trajectory pairs including a negative sample label; Based on the vehicle appearance features of each sample trajectory pair, the latitude and longitude data of each sample trajectory pair, the license plate recognition information of each sample trajectory pair, and the start and end time of each sample trajectory pair in the video stream, feature extraction is performed on each sample trajectory pair in the sample set to obtain a spatiotemporal appearance joint feature of each sample trajectory pair; An initial classification model is trained based on the spatiotemporal appearance joint features of each sample trajectory pair and a label of each sample trajectory pair to obtain the classification model.

7. The method according to claim 1, characterized in that The calibrated parameters of each camera include an intrinsic parameter matrix and an extrinsic parameter matrix of each camera. Based on the calibrated parameters of each camera, the detection frame parameters of each track are converted from the camera coordinate system to the world coordinate system to obtain the longitude and latitude data of each track, including: The detection frame parameters of each track are multiplied by the intrinsic parameter matrix and the extrinsic parameter matrix of the corresponding camera to obtain the longitude and latitude data of each track.

8. The method according to claim 1, characterized in that After constructing a candidate matching set based on the vehicle appearance features of each trajectory, the method further includes at least one of the following: For any track pair in the candidate matching set, compare the license plate recognition information of the two tracks in the track pair to obtain the license plate character difference between the two tracks in the track pair; if the license plate character difference exceeds the character difference threshold, delete the track pair from the candidate matching set; and / or, For any trajectory pair in the candidate matching set, if the time gap between the two trajectories in the trajectory pair exceeds a time gap threshold, the trajectory pair is deleted from the candidate matching set; and / or, For each time interval trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the time interval trajectory pair in the world coordinate system exceeds a first distance threshold, the time interval trajectory pair is deleted from the candidate matching set, and the start and end time periods of the two trajectories in each time interval trajectory pair do not have overlapping time segments; and / or, For each temporally overlapping trajectory pair in the candidate matching set, if the projected distance between the two trajectories in the temporally overlapping trajectory pair in the world coordinate system during the overlapping time segment exceeds a second distance threshold, the temporally overlapping trajectory pair is deleted from the candidate matching set. The start and end time segments of the two trajectories in each temporally overlapping trajectory pair have overlapping time segments.

9. A vehicle traffic data en route device based on 3D continuous trajectory technology, characterized in that: The device comprises: an identification module for identifying vehicle tracks in a video stream captured by each of a plurality of cameras, and obtaining detection frame parameters for each track in the video stream captured by each camera, vehicle appearance features for each track, and license plate recognition information for each track, wherein the detection frame parameters for each track include the position coordinates, width, and height of the vehicle in the camera coordinate system of the corresponding camera, the vehicle appearance features for each track are used to characterize the vehicle appearance features, and the license plate recognition information for each track is used to characterize the license plate number of the vehicle in the corresponding track; A coordinate system conversion module, configured to convert the detection frame parameters of each track corresponding to each camera from the camera coordinate system to the world coordinate system based on the calibrated parameters of each camera, thereby obtaining the longitude and latitude data of each track, wherein the longitude and latitude data of each track includes the position coordinates, width, and height of the vehicle in the world coordinate system; a matching set construction module, configured to construct a candidate matching set based on the vehicle appearance features of each trajectory, wherein the candidate matching set includes a plurality of candidate trajectory pairs, each candidate trajectory pair including a trajectory and other trajectories whose similarity to the vehicle appearance features of the one trajectory satisfies a constraint condition; a feature extraction module configured to extract features from each candidate track pair in the candidate matching set based on the vehicle appearance features of each track, the latitude and longitude data of each track, the license plate recognition information of each track, and the start and end times of each track in the video stream, to obtain a spatiotemporal appearance joint feature for each candidate track pair. The spatiotemporal appearance joint feature of each candidate track pair includes the vehicle appearance similarity, license plate association, spatial scale features, and spatiotemporal relationship features of the corresponding candidate track pairs. The spatial scale features are used to characterize the physical space occupancy differences of the vehicles in the corresponding candidate track pairs, and the spatiotemporal relationship features are used to characterize the temporal and spatial differences of the corresponding candidate track pairs. a classification module, configured to classify each candidate track pair based on the spatiotemporal appearance joint features of each candidate track pair using a classification model to obtain a classification result for each candidate track pair, wherein the classification model is configured to predict the probability that the corresponding candidate track pairs originate from the same vehicle; The merging module is configured to merge the trajectories in the trajectory pairs originating from the same vehicle based on the classification results of each candidate trajectory pair to obtain a vehicle driving path.

10. A computer device, characterized in that: The computer device includes: a processor, the processor is coupled to a memory, the memory stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by the processor, so that the computer device implements the method according to any one of claims 1 to 8.

Images