Vehicle-road cooperation multi-target tracking trajectory fusion method and device

Abstract

The embodiment of the application provides a kind of vehicle-road cooperation multi-target tracking trajectory fusion method and device, the method comprises: obtaining vehicle-road cooperation space-time synchronization frame pair;Whether the two-to-two between vehicle end trajectory is space-time relationship is recorded in vehicle end trajectory relationship table;Whether the two-to-two between road end trajectory is space-time relationship is recorded in road end trajectory relationship table;According to the matching result of vehicle-road cooperation space-time synchronization frame pair target frame, the number of target frame matched between the two-to-two between vehicle end trajectory and road end trajectory is obtained, and recorded in the first vehicle-road trajectory cooperation relationship table;Obtain the mis-matching cooperative trajectory pair that simultaneously space trajectory matching is generated in cooperative trajectory pair, the target frame number corresponding to mis-matching cooperative trajectory pair is 0, obtain the second vehicle-road trajectory cooperation relationship table, the vehicle end trajectory and road end trajectory that exist target frame match are fused, and vehicle-road cooperation multi-target tracking trajectory fusion result is obtained.The embodiment of the application improves the robustness of trajectory matching, and obtains high-quality vehicle-road cooperation trajectory.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a method and apparatus for vehicle-road cooperative multi-target tracking trajectory fusion. Background Technology

[0002] Multi-object tracking involves detecting and assigning IDs to multiple targets such as pedestrians, cars, and animals in a video, and then tracking their trajectories without knowing the number of targets beforehand. Different targets have different IDs to enable subsequent trajectory prediction and accurate location. Multi-object tracking faces challenges inherent in single-object tracking, such as occlusion, deformation, motion blur, crowded scenes, rapid movement, lighting changes, and scale changes. It also addresses complex issues like trajectory initialization and termination, and interference between similar targets. Therefore, multi-object tracking remains a highly challenging area within computer vision (CV).

[0003] Autonomous driving faces significant safety challenges. Single-vehicle intelligence suffers from blind spots and unstable perception at medium to long distances, limiting the operational design domain (ODD) of autonomous vehicles and hindering the deployment of single-vehicle intelligent autonomous driving. Vehicle-to-infrastructure (V2I) cooperation will be the only way to ensure the safe operation of autonomous driving.

[0004] Vehicle-road cooperative multi-target tracking refers to the process of combining vehicle-side tracking information with roadside tracking information to achieve multi-target tracking tasks, thereby ultimately improving the quality of multi-target tracking. How to effectively integrate the tracking trajectory information from the roadside with the vehicle-side tracking, i.e., the matching and fusion of trajectories at both ends of the vehicle-road system, is a key challenge in vehicle-road cooperative multi-target tracking.

[0005] The common trajectory matching method for vehicle-road cooperative multi-target tracking involves performing cooperative matching of target boxes frame by frame, and then fusing the trajectories corresponding to the matched target boxes to obtain the vehicle-road cooperative trajectory fusion result. However, this method is sensitive to outliers. If a pair of outliers (target boxes) is matched and associated, it will cause mismatch and association of the entire trajectory, resulting in poor vehicle-road cooperative trajectory quality. Summary of the Invention

[0006] To address the shortcomings of existing technologies, embodiments of the present invention provide a method and apparatus for vehicle-road cooperative multi-target tracking trajectory fusion.

[0007] This invention provides a vehicle-road cooperative multi-target tracking trajectory fusion method, comprising: performing temporal and spatial synchronization on vehicle-end sequence data and road-end sequence data to obtain a vehicle-road cooperative temporal and spatial synchronization frame pair consisting of temporally and spatially aligned vehicle-end frames and road-end frames; determining whether there is a temporal and spatial relationship between any two vehicle-end trajectories and recording it in a vehicle-end trajectory relationship table; determining whether there is a temporal and spatial relationship between any two road-end trajectories and recording it in a road-end trajectory relationship table; and determining the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories based on the target box matching results of the vehicle-road cooperative temporal and spatial synchronization frame pair and recording it in a first vehicle-road trajectory cooperative relationship table. The vehicle-side trajectory and the road-side trajectory with a target box count greater than 0 constitute a cooperative trajectory pair. Based on the first vehicle-road trajectory cooperative relationship table, the vehicle-side trajectory relationship table, and the road-side trajectory relationship table, mismatched cooperative trajectory pairs that produce simultaneous spatiotemporal trajectory matching are obtained from the cooperative trajectory pairs. The target box count corresponding to the mismatched cooperative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory cooperative relationship table. Based on the target box matching status in the second vehicle-road trajectory cooperative relationship table, the vehicle-side trajectory and the road-side trajectory with target box matching are fused to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0008] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The method includes: obtaining mismatched cooperative trajectory pairs that have simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs according to a first vehicle-road trajectory cooperative relationship table, a vehicle-end trajectory relationship table, and a road-end trajectory relationship table; setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0; and obtaining a second vehicle-road trajectory cooperative relationship table. The method further includes: obtaining a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory according to the first vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table; identifying a cooperative trajectory pair between one road-end trajectory and the vehicle-end trajectory in the road-end trajectory set as a correctly matched cooperative trajectory pair according to the number of target boxes; and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs as 0. The collaborative trajectory pairs between the remaining road-end trajectory and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the third vehicle-road trajectory collaborative relationship table. Based on the third vehicle-road trajectory collaborative relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one of the collaborative trajectory pairs between the vehicle-end trajectory in the set and the road-end trajectory is identified as a correctly matched collaborative trajectory pair. The remaining collaborative trajectory pairs between the vehicle-end trajectory in the set and the road-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the second vehicle-road trajectory collaborative relationship table.

[0009] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The method involves obtaining mismatched cooperative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs based on a first vehicle-road trajectory cooperative relationship table, a vehicle-end trajectory relationship table, and a road-end trajectory relationship table, and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0 to obtain a second vehicle-road trajectory cooperative relationship table. The method further includes: obtaining a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory based on the first vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table; identifying one vehicle-end trajectory in the vehicle-end trajectory set as a correctly matched cooperative trajectory pair based on the number of target boxes; and setting the number of target boxes in the vehicle-end trajectory set as a correctly matched cooperative trajectory pair. The remaining vehicle-end trajectories and the road-end trajectories are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a fourth vehicle-road trajectory collaborative relationship table. Based on the fourth vehicle-road trajectory collaborative relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one of the road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as correctly matched collaborative trajectory pairs. The remaining road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaborative relationship table.

[0010] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The step of identifying a cooperative trajectory pair between a road-end trajectory in the road-end trajectory set and a vehicle trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any road-end trajectory in the road-end trajectory set that has the most target boxes matching the vehicle trajectory, and identifying the cooperative trajectory pair between the road-end trajectory and the vehicle trajectory as a correctly matched cooperative trajectory pair; the step of identifying a cooperative trajectory pair between a vehicle trajectory in the vehicle trajectory set and a road-end trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any vehicle trajectory in the vehicle trajectory set that has the most target boxes matching the road trajectory, and identifying the cooperative trajectory pair between the vehicle trajectory and the road trajectory as a correctly matched cooperative trajectory pair.

[0011] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The step of obtaining whether two vehicle-end trajectories are in a simultaneous spatial-temporal relationship includes: obtaining a first number of times the target bounding boxes of two vehicle-end trajectories appear in the same vehicle-end frame; if the first number reaches a preset first threshold, the corresponding two vehicle-end trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two vehicle-end trajectories are determined not to be in a simultaneous spatial-temporal relationship. The step of obtaining whether two road-end trajectories are in a simultaneous spatial-temporal relationship includes: obtaining a second number of times the target bounding boxes of two road-end trajectories appear in the same road-end frame; if the second number reaches a preset second threshold, the corresponding two road-end trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two road-end trajectories are determined not to be in a simultaneous spatial-temporal relationship.

[0012] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided, wherein the vehicle-side sequence data and the road-side sequence data are online data or offline data.

[0013] This invention also provides a vehicle-road cooperative multi-target tracking trajectory fusion device, comprising: a spatiotemporal synchronization module, used to: perform temporal and spatial synchronization on vehicle-end sequence data and road-end sequence data to obtain a vehicle-road cooperative spatiotemporal synchronization frame pair composed of spatiotemporally aligned vehicle-end frames and road-end frames; a vehicle-end trajectory relationship table acquisition module, used to: acquire whether there is a spatiotemporal relationship between any two vehicle-end trajectories and record it in the vehicle-end trajectory relationship table; a road-end trajectory relationship table acquisition module, used to: acquire whether there is a spatiotemporal relationship between any two road-end trajectories and record it in the road-end trajectory relationship table; and a first vehicle-road trajectory cooperation relationship table acquisition module, used to: acquire the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories based on the target box matching results of the vehicle-road cooperative spatiotemporal synchronization frame pair. The quantity is recorded in the first vehicle-road trajectory coordination relationship table; wherein, the vehicle-end trajectory and the road-end trajectory with a target box number greater than 0 constitute a coordinated trajectory pair; the second vehicle-road trajectory coordination relationship table acquisition module is used to: according to the first vehicle-road trajectory coordination relationship table, the vehicle-end trajectory relationship table and the road-end trajectory relationship table, obtain the mismatched coordinated trajectory pairs that generate simultaneous spatiotemporal trajectory matching in the coordinated trajectory pairs, set the target box number corresponding to the mismatched coordinated trajectory pairs to 0, and obtain the second vehicle-road trajectory coordination relationship table; the trajectory fusion module is used to: according to the target box matching status in the second vehicle-road trajectory coordination relationship table, fuse the vehicle-end trajectory and the road-end trajectory with target box matching to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0014] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the vehicle-road cooperative multi-target tracking trajectory fusion method described above.

[0015] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the vehicle-road cooperative multi-target tracking trajectory fusion method described above.

[0016] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the vehicle-road cooperative multi-target tracking trajectory fusion methods described above.

[0017] The vehicle-road cooperative multi-target tracking trajectory fusion method and apparatus provided in this invention synchronizes vehicle-end sequence data and road-end sequence data in time and space to obtain a vehicle-road cooperative spatiotemporally synchronized frame pair consisting of spatiotemporally aligned vehicle-end frames and road-end frames. It then determines whether each pair of vehicle-end trajectories has a spatiotemporal relationship and records it in a vehicle-end trajectory relationship table. Similarly, it determines whether each pair of road-end trajectories has a spatiotemporal relationship and records it in a road-end trajectory relationship table. Based on the target box matching results of the vehicle-road cooperative spatiotemporally synchronized frame pair, it obtains the number of target boxes matched between each pair of vehicle-end and road-end trajectories and records them in a first vehicle-road trajectory cooperative relationship table. The system generates a road trajectory collaboration relationship table, a vehicle trajectory relationship table, and a road-end trajectory relationship table. It then identifies mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching. The number of target boxes corresponding to these mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaboration relationship table. Based on the target box matching in this table, vehicle-end trajectories and road-end trajectories with matching target boxes are fused to obtain the vehicle-road collaborative multi-target tracking trajectory fusion result. This process removes abnormal matching trajectories, improves the robustness of trajectory matching, and effectively enhances the quality of vehicle-road collaborative multi-target tracking trajectory fusion, resulting in high-quality vehicle-road collaborative trajectories. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the vehicle-road cooperative multi-target tracking trajectory fusion method provided in this embodiment of the invention;

[0020] Figure 2 This is a schematic diagram of the structure of the vehicle-road cooperative multi-target tracking trajectory fusion device provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] Figure 1 This is a flowchart illustrating the vehicle-road cooperative multi-target tracking trajectory fusion method provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0024] Step S1: Perform time and space synchronization on the vehicle-end sequence data and the road-end sequence data to obtain a vehicle-road cooperative time and space synchronized frame pair composed of time-space aligned vehicle-end frames and road-end frames.

[0025] Vehicle-side sequence data consists of sequences of vehicle-side frames collected by onboard equipment, while road-side sequence data consists of sequences of road-side frames collected by road-side equipment. Due to issues such as hardware asynchrony and transmission delay, the collected data at both ends of the vehicle and road exhibit spatiotemporal asynchrony. Therefore, it is necessary to synchronize the collected vehicle-side and road-side sequence data in both time and space to obtain spatiotemporally aligned vehicle-side and road-side frames, forming a vehicle-road cooperative spatiotemporally synchronized frame pair, before performing vehicle-road cooperative processing.

[0026] To achieve temporal and spatial synchronization of vehicle-end sequence data and road-end sequence data, a vehicle-road cooperative temporal and spatial synchronization frame pair is obtained, consisting of spatiotemporally aligned vehicle-end frames and road-end frames. One approach is to find the road-end frame in the road-end sequence data whose time is closest to that of the vehicle-end frame in the vehicle-end sequence data, thus forming a vehicle-road cooperative temporal and spatial synchronization frame pair. However, in finding this pair, a greedy algorithm is used to select the road-end frame with the closest timestamp to the vehicle-end frame as the matching frame pair. This method does not consider the overall temporal order of the target data and the globally optimal matching.

[0027] One method is to perform temporal and spatial synchronization on vehicle-end sequence data and road-end sequence data to obtain vehicle-road cooperative temporal and spatial synchronization frame pairs composed of temporally and spatially aligned vehicle-end frames and road-end frames.

[0028] Based on vehicle-end frames from vehicle-end sequence data and road-end frames from road-end sequence data, the optimization condition is that the time difference between the vehicle-end frames and road-end frames in the vehicle-road cooperative frame pair is less than a preset time difference threshold, and the sum of the timestamp differences of each vehicle-road cooperative frame pair is minimized. A global optimization algorithm is used to obtain a one-to-one matching vehicle-road cooperative frame pair.

[0029] If the timestamps of the vehicle-to-vehicle (V2V) frame and the road-to-vehicle (LTV) frame are different, the V2V frame and the LTV frame are synchronized in time to obtain a V2V time-synchronized frame pair.

[0030] For the corresponding vehicle-road cooperative time synchronization frame pair, based on the same coordinate system, the maximum sum of the number of target boxes matched by the vehicle-end frame and the road-end frame is used as the optimization condition, and the corresponding optimal offset is obtained by using a global optimization algorithm.

[0031] By using the optimal offset, the vehicle-end frames and road-end frames in the corresponding vehicle-road cooperative time synchronization frame pairs are spatially synchronized to obtain vehicle-road cooperative time-space synchronization frame pairs.

[0032] This method takes into account both the temporal and global information of the target, resulting in more accurate and higher-quality acquisition of vehicle-road cooperative spatiotemporal synchronization frame pairs.

[0033] Step S2: Determine whether there is a simultaneous spatial-temporal relationship between any two vehicle-side trajectories and record it in the vehicle-side trajectory relationship table.

[0034] A multi-target tracking algorithm is used to analyze the vehicle-side sequence data to obtain the vehicle-side trajectory analysis results. Each different trajectory includes target boxes arranged in time sequence, and the target boxes in the same trajectory have the same trajectory ID.

[0035] Based on the characteristic that the target boxes of two corresponding targets in two trajectories with simultaneous spatial and temporal relationships will appear in the same vehicle-end frame, we obtain whether there is a simultaneous spatial and temporal relationship between each pair of vehicle-end trajectories, and record the results of whether there is a simultaneous spatial and temporal relationship between each pair of vehicle-end trajectories in the vehicle-end trajectory relationship table.

[0036] Step S3: Determine whether there is a simultaneous spatiotemporal relationship between any two road end trajectories and record it in the road end trajectory relationship table.

[0037] A multi-target tracking algorithm is used to analyze roadside sequence data to obtain the trajectory analysis results of the roadside. Each different trajectory includes target boxes arranged in time sequence, and the target boxes in the same trajectory have the same trajectory ID.

[0038] Based on the characteristic that the target boxes of two corresponding targets in two trajectories with simultaneous spatial and temporal relationships will appear in the same road end frame, we obtain whether there is a simultaneous spatial and temporal relationship between each pair of road end trajectories, and record the results of whether there is a simultaneous spatial and temporal relationship between each pair of road end trajectories in the road end trajectory relationship table.

[0039] It should be noted that step S3 can be performed before step S2, or simultaneously with step S2.

[0040] Step S4: Based on the target box matching results of the vehicle-road cooperative spatiotemporal synchronization frame pair, obtain the number of target boxes that match between each pair of vehicle-end trajectory and road-end trajectory, and record them in the first vehicle-road trajectory cooperative relationship table; wherein, the vehicle-end trajectory and the road-end trajectory with a target box number greater than 0 constitute a cooperative trajectory pair.

[0041] The vehicle-road cooperative spatiotemporal synchronization frame pair includes a one-to-one matched spatiotemporally aligned vehicle-side frame and road-side frame. A vehicle-side frame may contain multiple bounding boxes, and a road-side frame may also contain multiple bounding boxes. Each bounding box has a trajectory ID. When obtaining the number of matched bounding boxes between vehicle-side and road-side trajectories, the matching results of bounding boxes with the corresponding vehicle-side trajectory ID and road-side trajectory ID in the vehicle-road cooperative spatiotemporal synchronization frame pair are obtained based on the vehicle-side trajectory ID and road-side trajectory ID. If a match is found, the number of matched bounding boxes is incremented by 1. After determining whether the bounding boxes with the corresponding vehicle-side trajectory ID and road-side trajectory ID in each vehicle-road cooperative spatiotemporal synchronization frame pair match, the number of matched bounding boxes between the corresponding vehicle-side and road-side trajectories is obtained, and this number is recorded in the first vehicle-road trajectory cooperative relationship table.

[0042] Vehicle-side trajectories and road-side trajectories with a target box count greater than 0 indicate that they have at least one matched target box, meaning that the corresponding vehicle-side trajectory and road-side trajectory have a matching relationship and constitute a cooperative trajectory pair.

[0043] Step S5: Based on the first vehicle-road trajectory coordination relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, obtain the mismatched coordinated trajectory pairs that generate simultaneous spatiotemporal trajectory matching in the coordinated trajectory pairs, set the number of target boxes corresponding to the mismatched coordinated trajectory pairs to 0, and obtain the second vehicle-road trajectory coordination relationship table.

[0044] According to the principle of spatiotemporal mutual exclusion, two trajectories that are simultaneously spatiotemporally related cannot be associated. For example, two road-side trajectories, Target 1 trajectory TrB1 and Target 2 trajectory TrB2, are simultaneously spatiotemporally related. The corresponding vehicle-side trajectories are Target 1 trajectory TrA1 and Target 2 trajectory TrA2, which are also simultaneously spatiotemporally related. That is, TrB1 and TrB2 cannot be associated through a single vehicle-side trajectory. Suppose that at a certain moment, TrA2 is absent due to occlusion. The target box matching algorithm causes the target boxes of TrA1 and TrB2 to coordinate at this moment, meaning TrB2 is associated with TrA1, and TrA1 is associated with TrB1. Thus, the coordination of target boxes at this moment causes TrB2 to be associated with TrB1. However, since TrB1 and TrB2 are simultaneously spatiotemporally related and cannot be associated, this is considered a mismatched coordinated trajectory pair and should be corrected.

[0045] The matching status of vehicle-end trajectories and road-end trajectories can be obtained from the first vehicle-road trajectory coordination relationship table. Vehicle-end trajectories with simultaneous spatial and temporal relationships can be obtained from the vehicle-end trajectory relationship table, and road-end trajectories with simultaneous spatial and temporal relationships can be obtained from the road-end trajectory relationship table. If a vehicle-end trajectory obtained from the first vehicle-road trajectory coordination relationship table simultaneously matches a road-end trajectory with a simultaneous spatial and temporal relationship, or vice versa, then a mismatched coordinated trajectory pair resulting in simultaneous spatial and temporal trajectory matching has occurred. It is necessary to remove these mismatched coordinated trajectory pairs (set the number of bounding boxes corresponding to the mismatched coordinated trajectory pair to 0, indicating that the two trajectories no longer match) to avoid simultaneous spatial and temporal trajectory matching.

[0046] The acquisition of mismatched cooperative trajectory pairs can be result-oriented, that is, after removing the corresponding mismatched cooperative trajectory pairs, no more spatiotemporal trajectory matching will be generated.

[0047] After the above corrections, the second vehicle-road trajectory coordination relationship table is obtained.

[0048] Step S6: Based on the target box matching status in the second vehicle-road trajectory coordination relationship table, fuse the vehicle-end trajectory and road-end trajectory with target box matching to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0049] Based on the target box matching in the second vehicle-road trajectory coordination relationship table, vehicle-end trajectories and road-end trajectories with matching target boxes (where the number of target boxes in a coordinated trajectory pair is greater than 0) are fused to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result. Specifically, when fusing vehicle-end trajectories and road-end trajectories with matching target boxes, for a target box of a certain target, if both vehicle-end and road-end trajectory target boxes exist in the vehicle-road cooperative spatiotemporal synchronization frame pair, one of the target boxes is retained, or the target boxes of the vehicle-end trajectory and the road-end trajectory are integrated to obtain a single target box (the size and position of the target box can be set). The fused vehicle-road trajectory is formed from the finally determined target boxes based on each vehicle-road cooperative spatiotemporal synchronization frame pair.

[0050] Among them, the criteria for determining the target box matching can be predefined. For example, the overlap between the vehicle-side target box and the road-side target box can be used to determine whether the target boxes match.

[0051] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this invention synchronizes vehicle-end sequence data and road-end sequence data in time and space to obtain a vehicle-road cooperative spatiotemporally synchronized frame pair consisting of spatiotemporally aligned vehicle-end frames and road-end frames. It then determines whether each pair of vehicle-end trajectories has a spatiotemporal relationship and records it in a vehicle-end trajectory relationship table. Similarly, it determines whether each pair of road-end trajectories has a spatiotemporal relationship and records it in a road-end trajectory relationship table. Based on the target box matching results of the vehicle-road cooperative spatiotemporally synchronized frame pair, it obtains the number of target boxes matched between each pair of vehicle-end and road-end trajectories and records them in a first vehicle-road trajectory cooperative relationship table. Finally, it determines the number of target boxes matched between each pair of vehicle-end and road-end trajectories based on the first vehicle-road cooperative relationship table. The system generates a trajectory collaboration relationship table, a vehicle-side trajectory relationship table, and a road-side trajectory relationship table. It then identifies mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching. The number of target boxes corresponding to these mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaboration relationship table. Based on the target box matching in this table, vehicle-side and road-side trajectories with matching target boxes are fused to obtain the vehicle-road collaborative multi-target tracking trajectory fusion result. This process removes abnormal matching trajectories, improves the robustness of trajectory matching, and effectively enhances the quality of vehicle-road collaborative multi-target tracking trajectory fusion, resulting in high-quality vehicle-road collaborative trajectories.

[0052] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The method includes: obtaining mismatched cooperative trajectory pairs that have simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs according to a first vehicle-road trajectory cooperative relationship table, a vehicle-end trajectory relationship table, and a road-end trajectory relationship table; setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0; and obtaining a second vehicle-road trajectory cooperative relationship table. The method further includes: obtaining a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory according to the first vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table; identifying a cooperative trajectory pair between one road-end trajectory and the vehicle-end trajectory in the road-end trajectory set as a correctly matched cooperative trajectory pair according to the number of target boxes; and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs as 0. The collaborative trajectory pairs between the remaining road-end trajectory and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the third vehicle-road trajectory collaborative relationship table. Based on the third vehicle-road trajectory collaborative relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one of the collaborative trajectory pairs between the vehicle-end trajectory in the set and the road-end trajectory is identified as a correctly matched collaborative trajectory pair. The remaining collaborative trajectory pairs between the vehicle-end trajectory in the set and the road-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the second vehicle-road trajectory collaborative relationship table.

[0053] In this embodiment of the invention, firstly, based on the first vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table, each vehicle-end trajectory is analyzed sequentially. By matching the target boxes of the vehicle-end trajectories with each road-end trajectory, mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching are obtained from the collaborative trajectory pairs. Since the results of whether each pair of road-end trajectories exhibits simultaneous spatiotemporal relationships in the road-end trajectory relationship table and the vehicle-end trajectory relationship table may not completely correspond, to ensure the comprehensiveness of the obtained mismatched collaborative trajectory pairs and improve the removal effect of mismatched trajectories, after obtaining the mismatched collaborative trajectory pairs based on the first vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table, each road-end trajectory is analyzed sequentially based on the updated vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table. By matching the target boxes of the road-end trajectories with each vehicle-end trajectory, mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching are obtained from the collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in the second vehicle-road trajectory coordination relationship table.

[0054] Based on the first vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships and target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one road-end trajectory and vehicle-end trajectory pair in the road-end trajectory set is identified as a correctly matched pair, while the remaining road-end trajectory and vehicle-end trajectory pairs are identified as incorrectly matched pairs. The number of target boxes corresponding to the incorrectly matched pairs is set to 0, resulting in the third vehicle-road trajectory coordination relationship table. A correctly matched pair can be any road-end trajectory and vehicle-end trajectory pair in the road-end trajectory set. By retaining only correctly matched pairs and setting the number of target boxes for the remaining road-end trajectory and vehicle-end trajectory pairs to 0 (i.e., removing the matching relationship), it is ensured that matching between vehicle-end trajectories will not result in simultaneous spatiotemporal matching, based on whether there is a simultaneous spatiotemporal relationship between any two road-end trajectories provided in the road-end trajectory relationship table.

[0055] Based on the third vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships and target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one vehicle-end trajectory and one road-end trajectory in the vehicle-end trajectory set are identified as correctly matched coordinated trajectory pairs, while the remaining vehicle-end trajectory and road-end trajectory pairs are identified as incorrectly matched coordinated trajectory pairs. The number of target boxes corresponding to the incorrectly matched coordinated trajectory pairs is set to 0, resulting in the second vehicle-road trajectory coordination relationship table. A correctly matched coordinated trajectory pair can be any vehicle-end trajectory and one road-end trajectory in the vehicle-end trajectory set. By retaining only correctly matched coordinated trajectory pairs and setting the number of target boxes for the remaining vehicle-end trajectory and road-end trajectory pairs to 0 (i.e., removing the matching relationship), it is ensured that matching vehicle-end trajectories with road-end trajectories will not result in simultaneous spatiotemporal matching, based on whether there is a simultaneous spatiotemporal relationship between any two vehicle-end trajectories provided in the vehicle-end trajectory relationship table.

[0056] After the above processing, the target box matching of vehicle-road trajectories, the spatiotemporal relationship between vehicle-end trajectories and the spatiotemporal relationship between road-end trajectories are combined to identify mismatched cooperative trajectory pairs, and the corresponding target box count is set to 0. The resulting second vehicle-road trajectory cooperative relationship table no longer contains spatiotemporal trajectory matching.

[0057] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this embodiment of the invention first removes mismatched trajectories according to the first vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table to obtain the third vehicle-road trajectory cooperative relationship table, and then removes mismatched trajectories according to the third vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table, thereby achieving accurate acquisition of the second vehicle-road trajectory cooperative relationship table.

[0058] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The method involves obtaining mismatched cooperative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs based on a first vehicle-road trajectory cooperative relationship table, a vehicle-end trajectory relationship table, and a road-end trajectory relationship table, and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0 to obtain a second vehicle-road trajectory cooperative relationship table. The method further includes: obtaining a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory based on the first vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table; identifying one vehicle-end trajectory in the vehicle-end trajectory set as a correctly matched cooperative trajectory pair based on the number of target boxes; and setting the number of target boxes in the vehicle-end trajectory set as a correctly matched cooperative trajectory pair. The remaining vehicle-end trajectories and the road-end trajectories are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a fourth vehicle-road trajectory collaborative relationship table. Based on the fourth vehicle-road trajectory collaborative relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one of the road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as correctly matched collaborative trajectory pairs. The remaining road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaborative relationship table.

[0059] In this embodiment of the invention, firstly, based on the first vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table, each road-end trajectory is analyzed sequentially. By matching the target boxes of the road-end trajectories with each vehicle-end trajectory, mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching are obtained from the collaborative trajectory pairs. Since the results of whether each pair of vehicle-end trajectories exhibits simultaneous spatiotemporal relationships in the vehicle-end trajectory relationship table and the results of whether each pair of road-end trajectories exhibits simultaneous spatiotemporal relationships in the road-end trajectory relationship table may not completely correspond, to ensure the comprehensiveness of the obtained mismatched collaborative trajectory pairs and improve the removal effect of mismatched trajectories, after obtaining the mismatched collaborative trajectory pairs based on the first vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table, each vehicle-end trajectory is analyzed sequentially based on the updated vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table. By matching the target boxes of the vehicle-end trajectories with each road-end trajectory, mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching are obtained from the collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in the second vehicle-road trajectory coordination relationship table.

[0060] Based on the first vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships and target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one vehicle-end trajectory and one road-end trajectory in the vehicle-end trajectory set are identified as correctly matched coordinated trajectory pairs, while the remaining vehicle-end trajectory and road-end trajectory pairs are identified as incorrectly matched coordinated trajectory pairs. The number of target boxes corresponding to the incorrectly matched coordinated trajectory pairs is set to 0, resulting in the fourth vehicle-road trajectory coordination relationship table. A correctly matched coordinated trajectory pair can be any vehicle-end trajectory and one road-end trajectory in the vehicle-end trajectory set. By retaining only correctly matched coordinated trajectory pairs and setting the number of target boxes for the remaining vehicle-end trajectory and road-end trajectory pairs to 0 (i.e., removing the matching relationship), it is ensured that matching vehicle-end trajectories with road-end trajectories will not result in simultaneous spatiotemporal matching, based on whether there is a simultaneous spatiotemporal relationship between any two vehicle-end trajectories provided in the vehicle-end trajectory relationship table.

[0061] Based on the fourth vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships and target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one road-end trajectory and vehicle-end trajectory pair in the road-end trajectory set is identified as a correctly matched pair, while the remaining road-end trajectory and vehicle-end trajectory pairs are identified as incorrectly matched pairs. The number of target boxes corresponding to the incorrectly matched pairs is set to 0, resulting in the second vehicle-road trajectory coordination relationship table. A correctly matched pair can be any road-end trajectory and vehicle-end trajectory pair in the road-end trajectory set. By retaining only correctly matched pairs and setting the number of target boxes for the remaining road-end trajectory and vehicle-end trajectory pairs to 0 (i.e., removing the matching relationship), it is ensured that matching between vehicle-end trajectories will not result in simultaneous spatiotemporal matching, based on whether there is a simultaneous spatiotemporal relationship between any two road-end trajectories provided in the road-end trajectory relationship table.

[0062] After the above processing, the target box matching of vehicle-road trajectories, the spatiotemporal relationship between vehicle-end trajectories and the spatiotemporal relationship between road-end trajectories are combined to identify mismatched cooperative trajectory pairs, and the corresponding target box count is set to 0. The resulting second vehicle-road trajectory cooperative relationship table no longer contains spatiotemporal trajectory matching.

[0063] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this embodiment of the invention first removes mismatched trajectories according to the first vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table to obtain the fourth vehicle-road trajectory cooperative relationship table, and then removes mismatched trajectories according to the fourth vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table to obtain the second vehicle-road trajectory cooperative relationship table, thereby achieving accurate acquisition of the second vehicle-road trajectory cooperative relationship table.

[0064] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The step of identifying a cooperative trajectory pair between a road-end trajectory in the road-end trajectory set and a vehicle trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any road-end trajectory in the road-end trajectory set that has the most target boxes matching the vehicle trajectory, and identifying the cooperative trajectory pair between the road-end trajectory and the vehicle trajectory as a correctly matched cooperative trajectory pair; the step of identifying a cooperative trajectory pair between a vehicle trajectory in the vehicle trajectory set and a road-end trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any vehicle trajectory in the vehicle trajectory set that has the most target boxes matching the road trajectory, and identifying the cooperative trajectory pair between the vehicle trajectory and the road trajectory as a correctly matched cooperative trajectory pair.

[0065] Since the more bounding boxes that match between the vehicle trajectory and the road trajectory, the higher the confidence that the vehicle trajectory and the road trajectory form a correctly matched collaborative trajectory pair, when identifying a collaborative trajectory pair between a road trajectory and a vehicle trajectory in the road trajectory set as a correctly matched collaborative trajectory pair based on the number of bounding boxes, the road trajectory with the most bounding boxes that match the vehicle trajectory in the road trajectory set is selected, and its collaborative trajectory pair with the vehicle trajectory is identified as a correctly matched collaborative trajectory pair. Specifically, if there is only one road trajectory in the road trajectory set with the most bounding boxes that match the vehicle trajectory, then the collaborative trajectory pair between that road trajectory and the vehicle trajectory is identified as a correctly matched collaborative trajectory pair; if there are two or more road trajectories in the road trajectory set with the most bounding boxes that match the vehicle trajectory (with the same number of bounding boxes), then any one of these road trajectory pairs with the vehicle trajectory is identified as a correctly matched collaborative trajectory pair.

[0066] For the same reason, when determining a correctly matched collaborative trajectory pair between a vehicle trajectory and a road trajectory in the vehicle trajectory set based on the number of target boxes, the vehicle trajectory with the most target boxes matching the road trajectory in the vehicle trajectory set is selected, and its collaborative trajectory pair with the road trajectory is determined as a correctly matched collaborative trajectory pair. Specifically, if there is only one vehicle trajectory in the vehicle trajectory set with the most target boxes matching the road trajectory, then that collaborative trajectory pair with the road trajectory is determined as a correctly matched collaborative trajectory pair; if there are two or more vehicle trajectories in the vehicle trajectory set with the most target boxes matching the road trajectory (with the same number of target boxes), then any one of these vehicle trajectory pairs with the road trajectory is determined as a correctly matched collaborative trajectory pair.

[0067] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this invention improves the accuracy of obtaining correctly matched cooperative trajectory pairs by acquiring any road-end trajectory in the road-end trajectory set that matches the vehicle trajectory with the most target boxes, based on the number of target boxes, and identifying the cooperative trajectory pair between the road-end trajectory and the vehicle trajectory as a correctly matched cooperative trajectory pair.

[0068] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided. The step of obtaining whether two vehicle-end trajectories are in a simultaneous spatial-temporal relationship includes: obtaining a first number of times the target bounding boxes of two vehicle-end trajectories appear in the same vehicle-end frame; if the first number reaches a preset first threshold, the corresponding two vehicle-end trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two vehicle-end trajectories are determined not to be in a simultaneous spatial-temporal relationship. The step of obtaining whether two road-end trajectories are in a simultaneous spatial-temporal relationship includes: obtaining a second number of times the target bounding boxes of two road-end trajectories appear in the same road-end frame; if the second number reaches a preset second threshold, the corresponding two road-end trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two road-end trajectories are determined not to be in a simultaneous spatial-temporal relationship.

[0069] A vehicle-side frame is a snapshot of a specific space captured by a vehicle-side device at a specific time. If the target bounding boxes of two vehicle-side trajectories appear simultaneously in the same vehicle-side frame, it indicates that the targets of the two trajectories are in a simultaneous spatial-temporal relationship when the corresponding vehicle-side frame was captured. A simultaneous spatial-temporal relationship between two vehicle-side trajectories actually means that the targets corresponding to the two vehicle-side trajectories are in a simultaneous spatial-temporal relationship. Since the target bounding boxes are obtained based on the analysis of the tracking algorithm, there may be false positives. Therefore, when determining whether two vehicle-side trajectories are in a simultaneous spatial-temporal relationship, the number of times the target bounding boxes of the two trajectories appear in the same vehicle-side frame is used. Specifically, the first number of times the target bounding boxes of two vehicle-side trajectories appear in the same vehicle-side frame is obtained. If the first count reaches a preset first threshold, the two vehicle-side trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the two vehicle-side trajectories are determined not to be in a simultaneous spatial-temporal relationship.

[0070] A roadside frame is a snapshot of a specific space captured by roadside equipment at a specific time. If the target bounding boxes of two roadside trajectories appear simultaneously in the same roadside frame, it indicates that the targets of the two trajectories were in a simultaneous spatial-temporal relationship when the corresponding roadside frame was captured. A simultaneous spatial-temporal relationship between two roadside trajectories actually means that the targets corresponding to the two roadside trajectories are in a simultaneous spatial-temporal relationship. Since the target bounding boxes are obtained based on the analysis of the tracking algorithm, there may be misjudgments. Therefore, when determining whether two roadside trajectories are in a simultaneous spatial-temporal relationship, the number of times the target bounding boxes of the two trajectories appear in the same roadside frame is used. Specifically, the second number of times the target bounding boxes of two roadside trajectories appear in the same roadside frame is obtained. If the second number reaches a preset second threshold, the corresponding two roadside trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two roadside trajectories are determined not to be in a simultaneous spatial-temporal relationship.

[0071] The first threshold and the second threshold can be the same or different.

[0072] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this invention improves the accuracy of determining whether vehicle-end trajectories and road-end trajectories are in a simultaneous spatial-temporal relationship by obtaining the first number of times the target boxes of two vehicle-end trajectories appear in the same vehicle-end frame. If the first number reaches a preset first threshold, the two vehicle-end trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, they are determined not to be in a simultaneous spatial-temporal relationship.

[0073] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion method is provided, wherein the vehicle-side sequence data and the road-side sequence data are online data or offline data.

[0074] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this invention can be applied to both online and offline scenarios.

[0075] In online scenarios, this technology enables online tracking of multiple targets and vehicle-road trajectory fusion. Both vehicle-side and road-side sequence data are online data; the online scenario essentially involves performing trajectory collaborative analysis by looking back from the current moment.

[0076] When applied to offline scenarios, it enables offline tracking of multiple targets and vehicle-road trajectory fusion. Vehicle-side sequence data and road-side sequence data are offline data. The offline scenario is equivalent to performing trajectory collaborative analysis by looking back from the last moment of a time segment.

[0077] The vehicle-road cooperative multi-target tracking trajectory fusion method provided in this invention uses online or offline data for vehicle-end sequence data and road-end sequence data, thereby realizing online or offline vehicle-road cooperative multi-target tracking trajectory fusion and improving flexibility.

[0078] It should be noted that the preferred embodiments given in this example can be freely combined, provided that there is no logical or structural conflict between them, and the present invention does not limit them.

[0079] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in the embodiments of the present invention is described below. The vehicle-road cooperative multi-target tracking trajectory fusion device described below and the vehicle-road cooperative multi-target tracking trajectory fusion method described above can be referred to in correspondence with each other.

[0080] Figure 2 This is a schematic diagram of the structure of the vehicle-road cooperative multi-target tracking trajectory fusion device provided in an embodiment of the present invention. Figure 2 As shown, the device includes: a spatiotemporal synchronization module 10 for: performing time and space synchronization on vehicle-end sequence data and road-end sequence data to obtain a vehicle-road cooperative spatiotemporal synchronization frame pair composed of spatiotemporally aligned vehicle-end frames and road-end frames; a vehicle-end trajectory relationship table acquisition module 20 for: obtaining whether there is a spatiotemporal relationship between any two vehicle-end trajectories and recording it in the vehicle-end trajectory relationship table; a road-end trajectory relationship table acquisition module 30 for: obtaining whether there is a spatiotemporal relationship between any two road-end trajectories and recording it in the road-end trajectory relationship table; and a first vehicle-road trajectory cooperative relationship table acquisition module 40 for: obtaining the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories based on the target box matching results of the vehicle-road cooperative spatiotemporal synchronization frame pair and recording it in the first vehicle-road trajectory relationship table. The system comprises: a track coordination relationship table; wherein, the vehicle-end trajectory and the road-end trajectory with a target box count greater than 0 constitute a coordinated trajectory pair; the second vehicle-road trajectory coordination relationship table acquisition module 50 is used to: obtain mismatched coordinated trajectory pairs that generate simultaneous spatiotemporal trajectory matching in the coordinated trajectory pairs according to the first vehicle-road trajectory coordination relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, and set the target box count corresponding to the mismatched coordinated trajectory pair to 0, thereby obtaining the second vehicle-road trajectory coordination relationship table; the trajectory fusion module 60 is used to: fuse the vehicle-end trajectory and the road-end trajectory with target box matching according to the target box matching status in the second vehicle-road trajectory coordination relationship table, thereby obtaining the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0081] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention synchronizes vehicle-end sequence data and road-end sequence data in time and space to obtain a vehicle-road cooperative spatiotemporally synchronized frame pair consisting of spatiotemporally aligned vehicle-end frames and road-end frames. It then determines whether there is a spatiotemporal relationship between any two vehicle-end trajectories and records this information in a vehicle-end trajectory relationship table. Similarly, it determines whether there is a spatiotemporal relationship between any two road-end trajectories and records this information in a road-end trajectory relationship table. Based on the target box matching results of the vehicle-road cooperative spatiotemporally synchronized frame pair, it obtains the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories and records this information in a first vehicle-road trajectory cooperative relationship table. Finally, it determines the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories based on the first vehicle-road cooperative relationship table. The system generates a trajectory collaboration relationship table, a vehicle-side trajectory relationship table, and a road-side trajectory relationship table. It then identifies mismatched collaborative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching. The number of target boxes corresponding to these mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaboration relationship table. Based on the target box matching in this table, vehicle-side and road-side trajectories with matching target boxes are fused to obtain the vehicle-road collaborative multi-target tracking trajectory fusion result. This process removes abnormal matching trajectories, improves the robustness of trajectory matching, and effectively enhances the quality of vehicle-road collaborative multi-target tracking trajectory fusion, resulting in high-quality vehicle-road collaborative trajectories.

[0082] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion device includes a second vehicle-road trajectory cooperative relationship table acquisition module 50. Specifically, when acquiring mismatched cooperative trajectory pairs that produce simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs based on the first vehicle-road trajectory cooperative relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0, the module 50 is configured to: acquire a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory based on the first vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table; and, based on the number of target boxes, identify one of the cooperative trajectory pairs between the road-end trajectory and the vehicle-end trajectory in the road-end trajectory set as a correctly matched cooperative trajectory pair. The remaining road-end trajectories in the road-end trajectory set and the collaborative trajectory pairs with the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the third vehicle-road trajectory collaborative relationship table. Based on the third vehicle-road trajectory collaborative relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one of the collaborative trajectory pairs between the vehicle-end trajectory set and the road-end trajectory is identified as a correctly matched collaborative trajectory pair. The remaining collaborative trajectory pairs between the vehicle-end trajectory set and the road-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the second vehicle-road trajectory collaborative relationship table.

[0083] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention first removes mismatched trajectories according to the first vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table to obtain the third vehicle-road trajectory cooperative relationship table, and then removes mismatched trajectories according to the third vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table, thereby achieving accurate acquisition of the second vehicle-road trajectory cooperative relationship table.

[0084] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion device includes a second vehicle-road trajectory cooperative relationship table acquisition module 50. Specifically, when acquiring mismatched cooperative trajectory pairs that exhibit simultaneous spatiotemporal trajectory matching from the cooperative trajectory pairs based on the first vehicle-road trajectory cooperative relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, and setting the number of target boxes corresponding to the mismatched cooperative trajectory pairs to 0, the module 50 obtains the second vehicle-road trajectory cooperative relationship table by: acquiring a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory based on the first vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table; and identifying a cooperative trajectory pair between one vehicle-end trajectory and the road-end trajectory in the vehicle-end trajectory set as a correctly matched cooperative trajectory pair based on the number of target boxes. The remaining vehicle-end trajectory pairs in the vehicle-end trajectory set and the road-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a fourth vehicle-road trajectory collaborative relationship table. Based on the fourth vehicle-road trajectory collaborative relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one of the collaborative trajectory pairs between the road-end trajectory in the road-end trajectory set and the vehicle-end trajectory is identified as a correctly matched collaborative trajectory pair. The remaining collaborative trajectory pairs between the road-end trajectory in the road-end trajectory set and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, resulting in a second vehicle-road trajectory collaborative relationship table.

[0085] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention first removes mismatched trajectories according to the first vehicle-road trajectory cooperative relationship table and the vehicle-end trajectory relationship table to obtain the fourth vehicle-road trajectory cooperative relationship table, and then removes mismatched trajectories according to the fourth vehicle-road trajectory cooperative relationship table and the road-end trajectory relationship table to obtain the second vehicle-road trajectory cooperative relationship table, thereby achieving accurate acquisition of the second vehicle-road trajectory cooperative relationship table.

[0086] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion device is provided. When the second vehicle-road trajectory cooperative relationship table acquisition module 50 is used to identify a cooperative trajectory pair between one of the road-end trajectories in the road-end trajectory set and the vehicle-end trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes, it is specifically used to: acquire any road-end trajectory in the road-end trajectory set that has the most target boxes matching the vehicle-end trajectory based on the number of target boxes, and identify the cooperative trajectory pair between the road-end trajectory and the vehicle-end trajectory as a correctly matched cooperative trajectory pair; when the second vehicle-road trajectory cooperative relationship table acquisition module 50 is used to identify a cooperative trajectory pair between one of the vehicle-end trajectories in the vehicle-end trajectory set and the road-end trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes, it is specifically used to: acquire any vehicle-end trajectory in the vehicle-end trajectory set that has the most target boxes matching the road-end trajectory based on the number of target boxes, and identify the cooperative trajectory pair between the vehicle-end trajectory and the road-end trajectory as a correctly matched cooperative trajectory pair.

[0087] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention obtains any road-end trajectory in the road-end trajectory set that matches the vehicle-end trajectory with the most target boxes based on the number of target boxes, and identifies the cooperative trajectory pair between the road-end trajectory and the vehicle-end trajectory as a correctly matched cooperative trajectory pair. Similarly, it obtains any vehicle-end trajectory in the vehicle-end trajectory set that matches the road-end trajectory with the most target boxes based on the number of target boxes, and identifies the cooperative trajectory pair between the vehicle-end trajectory and the road-end trajectory as a correctly matched cooperative trajectory pair. This improves the accuracy of obtaining correctly matched cooperative trajectory pairs and is conducive to further improving the quality of vehicle-road cooperative multi-target tracking trajectory fusion.

[0088] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion device is provided. The vehicle-end trajectory relationship table acquisition module 20, when determining whether there is a simultaneous spatiotemporal relationship between two vehicle-end trajectories, specifically acquires: a first number of times the target boxes of two vehicle-end trajectories appear in the same vehicle-end frame; if the first number reaches a preset first threshold, it determines that the corresponding two vehicle-end trajectories are simultaneously spatiotemporally related; otherwise, it determines that the corresponding two vehicle-end trajectories are not simultaneously spatiotemporally related. The road-end trajectory relationship table acquisition module 30, when determining whether there is a simultaneous spatiotemporal relationship between two road-end trajectories, specifically acquires: a second number of times the target boxes of two road-end trajectories appear in the same road-end frame; if the second number reaches a preset second threshold, it determines that the corresponding two road-end trajectories are simultaneously spatiotemporally related; otherwise, it determines that the corresponding two road-end trajectories are not simultaneously spatiotemporally related.

[0089] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention obtains the first number of times the target boxes of two vehicle-end trajectories appear in the same vehicle-end frame. If the first number reaches a preset first threshold, it is determined that the two corresponding vehicle-end trajectories are in a simultaneous spatial-temporal relationship; otherwise, it is determined that the two corresponding vehicle-end trajectories are not in a simultaneous spatial-temporal relationship. It also obtains the second number of times the target boxes of two road-end trajectories appear in the same road-end frame. If the second number reaches a preset second threshold, it is determined that the two corresponding road-end trajectories are in a simultaneous spatial-temporal relationship; otherwise, it is determined that the two corresponding road-end trajectories are not in a simultaneous spatial-temporal relationship. This improves the accuracy of determining whether two vehicle-end trajectories and two road-end trajectories are in a simultaneous spatial-temporal relationship.

[0090] According to an embodiment of the present invention, a vehicle-road cooperative multi-target tracking trajectory fusion device is provided, wherein the vehicle-side sequence data and the road-side sequence data are online data or offline data.

[0091] The vehicle-road cooperative multi-target tracking trajectory fusion device provided in this embodiment of the invention uses online or offline data for vehicle-end sequence data and road-end sequence data, thereby realizing online or offline vehicle-road cooperative multi-target tracking trajectory fusion and improving flexibility.

[0092] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a vehicle-road cooperative multi-target tracking trajectory fusion method. This method includes: synchronizing vehicle-end sequence data and road-end sequence data in time and space to obtain a vehicle-road cooperative time-space synchronized frame pair consisting of time-space aligned vehicle-end frames and road-end frames; determining whether there is a time-space relationship between any two vehicle-end trajectories and recording it in a vehicle-end trajectory relationship table; determining whether there is a time-space relationship between any two road-end trajectories and recording it in a road-end trajectory relationship table; and, based on the target box matching results of the vehicle-road cooperative time-space synchronized frame pair, obtaining the number of target boxes matched between any two vehicle-end trajectories and road-end trajectories and recording it in the [missing information]. A vehicle-road trajectory coordination relationship table is generated. Vehicle trajectories and roadside trajectories with a target box count greater than 0 constitute a coordinated trajectory pair. Based on the first vehicle-road trajectory coordination relationship table, the vehicle trajectory relationship table, and the roadside trajectory relationship table, mismatched coordinated trajectory pairs that produce simultaneous spatiotemporal trajectory matching are obtained from the coordinated trajectory pairs. The target box count corresponding to the mismatched coordinated trajectory pairs is set to 0, resulting in a second vehicle-road trajectory coordination relationship table. Based on the target box matching status in the second vehicle-road trajectory coordination relationship table, vehicle trajectories and roadside trajectories with target box matching are fused to obtain a vehicle-road cooperative multi-target tracking trajectory fusion result.

[0093] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] On the other hand, embodiments of the present invention also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the vehicle-road cooperative multi-target tracking trajectory fusion method provided by the above methods. This method includes: performing temporal and spatial synchronization on vehicle-end sequence data and road-end sequence data to obtain a vehicle-road cooperative temporal and spatial synchronization frame pair composed of temporally and spatially aligned vehicle-end frames and road-end frames; obtaining whether there is a temporal and spatial relationship between any two vehicle-end trajectories and recording it in a vehicle-end trajectory relationship table; obtaining whether there is a temporal and spatial relationship between any two road-end trajectories and recording it in a road-end trajectory relationship table; and determining the target bounding box of the vehicle-road cooperative temporal and spatial synchronization frame pair. The matching results are used to obtain the number of target boxes that match between each pair of vehicle-side trajectories and road-side trajectories, and record them in the first vehicle-road trajectory cooperation relationship table. Among them, the vehicle-side trajectories and road-side trajectories with a target box count greater than 0 constitute a cooperative trajectory pair. According to the first vehicle-road trajectory cooperation relationship table, the vehicle-side trajectory relationship table, and the road-side trajectory relationship table, the mismatched cooperative trajectory pairs that produce simultaneous spatiotemporal trajectory matching are obtained from the cooperative trajectory pairs. The number of target boxes corresponding to the mismatched cooperative trajectory pairs is set to 0, and a second vehicle-road trajectory cooperation relationship table is obtained. According to the target box matching status in the second vehicle-road trajectory cooperation relationship table, the vehicle-side trajectories and road-side trajectories with target box matching are fused to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0095] In another aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the vehicle-road cooperative multi-target tracking trajectory fusion method provided by the above methods. This method includes: performing temporal and spatial synchronization on vehicle-end sequence data and road-end sequence data to obtain a vehicle-road cooperative spatiotemporally synchronized frame pair consisting of spatiotemporally aligned vehicle-end frames and road-end frames; obtaining whether there is a simultaneous spatiotemporal relationship between any two vehicle-end trajectories and recording it in a vehicle-end trajectory relationship table; obtaining whether there is a simultaneous spatiotemporal relationship between any two road-end trajectories and recording it in a road-end trajectory relationship table; and obtaining the vehicle-end trajectory and road-end trajectory based on the target box matching result of the vehicle-road cooperative spatiotemporally synchronized frame pair. The number of target boxes that match between any two trajectories is recorded in the first vehicle-road trajectory cooperation relationship table. Vehicle trajectories and road-end trajectories with a target box count greater than 0 constitute a cooperative trajectory pair. Based on the first vehicle-road trajectory cooperation relationship table, the vehicle trajectory relationship table, and the road-end trajectory relationship table, mismatched cooperative trajectory pairs that produce simultaneous spatiotemporal trajectory matching are obtained from the cooperative trajectory pairs. The number of target boxes corresponding to the mismatched cooperative trajectory pairs is set to 0, resulting in the second vehicle-road trajectory cooperation relationship table. Based on the target box matching status in the second vehicle-road trajectory cooperation relationship table, vehicle trajectories and road-end trajectories with matching target boxes are fused to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle-road cooperative multi-target tracking trajectory fusion method, characterized in that, include: The vehicle-end sequence data and the road-end sequence data are synchronized in time and space to obtain a vehicle-road cooperative time and space synchronized frame pair consisting of vehicle-end frames and road-end frames that are aligned in time and space. Determine whether there is a simultaneous spatiotemporal relationship between any two vehicle-side trajectories and record it in the vehicle-side trajectory relationship table; Determine whether there is a simultaneous spatiotemporal relationship between any two road end trajectories and record it in the road end trajectory relationship table; Based on the target box matching results of the vehicle-road cooperative spatiotemporal synchronization frame pairs, the number of target boxes that match between each pair of vehicle-end trajectory and road-end trajectory is obtained and recorded in the first vehicle-road trajectory cooperative relationship table; wherein, the vehicle-end trajectory and the road-end trajectory with a target box number greater than 0 constitute a cooperative trajectory pair; Based on the first vehicle-road trajectory collaboration relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, obtain the mismatched collaborative trajectory pairs that generate simultaneous spatiotemporal trajectory matching in the collaborative trajectory pairs, set the number of target boxes corresponding to the mismatched collaborative trajectory pairs to 0, and obtain the second vehicle-road trajectory collaboration relationship table; Based on the target box matching in the second vehicle-road trajectory coordination relationship table, the vehicle-end trajectory and road-end trajectory with target box matching are fused to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

2. The vehicle-road cooperative multi-target tracking trajectory fusion method according to claim 1, characterized in that, The step involves obtaining mismatched collaborative trajectory pairs that produce simultaneous spatiotemporal trajectory matching from the collaborative trajectory pairs based on the first vehicle-road trajectory collaboration relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, setting the number of target boxes corresponding to the mismatched collaborative trajectory pairs to 0, and obtaining the second vehicle-road trajectory collaboration relationship table, including: Based on the first vehicle-road trajectory collaboration relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one of the road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as correctly matched collaborative trajectory pairs, and the remaining road-end trajectories in the road-end trajectory set and the vehicle-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the third vehicle-road trajectory collaboration relationship table. Based on the third vehicle-road trajectory coordination relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one vehicle-end trajectory in the set and the road-end trajectory are identified as correctly matched coordinated trajectory pairs, and the remaining vehicle-end trajectories in the set and the road-end trajectory are identified as mismatched coordinated trajectory pairs. The number of target boxes corresponding to the mismatched coordinated trajectory pairs is set to 0, thereby obtaining the second vehicle-road trajectory coordination relationship table.

3. The vehicle-road cooperative multi-target tracking trajectory fusion method according to claim 1, characterized in that, The step involves obtaining mismatched collaborative trajectory pairs that produce simultaneous spatiotemporal trajectory matching from the collaborative trajectory pairs based on the first vehicle-road trajectory collaboration relationship table, the vehicle-end trajectory relationship table, and the road-end trajectory relationship table, setting the number of target boxes corresponding to the mismatched collaborative trajectory pairs to 0, and obtaining the second vehicle-road trajectory collaboration relationship table, including: Based on the first vehicle-road trajectory collaboration relationship table and the vehicle-end trajectory relationship table, a set of vehicle-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same road-end trajectory is obtained. Based on the number of target boxes, one vehicle-end trajectory in the set of vehicle-end trajectories and the road-end trajectory are identified as correctly matched collaborative trajectory pairs, and the remaining vehicle-end trajectories in the set of vehicle-end trajectories and the road-end trajectory are identified as mismatched collaborative trajectory pairs. The number of target boxes corresponding to the mismatched collaborative trajectory pairs is set to 0, thus obtaining the fourth vehicle-road trajectory collaboration relationship table. Based on the fourth vehicle-road trajectory coordination relationship table and the road-end trajectory relationship table, a set of road-end trajectories with simultaneous spatiotemporal relationships that have target box matching with the same vehicle-end trajectory is obtained. Based on the number of target boxes, one road-end trajectory in the road-end trajectory set is identified as a correctly matched coordinated trajectory pair with the vehicle-end trajectory, and the remaining road-end trajectories in the road-end trajectory set are identified as mismatched coordinated trajectory pairs. The number of target boxes corresponding to the mismatched coordinated trajectory pairs is set to 0, thus obtaining the second vehicle-road trajectory coordination relationship table.

4. The vehicle-road cooperative multi-target tracking trajectory fusion method according to claim 2 or 3, characterized in that, The step of identifying a cooperative trajectory pair between one of the road-end trajectories in the road-end trajectory set and the vehicle-end trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any road-end trajectory in the road-end trajectory set that has the most target boxes matching the vehicle-end trajectory based on the number of target boxes, and identifying the cooperative trajectory pair between the road-end trajectory and the vehicle-end trajectory as a correctly matched cooperative trajectory pair; The step of identifying a cooperative trajectory pair between a vehicle trajectory in the vehicle trajectory set and the road trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes includes: obtaining any vehicle trajectory in the vehicle trajectory set that has the most target boxes matching the road trajectory, and identifying the cooperative trajectory pair between the vehicle trajectory and the road trajectory as a correctly matched cooperative trajectory pair based on the number of target boxes.

5. The vehicle-road cooperative multi-target tracking trajectory fusion method according to claim 1, characterized in that, The step of obtaining whether the two vehicle trajectories are in a simultaneous spatial-temporal relationship includes: obtaining the first number of times the target boxes of the two vehicle trajectories appear in the same vehicle frame; if the first number reaches a preset first threshold, the corresponding two vehicle trajectories are determined to be in a simultaneous spatial-temporal relationship; otherwise, the corresponding two vehicle trajectories are determined not to be in a simultaneous spatial-temporal relationship. The step of determining whether two road-end trajectories are in a simultaneous spatial-temporal relationship includes: obtaining the second number of times the target boxes of two road-end trajectories appear in the same road-end frame; if the second number reaches a preset second threshold, it is determined that the two corresponding road-end trajectories are in a simultaneous spatial-temporal relationship; otherwise, it is determined that the two corresponding road-end trajectories are not in a simultaneous spatial-temporal relationship.

6. The vehicle-road cooperative multi-target tracking trajectory fusion method according to claim 1, characterized in that, The vehicle-side sequence data and the road-side sequence data can be online or offline data.

7. A vehicle-road cooperative multi-target tracking trajectory fusion device, characterized in that, include: The spatiotemporal synchronization module is used to: synchronize vehicle-end sequence data and road-end sequence data in time and space to obtain a vehicle-road cooperative spatiotemporal synchronization frame pair composed of spatiotemporally aligned vehicle-end frames and road-end frames; The vehicle trajectory relationship table acquisition module is used to: obtain whether there is a simultaneous spatiotemporal relationship between any two vehicle trajectories and record it in the vehicle trajectory relationship table; The road end trajectory relationship table acquisition module is used to: obtain whether there is a simultaneous spatiotemporal relationship between any two road end trajectories and record it in the road end trajectory relationship table; The first vehicle-road trajectory coordination relationship table acquisition module is used to: obtain the number of target boxes that match between each pair of vehicle-end trajectory and road-end trajectory based on the target box matching result of the vehicle-road coordination spatiotemporal synchronization frame pair, and record them in the first vehicle-road trajectory coordination relationship table; wherein, the vehicle-end trajectory and the road-end trajectory with a target box number greater than 0 constitute a coordinated trajectory pair; The second vehicle-road trajectory coordination relationship table acquisition module is used to: obtain mismatched coordinated trajectory pairs that generate simultaneous spatiotemporal trajectory matching in the coordinated trajectory pairs according to the first vehicle-road trajectory coordination relationship table, the vehicle-end trajectory relationship table and the road-end trajectory relationship table, set the number of target boxes corresponding to the mismatched coordinated trajectory pairs to 0, and obtain the second vehicle-road trajectory coordination relationship table; The trajectory fusion module is used to: fuse vehicle-end trajectories and road-end trajectories with matching target boxes according to the target box matching in the second vehicle-road trajectory cooperation relationship table, so as to obtain the vehicle-road cooperative multi-target tracking trajectory fusion result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the vehicle-road cooperative multi-target tracking trajectory fusion method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle-road cooperative multi-target tracking trajectory fusion method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle-road cooperative multi-target tracking trajectory fusion method as described in any one of claims 1 to 6.

Images