Cross-device multi-target tracking method and system

Abstract

The invention discloses a cross-device multi-target tracking method and system. The method comprises the following steps: converting vehicle driving data collected by a camera and a radar into trajectory data under a global coordinate system; dividing all the sensing devices into a plurality of adjacent device pairs; checking the time dimension and the space dimension of the upstream track and the downstream track which are respectively formed by the adjacent equipment pair in sequence; and if the upstream track and the downstream track meet verification, splicing the track sequence according to a timestamp sequence to perform fusion of the global track, otherwise, temporarily storing the track as an isolated track independent of the global track. The system is used for implementing the method. The method has the advantages that feature extraction is not needed, the calculation complexity is low, and the real-time processing requirement of a large-scale sensing network can be met. Spatial similarity calculation methods are respectively designed for devices of the same type and different types, the cooperation requirements of mainstream sensing devices in an intelligent traffic scene are covered, and the compatibility is high.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method and system for multi-target tracking across devices. Background Technology

[0002] With the popularization of intelligent transportation systems, multi-camera collaborative tracking has become a core technical requirement in wide-area monitoring scenarios; especially in scenarios such as smart parks, transportation hubs, and urban roads, it is necessary to ensure that continuous tracking and trajectory matching of vehicle targets across cameras can be achieved.

[0003] However, existing mainstream methods heavily rely on vehicle appearance re-identification (ReID) features for cross-camera target association, which has the following inherent drawbacks: (1) High computational complexity: The deep ReID feature extraction model has a large computational load, which leads to a reduction in the real-time performance of the system and makes it difficult to meet the real-time processing requirements of large-scale monitoring networks.

[0004] (2) Poor environmental robustness: Appearance features are easily affected by changes in lighting, occlusion, weather conditions and differences in camera angles, resulting in insufficient feature stability and a decrease in matching accuracy.

[0005] (3) Strong data dependence: It requires a large amount of cross-camera target annotation data for model training, resulting in high annotation costs and limited generalization ability of the model in new, unseen scenes.

[0006] (4) Insufficient multi-device adaptation: No adaptation scheme was designed for the differences in trajectory data of different types of sensing devices such as radar and cameras, and the matching error caused by the mismatch in the number of trajectory points in the overlapping sensing area and the difference in sampling frequency was not resolved. Summary of the Invention

[0007] One objective of this application is to provide a cross-device multi-target tracking method that can overcome at least one of the deficiencies in the aforementioned background art.

[0008] Another objective of this application is to provide a cross-device multi-target tracking system that can solve at least one of the deficiencies in the aforementioned background art.

[0009] To achieve at least one of the above objectives, the technical solution adopted in this application is: a multi-target tracking method across devices, comprising the following steps: S100: Converts vehicle driving data collected by cameras and radar into trajectory data in a global coordinate system; S200: Based on the spatial coverage and topological relationship of different sensing devices, all sensing devices are divided into multiple adjacent device pairs; S300: Verify the upstream and downstream trajectories formed by adjacent devices in both time and spatial dimensions. S400: If the upstream and downstream trajectories in step S300 meet the time and space dimension verification, the trajectory sequence is spliced ​​according to the timestamp order to perform global trajectory fusion; otherwise, they will be temporarily stored as isolated trajectories independent of the global trajectory.

[0010] Preferably, the time-dimensional verification of the upstream and downstream trajectories in step S300 includes the following process: S310: Calculate the difference between the end time of the upstream trajectory and the start time of the downstream trajectory. If the difference is within the preset connection threshold range, proceed to the next step. Negative overlap between the upstream and downstream trajectories is allowed. S320: Calculate the time overlap value based on the time range of the upstream and downstream trajectories; if there is no perceived overlap area between the upstream and downstream trajectories, assign the time overlap value to 0; if there is a perceived overlap area between the upstream and downstream trajectories, determine the minimum overlap duration of the calculated time overlap value, and proceed to the next step for the upstream and downstream trajectories that meet the minimum overlap duration. S330: Based on the obtained time overlap value, calculate the proportion of the time overlap value to the duration of the upstream trajectory; if the calculated proportion is greater than the preset threshold, determine that the time matching of the overlapping area of ​​the upstream trajectory and the downstream trajectory has passed.

[0011] Preferably, the spatial dimension verification for adjacent device pairs with overlapping sensing areas includes the following process: S341: Extract trajectory points located within the overlapping area from the upstream and downstream trajectories respectively to form a candidate point pair set; S342: Filter the obtained candidate point pair set based on a distance threshold to obtain the longest common point pair sequence that satisfies the distance threshold; S343: Calculate the spatial similarity of overlapping area scenes based on the obtained longest common point pair sequence; S344: If the calculated spatial similarity is less than the set overlap spatial threshold, the spatial matching of the upstream trajectory and the downstream trajectory is deemed successful.

[0012] Preferably, spatial similarity S sim The calculation formula is as follows: ; Among them, T U and T D Let the upstream and downstream trajectories be represented respectively, and let K represent the number of longest common point pairs in the longest common point pair sequence. This represents the coordinates of the point on the upstream trajectory corresponding to the m-th longest common point pair. This represents the coordinates of the point on the downstream trajectory corresponding to the m-th longest common point pair.

[0013] Preferably, the spatial dimension verification of adjacent device pairs in areas with no perceived overlap includes the following process: S351: Select the end point of the upstream trajectory and the starting point of the downstream trajectory as key matching point pairs; S352: Calculate the spatial similarity of non-overlapping scene based on the obtained key matching point pairs; S353: If the calculated spatial similarity is less than the set connection space threshold, the spatial matching of the upstream trajectory and the downstream trajectory is deemed successful.

[0014] Preferably, spatial similarity S sim The calculation formula is as follows: ; Among them, (p U-end (X), p U-end (Y) represents the coordinates of the last point of the upstream trajectory in the key matching point pair, (p D-start (X), p D-start (Y) represents the coordinates of the starting point of the key matching point pair for the mid-to-lower reaches trajectory.

[0015] Preferably, for adjacent device pairs in areas of no perceived overlap, spatiotemporal evolution consistency verification is performed by introducing chain vectors, specifically including the following process: S361: Select the last two points p of the upstream trajectory U(n-1) and p U(n) and the two starting points p of the downstream trajectory D1 and p D2 ; S362: Based on point p U(n-1) and point p D2 Based on the spatiotemporal coordinates of point p, construct the first spatiotemporal chain vector; U(n) and point p D1 Using the spatiotemporal coordinates, construct the second spatiotemporal chain vector; S363: Calculate the cross product modulus of the first time chain vector and the second spatiotemporal chain vector; if the absolute value of the calculated cross product modulus is less than the set collinearity threshold, the spatiotemporal evolution consistency check of the upstream trajectory and the downstream trajectory is passed.

[0016] Preferably, in step S100, the vehicle driving data collected by the camera includes vehicle bounding box coordinates, detection confidence, vehicle type, and timestamp; based on the homography matrix of the camera, the vehicle bounding box coordinates are transformed from the pixel coordinate system to the global coordinate system, forming trajectory data including global coordinates of the vehicle bounding box, timestamp, device ID, local trajectory ID, and detection confidence; the vehicle driving data collected by the radar includes vehicle temporary ID, timestamp, spatial coordinates, and speed; the obtained raw radar trajectory data is optimized and then transformed to the global coordinate system, forming trajectory data including global spatial coordinates, timestamp, device ID, local trajectory ID, and speed.

[0017] Preferably, the optimization of the radar's raw trajectory data includes the following process: if the record interval between adjacent trajectory points in the raw trajectory data of the same vehicle temporary ID exceeds a threshold, it will be determined that the adjacent trajectory points correspond to different vehicles; then the raw trajectory data will be segmented based on the position of the adjacent trajectory points, and the vehicle temporary ID will be reassigned to the segmented trajectory segments; the number of trajectory points included in each raw trajectory data collected by the radar will be counted, and trajectories with a length less than the 5th percentile will be removed, and the remaining trajectories will be assigned the corresponding local trajectory ID.

[0018] A cross-device multi-target tracking system is provided for implementing the aforementioned cross-device multi-target tracking method. It includes a local tracking module, a trajectory preprocessing module, and a global tracker. The local tracking module is deployed on each camera to detect vehicles within its field of view and convert them into global coordinate system data. The trajectory preprocessing module is deployed on each radar to track the trajectories of vehicles within its field of view and convert them into global coordinate system data. The global tracker acquires data sent by the local tracking module and the trajectory preprocessing module to maintain global trajectory information for all vehicles and perform corresponding global trajectory fusion.

[0019] Compared with the prior art, the beneficial effects of this application are as follows: (1) No feature extraction is required. The core calculations only involve coordinate transformation, spatiotemporal parameter statistics and vector operations. The computational complexity is low and can meet the real-time processing requirements of large-scale perception networks.

[0020] (2) The tracking process does not depend on the vehicle's appearance features and is not affected by visual interference factors such as lighting, occlusion, and weather. At the same time, through time overlap quantification and spatial similarity analysis, it effectively resists matching fluctuations caused by differences in device sampling frequency and coordinate errors, and has excellent stability.

[0021] (3) Innovatively construct a unified radar-camera coordinate system, and design spatial similarity calculation methods for similar and dissimilar devices respectively, covering the collaborative needs of mainstream sensing devices in intelligent transportation scenarios, with strong compatibility. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall working steps of this application.

[0023] Figure 2 This is a schematic diagram of the trajectories of adjacent device pairs with overlapping sensing areas in this application.

[0024] Figure 3 This is a schematic diagram of the trajectories of adjacent device pairs in the non-perceptual overlapping area of ​​this application. Detailed Implementation

[0025] The present application will now be further described in conjunction with specific embodiments. It should be noted that, in the description of this specification, the use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicates that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0026] In the description of this application, it should be noted that the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., which indicate the orientation and positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of this application.

[0027] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0028] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0029] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0030] The terms “comprising” and “having”, and any variations thereof, in the specification and claims of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0031] One aspect of this application provides a multi-target tracking method across devices, such as Figure 1 As shown, one preferred embodiment includes the following steps: S100: Converts vehicle driving data collected by cameras and radar into trajectory data in a global coordinate system.

[0032] Understandably, both cameras and radar are commonly used sensing devices in semi-enclosed environments. Due to the different types of these two devices, it's difficult to achieve trajectory determination and fusion through direct pairing of the data they collect. Therefore, it's necessary to first unify the data collected by these two sensing devices. For the camera, a pre-calibrated homography matrix can be loaded. This matrix defines the projection relationship from the image pixel coordinate system to the global coordinate system, used for spatial positioning of the vehicle. For the radar, global transformation parameters are loaded to transform the radar's local coordinates to the global coordinate system, ensuring coordinate system consistency with the camera trajectory.

[0033] S200: Based on the spatial coverage and topological relationship of different sensing devices, all sensing devices are divided into multiple adjacent device pairs.

[0034] Understandably, multiple sensing devices are typically deployed in semi-enclosed environments. To facilitate vehicle trajectory tracking, it's necessary to obtain the sensing range, i.e., the spatial coverage, of each device. Based on the sensing range of each device, a topological relationship is constructed along all possible driving paths the vehicle can take within the semi-enclosed environment. According to this topological relationship, the sensing devices can be divided into multiple adjacent pairs.

[0035] S300: Perform time and spatial dimension verification on the upstream and downstream trajectories formed by adjacent devices respectively.

[0036] Understandably, since the time dimension verification requires less computation than the spatial dimension verification, the upstream and downstream trajectories can be verified first in the time dimension. This can filter out some irrelevant trajectories and retain a small number of upstream and downstream trajectories that may be generated by the same vehicle. Then, spatial consistency verification can be performed on the small number of retained trajectory data based on the spatial dimension, which can effectively reduce the computational load of verification and improve verification efficiency.

[0037] S400: If the upstream and downstream trajectories in step S300 meet the time and space dimension verification, the trajectory sequence is spliced ​​according to the timestamp order to perform global trajectory fusion; otherwise, they will be temporarily stored as isolated trajectories independent of the global trajectory.

[0038] Understandably, since both upstream and downstream trajectories carry timestamps and device identifiers, when fusing trajectories from adjacent devices, the two trajectories that meet the time and spatial dimension verifications can be concatenated into a single global trajectory based on their timestamps. The device identifiers of the upstream and downstream trajectories can be retained within the global trajectory to distinguish different sensing segments. The newly fused global trajectory can then continue to match with the trajectories of downstream adjacent devices, repeating the above steps to ultimately achieve full-link trajectory construction across multiple devices. Trajectories without a matching object can be temporarily stored independently of the global trajectory. If no match is found within a preset time, it can be marked as tracking complete; for example, if a vehicle travels a short distance within the camera's sensing range and then stops, its trajectory is only perceived by one camera.

[0039] Compared to traditional vehicle trajectory tracking methods that use ReID feature extraction, the technical solution in this application eliminates the need for feature extraction. The core computation involves only coordinate transformation, spatiotemporal parameter statistics, and vector operations, resulting in low computational complexity and meeting the real-time processing requirements of large-scale sensing networks. The tracking process is independent of vehicle appearance features and unaffected by visual interference factors such as lighting, occlusion, and weather. Furthermore, through temporal overlap quantification and spatial similarity analysis, it effectively resists matching fluctuations caused by differences in device sampling frequencies and coordinate errors, exhibiting excellent stability. In addition, the technical solution innovatively constructs a unified radar-camera coordinate system and designs spatial similarity calculation methods for similar and dissimilar devices, covering the collaborative needs of mainstream sensing devices in intelligent transportation scenarios and demonstrating strong compatibility.

[0040] In this embodiment, during step S100, the camera uses a lightweight detector, YOLOv11 or YOLO26, to perform real-time detection on video frames, thereby outputting the vehicle bounding box coordinates, detection confidence, and vehicle type. Then, the camera can continuously track the target vehicle using a deployed local tracking module, generating trajectory segments with local IDs and recording the coordinates of the bottom center point of the vehicle bounding box in each frame (representing the vehicle's ground position) and the corresponding timestamp. Since the bottom center point coordinates of the vehicle bounding box output by the camera are pixel coordinates p=(u, v), they need to be verified using the homography matrix H. c Complete the conversion from pixel coordinates to global coordinates to obtain the global coordinate position P of the target vehicle on the ground plane. w =(X, Y), and the specific conversion formula is as follows: .

[0041] in, Represents the homogeneous coordinate form of two-dimensional pixel coordinates (u, v), used for matrix operations; The pixel coordinates are homogeneous coordinates after homography transformation; X' and Y' represent intermediate variables of coordinates X and Y in homogeneous coordinates, respectively; w represents the scale factor in homogeneous coordinates, used to convert homogeneous coordinates into actual two-dimensional coordinates.

[0042] Based on the above transformation, the trajectory data corresponding to the camera in the global coordinate system can be obtained, specifically including the global coordinates of the vehicle bounding box, timestamp, device ID, local trajectory ID, and detection confidence.

[0043] In this embodiment, during step S100, the radar can receive real-time millimeter-wave data, including a vehicle temporary ID, timestamp, spatial coordinates, and speed. The collected data can then be filtered to remove abnormal data caused by perception errors, such as sudden speed changes or coordinates outside the road area. The filtered radar data can be optimized, and then the radar's local sensor coordinate system is transformed to a global coordinate system while retaining speed information. This yields the radar's trajectory data in the global coordinate system, specifically including global spatial coordinates, timestamp, device ID, local trajectory ID, and speed.

[0044] Understandably, when tracking the same vehicle, the raw trajectory data acquired by the radar after filtering may be split into multiple short trajectories due to signal fluctuations. This can cause the radar to mistakenly identify multiple short trajectories formed by the same vehicle as multiple vehicle trajectories when outputting data. At the same time, due to clutter and multipath acquisition, extremely short false trajectories may appear, that is, false trajectories correspond to only a few trajectory points. This reduces the efficiency of calculating false trajectories, and false trajectories may be misjudged as new trajectories, triggering incorrect associations. Therefore, the raw trajectory data acquired by the radar needs to be optimized before performing radar global coordinate transformation.

[0045] Specifically, the optimization of raw radar trajectory data includes the following processes: Trajectory Segmentation: If the record interval between adjacent trajectory points in the original trajectory data of the same vehicle temporary ID exceeds a threshold, it will be determined that the adjacent trajectory points correspond to different vehicles. The original trajectory data will then be segmented based on the position of these adjacent trajectory points, and the segmented trajectory segments will be reassigned vehicle temporary IDs. The threshold used to determine abnormal adjacent trajectory points can be selected according to the actual needs of those skilled in the art; for example, the radar trajectory point acquisition interval, such as 50ms, can be used.

[0046] Short trajectory elimination: Count the number of trajectory points included in each original trajectory data collected by the radar, and eliminate trajectories whose length is less than the 5th percentile. This can eliminate invalid trajectories caused by sensing noise. Then, assign the corresponding local trajectory ID to the remaining trajectories.

[0047] Understandably, optimizing radar trajectory processing by quickly removing invalid data through simple interval thresholds and length filtering can further improve processing efficiency.

[0048] In this embodiment, during step S200, the divided adjacent device pairs mainly include two cases: one is that the two sensing devices in the adjacent device pair are of the same type, such as two cameras or two radars; the other is that the two sensing devices in the adjacent device pair are of different types, i.e., a camera and a radar form an adjacent device pair. For ease of understanding, the following description will use adjacent device pairs formed by different types of sensing devices as examples.

[0049] It is important to note that, based on the spatial coverage and topological relationships of different sensing devices, not only are there adjacent device pairs, but also non-adjacent device pairs. When performing global trajectory matching, it is preferable to perform matching between trajectories of adjacent device pairs. Trajectories of non-adjacent device pairs are only matched when strong spatiotemporal constraints are met. That is, strong spatiotemporal constraints prevent erroneous trajectory matching of non-adjacent device pairs in situations where the trajectories are far apart and there is no overlapping sensing area. Generally, trajectory matching of non-adjacent device pairs is not considered; therefore, the following description will focus on trajectory matching of adjacent device pairs.

[0050] In this embodiment, for step S300, the upstream trajectory T U and downstream trajectory T D Consistency checks can be performed on a time-dimensional basis, focusing on both time continuity and time overlap. The specific process includes the following steps: S310: Calculate the upstream trajectory T U End time t U-end With downstream trajectory T D start time t D-start The difference Δt=t D-start -t U-end If the difference Δt is within the preset connection threshold range, the next step of time overlap verification can be performed.

[0051] It is understandable that for the upstream trajectory T representing the continuous travel of the same vehicle... U and downstream trajectory T D The trajectory can be continuous or intermittent, but the intervals are short; that is, the vehicle's trajectory can be considered essentially immutable during these intervals. Specifically, the upstream trajectory T... U and downstream trajectory T D If negative overlap is allowed, then negative values ​​are permitted within the connection threshold range. Specific values ​​can be selected based on the actual needs of those skilled in the art; for example, [-2s, 5s] could be used. Therefore, when -2s ≤ Δt ≤ 5s, it indicates that the upstream trajectory T... U and downstream trajectory T D Once the time continuity check is passed, the next step of time overlap check can be performed.

[0052] S320: Based on upstream trajectory T U and downstream trajectory T D Within the time range, calculate the time overlap value; if the upstream trajectory T U and downstream trajectory T D For regions with no perceived overlap, the time overlap value is assigned to 0; if the upstream trajectory T U and downstream trajectory T D There is a perceived overlapping region. The calculated temporal overlap values ​​are then judged based on the minimum overlap duration. The upstream trajectory T that satisfies the minimum overlap duration is then... U and downstream trajectory T D Proceed to the next step.

[0053] It is understandable that the upstream trajectory T U The time range is [t U-start , t U-end Downstream trajectory T D The time range is [t D-start , t D-end Then the time overlap value T overlap The calculation formula is: .

[0054] It is important to know the completion time overlap value T. overlap After calculation, there are two results; one is the time overlap value T. overlap >0, another result is the time overlap value T overlap =0. For the time overlap value T overlap If the value is greater than 0, the upstream trajectory T can be determined. U and downstream trajectory T D There is a perceptual overlap region; for the time overlap value T overlap In the case where =0, the upstream trajectory T can be determined. U and downstream trajectory T D Unperceived overlapping area.

[0055] For the upstream trajectory T U and downstream trajectory T D In scenarios with overlapping perceived regions, abnormal overlap may occur, such as an unrelated vehicle accidentally passing by the target vehicle, or false overlap in a single frame caused by jitter. Therefore, for scenarios with overlapping perceived regions, it is necessary to adjust the calculated temporal overlap value T. overlap The minimum overlap duration is determined only when the calculated time overlap T is reached. overlap Abnormal overlap can only be ruled out if the overlap duration is not less than the minimum overlap duration.

[0056] S330: Based on the obtained time overlap value, calculate the time overlap value relative to the upstream trajectory T. UThe proportion of duration; if the calculated proportion is greater than a preset threshold, the upstream trajectory T is determined. U and downstream trajectory T D The time matching of the overlapping areas was successful.

[0057] It is understandable that this can be achieved through the time overlap index T. index Indicates the time overlap value T overlap The time overlap index T is the ratio of the duration of the upstream trajectory to the duration of the upstream trajectory. index The calculation formula can be expressed as: .

[0058] The specific value of the threshold for determining whether the time matching of the overlapping area is successful can be determined by those skilled in the art based on their actual needs; for example, the value can be 10% to 30%.

[0059] It is important to note that the unit of minimum overlap duration is seconds (s), while the time overlap index T... index Dimensionless quantity; minimum overlap duration is used to determine whether the overlapping area is an abnormal overlap, time overlap index T index It is to ensure the upstream trajectory T U and downstream trajectory T D There is sufficient overlap between them for subsequent similarity calculations. To facilitate understanding, a specific example will be used to illustrate this in detail below.

[0060] Specifically, assuming the upstream trajectory T U Duration 20s, downstream trajectory T D The duration is 15s, and there is a 2s overlap between the two. Assuming the minimum overlap duration is 1s, and the threshold for determining whether the time match in the overlap area passes is 30%, then based on the given conditions, it can be known that the upstream trajectory T... U and downstream trajectory T D Meets the minimum overlap duration requirement; time overlap index T index =10%, failed time matching of overlapping areas.

[0061] Assume the upstream trajectory T U The duration is 5 seconds, and the downstream trajectory T D The duration of the first trajectory is 6 seconds, and there is a 2-second overlap between them. Assuming the minimum overlap duration is 1 second, and the threshold for determining whether the time match in the overlap area passes is 30%, then based on the given conditions, it can be known that the upstream trajectory T... U and downstream trajectory T D To meet the minimum overlap duration requirement, the time overlap index T index =40%, through time matching of overlapping areas.

[0062] Assume the upstream trajectory TU The duration is 1.5 seconds, and the downstream trajectory T D The duration of the first trajectory is 1 second, and there is an overlap region of 0.8 seconds between them. Assuming the minimum overlap duration is 1 second, and the threshold for determining whether the time match in the overlap region passes is 30%, then based on the given conditions, it can be known that the upstream trajectory T... U and downstream trajectory T D It does not meet the minimum overlap duration requirement, but the time overlap index T index =53.3%.

[0063] As can be understood from the above examples, if only the minimum overlap duration is used to determine the temporal matching of overlapping areas, then longer trajectories may not have enough overlapping areas for spatial similarity calculation, making it difficult to match the upstream trajectory T. U and downstream trajectory T D The accuracy of consistency judgment is significantly reduced if only the time overlap index T is used. index When performing time matching for overlapping regions, extremely short trajectories may pass the match due to jitter. Therefore, in this embodiment, it is necessary to first match the upstream trajectory T using the minimum overlap duration. U and downstream trajectory T D Perform an absolute judgment in the duration dimension, and then use the time overlap index T. index For the upstream trajectory T U and downstream trajectory T D A relative judgment is made based on the duration dimension to ensure the accuracy of subsequent spatial consistency judgments.

[0064] From the above, we can see that the upstream trajectory T U and downstream trajectory T D There are two scenarios: one is the presence of a perceptually overlapping region, and the other is the absence of a perceptually overlapping region. For ease of understanding, the following will provide a detailed description of the spatial dimension verification for each of these scenarios.

[0065] 1. Spatial dimension verification for adjacent device pairs with overlapping sensing areas.

[0066] like Figure 2 As shown, the specific verification process includes the following steps: S341: From the upstream trajectory T respectively U and downstream trajectory T D The trajectory points located within the overlapping region are extracted to form a candidate point pair set.

[0067] It is understandable that, such as Figure 2 and Figure 3As shown, since there is a sampling interval in the data collection of the target vehicle by the camera and radar, the first trajectory collected by the camera and the second trajectory collected by the radar can be equivalent to continuous trajectory points according to the sampling interval; then the upstream trajectory T U and downstream trajectory T D The perceptual overlap region can be viewed as the overlap region of multiple trajectory points. Within the perceptual overlap region, the upstream trajectory T can be... U The trajectory points and the downstream trajectory T D If the trajectory points corresponding to the time dimension are paired to form a candidate point pair, then after all the trajectory points in the overlapping area are paired, multiple candidate point pairs can be obtained, and these multiple candidate point pairs can form a candidate point pair set.

[0068] It should be noted that since the sampling intervals of cameras and radars may be different, for sensing devices with larger sampling intervals, interpolation can be used to supplement the trajectory points in the overlapping sensing areas to ensure that the number of trajectory points of cameras and radars in the overlapping sensing areas is basically consistent.

[0069] S342: Filter the obtained candidate point pair set based on a distance threshold to obtain the longest common point pair sequence that satisfies the distance threshold.

[0070] It is understandable that candidate point pairs are paired according to the time dimension, but the two may not correspond in the spatial dimension, that is, there is a gap between the two trajectory points (the spatial position of the target vehicle) in the paired candidate point pair. If the gap between the two trajectory points in the paired candidate point pair exceeds the set distance threshold, it can be basically determined that the two trajectory points do not correspond in space and can be eliminated. The specific value of the distance threshold can be selected according to the actual needs of those skilled in the art, for example, the distance threshold can be 2m, 3m, etc.

[0071] S343: Calculate the spatial similarity of overlapping areas based on the obtained longest common point pair sequence.

[0072] Specifically, we can assume that the number of longest common point pairs in the longest common point pair sequence is K, and the upstream trajectory T corresponding to any m-th longest common point pair is... U The coordinates of the point are The downstream trajectory T corresponding to any m-th longest common point pair D The coordinates of the point are Then the upstream trajectory T U and downstream trajectory T D Spatial similarity S between perceived overlapping regions sim The calculation formula is as follows: .

[0073] S344: If the calculated spatial similarity S sim If the overlap space threshold is less than the set threshold, the upstream trajectory T is determined. U and downstream trajectory T D Spatial matching passed.

[0074] It is understandable that, based on the spatial similarity S mentioned above... sim The calculation formula shows that spatial similarity S sim It can be viewed as the average of the Euclidean distances of all longest common pairs of points. Therefore, based on the definition of Euclidean distance, the spatial similarity S... sim The larger the value, the greater the distance between the two points, indicating poorer spatial consistency. Therefore, in this embodiment, an overlap spatial threshold can be set to improve the spatial similarity S. sim The upstream trajectory T can only be determined if it is less than the overlap space threshold. U and downstream trajectory T D Spatial matching was successful. The specific value of the overlap spatial threshold can be selected according to the actual needs of those skilled in the art; for example, it can be 3m.

[0075] II. Spatial dimension verification for adjacent device pairs in areas of non-perceptual overlap.

[0076] like Figure 3 As shown, the specific verification process includes the following steps: S351: Select the upstream trajectory T U The end point and the downstream trajectory T D The starting point is used as the key matching point pair.

[0077] It is understandable that the upstream trajectory T U The endpoint is the location where the target vehicle leaves the sensing range of the upstream sensing device; the downstream trajectory T D The starting point is the location point when the target vehicle just enters the sensing range of the downstream sensing device.

[0078] S352: Calculate the spatial similarity of non-overlapping scene pairs based on the obtained key matching point pairs.

[0079] Specifically, we can set the upstream trajectory T of the key matching point pair. U The coordinates of the end point are (p U-end (X), p U-end (Y)), key matching point pairs for mid-to-downstream trajectory T D The starting point coordinates are (p D-start (X), p D-start (Y)). Then the upstream trajectory T U and downstream trajectory T D Spatial similarity S of non-perceptual overlapping regions simThe calculation formula is as follows: .

[0080] S353: If the calculated spatial similarity S sim If the distance is less than the set connection space threshold, the upstream trajectory T is determined. U and downstream trajectory T D Spatial matching passed.

[0081] It is understandable that, based on the aforementioned spatial similarity S sim The calculation formula shows that spatial similarity S sim This can be viewed as the Euclidean distance between key matching point pairs; therefore, based on the definition of Euclidean distance, spatial similarity S... sim The larger the value, the greater the distance between the two points, indicating poorer spatial consistency. Therefore, in this embodiment, a connection space threshold can be set to adjust the spatial similarity S. sim The upstream trajectory T can only be determined if it is less than the threshold of the connection space. U and downstream trajectory T D Spatial matching was successful. The specific value of the connection space threshold can be selected according to the actual needs of those skilled in the art; for example, it can be 10m.

[0082] It should be noted that, based on the aforementioned upstream trajectory T U and downstream trajectory T D Spatial dimension verification of the non-perceptual overlapping area shows that the upstream trajectory T U and downstream trajectory T D The dimensionality is determined only by a single point, which leads to the upstream trajectory T in scenarios with no perceived overlap. U and downstream trajectory T D The spatial dimension verification accuracy is slightly poor. Therefore, in order to improve the accuracy of upstream trajectory T in scenarios with no perceived overlap, U and downstream trajectory T D The accuracy of spatial dimension verification can be achieved by introducing chain vectors for spatiotemporal evolution consistency verification.

[0083] In this embodiment, the specific process of performing spatiotemporal evolution consistency verification on adjacent device pairs in non-overlapping regions using chain vectors is as follows: S361: Select the upstream trajectory T U The last two points p U(n-1) and p U(n) and downstream trajectory T D The two starting points p D1 and p D2 .

[0084] S362: Based on point p U(n-1) and point pD2 Based on the spatiotemporal coordinates of point p, construct the first spatiotemporal chain vector; U(n) and point p D1 The spatiotemporal coordinates are used to construct a second spatiotemporal chain vector.

[0085] It is understandable that the spatiotemporal coordinates of a trajectory point consist of two-dimensional spatial coordinates and one-dimensional time coordinates; where point p U(n-1) The spacetime coordinates are (p U(n-1) (X), p U(n-1) (Y), t U(n-1) ), point p U(n) The spacetime coordinates are (p U(n) (X), p U(n) (Y), t U(n) ), point p D1 The spacetime coordinates are (p D1 (X), p D1 (Y), t D1 ), point p D2 The spacetime coordinates are (p D2 (X), p D2 (Y), t D2 Based on the coordinates described above, the expressions for the first spatiotemporal chain vector L1 and the second spatiotemporal chain vector L2 are as follows: .

[0086] .

[0087] S363: Calculate the cross-product magnitude of the first time chain vector and the second spatiotemporal chain vector; if the absolute value of the calculated cross-product magnitude is less than the set collinearity threshold, determine the upstream trajectory T. U and downstream trajectory T D The spatiotemporal evolution consistency check passed.

[0088] It is understandable that the specific value of the collinearity threshold can be selected according to the actual needs of those skilled in the art. Generally speaking, the closer the cross-product modulus is to 0, the stronger the upstream trajectory T. U and downstream trajectory T D The closer they are to being collinear in spatiotemporal evolution, the higher the probability that they belong to the same vehicle; therefore, the specific value of the collinearity threshold can be 0.1~0.2.

[0089] Specifically, the formula for calculating the cross product magnitude |P| of the first spatiotemporal chain vector L1 and the second spatiotemporal chain vector L2 is as follows: .

[0090] .

[0091] .

[0092] .

[0093] Among them, P X P Y and P Z These represent the three components of the spatiotemporal cross product in spatial coordinates, and are used to evaluate the degree of non-collinearity of the first spatiotemporal chain vector L1 and the second spatiotemporal chain vector L2 in the three spatiotemporal directions.

[0094] It should be understood that, in the technical solution of this application, for the upstream trajectory T where there is a sensing overlap area U and downstream trajectory T D Quantitative matching is achieved through temporal overlap index and spatial similarity; for upstream trajectories T in areas with no perceptible overlap... U and downstream trajectory T D By verifying the consistency of spatiotemporal evolution through the cross product modulus of chain vectors, a matching logic covering all scenarios and multiple dimensions is formed, which greatly improves the accuracy of trajectory association.

[0095] Another aspect of this application provides a cross-device multi-target tracking system for performing the aforementioned cross-device multi-target tracking method; a preferred embodiment includes a local tracking module, a trajectory preprocessing module, and a global tracker. The local tracking module is deployed on each camera to detect vehicles within its field of view and convert them into global coordinate system data; the trajectory preprocessing module is deployed on each radar to track the trajectories of vehicles within its field of view and convert them into global coordinate system data; the global tracker acquires data sent by the local tracking module and the trajectory preprocessing module to maintain global trajectory information for all vehicles and perform corresponding global trajectory fusion.

[0096] Understandably, upon system startup, it reads basic parameters from all cameras and radars, including device frame rate, time synchronization offset, camera homography matrix, and radar coordinate transformation coefficients. Then, it performs global coordinate transformation through a local tracking module deployed on the cameras and a trajectory preprocessing module deployed on the radars. The transformed global coordinates, timestamps, device IDs, local trajectory IDs, detection confidence (cameras), and speed (radar) are encapsulated into structured observation data, serving as the input source for the global tracker. The global tracker then executes the aforementioned multi-target tracking method based on the input source, ultimately outputting the complete trajectory of the target vehicle in the global coordinate system.

[0097] The basic principles, main features, and advantages of this application have been described above. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely the principles of this application. Various changes and modifications can be made to this application without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims. The scope of protection claimed by this application is defined by the appended claims and their equivalents.

Claims

1. A multi-target tracking method across devices, characterized in that, Includes the following steps: S100: Converts vehicle driving data collected by cameras and radar into trajectory data in a global coordinate system; S200: Based on the spatial coverage and topological relationship of different sensing devices, all sensing devices are divided into multiple adjacent device pairs; S300: Verify the upstream and downstream trajectories formed by adjacent devices in both time and spatial dimensions. S400: If the upstream and downstream trajectories in step S300 meet the time and space dimension verification, the trajectory point sequence is spliced ​​according to the timestamp order to perform global trajectory fusion; otherwise, they will be temporarily stored as isolated trajectories independent of the global trajectory.

2. The cross-device multi-target tracking method as described in claim 1, characterized in that, The time-dimensional verification of the upstream and downstream trajectories in step S300 includes the following process: S310: Calculate the difference between the end time of the upstream trajectory and the start time of the downstream trajectory. If the difference is within the preset connection threshold range, proceed to the next step. Negative overlap between the upstream and downstream trajectories is allowed. S320: Calculate the time overlap value based on the time range of the upstream and downstream trajectories; if there is no perceived overlap area between the upstream and downstream trajectories, assign the time overlap value to 0; if there is a perceived overlap area between the upstream and downstream trajectories, determine the minimum overlap duration of the calculated time overlap value, and proceed to the next step for the upstream and downstream trajectories that meet the minimum overlap duration. S330: Based on the obtained time overlap value, calculate the proportion of the time overlap value to the duration of the upstream trajectory; if the calculated proportion is greater than the preset threshold, determine that the time matching of the overlapping area of ​​the upstream trajectory and the downstream trajectory has passed.

3. The cross-device multi-target tracking method as described in claim 2, characterized in that, Spatial dimension verification for adjacent device pairs with overlapping sensing areas includes the following process: S341: Extract trajectory points located within the overlapping area from the upstream and downstream trajectories respectively to form a candidate point pair set; S342: Filter the obtained candidate point pair set based on a distance threshold to obtain the longest common point pair sequence that satisfies the distance threshold; S343: Calculate the spatial similarity of overlapping area scenes based on the obtained longest common point pair sequence; S344: If the calculated spatial similarity is less than the set overlap spatial threshold, the spatial matching of the upstream trajectory and the downstream trajectory is deemed successful.

4. The cross-device multi-target tracking method as described in claim 3, characterized in that, Spatial similarity S in step S343 sim The calculation formula is as follows: ； Among them, T U and T D Let the upstream and downstream trajectories be represented respectively, and let K represent the number of longest common point pairs in the longest common point pair sequence. This represents the coordinates of the point on the upstream trajectory corresponding to the m-th longest common point pair. This represents the coordinates of the point on the downstream trajectory corresponding to the m-th longest common point pair.

5. The cross-device multi-target tracking method as described in claim 2, characterized in that, Spatial dimension verification for adjacent device pairs in areas of non-perceptual overlap includes the following process: S351: Select the end point of the upstream trajectory and the starting point of the downstream trajectory as key matching point pairs; S352: Calculate the spatial similarity of non-overlapping scene based on the obtained key matching point pairs; S353: If the calculated spatial similarity is less than the set connection space threshold, the spatial matching of the upstream trajectory and the downstream trajectory is deemed successful.

6. The cross-device multi-target tracking method as described in claim 5, characterized in that, Spatial similarity S in step S352 sim The calculation formula is as follows: ； Among them, T U and T D Representing the upstream and downstream trajectories respectively, (p U-end (X), p U-end (Y) represents the coordinates of the last point of the upstream trajectory in the key matching point pair, (p D-start (X), p D-start (Y) represents the coordinates of the starting point of the key matching point pair for the mid-to-lower reaches trajectory.

7. The cross-device multi-target tracking method as described in claim 4, characterized in that, For adjacent device pairs in areas of non-perceptual overlap, spatiotemporal consistency verification is performed by introducing chain vectors, specifically including the following process: S361: Select the last two points p of the upstream trajectory U(n-1) and p U(n) and the two starting points p of the downstream trajectory D1 and p D2 ; S362: Based on point p U(n-1) and point p D2 Based on the spatiotemporal coordinates of point p, construct the first spatiotemporal chain vector; U(n) and point p D1 Using the spatiotemporal coordinates, construct the second spatiotemporal chain vector; S363: Calculate the cross product modulus of the first time chain vector and the second spatiotemporal chain vector; if the absolute value of the calculated cross product modulus is less than the set collinearity threshold, the spatiotemporal evolution consistency check of the upstream trajectory and the downstream trajectory is passed.

8. The cross-device multi-target tracking method as described in any one of claims 1-7, characterized in that, In step S100, the vehicle driving data collected by the camera includes the vehicle bounding box coordinates, detection confidence, vehicle type, and timestamp. Based on the homography matrix of the camera, the vehicle bounding box coordinates are transformed from the pixel coordinate system to the global coordinate system, forming trajectory data including the global coordinates of the vehicle bounding box, timestamp, device ID, local trajectory ID, and detection confidence. The vehicle driving data collected by radar includes the vehicle's temporary ID, timestamp, spatial coordinates, and speed; After optimizing the obtained raw radar trajectory data, it is converted to a global coordinate system to form trajectory data including global spatial coordinates, timestamp, device ID, local trajectory ID, and velocity.

9. The cross-device multi-target tracking method as described in claim 8, characterized in that, The optimization of raw radar trajectory data includes the following process: If the interval between records of adjacent trajectory points in the original trajectory data of the same vehicle temporary ID exceeds the threshold, it will be determined that the adjacent trajectory points correspond to different vehicles; then the original trajectory data will be segmented based on the position of the adjacent trajectory points, and the vehicle temporary ID will be reassigned to the segmented trajectory segments. The number of trajectory points included in each raw trajectory data collected by the statistical radar is counted. Trajectories with a length less than the 5th percentile are removed, and the remaining trajectories are assigned a corresponding local trajectory ID.

10. A cross-device multi-target tracking system for implementing the cross-device multi-target tracking method according to any one of claims 1-9; characterized in that, include: Local tracking module; The local tracking module is deployed on each camera to detect vehicles within the field of view and convert them into global coordinate system data; Trajectory preprocessing module; the trajectory preprocessing module is deployed in each radar and is used to track the trajectory of vehicles within the field of view and convert it into global coordinate system data; as well as Global tracer; The global tracker acquires data sent by the local tracking module and the trajectory preprocessing module to maintain global trajectory information for all vehicles and perform corresponding global trajectory fusion.

Images