Multi-source target trajectory semantic fusion method and system based on unified space-time coordinate system

Abstract

The application relates to a multi-source target trajectory semantic fusion method and system based on a unified space-time coordinate system. The method comprises the following steps: receiving target trajectory segments of multi-source heterogeneous sensing nodes; according to the processing time delay characteristics of each sensing node type, performing time delay compensation on the timestamps of each segment to align to a unified physical time axis, calculating three-dimensional joint correlation scores of the aligned different sensing node trajectory segments, and determining the segments belonging to the same target based on a global optimal correlation rule; sequentially connecting the same target segments, smoothing the joints, and generating a continuous global target trajectory; and according to the number of sensor covers, time delay compensation accuracy, correlation score and inference proportion, calculating a fusion quality confidence score for the global trajectory segments and storing the same in association. The method can adapt to the trajectory fusion requirements of a multi-layer heterogeneous sensor network, and comprehensively improves the accuracy, continuity and practicability of the multi-target global trajectory generation in a wide area space.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a method and system for semantic fusion of multi-source target trajectories based on a unified spatiotemporal coordinate system. Background Technology

[0002] With the rapid development of spatial perception, multi-sensor data fusion, and intelligent target tracking technologies, multi-layered sensing networks composed of heterogeneous sensing nodes such as visual cameras, LiDAR, millimeter-wave radar, UWB positioning beacons, and GPS have been widely applied in scenarios such as urban traffic control, low-altitude airspace management, logistics traceability, and security law enforcement evidence collection. These networks are used to continuously collect trajectory data of moving targets such as people, vehicles, and drones. Traditional multi-source target trajectory processing mainly adopts single-sensor tracking algorithms and visual re-identification (ReID) technology. Single-sensor tracking can only achieve continuous observation of targets within a single sensing node, while visual ReID extracts target appearance feature vectors and calculates similarity to achieve target matching and association across cameras. It is currently the mainstream implementation method for cross-device target trajectory stitching.

[0003] However, traditional trajectory processing methods centered on single-sensor tracking and visual ReID are only suitable for single-camera network deployment scenarios. They employ independent pairwise matching logic to complete target association, and their association judgment dimensions are only constructed around appearance features. In complex scenarios such as drastic changes in perspective, dense clusters of similar-looking targets, and partial target occlusion, the reliability of appearance feature matching drops significantly, easily leading to target identity confusion and association errors. Different types of sensors inherently have significant differences in data processing latency. Traditional solutions directly perform trajectory stitching based on the original timestamps of the sensors, and the latency difference is directly transmitted to the stitching result, causing time axis misalignment and producing obvious positional errors in high-speed moving target scenarios. Even if coordinate system unification is achieved, the inherent measurement errors of different sensors will still cause positional jumps at trajectory seams. Using simple linear interpolation to eliminate jumps will introduce trajectory segments that do not match the actual motion, making it difficult to match the actual motion state of the target. In multi-target dense scenarios, local greedy independent matching strategies can lead to the problem of explosive association combinations, which can easily generate chain association errors. At the same time, the output trajectory of existing solutions only contains basic motion position information, making it difficult to generate accurate, continuous, semantically complete and quantifiable global target trajectories. This makes it difficult to provide standardized data support for the reliable operation of upper-level applications such as path planning, airspace management, and law enforcement evidence collection. Summary of the Invention

[0004] Therefore, it is necessary to provide a method and system for semantic fusion of multi-source target trajectories based on a unified spatiotemporal coordinate system to address the above-mentioned technical problems.

[0005] A semantic fusion method for multi-source target trajectories based on a unified spatiotemporal coordinate system, the method comprising: Receives target trajectory segments output by multi-source heterogeneous sensing nodes; the trajectory segments include the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system. Based on the processing delay characteristics of various types of sensing nodes, delay compensation is performed on the timestamp sequence of the trajectory segments, the multi-source trajectory segments are aligned to a unified physical time axis, and the corresponding delay compensation accuracy is determined based on the delay compensation results. A trajectory association score is calculated for trajectory segments of different sensing nodes that have completed time alignment. Based on the trajectory association score and a pre-set global optimal association rule, trajectory segments belonging to the same physical target are determined. The trajectory association score is obtained at least based on spatiotemporal consistency and motion state similarity assessment. Trajectory segments belonging to the same physical target are sequentially connected in time. The position deviation and time gap at the trajectory joint are smoothed based on the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory. Based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio, a fusion quality confidence score is calculated for each segment of the global target trajectory; the trajectory inference ratio is the proportion of the number of trajectory points generated by inference in the corresponding segment to the total number of trajectory points in the segment. The global target trajectory is associated with and stored with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

[0006] A multi-source target trajectory semantic fusion system based on a unified spatiotemporal coordinate system, the system comprising: The trajectory segment receiving module is used to receive target trajectory segments output by multi-source heterogeneous sensing nodes; the trajectory segment includes the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system. The delay compensation module is used to perform delay compensation on the timestamp sequence of the trajectory segment according to the processing delay characteristics of each type of sensing node, align the multi-source trajectory segments to a unified physical time axis, and determine the corresponding delay compensation accuracy based on the delay compensation result. The target association module is used to calculate trajectory association scores for trajectory segments of different sensing nodes that are aligned with the completion time, and to determine trajectory segments belonging to the same physical target based on the trajectory association scores and pre-set global optimal association rules; the trajectory association scores are obtained at least based on spatiotemporal consistency and motion state similarity assessment; The global trajectory generation module is used to connect trajectory segments belonging to the same physical target in chronological order, and to smooth the position deviation and time gap at the trajectory joints based on the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory. The trajectory quality scoring module is used to calculate the fusion quality confidence score for the global target trajectory segments based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio; the trajectory inference ratio is the proportion of the number of trajectory points generated by inference in the corresponding segment to the total number of trajectory points in the segment. The global trajectory output module is used to associate and store the global target trajectory with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

[0007] The aforementioned multi-source target trajectory semantic fusion method and system based on a unified spatiotemporal coordinate system, by receiving target trajectory segments from multi-source heterogeneous sensing nodes in a unified three-dimensional spatiotemporal coordinate system, can lay a unified spatial benchmark and standardized input foundation for cross-node trajectory fusion. Through delay compensation and time axis alignment based on the processing delay characteristics of sensing nodes, it can eliminate time misalignment caused by delay differences between heterogeneous sensors, reducing trajectory splicing position errors for high-speed targets. Through trajectory association scoring based on spatiotemporal consistency and motion state similarity, and global optimal matching, it can achieve accurate identification of the same target across sensors, reducing target identity ambiguity in dense scenes. Through trajectory continuation and seam smoothing, it can eliminate positional jumps at trajectory splicing points, ensuring the continuity and authenticity of the global trajectory. Through segmented fusion quality confidence scores calculated by multi-dimensional factors and associated storage, it can achieve quantitative control of trajectory reliability and output standardized global semantic trajectory data. This invention can adapt to the trajectory fusion requirements of multi-layer heterogeneous sensor networks, comprehensively improving the accuracy, continuity, and practicality of multi-target global trajectory generation in a wide-area space. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a multi-source target trajectory semantic fusion method based on a unified spatiotemporal coordinate system in one embodiment. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] In one embodiment, such as Figure 1 As shown, a multi-source target trajectory semantic fusion method based on a unified spatiotemporal coordinate system is provided, including the following steps: Step 102: Receive the target trajectory fragment output by the multi-source heterogeneous sensing nodes. The trajectory fragment includes the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system.

[0011] Multi-source heterogeneous sensing nodes refer to various target detection and positioning devices deployed in different locations in space and employing different sensing principles, including UWB positioning beacons, millimeter-wave radar, lidar, visual cameras, and satellite positioning modules. A target trajectory segment refers to a continuous observation sequence containing target motion information generated after a single sensing node continuously observes a single target; it is the basic input unit of this scheme. A unified three-dimensional spatiotemporal coordinate system is a unified three-dimensional spatial coordinate system and time reference shared by all sensing nodes in space, ensuring spatial comparability of target position information output by different sensing nodes. A position sequence refers to the set of three-dimensional spatial position coordinates of the target in the unified three-dimensional spatiotemporal coordinate system, arranged in the order of observation within the trajectory segment. A timestamp sequence refers to the set of original time stamps corresponding one-to-one with each position coordinate in the trajectory segment, representing the time when the sensing node acquired that position information. Processing delay characteristics refer to core parameters such as the nominal processing delay and delay fluctuation range required for different types of sensing nodes to generate target trajectory segments from the acquisition of raw signals. The basic value of this parameter is determined through calibration measurements during system initialization and can be adaptively updated during system operation using cross-sensing data with the same target.

[0012] In one embodiment of the present invention, a TrajectoryFragment is the basic input unit of the present invention, corresponding to a continuous observation sequence of a single sensor node on a single target. Tables 1 and 2 below show an exemplary information composition of the trajectory fragment and its trajectory points: Table 1: Examples of TrajectoryFragment Information Composition

[0013] Table 2: Example of Waypoint Information Composition

[0014] To accommodate the subsequent processing requirements of trajectory inference ratio calculation and merging inference type annotations, optional additional fields such as trajectory point type markers and trajectory segment type markers can be added to the basic data structures of trajectory points (Waypoints) and trajectory segments (TrajectoryFragments) to distinguish between observation points and inference points, and between original observation segments and inference / merging type segments. The aforementioned basic data structures only contain the fields necessary to implement the core solution of this invention. Other reasonable additional fields can be added according to actual application needs. The addition or absence of such additional fields does not affect the complete implementation of the core technical solution of this invention, nor does it constitute a limitation on the scope of protection of this invention.

[0015] Step 104: Based on the processing delay characteristics of each type of sensing node, perform delay compensation on the timestamp sequence of the trajectory segment, align the multi-source trajectory segments to a unified physical time axis, and determine the corresponding delay compensation accuracy based on the delay compensation result.

[0016] Latency compensation refers to the process of correcting the original timestamp of each trajectory point in a trajectory segment based on the processing latency characteristics of sensing nodes, thus offsetting the time offset caused by differences in data processing links between different sensing nodes. A unified physical timeline refers to a unified time reference shared by all trajectory segments and perfectly aligned with real physical time after eliminating differences in processing latency among sensing nodes; it is the core foundation for cross-sensor node trajectory comparison and stitching. Latency compensation accuracy refers to the degree of alignment between the trajectory point timestamp and the real physical time after the latency compensation operation, which is determined by the latency fluctuation range of the sensing nodes.

[0017] It is understandable that this step can eliminate the time axis misalignment problem caused by the differences in processing links between different types of sensing nodes, reduce the position error of high-speed moving targets caused by time delay to the level of measurement error, improve the accuracy of cross-sensing node trajectory comparison, and provide a unified and accurate time reference for subsequent target association.

[0018] Step 106: Calculate trajectory association scores for trajectory segments from different sensing nodes that have completed time alignment. Based on the trajectory association scores and pre-set globally optimal association rules, determine the trajectory segments belonging to the same physical target. The trajectory association scores are obtained at least based on spatiotemporal consistency and motion state similarity assessments.

[0019] Trajectory association scoring refers to a quantitative indicator used to quantify the degree of matching between two trajectory segments output by different sensing nodes that belong to the same physical target. The higher the score, the greater the probability that the two trajectories belong to the same target. Global optimal association rule refers to modeling the trajectory association problem of multiple sensing nodes and multiple targets, aiming to maximize the sum of global association scores, and solving for the rule of globally optimal pairing scheme. Spatiotemporal consistency refers to the degree of matching between two trajectory segments in terms of time interval and spatial displacement, satisfying the feasibility of motion of the same physical target. Motion state similarity refers to the degree of continuous matching between the target motion state at the end of the first trajectory segment and the target motion state at the beginning of the second trajectory segment.

[0020] It is understandable that this step can achieve accurate cross-perception node target association in difficult scenarios such as drastic changes in perspective, dense clusters of similar-looking targets, and partial occlusion. It can avoid the chain matching errors caused by local greedy matching strategies, which helps to reduce the probability of target ID confusion in dense multi-target scenarios and greatly improves the accuracy and robustness of cross-node target recognition.

[0021] Step 108: The trajectory segments belonging to the same physical target are sequentially connected in time. The position deviation and time gap at the trajectory joint are smoothed according to the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory.

[0022] Trajectory splicing refers to the operation of stitching together multiple trajectory segments identified as belonging to the same physical target in chronological order along a unified physical timeline to form a complete motion link. Trajectory seam refers to the connecting area between the end of one trajectory segment and the beginning of the next in two consecutive trajectory segments. Global target trajectory refers to the continuous and complete motion trajectory of the same physical target within a complete observation period, formed after multi-source trajectory segment splicing and smoothing processing.

[0023] It is understandable that this step can eliminate trajectory position jumps caused by measurement errors of different sensing nodes, fill the time gaps between trajectory segments, restore the true motion behavior of the target, avoid motion trajectory distortion caused by artificial interpolation, and help improve the continuity and authenticity of the global trajectory, providing complete and smooth target motion link data for upper-layer applications.

[0024] Step 110: Based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio, calculate the fusion quality confidence score for each segment of the global target trajectory. The trajectory inference ratio is the proportion of the number of trajectory points generated through inference in the corresponding segment to the total number of trajectory points in the segment.

[0025] The fusion quality confidence score is a standardized rating calculated for each segment of the global target trajectory based on multiple influencing factors. It quantifies the reliability of a trajectory segment, ranging from 0 to 1, with higher scores indicating higher reliability. The number of sensing nodes covering the target refers to the number of independent sensing nodes effectively sensing the target within the time period corresponding to the trajectory segment. The trajectory inference ratio refers to the proportion of indirect observation trajectory points generated through interpolation, motion model extrapolation, target merging inference, etc., within a trajectory segment, relative to the total number of trajectory points in that segment.

[0026] Understandably, this step enables a quantitative credibility assessment of each segment of the global trajectory, providing quantifiable reference data on trajectory quality for upper-layer applications. This facilitates the selection of trajectory data with appropriate quality based on business needs in different scenarios such as law enforcement evidence collection and route planning, thereby achieving refined control and utilization of trajectory quality.

[0027] Step 112: Associate and store the global target trajectory with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

[0028] Global semantic trajectory data refers to the standardized trajectory dataset formed by associating and binding the complete global target trajectory with the fusion quality confidence score of the corresponding segment. It includes the complete motion link of the target, source sensing node information, quality confidence label, and unified spatiotemporal reference information. It is the final output of this solution.

[0029] It is understandable that this step can generate standardized, directly callable multi-source target trajectory output results, enabling traceability of the trajectory data source and assessment of its quality throughout the entire chain. This is beneficial for adapting to the application needs of various upper-level business scenarios such as path planning, airspace management, logistics traceability, and law enforcement evidence collection, while providing standardized basic data for subsequent tamper-proof evidence storage of the trajectory.

[0030] In the aforementioned multi-source target trajectory semantic fusion method based on a unified spatiotemporal coordinate system, by receiving target trajectory segments from multi-source heterogeneous sensing nodes in a unified three-dimensional spatiotemporal coordinate system, a unified spatial benchmark and standardized input foundation can be established for cross-node trajectory fusion. Through delay compensation and time axis alignment based on the processing delay characteristics of sensing nodes, time misalignment caused by delay differences in heterogeneous sensors can be eliminated, reducing trajectory splicing position errors for high-speed targets. Through trajectory association scoring based on spatiotemporal consistency and motion state similarity, and global optimal matching, accurate identification of the same target across sensors can be achieved, reducing target identity ambiguity in dense scenes. Through trajectory continuation and seam smoothing, positional jumps at trajectory splicing points can be eliminated, ensuring the continuity and authenticity of the global trajectory. Through segmented fusion quality confidence scores calculated by multi-dimensional factors and associated storage, standardized control of trajectory reliability can be achieved, outputting standardized global semantic trajectory data. This embodiment of the invention can adapt to the trajectory fusion requirements of multi-layer heterogeneous sensor networks, comprehensively improving the accuracy, continuity, and practicality of multi-target global trajectory generation in a wide-area space.

[0031] In one embodiment, the trajectory association score is also obtained based on multimodal feature similarity evaluation. The three-dimensional joint score is formed by the joint calculation of spatiotemporal consistency, motion state similarity, and multimodal feature similarity. Among them, the multimodal feature similarity of the same type of sensing nodes is calculated based on the cosine similarity of the target appearance feature vector, and the multimodal feature similarity of different type of sensing nodes is calculated based on the weighted comprehensive calculation after normalization of the corresponding feature types.

[0032] Specifically, for any two trajectory segments generated by different sensing nodes and The system establishes a three-dimensional joint scoring framework to comprehensively evaluate the probability that both scores correspond to the same physical target. This joint score can be expressed as a weighted sum of the three sub-scores:

[0033] Here, α, β, and γ are weighting coefficients that satisfy α + β + γ = 1, and can be adaptively adjusted according to the combination of sensor types involved in the association and the dynamics of the scene. For example, when both segments come from featureless sensors (such as UWB beacons), the system automatically reduces γ to 0, making the association judgment rely entirely on spatiotemporal consistency and motion state similarity; when the two segments come from the same type of visual sensor, the value of γ can be appropriately increased to make full use of the distinguishing ability of appearance features.

[0034] Spatiotemporal consistency score The physical meaning of this is to determine whether two trajectories satisfy the feasibility of continuous motion towards the same target in the spacetime dimension. Its calculation method is as follows: Let Δt be End time and The time interval between start times, Δd is the spatial distance between the corresponding end position and the starting position. , To cover the average velocity required to fill the gap at this time, The maximum possible speed preset for this target type, then when hour, (If physically unreachable, eliminate it directly); otherwise k is the attenuation coefficient and k≥1.

[0035] Motion state similarity score For comparison End-effector motion state and The initial state of motion is assessed to evaluate the continuity of the motion. , for The velocity vector and direction angle at the end point, , for Initial velocity vector and motion direction angle, , For the uncertainty parameter associated with the sensor type, then .

[0036] Multimodal feature similarity scoring To measure the degree of matching of target features from different sensor perspectives, the system supports unified comparison and calculation of cross-modal features and employs differentiated calculation strategies for different types of sensors. When both originate from dissimilar sensors... This equals the weighted overall similarity after normalizing the features of each modality; when any sensor does not support the generation of reliable feature vectors... Set to a preset constant to degenerate the feature dimension into a neutral factor. Example follows: For similar sensors (camera vs. camera): directly calculate the cosine similarity of appearance feature vectors.

[0037] Heterogeneous sensors (camera vs. LiDAR): After normalizing the appearance features and point cloud shape features respectively, a weighted comprehensive similarity is calculated.

[0038] Featureless sensor (UWB positioning): Set to a constant of 0.5, it does not participate in the differentiation; it mainly depends on... and .

[0039] In this embodiment, by incorporating spatiotemporal consistency, motion state similarity, and multimodal feature similarity into a unified joint scoring framework, the decision-making for cross-sensor target recognition no longer relies solely on appearance feature matching. Instead, even under conditions of drastic changes in viewpoint or severe degradation of appearance features due to partial occlusion, it can still maintain a high association accuracy by relying on spatiotemporal physical accessibility constraints and motion state continuity. By adaptively adjusting the weights of each dimension according to sensor type combinations, the system can obtain optimized association performance in both homogeneous and heterogeneous sensor networks, significantly improving its versatility.

[0040] In one embodiment, determining trajectory segments belonging to the same physical target based on trajectory association scores and pre-set global optimal association rules includes: selecting two different multi-source heterogeneous sensing nodes within the same time window and covering the target monitoring area as the first sensing node and the second sensing node; using all trajectory segments of the first sensing node as matrix rows and all trajectory segments of the second sensing node as matrix columns; calculating trajectory association scores for each pair of trajectory segments corresponding to the rows and columns; filling matrix elements to obtain an association candidate matrix; the association candidate matrix satisfies association constraints; with the goal of maximizing the sum of global association scores, using a bipartite graph optimal matching algorithm to solve the association candidate matrix to obtain the global optimal association scheme, determining the pairing of trajectory segments belonging to the same physical target; performing physical reachability verification on each pair of trajectory segments in the global optimal association scheme; if the physical reachability requirements are not met, the pairing is terminated, and the involved trajectory segments are marked as pending association; the association constraints limit each trajectory segment to be associated with at most one trajectory segment of another sensing node, while marking elements in the matrix with trajectory association scores lower than a preset association threshold as invalid pairings.

[0041] Specifically, in multi-target dense scenes, the first sensing node may simultaneously track M targets and generate M trajectory segments, while the second sensing node may simultaneously track N targets and generate N trajectory segments. Cross-sensor association faces a combination space of M×N candidate pairings. The system first constructs an M-row × N-column association scoring matrix, where the element in the i-th row and j-th column of the matrix is ​​the joint score between the i-th trajectory segment of the first sensing node and the j-th trajectory segment of the second sensing node. After the matrix is ​​filled, two constraints are imposed: first, elements with scores below a preset association threshold are directly marked as invalid pairs and set to zero. ∞ indicates that the candidate pair will not be considered in subsequent solutions; the second is to impose a one-to-one constraint, that is, each trajectory segment can only be associated with at most one trajectory segment of another sensing node, and each trajectory segment can only be associated once.

[0042] Based on this, the system aims to maximize the sum of global association scores and employs the Hungarian algorithm to solve for optimal bipartite graph matching of the score matrix. The Hungarian algorithm can find the globally optimal solution that maximizes the total score from all feasible association schemes satisfying the one-to-one constraint within polynomial time complexity. After solving, the system performs a secondary verification on each pair in the optimal scheme: if a pair... If the result is 0 (i.e. physically impossible), the pairing is terminated, and the two trajectory segments involved are marked as "association pending". Subsequently, attempts can be made to associate them with segments from other sensing nodes or to enter the manual confirmation process.

[0043] In this embodiment, the cross-sensor association problem is modeled as a globally optimal allocation problem rather than a local greedy matching problem. This effectively avoids cascading errors in scenarios such as: two targets A and B with similar appearances appearing simultaneously in the fields of view of two sensors. If independent pairwise judgment is used, A from sensor one might be incorrectly associated with B from sensor two (due to high appearance similarity), which in turn might force B from sensor one to be incorrectly associated with A from sensor two, or prevent them from being associated at all. Global optimal solution, however, can weigh the mutual constraints between all candidate pairs from a holistic perspective, ensuring that the association scheme is optimal in a globally meaningful sense. Furthermore, a secondary physical reachability check further eliminates pairings that, even with high scores, are physically impossible, improving the physical plausibility of the association results.

[0044] In one embodiment, the step of performing time delay compensation on the timestamp sequence of trajectory segments based on the processing time delay characteristics of each type of sensing node includes: when two different types of sensing nodes simultaneously and effectively sense the same confirmed associated target, the sensing node with the smallest time delay is used as the reference node; the actual time delay deviation of other sensing nodes relative to the reference node is calculated, and the processing time delay characteristics of the corresponding sensing node type are continuously updated in a moving average manner.

[0045] Specifically, the system performs the following delay compensation correction on the timestamp of each trajectory point in each trajectory segment to eliminate time axis misalignment caused by differences in processing link delays between different sensors:

[0046] in, The original timestamp of trajectory point i under the sensor's local clock is given by `latency_ms(sensor_type)`, which is the nominal processing latency corresponding to this sensor type (in milliseconds, divided by 1000 to convert to seconds). This is the timestamp after time delay compensation, aligned with a unified physical time axis. After this correction, the time delay-compensated timestamps of trajectory points generated by the same physical target in different sensors all correspond to the same physical moment, allowing for direct spatiotemporal continuity comparison and trajectory fusion on a unified time axis.

[0047] The aforementioned time delay compensation relies on the nominal processing time delay of each sensor type. This nominal time delay is obtained through calibration measurement during the system initialization phase. Specifically, the measurement method can be as follows: during the time period when the same physical target is effectively covered by two sensors simultaneously, using the timestamp of a known low-latency sensor (such as a UWB positioning beacon with a time delay of approximately 5-20 ms) as a reference, calculate the time offset of the other sensor relative to this reference. After multiple measurements, take the statistical average as the nominal time delay of that sensor type, and record its fluctuation variance.

[0048] However, in actual operation, the sensor's processing latency may drift due to factors such as temperature changes, computational load fluctuations, and communication link interference, causing the nominal latency set during initial calibration to gradually deviate from the actual value. Therefore, the system uses confirmed associated observation data of the same target to perform online adaptive updates to the latency characteristics during operation.

[0049] When two different types of sensors simultaneously and effectively sense the same confirmed associated target, the system uses the sensor with the smaller latency as the time reference. Let the nominal latency of the reference sensor be `latency_ref`, and the current nominal latency of the sensor to be calibrated be `latency_current`. The system acquires the position estimates of the target near the same physical moment from both the reference sensor and the sensor to be calibrated. It then calculates the actual observation time deviation `latency_measured`, i.e., the actual latency difference between the sensor to be calibrated and the reference sensor, through trajectory spatiotemporal back-calculation. This measured value is then fused with the currently recorded latency value using a moving average to obtain the updated latency value. latency_updated=λ×latency_current+(1 λ)×latency_measured Where λ is the smoothing coefficient of the moving average (0 < λ < 1), and latency_updated is the updated nominal latency, which will be used for latency compensation calculations for all trajectory segments of this type of sensor. Simultaneously, based on the statistical distribution of multiple measured latency values, the system synchronously updates the latency fluctuation variance for this sensor type; this variance will serve as the latency compensation accuracy factor in subsequent quality scoring. One of the computational inputs.

[0050] In this embodiment, the basic time delay compensation formula uses a simple and unified calculation method to correct the timestamps of each sensor trajectory point to the same physical time axis, fundamentally solving the problem of trajectory time axis misalignment caused by differences in sensor data link processing delays. For high-speed moving targets (such as vehicles traveling at 60 km / h), even a time delay difference of tens of milliseconds can lead to meter-level positional deviations. Time delay compensation can reduce such deviations to the level of measurement error. Furthermore, the online adaptive update mechanism allows the time delay characteristics to be continuously corrected according to changes in the actual operating conditions of the sensors, overcoming the problem of fixed calibration values ​​drifting over time and ensuring the time alignment accuracy of the fused trajectory in long-term continuous operation scenarios. In addition, the synchronous update and output of the time delay fluctuation variance allows upper-layer applications to quantitatively assess the uncertainty of time alignment of each sensor, providing transparent accuracy data for high-reliability applications such as law enforcement and evidence collection.

[0051] In one embodiment, the position deviation and time gap at the trajectory seam are smoothed based on the position sequence and timestamp sequence after time delay compensation. This includes: if there is a time gap at the trajectory seam and the time gap is less than a preset interpolation threshold, smoothing interpolation is performed based on the motion state of the trajectory segments at both ends of the seam to fill the trajectory points at the seam and mark the interpolation; if there is a position deviation at the trajectory seam and the position deviation is within a preset motion uncertainty range, the trajectory points in the seam area are smoothed to eliminate position jumps and record the smoothing adjustment amount; in the global target trajectory, the corresponding trajectory association score, position deviation, time gap, and interpolation or smoothing mark are recorded for each seam position.

[0052] Specifically, when two trajectory segments that have been confirmed to be related... and When splicing together on a unified timeline after time delay compensation, two types of discontinuities may occur. The first is a time gap: The last trajectory point time and There may be a time gap between the first trajectory point and the time point that is not covered by any sensor. If this gap Δt is less than a preset interpolation threshold... The system is based on The motion state of the end effector (position, velocity, acceleration) and The initial motion state is used to generate trajectory points within the gaps using a smooth interpolation algorithm (such as cubic spline interpolation or motion model-based predictive interpolation). These interpolated trajectory points are labeled as "interpolation points" to distinguish them from directly observed points.

[0053] The second type is positional jump: even if the two trajectory segments are continuous in time or have been filled by interpolation, due to differences in sensor measurement errors, End position and There may still be a positional deviation Δd between the starting positions. If Δd is within a reasonable range determined based on the target motion uncertainty parameters (related to sensor accuracy and target maneuverability), the system assumes that the deviation originates from measurement error rather than a real physical jump. The system then performs smoothing filtering on the trajectory points in the seam neighborhood (e.g., N points before and after the seam) to make the position transition continuous and natural, and records the smoothing adjustment amount (the difference between the original position and the smoothed position) for each trajectory point.

[0054] For each seam location, the system associates and stores the following information in the global target trajectory: the cross-sensor correlation score (Score), position deviation Δd, time interval Δt, whether it contains interpolation points and the number of interpolation points, whether it has undergone smoothing processing, and the maximum smoothing adjustment amount. This information provides calculation input for the subsequent quality scoring module.

[0055] In this embodiment, by performing conditional interpolation and smoothing at the seams, the continuity of the global trajectory in the spatiotemporal dimensions is ensured, while the transparency and traceability of the processing are maintained through detailed seam information recording. Time gap determination avoids the introduction of unreliable trajectory segments through blind interpolation in situations with long periods of no observation; position deviation smoothing eliminates false trajectory jitter caused by differences in sensor accuracy. Simultaneously, the recording of interpolation markers and smoothing adjustment amounts serves as the "inference ratio" factor in the quality score. This provides accurate calculation basis.

[0056] In one embodiment, the method further includes: when multiple sensing nodes provide location estimates exceeding a preset deviation threshold for the same confirmed associated target at the same time, the physical reachability of each location estimate is tested based on the maximum speed constraint of the target type, and the arbitration weight of the location estimate that does not meet the physical reachability is reduced to zero; for the location estimate that passes the physical reachability test, weighted location fusion is performed based on the product of the type accuracy coefficient, geometric accuracy factor and time delay compensation accuracy factor of the corresponding sensing node as the arbitration weight.

[0057] Specifically, when two sensors generate position estimates for the same confirmed associated target within an overlapping coverage area, the two estimates may deviate significantly due to anomalies in one of the sensors (such as multipath effects, temporary occlusion, or calibration drift). The system first calculates the spatial distance between the two estimates. If this distance exceeds a preset deviation threshold (e.g., 3 times the standard deviation 3σ of the normal measurement error range), a conflict arbitration process is triggered.

[0058] Arbitration is conducted in two steps. The first step is a hard constraint test of physical reachability: using the most recent high-confidence trajectory point before the current moment as a reference, the estimated velocity required for the target to travel from that reference point to two candidate locations is calculated. If the velocity required for a candidate location exceeds the maximum possible velocity for that target type... If the location estimate is deemed physically impossible, its arbitration weight is reduced to zero, and a data quality alarm is triggered for the corresponding sensor. If both candidate locations pass the physical reachability test, the process proceeds to the second step of precision weight soft constraint arbitration.

[0059] In the second step, the system performs a weighted fusion of the location estimates that pass the physical accessibility test using the following formula:

[0060] The arbitration weights of each sensor Determined by the product of three factors:

[0061] The accuracy coefficient for sensor type reflects the inherent position measurement accuracy level of this type of sensor under standard operating conditions (e.g., UWB positioning accuracy is usually higher than that of camera monocular ranging). The geometric accuracy factor reflects the impact of the target's spatial position relative to the sensor on the measurement accuracy (e.g., the closer the target is to the center of the sensor's field of view, the higher the geometric accuracy factor). The delay compensation accuracy factor is calculated based on the current delay fluctuation variance of the sensor. The greater the delay fluctuation, the lower the factor, indicating that the uncertainty of time alignment has a greater impact on position estimation.

[0062] In this embodiment, a hierarchical arbitration mechanism is used to first quickly identify and exclude abnormal sensor data using hard constraints based on physical reachability. Then, soft constraints based on accuracy weights are used to optimally fuse multi-source location estimates within the normal range, avoiding the problem of simple averaging being easily contaminated by outliers. The design of a three-level accuracy factor allows the fusion result to adaptively utilize sensor data with higher accuracy. At the same time, the introduction of a time delay compensation accuracy factor enables the fusion quality to adaptively reduce the weighting of sensors with time delay fluctuations. The introduction of a sensor anomaly alarm mechanism further improves the maintainability of the system and its robustness to sensor failures.

[0063] In one embodiment, the method further includes: when the positions of multiple tracked targets continue to converge and the number of targets reported by the sensing nodes decreases, recording the motion state and feature information of each target before aggregation; during the period of target aggregation, maintaining independent trajectory extrapolation based on the motion state of each target before aggregation; when the targets are redispersed, re-associating the dispersed targets with the tracked targets before aggregation based on the continuity of motion state, and labeling the trajectory segments during aggregation as merge inference type.

[0064] Specifically, in densely populated scenarios such as crowds and convoys, multiple independently tracked targets may converge due to their proximity. When the system detects that the spatial distance between two or more tracked targets continues to decrease to less than the spatial resolution of the sensor, and the number of targets reported by the sensor decreases accordingly, the system determines that a target merging event has occurred and enters the merging processing mode.

[0065] First, for each target participating in the merging process, the system records a "pre-merge snapshot" at the moment of merging: this includes the position, velocity vector, motion direction angle, and appearance feature vector of the last frame before merging. This snapshot data will be used for extrapolation during merging and reassociation after splitting.

[0066] During the merging process, the system can no longer directly distinguish the individual trajectories of each target. At this point, the system initiates independent trajectory extrapolation for each target based on the motion state in the snapshot before merging. The extrapolation can be based on a uniform velocity model, a uniform acceleration model, or a more complex motion prediction model; the specific model selection can be shared with the trajectory inference mechanism during the occlusion period. The trajectory points generated by the extrapolation are marked as "inference points".

[0067] When the sensors re-detect multiple independent targets and output their respective trajectory segments (targets are dispersed), the system initiates a re-association process. The core principle of re-association is to prioritize the use of similar motion states. Matching, rather than appearance feature similarity. The reason is that during the merging / splitting process, targets may undergo temporary changes in appearance features due to mutual occlusion or dense interaction, while changes in motion state before and after merging are usually smoother and more predictable. The system calculates the differences between the candidate targets detected after dispersion and each tracked target before merging. Rating, in Maximize the one-to-one association as the criterion.

[0068] The trajectory segments during the merging process are marked as "merging inference segment" (segment_type=MERGED) in the global target trajectory. The factor is reduced accordingly, indicating that the trajectory location of this segment is derived from extrapolation prediction rather than direct observation. In application scenarios that require high-precision location information (such as law enforcement evidence collection), the weight can be reduced or excluded as appropriate.

[0069] In this embodiment, through the system's merge detection, independent extrapolation, and motion state-priority re-association mechanisms, the system can maintain the continuity and correctness of the identity identifiers of each target in scenarios with dense target interactions, avoiding target identity switching caused by brief merging. By explicitly marking the merged segment as the inference type, upper-layer applications can flexibly decide whether to use the trajectory data segment based on their own accuracy requirements, achieving transparent management of trajectory data availability and reliability.

[0070] In one embodiment, after associating and storing the global target trajectory with the segmented fusion quality confidence score, the method further includes: calculating a cryptographic hash value for each segment of the global target trajectory, chaining the hash values ​​of subsequent trajectory segments with the hash values ​​of preceding trajectory segments, generating an immutable trajectory evidence chain, and storing it synchronously.

[0071] Specifically, in certain application scenarios (such as accident liability determination, law enforcement evidence collection, and logistics traceability), the immutability and verifiability of trajectory data are crucial. This embodiment, based on the aforementioned trajectory fusion and quality scoring, further adds cryptographic tamper-proof protection to the global target trajectory.

[0072] For a complete global target trajectory, the system divides it into several trajectory segments in chronological order. For the data content of the first trajectory segment (including the sequence of trajectory points, timestamps, segment type, list of source segment identifiers, and the TQS score of the segment), a preset cryptographic hash algorithm (such as SHA-256) is used to calculate its hash value H1. For the second trajectory segment, its data content is combined with the hash value H1 of the previous trajectory segment to calculate a hash value H2. This process continues, forming a hash chain.

[0073] In this context, "||" indicates data concatenation. The hash value of each trajectory segment is stored in the global trajectory database in association with its corresponding trajectory segment data, TQS score, source sensing node identifier, and time delay compensation record.

[0074] Users of the track can independently verify its integrity and tamper-resistance by recalculating the hash value of each track segment in the same order, starting from the track's origin, and comparing it with the stored hash values ​​one by one. If the calculated value at any stage does not match the stored value, it indicates that the track segment and all subsequent data may have been tampered with. Due to the chain structure of the hash chain, any modification to any segment of track data will cause a change in the hash value of that segment and all subsequent chain nodes, resulting in extremely high detection sensitivity. Furthermore, the TQS score for each track segment allows the verifier to selectively focus on the data integrity of high-confidence segments.

[0075] In this embodiment, by organically linking the cryptographic hash chain with the segmented quality scoring mechanism, a complete trustworthy assurance system is provided for the fused global semantic trajectory, encompassing perception, fusion, and tamper-proof evidence storage. Trajectory users do not need the endorsement of a centralized trusted institution; they can independently verify the authenticity of the trajectory using only the publicly available hash algorithm and the stored hash chain. This is of significant value in complex application scenarios involving the division of responsibilities among multiple parties and the acceptance of evidence.

[0076] In one embodiment, the system calculates a Trajectory Quality Score (TQS) for each time period of the fused global target trajectory.

[0077] Specifically, TQS is obtained by weighted summation of the following four quality factors:

[0078] The physical meaning and calculation method of each factor are as follows: Sensor coverage quantity factor Q_coverage reflects the contribution of the number of independent sensors that effectively perceive the target within a given time period to the geometric accuracy of the fused localization. It is calculated by dividing the number of sensors that actually provide effective observations of the target within that time period by the theoretically maximum number of sensors that can cover the spatial region where the target is located. A higher number of covering sensors results in stronger multi-view geometric intersection constraints, higher accuracy of the fused position estimation, and a smaller impact on overall localization when a single sensor fails; therefore, a higher Q_coverage value is desirable.

[0079] Delay compensation accuracy factor This reflects the impact of the latency uncertainties of each sensor involved in the fusion process on the time alignment accuracy. Its calculation method is as follows:

[0080] Here, `latency_uncertainty_ms` represents the maximum or mean variance of latency fluctuations among the sensors involved in the fusion within that time period, and `max_acceptable_latency_ms` represents the maximum acceptable latency uncertainty for the system's application scenario. The smaller the latency uncertainty, the more accurate the time alignment and the higher the timing accuracy of the fused trajectory. The closer it gets to 1.

[0081] Related rating factors This reflects the reliability of the cross-sensor correlation process relied upon during that time period. Its value is the lowest of all cross-sensor correlation scores within that time period. If the time period is composed of multiple sensor segments fused together, the correlation with the lowest score is selected as the baseline. The lowest association score determines the weakest identity association link in the fusion results. Therefore, the minimum value is used instead of the mean value to reflect the association risk in a conservative manner.

[0082] Inferring the scaling factor This reflects the proportion of trajectory points within a given time period that are derived from inference rather than direct observation. The calculation method is as follows: =1 (Number of inferred trajectory points / Total number of trajectory points in this time period) The inferred trajectory points include: trajectory points generated by extrapolation during the occlusion period, trajectory points generated by interpolation at the seams, and trajectory points generated by extrapolation during target merging. A higher inference ratio indicates a lower degree of agreement between that trajectory segment and the actual sensor observations. The value will be reduced accordingly.

[0083] The default weighting of the four factors is: w1=0.35, w2=0.25, w3=0.25, w4=0.15. The weights can be customized according to specific application scenarios. The TQS value ranges from 0 to 1, where 0 represents completely untrusted and 1 represents the highest quality.

[0084] The TQS scores for each time period are stored in segments within the global target trajectory record, and are associated with information such as the time range, segment type, and list of source sensing nodes for the corresponding trajectory segment. Users of the trajectory can use the query interface provided by the global trajectory database to filter low-quality trajectory segments by TQS thresholds (e.g., a threshold of TQS ≥ 0.85 can be set in law enforcement evidence collection scenarios), or dynamically adjust the dependency weights on each trajectory segment based on TQS in applications such as path planning and situation assessment.

[0085] The beneficial effects of this embodiment are as follows: By establishing a multi-factor comprehensive trajectory quality confidence scoring system, the system provides a structured and quantifiable confidence evaluation standard for fused trajectories for the first time. Four quality factors characterize trajectory quality from four key dimensions: geometric coverage redundancy, time alignment accuracy, identity association confidence, and observation directness. This allows upper-layer applications to flexibly select or de-weight trajectory segments of different quality levels according to their differentiated reliability needs, achieving refined management and utilization of trajectory data from "existence" to "confidence." Simultaneously, the segmented storage of TQS scores combined with the source traceability information of trajectory segments ensures that the confidence evaluation conclusion of any trajectory segment is itself traceable and interpretable, providing a complete data foundation for subsequent auditing and quality backtracking.

[0086] In one embodiment, the final output of the present invention is a global semantic trajectory database. This database stores each fused global target trajectory and its segmentation quality information in a structured manner, which can be accessed by upper-layer applications through a query interface.

[0087] Tables 3 and 4 below show an exemplary composition of information about the global target trajectory and its trajectory segments.

[0088] Table 3: Examples of Global Trajectory Information Components

[0089] Table 4: Example of Segment Information Composition

[0090] The information composition described in Tables 3 and 4 above is merely an exemplary structured definition for implementing the technical solution of this invention. The above information composition only includes the parts necessary for implementing the core solution of this invention and does not constitute an absolute limitation on the completeness of the global trajectory database information. Depending on the actual application scenario, other reasonable information items (such as target appearance feature snapshots, the version number of the fusion algorithm that generated the trajectory, and the association confirmation time, etc.) can be added on this basis. Whether or not such additions are made, they do not change the complete implementation of the core technical solution of this invention, nor should they be construed as limiting the scope of protection of this invention.

[0091] Furthermore, in this embodiment, the aforementioned global semantic trajectory database also provides the following query interfaces: querying the complete trajectory and quality scores of each segment of a target by a global target identifier; querying all target trajectories passing through a specified time range and spatial region by a specified time range and spatial region; filtering by a fusion quality confidence threshold to return only trajectory segments that meet the specified TQS quality requirements; and querying all global target trajectories composed of trajectory segments of that node by a source sensing node identifier. These query interfaces enable upper-layer applications to flexibly obtain trajectory data of the required granularity and quality level according to their own needs.

[0092] In this embodiment, by establishing a structured global semantic trajectory database output format, the outputs of the aforementioned processing steps (including latency-compensated trajectory points, cross-sensor correlation results, seam quality information, and segmented TQS scores) are uniformly organized into a semantically complete, quality-transparent, and traceable fused trajectory data body. The segmented storage mechanism for TQS scores allows trajectory users to understand the differences in reliability between different trajectory segments, rather than simply obtaining a single trajectory line without quality differentiation. Rich query interfaces enable the database to efficiently support the differentiated data needs of various upper-level application scenarios such as path planning, airspace management, logistics traceability, and law enforcement evidence collection. Complete source segment identification and associated score records ensure that each location point in the fused trajectory is interpretable and auditable, meeting the stringent data traceability requirements of high-reliability application scenarios.

[0093] In one specific embodiment, the method of the present invention includes the following steps: (a) Trajectory segment receiving step: Receive target trajectory segments from multiple independent sensing nodes. The trajectory segments include the position sequence of the target in the local coordinate system or unified coordinate system of the sensing node, the corresponding timestamp sequence, and the type identifier of the sensing node; (b) Delay compensation step: Based on the known processing delay characteristics of each sensing node type, delay compensation is performed on the timestamp sequence of each trajectory segment to generate a time-delay compensated aligned timestamp sequence, so that the timestamps of trajectory points from different sensing nodes correspond to a unified physical time axis. (c) Cross-sensor same target identification step: For trajectory segments from different sensing nodes, jointly evaluate the association scores of at least two of the following dimensions: the continuity constraint score of the trajectory segment in time and space, and the similarity score of the motion state between the end and the beginning of the trajectory; based on the joint score, determine whether trajectory segments from different sources correspond to the same physical target; (d) Trajectory continuation steps: Multiple trajectory segments identified as the same target are continuated in chronological order, and the positional deviation at the seam is smoothed to generate a continuous global target trajectory across sensing nodes. (e) Quality scoring steps: Calculate the fusion quality confidence score for each time period of the global target trajectory. The confidence score is calculated based on at least the number of sensing nodes participating in the fusion and the correlation score of the trajectory segments, and is stored in the global trajectory record.

[0094] In step (b), the system corrects the sensor delay estimation during runtime in the following way: when two different types of sensing nodes simultaneously and effectively sense the same confirmed associated target, the sensing node with the smallest delay is used as the time reference, the actual delay deviation of other sensing nodes relative to the reference node is calculated, and the delay characteristics of the sensing node type are continuously updated in a moving average manner; the updated delay characteristics are used for delay compensation calculation of subsequent trajectory segments.

[0095] Step (c) also includes a multimodal feature similarity scoring dimension. For similar sensing nodes (such as camera to camera), the similarity of the target appearance feature vectors is calculated. For different types of sensing nodes (such as camera to LiDAR), the similarity is calculated separately according to the feature types supported by each sensor type and then weighted and integrated. When the sensing node type does not generate reliable feature vectors (such as wireless positioning beacons), the weight of the feature similarity score is reduced to a level that does not affect the association judgment.

[0096] In step (c), when multiple sensing nodes generate trajectory segments of multiple targets within the same time window, the system models the cross-sensor association problem as a global optimal allocation problem. With the goal of maximizing the sum of global association scores, and under the constraint that each trajectory segment can be associated with at most one trajectory segment of another sensor, the system uses the bipartite graph optimal matching algorithm to solve the global optimal association scheme.

[0097] In step (e), the fusion quality confidence score also includes an inference ratio factor. The inference ratio factor is calculated based on the proportion of trajectory points generated by trajectory interpolation, motion model extrapolation, or target merging in the time period to the total number of trajectory points in the time period. The higher the inference ratio, the lower the confidence score. The inferred segments and the directly observed segments are marked separately in the global trajectory record so that the trajectory user can independently access the quality information of each type of segment.

[0098] The method also includes a conflict arbitration step: when multiple sensing nodes provide position estimates for the same target (which has been confirmed to be associated) that exceed a preset deviation threshold at the same time, the physical reachability of each position estimate is first checked (based on the target type maximum speed constraint), and the weight of the position estimate that does not meet the physical constraint is reduced to zero; for multiple position estimates that pass the physical constraint check, the product of the type accuracy coefficient, geometric accuracy factor and time delay compensation accuracy factor of each sensing node is used as the arbitration weight to perform weighted position fusion.

[0099] The method also includes target splitting and merging processing steps: when multiple tracked targets are detected to continuously converge in space and enter a clustering state, the motion state of each target before clustering is recorded; during the clustering period, independent motion trajectory extrapolation is maintained based on the motion state of each target before clustering; when the targets are redispersed, the reassociation of the dispersed targets and the targets before clustering is prioritized based on the continuity of motion state rather than appearance features, and the trajectory segments during clustering are labeled as merging inference types.

[0100] After step (d) is completed, the system calculates the cryptographic hash value for each global trajectory record and links the hash values ​​of subsequent trajectory segments with the hash values ​​of previous trajectory segments in a chain to generate an immutable trajectory evidence chain. The evidence chain, together with the source sensing node identifier, delay compensation record and associated score, is stored together so that the authenticity of the trajectory can be independently verified by verifying the integrity of the hash chain.

[0101] The global trajectory database also provides the following query interfaces: query the complete trajectory and quality scores of each segment by global target ID; query all target trajectories passing through the region by time range and spatial region; filter by fusion quality confidence threshold to return only trajectory segments that meet the quality requirements; and query all global trajectories composed of trajectory segments of that node by source sensing node ID.

[0102] It is understood that this invention integrates spatiotemporal continuity constraints and motion state similarity into cross-sensor association through a three-dimensional joint scoring framework, which can effectively improve the accuracy of target association and address the failure problem of ReID, which relies solely on appearance features, in scenarios with drastic changes in viewpoint and similar target appearances. Simultaneously, it avoids the chain of errors caused by greedy strategies in dense multi-target scenarios by leveraging globally optimal association. Through precise modeling and real-time correction of time delay differences between heterogeneous sensors, it can eliminate the time axis misalignment problem caused by sensor delay, reducing the trajectory stitching error caused by delay from hundreds of milliseconds to the level of measurement error, significantly improving the trajectory accuracy of high-speed moving targets. Furthermore, through a conflict arbitration scheme combining physical constraint verification and precision weight arbitration, it can provide reliable support for heterogeneous sensor fusion, enabling multi-sensor fusion in various scenarios. When sensors produce conflicting estimates of the same target location, they output physically reasonable fusion results and rely on alarm mechanisms to identify sensor anomalies, enhancing the system's robustness in dealing with sensor failures. By constructing a segmented TQS quality confidence scoring system, it can support differentiated applications of trajectories. Trajectory users can filter low-quality trajectory segments according to scenario requirements, balancing the requirements of high-confidence trajectories in law enforcement and evidence collection scenarios with the need for trajectory continuity in navigation planning scenarios, thus achieving refined utilization of trajectory quality. By completely retaining the source sensing nodes, associated scores, time delay compensation amounts, and segment type labels for each segment in the global trajectory record, complete source traceability can be achieved, allowing each location point on the trajectory to be traced back to the corresponding sensing node and original timestamp, providing stable support for post-event auditing and quality backtracking.

[0103] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0104] In one embodiment, a multi-source target trajectory semantic fusion system based on a unified spatiotemporal coordinate system is provided, comprising: The trajectory segment receiving module is used to receive target trajectory segments output by multi-source heterogeneous sensing nodes; the trajectory segment includes the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system; The delay compensation module is used to compensate for the delay of the timestamp sequence of trajectory segments according to the processing delay characteristics of various types of sensing nodes, align the multi-source trajectory segments to a unified physical time axis, and determine the corresponding delay compensation accuracy based on the delay compensation results. The target association module is used to calculate the trajectory association score for trajectory segments of different sensing nodes that are aligned with the completion time. Based on the trajectory association score and the pre-set global optimal association rule, the module determines the trajectory segments that belong to the same physical target. The trajectory association score is obtained at least based on the evaluation of spatiotemporal consistency and motion state similarity. The global trajectory generation module is used to connect trajectory segments belonging to the same physical target in chronological order, and to smooth the position deviation and time gap at the trajectory joints based on the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory. The trajectory quality scoring module is used to calculate the fusion quality confidence score for global target trajectory segments based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio. The trajectory inference ratio is the proportion of the number of trajectory points generated by inference in the corresponding segment to the total number of trajectory points in the segment. The global trajectory output module is used to associate and store the global target trajectory with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A semantic fusion method for multi-source target trajectories based on a unified spatiotemporal coordinate system, characterized in that, The method includes: Receives target trajectory segments output by multi-source heterogeneous sensing nodes; the trajectory segments include the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system. Based on the processing delay characteristics of various types of sensing nodes, delay compensation is performed on the timestamp sequence of the trajectory segments, the multi-source trajectory segments are aligned to a unified physical time axis, and the corresponding delay compensation accuracy is determined based on the delay compensation results. A trajectory association score is calculated for trajectory segments of different sensing nodes that have completed time alignment. Based on the trajectory association score and a pre-set global optimal association rule, trajectory segments belonging to the same physical target are determined. The trajectory association score is obtained at least based on spatiotemporal consistency and motion state similarity assessment. Trajectory segments belonging to the same physical target are sequentially connected in time. The position deviation and time gap at the trajectory joint are smoothed based on the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory. Based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio, a fusion quality confidence score is calculated for each segment of the global target trajectory; the trajectory inference ratio is the proportion of the number of trajectory points generated by inference in the corresponding segment to the total number of trajectory points in the segment. The global target trajectory is associated with and stored with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

2. The method according to claim 1, characterized in that, The trajectory association score is also based on multimodal feature similarity assessment, which is a three-dimensional joint score calculated by spatiotemporal consistency, motion state similarity and multimodal feature similarity. Among them, the multimodal feature similarity of the same type of sensing nodes is calculated based on the cosine similarity of the target appearance feature vector, and the multimodal feature similarity of different type of sensing nodes is calculated based on the weighted comprehensive calculation after normalization of the corresponding feature types.

3. The method according to claim 1, characterized in that, The step of performing time delay compensation on the timestamp sequence of the trajectory segment based on the processing delay characteristics of various types of sensing nodes includes: When two different types of sensing nodes simultaneously perform effective sensing of the same confirmed associated target, the sensing node with the least latency is used as the reference node. Calculate the actual time delay deviation of other sensing nodes relative to the reference node, and continuously update the processing time delay characteristics of the corresponding sensing node type using a moving average method.

4. The method according to claim 1, characterized in that, The determination of trajectory segments belonging to the same physical target based on trajectory association scoring and pre-set global optimal association rules includes: Two different multi-source heterogeneous sensing nodes within the same time window and covering the target monitoring area are selected as the first sensing node and the second sensing node. Using all trajectory segments of the first sensing node as the matrix rows and all trajectory segments of the second sensing node as the matrix columns, the trajectory association score is calculated for each pair of trajectory segments corresponding to the row and column, and the matrix elements are filled to obtain the association candidate matrix. With the goal of maximizing the sum of global association scores, the bipartite graph optimal matching algorithm is used to solve the association candidate matrix to obtain the global optimal association scheme, and the pairing of trajectory segments belonging to the same physical target is determined. For each pair of trajectory segments in the global optimal association scheme, physical reachability is verified. If the physical reachability requirement is not met, the pairing is terminated, and the trajectory segments involved are marked as pending association.

5. The method according to claim 4, characterized in that, The association constraint condition limits each trajectory segment to be associated with at most one trajectory segment of another sensing node, while marking elements in the matrix whose trajectory association score is lower than a preset association threshold as invalid pairs.

6. The method according to claim 1, characterized in that, The smoothing process for position deviations and time gaps at trajectory seams based on the time-delay-compensated position sequence and timestamp sequence includes: If there is a time gap at the trajectory seam, and the time gap is less than the preset interpolation threshold, smooth interpolation is performed based on the motion state of the trajectory segments at both ends of the seam, the trajectory points at the seam are filled and the interpolation markers are marked. If there is a positional deviation at the trajectory seam, and the positional deviation is within the preset motion uncertainty range, the trajectory points in the seam area are smoothed to eliminate positional jumps and the smoothing adjustment amount is recorded. In the global target trajectory, for each seam position, the corresponding trajectory association score, position deviation, time gap, and interpolation or smoothing mark are recorded.

7. The method according to claim 1, characterized in that, The method further includes: When multiple sensing nodes provide location estimates for the same confirmed associated target that exceed a preset deviation threshold at the same time, the physical reachability of each location estimate is checked based on the maximum velocity constraint of the target type, and the arbitration weight of the location estimate that does not meet the physical reachability is reduced to zero. For the location estimates that pass the physical reachability test, weighted location fusion is performed based on the product of the type accuracy coefficient, geometric accuracy factor and time delay compensation accuracy factor of the corresponding sensing node as the arbitration weight.

8. The method according to claim 1, characterized in that, The method further includes: When the positions of multiple tracked targets continue to converge and the number of targets reported by the sensing nodes decreases, record the motion state and feature information of each target before they converge. During the period of target aggregation, independent trajectory extrapolation is maintained based on the motion state of each target before aggregation; After the target is redispersed, the dispersed target is re-associated with the tracked target before aggregation based on the continuity of motion state, and the trajectory segments during aggregation are labeled as merge inference types.

9. The method according to claim 1, characterized in that, After associating and storing the global target trajectory with the segmented fusion quality confidence score, the method further includes: Calculate the cryptographic hash value for each segment of the global target trajectory, and chain the hash values ​​of subsequent trajectory segments with the hash values ​​of previous trajectory segments to generate an immutable trajectory evidence chain and store it synchronously.

10. A multi-source target trajectory semantic fusion system based on a unified spatiotemporal coordinate system, characterized in that, The system includes: The trajectory segment receiving module is used to receive target trajectory segments output by multi-source heterogeneous sensing nodes; the trajectory segment includes the target's position sequence, timestamp sequence, sensing node type, and corresponding processing delay characteristics in a unified three-dimensional spatiotemporal coordinate system. The delay compensation module is used to perform delay compensation on the timestamp sequence of the trajectory segment according to the processing delay characteristics of each type of sensing node, align the multi-source trajectory segments to a unified physical time axis, and determine the corresponding delay compensation accuracy based on the delay compensation result. The target association module is used to calculate trajectory association scores for trajectory segments of different sensing nodes that are aligned with the completion time, and to determine trajectory segments belonging to the same physical target based on the trajectory association scores and pre-set global optimal association rules; the trajectory association scores are obtained at least based on spatiotemporal consistency and motion state similarity assessment; The global trajectory generation module is used to connect trajectory segments belonging to the same physical target in chronological order, and to smooth the position deviation and time gap at the trajectory joints based on the position sequence and timestamp sequence after time delay compensation, so as to generate a continuous and complete global target trajectory. The trajectory quality scoring module is used to calculate the fusion quality confidence score for the global target trajectory segments based on the number of sensing nodes involved in the fusion, the latency compensation accuracy, the trajectory association score, and the trajectory inference ratio; the trajectory inference ratio is the proportion of the number of trajectory points generated by inference in the corresponding segment to the total number of trajectory points in the segment. The global trajectory output module is used to associate and store the global target trajectory with the segmented fusion quality confidence score to obtain standardized multi-source target global semantic trajectory data.

Images