Multi-target real-time state tracking prediction method and system based on trajectory association

Abstract

The invention provides a multi-target real-time state tracking prediction method and system based on trajectory association, and relates to the technical field of multi-target tracking, and the method comprises the steps: collecting the motion data of a plurality of targets in real time through a multi-source sensor, and obtaining a target detection data set; performing feature analysis on the plurality of targets based on the target detection data set to obtain a plurality of target feature parameters, and performing trajectory association on the target detection data set according to the plurality of target feature parameters to generate a trajectory association result; state tracking is carried out based on a track association result, and a real-time target state is obtained; and predicting the plurality of targets according to the real-time target state, and outputting a multi-target state prediction result. According to the method and the device, the technical problems of inaccurate target state updating and even target loss caused by the fact that different targets are often difficult to effectively distinguish in multi-target tracking in the prior art and target misassociation or omission is easy to occur when interaction between the targets is relatively complex are solved.

Description

Technical Field

[0001] This invention relates to the field of multi-target tracking technology, and more specifically to a method and system for real-time state tracking and prediction of multiple targets based on trajectory association. Background Technology

[0002] The core task of multi-target tracking is to accurately identify, predict, and plan the states and behaviors of multiple targets through real-time data acquisition and analysis. However, ensuring high accuracy, real-time performance, and reliability remains a challenge for technological development, especially in complex dynamic environments. In multi-target scenarios, targets often exhibit similar motion patterns or trajectories. Particularly in dense environments, the movements of multiple targets may overlap. Existing multi-target tracking technologies often struggle to effectively distinguish between different targets, and when the interactions between targets are complex, mis-associations or omissions can easily occur, leading to inaccurate target state updates or even target loss, thus affecting the stability and reliability of the system. Summary of the Invention

[0003] This application provides a method and system for real-time state tracking and prediction of multiple targets based on trajectory association, which aims to solve the technical problems of existing multi-target tracking, which often has difficulty in effectively distinguishing different targets, and when the interaction between targets is complex, it is easy to misassociate or miss targets, which leads to inaccurate target state updates or even target loss.

[0004] The first aspect disclosed in this application provides a multi-target real-time state tracking and prediction method based on trajectory association. The method includes: acquiring motion data of multiple targets in real time through multi-source sensors to obtain a target detection dataset; performing feature analysis on the multiple targets based on the target detection dataset to obtain multiple target feature parameters; performing trajectory association on the target detection dataset according to the multiple target feature parameters to generate a trajectory association result; performing state tracking based on the trajectory association result to obtain the real-time target state; and predicting the multiple targets according to the real-time target state to output a multi-target state prediction result.

[0005] The second aspect disclosed in this application provides a multi-target real-time state tracking and prediction system based on trajectory association. The system is used in the aforementioned multi-target real-time state tracking and prediction method based on trajectory association. The system includes: a motion data acquisition module for acquiring motion data of multiple targets in real time using multi-source sensors to obtain a target detection dataset; a trajectory association module for performing feature analysis on the multiple targets based on the target detection dataset to obtain multiple target feature parameters, and performing trajectory association on the target detection dataset according to the multiple target feature parameters to generate a trajectory association result; a state tracking module for performing state tracking based on the trajectory association result to obtain the real-time target state; and a target prediction module for predicting the state of multiple targets based on the real-time target state and outputting a multi-target state prediction result.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects:

[0007] By acquiring motion data from multiple targets in real time using multi-source sensors, more comprehensive and accurate target motion information can be obtained using different types of sensors, thereby improving the reliability and accuracy of target detection data. Feature analysis of the target detection dataset extracts feature parameters from multiple targets, enabling precise description of the motion state of each target. Correlation processing of these feature parameters effectively matches data from different sensors, generating accurate trajectory association results. This process significantly reduces mismatches and data omissions between targets, ensuring accurate tracking of each target's trajectory. Real-time target state updates based on trajectory association results, employing efficient state tracking algorithms, allow for real-time updates of dynamic information such as the current position and velocity of each target based on historical data and real-time observations. This enables continuous target monitoring and exhibits good adaptability to environmental changes or sensor errors. Predicting the future behavior of multiple targets based on their real-time states is suitable for applications requiring future path planning or decision-making for multiple targets. By combining target motion patterns, the future position and state of targets can be predicted, and corresponding adjustments or warnings can be made based on the prediction results, enhancing the system's decision-making capabilities and ability to cope with complex dynamic environments.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1This is a schematic diagram of the process of a multi-target real-time state tracking and prediction method based on trajectory association provided in an embodiment of this application.

[0010] Figure 2 A schematic diagram of the structure of a multi-target real-time state tracking and prediction system based on trajectory association provided in an embodiment of this application.

[0011] Figure labeling: 10 motion data acquisition module, 20 trajectory association module, 30 state tracking module, 40 target prediction module. Detailed Implementation

[0012] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0013] This application provides a method and system for real-time state tracking and prediction of multiple targets based on trajectory association. It solves the technical problems of existing multi-target tracking, which often has difficulty in effectively distinguishing different targets and is prone to mis-association or omission of targets when the interaction between targets is complex, thus leading to inaccurate target state updates or even target loss.

[0014] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0015] Example 1, as Figure 1 As shown in the embodiments of this application, a multi-target real-time state tracking and prediction method based on trajectory association is provided, the method comprising:

[0016] A target detection dataset is obtained by collecting motion data of multiple targets in real time using multi-source sensors.

[0017] By interleaving multiple sensors, such as radar, infrared cameras, vision cameras, and lidar, within the monitoring area, a multimodal sensor network is formed to collect motion data of multiple targets in real time. By setting a unified time base and trigger clock parameters, when a target enters the monitoring area, the multimodal sensor network sends a collection trigger signal based on the clock parameters. Each sensor synchronously collects data according to the trigger signal, forming a spatiotemporally aligned target detection dataset. The data includes information such as motion trajectory, target position, speed, and appearance.

[0018] Based on the target detection dataset, feature analysis is performed on multiple targets to obtain multiple target feature parameters. Trajectory association is then performed on the target detection dataset according to the multiple target feature parameters to generate trajectory association results.

[0019] For each target in the target detection dataset, feature extraction is performed. Computer vision, signal processing, and other techniques are used to extract feature parameters, forming a feature description for each target. These feature parameters include motion parameters (such as velocity and acceleration), appearance parameters (such as image features and color information), and spatiotemporal features (such as the target's trajectory). Based on these feature parameters, a trajectory association algorithm is used to associate the targets. The goal of trajectory association is to determine the consistency and continuity of the target at different time points by comparing the features of historical trajectories and the current target. By calculating the feature similarity of the targets, the detected targets are associated with historical trajectories, thereby generating trajectory association results.

[0020] Based on the trajectory association results, state tracking is performed to obtain the real-time target state.

[0021] A state tracking filter is deployed for each target's trajectory. Commonly used filtering algorithms include Kalman filtering or particle filtering. These filters can update the target's state based on the target's motion model and detection data. The matching status of the monitored trajectory with multiple targets in the trajectory association results is assessed. If multiple monitored trajectories match a target, the target's trajectory is valid and tracking can continue. If no matching occurs, it indicates the trajectory is lost or the target has left the monitoring area. In this case, the target's historical motion trajectory can be extracted, and its current state can be inferred using prediction methods. The state tracking filter updates the observed target data to obtain the real-time target state, including information such as the target's position, velocity, and acceleration.

[0022] Based on the real-time target state, multiple targets are predicted, and the multi-target state prediction result is output.

[0023] Based on the real-time target state, motion pattern analysis is performed on each target. Targets may have different motion patterns, such as linear motion and circular motion. By analyzing the historical motion state of the targets, motion pattern features are extracted. Based on the motion pattern features of each target, a kinematic model is used to predict the future state of the target. The prediction includes the target's position, velocity, acceleration, etc. In a multi-target scenario, the spatial interaction and coupling relationship between targets are analyzed based on the motion pattern features of the targets. Targets may have interactive behaviors, such as following and avoidance. Based on the interaction influence between targets, the predicted state of each target is corrected. By combining the target state prediction sequence and interaction information, the predicted trajectory of the targets is adjusted, and a more accurate multi-target state prediction result is output.

[0024] Furthermore, by acquiring motion data of multiple targets in real time using multi-source sensors to obtain a target detection dataset, the methods include:

[0025] A multi-source sensor network is deployed to cover the monitoring area using a staggered layout. A time base is set for the multi-source sensor network, and trigger clock parameters are configured. When a target enters the monitoring area, a data acquisition trigger signal is generated based on the trigger clock parameters. The multi-source sensor network is synchronously triggered to acquire data at a sampling frequency according to the acquisition trigger signal to obtain multimodal detection data. The multimodal detection data is then aligned to generate a target detection dataset.

[0026] Within the monitored area, the sensors are deployed in a staggered layout. This layout involves placing multiple sensors at certain intervals and angles in different locations within the monitored area to ensure comprehensive coverage of the target. The main advantages of a staggered layout are that it reduces blind spots, improves the accuracy and coverage of data collection, and maximizes the probability of target detection by rationally selecting the location and number of sensors. The deployed multi-source sensor network includes multiple different types of sensors to ensure multi-angle and multi-dimensional perception of the target, specifically including radar, infrared cameras, visual cameras, and lidar.

[0027] To ensure that a multi-source sensor network can acquire data on the same time scale, a unified time reference is first established. This is typically achieved by configuring a time synchronization mechanism for each sensor. The time reference ensures consistency in cooperation between different sensors; all sensors synchronize according to this unified time reference when acquiring data. Configuring trigger clock parameters enables the multi-source sensors to trigger data acquisition at predetermined time intervals. The trigger clock is a key parameter controlling the data acquisition frequency and is usually configured based on the size of the monitored area, the target's movement speed, and the required monitoring accuracy to ensure strict time alignment of the data.

[0028] Physical detection is performed using radar and other means. When a target is detected entering the monitoring area, a data acquisition trigger signal is generated based on the set clock parameters. The data acquisition trigger signal is the mechanism for starting data acquisition. It can be triggered by setting a threshold, such as when the target moves to a specific location, and ensures that data acquisition can be carried out at the accurate time.

[0029] After the trigger signal is issued, all deployed multi-source sensors begin data acquisition according to the configured sampling frequency. The multi-source sensor network operates synchronously based on a unified time reference, ensuring that the acquired data is correlated and synchronized. Different sensors acquire different types of data based on their different operating principles; for example, vision sensors acquire image and video data of the target, while radar acquires the target's motion information, such as speed and direction.

[0030] Multimodal detection data comes from different types of sensors, so spatiotemporal alignment of this data is necessary. The spatiotemporal alignment process involves synchronizing information such as the spatial location and timestamps of the target, so that data from different sensors can accurately reflect the state of the same target at the same time. The aligned data are then merged to form a complete target detection dataset. This dataset includes information such as the trajectory, position, velocity, and appearance of each target within the monitored area, providing a foundation for subsequent target state tracking and feature analysis.

[0031] Furthermore, based on the target detection dataset, feature analysis is performed on multiple targets to obtain multiple target feature parameters. Trajectory association is then performed on the target detection dataset according to the multiple target feature parameters to generate trajectory association results. The method includes:

[0032] Based on the target detection dataset, multi-feature analysis is performed to extract multiple feature parameters for each target; based on the multiple feature parameters, the feature similarity of multiple targets is calculated to construct a multi-dimensional similarity matrix; a multi-feature fusion strategy is used to perform weighted fusion of the multi-dimensional similarity matrix to generate a comprehensive similarity matrix; based on the comprehensive similarity matrix, association solving is performed to generate association matching results; based on the association matching results, multiple targets are associated and matched with historical trajectories to generate the trajectory association results.

[0033] For each target, multi-dimensional features are extracted, including: motion features such as velocity, acceleration, and trajectory direction, which help describe the target's motion state; appearance features such as shape, color, texture, and size, which can be extracted from image data provided by the visual sensor; and spatiotemporal features such as spatial location and timestamps, which help describe the target's time-series behavior and positional changes at different points in time. Based on the target detection dataset, feature analysis is performed on each target to extract motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters.

[0034] The extracted feature parameters are calculated to generate the feature similarity between each pair of targets. For example, cosine similarity is used to evaluate the similarity between targets by calculating the angle between feature vectors. The similarity values ​​of each pair of targets are organized into a matrix form, called a multi-dimensional similarity matrix. The rows and columns of this matrix represent different targets, and each element of the matrix represents the similarity between two targets.

[0035] Since different features contribute differently to target identification and association, the purpose of multi-feature fusion is to generate a comprehensive similarity assessment by weighting the similarity of these features. Multi-feature fusion strategies can employ weighted averaging methods to handle the importance of different features; for example, motion features can be given higher weights because they are more crucial in target association. The similarity of each feature is combined using weighted fusion techniques to obtain a weighted comprehensive similarity matrix, where each element reflects the target similarity after combining multiple features.

[0036] Based on the fused comprehensive similarity matrix, matching algorithms such as the Hungarian algorithm, nearest neighbor algorithm, and graph theory algorithms are used to solve the target association problem. These algorithms can find the optimal matching scheme, i.e., which target is most similar to the historical trajectory. Among them, the Hungarian algorithm is a classic method for solving the maximum matching problem in a bipartite graph, and is suitable for the target-trajectory association problem. The current target is matched with historical trajectories, which are previously recorded target movement trajectories. By comparing the similarity between the current target and historical trajectories, the most suitable trajectory is selected for association. If a target has a high similarity with multiple historical trajectories, the most suitable trajectory is selected for matching, or selection is based on the chronological order of the time series. After completing the association between targets and historical trajectories, trajectory association results are generated, representing the matching relationship between each target and its corresponding historical trajectory. This provides data support for subsequent state tracking, prediction, and behavior analysis.

[0037] Furthermore, based on the aforementioned multiple feature parameters, the feature similarity of multiple targets is calculated to construct a multi-dimensional similarity matrix. The method includes:

[0038] A set of feature parameters for multiple targets is obtained, including motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters. Based on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters, motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity of the multiple targets are calculated. The motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity are normalized to obtain a normalization result. Based on the normalization result, adaptive weight allocation is performed on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters to obtain multiple weight coefficients. Based on the multiple weight coefficients, the motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity are weighted and fused to generate a comprehensive similarity score, thus constructing the multi-dimensional similarity matrix.

[0039] Spatial distance similarity is calculated based on the target's location information, with higher similarity as the distance is closer; velocity matching similarity is calculated based on the target's motion velocity, with higher similarity as the magnitude and direction of the velocity are closer; acceleration consistency similarity is calculated based on the target's motion acceleration, with higher similarity as the acceleration patterns are more consistent; and motion feature similarity is generated by combining spatial distance similarity, velocity matching similarity, and acceleration consistency similarity.

[0040] Extract the depth appearance feature vectors of the target and calculate the cosine similarity between the feature vectors; analyze the color distribution characteristics of the target and calculate the overlap of the color histograms; compare the shape features of the target, including the similarity of the bounding rectangle size and aspect ratio. Fuse the depth feature similarity, color feature similarity, and shape feature similarity to generate an appearance feature similarity score.

[0041] The analysis process involves analyzing the temporal continuity of target appearance and calculating time interval similarity; examining the spatial consistency of target trajectories and calculating trajectory shape similarity; and assessing the spatiotemporal correlation of target movement patterns and calculating behavioral pattern similarity. Finally, combining temporal, spatial, and behavioral characteristics, a spatiotemporal feature similarity is generated.

[0042] To ensure that the similarity values ​​of each feature are compared under the same dimension, it is necessary to normalize the similarity of motion features, appearance features, and spatiotemporal features. For example, the min-max normalization method can be used to compress the similarity values ​​of different features into a common range, usually [0,1]. This can avoid the influence of a feature value that is too large or too small on the final result.

[0043] By using historical data or previous experience, the weights of each feature can be set. For example, in target tracking, motion features are more important than appearance features, so motion features are assigned a higher weight. The feature weights can be automatically assigned through intelligent methods to ensure that the most relevant features dominate the final similarity calculation.

[0044] Based on the obtained multiple weight coefficients, the similarity values ​​of each feature are weighted and fused. The goal of the weighted fusion is to comprehensively derive the final similarity score between targets based on the importance of different features. The comprehensive similarity values ​​between each pair of targets form a multi-dimensional similarity matrix. Each element of this matrix represents the comprehensive similarity between the target pair. Each row or column in the matrix represents a target, and each element in the matrix is ​​the similarity between that target and other targets.

[0045] Furthermore, based on the trajectory association results, state tracking is performed to obtain the real-time target state, and the method includes:

[0046] The trajectory association results are iterated through, and a state tracking filter is deployed for each trajectory. It is determined whether there is an association between multiple monitoring trajectories and multiple targets in the trajectory association results. When multiple monitoring trajectories are associated with multiple targets, the trajectory association is successful, and multiple target observation data are obtained. The state tracking filter is updated according to the multiple target observation data to obtain the real-time target state. When multiple monitoring trajectories are not associated with multiple targets, the trajectory is lost, and the historical motion trajectories of multiple targets are extracted for state prediction to obtain the real-time target state.

[0047] For each trajectory associated with the target, a corresponding state tracking filter is deployed. The purpose of the state tracking filter is to estimate the target's motion state during multiple observations and updates, and to continuously track the target's current position, velocity, acceleration, and other information. Commonly used filters include Kalman filters and particle filters. The specific choice depends on the nonlinearity and noise characteristics of the system. By updating the target's state in real time through the state tracking filter, the target's state at the next moment can be dynamically estimated.

[0048] Based on a multi-dimensional similarity matrix, it is determined whether the current multiple monitoring trajectories match the trajectories of multiple targets. Target trajectory association is based on similarity calculation results, determining whether the current trajectory is a continuation of previously tracked targets. The association criteria include the target's position, velocity, and appearance features. If the target's trajectory successfully matches the current monitoring trajectory, i.e., the similarity exceeds a set threshold, then the target is considered successfully associated with the monitoring trajectory. Successful association means that the target's motion state can be updated based on these observation data. When association is successful, the latest target observation data is obtained, including the target's spatial position, velocity, acceleration, and other motion states.

[0049] The state tracking filter is updated based on multiple target observations. State updates involve adjusting the target's motion state; for example, updating the position state based on the latest position observations, and updating the target's velocity and acceleration using a kinematic model or filter. If a Kalman filter is used for state tracking, the update process involves a prediction step and an update step: in the prediction step, the state at the next time step is predicted based on the current target state and the motion model; in the update step, the new observation data is combined with the prediction results to update the target state. The filter performs weighted fusion based on the observation data and model uncertainties to optimize the target's state estimate.

[0050] If the target's monitoring trajectory cannot be successfully matched with its historical trajectory, i.e., the association fails, the target trajectory is considered lost. When trajectory loss occurs, the target's state is predicted based on its historical motion trajectory. The target's historical motion trajectory is usually extracted from the observation data of the previous few frames. Based on the position, velocity, acceleration and other information in the historical trajectory, the state prediction algorithm is used to estimate the target's current state and obtain the real-time target state.

[0051] Furthermore, the method for predicting multiple targets based on the real-time target state and outputting multi-target state prediction results includes:

[0052] Based on the real-time target state, motion analysis is performed on multiple targets to obtain multiple motion pattern features; motion prediction is performed on multiple targets according to the multiple motion pattern features to obtain a state prediction sequence for multiple targets; interaction analysis is performed on multiple targets according to the multiple motion pattern features to generate multiple target interaction information; interaction influence calculation is performed based on the multiple target interaction information and the state prediction sequence to obtain multiple target state interaction influence coefficients; the state prediction sequence for multiple targets is corrected according to the multiple target state interaction influence coefficients, and the multi-target state prediction result is output.

[0053] Based on the real-time target status, motion analysis is performed on each target to extract its motion pattern features, including: motion trajectory features, such as direction and shape (e.g., straight line, curve, turning); velocity and acceleration features, identifying whether the target is in a uniform, accelerating, or decelerating state using real-time velocity and acceleration information; and motion patterns, such as linear, nonlinear, or periodic. This step extracts motion features using mathematical models, including fitted curves and acceleration models.

[0054] Based on the extracted motion pattern features, kinematic or dynamic models are applied to predict the future motion of each target. Prediction methods include prediction based on physical motion models (such as uniform velocity and uniform acceleration models), Kalman filtering, particle filtering, etc. These models use the target's motion state and motion pattern features to infer the target's future position and state.

[0055] In multi-target scenarios, targets may interact with each other, such as avoidance, following, and collision. Analyzing the interaction between targets involves analyzing their relative positional relationships. Based on the target's motion pattern characteristics, it is determined whether the target interacts with other targets, such as chasing or avoiding. By calculating the relative speed, acceleration, and path intersection points between targets, the motion coordination of the targets is analyzed, and cluster behaviors among multiple targets are identified, such as multi-target cooperative motion and avoidance behavior, thereby generating corresponding target interaction information.

[0056] By integrating target interaction information and state prediction sequences between targets, the interaction effects between targets are calculated. Specifically, based on the spatial relationship, velocity difference, acceleration difference, etc. between targets, an influence coefficient function is designed to quantify the interaction effects between targets. Common influence measurement methods include distance-based functions and velocity difference weighted functions. Based on the quantification results of the above interaction effects, the target state interaction influence coefficient of each target on other targets is generated as the basis for further prediction and correction.

[0057] Based on the target state interaction coefficient, the state prediction sequence of each target is corrected. For example, if two targets are very close and there is an interaction effect of avoidance behavior, the trajectory or predicted speed of one of the targets needs to be adjusted. According to the magnitude of the target state interaction coefficient, a weighted correction is made. The target with a larger target state interaction coefficient has a greater impact on the state prediction, and the state prediction is corrected more significantly accordingly. The corrected state prediction sequence is used as the final multi-target state prediction result.

[0058] Furthermore, the method for generating multiple target interaction information by performing interaction analysis on multiple targets based on the aforementioned multiple motion pattern features includes:

[0059] Based on the multiple motion pattern features, spatial interaction recognition is performed on multiple targets to extract interaction group information; based on the multiple motion pattern features, coupling analysis is performed on multiple targets, and the interaction group information is matched according to the coupling coefficient to extract interaction following relationships; based on the interaction group information, interaction overlap analysis is performed on multiple targets to identify interaction overlap behaviors and perform interaction coordination identification; the interaction coordination identification and the interaction following relationships are integrated into the interaction group information to associate multiple targets and generate the interaction information of the multiple targets.

[0060] The relative positions between targets are calculated based on the motion pattern characteristics of each target. Using this information, it is possible to identify which targets are in the same area and may interact spatially. By analyzing the spatial position, velocity vector and other characteristics of the targets, interaction groups are defined. If the trajectories of multiple targets overlap or are close and their motion behaviors are similar, they can be grouped into one interaction group. Based on spatial proximity and similarity of motion patterns, targets are divided into different interaction groups. Each interaction group represents a cluster of targets that are moving in coordination or avoiding each other.

[0061] Coupling analysis is performed on the motion patterns of targets. This involves calculating the relative motion relationships (e.g., the degree of matching in velocity and acceleration) between targets to determine whether they exhibit cooperative motion or mutual following behavior. Specifically, a coupling coefficient can be used to measure the mutual influence between targets. For example, if two targets have similar motion directions and velocities, their coupling coefficient is relatively large, indicating a possible following relationship. Within an interactive group, targets may exhibit some degree of mutual following; for instance, one target may follow the trajectory of another, especially in traffic flow. Based on coupling analysis, following relationships between targets are identified, such as target A consistently maintaining a certain distance from target B. Based on the coupling coefficient, the interactive group is traversed, matching the interaction relationships between different targets to extract each pair of targets with a following relationship.

[0062] In an interactive group, the trajectories of different targets may overlap, especially at the intersection of their paths. Interactive overlap refers to multiple targets overlapping or approaching each other in time or space, which may trigger collisions, avoidance, or other interactive behaviors. Based on the targets' trajectories and current spatial positions, identifying which targets have overlapping trajectories can be done by calculating indicators such as changes in distance and velocity between targets. When overlapping behavior is identified, the interactive coordination behavior between the targets is marked.

[0063] By integrating information such as interactive following relationships and interactive coordination identifiers, complete target interaction information is generated. This target interaction information not only describes the target's movement trajectory but also reflects the interactive behaviors between targets, such as whether there is following, avoidance, or cooperation.

[0064] Example 2 is based on the same inventive concept as the trajectory-association-based multi-target real-time state tracking and prediction method in the previous examples, such as... Figure 2 As shown in the embodiment of this application, a multi-target real-time state tracking and prediction system based on trajectory association is provided. The system includes:

[0065] The motion data acquisition module 10 is used to acquire motion data of multiple targets in real time through multi-source sensors to obtain a target detection dataset; the trajectory association module 20 is used to perform feature analysis on multiple targets based on the target detection dataset to obtain multiple target feature parameters, and perform trajectory association on the target detection dataset according to the multiple target feature parameters to generate trajectory association results; the state tracking module 30 is used to perform state tracking based on the trajectory association results to obtain the real-time target state; and the target prediction module 40 is used to predict multiple targets based on the real-time target state and output multi-target state prediction results.

[0066] Furthermore, the motion data acquisition module 10 is used to perform the following operation steps:

[0067] A multi-source sensor network is deployed to cover the monitoring area using a staggered layout. A time base is set for the multi-source sensor network, and trigger clock parameters are configured. When a target enters the monitoring area, a data acquisition trigger signal is generated based on the trigger clock parameters. The multi-source sensor network is synchronously triggered to acquire data at a sampling frequency according to the acquisition trigger signal to obtain multimodal detection data. The multimodal detection data is then aligned to generate a target detection dataset.

[0068] Furthermore, the trajectory association module 20 is used to perform the following operation steps:

[0069] Based on the target detection dataset, multi-feature analysis is performed to extract multiple feature parameters for each target; based on the multiple feature parameters, the feature similarity of multiple targets is calculated to construct a multi-dimensional similarity matrix; a multi-feature fusion strategy is used to perform weighted fusion of the multi-dimensional similarity matrix to generate a comprehensive similarity matrix; based on the comprehensive similarity matrix, association solving is performed to generate association matching results; based on the association matching results, multiple targets are associated and matched with historical trajectories to generate the trajectory association results.

[0070] Furthermore, the trajectory association module 20 is used to perform the following operation steps:

[0071] A set of feature parameters for multiple targets is obtained, including motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters. Based on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters, motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity of the multiple targets are calculated. The motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity are normalized to obtain a normalization result. Based on the normalization result, adaptive weight allocation is performed on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters to obtain multiple weight coefficients. Based on the multiple weight coefficients, the motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity are weighted and fused to generate a comprehensive similarity score, thus constructing the multi-dimensional similarity matrix.

[0072] Furthermore, the state tracking module 30 is used to perform the following operation steps:

[0073] The trajectory association results are iterated through, and a state tracking filter is deployed for each trajectory. It is determined whether there is an association between multiple monitoring trajectories and multiple targets in the trajectory association results. When multiple monitoring trajectories are associated with multiple targets, the trajectory association is successful, and multiple target observation data are obtained. The state tracking filter is updated according to the multiple target observation data to obtain the real-time target state. When multiple monitoring trajectories are not associated with multiple targets, the trajectory is lost, and the historical motion trajectories of multiple targets are extracted for state prediction to obtain the real-time target state.

[0074] Furthermore, the target prediction module 40 is used to perform the following operation steps:

[0075] Based on the real-time target state, motion analysis is performed on multiple targets to obtain multiple motion pattern features; motion prediction is performed on multiple targets according to the multiple motion pattern features to obtain a state prediction sequence for multiple targets; interaction analysis is performed on multiple targets according to the multiple motion pattern features to generate multiple target interaction information; interaction influence calculation is performed based on the multiple target interaction information and the state prediction sequence to obtain multiple target state interaction influence coefficients; the state prediction sequence for multiple targets is corrected according to the multiple target state interaction influence coefficients, and the multi-target state prediction result is output.

[0076] Furthermore, the target prediction module 40 is used to perform the following operation steps:

[0077] Based on the multiple motion pattern features, spatial interaction recognition is performed on multiple targets to extract interaction group information; based on the multiple motion pattern features, coupling analysis is performed on multiple targets, and the interaction group information is matched according to the coupling coefficient to extract interaction following relationships; based on the interaction group information, interaction overlap analysis is performed on multiple targets to identify interaction overlap behaviors and perform interaction coordination identification; the interaction coordination identification and the interaction following relationships are integrated into the interaction group information to associate multiple targets and generate the interaction information of the multiple targets.

[0078] Through the foregoing detailed description of the multi-target real-time state tracking and prediction method based on trajectory association, those skilled in the art can clearly understand the multi-target real-time state tracking and prediction system based on trajectory association in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section.

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A multi-target real-time state tracking and prediction method based on trajectory association, characterized in that, The method includes: A target detection dataset is obtained by collecting motion data of multiple targets in real time using multi-source sensors. Based on the target detection dataset, feature analysis is performed on multiple targets to obtain multiple target feature parameters. Trajectory association is then performed on the target detection dataset according to the multiple target feature parameters to generate trajectory association results. Based on the trajectory association results, state tracking is performed to obtain the real-time target state; Based on the real-time target state, multiple targets are predicted, and the multi-target state prediction result is output.

2. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 1, characterized in that, A target detection dataset is obtained by acquiring motion data of multiple targets in real time using multi-source sensors. The methods include: A multi-source sensor network is deployed to cover the monitoring area using a staggered layout. A time base is set for the multi-source sensor network, and trigger clock parameters are configured. When a target enters the monitoring area, a data acquisition trigger signal is generated based on the clock parameters. The multi-source sensor network is synchronously triggered according to the acquisition trigger signal to acquire data at the sampling frequency, thereby obtaining multimodal detection data; The multimodal detection data is aligned to generate a target detection dataset.

3. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 1, characterized in that, Based on the target detection dataset, feature analysis is performed on multiple targets to obtain multiple target feature parameters. Trajectory association is then performed on the target detection dataset according to the multiple target feature parameters to generate trajectory association results. The method includes: Multi-feature analysis is performed based on the target detection dataset to extract multiple feature parameters for each target; Based on the aforementioned multiple feature parameters, the feature similarity of multiple targets is calculated, and a multi-dimensional similarity matrix is ​​constructed. A multi-feature fusion strategy is used to weight and fuse the multi-dimensional similarity matrix to generate a comprehensive similarity matrix; Based on the comprehensive similarity matrix, the association is solved to generate the association matching result. Based on the association matching result, multiple targets are associated and matched with historical trajectories to generate the trajectory association result.

4. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 3, characterized in that, The method for calculating the feature similarity of multiple targets based on the aforementioned feature parameters and constructing a multi-dimensional similarity matrix includes: Obtain a set of feature parameters for multiple targets, including motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters; Based on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters, the motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity of multiple targets are calculated. The motion feature similarity, appearance feature similarity, and spatiotemporal feature similarity are normalized to obtain the normalization result. Based on the normalization result, adaptive weight allocation is performed on the motion feature parameters, appearance feature parameters, and spatiotemporal feature parameters to obtain multiple weight coefficients; The similarity of motion features, appearance features, and spatiotemporal features is weighted and fused based on the multiple weight coefficients to generate a comprehensive similarity score, and the multi-dimensional similarity matrix is ​​constructed.

5. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 1, characterized in that, Based on the trajectory association results, state tracking is performed to obtain the real-time target state. The method includes: The trajectory association results are iterated through, and a state tracking filter is deployed for each trajectory. Determine whether there is a correlation between multiple monitoring trajectories and multiple targets in the trajectory association results. If there is a correlation between the multiple monitoring trajectories and the multiple targets, the trajectory association is successful and observation data of multiple targets are obtained. The state tracking filter is updated based on the multiple target observation data to obtain the real-time target state; When the multiple monitoring trajectories are not associated with the multiple targets, the trajectories are lost. The historical motion trajectories of the multiple targets are extracted for state prediction to obtain the real-time target state.

6. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 1, characterized in that, The method includes predicting multiple targets based on the real-time target state and outputting the multi-target state prediction result. Based on the real-time target state, motion analysis is performed on multiple targets to obtain multiple motion pattern features; Based on the aforementioned multiple motion pattern features, motion prediction is performed on multiple targets to obtain a state prediction sequence for multiple targets; Based on the characteristics of the multiple motion patterns, the interaction of multiple targets is analyzed to generate multiple target interaction information; Based on the interaction information of the multiple targets and the state prediction sequence, the interaction influence is calculated to obtain the interaction influence coefficient of the multiple target states. The state prediction sequence of multiple targets is corrected according to the interaction influence coefficient of the multiple target states, and the multi-target state prediction result is output.

7. The multi-target real-time state tracking and prediction method based on trajectory association as described in claim 6, characterized in that, The method involves performing interaction analysis on multiple targets based on the aforementioned multiple motion pattern features to generate interaction information for multiple targets, including: Based on the aforementioned multiple motion pattern features, spatial interaction recognition is performed on multiple targets to extract interaction group information; Based on the multiple motion pattern features, a coupling analysis is performed on multiple targets, and the interaction group information is traversed and matched according to the coupling coefficient to extract the interaction following relationship. Based on the interactive group information, perform interaction overlap analysis on multiple targets, identify interactive overlap behaviors, and perform interaction coordination identification. The interaction coordination identifier and the interaction follow relationship are integrated into the interaction group information to associate multiple targets and generate the interaction information of the multiple targets.

8. A multi-target real-time state tracking and prediction system based on trajectory association, characterized in that, The system for implementing the multi-target real-time state tracking and prediction method based on trajectory association as described in any one of claims 1-7, the system comprising: The motion data acquisition module is used to acquire motion data of multiple targets in real time through multi-source sensors to obtain a target detection dataset. The trajectory association module is used to perform feature analysis on multiple targets based on the target detection dataset, obtain multiple target feature parameters, and perform trajectory association on the target detection dataset according to the multiple target feature parameters to generate trajectory association results; The status tracking module is used to track the status based on the trajectory association results and obtain the real-time target status. The target prediction module is used to predict multiple targets based on the real-time target status and output the multi-target status prediction result.

Images