A multi-sensor fusion intelligent moving target tracking method

Abstract

Intelligent motion target tracking technology provides intelligent support for personal fitness training by analyzing and monitoring the user's movement status in real time. Current research mainly focuses on methods utilizing inertial measurement units and optical sensors, as well as the application of virtual reality and augmented reality technologies. However, these technologies have some limitations, such as challenges in accurately capturing complex movements and protecting privacy. To address these issues, this invention proposes a multi-sensor fusion intelligent motion target tracking technology. This technology utilizes multiple millimeter-wave sensors for refined analysis and monitoring of moving targets, providing accurate motion data and feedback.

Description

Technical Field

[0001] This invention belongs to the field of intelligent moving target tracking, and more specifically, it relates to an intelligent moving target tracking method that integrates multiple sensors. Background Technology

[0002] Intelligent motion target tracking technology can help users understand their own exercise status and progress, providing real-time suggestions and movement corrections to avoid errors and injuries during exercise. This allows users to conduct personal fitness training more intelligently and efficiently, improving their health and quality of life. Currently, intelligent motion target tracking technology is in a stage of continuous research and development. For example, many existing products use a combination of inertial measurement units (IMUs) and optical sensors to capture and analyze user movements. Furthermore, the development of virtual reality (VR) and augmented reality (AR) technologies has provided richer interactive and feedback methods for intelligent fitness target tracking. However, existing intelligent motion target tracking technologies still have some shortcomings. First, for complex movements or high-intensity exercise, tracking technology using IMUs and optical sensors may not be able to accurately capture and track every subtle movement. Second, tracking technologies using video or AR often involve the collection and storage of sensitive personal data, posing a serious challenge to user privacy.

[0003] To address this, this invention proposes a multi-sensor fusion intelligent moving target tracking method. This method utilizes multiple millimeter-wave sensors to track moving targets, enabling refined analysis and monitoring of the targets and providing users with accurate motion data and feedback. Compared to traditional tracking technologies, millimeter-wave sensor technology primarily relies on the reflection and echo of wireless signals, avoiding direct acquisition of user images or personal identification information, thus effectively protecting user privacy. Furthermore, millimeter-wave sensors possess high ranging and angular resolution, enabling precise capture of subtle motion details and attitude changes, providing more accurate motion analysis and tracking. Summary of the Invention

[0004] The present invention aims to provide an intelligent moving target tracking method based on multi-sensor fusion. The technical solution is as follows:

[0005] S1. Deploy two millimeter-wave sensors in a motion scene to track the same moving target. Each sensor periodically sends its sensed position trajectory data and corresponding timestamp information to the fusion center. For ease of representation, use... The trajectory from sensor 1 contains a sequence of target position estimates. Representative from sensor The trajectories generated by each millimeter-wave sensor may overlap in time or not, and the lengths of the trajectories generated by different sensors may not be the same.

[0006] S2. Time-align the position trajectory information sensed by different millimeter-wave sensors to make the trajectory Each location estimate is related to the trajectory The location estimation is correlated with minimizing the time difference between two acquisition times.

[0007] S202, the trajectory of millimeter-wave sensor 1 Each position is estimated as a reference to find the trajectory. The position estimate with the smallest time difference is if in If no corresponding element is found within a second (the time step of fusion), the position estimate is discarded.

[0008] S204: Reduce two trajectories by step S202. and arrive A time-aligned position sequence, respectively using and This represents the time-aligned position index sequence of sensor 1 and sensor 2, which have the same length.

[0009] S206: In step S204, by tracing... and Time alignment was performed, resulting in a length of Time-aligned position index sequence and , Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. The difference between timestamps at other locations within the timeframe. Similarly, Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. The difference between timestamps at other locations is calculated. The average time deviation of the two sensor position sequences after alignment is then used to calculate the difference. This is used to evaluate the alignment quality of the two trajectories, and the specific calculation is as follows:

[0010]

[0011] S3. By considering three factors—trajectory length, time alignment, and superposition quality—the association cost of trajectory pairs is calculated, and the Hungarian algorithm is used to select the trajectory pairs with the lowest cost, forming an associated trajectory set. Based on a set cost threshold, trajectory pairs with association costs below the threshold are selected as trajectory pairs for the same moving target and included in the final associated set.

[0012] S302, taking into account three factors—track length, track time alignment, and the quality of track overlay on different sensors—for each pair of tracks... The related costs are calculated as follows:

[0013]

[0014] in, It is the sum of the least-squares residuals after time alignment and rigid body transformation, and It is a balancing factor that favors trajectory pairs with longer overlap and lower average time offset.

[0015] S304. Combine the calculated cost values ​​into a... The cost matrix, where This represents the number of sensing location trajectories collected by millimeter-wave sensor 1, while This represents the number of sensing location trajectories collected by millimeter-wave sensor 2. Using the Hungarian algorithm, the optimal correlation of the trajectories is obtained by minimizing the overall cost. (express The smaller value in the associated trajectory implies the same moving target tracked by the two millimeter-wave sensors.

[0016] S306. Due to the presence of false trajectories, ghost targets, and clutter, a cost threshold is set. For trajectory pairs whose association cost is lower than the cost threshold calculated in step S302, they are considered as trajectory pairs generated by tracking the same moving target and are selected into the final association set. Among them Indicates source from millimeter-wave sensor The A selected trajectory.

[0017] S4. By stacking and calculating the association cost of the sensing position trajectory pairs of sensor 1 and sensor 2, the trajectory pair with the lowest association cost is selected and used to solve the self-calibration parameters of multi-sensor fusion, including the rotation matrix and translation vector.

[0018] S402, Use In step S306 The set of all possible non-empty subsets. Each element is a set of perceived position trajectory pairs from sensor 1 and sensor 2. Each element of the vector stacks all trajectories from sensor 1. Stack all trajectories from sensor 2 into a vector. Then, the same operation is performed on the corresponding timestamp sequence. Finally, the associated cost is calculated according to step S302. And store all associated costs in a single dimension. In the matrix.

[0019] S404, from Select the element with the lowest associated cost; this minimum cost element contains... The trajectory pairs will be used to calculate the self-calibration parameters for multi-sensor fusion in subsequent calculations.

[0020] S406, Selected in step S404 A pair of trajectories are stacked together, represented as A rigid transformation problem is established and solved to obtain the final rotation matrix used for calibrating sensors 1 and 2. Translation vector .

[0021] S5. In the time slot operation of the fusion center, the set of the latest unreceived trajectories is filtered out, and the sensed position trajectories are propagated in each time slot using the extended translation vector and rotation matrix to update them to the current fusion center time point. Then, the Kalman filter prediction equation is used to predict all trajectories to obtain the prediction state matrix and covariance matrix of the current time step.

[0022] S502, The fusion center maintains a central time variable, represented as ,in This refers to the start time of the fusion center. To address the random variations in the acquisition, processing, and communication times of millimeter-wave sensors, the fusion center operates using a time-slot method, with the duration of each time slot set to... Using the timestamp information from the received trajectory, in time... The fusion center filters out those in the interval It contains all unreceived trajectory sets, retaining only the latest trajectory set for each sensor. This indicates the end time of the previous central time step.

[0023] S504. The translation vector obtained in step S406 Extend to obtain and rotation matrix Extend to obtain , in This represents the constructed block diagonal matrix. Based on and Within each time slot of the fusion center, the state matrix and covariance matrix of each millimeter-wave sensor are calculated based on the selected sensing location trajectory. Then, based on the sensor's state matrix and covariance matrix, the sensor trajectory is propagated to update the sensing location trajectory from its sensor time to the current time point of the fusion center, so as to fuse it with the sensing location trajectories of other sensors.

[0024] S506. Based on the motion model of each millimeter-wave sensor, the fusion center uses the Kalman filter prediction equation to predict all the sensing position trajectories it maintains. According to the Kalman filter prediction equation, the predicted state matrix and covariance matrix of the current time step are obtained by transforming the state and covariance of the previous time step into a state evolution matrix.

[0025] S6. Associate and map all position trajectories maintained by the fusion center with the position trajectories from the millimeter-wave sensors, verify whether the previous association is still valid, find the new association with the minimum total cost, and update the correspondence between the fusion center position trajectories and the sensor position trajectories.

[0026] S602. For each pair of millimeter-wave sensor sensing position trajectories and fusion center position trajectories, check whether the previous association is still valid. For each pair of sensor sensing position trajectories and fusion center trajectories, calculate the Mahalanobis distance to determine whether the association in step S4 is still valid. If the Mahalanobis distance between the sensor sensing position trajectory and the fusion center trajectory is less than a preset threshold... Then the association between the two trajectories is saved.

[0027] S604. For any two sensing position trajectories of different sensors and The association cost is calculated using step S302, and the Hungarian algorithm is used to find the association mapping relationship with the minimum total cost. If the minimum total cost association cost is less than a threshold... If so, the association between the two trajectories will continue to be preserved.

[0028] S7. Calculate the state estimate and covariance matrix of the sensor trajectory. Calculate a weighted combination based on the state matrix and covariance matrix to achieve the fusion of the position trajectories perceived by the two sensors and generate a new central trajectory. At the same time, determine whether to use an initialization rule or an update rule for fusion by comparing with the timestamp of the central trajectory, and determine whether to perform fusion based on the correlation, thereby realizing intelligent moving target tracking through multi-sensor fusion.

[0029] S702, For the fusion of the perceived position trajectory of two sensors, for each sensor Calculate sensor trajectory For two correlated sensor trajectories, the state estimation matrix and covariance matrix are used to fuse them into a new central trajectory using a weighted combination based on their state estimation and covariance matrices. The weighted combination of states during fusion uses the accuracy matrix of the estimation error.

[0030] S704. For each fusion center trajectory, find its corresponding sensor trajectory. Based on the timestamps of the sensor trajectory and the center trajectory, determine whether fusion is needed. If the timestamp of the sensor trajectory is earlier than a preset timestamp threshold, initialize it using the same fusion rules as in step S702; otherwise, update the fusion center trajectory to the result fused with the corresponding sensor trajectory.

[0031] S706. Receive selected trajectories from the sensors and determine whether they can be correlated according to step S604. If correlated trajectories exist, fuse them into a new central trajectory using step S702. Based on the received sensor trajectories and the existing central trajectory, find the corresponding correlated trajectory pairs according to step S602, and perform sensing position trajectory fusion using step S702. If the timestamp of the sensor trajectory is earlier than a preset time threshold, perform fusion using the same fusion rules as the tracking in step S704.

[0032] The beneficial effects of this approach are as follows: by using multi-sensor fusion technology, data from multiple sensors can be acquired simultaneously to track a target, which can more accurately determine the target's position, attitude, velocity, and other information, thereby improving the accuracy of intelligent moving target tracking and having great potential and significance for expanding the scope of applications. Attached Figure Description

[0033] Figure 1 The accompanying drawings are for the abstract of this invention.

[0034] Figure 2 This is a flowchart illustrating the time alignment of multi-sensor sensing position trajectory in an embodiment of the present invention.

[0035] Figure 3 This is a flowchart illustrating the cost calculation process for multi-sensor perceived trajectory pairs in an embodiment of the present invention.

[0036] Figure 4 This is a flowchart illustrating the calculation of correction parameters for multi-sensor sensing trajectory pairs in an embodiment of the present invention.

[0037] Figure 5 This is a flowchart illustrating the multi-sensor trajectory association update process in an embodiment of the present invention.

[0038] Figure 6 This is a flowchart of the state estimation and covariance matrix calculation of multi-sensor trajectories in an embodiment of the present invention;

[0039] Figure 7 This is a flowchart illustrating the multi-sensor perception location trajectory fusion process in an embodiment of the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to embodiments. The description of the embodiments below is only for the purpose of helping to understand the present invention. It should be noted that those skilled in the art can make several modifications to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0041] Example 1: This application provides a multi-sensor fusion intelligent moving target tracking method, including the following steps:

[0042] S1. Deploy two millimeter-wave sensors in a motion scene to track the same moving target. Each sensor periodically sends its sensed position trajectory data and corresponding timestamp information to the fusion center. For ease of representation, use... The trajectory from sensor 1 contains a sequence of target position estimates. Representative from sensor The trajectories generated by each millimeter-wave sensor may overlap in time or not, and the lengths of the trajectories generated by different sensors may not be the same.

[0043] S2. Time-align the position trajectory information sensed by different millimeter-wave sensors to make the trajectory Each location estimate is related to the trajectory The location estimation is correlated with minimizing the time difference between two acquisition times;

[0044] S3. By considering three factors, namely trajectory length, time alignment and superposition quality, the association cost of trajectory pairs is calculated, and the Hungarian algorithm is used to select the trajectory pairs with the lowest cost to form an associated trajectory set. According to the set cost threshold, trajectory pairs with association costs below the threshold are selected as trajectory pairs of the same moving target and included in the final associated set.

[0045] S4. By stacking and calculating the association cost of the sensing position trajectory pairs of sensor 1 and sensor 2, select the trajectory pair with the lowest association cost and use it to solve the self-calibration parameters of multi-sensor fusion, including rotation matrix and translation vector.

[0046] S5. In the time slot operation of the fusion center, the set of the latest trajectory that has not been received is filtered out, and the sensed position trajectory is propagated in each time slot using the extended translation vector and rotation matrix to update it to the current fusion center time point. Then, the Kalman filter prediction equation is used to predict all trajectories to obtain the prediction state matrix and covariance matrix of the current time step.

[0047] S6. Associate and map all position trajectories maintained by the fusion center with the position trajectories from the millimeter-wave sensors, verify whether the previous association is still valid, find the new association with the minimum total cost, and update the correspondence between the fusion center position trajectories and the sensor position trajectories.

[0048] S7. Calculate the state estimate and covariance matrix of the sensor trajectory. Calculate a weighted combination based on the state matrix and covariance matrix to achieve the fusion of the position trajectories perceived by the two sensors and generate a new central trajectory. At the same time, determine whether to use an initialization rule or an update rule for fusion by comparing with the timestamp of the central trajectory, and determine whether to perform fusion based on the correlation, thereby realizing intelligent moving target tracking through multi-sensor fusion.

[0049] Example 2, as follows Figure 2 As shown, embodiment S2 of this application further includes the following steps:

[0050] S202, the trajectory of millimeter-wave sensor 1 Each position is estimated as a reference to find the trajectory. The position estimate with the smallest time difference is discarded if no corresponding element is found within the fusion time step Tc seconds.

[0051] S204, Reduce two trajectories by step S202. and arrive A time-aligned position sequence, respectively using and This represents the time-aligned position index sequence of sensor 1 and sensor 2, which have the same length.

[0052] S206. In step S204, by tracing... and Time alignment was performed, resulting in a length of Time-aligned position index sequence and , Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. Similarly, the difference between timestamps at other locations in the timeline, Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. The difference between timestamps at other locations is used to calculate the average time deviation of the two sensor position sequences after alignment. This is used to evaluate the alignment quality of the two trajectories, and the specific calculation is as follows:

[0053] .

[0054] Example 3, such as Figure 3 As shown, Embodiment 3 of this application includes the following steps:

[0055] S302, taking into account three factors—track length, track time alignment, and the quality of track overlay on different sensors—for each pair of tracks... The related costs are calculated as follows:

[0056]

[0057] in, It is the sum of the least-squares residuals after time alignment and rigid body transformation, and It is a balancing factor that favors trajectory pairs with longer overlap and lower average time offset;

[0058] S304. Combine the calculated cost values ​​into a... The cost matrix, where This represents the number of sensing location trajectories collected by millimeter-wave sensor 1, while This represents the number of sensing location trajectories collected by millimeter-wave sensor 2. Using the Hungarian algorithm, the optimal correlation of these trajectories is obtained by minimizing the overall cost. , wherein express The smaller value in the associated trajectory implies the same moving target tracked by both millimeter-wave sensors;

[0059] S306. Due to the presence of false trajectories, ghost targets, and clutter, a cost threshold is set. For trajectory pairs whose association cost is lower than the cost threshold calculated in step S302, they are considered as trajectory pairs generated by tracking the same moving target and are selected into the final association set. Among them Indicates source from millimeter-wave sensor The A selected trajectory.

[0060] Example 4, such as Figure 4 As shown, Embodiment 4 of this application includes the following steps:

[0061] S402, Use In step S306 The set of all possible non-empty subsets, Each element is a set of perceived position trajectory pairs from sensor 1 and sensor 2, for Each element of the vector stacks all trajectories from sensor 1. Stack all trajectories from sensor 2 into a vector. Then, calculate the associated costs according to step S302. And store all associated costs in a single dimension. In the matrix;

[0062] S404, from Select the element with the lowest association cost, which contains... The trajectory pairs will be used for subsequent calculation of self-calibration parameters for multi-sensor fusion;

[0063] S406, Selected in step S404 A series of trajectory pairs are stacked together, represented as A rigid transformation problem is established and solved to obtain the final rotation matrix used for calibrating sensors 1 and 2. Translation vector .

[0064] Example 5, as follows Figure 5 As shown, Embodiment 5 of this application includes the following steps:

[0065] S502, The fusion center maintains a central time variable, represented as ,in This refers to the start time of the fusion center. To address the random variations in the acquisition, processing, and communication times of millimeter-wave sensors, the fusion center operates using a time-slot method, with the duration of each time slot set to... Using the timestamp information from the received trajectory, in time The fusion center filters out those in the interval The set of all unreceived trajectories is used, and only the latest trajectories from each sensor are retained. Indicates the end time of the previous central time step;

[0066] S504. The translation vector obtained in step S406 Extend to obtain and rotation matrix Extend to obtain , in Represents the constructed block diagonal matrix, based on and Within each time slot of the fusion center, the state matrix and covariance matrix of each millimeter-wave sensor are calculated based on the selected sensing position trajectory. Then, the sensor trajectory is propagated based on the sensor's state matrix and covariance matrix. The purpose is to update the sensing position trajectory from its sensor's time to the current time point of the fusion center so as to fuse it with the sensing position trajectories of other sensors.

[0067] S506. Based on the motion model of each millimeter-wave sensor, the fusion center uses the Kalman filter prediction equation to predict all the sensing position trajectories it maintains. According to the Kalman filter prediction equation, the predicted state matrix and covariance matrix of the current time step are obtained by transforming the state and covariance of the previous time step into a state evolution matrix.

[0068] Example 6, as Figure 6 As shown, Embodiment 6 of this application includes the following steps:

[0069] S602. For each pair of millimeter-wave sensor sensing position trajectories and fusion center position trajectories, check whether the previous association is still valid. For each pair of sensor sensing position trajectories and fusion center trajectories, calculate the Mahalanobis distance to determine whether the association in step S4 is still valid. If the distance between the sensor sensing position trajectory and the fusion center trajectory is less than a preset threshold... Then the correlation between the two trajectories is saved;

[0070] S604. For any two sensing position trajectories of different sensors and The association cost is calculated using step S302, and the Hungarian algorithm is used to find the association mapping relationship with the minimum total cost. If the minimum total cost association cost is less than a threshold... If so, the association between the two trajectories will continue to be preserved.

[0071] Example 7, as follows Figure 7 As shown, Embodiment 7 of this application includes the following steps:

[0072] S702, For the fusion of the perceived position trajectory of two sensors, for each sensor Calculate sensor trajectory For two correlated sensor trajectories, the state estimation matrix and covariance matrix are used to fuse them into a new central trajectory using a weighted combination based on their state estimation and covariance matrices. The weighted combination of states during fusion uses the accuracy matrix of the estimation error.

[0073] S704. For each fusion center trajectory, find the corresponding sensor trajectory. Based on the timestamps of the sensor trajectory and the center trajectory, determine whether fusion is needed. If the timestamp of the sensor trajectory is earlier than the preset threshold, initialize it using the same fusion rule as in step S702; otherwise, update the fusion center trajectory to the result fused with the corresponding sensor trajectory.

[0074] S706. Receive selected trajectories from the sensor. Determine whether they can be associated according to step S604. If there are associative trajectories, use the fusion step in step S702 to fuse them into a new central trajectory. Based on the received sensor trajectory and the existing central trajectory, find the corresponding associated trajectory pair according to step S602. Use step S702 to realize the sensing position trajectory fusion. If the timestamp of the sensor trajectory is earlier than the preset time threshold, use the same fusion rule as the tracking in step S704 to perform fusion.

[0075] In summary, the multi-sensor fusion technology proposed in this invention can improve the accuracy and robustness of moving target tracking systems and can be widely used in fields such as robotics, autonomous driving, and intelligent video. This technology can obtain more accurate and comprehensive information such as target position, velocity, and attitude by fusing data collected from multiple sensors.

Claims

1. A multi-sensor fusion intelligent moving target tracking method, characterized in that, include: S1. Deploy two millimeter-wave sensors in a motion scene to track the same moving target. Each sensor periodically sends the sensed position trajectory data and corresponding timestamp information to the fusion center. For ease of representation, use... The trajectory from sensor 1 contains a sequence of target position estimates. Representative from sensor The trajectories generated by each millimeter-wave sensor may overlap in time or not, and the lengths of the trajectories generated by different sensors may not be the same. S2. Time-align the position trajectory information sensed by different millimeter-wave sensors to make the trajectory Each location estimate is related to the trajectory The location estimation is correlated with minimizing the time difference between two acquisition times; S3. By considering three factors, namely trajectory length, time alignment and superposition quality, the association cost of trajectory pairs is calculated, and the Hungarian algorithm is used to select the trajectory pairs with the lowest cost to form an associated trajectory set. According to the set cost threshold, trajectory pairs with association costs below the threshold are selected as trajectory pairs of the same moving target and included in the final associated set. S4. By stacking and calculating the association cost of the sensing position trajectory pairs of sensor 1 and sensor 2, the trajectory pair with the lowest association cost is selected and used to solve the self-calibration parameters of multi-sensor fusion, including the rotation matrix and translation vector. S5. In the time slot operation of the fusion center, the set of the latest trajectory that has not been received is filtered out, and the sensed position trajectory is propagated in each time slot using the extended translation vector and rotation matrix to update it to the current fusion center time point. Then, the Kalman filter prediction equation is used to predict all trajectories to obtain the prediction state matrix and covariance matrix of the current time step. S6. Associate and map all position trajectories maintained by the fusion center with position trajectories from millimeter-wave sensors, verify whether the previous association is still valid, find a new association with the minimum total cost, and update the correspondence between the fusion center position trajectories and the sensor position trajectories. S7. Calculate the state estimate and covariance matrix of the sensor trajectory. Calculate a weighted combination based on the state matrix and covariance matrix to achieve the fusion of the position trajectories perceived by the two sensors and generate a new central trajectory. At the same time, determine whether to use an initialization rule or an update rule for fusion by comparing with the timestamp of the central trajectory, and determine whether to perform fusion based on the correlation, thereby realizing intelligent moving target tracking through multi-sensor fusion.

2. The intelligent moving target tracking method based on multi-sensor fusion according to claim 1, characterized in that, Step S2 includes: S202, the trajectory of millimeter-wave sensor 1 Each position is estimated as a reference to find the trajectory. The position estimate with the smallest time difference is discarded if no corresponding element is found within the fusion time step Tc seconds. S204, Reduce two trajectories by step S202. and arrive A time-aligned position sequence, respectively using and This represents the time-aligned position index sequence of sensor 1 and sensor 2, which have the same length. S206. In step S204, by tracing... and Time alignment was performed, resulting in a length of Time-aligned position index sequence and , Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. Similarly, the difference between timestamps at other locations in the timeline, Representing the trajectory The Middle The time offset of each time alignment position indicates the time difference between the timestamp at that position and the trajectory. The difference between timestamps at other locations is used to calculate the average time deviation of the two sensor position sequences after alignment. This is used to evaluate the alignment quality of the two trajectories, and the specific calculation is as follows: 。 3. The intelligent moving target tracking method based on multi-sensor fusion according to claim 1, characterized in that, Step S3 includes: S302, taking into account three factors—track length, track time alignment, and the quality of track superposition on different sensors—for each pair of tracks... The related costs are calculated as follows: in, It is the sum of the least-squares residuals after time alignment and rigid body transformation, and It is a balancing factor that favors trajectory pairs with longer overlap and lower average time offset; S304. Combine the calculated cost values ​​into a... The cost matrix, where This represents the number of sensing location trajectories collected on millimeter-wave sensor 1, while This represents the number of sensing location trajectories collected by millimeter-wave sensor 2. Using the Hungarian algorithm, the optimal correlation of these trajectories is obtained by minimizing the overall cost. , wherein express The smaller value in the associated trajectory implies the same moving target tracked by both millimeter-wave sensors; S306. Due to the presence of false trajectories, ghost targets, and clutter, a cost threshold is set. For trajectory pairs whose association cost is lower than the cost threshold calculated in step S302, they are considered as trajectory pairs generated by tracking the same moving target and are selected into the final association set. Among them Indicates source from millimeter-wave sensor The A selected trajectory.

4. The intelligent moving target tracking method based on multi-sensor fusion according to claim 3, characterized in that, Step S4 includes: S402, Use In step S306 The set of all possible non-empty subsets, Each element is a set of perceived position trajectory pairs from sensor 1 and sensor 2, for Each element of the vector stacks all trajectories from sensor 1 into a vector. Stack all trajectories from sensor 2 into a vector. Then, calculate the associated costs according to step S302. And store all associated costs in a single dimension. In the matrix; S404, from Select the element with the lowest association cost, which contains... The trajectory pairs will be used for subsequent calculation of self-calibration parameters for multi-sensor fusion; S406, Selected in step S404 A series of trajectory pairs are stacked together, represented as A rigid transformation problem is established and solved to obtain the final rotation matrix used for calibrating sensors 1 and 2. Translation vector .

5. The intelligent moving target tracking method based on multi-sensor fusion according to claim 4, characterized in that, Step S5 includes: S502, The fusion center maintains a central time variable, represented as ,in This refers to the start time of the fusion center. To address the random variations in the acquisition, processing, and communication times of millimeter-wave sensors, the fusion center operates using a time-slot method, with the duration of each time slot set to... Using the timestamp information from the received trajectory, in time The fusion center filters out those in the interval The set of all unreceived trajectories is used, and only the latest trajectories from each sensor are retained. Indicates the end time of the previous central time step; S504. The translation vector obtained in step S406 Extend to obtain and rotation matrix Extend to obtain , in Represents the constructed block diagonal matrix, based on and Within each time slot of the fusion center, the state matrix and covariance matrix of each millimeter-wave sensor are calculated based on the selected sensing position trajectory. Then, the sensor trajectory is propagated based on the sensor's state matrix and covariance matrix. The purpose is to update the sensing position trajectory from its sensor's time to the current time point of the fusion center so as to fuse it with the sensing position trajectories of other sensors. S506. Based on the motion model of each millimeter-wave sensor, the fusion center uses the Kalman filter prediction equation to predict all the sensing position trajectories it maintains. According to the Kalman filter prediction equation, the predicted state matrix and covariance matrix of the current time step are obtained by transforming the state and covariance of the previous time step into a state evolution matrix.

6. The intelligent moving target tracking method based on multi-sensor fusion according to claim 3, characterized in that, Step S6 includes: S602. For each pair of millimeter-wave sensor sensing position trajectories and fusion center position trajectories, check whether the previous association is still valid. For each pair of sensor sensing position trajectories and fusion center trajectories, calculate the Mahalanobis distance to determine whether the association in step S4 is still valid. If the distance between the sensor sensing position trajectory and the fusion center trajectory is less than a preset threshold... Then the correlation between the two trajectories is saved; S604. For any two sensing position trajectories of different sensors and The association cost is calculated using step S302, and the Hungarian algorithm is used to find the association mapping relationship with the minimum total cost. If the minimum total cost association cost is less than a threshold... If so, the association between the two trajectories will continue to be preserved.

7. The intelligent moving target tracking method based on multi-sensor fusion according to claim 6, characterized in that, Step S7 includes: S702, For the fusion of the perceived position trajectory of two sensors, for each sensor Calculate sensor trajectory For two correlated sensor trajectories, the state estimation matrix and covariance matrix are used to fuse them into a new central trajectory using a weighted combination based on their state estimation and covariance matrices. The weighted combination of states during fusion uses the accuracy matrix of the estimation error. S704. For each fusion center trajectory, find the corresponding sensor trajectory. Based on the timestamps of the sensor trajectory and the center trajectory, determine whether fusion is needed. If the timestamp of the sensor trajectory is earlier than the preset threshold, initialize it using the same fusion rule as in step S702; otherwise, update the fusion center trajectory to the result fused with the corresponding sensor trajectory. S706. Receive selected trajectories from the sensor. Determine whether they can be associated according to step S604. If there are associative trajectories, use the fusion step in step S702 to fuse them into a new central trajectory. Based on the received sensor trajectory and the existing central trajectory, find the corresponding associated trajectory pair according to step S602. Use step S702 to realize the sensing position trajectory fusion. If the timestamp of the sensor trajectory is earlier than the preset time threshold, use the same fusion rule as the tracking in step S704 to perform fusion.

Images