Multi-target tracking millimeter wave radar measurement point dynamic correlation method

Abstract

This application belongs to the field of target association technology, and more specifically, relates to a dynamic association method for millimeter-wave radar measurement points in multi-target tracking. In this invention, a dynamic spatiotemporal graph is constructed, and edges between nodes are dynamically built based on dynamic consistency and spatial proximity rules. Combined with motion, environment, and temporal feature fusion, the feature graph can reflect the dynamic changes in node relationships in real time, achieving dynamic adaptation to changes in node relationships. Then, a lightweight relationship network is used to accurately calculate the similarity between nodes and trajectories and screen candidate matching pairs. Combined with the Hungarian algorithm that introduces an environment adaptive cost matrix, the matching process can dynamically adjust the weights according to the environment, effectively resisting environmental interference and improving matching stability. Finally, continuous occlusion is detected by the association results and an occlusion recovery procedure is initiated, so that the trajectory can continue even when the target is occluded, ensuring the continuity of the association.

Description

Technical Field

[0001] This application belongs to the field of target association technology, and more specifically, relates to a method for dynamic association of measurement points of millimeter-wave radar for multi-target tracking. Background Technology

[0002] Multi-target tracking is a core supporting technology in fields such as autonomous driving, intelligent security, and drone navigation. It estimates the motion trajectory and identity information of multiple targets in real time through sensor data, providing key inputs for decision-making systems. Millimeter-wave radar has become one of the preferred sensors for MOT (Multi-Target Tracking) in complex environments due to its advantages such as all-weather operation, resistance to light interference, and strong penetration. Especially in autonomous driving, millimeter-wave radar needs to output the accurate trajectory of targets such as vehicles, pedestrians, and cyclists in real time, which directly affects the reliability of path planning and safety control.

[0003] The association between measurement points and target trajectories is the core environment of multi-target tracking. Essentially, it involves matching the millimeter-wave radar measurement points of the current frame with the historically tracked target trajectories to determine which target each measurement point belongs to, and then updating the trajectory status. The accuracy of the association results directly determines the tracking accuracy. If the association is incorrect, it will lead to trajectory drift. If the association is missed, it will lead to target loss or trajectory breakage. Therefore, measurement-trajectory association is considered a bottleneck problem in multi-target tracking, and its robustness is a key indicator for measuring the performance of the tracking system.

[0004] In real-world dynamic scenarios, the association between millimeter-wave radar measurement points and trajectories faces multiple challenges. First, the spatial distribution and association of measurement points are dynamic and changeable. Measurement points of the same target may exhibit dynamic spatial distribution due to changes in dynamic state. Measurement points of different targets may overlap spatially due to relative motion, causing the association between measurement points to change dynamically over time. Traditional association methods are mostly based on fixed spatial distance thresholds or single motion features to construct association logic, without considering the dynamic topological relationships between measurement points and between measurement points and trajectories, making it difficult to adapt to the dynamic changes in node association.

[0005] Furthermore, when the target is obscured, the millimeter-wave radar measurement points will temporarily disappear. Traditional correlation methods, lacking in-depth mining of historical trajectory temporal characteristics and dynamic discrimination of obstruction status, are difficult to effectively continue trajectory correlation, and are prone to trajectory interruption or incorrect correlation to other target measurement points, further disrupting the continuity of tracking. Summary of the Invention

[0006] This invention provides a dynamic association method for millimeter-wave radar measurement points for multi-target tracking, aiming to solve the technical problem that the association logic of existing methods cannot dynamically adapt to changes in node relationships, environmental interference, and occlusion, making it difficult to maintain the continuity and stability of the association.

[0007] A method for dynamic association of measurement points in millimeter-wave radar for multi-target tracking includes the following steps: Target signal preprocessing: Acquire raw millimeter-wave radar data and preprocess the raw millimeter-wave radar data to obtain an enhanced three-dimensional measurement point set; Constructing a dynamic spatiotemporal graph: Each enhanced 3D measurement point in the current frame is taken as a node of the graph. Edges between nodes are constructed according to motion consistency rules and spatial proximity rules. Motion features, environmental features and temporal features are fused for each node to obtain a feature graph with topological relationships. Hierarchical quantization association: A lightweight relational network is used to calculate the similarity between each node in the feature map and the existing trajectory nodes. The top k candidate matching pairs with the highest similarity are retained to obtain a candidate matching set. The Hungarian algorithm is used to match each pair of measurement points and trajectories in the candidate matching set to obtain the association result between the target in the current frame and the existing trajectory. The Hungarian algorithm introduces an environment adaptive cost matrix. Association result judgment: The association result is judged. If the association result for a certain target shows continuous occlusion for more than N frames, the occlusion recovery procedure is started to restore the trajectory of the occluded target. If no occlusion occurs, the trajectory data is directly updated to obtain the trajectory continuation result after occlusion recovery.

[0008] In this invention, a dynamic spatiotemporal graph is constructed, and edges between nodes are dynamically built based on dynamic consistency and spatial proximity rules. Combined with motion, environment, and temporal feature fusion, the feature graph can reflect the dynamic changes in node relationships in real time, achieving dynamic adaptation to changes in node relationships. Then, a lightweight relationship network is used to accurately calculate the similarity between nodes and trajectories and screen candidate matching pairs. With the introduction of the Hungarian algorithm with an environment adaptive cost matrix, the matching process can dynamically adjust the weights according to the environment, effectively resisting environmental interference and improving matching stability. Finally, continuous occlusion is detected by the association results and the occlusion recovery procedure is initiated, so that the trajectory can continue even when the target is occluded, ensuring the continuity of the association.

[0009] Preferably, the preprocessing of the raw millimeter-wave radar data includes the following steps: The raw data from the millimeter-wave radar is processed by range-Doppler to obtain a complex domain 2D matrix. Then, a three-level complex domain wavelet packet transform is performed on the complex domain 2D matrix. The complex Morlet wavelet is used as the basis function to decompose the input signal into eight sub-band signals, with each level of decomposition focusing on a different frequency range. For each sub-band signal, the cross-pulse phase coherence coefficient is calculated to obtain the coherent phase coefficients of each of the eight sub-bands. Then, the weights are dynamically allocated according to the signal-to-noise ratio of each sub-band, and the correlation phase coefficients of the eight sub-bands are weighted and summed to obtain the comprehensive coherence coefficient. Based on a set threshold, points with a comprehensive correlation coefficient greater than the threshold are retained as candidate target measurement points. The candidate target measurement points are converted from polar coordinates to rectangular coordinates, and Doppler velocity information is extracted to obtain the set of candidate target measurement points. An enhanced three-dimensional measurement point set is obtained by performing adaptive particle filtering based on the candidate target measurement point set.

[0010] Preferably, when using adaptive particle filtering, resampling and state estimation are introduced, as follows: Resampling: First, calculate the number of valid particles. If the number of valid particles is less than 50% of the total number of particles, resampling is triggered. The top 50% of particles by weight are retained, and new particles that follow the prior distribution are added. State estimation: The weighted average of the resampled particle swarm is used to obtain the compensated position and velocity, and based on this, the position and velocity of each candidate measurement point after motion compensation are obtained.

[0011] Preferably, constructing the edges between nodes based on motion consistency rules and spatial proximity rules includes the following steps: For any two nodes, calculate their Euclidean distance. If the Euclidean distance is less than or equal to a preset distance threshold, then the two nodes are spatially adjacent. For the same pair of nodes, calculate the angle between their velocity vectors. If the target is stationary, the angle is set to 0 by default. Otherwise, if the angle is less than or equal to a preset angle threshold, the two nodes move in the same direction. For node pairs that simultaneously satisfy spatial proximity and known motion, add undirected edges and calculate weights based on distance and included angle to obtain a weighted edge set.

[0012] Preferably, the motion characteristics include velocity, velocity modulus, acceleration, acceleration modulus, velocity direction angle, and acceleration direction angle; The environmental features include surrounding target density, reflection intensity, phase coherence confidence, and occlusion probability; The temporal features include the rate of change of position, the variance of the rate of change of position, and the position offset.

[0013] Preferably, the environment adaptive cost matrix is ​​obtained by weighted summation of three sub-cost terms, namely motion cost, comprehensive state cost, and environment cost; wherein the motion cost is used to quantify the consistency of motion state between the measurement point and the trajectory; the comprehensive state cost is used to quantify the feature similarity between the measurement point and the trajectory; and the environment cost is used to quantify the impact of environmental complexity on matching. The weight coefficients of each sub-cost item are dynamically adjusted based on the constructed scenario type, which is divided into low-density, medium-density, and high-density scenarios based on the target density. In the low-density scenario, the weight of motion cost is increased. In the medium-density scenario, the weights of motion cost and comprehensive state cost are equal. In the high-density scenario, comprehensive state cost is dominant. The sum of the weights of motion cost, comprehensive state cost, and environmental cost is 1.

[0014] Preferably, when executing the Hungarian algorithm, a cost matrix is ​​constructed based on the candidate matching set, and the matching costs of candidate pairs in the candidate matching set are filled into the matrix. For candidate pairs not in the candidate matching set, the cost is set to invalid. Then, the improved Hungarian algorithm based on the Kuhn-Munkres algorithm framework is called to perform the optimal matching solution. In the solution process, a cost threshold filter is introduced. If the minimum matching cost of a matching pair exceeds a predetermined threshold, the corresponding match is rejected. Based on this, the optimal matching result is obtained, that is, the association result between the target in the current frame and the existing trajectory.

[0015] Preferably, the occlusion recovery procedure includes the following steps: The target's own trajectory segment is extracted based on the trajectory segments of the target in consecutive n frames before the occlusion occurs; a surrounding situation map is constructed based on the motion state information of other targets within a predetermined range around the target's disappearance location; The target's own trajectory segment is used as input to the LSTM network to extract motion features in the time dimension; the surrounding situation map is used as input to the fully connected layer to extract situation features; the motion features and situation features are concatenated to form a joint feature vector; and the joint feature vector is then processed by the fully connected layer to obtain the optimal action number, which represents the predicted direction of motion. The optimal action number is mapped to a specific angle. Assuming that the target moves at a constant speed according to the speed before occlusion and the predicted direction, the predicted position is calculated based on the duration of occlusion. A two-dimensional Gaussian distribution region is constructed with the predicted position as the center to generate a probability heatmap. The variance of the Gaussian distribution region increases with the occlusion time. Calculate the probability value of the new measurement point in the probability heatmap. If the probability value is greater than the predetermined threshold, perform motion consistency verification; otherwise, ignore it. The motion consistency verification involves calculating the absolute difference between the velocity magnitude of the new measurement point and the historical velocity magnitude of the target, converting it into a consistency score using an exponential function; then calculating the cosine similarity between the feature vector of the new measurement point and the historical feature vector of the target; adjusting the weights of the consistency score and cosine similarity based on the phase coherence confidence of the target before occlusion, and performing a weighted summation. If the summation value exceeds a predetermined threshold, the new measurement point is determined to belong to the original target; otherwise, the association is rejected, and the occlusion recovery procedure continues.

[0016] Preferably, for measurement points with probability values ​​less than or equal to a predetermined threshold, it is determined whether this is the first occurrence. If it is the first occurrence, a temporary tracking document is created; if it is not the first occurrence, the tracking document is updated and the number of consecutive occurrence frames is increased. When the number of consecutive occurrence frames reaches a predetermined number of frames, cross-frame correlation analysis is activated. Calculate the angle difference between the actual motion direction and the predicted motion direction to obtain the motion consistency coefficient. Check whether the target continues to move towards the center of the probability heatmap and whether the motion consistency coefficient is greater than a predetermined threshold. If the condition is met, a re-evaluation is performed. For measurement points that meet the conditions, a re-evaluation is performed, that is, after updating their own trajectory segments and surrounding situation maps, a new prediction is made, and a new probability heatmap is generated based on the new prediction results. The probability value of the measurement point is calculated based on the new probability heatmap. If the probability value is greater than a predetermined threshold, motion consistency verification is performed. If it is less than or equal to the predetermined threshold, the system returns to the memory retention state to continue observation.

[0017] The beneficial effects of the invention include: In this invention, a dynamic spatiotemporal graph is constructed, and edges between nodes are dynamically built based on dynamic consistency and spatial proximity rules. Combined with motion, environment, and temporal feature fusion, the feature graph can reflect the dynamic changes in node relationships in real time, achieving dynamic adaptation to changes in node relationships. Then, a lightweight relationship network is used to accurately calculate the similarity between nodes and trajectories and screen candidate matching pairs. With the introduction of the Hungarian algorithm with an environment adaptive cost matrix, the matching process can dynamically adjust the weights according to the environment, effectively resisting environmental interference and improving matching stability. Finally, continuous occlusion is detected by the association results and the occlusion recovery procedure is initiated, so that the trajectory can continue even when the target is occluded, ensuring the continuity of the association. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 An overall flowchart is provided for embodiments of the present invention.

[0020] Figure 2 A flowchart of the DRQN ​​prediction network prediction steps provided for an embodiment of the present invention.

[0021] Figure 3 A flowchart illustrating the tracking file update process provided in this embodiment of the invention. Detailed Implementation

[0022] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0023] See Figure 1 As shown, the method for dynamic association of measurement points in millimeter-wave radar for multi-target tracking includes the following steps: Target signal preprocessing: Acquire raw millimeter-wave radar data and preprocess the raw millimeter-wave radar data to obtain an enhanced three-dimensional measurement point set; The complex domain echo signal from the original millimeter-wave radar output is processed by range-Doppler to obtain a 2D complex matrix of range-Doppler cells. Where M represents the number of distance units and N represents the number of Doppler units; for complex matrices Perform a 3-level complex domain wavelet transform to decompose the signal into 8 sub-bands. ; Calculate the pulse phase coherence coefficient for each sub-band signal. Quantifying the consistency of phase changes: In the formula: Indicates the number of coherent cumulative pulses; This represents the phase of the t-th pulse of the k-th child. Represents the imaginary unit; The base of the natural logarithm; The correlation coefficients of the eight sub-bands are then weighted and fused to obtain the comprehensive coherence coefficient. The weights are dynamically allocated based on the sub-band signal-to-noise ratio (SNR), meaning the weight of any sub-band is its SNR divided by the sum of the SNRs of all sub-bands. Based on a set threshold (e.g., 0.6), points with a comprehensive coherence coefficient greater than or equal to the threshold are retained as candidate target measurement points. Then, the selected points are converted from polar coordinates (original millimeter-wave radar coordinates) to rectangular coordinates, and Doppler velocity information is extracted to obtain the set of candidate target measurement points. Each measurement point includes rectangular coordinates and Doppler velocity; Since the phase and time of static clutter fluctuate randomly, while the phase change of moving targets is periodic due to the micro-Doppler effect, the phase coherence can be used to distinguish between targets and clutter, which can solve the problem of misjudgment caused by amplitude superposition in the overlapping area of ​​targets by amplitude filtering.

[0024] Furthermore, in this embodiment, an adaptive particle filter is used to dynamically switch the motion model to compensate for nonlinear motion errors. Specifically, for candidate measurement points... Establish state vector , including location ,speed acceleration Turning angular velocity Three motion models were constructed, and the optimal model was selected through real-time likelihood, including the uniform velocity model, the uniform acceleration model, and the turning model. For each candidate measurement point, the optimal motion model is selected by maximizing the model likelihood; that is, a particle set is generated based on the three models respectively, with 200 particles generated for each model, and each particle representing a possible motion state. Calculate the weight of each particle, where the weight is determined by the observation likelihood, i.e. That is, the original weights of particle i under model m. equal to the measured value Regarding particle states The observation likelihood is the probability density function of a multivariate Gaussian distribution.

[0025] For each model, the weights of all its particles are summed to obtain the total likelihood of the model. The model with the largest total likelihood is selected as the optimal model for the current measurement point. Based on this, the optimal motion model corresponding to each candidate measurement point is obtained. Then, the number of effective particles is calculated. When the number of effective particles is less than 50% of the total number of particles, resampling is triggered. The top 50% of the particles by weight are retained, and new particles that follow the prior distribution are added to avoid particle degradation. In this embodiment, resampling first involves resetting the original weights of each particle. The normalized weights are obtained by dividing by the sum of the original weights of all particles. The 200 particles are then sorted in descending order of normalized weights, and the top 50% of particles are selected as elite particles. The states (position, velocity, acceleration, angular velocity, etc.) of the elite particles are then weighted and averaged (the weights are the normalized weights of the elite particles) to obtain the state prediction mean, which serves as the center of the prior distribution. Based on the process noise covariance of the current optimal motion model, the extended covariance is obtained by multiplying it by an expansion factor of 1.5. From the Gaussian distribution centered on the state prediction mean and with the extended covariance as the noise range, 100 new particles are randomly sampled. At the same time, physical constraints are applied to the states of the new particles (e.g., the absolute value of velocity does not exceed 30 m / s, and the absolute value of angular velocity does not exceed 0.5 rad / s) to ensure that the states conform to the actual physical range of the detected target. Based on this, 100 new particles that conform to the prior distribution are obtained. The retained elite particles are merged with the newly added particles to form a new set of particles after resampling. The weights of all particles are then evenly distributed, with each particle having a weight of 1 / total number of particles. The compensated target state (position and velocity) is then calculated based on the new particle set. This involves taking a weighted average of the position and velocity states of each particle with the corresponding evenly distributed particle weights to obtain the final compensated position and velocity.

[0026] Constructing a dynamic spatiotemporal graph: Each enhanced 3D measurement point in the current frame is taken as a node of the graph. Edges between nodes are constructed according to motion consistency rules and spatial proximity rules. Motion features, environmental features and temporal features are fused for each node to obtain a feature graph with topological relationships. A dynamic spatiotemporal graph is constructed based on the enhanced 3D measurement point set, which includes multiple measurement points. Each measurement point contains position, velocity, phase coherence confidence (phase coherence coefficient), reflection intensity (millimeter-wave radar echo reflection intensity), and historical position cache (position data of the past 3 frames, used for temporal feature calculation). Spatial proximity rule filtering: For any two nodes, calculate the Euclidean distance between the two nodes (i.e., the square root of the difference in three-dimensional position coordinates). If the Euclidean distance is less than or equal to a preset distance threshold (e.g., 5 meters), the two nodes are considered to be spatially adjacent. Motion consistency rule filtering: For the same node pair, calculate the angle between their velocity vectors. If the target is stationary, the angle is 0 degrees by default; otherwise, if the angle is less than or equal to the preset angle threshold (e.g., 30 degrees), the two nodes are considered to have consistent motion. Undirected edges are added only to node pairs that simultaneously satisfy the spatial proximity rule and the motion consistency rule. The weight of the edge is determined by the Euclidean distance and the angle between their velocity vectors. Specifically, it is obtained by taking the negative exponent of the ratio of the Euclidean distance to the distance threshold and then multiplying it by the cosine of the angle. The weight range is 0-1. Based on this, a weighted edge set is obtained.

[0027] Then, features are extracted based on the three-dimensional measurement point set to form a 128-dimensional feature vector for each node, which includes motion features, environmental features and temporal features; The motion features include basic velocity (velocity in the x, y, and z directions), velocity magnitude (three-dimensional magnitude of the velocity vector), acceleration (acceleration in the x, y, and z directions), acceleration magnitude (three-dimensional magnitude of the acceleration vector), velocity direction angle (azimuth and elevation angles), and acceleration direction angle (azimuth and elevation angles), totaling 12 features.

[0028] The environmental features comprise four dimensions: surrounding target density, reflection intensity, phase position confidence, and occlusion probability. The surrounding target density is calculated by dividing the number of measurement points within a 10-meter radius centered on the current node by this circular area. The occlusion probability is calculated based on the surrounding target density and coherence confidence, and the specific expression is as follows: In the formula: Indicates the probability of occlusion; This represents the density compensation coefficient (dimension L²). Indicates the density of surrounding targets; This represents the coherence confidence level (phase coherence coefficient).

[0029] The temporal features consist of 15 dimensions, including the rate of change of position, the variance of the rate of change of position, and the position offset. The position change rate has 9 dimensions. Based on the position data of the past 3 frames, the average velocity of the past 1, 2 and 3 frames is calculated respectively, and each frame corresponds to 3-dimensional coordinates; for example, the change rate in the x direction, the change rate in the y direction and the change rate in the z direction of the past 1 frame. The variance of the rate of position change is three-dimensional. The variance of the rate of position change in the x, y, and z directions of the past three frames is calculated to characterize motion stability. The displacement offset is 3-dimensional, and the deviation between the current position and the average position of the past 3 frames is calculated.

[0030] Motion features, environmental features, and temporal features are concatenated in sequence to form a 34-dimensional original feature vector; then, a two-layer fully connected neural network is used to map the 34-dimensional vector to a 128-dimensional feature vector; the 128-dimensional feature vector is used as the feature vector of each node; based on this, a feature map with topological relationships is output, including a node set, an edge set, and a feature set.

[0031] Hierarchical quantization association: A lightweight relational network is used to calculate the similarity between each node in the feature map and the existing trajectory nodes. The top k candidate matching pairs with the highest similarity are retained to obtain a candidate matching set. The Hungarian algorithm is used to match each pair of measurement points and trajectories in the candidate matching set to obtain the association result between the target in the current frame and the existing trajectory. The Hungarian algorithm introduces an environment adaptive cost matrix. In this embodiment, a lightweight relational network (RN) is used to take the current set of measurement point nodes and the set of active trajectories as input to calculate the matching similarity between measurement points and trajectories. The current set of measurement points includes M measurement points, each of which includes a 128-dimensional feature vector. The set of active trajectories includes N existing trajectories, each of which carries the feature vectors of 3 historical frames and the current predicted motion state (the result obtained from the association result discrimination). In this embodiment, the 128-dimensional feature vector of the current measurement point is concatenated with the average value of the feature vectors of the corresponding historical 3 frames to form a 256-dimensional combined vector. The combined vector is then subjected to dimensionality reduction and feature extraction through two fully connected layers. The first fully connected network compresses the 256-dimensional vector to 64 dimensions and uses the ReLU activation function to enhance the nonlinear expression. The second fully connected network further compresses it to 32 dimensions, and the training process is stabilized through BatchNorm (batch normalization), finally outputting a 32-dimensional associated feature vector. Finally, a fully connected network is used to map the 32-dimensional associated feature vector to a 1-dimensional value, and then the Sigmoid function is used to activate and normalize it to obtain the similarity score between 0 and 1. In this embodiment, the above-mentioned relationship network is trained offline in advance. Successful measurement-trajectory association pairs in historical data are used as positive samples, and random measurement-trajectory pairs are used as negative samples. The network parameters are optimized by the binary cross-entropy loss function.

[0032] After calculating the similarity score between each measurement point and all trajectories, only the top k trajectories with the highest scores are retained as candidate matching objects to form a candidate matching set. The value of k is dynamically adjusted according to the current number of active trajectories N. Specifically, it is 20% of your technology rounded up, and it is determined whether the rounded value exceeds 20. If it does, then 20 is taken; if it does not exceed 20, then the rounded value is taken. For example, if there are 50 trajectories, 20% is 10 (which is still 10 after rounding up), then k equals 10. If there are 150 trajectories, 20% is 30, and the smaller value of 20 is taken.

[0033] Then, the matching cost is calculated based on the measurement points and trajectories in the selected candidate matching set. The matching cost is obtained by weighted summation of three sub-cost items, namely motion cost, comprehensive state cost and environmental cost. The expression for the motion cost is as follows: In the formula: Indicates the cost of movement; Indicates the three-dimensional position of the current measurement point; Representing the trajectory The predicted position of the current frame; This represents the position normalization constant; This represents the velocity vector of the current measurement point; Representing the trajectory The current frame predicted velocity vector; Represents the velocity normalization constant; regarding and The predictions are not within the scope of this application and can be obtained through existing models.

[0034] The expression for the comprehensive state cost is as follows: In the formula: The 128-dimensional feature vector representing the current measurement point (obtained based on the dynamic spatiotemporal graph); Representing the trajectory The historical 128-dimensional feature vector; Indicates the overall state cost; The expression for the environmental cost is as follows: ; In the formula: Indicates environmental cost; Indicates the current environmental density; Indicates scene reference density; The matching cost is obtained by weighted summation of the values ​​of the three sub-costs mentioned above, wherein the weights are set based on the following steps: The weights of the sub-costs are adjusted based on the environment density, as follows: Based on environmental density, scenes are categorized into low-density, medium-density, and high-density scenes, as detailed below: In low-density scenes (ρ < 1 target / 100 square meters): targets are sparse, occlusion is minimal, and the continuity of motion is strong. In this case, the motion cost weight is increased, and α = 0.6, β = 0.2, and γ = 0.2 are set (motion cost is dominant).

[0035] Medium-density scenarios (1≤ρ≤3 targets / 100 square meters): With increased target intersections, it is necessary to balance motion and feature information. Set α=0.4, β=0.4, γ=0.2 (motion cost and comprehensive state cost have equal weights).

[0036] High-density scenes (ρ>3 / 100 square meters): Targets are severely overlapping, and feature differences are the key to differentiation. In this case, increase the weight of the comprehensive state cost and set α=0.2, β=0.6, γ=0.2 (comprehensive state cost is dominant).

[0037] Multiply the motion cost, comprehensive state cost, and environmental cost by their respective weighting coefficients α, β, and γ, and then add them together to obtain the total matching cost for each pair of measurement points and trajectories.

[0038] In this embodiment, since the traditional Hungarian algorithm requires calculation of the entire N×N cost matrix, which is highly complex, this invention constructs a sparse cost matrix (with dimensions M×K, M≤100, K≤20) based on the candidate matching set, as follows: The total matching cost of measurement point-trajectory pairs in the candidate matching set is filled into the matrix. The cost of measurement point-trajectory pairs not in the candidate matching set is set to the maximum value, which is considered an invalid match and is not included in the subsequent calculation. An improved Hungarian algorithm based on the classic Kuhn-Munkres algorithm framework is used to iteratively calculate the matching cost. If the calculated minimum matching cost exceeds the matching cost threshold (the matching cost threshold can be determined based on historical data statistics, i.e., by statistically analyzing the cost distribution of correct and incorrect results), the match is rejected to avoid incorrectly associating measurement points with the trajectory.

[0039] The output of the optimal matching result is divided into three categories: successful association pairs: measurement points and trajectories are successfully matched and used to update the trajectory status; unmatched measurement points: measurement points that are not matched with any trajectory, which may be newly emerging targets; unmatched trajectories: trajectories that are matched with any measurement points, which may be in an occluded state.

[0040] Association result judgment: The association result is judged. If the association result for a certain target shows continuous occlusion for more than N frames, the occlusion recovery procedure is started to restore the trajectory of the occluded target. If no occlusion occurs, the trajectory data is directly updated to obtain the trajectory continuation result after occlusion recovery.

[0041] The specific steps of the occlusion recovery procedure are as follows: First, extract the trajectory segments of the target for 20 consecutive frames before the occlusion occurs (including position, velocity, heading angle, and phase coherence confidence); and construct a surrounding situation map based on the motion state information of other targets within a 10-meter radius of the target's disappearance location. That is, for each neighboring target, calculate its relative position, relative velocity magnitude, and velocity vector angle relative to the occluded target. See Figure 2 As shown, the obtained trajectory fragments of the target itself and the surrounding situation map are input into the DRQN ​​prediction network for processing to obtain the optimal action number. The specific steps are as follows: The trajectory segment is input into the LSTM encoder. The LSTM network processes each frame of data in the trajectory segment step by step in time, extracting 128-dimensional motion features in the time dimension. The surrounding situation map is used as the input of the fully connected layer to extract 128-dimensional situation features. The motion features output from the LSTM and the situational features output from the fully connected layer are concatenated to form a 256-dimensional joint feature vector. This joint feature vector is then processed by a fully connected layer to output value scores for nine actions (corresponding to eight motion directions and one stationary action, such as...). The action with the highest value score is taken as the optimal action for the current state; the optimal action is numbered from 0 to 8, representing the predicted movement method, including staying still.

[0042] The determined optimal action is then mapped to a specific angle or stationary state. Assuming the target moves at a constant speed according to the speed before occlusion and the predicted direction, the predicted position is calculated based on the occlusion time (the position is updated once for each frame of occlusion). Then, a two-dimensional Gaussian distribution region is constructed with the predicted position as the center. The variance of the Gaussian distribution increases with the occlusion time (e.g., the initial standard deviation is 0.5 meters, and the standard deviation increases by 0.2 meters for each additional frame). Based on this, a probability heatmap is obtained.

[0043] By adding new measurement points Substitute the values ​​centered at the predicted location and with a standard deviation of The probability value of the new measurement point in the probability heatmap is calculated from the probability density function of the two-dimensional Gaussian distribution. The specific expression is as follows: In the formula: Indicates the location coordinates of the new measurement point; Indicates the predicted location coordinates; If the calculated probability value If the value exceeds a predetermined threshold (e.g., 0.6), then proceed to the motion consistency test and the comprehensive feature consistency test; otherwise, ignore. Motion consistency test: ,in Indicates the magnitude of the velocity at the new measurement point; Indicates the target's historical speed; This represents the motion consistency sensitivity coefficient, with a value of 0.5. The larger the value, the more stringent the sensitivity. Indicates the motor consistency score; Comprehensive feature consistency verification: ,in The 128-dimensional feature vector representing the target before it disappears (obtained based on a dynamic spatiotemporal graph); 128-dimensional feature vector of the target for sex detection (obtained based on dynamic spatiotemporal graph); Indicates the consistency score of comprehensive characteristics; The motion consistency score and the comprehensive feature consistency score are weighted and fused. When the total score after weighted fusion exceeds a predetermined threshold (e.g., 0.8), it is determined that the new measurement point belongs to the original target; otherwise, the association is rejected. If the new measurement point belongs to the original target, the acquisition is successful and the target trajectory is restored. If it fails, the occlusion recovery procedure continues. The fusion weight is dynamically adjusted according to the phase coherence confidence before the target is occluded. That is, the higher the phase coherence confidence, the higher the weight of the motion consistency score, with a maximum of 0.8 and a minimum of 0.3. The sum of the motion consistency score and the comprehensive feature consistency score is 1.

[0044] In this invention, a dynamic spatiotemporal graph is constructed, and edges between nodes are dynamically built based on dynamic consistency and spatial proximity rules. Combined with motion, environment, and temporal feature fusion, the feature graph can reflect the dynamic changes in node relationships in real time, achieving dynamic adaptation to changes in node relationships. Then, a lightweight relationship network is used to accurately calculate the similarity between nodes and trajectories and screen candidate matching pairs. With the introduction of the Hungarian algorithm with an environment adaptive cost matrix, the matching process can dynamically adjust the weights according to the environment, effectively resisting environmental interference and improving matching stability. Finally, continuous occlusion is detected by the association results and the occlusion recovery procedure is initiated, so that the trajectory can continue even when the target is occluded, ensuring the continuity of the association.

[0045] See Figure 3 As shown, in this embodiment, for probability values Measurement points less than or equal to a predetermined threshold (0.6) are determined to be first occurrences. If they are first occurrences, a temporary tracking document is created. If they are not first occurrences, the tracking document is updated and the number of consecutive occurrences is increased. When the number of consecutive occurrences reaches a predetermined number of frames (3), cross-frame correlation analysis is activated. Calculate the angle difference between the actual motion direction and the predicted motion direction to obtain the motion consistency coefficient. Check whether the target continues to move towards the center of the probability heatmap and whether the motion consistency coefficient is greater than a predetermined threshold. If the condition is met, a re-evaluation is performed. For measurement points that meet the conditions, a re-evaluation is performed, that is, after updating their own trajectory segments and surrounding situation maps, a new prediction is made, and a new probability heatmap is generated based on the new prediction results. The probability value of the measurement point is calculated based on the new probability heatmap. If the probability value is greater than a predetermined threshold, motion consistency verification is performed. If it is less than or equal to the predetermined threshold, the system returns to the memory retention state to continue observation.

[0046] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for dynamic correlation of measurement points in millimeter-wave radar for multi-target tracking, characterized in that, Includes the following steps: Target signal preprocessing: Acquire raw millimeter-wave radar data and preprocess the raw millimeter-wave radar data to obtain an enhanced three-dimensional measurement point set; Constructing a dynamic spatiotemporal graph: Each enhanced 3D measurement point in the current frame is taken as a node of the graph. Edges between nodes are constructed according to motion consistency rules and spatial proximity rules. Motion features, environmental features and temporal features are fused for each node to obtain a feature graph with topological relationships. Hierarchical quantization association: A lightweight relational network is used to calculate the similarity between each node in the feature map and the existing trajectory nodes. The top k candidate matching pairs with the highest similarity are retained to obtain a candidate matching set. The Hungarian algorithm is used to match each pair of measurement points and trajectories in the candidate matching set to obtain the association result between the target in the current frame and the existing trajectory. The Hungarian algorithm introduces an environment adaptive cost matrix. Association result judgment: The association result is judged. If the association result for a certain target shows continuous occlusion for more than N frames, the occlusion recovery procedure is started to restore the trajectory of the occluded target. If no occlusion occurs, the trajectory data is directly updated to obtain the trajectory continuation result after occlusion recovery.

2. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, The preprocessing of raw millimeter-wave radar data includes the following steps: The raw data from the millimeter-wave radar is processed by range-Doppler to obtain a complex domain 2D matrix. Then, a three-level complex domain wavelet packet transform is performed on the complex domain 2D matrix. The complex Morlet wavelet is used as the basis function to decompose the input signal into eight sub-band signals, with each level of decomposition focusing on a different frequency range. For each sub-band signal, the cross-pulse phase coherence coefficient is calculated to obtain the coherent phase coefficients of each of the eight sub-bands. Then, the weights are dynamically allocated according to the signal-to-noise ratio of each sub-band, and the correlation phase coefficients of the eight sub-bands are weighted and summed to obtain the comprehensive coherence coefficient. Based on a set threshold, points with a comprehensive correlation coefficient greater than the threshold are retained as candidate target measurement points. The candidate target measurement points are converted from polar coordinates to rectangular coordinates, and Doppler velocity information is extracted to obtain the set of candidate target measurement points. An enhanced three-dimensional measurement point set is obtained by performing adaptive particle filtering based on the candidate target measurement point set.

3. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 2, characterized in that, When using adaptive particle filtering, resampling and state estimation are introduced, as follows: Resampling: First, calculate the number of valid particles. If the number of valid particles is less than 50% of the total number of particles, resampling is triggered. The top 50% of particles by weight are retained, and new particles that follow the prior distribution are added. State estimation: The weighted average of the resampled particle swarm is used to obtain the compensated position and velocity, and based on this, the position and velocity of each candidate measurement point after motion compensation are obtained.

4. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, The process of constructing edges between nodes based on motion consistency rules and spatial proximity rules includes the following steps: For any two nodes, calculate their Euclidean distance. If the Euclidean distance is less than or equal to a preset distance threshold, then the two nodes are spatially adjacent. For the same pair of nodes, calculate the angle between their velocity vectors. If the target is stationary, the angle is set to 0 by default. Otherwise, if the angle is less than or equal to a preset angle threshold, the two nodes move in the same direction. For node pairs that simultaneously satisfy spatial proximity and known motion, add undirected edges and calculate weights based on distance and included angle to obtain a weighted edge set.

5. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, The motion characteristics include velocity, velocity magnitude, acceleration, acceleration magnitude, velocity direction angle, and acceleration direction angle; The environmental features include surrounding target density, reflection intensity, phase coherence confidence, and occlusion probability; The temporal features include the rate of change of position, the variance of the rate of change of position, and the position offset.

6. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, The environment adaptive cost matrix is ​​obtained by weighted summation of three sub-cost terms, namely motion cost, comprehensive state cost, and environment cost. The motion cost is used to quantify the consistency of motion state between the measurement point and the trajectory; the comprehensive state cost is used to quantify the feature similarity between the measurement point and the trajectory; and the environment cost is used to quantify the impact of environmental complexity on matching. The weight coefficients of each sub-cost item are dynamically adjusted based on the constructed scene type, which is divided into low-density scene, medium-density scene, and high-density scene based on the target density. In the low-density scene, the weight of motion cost is increased. In the medium-density scene, the weights of motion cost and comprehensive state cost are equal. In the high-density scene, comprehensive state cost is dominant. The weights of motion cost, comprehensive state cost, and environmental cost are summed to 1.

7. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, When executing the Hungarian algorithm, a cost matrix is ​​constructed based on the candidate matching set. The matching costs of candidate pairs in the candidate matching set are filled into the matrix, and the costs of candidate pairs not in the candidate matching set are set to invalid. Then, the improved Hungarian algorithm based on the Kuhn-Munkres algorithm framework is called to perform the optimal matching solution. In the solution process, a cost threshold filter is introduced. If the minimum matching cost of a matching pair exceeds a predetermined threshold, the corresponding match is rejected. Based on this, the optimal matching result is obtained, that is, the association result between the target in the current frame and the existing trajectory.

8. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 1, characterized in that, The occlusion restoration procedure includes the following steps: The target's own trajectory segment is extracted based on the trajectory segments of the target in consecutive n frames before the occlusion occurs; a surrounding situation map is constructed based on the motion state information of other targets within a predetermined range around the target's disappearance location; The target's own trajectory segment is used as input to the LSTM network to extract motion features in the time dimension; the surrounding situation map is used as input to the fully connected layer to extract situation features; the motion features and situation features are concatenated to form a joint feature vector; and the joint feature vector is then processed by the fully connected layer to obtain the optimal action number, which represents the predicted direction of motion. The optimal action number is mapped to a specific angle. Assuming that the target moves at a constant speed according to the speed before occlusion and the predicted direction, the predicted position is calculated based on the duration of occlusion. A two-dimensional Gaussian distribution region is constructed with the predicted position as the center to generate a probability heatmap. The variance of the Gaussian distribution region increases with the occlusion time. Calculate the probability value of the new measurement point in the probability heatmap. If the probability value is greater than the predetermined threshold, perform motion consistency verification; otherwise, ignore it. The motion consistency verification involves calculating the absolute difference between the velocity magnitude of the new measurement point and the historical velocity magnitude of the target, converting it into a consistency score using an exponential function; then calculating the cosine similarity between the feature vector of the new measurement point and the historical feature vector of the target; adjusting the weights of the consistency score and cosine similarity based on the phase coherence confidence of the target before occlusion, and performing a weighted summation. If the summation value exceeds a predetermined threshold, the new measurement point is determined to belong to the original target; otherwise, the association is rejected, and the occlusion recovery procedure continues.

9. The method for dynamic association of millimeter-wave radar measurement points for multi-target tracking according to claim 8, characterized in that, For measurement points with probability values ​​less than or equal to a predetermined threshold, determine whether it is the first occurrence. If it is the first occurrence, create a temporary tracking document; otherwise, update the tracking document and increase the number of consecutive occurrence frames. When the number of consecutive occurrence frames reaches a predetermined number, activate cross-frame correlation analysis. Calculate the angle difference between the actual motion direction and the predicted motion direction to obtain the motion consistency coefficient. Check whether the target continues to move towards the center of the probability heatmap and whether the motion consistency coefficient is greater than a predetermined threshold. If the condition is met, a re-evaluation is performed. For measurement points that meet the conditions, a re-evaluation is performed, that is, after updating their own trajectory segments and surrounding situation maps, a new prediction is made, and a new probability heatmap is generated based on the new prediction results. The probability value of the measurement point is calculated based on the new probability heatmap. If the probability value is greater than a predetermined threshold, motion consistency verification is performed. If it is less than or equal to the predetermined threshold, the system returns to the memory retention state to continue observation.

Images