A high-robust multi-target trajectory tracking method based on millimeter wave radar

Abstract

This invention belongs to the field of wireless sensing technology and discloses a robust multi-target trajectory tracking method based on millimeter-wave radar. The steps are as follows: First, millimeter-wave radar signal acquisition and preprocessing are performed to obtain a range-angle map of the target after removing the static background; then, target point detection is performed; next, target trajectory prediction and matching updates are performed; finally, by combining an adaptive Kalman correction mechanism based on sharp turn detection with a stationary person recognition and trajectory maintenance mechanism based on respiratory micro-motion features, stable tracking of personnel trajectories in complex indoor scenes is achieved. Compared with traditional tracking methods, the robust multi-target trajectory tracking method of this invention can maintain trajectory stability even in scenarios where the target makes sharp turns or remains stationary, exhibiting higher robustness.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of wireless sensing technology, and relates to a high-robustness multi-target trajectory tracking method based on a millimeter wave radar. BACKGROUND

[0002] Accurate real-time personnel trajectory tracking is a key enabling technology for emerging scenarios such as smart home, elderly care monitoring, factory security, etc. The commonly used indoor personnel detection method at present is mainly visual detection based on a visible light camera. Although this kind of method can provide high-resolution image information, it is prone to detection failure under poor lighting conditions such as insufficient light and strong backlight, and its dependence on imaging also brings a greater privacy risk, limiting its popularization and application in sensitive scenarios such as the home.

[0003] Compared with a camera, a millimeter wave radar, as a new emerging sensing means, has the advantages of all-weather, privacy protection, and penetration, and has gradually become an important supplement to the visual solution. In the field of indoor personnel sensing based on a millimeter wave radar, the existing multi-target tracking method usually adopts a relatively fixed processing flow: first, a target point cloud is extracted through a constant false alarm rate (CFAR) detection algorithm and a clustering algorithm, then target state prediction is performed based on a Kalman filter, and multi-target matching is achieved through a Hungarian algorithm or a similar data association method, and finally, the associated data is fused and updated through the Kalman filter. This kind of technical system of “point cloud clustering + Kalman prediction + Hungarian association + Kalman update” has been applied in many patents. For example, patent CN110361727A proposes a millimeter wave radar multi-target tracking method, which realizes a typical multi-target tracking process based on point cloud clustering and prediction update by performing a density-based spatial clustering algorithm (DBSCAN) on point clouds and combining Kalman filtering and data association algorithms, representing the current general technical route of millimeter wave radar multi-target tracking.

[0004] The above-mentioned existing millimeter wave radar trajectory tracking method is generally applicable to scenarios where the target motion is continuous and the direction is smooth. However, in actual scenarios, indoor personnel behavior may have the following two situations, which may cause the existing method to track inaccurately:

[0005] (1) When a person makes a sharp turn, suddenly turns around, or has other drastic changes in direction, since the Kalman filter is used to predict along the direction of the previous frame speed, the predicted position will deviate from the true position, and this deviation will accumulate, causing trajectory drift;

[0006] (2) When a person suddenly stops moving, the echo reflection will be considered as static background and suppressed in the background noise processing stage, causing the detection points to completely disappear, thereby causing the trajectory to be interrupted.

[0007] Among them, in order to improve the detection ability of static personnel, the prior art has carried out exploration. For example, patent CN111289967A constructs a state machine mechanism of "detection-activation-still-release", infers the personnel state through the change of the point trace, and maintains the trace continuity under the still condition. However, this method completely depends on the point trace change rule and lacks the ability to judge the existence of the human body from the signal itself. Therefore, when the point trace abnormally decreases or temporarily disappears due to occlusion, posture change or noise threshold fluctuation, the method cannot distinguish between "the real still target still exists" and "the target really leaves", which easily leads to problems such as track deletion and activation error.

[0008] In summary, although the prior art has tried to improve the detection ability of static personnel to some extent, it still lacks a high-robustness technical solution that can simultaneously solve the problems of "prediction deviation accumulation caused by sharp turning" and "track disappearance of static personnel caused by background suppression". SUMMARY

[0009] The present application provides a high-robustness multi-target track tracking method based on millimeter wave radar, which effectively solves the two common problems in the above-mentioned actual scene by introducing an adaptive Kalman correction mechanism based on sharp turning detection and a still personnel identification and track keeping mechanism based on breathing micro-motion features into the traditional multi-target tracking framework.

[0010] The present application utilizes the multi-dimensional features of radar echoes, combines multi-frame motion direction angle analysis, velocity state reconstruction and adaptive noise adjustment, realizes track correction of the target in the case of sharp turning, and at the same time, realizes the existence identification of static personnel based on the phase sequence autocorrelation analysis of the prediction area, so that the multi-target track can maintain continuity and reliability. Specifically, first, the echo signal collected by the millimeter wave radar is preprocessed to obtain a range-angle graph, and the detection points of each target personnel are extracted through CFAR and clustering algorithm; then, a Kalman filter is used for track prediction and update, wherein an adaptive Kalman correction mechanism based on sharp turning detection is introduced to adaptively adjust the Kalman state parameters to maintain the stability of the track when sharp turning is detected; a still personnel identification and track keeping mechanism based on breathing micro-motion features is designed, when the track is temporarily lost, the identification and recovery of the static target are realized in combination with the breathing phase detection, and finally the track and existence state of the multi-target are output.

[0011] The specific steps are as follows:

[0012] Step one: millimeter wave radar signal acquisition and preprocessing;

[0013] The millimeter wave radar sends a frequency-modulated continuous wave (FMCW) signal through a transmitting antenna, and the frequency-modulated continuous wave signal is reflected by a target and received by a receiving antenna array to obtain a return signal; the return signal is mixed with the frequency-modulated continuous wave signal to obtain an intermediate frequency signal; a distance fast Fourier transform (distance-FFT) is performed on the intermediate frequency signal to extract the distance of the target, and a Capon digital beamforming algorithm is used to estimate the angle of the return signal received by the receiving antenna array to obtain the angle of the target, and then the distance and angle of the target are combined to form a distance-angle map of the target;

[0014] To suppress the background static reflection (such as walls, furniture, etc.), the distance-angle maps of the targets of the past several frames are averaged to establish a background model, and the distance-angle map of the target of the current frame is subtracted from the background model to obtain a distance-angle map of the target after removing the static background;

[0015] Step two: target point detection;

[0016] The constant false alarm rate detection algorithm is used to extract the scattering points from the distance-angle map of the target after removing the static background, and the density-based spatial clustering algorithm is used to cluster and group the scattering points; for each cluster of scattering points, the centroid position is calculated, and the centroid position is taken as the detection point position of the target to extract the detection point of each target;

[0017] Step three: target trajectory prediction and matching update;

[0018] The detection points of each target obtained in step two are combined to predict and update the target trajectories established in the historical frames; each target trajectory is configured with an independent Kalman filter for continuous estimation of the motion state of the target, including the position vector and velocity vector of the target in the plane coordinate system, and a covariance matrix and a process noise matrix are configured to describe the uncertainty of the motion state of the target and the change of the prediction error;

[0019] The Kalman filter updates the target trajectory according to the following process:

[0020] (1) Prediction stage:

[0021] According to the motion state of the target in the last frame, the Kalman filter predicts the target trajectory in the current frame to obtain a prediction result, including a position vector and a velocity vector, and the position vector is a prediction point; at the same time, the covariance matrix of the motion state is updated by using the process noise matrix;

[0022] (2) Matching stage:

[0023] The distance or cost between the prediction point and the detection point of each target obtained in step two is calculated to construct a cost matrix, and the Hungarian algorithm is used for global optimal matching to ensure that each target trajectory is matched to the most suitable detection point;

[0024] (3) Update stage:

[0025] When the target trajectory is successfully matched to the detection point, the prediction result of the target trajectory is corrected using the detection point, and the position vector and velocity vector of the target and the corresponding covariance matrix are updated to obtain the target trajectory update result of the current frame;

[0026] Step four: adaptive Kalman correction mechanism based on sudden turning detection;

[0027] When the target suddenly turns, its true motion direction will change significantly in a short time, and the Kalman prediction in step three will continue to calculate the current position based on the speed direction of the last frame, which will cause a large deviation between the predicted point and the actual detection point. If the Kalman update is still performed using the predicted point with a large deviation, the trajectory position will deviate from the true trajectory frame by frame, further causing the prediction direction to continue to be wrong, and eventually may cause the trajectory direction to be abnormal or even the trajectory to be broken.

[0028] To solve the above problems, the present application proposes an adaptive Kalman correction mechanism based on sudden turning detection. When the target suddenly turns, its motion direction changes significantly in a short time. The adaptive Kalman correction mechanism based on sudden turning detection is proposed. First, it is detected whether the target suddenly turns. If so, the update stage in step three (3) is modified.

[0029] Specifically: 1) Estimate the motion direction of the target using the motion state of three consecutive frames: first, calculate the speed vector of the last frame using the position vector of the target updated in the last two frames; then, calculate the speed vector of the current frame using the position vector of the target updated in the last frame and the detection point of the current frame; finally, calculate the included angle between the two speed vectors. When the included angle exceeds 135°, it is determined that the target suddenly turns in the current frame;

[0030] 2) After detecting that the target suddenly turns, the update stage in step three (3) is modified as follows: first, ignore the predicted point given by the Kalman filter in the current frame, and directly use the detection point in the current frame as the updated position vector to prevent the target trajectory from continuing to extend in the wrong direction; second, update the speed vector in the Kalman filter to the speed vector calculated according to the current detection point; finally, increase the process noise of the Kalman filter to reduce the confidence of the Kalman filter in the prediction state in the update stage; thereby correspondingly increasing the weight of the detection point, so that the state estimation is more dependent on the real observation data, in order to quickly adapt to the possible changing motion situation.

[0031] Step five: still person recognition and trajectory keeping mechanism based on breathing micro-motion characteristics;

[0032] After the above trajectory update is completed, since the static background modeling and background difference method is adopted in step one, the reflection signal of fixed objects such as furniture and walls is suppressed. Based on the fact that when the target is static, the echo signal does not produce dynamic changes, it is regarded as a static background and is filtered out, resulting in the target point being unable to be detected.

[0033] To solve the above problems, the present application proposes a still personnel identification and trajectory keeping mechanism based on breathing micro-motion characteristics. Specifically, after detecting that the target trajectory does not match the detection point in the current frame, the radar phase signal is extracted in the neighborhood region of the target trajectory termination position, and breathing micro-motion analysis is performed. Since the human body still produces periodic chest movements in the static state, the periodic chest movements are reflected in the radar phase signal. If the radar phase signal is detected to have stable periodic micro-motion, it is determined that the target is a still personnel, and the target trajectory is kept. At the same time, the distribution of the detection point around the target trajectory termination position is continuously monitored in the subsequent frames. Once the detection point is detected again near the target trajectory termination position, the detection point is automatically associated with the target trajectory and the step three (3) update phase is executed, realizing the trajectory continuation when the target moves again after being static.

[0034] Finally, according to the state of all targets, the output includes position, trajectory and state category (moving / still), realizing real-time detection and continuous trajectory tracking of multiple targets.

[0035] The beneficial effects of the present application are:

[0036] (1) The present application designs a high-robustness multi-target trajectory tracking method based on millimeter wave radar, which can maintain stable trajectory in the scene of target sharp turning or keeping still, and has higher robustness compared with traditional tracking methods.

[0037] (2) The adaptive Kalman correction mechanism based on sharp turning detection proposed by the present application can automatically reset the speed state of the Kalman filter and adaptively adjust the process noise when the target sharp turns or stops, so as to quickly correct the deviation of the trajectory. Compared with the traditional radar tracking method which depends on the uniform motion assumption, the present application can effectively handle the nonlinear motion behaviors such as frequent turning and sudden stopping, and significantly reduce the trajectory drift and break caused by the accumulation of prediction deviation. The mechanism greatly improves the continuity and stability of the trajectory, and ensures that the system can still realize reliable and accurate tracking of multiple targets in complex indoor environments.

[0038] (3) The application proposes a still person recognition and trajectory keeping mechanism based on respiratory micro-motion characteristics, realizing high-reliability person detection in a static state. By analyzing the weak periodic respiratory motion in the radar phase signal, the static human body can be accurately distinguished from static reflectors such as furniture and walls, and the technical bottleneck that the traditional background difference is easy to delete the static target in detection is fundamentally overcome. This method can still maintain the trajectory continuity when the target is completely static, and has significant application value for smart home, elderly care, indoor security monitoring and other scenes. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is the trajectory tracking result of the method of the application in a scene. DETAILED DESCRIPTION

[0040] The specific embodiments of the application will be further described below in combination with the drawings and technical solutions.

[0041] EMBODIMENT

[0042] Based on a 24GHz millimeter wave radar platform, real-time detection of multiple personnel targets, stable tracking of trajectory in sudden turning, and accurate identification of static personnel can be realized. The specific steps are as follows:

[0043] First, a 24GHz millimeter wave radar is used as a sensing device to continuously scan the target scene. The 24GHz millimeter wave radar periodically sends a frequency-modulated continuous wave signal through the transmitting antenna, and each transmitting period forms a frame of data; the return signal is received by the receiving antenna array, and the intermediate frequency signal is obtained by mixing, and is transmitted to the host computer in real time through the data acquisition board.

[0044] In the host computer, for each frame of received intermediate frequency signal, according to step one, the Capon digital beamforming algorithm operation is performed on the return signal to form a range-angle diagram. Then, the background model is obtained by averaging the past continuous multiple frames of range-angle diagrams, and the current frame range-angle diagram is subtracted from the background model to obtain the range-angle diagram of the target after removing the static background.

[0045] According to step two, the constant false alarm detection algorithm is used to extract the scattering points from the range-angle diagram of the target after removing the static background, and the density-based spatial clustering algorithm is used to group the scattering points. The centroid of each clustering cluster is used as the detection point of the corresponding target for subsequent trajectory tracking.

[0046] According to step three, after obtaining the detection points of the current frame, the Kalman filter is used to predict and update each target trajectory. First, according to the motion state of the target in the previous frame, the predicted position (predicted point) and the predicted speed of the target in the current frame are predicted, and the state covariance matrix is updated. Then, a cost matrix is constructed based on the distance between the predicted point and the detection point, and the Hungarian algorithm is used to achieve global optimal matching. Finally, the matched detection point is used to correct the predicted state of the trajectory, update the position vector and the speed vector of the target, and obtain the trajectory result of the current frame.

[0047] According to step four, an adaptive Kalman correction mechanism based on sudden turning detection is introduced in the trajectory prediction and update process. The motion direction of the target is estimated using the motion state of three consecutive frames: first, the speed vector of the previous frame is calculated using the updated position vector of the target in the previous two frames; then, the speed vector of the current frame is calculated using the updated position vector of the target in the previous frame and the detection point of the current frame; finally, the included angle between the two speed vectors is calculated, and when the included angle exceeds 135°, it is determined that the target has a sudden turning in the current frame; after detecting the sudden turning of the target, the update stage of step three (3) is modified as follows: first, ignore the predicted point given by the Kalman filter in the current frame, and directly use the detection point of the current frame as the updated position vector, to prevent the target trajectory from continuing to extend in the wrong direction; second, update the speed vector in the Kalman filter to the speed vector calculated based on the current detection point; finally, increase the process noise of the Kalman filter to reduce the trust degree of the Kalman filter in the update stage.

[0048] Finally, according to step five, after detecting that the target trajectory does not match the detection point in the current frame, the radar phase signal is extracted in the neighborhood region of the target trajectory termination position, and the breathing micro-motion analysis is performed; since the human body still produces periodic chest movement in the static state, the periodic chest movement is reflected in the radar phase signal; if the radar phase signal is detected to have stable periodic micro-motion, it is determined that the target is a stationary person, and the target trajectory is maintained; at the same time, the distribution of detection points around the target trajectory termination position is continuously monitored in the subsequent frames; once a detection point is detected again near the target trajectory termination position, the detection point is automatically associated with the target trajectory and the update stage of step three (3) is executed, realizing the trajectory continuation when the target is stationary again.

[0049] Finally, the system outputs the position coordinates, trajectory and motion category (moving / stationary) of multiple targets, realizing continuous detection and tracking of multiple targets in indoor environment.

[0050] In this embodiment, an experimental scene is constructed in a typical indoor environment, and the trajectories of two subjects are tracked and verified. One of the subjects walks freely in the indoor area and contains a sudden turning action; the other subject walks to a specified position and remains stationary for about 10 seconds. Under the above experimental scene,Figure 1 The trajectory tracking results of the method of the present application in this scene are shown. The solid line represents the trajectory of the moving subject, and the method effectively suppresses the prediction deviation through the sharp turning detection and adaptive Kalman correction mechanism, so that the trajectory remains smooth at the sharp turning position without obvious drift and breakage. The dashed line represents the stationary subject, and due to the stationarity, its trajectory fails to match the detection points in multiple frames. The method successfully detects the breathing micro-movement of the stationary person and keeps the trajectory continuously existing through the stationary person recognition and trajectory keeping mechanism based on the breathing micro-movement characteristics, without false deletion, fully verifying the high robustness and effectiveness of the method in actual indoor applications.

Claims

1. A highly robust multi-target trajectory tracking method based on millimeter wave radar, characterized in that, The method comprises the following steps: Step one: millimeter wave radar signal acquisition and preprocessing, obtaining the distance-angle map of the target after removing the static background; Step two: target point detection; The constant false alarm rate detection algorithm is used to extract the scattering points from the distance-angle map of the target after removing the static background, and the density-based spatial clustering algorithm is used for clustering and grouping the scattering points; for each cluster of scattering points, the centroid position is calculated, and the centroid position is taken as the detection point position of the target, and the detection point of each target is extracted; Step three: target trajectory prediction and matching update; In combination with the detection point of each target obtained in step two, the target trajectory established in the historical frame is predicted and updated; each target trajectory is configured with an independent Kalman filter for continuous estimation of the motion state of the target, including the position vector and velocity vector of the target in the plane coordinate system, and a covariance matrix and a process noise matrix are configured to describe the uncertainty of the motion state of the target and the change of the prediction error; Step four: adaptive Kalman correction mechanism based on sudden turning detection; When the target suddenly turns, the motion direction of the target changes significantly in a short time, and the adaptive Kalman correction mechanism based on sudden turning detection is proposed, that is, first, it is detected whether the target suddenly turns, and if so, the target trajectory update in step three is corrected; The specific implementation process of step four is as follows: Step 4.1, the motion direction of the target is estimated by using the motion state of three consecutive frames: first, the position vector of the target updated in the previous two frames is used to calculate the velocity vector of the previous frame; then, the position vector of the target updated in the previous frame and the detection point of the current frame are used to calculate the velocity vector of the current frame; finally, the included angle between the two velocity vectors is calculated, and when the included angle exceeds 135°, it is determined that the target suddenly turns in the current frame; Step 4.2, after detecting that the target suddenly turns, the update stage in step three is corrected as follows: first, the prediction point given by the Kalman filter in the current frame is ignored, and the detection point in the current frame is directly taken as the updated position vector, to prevent the target trajectory from continuing to extend in the wrong direction; secondly, the velocity vector in the Kalman filter is updated to the velocity vector calculated according to the current detection point; finally, the process noise of the Kalman filter is increased, so that the Kalman filter reduces the trust degree of the predicted state in the update stage; Step five: static person identification and trajectory keeping mechanism based on breathing micro-motion characteristics; Based on the fact that when the target is static, the echo signal does not produce dynamic changes, so it is filtered out as static background, resulting in the inability to detect the target, a static target identification and trajectory keeping mechanism based on breathing micro-motion characteristics is proposed; The specific implementation process of step five is as follows: After detecting that the target trajectory is not matched to the detection point in the current frame, the radar phase signal is extracted in the neighborhood region of the target trajectory termination position, and the breathing micro-motion analysis is performed; since the human body still produces periodic chest movement in the static state, the periodic chest movement is reflected in the radar phase signal; if the radar phase signal is detected to have stable periodic micro-motion, it is determined that the target is a static person, and the target trajectory is kept existing; at the same time, the distribution of the detection point around the target trajectory termination position is continuously monitored in the subsequent frame; once the detection point is detected again near the target trajectory termination position, the detection point is automatically associated with the target trajectory, and step three update stage is performed, realizing the trajectory continuation when the static person moves again; Finally, the position, trajectory and state category are output according to the state of all targets, realizing the real-time detection and continuous trajectory tracking of multiple targets.

2. The high-robust multi-target trajectory tracking method based on millimeter wave radar according to claim 1, characterized in that, The specific implementation process of step one is as follows: The millimeter wave radar sends a frequency-modulated continuous wave signal through a transmitting antenna, and the frequency-modulated continuous wave signal is reflected by a target and received by a receiving antenna array; the return signal and the frequency-modulated continuous wave signal are mixed to obtain an intermediate frequency signal; the intermediate frequency signal is subjected to distance fast Fourier transform to extract the distance of the target, and a Capon digital beamforming algorithm is used to estimate the angle of the return signal received by the receiving antenna array to obtain the angle of the target, and then the distance and angle of the target are combined to form a distance-angle map of the target; The distance-angle map of the target of the past several frames is averaged to establish a background model, and the distance-angle map of the target of the current frame is subtracted from the background model to obtain a target distance-angle map without static background.

3. The high-robust multi-target trajectory tracking method based on millimeter wave radar according to claim 2, characterized in that, The Kalman filter in step three updates the target trajectory as follows: Step 3.1, prediction stage: according to the motion state of the target in the last frame, the Kalman filter predicts the target trajectory in the current frame to obtain a prediction result, including a position vector and a velocity vector, the position vector being a prediction point; at the same time, the covariance matrix of the motion state is updated by using the process noise matrix; Step 3.2, matching stage: the prediction point is subjected to distance or cost calculation with each detection point of the target obtained in step two to construct a cost matrix, and the Hungarian algorithm is used for global optimal matching to ensure that each target trajectory is matched to the most suitable detection point; Step 3.3, update stage: After the target trajectory is successfully matched to the detection point, the prediction result of the target trajectory is corrected by using the detection point to update the position vector and the velocity vector of the target and the corresponding covariance matrix, and the target trajectory update result in the current frame is obtained.

Images