Stable multi-target detection and positioning tracking method based on millimeter radar

Abstract

The invention discloses a stable multi-target detection and positioning tracking method based on millimeter radar. According to the method, firstly, abnormal values are eliminated through a Dijkstra algorithm, targets are accurately recognized, and the number, center points and geometric parameters of the targets are obtained; then, a Kalman filter is used to predict and correct the trajectory so as to deal with the condition that the sensor is shielded or the target is temporarily lost. In addition, trajectory allocation is optimized by adopting a Hungary algorithm, association between the target and the trajectory is effectively managed, and waste of computing resources is reduced. And through cost matrix enhancement processing, the accuracy and the high efficiency of track distribution are ensured. According to the method, the performance of a multi-target tracking system can be remarkably improved, accidents caused by missing detection are reduced, and the method has very high practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-target tracking, and in particular to a stable multi-target detection and positioning tracking method based on millimeter radar. Background Art

[0002] The boundaries of tracking and sensing solutions are constantly being pushed. One of the stimuli in the field in recent years has been the use of the characteristics of millimeter wave (mmWave) radar to achieve simultaneous tracking and detection of multiple targets. There is a large literature on single target tracking using mmWave radar, which proves the maturity of single target tracking systems. However, the field of mmWave radar multi-target tracking also needs innovative research and progress.

[0003] The most widely used millimeter wave radar technology at present is frequency modulated continuous wave (FMCW) technology. Compared with the traditional pulse radar system that periodically transmits short pulses, FMCW radar continuously transmits frequency modulated signals to measure the distance and angle of objects. Generally, according to the traditional signal processing position detection method, the noise of the reflected millimeter wave radar signal is removed, and the position information contained in the signal is obtained using the FFT method. However, the complexity of the harsh signal propagation environment makes the tracking and detection of multiple targets very difficult.

[0004] In particular, stable tracking of multiple targets that interrupt the field of view is also one of the key technologies in the field of multi-target tracking of millimeter-wave radar.

[0005] Millimeter-wave radar has been widely studied in recent years for multi-target tracking and detection. The potential and power of millimeter-wave radar in this field is mainly driven by the micro-Doppler information that can be extrapolated. Micro-Doppler generally refers to the Doppler information generated by the motion of various parts of a specific target. The micro-Doppler function can be used to determine the characteristics of multiple targets for tracking and detection. The identified characteristics can eventually be converted into a single motion of the target below the millimeter. This is due to the high sensitivity of mmWave radar driven by very short wavelengths.

[0006] If multiple targets move in the same environment, three additional problems need to be solved.

[0007] First, if the dataset has many more outliers than individual targets, removing these outliers is critical to performance. Second, the radar points must be clustered and accurately associated with other moving targets. Finally, the time-varying positions of these clusters must consistently indicate the location of the target.

[0008] The usual multi-target tracking methods include linking measurements to tracks, managing the birth and death of tracks, and predicting track parameters and measurements. In the past, various complex algorithms including Multi-Hypothesis Tracker (MHT) were developed to generate potential hypothesis trees for each target, so as the number of targets increases, combinatorial explosion and computational overload will occur.

[0009] Joint Probabilistic Data Association (JPDA) calculates all possible joint events to calculate the association probability and requires knowledge about the clutch speed and detection probability to calculate. Summary of the invention

[0010] Based on this, the embodiment of the present application provides a stable multi-target detection and positioning tracking method based on millimeter radar, which uses the Dijkstra algorithm to remove outliers and detect and classify targets on millimeter radar point cloud data, and uses the Hungarian algorithm to allocate tracks. Finally, the Kalman detector is used to estimate the allocated tracks, and the detected tracks are updated and deleted, so that stable and accurate results can be returned.

[0011] In a first aspect, a stable multi-target detection and positioning tracking method based on millimeter radar is provided, the method comprising:

[0012] S10: Remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters;

[0013] S20: predict the next position of the millimeter wave radar trajectory;

[0014] S30: Determine the numbers of the detected targets associated with the existing tracks, the targets for which new tracks need to be created, and the unassigned tracks;

[0015] S40: Correcting the trajectory, allocating the newly detected trajectory and deleting the undetected trajectory.

[0016] Optionally, S10 specifically includes:

[0017] Use SKY32B750_EVB millimeter wave radar to obtain the point cloud data of the target, including the number of point cloud data, distance information of X, Y, and Z axes, and speed information in the radial direction;

[0018] Initialize to process each radar point cloud data;

[0019] For each point cloud data, calculate the distance between it and other point cloud data. If the distance is less than the set threshold, they are considered to belong to the same target point. Otherwise, they are considered as outliers and removed.

[0020] For the point cloud data determined as the target, its center point and geometric parameters are calculated; wherein the geometric parameters include length, width, and height.

[0021] Optionally, S20 specifically includes:

[0022] Use Kalman filters to correct for sensor noise and predict the next position of tracked objects that cannot be detected due to occlusion or temporary loss of the sensing area;

[0023] For each trajectory, a state vector is maintained, including the position and velocity along the x and y axes, and the next state is predicted based on the current state.

[0024] Optionally, S30 specifically includes:

[0025] S301: Calculate the cost matrix using the trajectory state and the distance between the detection target center point;

[0026] S302: enhancing the cost matrix using stacking items;

[0027] S303: Apply the Hungarian algorithm to the cost matrix to obtain the track number and target number with the lowest distance loss, thereby determining the association between the detected target and the existing track, and obtaining the allocation result between the detected target and the existing track.

[0028] Optionally, S40 specifically includes:

[0029] S401: predict and correct the existing trajectory, and use the Kalman filter to update the predicted value and uncertainty of the detected target;

[0030] S402: For a newly detected target, a new track is created and its position, velocity and geometric parameters are initialized. For an existing track, its status is updated, including track age and target appearance count;

[0031] S403: If the track object is not detected in a plurality of consecutive frames, it is marked as inactive and deleted from the track set.

[0032] Optionally, S402 specifically includes:

[0033] Give each trajectory a trajectory number, and give the trajectory position and the velocity and geometric parameters of the trajectory target, and the trajectory position gives the center point of the trajectory target;

[0034] Initialize the Kalman filter of each track, initialize the track age and the number of target appearances to the number of discontinuous targets; the track age is the total number of frames output from the millimeter wave radar, and the number of target appearances is the value that increases one by one when the target detected in each frame appears on the corresponding track;

[0035] In the update of an existing track, the age of the track is increased each frame, if the target appears on the track, the number of target appearances is increased, and the number of discontinuous target appearances is initialized to zero, and if the target does not appear in the track, the number of discontinuous target appearances is increased by one.

[0036] In the second aspect, a stable multi-target detection and positioning tracking system based on millimeter radar is provided, the system comprising:

[0037] The acquisition module is used to remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters;

[0038] A prediction module, used to predict the next position of the millimeter-wave radar trajectory;

[0039] a determination module, used to determine the number of the detected targets associated with the existing tracks, the targets for which new tracks need to be created, and the unassigned tracks;

[0040] Tracking module, which is used to correct tracks, assign newly detected tracks and delete undetected tracks.

[0041] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the stable multi-target detection and positioning tracking method described in any one of the first aspects is implemented.

[0042] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the stable multi-target detection and positioning tracking method described in any one of the first aspects is implemented.

[0043] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the stable multi-target detection and positioning tracking method described in any one of the first aspects above.

[0044] The technical solution provided in the embodiment of the present application proposes a method for reasonably managing the connection between the detection target and the trajectory, the generation and disappearance of the trajectory, and predicting and modifying the trajectory on the millimeter wave radar, thereby supporting the collision avoidance of the autonomous system and reducing accidents caused by missing detection. It is very practical and promotes the rapid implementation of actual development and testing.

[0045] By using the Dijkstra algorithm to eliminate outliers, the present invention can more accurately identify targets from millimeter-wave radar point cloud data, reducing false detections and missed detections.

[0046] By using the Kalman filter to predict and correct the trajectory, the present invention can maintain stable tracking when the target is temporarily lost, thereby improving the continuity and reliability of multi-target tracking.

[0047] By performing trajectory allocation and enhanced processing of the cost matrix using the Hungarian algorithm, the present invention can efficiently manage the association between targets and trajectories, reduce the waste of computing resources, and improve the processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0049] Figure 1 The present invention is a flowchart of a stable multi-target detection and positioning tracking method based on millimeter radar in an embodiment.

[0050] Figure 2 It is a flowchart for removing outliers from millimeter radar point cloud data and obtaining the number of targets, center points and geometric parameters.

[0051] Figure 3 The present invention is a flowchart for determining the numbers of detected targets associated with existing tracks, targets for which new tracks need to be created, and unassigned tracks.

[0052] Figure 4 It is a flowchart for predicting and correcting trajectories, allocating newly detected trajectories, updating existing trajectories, and deleting undetected trajectories. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In the description of the present invention, the terms "comprises", "has" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may also include other steps or units that are not explicitly listed but are inherent to these processes, methods, products or apparatuses, or steps or units added based on further optimization schemes conceived by the present invention.

[0055] In this paper, Dijkstra algorithm is used to remove outliers and detect and classify targets from millimeter radar point cloud data, and the Hungarian algorithm is used to assign tracks. Finally, Kalman detector is used to estimate the assigned tracks, and the detected tracks are updated and deleted. The proposed method is experimentally validated using a dataset collected by SKY32B750 module, which uses RC6011B-AIP (59-64GHz) radar transceiver chip. The reported results show that the developed algorithm will return stable and accurate results if properly tuned.

[0056] Please refer to Figure 1 , which shows a flow chart of a stable multi-target detection and positioning tracking method based on millimeter radar in one embodiment, the method may include the following steps:

[0057] S10: Remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters.

[0058] The present invention utilizes the Dijkstra algorithm to eliminate abnormal values ​​and detect targets in millimeter wave radar point cloud data.

[0059] Dijkstra's algorithm is a popular algorithm used to solve many single-source shortest path problems with non-negative edge weights in graphs, that is, to find the shortest distance between two vertices on a graph.

[0060] However, the present invention does not focus on using the Dijkstra algorithm to find the shortest distance. Here, a certain distance limit is given. For points that enter the distance limit, if the number of point sets exceeds the set value, they are regarded as the same target point, saved in the memory area, and the number of targets is increased. Otherwise, they are regarded as abnormal values ​​and deleted.

[0061] In this way, for each target point set obtained, the center point and geometric integers, that is, the center point and size (length, width, height) of the object are calculated.

[0062] S20: Predict the next position of the millimeter wave radar trajectory.

[0063] The present invention uses a Kalman filter to correct for sensor noise and provide predictive guidance in situations where the tracked object cannot be detected due to occlusion or temporary loss of the sensing area.

[0064] For each trajectory, a state consisting of the position and velocity along the x and y axes is maintained. For each trajectory, the initial state consists of the first detected position and velocity.

[0065] S30: Determine the numbers of the detected targets associated with the existing tracks, the targets for which new tracks need to be created, and the unassigned tracks.

[0066] For each object detection, a new track is generated and cannot be associated with an existing track; in each frame, the Hungarian algorithm is used to associate the detected objects with the corresponding track.

[0067] The Hungarian algorithm is a combinatorial optimization algorithm for solving task allocation problems in polynomial time for tracking multiple objects in autonomous driving, video processing, and other fields.

[0068] The purpose of this application is to establish the correlation between each detected target and the maintained tracked target, and minimize the combined distance loss. Among them, since the number of existing trajectories K1 and the number of targets detected in the current frame K2 may be different, the cost matrix may not be square, so the cost matrix is ​​enhanced with stacking items to construct a K×K matrix M.

[0069] Where, M=[M ij ] K×K ,K=K1+K2

[0070] If a detection is mapped to an augmented dimension, the application ignores these mappings and creates a new track for this detection. Similarly, if a track object is mapped to an augmented dimension, the application treats the track object as undetected. This approach enables the application to successfully keep track of detections.

[0071] S40: Correcting the trajectory, allocating the newly detected trajectory and deleting the undetected trajectory.

[0072] Trajectory corrections are performed on the targets of the assigned trajectory, which updates the predicted values ​​of the detected targets and their uncertainties through the Kalman filter.

[0073] At each successive time step, the Kalman filter updates the current state variables using the transition matrix and the corresponding uncertainties.

[0074] Based on the current position and velocity, it estimates a new position / velocity and a new covariance.

[0075] Deleting tracks: If a track object is not detected in D consecutive frames, it is marked as inactive and excluded from the consecutive associations and deleted.

[0076] In addition, a recommended method for stable multi-target detection and positioning and tracking based on millimeter radar is also provided.

[0077] Figure 2 A flowchart is shown for deleting outliers from millimeter radar point cloud data and obtaining the number of targets, center points and geometric parameters according to an embodiment of the present disclosure.

[0078] like Figure 2 As shown, step S10 is implemented using the Dijkstra algorithm, including the following steps:

[0079] A: Initialize to process radar point cloud data.

[0080] The present invention uses the SKY32B750_EVB millimeter wave radar module to obtain point cloud data of the target, which provides the number of point cloud data, distance information of the X, Y, and Z axes, and speed information in the radial direction.

[0081] Display N radar point cloud data as follows:

[0082] TGT={tgt i} i＝0,…,N-1

[0083]

[0084] Among them, tgt i : The i-th point cloud data.

[0085] : The distance value of the i-th point cloud data in the X-axis direction.

[0086] : The distance value of the ith point cloud data in the Y-axis direction.

[0087] : The distance value of the i-th point cloud data in the Z-axis direction.

[0088] : The velocity value in the radial direction of the i-th point cloud data.

[0089] Determine the distance threshold d lim , which is a threshold value. If the distance between points in the point cloud data is less than d lim , it can be considered as the same target point. The present invention determines d lim =0.4.

[0090] Initialize the array variable Ids to store the target IDs obtained from the radar point cloud obj .

[0091] Ids obj ={-1} 1×N

[0092] Set the target ID c Initialized to 0.

[0093] B: Select the i-th radar point cloud (i=0,…,N-1).

[0094] If Ids obsIf [i]=-1, run C, otherwise add 1 to i and run B.

[0095] C:Ids obs [i]=Id c ,

[0096] Ids new =[],

[0097] Ids new =[Ids new ;i],

[0098] Among them, Ids new It is a buffer used to manage the numbers of radar point clouds that can be regarded as the same target.

[0099] D: Calculate the IDs new Each point cloud data of the point set and tgt i The distance between them is used to determine whether they are points of the same target.

[0100] id=Ids new [j], j=0,…,M-1.

[0101] Where M is Ids new Length.

[0102] The idth point cloud data is

[0103] The kth point cloud data is

[0104] Where k = 0,…,N-1,Ids obj [k] = -1.

[0105] The distance between x1 and x2 is calculated as:

[0106]

[0107] If d is less than d lim , then x1 and x2 are considered as points within a target, in the buffer Ids next Add k and update Ids obj [k].

[0108] Ids obj [k]=Id c

[0109] Buffer Ids next Initialized before k loops, k=0,…,N-1 loops and j=0,…,M-1 loops, after which the data is handed over to Ids new Idsnew =Ids next .

[0110] If Ids new If the number of data is zero, Ids new The sum i is increased one by one, and it starts again from B, and runs E when i=N.

[0111] Ids obj At the end of the update, Id c Indicates the number of targets, Id c -1 indicates the number of the last target.

[0112] E: Get the center point and geometric parameters of each object.

[0113] For the i-th target, in Ids obj Find the dataset with the same point cloud number as i. That is, IDX = {idx l |Ids obj [idx l ]=i}, where l=0,…,L, L is the number of point cloud data in the i-th target.

[0114] The calculation formula for the center point of the i-th target is:

[0115]

[0116] in, is the X coordinate value of the center point of the i-th target, is the Y coordinate value of the center point of the i-th target. For idx l The distance value of the point cloud data in the X-axis direction, For idx l The distance value of the point cloud data in the Y-axis direction.

[0117] The calculation formula of the geometric parameters of the i-th target is:

[0118]

[0119] Among them, a i is the length of the i-th target,

[0120] b i is the width of the i-th target,

[0121] c i is the height of the i-th target,

[0122] is the maximum distance value of the i-th target in the X-axis direction,

[0123] is the maximum distance value of the Y axis direction of the i-th target,

[0124] is the maximum distance value of the i-th target in the Z-axis direction,

[0125] is the minimum distance value of the i-th target in the X-axis direction,

[0126] is the minimum distance value of the i-th target in the Y-axis direction,

[0127] The minimum distance value of the i-th target in the Z-axis direction.

[0128] Put the obtained center point and geometric parameters into the buffer centroids and bboxes, and repeat E while increasing i until i = Id c until.

[0129] Figure 3 FIG. 4 shows a flow chart of S30 according to an embodiment of the present disclosure. Figure 3 As shown, step S30 includes the following steps S301-S303:

[0130] In step S301, the cost matrix is ​​calculated using the trajectory state and the distance between the detection target center point.

[0131] The existing tracks are marked as

[0132] Among them, t i is the ith trajectory, is the distance value of the i-th trajectory in the X-axis direction, is the distance value of the ith track in the Y-axis direction, and K1 is the number of existing tracks.

[0133] The target detected in the current frame is marked as

[0134] Among them, p j is the jth target, is the X coordinate value of the jth target center point, is the Y coordinate value of the center point of the jth target, and K2 is the number of detected targets. That is:

[0135]

[0136] K2=Id c

[0137] Then, calculate the cost matrix, which is calculated as follows:

[0138]

[0139] Where C is the K1×K2 cost matrix, c i,j is the distance between the i-th trajectory and the j-th target.

[0140] In step S302, the cost matrix is ​​enhanced with stacked terms.

[0141] Typically, since the number of existing trajectories K1 and the number of targets detected in the current frame K2 may be different, the cost matrix may not be square, so the cost matrix is ​​enhanced with stacking terms to construct a K×K matrix M.

[0142] Cost matrix enhancement involves first constructing a new K×K matrix M and giving each element an initial value of a large number (1.79e+308).

[0143]

[0144] Where K = K1 + K2, m i,j =BN, BN=1.79e+308.

[0145] The matrix M is notated as:

[0146]

[0147] Among them, M1 is a K1×K2 matrix, M2 is a K1×K1 matrix, M3 is a K2×K2 matrix, and M4 is a K2×K1 matrix. Next, perform the following operations:

[0148] M1=C,

[0149] diag(M2)=cost lim ,

[0150] diag(M3)=cost lim ,

[0151]

[0152] Among them, cost lim is the distance threshold of the correlation between the trajectory and the detected target, is a K2×K1 zero matrix.

[0153] Then, M can be expressed as follows:

[0154]

[0155] In step S303, the Hungarian algorithm is used to obtain the allocation result between the detection target and the existing track. The Hungarian algorithm is applied to the cost matrix M to obtain the track number and target number with the lowest distance loss.

[0156] At this time, the obtained trajectory number and target number are marked as Id tra ,Id obj .

[0157]

[0158] Determine the unassigned track numbers as follows:

[0159]

[0160] in, is the unassigned trajectory number, and N1 is the number of unassigned trajectories.

[0161] Determine the number of the target for which a new track needs to be created, specifically:

[0162]

[0163] in, is the number of the newly detected target, and N2 is the number of newly detected targets.

[0164] Determine the number of existing tracks and associated targets, specifically:

[0165]

[0166] Among them, A is the allocation matrix, the first column of matrix A is the number of the existing trajectory, and the second column of matrix A is the number of the related target. N3 is the number of existing trajectories.

[0167] For example, the first row of matrix A is It means the first Targets occupy the of existing trajectories.

[0168] Figure 4 A flowchart is shown of predicting and correcting the trajectory, allocating a newly detected trajectory, updating an existing trajectory, and deleting an undetected trajectory according to an embodiment of the present disclosure.

[0169] like Figure 4 As shown, steps S20 and S40 include the following steps S401-S403:

[0170] In step S401, the existing trajectory is predicted and corrected.

[0171] The prediction and correction of the existing trajectory is performed through the Kalman filter to predict the existing trajectory, specifically:

[0172] First, consider the state at the next moment.

[0173]

[0174] in, is the estimate of the system state vector at time n,

[0175] is the prediction of the system state at time n+1 at time n,

[0176] F is the state transfer matrix:

[0177]

[0178] in, They are the estimated target center distance and speed values ​​of the X-axis and Y-axis at time n, They are respectively the prediction of the target center distance value and speed value of the X-axis and Y-axis at time n for time n+1.

[0179] At the same time, the uncertainty of this prediction is determined, specifically:

[0180] P n+1,n =FP n,n F T +Q

[0181] Among them, P n,n is the square of the uncertainty of the current state estimate (covariance matrix),

[0182] P n+1,n is the square of the uncertainty of the next state prediction (covariance matrix),

[0183] Q is the process noise matrix, which corrects the existing trajectory, specifically:

[0184] First, calculate the Kalman gain, which is calculated as follows:

[0185] K n =P n,n-1 H T (HP n,n-1 H T +R n ) -1

[0186] Among them, K n is the Kalman gain,

[0187] H is the observation matrix,

[0188] R n is the measurement covariance.

[0189] The estimated value is then updated with the measured value, which is calculated as:

[0190]

[0191] Among them, z n is the measurement of the current state, is the predicted value of the current state, is an estimate of the current state.

[0192] and

[0193] in, It is Then, the prediction uncertainty is updated, which is calculated as:

[0194]

[0195] In step S402, a trajectory is assigned to the newly detected target and the existing trajectory is updated.

[0196] The ID of the target for which a new track needs to be created ntr , create N2 trajectories, the specific process is:

[0197] Each trajectory is given a trajectory number by adding one to the last number of the existing trajectory.

[0198] The position of the trajectory, the velocity of the trajectory target and the geometric parameters are given, and the position of the trajectory gives the center point of the trajectory target.

[0199] Initialize the Kalman filter for each trajectory.

[0200] Here the Kalman filter model and the initialization of the input and state uncertainties are performed.

[0201] Secondly, the trajectory age and the number of target appearances are initialized to the number of targets that appear discontinuously.

[0202] The age of the track is the total number of frames output from the millimeter-wave radar, and the number of target appearances is a value that increases one by one when the target detected in each frame appears on the corresponding track.

[0203] The number of targets that do not appear consecutively is a value that increases one by one each frame when targets do not appear consecutively.

[0204] In the update of an existing track, the age of the track is increased each frame, if the target appears on the track, the number of target appearances is increased, and the number of target appearances that are not consecutive is initialized to zero.

[0205] If no target appears in the trajectory, the number of discontinuous targets is increased by one.

[0206] In step S403, the undetected tracks are deleted.

[0207] To delete a track, do the following:

[0208] If a target does not appear D times consecutively on a certain trajectory, the trajectory is excluded and deleted from the trajectory set.

[0209] Secondly, if the ratio of the number of target occurrences to the trajectory age exceeds a certain threshold, the trajectory is excluded from the trajectory set and deleted.

[0210] A recommended method comprises the following steps:

[0211] Step 1: Remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points, and geometric parameters. The specific process is as follows:

[0212] Firstly, the Dijkstra algorithm is used to eliminate outliers in millimeter-wave radar point cloud data and determine the number of detected targets.

[0213] Secondly, obtain the center point and geometric parameters of the detected target.

[0214] Step 2: Predict the next position of the mmWave radar track.

[0215] Trajectory prediction is performed on existing trajectories, predicting the location of the existing trajectory and its state uncertainty.

[0216] Step 3: Determine the numbers of detected targets associated with existing tracks, targets for which new tracks need to be created, and unassigned tracks. The specific process is as follows:

[0217] The cost matrix is ​​calculated using the distance between the trajectory state and the detected target center point.

[0218] Augment the cost matrix with stacked terms.

[0219] The Hungarian algorithm is used to obtain the assignment results between the detected targets and the existing trajectories.

[0220] Step 4: Correct the trajectories, assign newly detected trajectories and delete undetected trajectories.

[0221] Make predictions and corrections to existing trajectories.

[0222] Assign tracks to newly detected targets and update existing ones.

[0223] Delete the undetected tracks.

[0224] In one embodiment of the present application, a stable multi-target detection and positioning tracking system based on millimeter radar is also provided, and the system includes:

[0225] The acquisition module is used to remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters;

[0226] A prediction module, used to predict the next position of the millimeter-wave radar trajectory;

[0227] a determination module, used to determine the number of the detected targets associated with the existing tracks, the targets for which new tracks need to be created, and the unassigned tracks;

[0228] Tracking module, which is used to correct tracks, assign newly detected tracks and delete undetected tracks.

[0229] For the specific limitations of the millimeter radar-based stable multi-target detection and positioning tracking system, please refer to the limitations of the millimeter radar-based stable multi-target detection and positioning tracking method above, which will not be repeated here. Each module in the above-mentioned millimeter radar-based stable multi-target detection and positioning tracking system can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0230] In one embodiment, an electronic device is provided, which may be a computer. The electronic device includes a processor, a memory and a network interface connected via a system bus. The processor of the device is used to provide computing and control capabilities. The memory of the device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for stable multi-target detection and positioning and tracking data based on millimeter radar. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a stable multi-target detection and positioning and tracking method based on millimeter radar is implemented.

[0231] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned stable multi-target detection and positioning and tracking method based on millimeter radar are implemented.

[0232] In one embodiment of the present application, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned millimeter radar-based stable multi-target detection and positioning and tracking method.

[0233] The computer-readable storage medium and computer program product provided in this embodiment have similar implementation principles and technical effects to those of the above method embodiments, and are not described in detail here.

[0234] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SyMchliMk) DRAM (SLDRAM), memory bus (RaMbus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0235] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0236] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A stable multi-target detection and positioning tracking method based on millimeter radar, characterized in that: The method comprises: S10: Remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters; S20: predict the next position of the millimeter wave radar trajectory; S30: Determine the numbers of the detected targets associated with the existing tracks, the targets for which new tracks need to be created, and the unassigned tracks; S40: Correcting the trajectory, allocating the newly detected trajectory and deleting the undetected trajectory.

2. The stable multi-target detection and positioning tracking method according to claim 1, characterized in that: S10 specifically includes: Use SKY32B750_EVB millimeter wave radar to obtain the point cloud data of the target, including the number of point cloud data, distance information of X, Y, and Z axes, and speed information in the radial direction; Initialize to process each radar point cloud data; For each point cloud data, calculate the distance between it and other point cloud data. If the distance is less than the set threshold, they are considered to belong to the same target point. Otherwise, they are considered as outliers and removed. For the point cloud data determined as the target, its center point and geometric parameters are calculated; wherein the geometric parameters include length, width, and height.

3. The stable multi-target detection and positioning tracking method according to claim 1, characterized in that: S20 specifically includes: Use Kalman filters to correct for sensor noise and predict the next position of tracked objects that cannot be detected due to occlusion or temporary loss of the sensing area; For each trajectory, a state vector is maintained, including the position and velocity along the x and y axes, and the next state is predicted based on the current state.

4. The stable multi-target detection and positioning tracking method according to claim 1, characterized in that: S30 specifically includes: S301: Calculate the cost matrix using the trajectory state and the distance between the detection target center point; S302: enhancing the cost matrix using stacking items; S303: Apply the Hungarian algorithm to the cost matrix to obtain the track number and target number with the lowest distance loss, thereby determining the association between the detected target and the existing track, and obtaining the allocation result between the detected target and the existing track.

5. The stable multi-target detection and positioning tracking method according to claim 1, characterized in that: S40 specifically includes: S401: predict and correct the existing trajectory, and use the Kalman filter to update the predicted value and uncertainty of the detected target; S402: For a newly detected target, a new track is created and its position, velocity and geometric parameters are initialized. For an existing track, its status is updated, including track age and target appearance count; S403: If the track object is not detected in a plurality of consecutive frames, it is marked as inactive and deleted from the track set.

6. The stable multi-target detection and positioning tracking method according to claim 5, characterized in that: S402 specifically includes: Give each trajectory a trajectory number, and give the trajectory position and the velocity and geometric parameters of the trajectory target, and the trajectory position gives the center point of the trajectory target; Initialize the Kalman filter of each track, initialize the track age and the number of target appearances to the number of discontinuous targets; the track age is the total number of frames output from the millimeter wave radar, and the number of target appearances is the value that increases one by one when the target detected in each frame appears on the corresponding track; In the update of an existing track, the age of the track is increased each frame, if the target appears on the track, the number of target appearances is increased, and the number of discontinuous target appearances is initialized to zero, and if the target does not appear in the track, the number of discontinuous target appearances is increased by one.

7. A stable multi-target detection and positioning tracking system based on millimeter radar, characterized in that: The system comprises: The acquisition module is used to remove outliers from the millimeter radar point cloud data and obtain the number of targets, center points and geometric parameters; A prediction module, used to predict the next position of the millimeter-wave radar trajectory; a determination module, for determining, among the detected targets, targets associated with existing tracks, targets for which new tracks need to be created, and numbers of unassigned tracks; Tracking module, which is used to correct tracks, assign newly detected tracks and delete undetected tracks.

8. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, it implements the stable multi-target detection and positioning tracking method based on millimeter radar as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the stable multi-target detection and positioning tracking method as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the stable multi-target detection and positioning tracking method as described in any one of claims 1 to 6 is implemented.

Images