Target tracking method and device based on passive speed estimation of target detection frame

Abstract

The invention discloses a passive speed estimation target tracking method and device based on a target detection frame. The method comprises the following steps: acquiring a matching relationship between a target and the detection frame; detecting the stability of the detection frame and preliminarily judging the dynamic and static states of the target according to the obtained matching relationship; performing comprehensive judgment based on a detection result of the stability of the detection frame and a preliminary judgment result of a target dynamic and static state, and judging whether the target is in a static state or a moving state; performing speed selection according to the final judgment result of the dynamic and static states of the target, performing updating through a filter, and outputting the filtered speed; and realizing target tracking based on the filtered speed. According to the invention, the accuracy and smoothness of speed estimation can be improved, and the accuracy of target tracking is further improved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of autonomous driving, intelligent transportation, etc., and relates to a target tracking method, and specifically to a target tracking method and device based on passive speed estimation of a target detection frame. Background Art

[0002] With the development of autonomous driving technology, driving safety is a key link. In order to perceive the driving environment, autonomous driving vehicles are usually equipped with multiple sensors (cameras, lidar, millimeter-wave radar, etc.). By processing and analyzing sensor data, they can obtain information such as the surrounding environment, targets, obstacles, etc., to provide support for autonomous driving planning and control.

[0003] Autonomous driving perception technology requires tracking and trajectory prediction of detected targets. Therefore, multi-target tracking technology is also one of the hot topics in current autonomous driving perception technology research. One of the core aspects of target tracking is to stably track the static and dynamic state of the target and the speed of movement. The static and dynamic state of the target involves the need to accurately judge the static or dynamic state, and provide stable and reliable information assistance for downstream planning and obstacle avoidance. Furthermore, for targets in motion, it is necessary to accurately and steadily estimate the target speed, improve the accuracy of target position derivation and target association, and thus improve the stability of target tracking.

[0004] The basic principle of current speed measurement methods based on LiDAR is passive speed measurement, that is, the speed is obtained by differentially measuring the change in the position of the same target in the previous and next frames. The speed obtained in this way is limited by the stability of the detection frame or the stability of the target cluster point cloud set, resulting in a large range of speed fluctuations. Although the speed fluctuation can be suppressed after filtering, the instability of the filter input data will affect the convergence of the filter, resulting in deviations when the target predicted position is deduced in the next frame, which may cause target association failure and affect the stability of target tracking.

[0005] At present, there are two methods for target detection by LiDAR: deep learning model and traditional clustering algorithm. For the above two target detection algorithms, the principle of LiDAR speed measurement is the same, which is to obtain the speed by differential position of target detection between frames. The slight difference is that the LiDAR speed measurement method based on deep learning model is obtained through model training, and the speed of the target can be given at the same time as the target detection frame; while the traditional algorithm first divides and detects the target through algorithms such as clustering to obtain a point cloud set of the target. At this time, there is only a point cloud set with the target, which does not carry any speed information until the next frame clusters the point cloud data. After obtaining the matching information of the target through the target association algorithm, the position of the point cloud set is differentiated to obtain the target detection speed.

[0006] The following will further explain the technical details of the two lidar speed measurement methods:

[0007] First, the laser radar speed measurement method based on the deep learning model of point cloud overlay frames. The laser radar speed measurement method based on the deep learning model is divided into two stages: training and reasoning. In the training stage, the collected laser radar data needs to be labeled, the target ID is labeled, and the labeled target trajectory is smoothed and post-processed to obtain the true value of the target speed for training the model. In addition, the laser radar deep learning model speed measurement method is also a passive speed measurement between frames. It needs to accumulate the point clouds of the previous and next frames for registration and then input into the model for training.

[0008] After the training is completed, the model inference stage begins. First, two adjacent frames of point cloud images need to be extracted from the acquired point cloud data for registration to obtain the spliced ​​point cloud data; secondly, the point cloud data is input into the target recognition model to obtain the target detection results and speed output by the target recognition model.

[0009] Second, the laser radar speed measurement method based on the point cloud algorithm. The laser radar speed measurement method based on the traditional point cloud algorithm only needs to use a single frame of point cloud data for point cloud clustering or input it into the deep learning target detection model to obtain the single frame target detection result and the point cloud set of the detected target; then, the target is associated and matched with the detection result of the previous frame to obtain the association relationship; then, the speed of the target is obtained by calculating the centroid offset distance of the point cloud set or the offset distance of the position center point of the detection frame. In summary, the laser radar speed measurement method based on the point cloud algorithm is also a passive speed measurement method. Compared with the stacked frame point cloud target detection algorithm of the deep learning model, only a single frame is needed to complete target detection and clustering of the target point cloud set.

[0010] The current lidar speed measurement method is based on frame superposition or frame association, and the speed is obtained through model reasoning or by calculating the position offset between frames. However, this method is not sensitive to target speed resolution in low-speed displacement scenarios, and moving targets often give zero speed or stationary targets give speed. In addition, due to the limitation of computing resources, there are often cases of frame loss in detection, and the detection speed in this case is unreliable. In addition, it is also limited by the stability of the detection frame, and the speed given by the model or the speed obtained by differential calculation is not smooth and stable.

[0011] The existing patent CN 114120255 A discloses a target recognition method and device based on laser radar speed measurement. The patent extracts two adjacent frames of point cloud images from the acquired laser radar point cloud data and splices them to obtain spliced ​​point cloud data; the spliced ​​point cloud data is input into the target recognition model to obtain the target recognition result output by the target recognition model; wherein the target recognition model is obtained by training based on the point cloud training data and its corresponding target recognition true value.

[0012] This patent is based on the detection speed obtained by training the model after inter-frame point cloud image registration. The target recognition model is used to perform target recognition on the spliced ​​point cloud data to improve the target recognition accuracy, thereby avoiding misdetection or misassociation of static targets across the road, further ensuring the safety of vehicle autonomous driving and reducing the probability of accidents.

[0013] However, the target speed recognition accuracy in low-speed scenarios and the effect after detection frame loss due to other factors such as computing resources cannot be guaranteed.

[0014] The existing patent CN 107705563 B discloses a continuous vehicle speed detection method based on laser radar. The patent is based on the method of associating vehicles in multi-frame data to achieve vehicle association, and then to detect vehicle speed, which completely avoids the problem of laser radar frame loss affecting measurement, and also avoids the problem of vehicles blocking each other affecting measurement.

[0015] Although the method of this patent avoids the speed impact caused by frame loss, this patent is based on the speed obtained by position difference. It is limited by the stability of the detection frame or the influence of the point cloud set. The speed fluctuation obtained by the difference is large, and the speed is not smooth and stable, which affects the position derivation of the next frame and the stability of target tracking. Summary of the invention

[0016] In order to solve the above technical problems, the present invention proposes a target tracking method and device based on passive velocity estimation of a target detection frame.

[0017] In order to achieve the above object, the technical solution of the present invention is as follows:

[0018] In a first aspect, the present invention discloses a target tracking method based on passive velocity estimation of a target detection frame, comprising:

[0019] Step S1: Obtain the matching relationship between the target and the detection frame;

[0020] Step S2: According to the obtained matching relationship, the stability of the detection frame and the preliminary judgment of the dynamic and static state of the target are respectively performed;

[0021] Step S3: making a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target;

[0022] If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state;

[0023] If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion;

[0024] If the initial judgment result of the target's static or dynamic state is motion, and the detection result of the detection frame stability is unstable, the point cloud registration algorithm is used to perform a secondary analysis of the target's static or dynamic state to comprehensively judge whether the target is static or moving.

[0025] Step S4: Select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed;

[0026] Step S5: Target tracking is achieved based on the filtered velocity.

[0027] Based on the above technical solution, the following improvements can be made:

[0028] As a preferred solution, in step S2, the detection of the stability of the detection frame is implemented by the following method, including:

[0029] Step A1: Obtain a set of center points of the detection frame position within a specified time period;

[0030] Step A2: performing pairwise difference calculations on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtaining the target position orientation angle set based on the results;

[0031] Step A3: Calculate the variance of the target position orientation angle set to obtain the orientation angle variance;

[0032] Step A4: judging the orientation angle variance and the detection frame stability threshold;

[0033] If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state;

[0034] Otherwise, the detection box is in a stable state.

[0035] As a preferred solution, in step S2, the preliminary determination of the dynamic and static state of the target is achieved by the following method, including:

[0036] Step B1: Obtain the target trajectory within a specified time period;

[0037] Step B2: Calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using the principal component analysis method;

[0038] Step B3: Calculate the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector;

[0039] Step B4: judging the principal component change equivalent distance and the first target motion threshold;

[0040] If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion;

[0041] Otherwise, the target is at rest.

[0042] As a preferred solution, in step S3, when the preliminary determination result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is instability, the following method is used to perform a secondary analysis of the target's dynamic and static state, including:

[0043] Step C1: Obtain the point cloud set of the current frame and the point cloud set of the previous frame of the target;

[0044] Step C2: Use a registration algorithm to register the two point cloud sets and obtain the registered rotation matrix and translation matrix;

[0045] Step C3: Solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance;

[0046] Step C4: judging the equivalent transformation distance and the second target motion threshold;

[0047] If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion;

[0048] Otherwise, the target is at rest.

[0049] As a preferred solution, step S4 includes:

[0050] Step S4.1: Calculate multiple speeds and construct a speed candidate set based on the final judgment result of the target's dynamic and static state;

[0051] Step S4.2: using the acceleration state information of the filter to obtain the reference selection speed of the current frame;

[0052] Step S4.3: Calculate the difference between the speed candidate set and the reference selected speed to obtain a speed gain set;

[0053] Step S4.4: sort the speed gain set from small to large to obtain the speed index with the minimum gain;

[0054] Step S4.5: according to the speed index, taking the speed corresponding to the index from the speed candidate set;

[0055] Step S4.6: Input the retrieved speed into the filter, update it, and obtain the filtered speed.

[0056] In a second aspect, the present invention further discloses a target tracking device based on passive velocity estimation of a target detection frame, comprising:

[0057] The acquisition module is used to obtain the matching relationship between the target and the detection frame;

[0058] A detection frame stability detection module is used to perform detection frame stability detection based on the obtained matching relationship;

[0059] A module for initially judging the dynamic and static state of a target, for initially judging the dynamic and static state of a target based on the obtained matching relationship;

[0060] A comprehensive judgment module is used to make a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target;

[0061] If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state;

[0062] If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion;

[0063] If the preliminary determination result of the target's motion state is motion, and the detection result of the detection frame's stability is instability, then continue to execute the method in the target's motion state secondary determination module;

[0064] The target static and dynamic state secondary judgment module is used to perform secondary analysis on the target static and dynamic state by using the point cloud registration algorithm to determine whether the target is in a static state or a moving state;

[0065] The speed update module is used to select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed;

[0066] The target tracking module is used to realize target tracking based on the filtered velocity.

[0067] As a preferred solution, the detection frame stability detection module includes:

[0068] A detection frame position center point obtaining unit, used to obtain a set of detection frame position center points within a specified time period;

[0069] A position orientation angle calculation unit is used to perform pairwise difference calculation on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtain the target position orientation angle set based on the result;

[0070] A heading angle variance calculation unit is used to calculate the variance of the target position heading angle set to obtain the heading angle variance;

[0071] A stability judgment unit, used to judge the orientation angle variance and the detection frame stability threshold;

[0072] If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state;

[0073] Otherwise, the detection box is in a stable state.

[0074] As a preferred solution, the target dynamic and static state preliminary judgment module includes:

[0075] A target trajectory obtaining unit, used to obtain the target trajectory within a specified time period;

[0076] A feature calculation unit, used to calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using a principal component analysis method;

[0077] An equivalent distance calculation unit, used for calculating the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector;

[0078] A preliminary judgment unit, used for judging the principal component change equivalent distance and the first target motion threshold;

[0079] If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion;

[0080] Otherwise, the target is at rest.

[0081] As a preferred solution, the target dynamic and static state secondary judgment module includes:

[0082] A point cloud acquisition unit, used to obtain a point cloud set of a current frame and a point cloud set of a previous frame of a target;

[0083] A registration unit, used to register two point cloud sets using a registration algorithm to obtain a registered rotation matrix and translation matrix;

[0084] The equivalent transformation distance calculation unit is used to solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance;

[0085] A secondary judgment unit, used for judging the equivalent transformation distance and the second target motion threshold;

[0086] If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion;

[0087] Otherwise, the target is at rest.

[0088] As a preferred solution, the speed update module includes:

[0089] A speed calculation unit, used to calculate multiple speeds and construct a speed candidate set according to the final judgment result of the target dynamic and static state;

[0090] A reference selection speed calculation unit, used to obtain a reference selection speed of a current frame using acceleration state information of the filter;

[0091] A speed gain calculation unit, used for performing difference calculation between the speed candidate set and the reference selected speed to obtain a speed gain set;

[0092] A speed index obtaining unit, used for sorting the speed gain set from small to large to obtain the speed index with the minimum gain;

[0093] A speed selection unit, used for taking out a speed corresponding to an index from a speed candidate set according to a speed index;

[0094] The filtering unit is used to input the extracted speed into the filter, update it, and obtain the filtered speed.

[0095] In a third aspect, the present invention further discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing any of the above-mentioned target tracking methods based on passive velocity estimation of a target detection frame.

[0096] The present invention discloses a target tracking method and device based on passive velocity estimation of a target detection frame, which has the following beneficial effects:

[0097] First, the present invention uses a combination of methods such as motion orientation consistency, principal component analysis (PCA), and ICP analysis to accurately judge the moving or stationary state of the target, thereby improving the speed response. It improves the speed response problem in scenarios such as intersection merging, reversing, and low-speed turning, and gives the speed in a timely manner, so that the downstream can make preventive avoidance and detour strategies earlier, improves the accuracy of target state tracking, and can provide a more accurate and reliable state for the downstream.

[0098] Second, the present invention discloses a speed gain selection method using the target acceleration and deceleration situation. On the one hand, it avoids the influence of fluctuation error caused by the detection speed, and the speed selection is smoother, which can predict a more accurate position; on the other hand, the target speed selection method using the acceleration and deceleration situation improves the response of the target speed, accelerates the speed convergence process, and improves the accuracy of the target tracking state prediction, thereby improving the accuracy and stability of target tracking.

[0099] Third, the present invention has a wide range of applications and can be used in scenarios such as park security, intelligent transportation, and automotive fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0101] Figure 1 A flow chart of a target tracking method provided by an embodiment of the present invention.

[0102] Figure 2 A flow chart of detection frame stability detection provided by an embodiment of the present invention.

[0103] Figure 3 A flow chart for preliminary determination of the dynamic and static state of a target provided by an embodiment of the present invention.

[0104] Figure 4 A flow chart of secondary analysis of target dynamic and static states provided by an embodiment of the present invention.

[0105] Figure 5 A flow chart of speed filtering provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0106] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0107] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0108] Using ordinal numbers “first,” “second,” “third,” etc. to describe common objects merely indicates that different instances of similar objects are involved and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0109] In addition, the expression of “comprising” an element is an “open” expression, which merely means that corresponding components or steps exist, and should not be interpreted as excluding additional components or steps.

[0110] In order to achieve the purpose of the present invention, a method and device for target tracking based on passive velocity estimation of a target detection frame are provided in some embodiments, such as Figure 1 As shown, the target tracking method includes:

[0111] Step S1: Obtain the matching relationship between the target and the detection frame;

[0112] Step S2: According to the obtained matching relationship, the stability of the detection frame and the preliminary judgment of the dynamic and static state of the target are respectively performed;

[0113] Step S3: making a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target;

[0114] If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state;

[0115] If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion;

[0116] If the initial judgment result of the target's static or dynamic state is motion, and the detection result of the detection frame stability is unstable, the point cloud registration algorithm is used to perform a secondary analysis of the target's static or dynamic state to comprehensively judge whether the target is static or moving.

[0117] Step S4: Select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed;

[0118] Step S5: Target tracking is achieved based on the filtered velocity.

[0119] At present, it is relatively easy to estimate the speed of targets moving at medium and high speeds because of the obvious position difference changes. However, for low-speed scenarios such as left and right turns, U-turns, and reversing, the speed is low and the displacement changes are relatively insignificant, which is a major difficulty in estimating the speed. Because when estimating such low-speed targets, if the speed is directly derived based on the position difference of the detection frame, there will be false speeds caused by stationary obstacles due to the instability of the detection frame, which will affect the trajectory planning of the vehicle in the subsequent tracking algorithm.

[0120] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining feature technologies are the same, except that, in step S2, the detection of the stability of the detection frame is implemented by the following method, including:

[0121] Step A1: Obtain a set of center points of the detection frame position within a specified time period;

[0122] Step A2: performing pairwise difference calculations on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtaining the target position orientation angle set based on the results;

[0123] Step A3: Calculate the variance of the target position orientation angle set to obtain the orientation angle variance;

[0124] Step A4: judging the orientation angle variance and the detection frame stability threshold;

[0125] If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state;

[0126] Otherwise, the detection box is in a stable state.

[0127] The following is a detailed description of the above methods:

[0128] like Figure 2 As shown, a target trajectory is defined as Traj = {p t ,p t-1 ,p t-2 ,...,p t-n}.

[0129] Where: p t ={px t ,py t ,pz t}.

[0130] Traj is the set of target locations from time tn to time t, p t It is the position coordinates in the X, Y, and Z directions at time t.

[0131] According to the target position trajectory, the position orientation is calculated and obtained:

[0132]

[0133] in, Use the detection box position center point {px at time t t ,py t} and the center point of the detection box at time t-1 {px t-1 ,py t-1}, and use the result to get the position and orientation angle of the target in the X and Y directions of the coordinate system.

[0134] Then, the variance of the position and orientation angle set is calculated to measure the stability of position changes. The orientation angle variance var is obtained and the threshold is judged:

[0135]

[0136] After the detection stability is obtained, the target motion trajectory will be analyzed in combination with the subsequent principal component analysis to determine whether the target's motion state is moving, still, or other (unknown) states (other states represent states between moving and still).

[0137] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining characteristic technologies are the same, except that, in step S2, the preliminary determination of the dynamic and static state of the target is achieved by the following method, including:

[0138] Step B1: Obtain the target trajectory within a specified time period;

[0139] Step B2: Calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using the principal component analysis method;

[0140] Step B3: Calculate the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector;

[0141] Step B4: judging the principal component change equivalent distance and the first target motion threshold;

[0142] If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion;

[0143] Otherwise, the target is at rest.

[0144] Principal Components Analysis (PCA) is an unsupervised multivariate statistical analysis method. Its main principle is to extract the main features of the data by projecting high-dimensional data into a low-dimensional data space. When analyzing the target trajectory, the positions in the X and Y directions are mainly analyzed. Through principal component analysis, redundant and repeated variables are deleted, new variables are established, and the new variables are independent of each other. Then, the largest principal component variable and secondary component variable are found, and the threshold is used to determine whether the target is in motion, stationary, or cannot be analyzed.

[0145] like Figure 3 As shown, the above method is described in detail below:

[0146] For a target trajectory Traj = {p t ,p t-1 ,p t-2 ,...,p t-n}.

[0147] Among them, p t ={px t ,py t ,pz t}, Traj is the set of target locations from time tn to time t, p t It is the position coordinates in the X, Y, and Z directions at time t.

[0148] In target tracking, we pay more attention to the longitudinal and lateral changes in the X and Y directions, but are not sensitive to the position changes in the Z axis. Therefore, we extract the position coordinates in the X and Y directions of the target trajectory to form a set M, which contains n+1 2D position data from time t to time tn.

[0149]

[0150] Rearrange the original data set by columns to generate a new data set X with 2 rows and n+1 columns;

[0151]

[0152] Among them, x1=|px t ,px t-1 ,...,px t-n | T , x2=|py t ,py t-1 ,...,py t-n | T ; Zero the mean of each row of X

[0153]

[0154] Find the covariance matrix

[0155]

[0156] The eigenvalues ​​of the covariance matrix and their corresponding eigenvectors are obtained (this solution process is prior art and will not be described in detail here).

[0157] Through the calculated eigenvalues ​​and eigenvectors, select the first eigenvalue E1, eigenvector E2, and eigenvector E3 from large to small. and the second eigenvalue E2, eigenvector The first principal component change distance ΔD1 and the second principal component change distance ΔD2 are calculated, where:

[0158]

[0159] After obtaining ΔD1 and ΔD2 by calculation, the comprehensive principal component change equivalent distance ΔD is obtained, ΔD=ΔD1+ΔD2.

[0160] The minimum weighted feature distance threshold M of the motion obtained by statistics thrd and the maximum weighted feature distance threshold S in the static state thrd , determine whether the target is in motion or stationary state, or in other (unknown) state.

[0161]

[0162] It should be noted that the error introduced by the jitter of the detection frame will be transmitted to the principal component analysis calculation, causing the originally stationary target to be mistakenly judged as a moving target. Therefore, it is necessary to use the result of the position and orientation consistency judgment to further judge the state of the target during movement.

[0163] When PCA judges that the target is in motion, and the consistency of the target position and orientation is judged to be stable, the target is considered to be in motion;

[0164] When PCA determines that the target is in motion and the target orientation consistency is judged to be unstable, it is considered that the target is in misjudged motion due to the jitter of the detection frame and is converted to other (unknown) states. The point cloud registration algorithm (such as ICP) is further used to analyze the motion state.

[0165] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining feature technologies are the same, except that, in step S3, when the preliminary judgment result of the target dynamic and static state is motion, and the detection result of the detection frame stability is unstable, the following method is used to perform a secondary analysis on the target dynamic and static state, including:

[0166] Step C1: Obtain the point cloud set of the current frame and the point cloud set of the previous frame of the target;

[0167] Step C2: Use the ICP algorithm (Iterative Closest Point) to align the two point cloud sets and obtain the rotation matrix and translation matrix of the alignment;

[0168] Step C3: Solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance;

[0169] Step C4: judging the equivalent transformation distance and the second target motion threshold;

[0170] If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion;

[0171] Otherwise, the target is at rest.

[0172] The ICP (Iterative Closest Point) algorithm is a commonly used point cloud registration algorithm, which is used to align the coordinate systems of two or more point clouds. Its principle is to continuously optimize the matching error between two point clouds through iteration until the error is minimized.

[0173] Define two point cloud sets P = {p1, p2, ..., p n}, Q={q1,q2,...,q n},

[0174] Among them, P is the reference point cloud set (the target point cloud set at time t-1), and Q is the data point cloud set (the target point cloud set at time t).

[0175] The point cloud registration algorithm needs to transform Q into the coordinate system of set P through a series of translations and rotations to make it as close to set P as possible. Because the two data will not completely overlap after rotation and translation transformation, the above problem can be described as solving the rotation matrix R and translation matrix T, so that:

[0176]

[0177] The ICP algorithm performs iterative calculation based on the least squares method to minimize the sum of squared errors:

[0178]

[0179] The derivation of the formula and the calculation process for solving the rotation matrix R and the translation matrix T will not be introduced in detail here.

[0180] After ICP reaches the set number of iterations, it determines whether the target point cloud set has undergone displacement changes by adjusting the changes in the rotation matrix R and the translation matrix T.

[0181] Rotation matrix R (rotation around the Z axis):

[0182]

[0183] Translation matrix T:

[0184] T=|T x T y T z

[0185] Calculate the weighted results of the displacement change and rotation change of the point cloud set:

[0186]

[0187] Where yaw = tan -1 (sin(yaw),cos(yaw)), we get the weighted equivalent transformation distance Δd, which is used to judge motion or stillness.

[0188]

[0189] If the equivalent transformation distance Δd is less than or equal to the set threshold d thrd , the target is judged to be in a stationary state; otherwise, it is judged to be in a moving state.

[0190] The above ICP algorithm may also be NDT or other registration algorithms.

[0191] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining characteristic technologies are the same, except that step S4 includes:

[0192] Step S4.1: Calculate multiple speeds and construct a speed candidate set based on the final judgment result of the target's dynamic and static state;

[0193] Step S4.2: using the acceleration state information of the filter to obtain the reference selection speed of the current frame;

[0194] Step S4.3: Calculate the difference between the speed candidate set and the reference selected speed to obtain a speed gain set;

[0195] Step S4.4: sort the speed gain set from small to large to obtain the speed index with the minimum gain;

[0196] Step S4.5: according to the speed index, taking the speed corresponding to the index from the speed candidate set;

[0197] Step S4.6: Input the retrieved speed into the filter, update it, and obtain the filtered speed.

[0198] Steps S1 to S3 make a final judgment on the target's static or dynamic state. If the target is static, the speed is directly set to 0; if it is in motion, a speed selection is performed. Here, the following speeds are defined to form a speed candidate set. Then, the speed gain is selected based on the acceleration and deceleration trend. The optimal speed is selected to ensure the smoothness and stability of the speed, improve the stability of the prediction frame, and thus improve the accuracy of the next frame target matching association and the stability of the target tracking.

[0199] The method in step S4 is described in detail below:

[0200] The speed set includes the following speeds:

[0201] Detect speed: The speed given by the detection end, denoted as {V Det_x ,V Det_y ,V Det_z};

[0202] Anchor velocity: The center of gravity of the point cloud set. The calculation formula is:

[0203]

[0204] Center speed: The center point of the detection frame. The calculation formula is:

[0205]

[0206] Period speed: The speed calculated from the center point position difference of the detection frame from time tn to time t. The calculation formula is:

[0207]

[0208] Corner speed: The speed of the four corner points of the detection box in the top-down plane. The order of the corner points is defined as clockwise, and the first point is the upper left vertex. The formula for calculating the corner speed is:

[0209]

[0210] Calculate the composite speed from all the speeds to get the speed candidate set V C :

[0211]

[0212] in, is the composite speed of the detection speed, and the others represent the composite speed calculated by the anchor point, the composite speed calculated by the center point, the composite speed calculated by the nearest Corner, the composite speed calculated by the period interval, and the composite speed calculated by the four Corner points, which together constitute the speed candidate list V C .

[0213] For velocity filtering, the Constant Acceleration (CA) Kalman filter model is selected.

[0214] The model is defined as follows:

[0215]

[0216] In the constant acceleration filter model, velocity is used as the observed quantity, while acceleration is used as the process state quantity. By analyzing the acceleration, it can be determined whether the target is in an accelerating or decelerating state.

[0217] Take the output speed of the previous frame as the initial reference speed of the current frame And extract the acceleration of the filter model Calculates the reference selection speed for the current frame.

[0218]

[0219] Calculate the composite speed at time t

[0220] Calculate the speed gain. The reference speed at time t is calculated For the speed candidate set V C The speed gain is calculated to obtain a speed gain set G and a subscript index set I.

[0221]

[0222] I={0,1,...,7

[0223] Sort the speed gain set G from small to large to get the sorted gain list G s and the subscript index set I s .

[0224] In order to ensure speed smoothness, the speed with the smallest gain is preferred, that is, the speed gain list G s The first speed gain, take out the subscript index represented by the gain Index = I s [0], and from the corresponding speed candidate set V C Extraction speed V = V C[Index], input into the filter model for update, and obtain the filtered velocity V at time t o .

[0225] The present invention calculates the speed gain through the acceleration and deceleration trend, and inputs the selected speed into the filter, thereby finally achieving the purpose of improving the speed accuracy and providing more reliable prior information for the predicted position of the next frame, thereby improving the target association accuracy and target tracking stability.

[0226] In summary, the present invention discloses a target tracking method based on passive velocity estimation of a target detection frame.

[0227] Firstly, the present invention makes a comprehensive judgment on the dynamic and static state of the target through position orientation consistency, principal component analysis, ICP algorithm, etc., to judge whether the target is in a static state or a moving state, thereby improving speed sensitivity.

[0228] It is worth noting that the present invention discloses a target state analysis method based on a combination of multiple algorithms, which is used to distinguish the dynamic and static state of the target. According to the position state information of the target and the point cloud information between frames, accurate dynamic and static information of the target is given for subsequent speed screening. The protection scope of the present invention is not limited to the orientation consistency analysis, principal component analysis, ICP algorithm, etc. disclosed in the above embodiments, but can also be speed orientation consistency analysis, target detection deep learning feature analysis, etc.

[0229] Secondly, the present invention uses a variety of speed combinations to generate a speed candidate set, and uses the motion trend of acceleration and deceleration to calculate the reference selection speed, select the speed that meets the minimum gain, and specifically select the speed that meets the motion trend, thereby improving the speed response, ensuring the smoothness of the speed curve, and being more conducive to the derivation of the target position and the stability of target tracking.

[0230] The present invention can provide more reliable prior information for predicting the position of the next frame, thereby improving the accuracy of target association and the stability of target tracking.

[0231] It is worth noting that the above-mentioned detection frame analysis can, but is not limited to, use other sensor detection results that can be analyzed by detection frames, such as camera image 3D detection technology, millimeter wave radar detection technology, etc.

[0232] In addition, an embodiment of the present invention further discloses a target tracking device based on passive velocity estimation of a target detection frame, comprising:

[0233] The acquisition module is used to obtain the matching relationship between the target and the detection frame;

[0234] A detection frame stability detection module is used to perform detection frame stability detection based on the obtained matching relationship;

[0235] A module for initially judging the dynamic and static state of a target, for initially judging the dynamic and static state of a target based on the obtained matching relationship;

[0236] A comprehensive judgment module is used to make a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target;

[0237] If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state;

[0238] If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion;

[0239] If the preliminary determination result of the target's motion state is motion, and the detection result of the detection frame's stability is instability, then continue to execute the method in the target's motion state secondary determination module;

[0240] The target static and dynamic state secondary judgment module is used to perform secondary analysis on the target static and dynamic state by using the point cloud registration algorithm to determine whether the target is in a static state or a moving state;

[0241] The speed update module is used to select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed;

[0242] The target tracking module is used to realize target tracking based on the filtered velocity.

[0243] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining feature technologies are the same, except that the detection frame stability detection module includes:

[0244] A detection frame position center point obtaining unit, used to obtain a set of detection frame position center points within a specified time period;

[0245] A position orientation angle calculation unit is used to perform pairwise difference calculation on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtain the target position orientation angle set based on the result;

[0246] A heading angle variance calculation unit is used to calculate the variance of the target position heading angle set to obtain the heading angle variance;

[0247] A stability judgment unit, used to judge the orientation angle variance and the detection frame stability threshold;

[0248] If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state;

[0249] Otherwise, the detection box is in a stable state.

[0250] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining feature technologies are the same, except that the target dynamic and static state preliminary judgment module includes:

[0251] A target trajectory obtaining unit, used to obtain the target trajectory within a specified time period;

[0252] A feature calculation unit, used to calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using a principal component analysis method;

[0253] An equivalent distance calculation unit, used for calculating the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector;

[0254] A preliminary judgment unit, used for judging the principal component change equivalent distance and the first target motion threshold;

[0255] If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion;

[0256] Otherwise, the target is at rest.

[0257] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining feature technologies are the same, except that the target dynamic and static state secondary judgment module includes:

[0258] A point cloud acquisition unit, used to obtain a point cloud set of a current frame and a point cloud set of a previous frame of a target;

[0259] A registration unit, used to register two point cloud sets using a registration algorithm to obtain a registered rotation matrix and translation matrix;

[0260] The equivalent transformation distance calculation unit is used to solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance;

[0261] A secondary judgment unit, used for judging the equivalent transformation distance and the second target motion threshold;

[0262] If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion;

[0263] Otherwise, the target is at rest.

[0264] In order to further optimize the implementation effect of the present invention, in some other implementations, the remaining characteristic technologies are the same, except that the speed update module includes:

[0265] A speed calculation unit, used to calculate multiple speeds and construct a speed candidate set according to the final judgment result of the target dynamic and static state;

[0266] A reference selection speed calculation unit, used to obtain a reference selection speed of a current frame using acceleration state information of the filter;

[0267] A speed gain calculation unit, used for performing difference calculation between the speed candidate set and the reference selected speed to obtain a speed gain set;

[0268] A speed index obtaining unit, used for sorting the speed gain set from small to large to obtain the speed index with the minimum gain;

[0269] A speed selection unit, used for taking out a speed corresponding to an index from a speed candidate set according to a speed index;

[0270] The filtering unit is used to input the extracted speed into the filter, update it, and obtain the filtered speed.

[0271] In this embodiment, the specific content of the target tracking device based on passive speed estimation of the target detection frame is similar to the content of the target tracking method based on passive speed estimation of the target detection frame disclosed in the above embodiment, and will not be repeated here.

[0272] An embodiment of the present invention further discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing the target tracking method based on passive velocity estimation of a target detection frame disclosed in any of the above embodiments.

[0273] The present invention discloses a target tracking method and device based on passive velocity estimation of a target detection frame, which has the following beneficial effects:

[0274] First, the present invention uses a combination of methods such as motion orientation consistency, principal component analysis (PCA), and ICP analysis to accurately judge the moving or stationary state of the target, thereby improving the speed response. It improves the speed response problem in scenarios such as intersection merging, reversing, and low-speed turning, and gives the speed in a timely manner, so that the downstream can make preventive avoidance and detour strategies earlier, improves the accuracy of target state tracking, and can provide a more accurate and reliable state for the downstream.

[0275] Second, the present invention discloses a speed gain selection method using the target acceleration and deceleration situation. On the one hand, it avoids the influence of fluctuation error caused by the detection speed, and the speed selection is smoother, which can predict a more accurate position; on the other hand, the target speed selection method using the acceleration and deceleration situation improves the response of the target speed, accelerates the speed convergence process, and improves the accuracy of the target tracking state prediction, thereby improving the accuracy and stability of target tracking.

[0276] Third, the present invention has a wide range of applications and can be used in scenarios such as park security, intelligent transportation, and automotive fields.

[0277] The present invention is a method for improving target speed estimation, which can improve the accuracy and smoothness of speed estimation and further improve the accuracy of target tracking.

[0278] It should be understood that the various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or portions of the method and apparatus of the present invention, can take the form of program codes (i.e., instructions) embedded in a tangible medium, such as a floppy disk, a CD-ROM, a hard disk drive, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0279] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A target tracking method based on passive velocity estimation of a target detection frame, characterized in that: include: Step S1: Obtain the matching relationship between the target and the detection frame; Step S2: According to the obtained matching relationship, the stability of the detection frame and the preliminary judgment of the dynamic and static state of the target are respectively performed; Step S3: making a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target; If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state; If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion; If the initial judgment result of the target's static or dynamic state is motion, and the detection result of the detection frame stability is unstable, the point cloud registration algorithm is used to perform a secondary analysis of the target's static or dynamic state to comprehensively judge whether the target is static or moving; Step S4: Select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed; Step S5: Target tracking is achieved based on the filtered velocity.

2. The target tracking method according to claim 1, characterized in that: In step S2, the detection of the stability of the detection frame is implemented by the following method, including: Step A1: Obtain a set of center points of the detection frame position within a specified time period; Step A2: performing pairwise difference calculations on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtaining the target position orientation angle set based on the results; Step A3: Calculate the variance of the target position orientation angle set to obtain the orientation angle variance; Step A4: judging the orientation angle variance and the detection frame stability threshold; If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state; Otherwise, the detection box is in a stable state.

3. The target tracking method according to claim 1, characterized in that: In step S2, the preliminary determination of the target's dynamic and static state is achieved by the following method, including: Step B1: Obtain the target trajectory within a specified time period; Step B2: Calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using the principal component analysis method; Step B3: Calculate the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector; Step B4: judging the principal component change equivalent distance and the first target motion threshold; If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion; Otherwise, the target is at rest.

4. The target tracking method according to claim 1, characterized in that: In step S3, when the preliminary determination result of the target's dynamic and static state is motion, and the detection result of the detection frame's stability is instability, the following method is used to perform a secondary analysis of the target's dynamic and static state, including: Step C1: Obtain the point cloud set of the current frame and the point cloud set of the previous frame of the target; Step C2: Use a registration algorithm to register the two point cloud sets and obtain the registered rotation matrix and translation matrix; Step C3: Solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance; Step C4: judging the equivalent transformation distance and the second target motion threshold; If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion; Otherwise, the target is at rest.

5. The target tracking method according to claim 1, characterized in that: The step S4 comprises: Step S4.1: Calculate multiple speeds and construct a speed candidate set based on the final judgment result of the target's dynamic and static state; Step S4.2: using the acceleration state information of the filter to obtain the reference selection speed of the current frame; Step S4.3: Calculate the difference between the speed candidate set and the reference selected speed to obtain a speed gain set; Step S4.4: sort the speed gain set from small to large to obtain the speed index with the minimum gain; Step S4.5: according to the speed index, taking the speed corresponding to the index from the speed candidate set; Step S4.6: Input the retrieved speed into the filter, update it, and obtain the filtered speed.

6. A target tracking device based on passive velocity estimation of a target detection frame, characterized in that: include: The acquisition module is used to obtain the matching relationship between the target and the detection frame; A detection frame stability detection module is used to perform detection frame stability detection based on the obtained matching relationship; A module for initially judging the dynamic and static state of a target, for initially judging the dynamic and static state of a target based on the obtained matching relationship; A comprehensive judgment module is used to make a comprehensive judgment based on the detection result of the stability of the detection frame and the preliminary judgment result of the dynamic and static state of the target; If the preliminary judgment result of the target's dynamic and static state is static, then the target is comprehensively judged to be in a static state; If the preliminary judgment result of the target's dynamic and static state is motion, and the detection result of the detection frame stability is stability, then it is comprehensively judged that the target is in motion; If the preliminary determination result of the target's motion state is motion, and the detection result of the detection frame's stability is instability, then continue to execute the method in the target's motion state secondary determination module; The target static and dynamic state secondary judgment module is used to perform secondary analysis on the target static and dynamic state by using the point cloud registration algorithm to determine whether the target is in a static state or a moving state; The speed update module is used to select the speed according to the final judgment result of the target's dynamic and static state, update it through the filter, and output the filtered speed; The target tracking module is used to realize target tracking based on the filtered velocity.

7. The target tracking device according to claim 6, characterized in that: The detection frame stability detection module includes: A detection frame position center point obtaining unit, used to obtain a set of detection frame position center points within a specified time period; A position orientation angle calculation unit is used to perform pairwise difference calculation on the detection frame position center points of the previous and next frames within the detection frame position center point set, and obtain the target position orientation angle set based on the result; A heading angle variance calculation unit is used to calculate the variance of the target position heading angle set to obtain the heading angle variance; A stability judgment unit, used to judge the orientation angle variance and the detection frame stability threshold; If the orientation angle variance is greater than the detection frame stability threshold, the detection frame is in an unstable state; Otherwise, the detection box is in a stable state.

8. The target tracking device according to claim 6, characterized in that: The target dynamic and static state preliminary judgment module includes: A target trajectory obtaining unit, used to obtain the target trajectory within a specified time period; A feature calculation unit, used to calculate the maximum eigenvalue of the target trajectory and its corresponding eigenvector using a principal component analysis method; An equivalent distance calculation unit, used for calculating the principal component change equivalent distance based on the maximum eigenvalue and its corresponding eigenvector; A preliminary judgment unit, used for judging the principal component change equivalent distance and the first target motion threshold; If the principal component change equivalent distance is greater than the first target motion threshold, the target is in motion; Otherwise, the target is at rest.

9. The target tracking device according to claim 6, characterized in that: The target dynamic and static state secondary judgment module includes: A point cloud acquisition unit, used to obtain a point cloud set of a current frame and a point cloud set of a previous frame of a target; A registration unit, used to register two point cloud sets using a registration algorithm to obtain a registered rotation matrix and translation matrix; The equivalent transformation distance calculation unit is used to solve the rotation angle of the rotation matrix and fuse it with the translation matrix through weighted calculation to obtain the equivalent transformation distance; A secondary judgment unit, used for judging the equivalent transformation distance and the second target motion threshold; If the equivalent transformation distance is greater than the second target motion threshold, the target is in motion; Otherwise, the target is at rest.

10. A storage medium, characterized in that The storage medium stores one or more computer-readable programs, and the one or more programs include instructions, which are suitable for being loaded by the memory and executing the target tracking method based on passive velocity estimation of the target detection frame as described in any one of claims 1-5.

Images