Target tracking methods, devices, storage media and electronic devices

Abstract

This invention discloses a target tracking method, apparatus, storage medium, and electronic device. The method includes: acquiring target detection information using a target detection model; performing state prediction on the target detection information based on a target tracking model to obtain state prediction information; constructing an association matrix based on the target detection information and the state prediction information to obtain a matching result; and determining the location information of the target object based on the matching result. This invention solves the technical problem in related technologies where the matching process between the tracked target and the detected target is not accurate enough due to the lack of significant differences in their features.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a target tracking method, apparatus, storage medium, and electronic device. Background Technology

[0002] In target tracking, the traditional Kalman filter method is only suitable for noise that conforms to a Gaussian distribution. However, in many real-world applications, noise is diverse, and for non-Gaussian noise, the Kalman filter's performance in target perception is less than ideal. The Kalman filter is limited to linear Gaussian systems, while particle filtering techniques offer better filtering performance in nonlinear, non-Gaussian systems.

[0003] In the process of tracking targets using particle filtering algorithms, methods such as Euclidean distance are typically used to determine whether the tracked target matches the detected target. However, this method cannot amplify the feature differences between the two. In practical applications, when the two vectors being compared contain multiple dimensions, simply using Euclidean distance to determine the differences results in a large error.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a target tracking method, apparatus, storage medium, and electronic device to at least solve the technical problem in the related art where the matching between the tracked target and the detected target is not accurate due to the lack of significant differences in their features.

[0006] According to one aspect of the present invention, a target tracking method is provided, comprising:

[0007] Target detection information is obtained using a target detection model, including the target object's position, size, orientation angle, velocity components, and acceleration components. State prediction information is then obtained by using a target tracking model to predict the target object's state, including its position, size, orientation angle, velocity components, and acceleration components. An association matrix is ​​constructed based on the target detection information and the state prediction information to obtain a matching result, which represents the matching status between the target detection model and the target tracking model. Finally, the target object's position information is determined based on the matching result.

[0008] Optionally, the target tracking model is used to predict the state of the target detection information. The state prediction information includes: determining the target state information of each particle among multiple particles based on the target detection information; and inputting the target state information into the uniformly accelerated linear motion model to obtain the state prediction information.

[0009] Optionally, constructing an association matrix using target detection information and state prediction information to obtain matching results includes: calculating a first parameter, a second parameter, and a third parameter using target detection information and state prediction information, wherein the first parameter represents the overlap between target detection information and state prediction information, the second parameter represents the directional error between target detection information and state prediction information, and the third parameter represents the feature difference between target detection information and state prediction information; weighting the first parameter, the second parameter, and the third parameter to obtain a weighted result, wherein the weighted result is used to construct an association matrix between the target detection model and the target tracking model; and comparing each element value in the association matrix with a target threshold to obtain a matching result.

[0010] Optionally, determining the location information of the target object based on the matching result includes: in response to the existence of a target detection model and a target tracking model matching, fusing the target detection information and the state prediction information to obtain the location information of the target object.

[0011] Optionally, the target tracking method further includes: obtaining candidate velocity values ​​between multiple corresponding points of the target detection model and the target tracking model; calculating the difference between the multiple candidate velocity values ​​and the velocity component detection information to obtain multiple difference results; and determining the candidate velocity value corresponding to the minimum value among the multiple difference results as the velocity measurement value of the target object.

[0012] Optionally, determining the location information of the target object based on the matching result includes: in response to the absence of a match between the target detection model and the target tracking model, determining the location information of the target object based on the state prediction information.

[0013] Optionally, determining the location information of the target object based on the matching result includes: in response to the absence of a match between the target tracking model and the target detection model, obtaining the number of times the target tracking model is lost; in response to the number of times the target tracking model is lost being greater than a preset value, deleting the target detection information; or, in response to the number of times the target tracking model is lost being less than a preset value, determining the location information of the target object based on the target detection information.

[0014] According to one embodiment of this application, a target tracking device is also provided, comprising:

[0015] The module comprises the following components: an acquisition module, which acquires target detection information using a target detection model, including the target object's position, size, orientation angle, velocity components, and acceleration components; a prediction module, which performs state prediction on the target detection information based on a target tracking model, obtaining state prediction information, including the target object's position, size, orientation angle, velocity components, and acceleration components; a construction module, which constructs an association matrix based on the target detection information and state prediction information, obtaining a matching result, which represents the matching status between the target detection model and the target tracking model; and a determination module, which determines the target object's position information based on the matching result.

[0016] Optionally, the prediction module is also used to determine the target state information of each of the multiple particles based on the target detection information; and input the target state information into the uniformly accelerated linear motion model to obtain state prediction information.

[0017] Optionally, the construction module is further configured to: calculate a first parameter, a second parameter, and a third parameter using target detection information and state prediction information, wherein the first parameter represents the overlap between target detection information and state prediction information, the second parameter represents the directional error between target detection information and state prediction information, and the third parameter represents the feature difference between target detection information and state prediction information; perform weighted processing on the first parameter, the second parameter, and the third parameter to obtain a weighted result, wherein the weighted result is used to construct the correlation matrix between the target detection model and the target tracking model; and compare each element value in the correlation matrix with a target threshold to obtain a matching result.

[0018] Optionally, the determining module is also used to: in response to the existence of a target detection model and a target tracking model matching, fuse the target detection information and the state prediction information to obtain the position information of the target object.

[0019] Optionally, the target tracking device further includes: a calculation module, used to obtain candidate velocity values ​​between multiple corresponding points of the target detection model and the target tracking model; to perform difference calculation using multiple candidate velocity values ​​and velocity component detection information to obtain multiple difference results; and to determine the candidate velocity value corresponding to the minimum value among the multiple difference results as the velocity measurement value of the target object.

[0020] Optionally, the determination module is also used to: determine the location information of the target object based on the state prediction information in response to the absence of a match between the target detection model and the target tracking model.

[0021] Optionally, the determining module is further configured to: in response to the absence of a match between the target tracking model and the target detection model, obtain the number of times the target tracking model is lost; in response to the number of times the target tracking model is lost being greater than a preset value, delete the target detection information; or, in response to the number of times the target tracking model is lost being less than a preset value, determine the location information of the target object based on the target detection information.

[0022] According to one embodiment of this application, a non-volatile storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program is configured to execute the target tracking method described in any of the above claims when running.

[0023] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to perform the target tracking method of any of the preceding claims.

[0024] In this embodiment of the invention, target tracking is achieved through a particle filtering algorithm. During implementation, an association matrix is ​​established between the tracked target and the detected target, and the difference between them is calculated. The association matrix is ​​then optimized, and a match is determined. Finally, target tracking management is achieved based on the matching result. This solves the technical problem in related technologies where the matching between the tracked target and the detected target is not accurate due to the insignificant difference in their features. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a flowchart of a target tracking method according to one embodiment of this application;

[0027] Figure 2 This is a schematic diagram of a detection box and a tracking prediction box according to one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the ground truth bounding box and detection bounding box of a radar point cloud according to one embodiment of this application;

[0029] Figure 4 This is a schematic diagram of the target optimal estimate of the previous time step and the target detection information of the next time step matched according to one embodiment of this application;

[0030] Figure 5 This is a schematic diagram of a target tracking method according to one embodiment of this application;

[0031] Figure 6 This is a structural block diagram of a target tracking device according to one embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] According to an embodiment of the present invention, an embodiment of a target tracking method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] Figure 1 This is a flowchart of a target tracking method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0036] Step S11: Obtain target detection information using a target detection model, wherein the target detection information includes the target object's position detection information, size detection information, orientation angle detection information, velocity component detection information, and acceleration component detection information;

[0037] In the target detection model, target detection information is obtained by detecting targets based on a deep learning model. The target tracking model includes a target tracking sequence, which contains target state information and target state prediction information.

[0038] In the object detection model, object detection information includes the following 7 dimensions {x} i -1,y i-1 ,z i-1 ,l i-1 ,h i-1 ,w i-1 ,yaw i-1}, where x i-1 ,y i-1 ,z i-1 For target location detection information, l i-1 ,h i-1 For target size detection information, w i-1 ,yaw i-1 Detects information on the target's orientation angle.

[0039] Alternatively, the object detection model can be a Region-Convolutional Neural Network (R-CNN) model.

[0040] Step S12: Based on the target tracking model, the target detection information is used to predict the state to obtain state prediction information, which includes the target object's position state information, size state information, orientation angle state information, velocity component state information, and acceleration component state information.

[0041] This application embodiment uses a particle filter algorithm to achieve target tracking. Specifically, the particle filter algorithm implementation process includes the following steps:

[0042] (1) Particle initialization stage

[0043] The target motion state is simulated by a large number of particles. The particles conform to a certain mathematical distribution in space, and the weight of each particle conforms to a uniform distribution.

[0044] (2) Particle Prediction Stage

[0045] Based on the prediction model, the particle state information of each particle at the previous moment is substituted into the model to obtain the particle state prediction information at the current moment.

[0046] (3) Particle Renewal Phase

[0047] The weights of each particle are updated based on the particle state prediction information and target detection information at the current moment.

[0048] (4) Particle resampling stage

[0049] Particles are filtered based on their weights. During the filtering process, a large number of particles with high weights are retained, while a small number of particles with low weights are also retained.

[0050] (5) Particle filtering stage

[0051] The resampled particles are fused with the target detection information at the current moment to obtain the optimal estimate of the particle at the current moment. This optimal estimate is used as the state information of the particle at the current moment and returned to the prediction stage (2).

[0052] In the target tracking model, the target state information includes the following 11 dimensions {x} j -1,y j-1 ,z j-1 ,l j-1 ,h j-1 ,w j-1 ,yaw j-1 Vx j-1 Vy j-1 Ax j-1 Ay j-1}, where x j-1 ,y j-1 ,z j-1 For the target's location and status information, l j-1 ,h j-1 For the target's size and state information, w j-1 ,yaw j-1 Vx represents the target's orientation angle state information. j-1 Vy j-1 For the target's velocity component state information, Ax j-1 Ay j-1 The state information of the acceleration components of the target.

[0053] Initialization of target state information: Initialization of target state information refers to the initialization of the target state information value, which is the target state information value at the first moment in the tracking sequence of the target tracking model. This first moment target state information value is assigned by the target detection information at the first moment in the corresponding target detection model, that is:

[0054] {x j-1 ,y j-1 ,z j-1 ,l j-1 ,h j-1 ,w j-1 ,yaw j-1 Vx j-1 Vy j-1 Ax j-1 Ay j-1}={x i-1 ,y i-1 ,z i-1 ,l i-1 ,h i-1 ,w i-1 ,yaw i-1 ,0,0,0,0}

[0055] Where i and j represent time information, and i = j. Since the initial velocity and acceleration are unavailable, Vx in the target detection information... j-1 Vy j-1 Ax j-1 Ay j-1 All four components have a value of 0.

[0056] In the target tracking model of this application, the target state information value at the first moment is the value of the corresponding component of the target detection information at the first moment in the corresponding target detection model, and the target state information value at the current moment is the target state information value obtained by the target tracking model using the particle filter algorithm.

[0057] Step S13: Construct an association matrix based on target detection information and state prediction information to obtain matching results, wherein the matching results are used to represent the matching situation between the target detection model and the target tracking model;

[0058] At the same time, the target detection information in each target detection model and the state prediction information in each target tracking model are used to construct an association matrix through a certain calculation method. This association matrix is ​​used to represent the differences between the obstacles detected and the tracking sequences of each target detection model and each target tracking model.

[0059] Construct the correlation matrix between each target detection model and each target tracking model, as follows:

[0060] At any given time, there are m target detection models (detection boxes) and n target tracking models (tracking prediction boxes), resulting in an (m×n) M matrix:

[0061]

[0062] Where, m mn This parameter represents the feature difference between the m-th obstacle and the n-th tracking sequence. It is mainly composed of three parameters: IOU error, orientation error, and Mahalanobis distance.

[0063] By analyzing m mn The matching result between the target detection model and the target tracking model is obtained through the judgment, and the result includes the following two cases:

[0064] (1) Matching of target detection model and target tracking model;

[0065] (2) The mismatch between the target detection model and the target tracking model includes the following two situations:

[0066] 1) No object detection model matches the object tracking model;

[0067] 2) No target tracking model matches the target detection model.

[0068] Step S14: Determine the location information of the target object based on the matching results.

[0069] The target detection model is matched with the target tracking model, and the target state information value obtained by the target tracking model through the particle filter algorithm is used as the position information of the target object at the current moment.

[0070] The mismatch between the object detection model and the object tracking model includes the following two situations:

[0071] (1) No target detection model matches the target tracking model;

[0072] The target detection model does not match any of the target tracking models. This target detection model corresponds to the newly observed target, and new trajectory information about the target is created.

[0073] (2) No target tracking model matches the target detection model.

[0074] The target tracking model does not match any of the target detection models, and the target tracking model loses the corresponding target.

[0075] Through steps S11 to S14, this process achieves target tracking using a particle filtering algorithm. In its specific implementation, an association matrix is ​​established between the tracked target and the detected target, and the difference between them is calculated. The association matrix is ​​then optimized, and a match is determined. Finally, target tracking and management are achieved based on the matching result. This solves the technical problem in related technologies where the matching between the tracked target and the detected target is not accurate due to the insignificant difference in their features.

[0076] Optionally, in step S12, state prediction is performed on the target detection information based on the target tracking model to obtain state prediction information. This step includes the following execution steps:

[0077] Step S121: Determine the target state information of each particle among multiple particles based on the target detection information;

[0078] Step S122: Input the target state information into the uniformly accelerated linear motion model to obtain state prediction information.

[0079] Acquisition of target state prediction information: Optionally, this application uses a uniformly accelerated linear motion model to obtain target state prediction information. The target state information of the previous time step (j-1) is input into the uniformly accelerated linear motion model to obtain the target state prediction information of the target at the current time step (j).

[0080]

[0081] Where Q represents the process covariance of state information in each dimension.

[0082] Through steps S121 to S122, the target state information of the previous moment in the tracking sequence of the target tracking model is brought into the uniformly accelerated linear motion model to obtain the target state prediction information at the current moment.

[0083] Optionally, in step S13, an association matrix is ​​constructed based on the target detection information and state prediction information to obtain the matching result. This step includes the following execution steps:

[0084] Step S131: Calculate the first parameter, the second parameter, and the third parameter using the target detection information and the state prediction information. The first parameter is used to represent the overlap between the target detection information and the state prediction information, the second parameter is used to represent the directional error between the target detection information and the state prediction information, and the third parameter is used to represent the feature difference between the target detection information and the state prediction information.

[0085] The first parameter is obtained by calculating IOU, and the formula for calculating IOU is as follows:

[0086]

[0087] Where, Detectionresult(m) represents the information of the m-th detected obstacle, that is, the target detection information of the obstacle in the m-th target detection model at the current time, i.e., the detection box information of the obstacle; Trackpredictresult(n) represents the information of the n-th track prediction box, that is, the target state prediction information at the current time obtained by the n-th target tracking model based on the target state information of the previous time step in its tracking sequence. For example, as Figure 2 As shown, Figure 2 This is a schematic diagram of detection boxes and tracking prediction boxes according to one embodiment of this application. Vehicle number 0 represents the vehicle itself, and the remaining numbers are represented by solid black lines, indicating the detection box information of the current obstacle. The dashed black lines represent the tracking prediction box information. Vehicle number 1's detection box does not overlap with the adjacent tracking prediction box, so its IOU value is 0, indicating a very low probability of matching. That is, the target tracking model represented by the adjacent tracking prediction box is unlikely to match the target detection model represented by Vehicle number 1's detection box. For vehicles 3 and 5, the tracking prediction boxes overlap significantly with the detection boxes, resulting in higher IOU values ​​and a higher probability of matching. That is, the target tracking model represented by the corresponding tracking prediction box is more likely to match the target detection model represented by the corresponding detection box. The higher the IOU value, the greater the probability of a match between the calculated target detection model represented by the detection box and the target tracking model represented by the tracking prediction box.

[0088] The second parameter is obtained by calculating the direction error, and the calculation formula is as follows:

[0089]

[0090]

[0091] Among them, yaw error (m,n) represents the orientation angle value of the m-th detected obstacle, i.e., the angle value yaw of the obstacle in the target detection information at the current time t in the m-th target detection model. t (m) and the angle value in the target state prediction information at the current time t obtained by predicting the target state information at the previous time t-1 in the tracking sequence of the nth target tracking model. The difference; the Direction(m,n) value represents the direction error between the m-th detected obstacle and the n-th tracking sequence, and this value should be as small as possible. t represents the current time, t-1 represents the previous time, and t = i = j.

[0092] The third parameter is obtained by calculating the feature distance. Based on the idea of ​​nearest neighbor association, the features of obstacles between the target detection model and the tracking box are calculated. The specific calculation method is as follows:

[0093]

[0094] Among them, {x t (m),y t (m),z t (m),l t (m),h t (m),w t (m),yaw t (m)} represents the target detection information of the obstacle at the current time t in the m-th detection frame; This represents the target state prediction information at the current time t obtained from the prediction of the obstacle at the previous time t-1 in the nth tracking sequence. Distance(m,n) represents the feature difference value between the detection box and the tracking sequence, and S represents the covariance matrix of the dimensional state quantities of the detection box and the tracking sequence. t represents the current time, t-1 represents the previous time, and t = i = j.

[0095] Step S132: The first parameter, the second parameter, and the third parameter are weighted to obtain a weighted result, which is used to construct the correlation matrix between the target detection model and the target tracking model.

[0096] The correlation matrix M between each target detection model and each target tracking model represents the difference in features between the target detection information at the current moment for each detected obstacle and the target state prediction information at the current moment obtained from the target state information at the previous moment in each tracking sequence, i.e., each element M mn Let represent the feature difference between the (m)th obstacle and the (n)th tracking sequence.

[0097] M mn =α·IOU(m,n)-β·Direction(m,n)-γ·Distance(m,n)

[0098] α+β+γ=1.

[0099] Step S133: Compare each element value in the association matrix with the target threshold to obtain the matching result.

[0100] Step S1331: Obtain the target threshold;

[0101] Step S13311: Perform IOU matching using the labeled radar point cloud ground truth data and target detection information, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the ground truth bounding box and detection bounding box of a radar point cloud according to one embodiment of this application. The black dashed box represents the ground truth bounding box, and the black solid box represents the detection bounding box of an obstacle. The IOU calculation formula is as follows:

[0102]

[0103] Here, Det(i) represents the detection bounding box information of the detected obstacle, and Gt(j) represents the ground truth bounding box information of the ground truth target. After calculating the IOU matrix, an offline greedy algorithm is used for matching to obtain matching pairs {Det(i), Gt(j)}. By adjusting the greedy strategy of the offline greedy algorithm, a suitable number of matches is obtained, that is, one or more matching pairs of ground truth bounding boxes and detection bounding boxes are obtained through the offline greedy algorithm.

[0104] Step S13312: Obtain the target detection information and ground truth information of each matching pair obtained in step S13311, substitute them into the above-mentioned third parameter Distance(m,n) calculation formula, obtain the feature difference value of each matching pair, and calculate the average value of these feature difference values. Finally, the average value is the target threshold used for matching in the correlation matrix M.

[0105] Step S1332: Match each element value in the association matrix with the target threshold.

[0106] The elements in matrix M are matched with the target threshold, and a greedy algorithm is used to find the optimal solution.

[0107] The elements M in the correlation matrix M mn The elements are compared with the target threshold obtained in step S13312. Elements smaller than the target threshold are considered matches. If the number of matching tracking sequences or obstacle detection information for the same obstacle detection information or the same tracking sequence is greater than 1, a greedy algorithm is used to ensure that at most one tracking sequence matches the same obstacle detection information or at most one obstacle detection information matches the same tracking sequence. Elements in the correlation matrix M that are greater than or equal to the target threshold are considered non-matches.

[0108] When the detection information of the same obstacle does not match any of the tracking sequences, the obstacle is considered a new object of observation.

[0109] If the same tracking sequence does not match any of the obstacle detection information, the tracking sequence loses its tracking target.

[0110] By establishing the correlation matrix between each target detection model and each target tracking model through steps S131 to S133, and comparing the feature differences between the obstacle detection information and the tracking sequence, it can be determined whether the two match.

[0111] Optionally, in step S14, the location information of the target object is determined based on the matching result. This step includes the following execution steps:

[0112] Step S141: In response to the existence of target detection model and target tracking model matching, the target detection information and state prediction information are fused to obtain the position information of the target object.

[0113] When the target detection model and the target tracking model are matched, a particle filtering algorithm is used to combine the target detection information at the current moment in the target detection model with the particle state prediction information of each particle based on the particle state information of the previous moment. The two are then fused to obtain the target state information at the current moment in the tracking sequence of the target tracking model. The specific details of this fusion calculation are as follows:

[0114] Step S1411, Particle initialization phase: Set up np particles, and the state information of each particle includes {x p-1 ,y p-1 ,z p-1 ,l p-1 ,h p-1 ,w p-1 ,yaw p-1 Vx p-1 Vy p-1 Ax p-1 Ay p-1} 11 dimensions of information. Each particle is normally distributed in space based on the target detection information, and the initial weights of each particle follow a uniform distribution, i.e., the weights of each particle. p-1 = 1 / np. p-1 represents the previous time step, p represents the current time step, and p = i = j.

[0115] Step S1412, Particle Prediction Stage: Optionally, this application uses a uniformly accelerated linear motion model to obtain particle state prediction information. Taking one particle as an example, the particle state information at the previous time (p-1) is input into the uniformly accelerated linear motion model to obtain the particle state prediction information for the target at the current time (p).

[0116]

[0117] weight p =1 / np

[0118] Where Q represents the process covariance of state information in each dimension. The angle output by the particle filter algorithm The corresponding range is [-π, π]. The predicted angles are constrained to prevent exceeding this range, thus obtaining the predicted particle state information.

[0119] Step S1413, Particle Update Phase: Update the weights of each particle. Taking a single particle as an example, the specific calculation is as follows:

[0120] First, calculate the likelihood probability of each state information attribute of the particle.

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130] Among them, Rx ,R y ,R z ,R l ,R w ,R h ,R yaw ,R Vx ,R Vy , representing the observation error of each dimension of the state information; {x i ,y i ,z i ,l i ,h i ,w i ,yaw i Vx i Vy i Ax i Ay i} represents the target detection information at the current time p in the target detection model; This is the particle state prediction information at the current time p, which is obtained by predicting the particle state information at the previous time p-1.

[0131] Next, calculate the likelihood probability density of the particle. The specific calculation method is as follows:

[0132]

[0133] The particle's non-normalized weights are then updated, and the specific calculation method is as follows:

[0134]

[0135] After updating the non-normalized weights of each particle according to the above calculation method, the new normalized weights of all particles are then calculated.

[0136] Step S1414, Particle Resampling Stage: Normalizing the weights of each particle in step S1411 The values ​​are superimposed to generate values ​​in the range of 0-1, which are the normalized weights of each particle. The sum of the generated 0-1 values ​​is 1. There are a total of np particles, and the above particles... The generated 0-1 values ​​can be divided into np intervals. The random machine simultaneously generates a random number between 0 and 1, determines which interval the random number belongs to, and copies the current state prediction information of the particle corresponding to that interval to the current state prediction information of the new particle. In the process, np random number generation is performed sequentially, ultimately resulting in np new particle pairs for the current time. Particle state prediction information.

[0137] For example, in the particle resampling stage, there are 20 updated and normalized weights. Twenty 0-1 values ​​are generated by superimposing these values, meaning the sum of these 20 0-1 values ​​is 1. The larger the particle weight, the larger the corresponding generated value. Assume particle 1 generates a 0-1 value w1, particle 2 generates a 0-1 value w2, particle 3 generates a 0-1 value w3, and particle 20 generates a 0-1 value w20. w1, w2, ..., w19, w20 form 20 intervals between w1-w2-...w19-w20, meaning the larger the particle weight, the larger the interval range. A random 0-1 value is generated by the machine, and it is determined which interval range it belongs to. For example, if it belongs to the interval w2-w3, the state prediction information of the particle corresponding to w2 at the current moment is copied to a new particle. This process is repeated 20 times, generating 20 new values ​​and corresponding state prediction information for the current moment for each of the 20 generated particles. When a particle has a larger weight, the probability of the generated random number falling within the corresponding interval is greater. This allows for the multiple replication of particles with large weights and the removal of particles with small weights. In this resampling process, not all particles with large weights are replicated; particles with small weights may also be replicated. This is because, even though the interval is small, there is a probability that the generated random number will fall within the interval corresponding to the particle with the small weight, thus ensuring particle diversity to a certain extent.

[0138] Step S1415, Particle Filtering Stage: The resampled particles are fused with the target detection information to obtain the optimal estimate of the target at the current moment. This optimal estimate is used as the target state information at the current moment, and the target state information at the current moment is also used as the target's forward vehicle position information at the current moment.

[0139] Current position information of the target vehicle in front {x j ,y j ,z j ,l j ,h j ,w j ,yaw j Vx j Vy j Ax j Ay j That is, the target state information at the current moment is obtained by performing the following calculations on the state prediction information of each of the np resampled particles:

[0140]

[0141] Therefore, by using the information of each particle's state at the current moment, we can predict the state of each particle. and the target detection information at the current moment {x i ,y i ,z i ,l i ,h i ,w i ,yaw i Vx i Vy i Ax i Ay i The target vehicle's position information at the current moment was obtained through step-by-step fusion calculations. j ,y j ,z j ,l j ,h j ,w j ,yaw j Vx j Vy j Ax j Ay j}

[0142] Step S141 involves obtaining the position information of the target vehicle ahead at the current moment using a particle filter algorithm.

[0143] Optionally, the above target tracking method may include the following execution steps:

[0144] Step S15: Obtain alternative velocity values ​​between multiple corresponding points of the target detection model and the target tracking model;

[0145] Step S16: Calculate the difference between multiple candidate velocity values ​​and velocity component detection information to obtain multiple difference results;

[0146] Step S17: Determine the candidate speed value corresponding to the minimum value among multiple difference results as the speed measurement value of the target object.

[0147] For a matching target detection model and target tracking model, the velocity measurement at the current moment can be calculated by using the position information of the obstacle at the current moment corresponding to the target detection model and the obstacle at the previous moment corresponding to the target tracking model, and the ratio of the displacement difference between the two to the time interval. For example, Figure 4 As shown, Figure 4This is a schematic diagram illustrating the matching of the optimal target estimate from the previous time step and the target detection information from the next time step according to one embodiment of this application. The solid black boxes and dashed black boxes represent vehicles, and they are matched. The vehicle represented by the solid black box indicates the vehicle information detected at time i, and the vehicle represented by the dashed black box indicates the optimal estimate for that vehicle at time i-1. The relationship between corresponding points is represented by dashed lines. Five pairs of candidate speed measurements at time i are obtained by performing the following calculations on five pairs of vehicle position points from the two time steps. The specific calculations are as follows:

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] Where PM represents the current location of the obstacle, and PT represents the location of the obstacle in the optimal estimate at the previous time step. The five alternative velocity measurements at time i are compared with the velocity V in the optimal estimate obtained at time i-1. j-1 The difference calculation is performed, and the candidate velocity measurement value with the smallest gain is used as the current velocity measurement value V of the obstacle. i This is to ensure the smoothness of the speed.

[0154] Through steps S15 to S17, alternative velocity measurement values ​​for the corresponding position points are obtained by comparing the displacement difference between the current position point of the target obstacle and the position point in the optimal estimate at the previous moment with the time interval between the two. Then, the difference between these alternative velocity measurement values ​​and the velocity in the optimal estimate at the previous moment is calculated. Finally, the alternative velocity measurement value with the smallest gain obtained from the difference calculation is taken as the velocity measurement value of the obstacle at the current moment.

[0155] Optionally, in step S14, the location information of the target object is determined based on the matching result. This step includes the following steps:

[0156] Step S142: In response to the absence of a match between the target detection model and the target tracking model, the position information of the target object is determined based on the state prediction information.

[0157] The target detection model does not match any of the target tracking models. The obstacle corresponding to this target detection model is the newly observed target, and new trajectory information about the target is created.

[0158] The target detection model does not match any of the target tracking models. Therefore, a target tracking model that is currently idle and has no tracked targets is invoked to track the obstacle. The current target state prediction information is obtained by directly predicting the target state information at the current moment based on the previous time step's state information value in the tracking sequence of that target tracking model. The current position information of the target vehicle ahead {x j ,y j ,z j ,l j ,h j ,w j ,yaw j Vx j Vy j Ax j Ay j}

[0159] Optionally, in step S14, the location information of the target object is determined based on the matching result. This step includes the following steps:

[0160] Step S143: In response to the absence of a match between the target tracking model and the target detection model, obtain the number of times the target tracking model has been lost;

[0161] The target tracking model does not match any of the target detection models, and the target tracking model loses the corresponding target.

[0162] Step S144: In response to the target tracking framework losing more than a preset value, delete the target detection information; or, in response to the target tracking model losing less than a preset value, determine the position information of the target object based on the target detection information.

[0163] If the number of times the target tracking model loses its target is greater than or equal to a preset value, the detection information in the corresponding target detection model is deleted, that is, the target detection information {x} at the current time step in the corresponding target detection model is deleted. i ,y i ,z i ,l i ,h i ,w i ,yaw i Vx i Vy i Ax i Ay i}; When the number of times the target tracking model loses track is less than a preset value, the target detection information {x} in the corresponding target detection model is used. i ,y i ,z i ,l i ,h i ,w i ,yawi Vx i Vy i Ax i Ay i This serves as the current state information of the tracking sequence, i.e., the current position information of the target vehicle ahead.

[0164] Through steps S141 to S144, the target detection model and the target tracking model are matched, and then the tracker is deleted or added, thereby realizing the tracking management of each dynamic target.

[0165] Figure 5 This is a schematic diagram of a target tracking method according to one embodiment of this application, as shown below. Figure 5 As shown, the method includes the following steps:

[0166] Step S501: Obtain the target state information of the current moment in the tracking sequence of each target tracking model;

[0167] Step S502: Obtain the target state prediction information for the next moment from the target state information at the current moment according to the state transition equation;

[0168] Step S503: Obtain the target detection information of each target detection model at the next time step;

[0169] Step S504: Calculate the difference information between each target tracking model and each target detection model;

[0170] Step S505: Establish a correlation matrix between each target detection model and each target tracking model using the difference information between each target tracking model and each target detection model;

[0171] Step S506: Determine whether the corresponding target detection model and target tracking model match;

[0172] Step S507: If the target detection model and the target tracking model match, the target detection information at the next moment and the particle state prediction information of each particle at the next moment are fused and calculated using the particle filtering algorithm to obtain the target state information at the next moment, and the target tracking model is updated with the target state information at the next moment in its tracking sequence.

[0173] Step S508: If the target detection model does not match any of the target tracking models, the obstacle corresponding to the target detection model is the new observed object;

[0174] Step S509: Create trajectory information for the target;

[0175] Step S510: If the target tracking model does not match any of the target detection models, the target tracking model loses the target it is tracking.

[0176] Step S511: Determine whether the number of times the target tracking model has lost tracking targets is greater than a threshold. If the number of times the target tracking model has lost tracking targets is greater than the threshold, proceed to step S512; if the number of times the target tracking model has lost targets is not greater than the threshold, use the target detection information as the target state information for the next moment and proceed to step S513.

[0177] Step S512: Delete the target detection information of the lost tracked target;

[0178] Step S513: Take the target state information of the next moment as input and proceed to step S501.

[0179] Through steps S501 to S513, the predicted state information of each target at the current moment is obtained by predicting the state information of each target at the next moment. The predicted state information of each target at the next moment is then fused with the target detection information of each target at the next moment to calculate the correlation matrix between the target detection model and the target tracking model. By determining whether the target detection model and the target tracking model corresponding to each element in the correlation matrix match, if they match, the target is further tracked; otherwise, the target trajectory information is deleted or created, thereby achieving dynamic target tracking management.

[0180] Figure 6 This is a structural block diagram of a target tracking device according to one embodiment of this application, such as... Figure 6 As shown, the target tracking device 600 includes:

[0181] The acquisition module 601 is used to acquire target detection information using a target detection model, wherein the target detection information includes the target object's position detection information, size detection information, orientation angle detection information, velocity component detection information, and acceleration component detection information; the prediction module 602 is used to perform state prediction on the target detection information based on a target tracking model to obtain state prediction information, wherein the state prediction information includes the target object's position state information, size state information, orientation angle state information, velocity component state information, and acceleration component state information; the construction module 603 is used to construct an association matrix based on the target detection information and the state prediction information to obtain a matching result, wherein the matching result is used to represent the matching situation between the target detection model and the target tracking model; and the determination module 604 is used to determine the target object's position information based on the matching result.

[0182] Optionally, the prediction module 602 is further configured to determine the target state information of each of the multiple particles based on the target detection information; and input the target state information into the uniformly accelerated linear motion model to obtain state prediction information.

[0183] Optionally, the construction module 603 is further configured to: calculate a first parameter, a second parameter, and a third parameter using target detection information and state prediction information, wherein the first parameter represents the overlap between target detection information and state prediction information, the second parameter represents the directional error between target detection information and state prediction information, and the third parameter represents the feature difference between target detection information and state prediction information; perform weighted processing on the first parameter, the second parameter, and the third parameter to obtain a weighted result, wherein the weighted result is used to construct the correlation matrix between the target detection model and the target tracking model; and compare each element value in the correlation matrix with a target threshold to obtain a matching result.

[0184] Optionally, the determining module 604 is further configured to: in response to the existence of a target detection model and a target tracking model matching, fuse the target detection information and the state prediction information to obtain the position information of the target object.

[0185] Optionally, the target tracking device 600 further includes: a calculation module 605, used to obtain candidate velocity values ​​between multiple corresponding points of the target detection model and the target tracking model; to perform difference calculation using multiple candidate velocity values ​​and velocity component detection information to obtain multiple difference results; and to determine the candidate velocity value corresponding to the minimum value among the multiple difference results as the velocity measurement value of the target object.

[0186] Optionally, the determining module 604 is further configured to: determine the position information of the target object based on the state prediction information in response to the absence of a match between the target detection model and the target tracking model.

[0187] Optionally, the determining module 604 is further configured to: in response to the absence of a target tracking model matching the target detection model, obtain the number of times the target tracking model is lost; in response to the number of times the target tracking model is lost being greater than a preset value, delete the target detection information; or, in response to the number of times the target tracking model is lost being less than a preset value, determine the position information of the target object based on the target detection information.

[0188] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0189] Embodiments of this application also provide a non-volatile storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0190] Optionally, in this embodiment, the non-volatile storage medium described above can be configured to store a computer program for performing the following steps:

[0191] S1. Obtain target detection information using a target detection model. The target detection information includes the target object's position detection information, size detection information, orientation angle detection information, velocity component detection information, and acceleration component detection information.

[0192] S2, based on the target tracking model, perform state prediction on the target detection information to obtain state prediction information, which includes the target object's position state information, size state information, orientation angle state information, velocity component state information, and acceleration component state information.

[0193] S3. Based on the target detection information and state prediction information, an association matrix is ​​constructed to obtain the matching result, which is used to represent the matching situation between the target detection model and the target tracking model.

[0194] S4, determine the location information of the target object based on the matching results.

[0195] Optionally, in this embodiment, the aforementioned non-volatile storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0196] Embodiments of this application also provide a wireless programming and transmitting device, including a memory, a processor, a first wireless network chip, and a second wireless network chip, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0197] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0198] S1. Obtain target detection information using a target detection model. The target detection information includes the target object's position detection information, size detection information, orientation angle detection information, velocity component detection information, and acceleration component detection information.

[0199] S2, based on the target tracking model, perform state prediction on the target detection information to obtain state prediction information, which includes the target object's position state information, size state information, orientation angle state information, velocity component state information, and acceleration component state information.

[0200] S3. Based on the target detection information and state prediction information, an association matrix is ​​constructed to obtain the matching result, which is used to represent the matching situation between the target detection model and the target tracking model.

[0201] S4, determine the location information of the target object based on the matching results.

[0202] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0203] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0205] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0206] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0207] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0208] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A target tracking method characterized by, The method comprises the following steps: obtaining target detection information by using a target detection model, wherein the target detection information comprises position detection information, size detection information, orientation angle detection information, velocity component detection information and acceleration component detection information of a target object; performing state prediction on the target detection information according to a target tracking model to obtain state prediction information, wherein the state prediction information comprises position state information, size state information, orientation angle state information, velocity component state information and acceleration component state information of the target object; constructing a correlation matrix based on the target detection information and the state prediction information to obtain a matching result, wherein the matching result is used to represent the matching situation of the target detection model and the target tracking model; determining position information of the target object based on the matching result; wherein constructing the correlation matrix based on the target detection information and the state prediction information to obtain the matching result comprises: calculating a first parameter, a second parameter and a third parameter based on the target detection information and the state prediction information, wherein the first parameter is used to represent the overlap between the target detection information and the state prediction information, the second parameter is used to represent the direction error between the target detection information and the state prediction information, and the third parameter is used to represent the feature difference between the target detection information and the state prediction information; performing weighted processing on the first parameter, the second parameter and the third parameter to obtain a weighted result, wherein the weighted result is used to construct the correlation matrix between the target detection model and the target tracking model; and performing comparison processing on each element value in the correlation matrix with a target threshold to obtain the matching result; The step of obtaining the target threshold is: performing IOU matching on labeled radar point cloud ground truth data and target detection information by using an offline greedy algorithm to obtain one or more matching pairs of true value boxes and detection boxes; calculating the feature difference value of each matching pair by using the target detection information and the true value information of each matching pair; and performing average value calculation on the feature difference values of each matching pair to obtain the target threshold.

2. The object tracking method of claim 1, wherein, The step of performing state prediction on the target detection information by using the target tracking model to obtain the state prediction information comprises: determining target state information of each particle in a plurality of particles according to the target detection information; inputting the target state information into a uniform acceleration straight line motion model to obtain the state prediction information.

3. The object tracking method of claim 1, wherein, The step of determining the position information of the target object based on the matching result comprises: in response to the existence of the matching of the target detection model and the target tracking model, performing fusion processing on the target detection information and the state prediction information to obtain the position information of the target object.

4. The target tracking method according to claim 3, characterized by, The method further comprises: obtaining candidate velocity values between a plurality of corresponding points of the target detection model and the target tracking model; performing difference value calculation on a plurality of candidate velocity values and the velocity component detection information to obtain a plurality of difference value results; determining the candidate velocity value corresponding to the minimum value in the plurality of difference value results as the velocity measurement value of the target object.

5. The object tracking method of claim 1, wherein, The position information of the target object is determined based on the matching result, and the position information of the target object is determined based on the state prediction information in response to the target detection model not matching the target tracking model. The position information of the target object is determined based on the matching result, and the position information of the target object is determined based on the state prediction information in response to the target detection model not matching the target tracking model.

6. The object tracking method of claim 1, wherein, The position information of the target object is determined based on the matching result, and the position information of the target object is determined based on the state prediction information in response to the target detection model not matching the target tracking model. In response to the target tracking model not matching the target detection model, the number of losses of the target tracking model is obtained. In response to the number of losses of the target tracking model being greater than a preset value, the target detection information is deleted, or in response to the number of losses of the target tracking model being less than the preset value, the position information of the target object is determined based on the target detection information.

7. A target tracking device, characterized by Comprise: An acquisition module is configured to acquire target detection information by using a target detection model, wherein the target detection information comprises position detection information, size detection information, orientation angle detection information, velocity component detection information and acceleration component detection information of a target object; A prediction module is configured to perform state prediction on the target detection information according to a target tracking model to obtain state prediction information, wherein the state prediction information comprises position state information, size state information, orientation angle state information, velocity component state information and acceleration component state information of the target object; A construction module is configured to construct an association matrix based on the target detection information and the state prediction information to obtain a matching result, wherein the matching result is used to represent the matching of the target detection model and the target tracking model; A determination module is configured to determine the position information of the target object based on the matching result; The construction module is further configured to calculate a first parameter, a second parameter and a third parameter by using the target detection information and the state prediction information, wherein the first parameter is used to represent the overlap between the target detection information and the state prediction information, the second parameter is used to represent the direction error between the target detection information and the state prediction information, and the third parameter is used to represent the feature difference between the target detection information and the state prediction information; the first parameter, the second parameter and the third parameter are weighted to obtain a weighted result, wherein the weighted result is used to construct the association matrix between the target detection model and the target tracking model; each element value in the association matrix is compared with a target threshold to obtain the matching result; The target threshold is obtained by using an offline greedy algorithm to perform IOU matching on labeled radar point cloud ground truth data and target detection information to obtain one or more matching pairs of ground truth boxes and detection boxes; the feature difference values of each matching pair are calculated by using the target detection information and the ground truth information of each matching pair; the feature difference values of each matching pair are averaged to obtain the target threshold.

8. A non-volatile storage medium, comprising: The storage medium stores a computer program, wherein the computer program is configured to execute the target tracking method described in any one of claims 1 to 6 when running. 9.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to perform the target tracking method in any one of claims 1 to 6.

Images