Target tracking methods, devices, electronic equipment and storage media

Abstract

This invention relates to the field of target tracking technology, specifically to target tracking methods, apparatus, electronic devices, and storage media. The method includes: acquiring the position of a detected target in the current frame, the detection confidence level of the detected target, and state parameters from the previous frame; predicting the state of a predicted target in the current frame based on the state parameters from the previous frame, obtaining a state prediction result for the predicted target; updating the state prediction result based on the detection confidence level, determining a target state prediction result for the predicted target; and performing target tracking based on the position of the detected target and the target state prediction result, determining a tracking result. This method introduces the detection confidence level of the detected target to update the state prediction result, which can reduce prediction errors.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and more specifically to target tracking methods, devices, electronic devices, and storage media. Background Technology

[0002] Currently, the most mature multi-target tracking method in the industrial field is based on a detection-plus-tracking paradigm, which is divided into two stages: the first stage is target detection, and the second stage is tracking and matching. The second stage has two key points: Kalman filtering (prediction) and linear matching. Kalman filtering is used to predict the state of the tracked target in the previous frame in the current frame; then, a cost matrix is ​​calculated based on the Kalman prediction result and the detection result of the current frame. Linear matching is the process of matching the Kalman prediction result with the detection result of the current frame based on the cost matrix.

[0003] However, when a target enters the tracking area at a relatively fast speed, because detection occurs every frame, the target gradually enters the tracking area. In the first few frames, the target is not detected to have fully entered the tracking area; for example, only 1 / 2 or 1 / 3 of it enters the target area. At this time, the target aspect ratio is not the true target state. Therefore, during this period, the aspect ratio error of the tracked target predicted by Kalman Array is relatively large, resulting in lower target tracking accuracy. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a target tracking method, apparatus, electronic device, and storage medium to solve the problem of low accuracy in target tracking.

[0005] According to a first aspect, embodiments of the present invention provide a target tracking method, comprising:

[0006] Obtain the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame;

[0007] Based on the state parameters of the previous frame, the state of the target in the current frame is predicted to obtain the state prediction result of the target.

[0008] The state prediction result is updated based on the detection confidence level to determine the target state prediction result of the predicted target;

[0009] Target tracking is performed based on the location of the detected target and the target state prediction result of the predicted target, and the tracking result is determined.

[0010] The target tracking method provided in this invention reduces prediction errors by incorporating the detection confidence level of the target into the state prediction results. This effectively solves the problem of inaccurate prediction caused by large changes in the target's width and height when the target enters the tracking area at a relatively high speed. Based on accurate prediction, the accuracy of the tracking results can be improved. Since a higher detection confidence level indicates a more realistic and reliable target during detection, the method updates the state prediction results using the target's detection confidence level.

[0011] In some implementations, updating the state prediction result based on the detection confidence level to determine the target state prediction result of the predicted target includes:

[0012] The first noise covariance matrix of the predicted target is updated using the detection confidence to reduce the value of the covariance in the first noise covariance matrix to determine the second noise covariance matrix, wherein the first noise covariance matrix is ​​obtained based on Kalman prediction;

[0013] Based on the second noise covariance matrix, a Kalman update is performed on the state prediction result to determine the target state prediction result of the predicted target.

[0014] The target tracking method provided in this invention addresses the issue that the target's high speed results in significant aspect ratio variations across multiple consecutive image frames, leading to highly inaccurate Kalman predictions of aspect ratio. Therefore, by combining the detection confidence level of the detected target with a Kalman update of the state prediction result, and leveraging the meaning of the detection confidence level—a higher confidence level indicates a more reliable target—the first noise covariance matrix is ​​updated based on the detection confidence level to reduce the covariance value within it. This improves the accuracy of the resulting second noise covariance matrix, thereby reducing the errors introduced during the prediction process.

[0015] In some implementations, updating the first noise covariance matrix of the predicted target using the detection confidence and determining the second noise covariance matrix includes:

[0016] Obtain the first noise covariance matrix of the predicted target;

[0017] The update coefficient is determined based on the detection confidence level, wherein the detection confidence level is negatively correlated with the update coefficient.

[0018] The second noise covariance matrix is ​​determined based on the product of the update coefficients and the first noise covariance matrix.

[0019] The target tracking method provided in this invention uses the difference between 1 and the target detection confidence to determine the update coefficient. Therefore, the higher the detection confidence, the smaller the update coefficient, which can reduce the value of the first noise covariance matrix more significantly; conversely, the lower the detection confidence, the larger the update coefficient, which can reduce the value of the first noise covariance matrix less significantly. This processing ensures that the noise covariance matrix and the detection confidence are closely linked, guaranteeing the accuracy of the second noise covariance matrix used for subsequent Kalman updates.

[0020] In some implementations, the step of predicting the state of the target in the current frame based on the state parameters of the previous frame to obtain the state prediction result of the target includes:

[0021] When there is a lost tracking target in the previous frame of the current frame, the historical state parameters of the lost tracking target in multiple historical frames are obtained to obtain multiple historical state parameters.

[0022] The multiple historical state parameters are fused to determine the state prediction result of the lost tracking target in the current frame.

[0023] The target tracking method provided in this invention addresses the issue that, during target tracking, if target occlusion exists, the unoccluded portion of the tracked target remains essentially unchanged across several consecutive frames. This minimal change in positional state makes the tracked target undetectable, meaning its potential location cannot be predicted. Therefore, for lost tracked targets, the method combines historical state parameters from multiple historical frames to obtain a state prediction result for the target in the current frame, thus resolving the prediction inaccuracy caused by gradual target occlusion.

[0024] In some implementations, the step of fusing the plurality of historical state parameters to determine the state prediction result of the lost tracking target in the current frame includes:

[0025] Obtain the historical detection confidence of the lost tracking target in the multiple historical frames to obtain multiple historical detection confidence scores;

[0026] Based on the fusion processing of the multiple historical detection confidence scores and the multiple historical state parameters, the state prediction result of the lost tracking target in the current frame is determined.

[0027] The target tracking method provided in this invention addresses the issue that the more severely the target is occluded, the lower the corresponding historical detection confidence level, and consequently, the smaller the proportion of its state parameters in historical frames. Therefore, fusing historical state parameters with historical detection confidence levels can improve the accuracy of state prediction results.

[0028] In some implementations, the process of fusing the multiple historical state parameters based on the multiple historical detection confidence levels to determine the state prediction result of the lost tracking target in the current frame includes:

[0029] The fusion coefficient is determined using the historical detection confidence level.

[0030] Based on the fusion coefficient, the historical state parameters are processed by exponential moving average to determine the state prediction result of the lost tracking target in the current frame.

[0031] The target tracking method provided in this embodiment of the invention fuses historical state parameters through exponential moving average, which is simple to process and also incorporates historical detection confidence during processing, thereby improving the accuracy of loading prediction results.

[0032] In some implementations, when a lost tracking target exists in the previous frame of the current frame, the historical state parameters of the lost tracking target in multiple historical frames are obtained, resulting in multiple historical state parameters, including:

[0033] Detect whether there is a missing flag bit in the previous frame;

[0034] When the loss flag bit exists in the previous frame, the lost tracking target is determined based on the loss flag bit;

[0035] The historical state parameters of the lost tracking target in multiple historical frames are obtained, resulting in multiple historical state parameters.

[0036] The target tracking method provided in this embodiment of the invention uses a loss flag to identify lost tracking targets, which simplifies the determination of lost tracking targets and improves the efficiency of target tracking.

[0037] According to a second aspect, embodiments of the present invention also provide a target tracking device, comprising:

[0038] The acquisition module is used to acquire the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame;

[0039] The prediction module is used to predict the state of the target in the current frame based on the state parameters of the previous frame, and obtain the state prediction result of the target.

[0040] The update module is used to update the state prediction result based on the detection confidence level, and determine the target state prediction result of the predicted target;

[0041] The tracking module is used to track the target based on the location of the detected target and the target state prediction result of the predicted target, and to determine the tracking result.

[0042] According to a third aspect, embodiments of the present invention provide an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the target tracking method described in the first aspect or any embodiment of the first aspect.

[0043] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing the computer to perform the target tracking method described in the first aspect or any embodiment of the first aspect.

[0044] It should be noted that the corresponding beneficial effects of the target tracking device, electronic device, and computer-readable storage medium provided in the embodiments of the present invention can be found in the description of the corresponding beneficial effects of the target tracking method above, and will not be repeated here. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figures 1a-1c This is a diagram illustrating the target entering the tracking area at a relatively high speed.

[0047] Figures 2a-2e This is a diagram illustrating the target being gradually obscured.

[0048] Figure 3 This is a flowchart of a target tracking method according to an embodiment of the present invention;

[0049] Figure 4 This is a flowchart of a target tracking method according to an embodiment of the present invention;

[0050] Figure 5 This is a flowchart of a target tracking method according to an embodiment of the present invention;

[0051] Figure 6 This is a structural block diagram of a target tracking device according to an embodiment of the present invention;

[0052] Figure 7This is a schematic diagram of the hardware structure of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention provides a target tracking method based on Kalman filtering. This method primarily improves upon Kalman filtering. In existing technologies, inaccurate detection results or target occlusion can lead to inaccurate Kalman filtering, resulting in inaccurate predictions and even loss of the tracked target.

[0055] The following description will describe the target tracking method from two aspects, namely, Figure 3 as well as Figure 4 The illustrated embodiment is described from the perspective of state update. Figure 5 The illustrated embodiment is described from the perspective of state prediction. That is, it is described in detail from the perspectives of state update and state prediction respectively. This is used to solve the problem of tracking inaccuracy in two situations: first, the tracking inaccuracy caused by the target entering the tracking area at a relatively fast speed; and second, the prediction inaccuracy caused by the target being occluded by an obstacle, i.e., due to a gradual occlusion process.

[0056] Specifically, regarding the first question, such as Figures 1a-1c As shown, when a target enters the tracking area from bottom to top at a relatively fast speed, since detection is performed in every frame, the target gradually enters the tracking area. In the first few frames, the target is not fully detected, with only 1 / 2 or 1 / 3 entering the target area. At this time, the target's aspect ratio is not the true target state, so the Kalman prediction of the tracked target's aspect ratio has a large error during this period. To solve this problem, this embodiment of the invention introduces a detection confidence parameter. The Kalman updates the tracking target state based on the [x, y, r, h] of the matched detection target bounding boxes, but different detection boxes have different confidence levels. The higher the detection confidence of the detected target, the more realistic and reliable the detection model considers the target to be, and vice versa. Therefore, introducing detection confidence during state updates can effectively utilize detection confidence and reduce errors.

[0057] Regarding the second question, such as Figures 2a-2eAs shown, during target tracking, such as a car traveling from right to left, the tracked vehicle is gradually obscured by a pillar. During this gradual process, the vehicle is actually moving forward, but because the front of the vehicle is obscured, the detection algorithm can only detect the unobstructed portion. In several consecutive frames, the position of the unobstructed portion of the tracked target remains essentially unchanged, causing the position state change parameter in the Kalman filter to become very small, even close to zero. Thus, when the tracked target cannot be detected, the Kalman filter cannot predict the possible position of the tracked target. When the target reappears, the detection box and the Kalman prediction result cannot match, causing a switch in the tracked target ID. To address this issue, a method is proposed that the most recent N frames of Kalman state parameters be retained. When the target is lost, the state parameters of these N frames can be fused to obtain a more realistic tracking state before the target was lost. The detection confidence of the tracked target is also incorporated into the fusion process. As mentioned above, the more severely the target is obscured, the lower the confidence of the detection box, and the less weight the tracking target's state parameters carry in these 10 frames. The final fused state parameters can effectively represent the state before the target was lost, and can effectively predict the target's position in the frames during which the target was lost. If the tracked target reappears, the prediction and detection results can be matched again, and the target ID will not be lost.

[0058] Since the target tracking method in this embodiment of the invention is related to Kalman filtering, Kalman filtering will be described first. The application of Kalman filtering in target tracking consists of two parts: state prediction and state update.

[0059] Specifically, the state prediction part can be represented by the following formula:

[0060] (1) State prediction involves calculating the state parameters for the current frame based on the optimal estimate from the previous frame.

[0061] in, This represents the optimal value of the previous frame. This represents the estimated value for the current frame.

[0062] (2) This represents the covariance of the previous frame. Let F represent the estimated covariance for this frame, Q represent the error matrix, and F represent the state transition matrix.

[0063] The input to the state update part is: the detection value z k ; Detection confidence level c k ; Predicted state Predicted noise covariance Observation model H kBecause the observed parameters differ from the system parameters, it is equivalent to a mapping matrix; the measurement noise covariance matrix R k .

[0064] The output is: Updated state estimate Update covariance

[0065] It is expressed using the following formula:

[0066] (1) Difference between detected and predicted values

[0067] (2) Prefit covariance residual S k

[0068] (3) Kalman gain K k

[0069] (4) Update state estimation

[0070] (5) Update estimated covariance

[0071] According to an embodiment of the present invention, a target tracking method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0072] This embodiment provides a target tracking method that can be used in electronic devices such as mobile terminals, servers, and computers. Figure 3 This is a flowchart of a target tracking method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0073] S11, obtain the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame.

[0074] For target tracking, the process involves analyzing and determining the image frames in the video to be analyzed. As mentioned above, the processing for target tracking in a single image frame includes predicting the position of the target that appeared in the previous frame in the current frame, detecting the target in the current frame, and finally fusing the predicted target with the detection module to determine the tracking result of the current frame.

[0075] The location of the detected target in the current frame is obtained after performing target detection on the current frame. Target detection is implemented based on a target detection algorithm, and the targets here include, but are not limited to, vehicles, pedestrians, etc., depending on the specific requirements. The detection confidence score refers to the confidence score of the detection box when detecting the target in the current frame.

[0076] It should be noted that there may be one, two, or more targets in the current frame; the number of targets is not limited here. If multiple targets exist, each target has a unique identifier ID. When tracking targets in the video to be analyzed, the identifier ID of the same target remains unchanged. The position and detection confidence of each target in each frame are obtained through the detection algorithm, and the target's identifier ID can be used for differentiation when saving the data.

[0077] The state parameters of the previous frame refer to the state parameters of the detected target, including but not limited to the target's center point, aspect ratio, etc. These state parameters are also stored and distinguished using an identifier (ID). The previous frame refers to the frame preceding the current frame; target prediction for the current frame is based on the state parameters of the previous frame.

[0078] S12, based on the state parameters of the previous frame, perform state prediction on the target in the current frame to obtain the state prediction result of the target.

[0079] The prediction objective is to predict the state of target A in the current frame by using the state parameters of target A in the previous frame, and obtain the state prediction result of target A. To distinguish it from the detection process, the predicted target is referred to as the predicted target.

[0080] State prediction is based on Kalman filtering. For example, as mentioned above, the state prediction result of the target is obtained by using the formula in the state prediction part.

[0081] S13, update the state prediction results based on the detection confidence to determine the target state prediction results of the predicted target.

[0082] In existing Kal filters, the state update part updates the state prediction result using the detection value. However, since the prediction becomes inaccurate when the target enters the tracking area at a high speed, this embodiment updates the state prediction result by combining the detection confidence of the detected target. Since the detection confidence represents the degree to which the detection algorithm considers the target to be real and reliable, updating the state prediction result by combining the detection confidence can ensure the accuracy of the target state prediction result.

[0083] The state update is based on the state update mechanism of Kalman filtering, and the detection confidence level is incorporated into the calculation formula. For example, when calculating the difference between the measured and observed values, the detection confidence level is used to increase or decrease the difference accordingly. Alternatively, the detection confidence level can also be incorporated into other formulas for the state update, and no restrictions are placed on this approach.

[0084] S14, perform target tracking based on the detected target's location and the predicted target's state, and determine the tracking result.

[0085] After obtaining the target state prediction result of the predicted target, the intersection-union ratio (IUGR) is calculated by combining the position information of the detected target to determine the tracking result. Alternatively, other methods can be used for target tracking, which are not limited here, as long as the target tracking is based on the position of the detected target and the target state prediction result of the predicted target.

[0086] The target tracking method provided in this embodiment reduces prediction errors by incorporating the detection confidence level of the target into the state prediction results. This effectively solves the problem of inaccurate prediction caused by large changes in the target's width and height when the target enters the tracking area at a relatively high speed. Based on accurate prediction, the accuracy of the tracking results can be improved. Since a higher detection confidence level indicates a more realistic and reliable target during detection, the method updates the state prediction results using the target's detection confidence level.

[0087] This embodiment provides a target tracking method that can be used in electronic devices such as mobile terminals, servers, and computers. Figure 4 This is a flowchart of a target tracking method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0088] S21, obtain the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame.

[0089] Please see details Figure 3 S11 of the illustrated embodiment will not be described again here.

[0090] S22, based on the state parameters of the previous frame, perform state prediction on the target in the current frame to obtain the state prediction result of the target.

[0091] The state prediction is obtained through Kalman prediction. For the specific processing steps, please refer to the relevant formulas for state prediction described above.

[0092] For further details, please see Figure 3 S12 of the illustrated embodiment will not be described again here.

[0093] S23, update the state prediction results based on the detection confidence level, and determine the target state prediction results of the predicted target.

[0094] Specifically, S23 above includes:

[0095] S231, update the first noise covariance matrix of the predicted target using the detection confidence to reduce the value of the covariance in the first noise covariance matrix and determine the second noise covariance matrix.

[0096] The first noise covariance matrix is ​​obtained based on Kalman prediction.

[0097] The first noise covariance matrix is ​​R in the state update formula above. k Using detection confidence level to evaluate R k Update to reduce R k The magnitude of the covariance value yields the second noise covariance matrix. Noise is essential for target tracking and should be minimized. The magnitude of the first noise covariance matrix is ​​determined by the predicted aspect ratio of the target. Based on the analysis of the first problem mentioned above, the high speed of the target movement can lead to inaccurate aspect ratio predictions. Therefore, the first noise covariance matrix based on the aspect ratio needs to be updated. This update incorporates the detection confidence level. Since the detection confidence level reflects the reliability of the target, updating the first noise covariance matrix using the detection confidence level reduces the covariance value within it, thereby mitigating the prediction error caused by inaccurate aspect ratio predictions.

[0098] In some embodiments, S231 includes:

[0099] (1) Obtain the first noise covariance matrix of the target to be predicted.

[0100] (2) Determine the update coefficient based on the difference in the detection confidence, wherein the detection confidence is negatively correlated with the update coefficient.

[0101] (3) Determine the second noise covariance matrix based on the product of the update coefficients and the first noise covariance matrix.

[0102] The detection confidence level is negatively correlated with the update coefficient, meaning that the higher the detection confidence level, the smaller the update coefficient; conversely, the lower the detection confidence level, the larger the update coefficient. Specifically, a higher detection confidence level indicates higher target reliability, and correspondingly, a more reliable aspect ratio prediction, leading to the use of a smaller update coefficient to update the first noise covariance matrix. Conversely, a lower detection confidence level indicates lower target reliability, and correspondingly, a less reliable aspect ratio prediction, leading to the use of a larger update coefficient to update the first noise covariance matrix.

[0103] Specifically, the second noise covariance matrix is ​​calculated using the following formula.

[0104]

[0105] Among them, c k The detection confidence of target k is 0. <c k<1 , (1-c k ) represents the update coefficient.

[0106] The update coefficients are determined using the difference between 1 and the detection confidence level. A higher detection confidence level results in a smaller update coefficient, thus reducing the covariance value in the first noise covariance matrix more significantly. Conversely, a lower detection confidence level results in a larger update coefficient, reducing the value of the first noise covariance matrix less. This approach tightly integrates the noise covariance matrix with the detection confidence level, ensuring the accuracy of the second noise covariance matrix used for subsequent Kalman updates.

[0107] S232, perform Kalman update on the state prediction results based on the second noise covariance matrix to determine the target state prediction results of the predicted target.

[0108] After obtaining the second noise covariance matrix Then, based on the state update formula described above, a Kalman update is performed on the state prediction results, that is, the state parameters are updated. and covariance Specifically, the following formula is used for calculation:

[0109] (1) Prefit covariance residual S k

[0110] (2) Kalman gain K k

[0111] (3) Update status parameters

[0112] (4) Update estimated covariance

[0113] S24, Based on the detected target's location and the predicted target's state, perform target tracking and determine the tracking result.

[0114] Please see details Figure 3 S14 of the illustrated embodiment will not be described again here.

[0115] The target tracking method provided in this embodiment suffers from significant errors due to the high speed of the target and the large changes in its aspect ratio across multiple consecutive image frames. This leads to highly inaccurate Kalman predictions of the aspect ratio. Therefore, by combining the detection confidence of the detected target with the Kalman update of the state prediction results, and leveraging the meaning of the detection confidence—that is, the higher the confidence, the more reliable the target—the error introduced by the prediction process can be reduced.

[0116] This embodiment provides a target tracking method that can be used in electronic devices such as mobile terminals, servers, and computers. Figure 5 This is a flowchart of a target tracking method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0117] S31, obtain the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame.

[0118] Please see details Figure 3 S11 of the illustrated embodiment will not be described again here.

[0119] S32, based on the state parameters of the previous frame, perform state prediction on the target in the current frame to obtain the state prediction result of the target.

[0120] Specifically, S32 includes:

[0121] S321, when there is a lost tracking target in the previous frame of the current frame, obtain the historical state parameters of the lost tracking target in multiple historical frames to obtain multiple historical state parameters.

[0122] Regarding the second problem mentioned above, due to gradual occlusion causing the position state change output during Kalman prediction to become very small, even close to zero, the Kalman filter cannot predict the possible position of the tracked target. Therefore, in this embodiment, for a lost tracked target, the state prediction result of the lost tracked target in the current frame is determined by combining historical state parameters from multiple historical frames.

[0123] Multiple historical frames refer to the N historical frames preceding the current frame. The specific value of N is set according to actual needs. For example, N=10 or N=15. These N historical frames can be N consecutive historical frames preceding the current frame, or N historical frames obtained by extracting one historical frame every n frames preceding the current frame, etc. There are no restrictions on the method of obtaining multiple historical frames; the specific setting depends on actual needs. It should be noted that to ensure accuracy, the value of n should not be too large.

[0124] As mentioned above, the state parameters of each target in each frame are stored and distinguished by the target's identifier ID. For the same target, there is a set of state parameters in each historical frame; correspondingly, N historical frames can yield N sets of historical state parameters. These historical state parameters include, but are not limited to, the target's position, area s, and aspect ratio r in the historical frames.

[0125] S322, fuses multiple historical state parameters to determine the state prediction result of the lost tracking target in the current frame.

[0126] The fusion of multiple historical state parameters includes, but is not limited to, smoothing or weighted processing, etc., which are set according to actual needs.

[0127] In some implementations, the historical detection confidence levels are fused together. Specifically, S322 includes:

[0128] (1) Obtain the historical detection confidence of the lost tracking target in multiple historical frames to obtain multiple historical detection confidence.

[0129] (2) Based on the fusion processing of multiple historical state parameters using multiple historical detection confidence scores, the state prediction result of the lost tracking target in the current frame is determined.

[0130] As mentioned above, each target has a unique identifier ID, which can be used to extract its historical detection confidence across multiple historical frames. The historical detection confidence corresponds to the historical state parameters; each target in each frame has both detection confidence and state parameters. Therefore, during the fusion process, the detection confidence can be used as a weight to perform a weighted average of the corresponding state parameters, determining the predicted state of the lost tracked target in the current frame.

[0131] The more severely the target is occluded, the greater the error in the width and height of the corresponding detection box compared to the normal width and height, resulting in a lower historical detection confidence level. Consequently, its state parameters carry less weight in historical frames. Therefore, fusing historical state parameters with historical detection confidence levels can improve the accuracy of state prediction results.

[0132] In some implementations, step (2) of S322 above includes:

[0133] 2.1) Determine the fusion coefficient using historical detection confidence.

[0134] 2.2) Based on the fusion coefficient, the historical state parameters are processed by exponential moving average to determine the state prediction result of the lost tracking target in the current frame.

[0135] For example, multiple historical frames are 10 historical frames. The historical detection confidence and historical state parameters of these 10 historical frames are obtained, where the historical detection confidence C... i Represented as C i i = 1, 2, 3...10, historical state parameters Represented as

[0136] The following formula is used for fusion processing to determine the state prediction result of the lost tracking target in the current frame:

[0137] (1) It is a constant, for example, set to 0.6;

[0138] (2)

[0139] (3)

[0140] Where x = [x, y, s, r, Δx, Δy, Δs, Δr] represents the center point (x, y) of the tracked target, the area s, the aspect ratio r, and Δ represents the corresponding rate of change. In this process, only six values ​​are updated: [s, r, Δx, Δy, Δs, Δr]. F represents the state transition matrix, which transfers the values ​​from the previous time step to the current time step.

[0141] By fusing historical state parameters through exponential moving averages, the processing method is simple, and the process also incorporates historical detection confidence, which can improve the accuracy of loading prediction results.

[0142] In some embodiments, step (1) of S322 above includes:

[0143] 1.1) Detect whether there is a missing flag bit in the previous frame.

[0144] 1.2) If a loss flag exists in the previous frame, determine the lost tracking target based on the loss flag.

[0145] 1.3) Obtain the historical state parameters of the lost tracking target in multiple historical frames to obtain multiple historical state parameters.

[0146] In the previous frame, a loss flag was set for undetected targets. This loss flag corresponds to the target's identifier ID and is used to record which target was lost during tracking. That is, when a loss flag exists, the lost tracking target can be identified using this flag. Then, using the identifier ID of the lost tracking target, its historical state parameters in multiple historical frames can be extracted.

[0147] Using a loss flag to identify lost tracking targets simplifies the identification of lost tracking targets and improves the efficiency of target tracking.

[0148] If there are no lost tracking targets, the state prediction is still based on the Kalman prediction formula.

[0149] S33, update the state prediction results based on the detection confidence to determine the target state prediction results of the predicted target.

[0150] Please see details Figure 4 S23 of the illustrated embodiment will not be described again here.

[0151] S34, Based on the detected target's position and the predicted target's state, perform target tracking and determine the tracking result.

[0152] Please see details Figure 3 S14 of the illustrated embodiment will not be described again here.

[0153] The target tracking method provided in this embodiment addresses the issue that if target occlusion exists during tracking, the unoccluded portion of the tracked target remains largely unchanged across several consecutive frames, resulting in minimal positional changes and making the target undetectable, i.e., its possible location unpredictable. Therefore, for lost targets, the method combines historical state parameters from multiple historical frames to obtain its state prediction in the current frame, resolving the prediction inaccuracies caused by gradual target occlusion. Furthermore, since this method can obtain the state prediction of lost targets in the current frame, it also solves the problem of mismatch between the detection box and Kalman prediction results when the target reappears, necessitating target ID switching. This ensures real-time, continuous prediction, where the error between the prediction and detection box is minimal when the target reappears, allowing for a match without requiring target ID switching.

[0154] This embodiment also provides a target tracking device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0155] This embodiment provides a target tracking device, such as Figure 6 As shown, it includes:

[0156] The acquisition module 41 is used to acquire the position of the detected target in the current frame, the detection confidence of the detected target, and the state parameters of the previous frame;

[0157] Prediction module 42 is used to predict the state of the target in the current frame based on the state parameters of the previous frame, and obtain the state prediction result of the target.

[0158] Update module 43 is used to update the state prediction result based on the detection confidence level, and determine the target state prediction result of the predicted target;

[0159] The tracking module 44 is used to track the target based on the position of the detected target and the target state prediction result of the predicted target, and determine the tracking result.

[0160] In some implementations, the update module 43 includes:

[0161] The first update unit is used to update the first noise covariance matrix of the predicted target using the detection confidence, so as to reduce the value of the covariance in the first noise covariance matrix to determine the second noise covariance matrix, wherein the first noise covariance matrix is ​​obtained based on Kalman prediction;

[0162] The second update unit is used to perform a Kalman update on the state prediction result based on the second noise covariance matrix to determine the target state prediction result of the predicted target.

[0163] In some implementations, the first update unit includes:

[0164] The first acquisition subunit is used to acquire the first noise covariance matrix of the predicted target;

[0165] A calculation subunit is used to determine an update coefficient based on the difference in the detection confidence levels, wherein the detection confidence levels are negatively correlated with the update coefficients.

[0166] The first determining subunit is used to determine the second noise covariance matrix based on the product of the update coefficient and the first noise covariance matrix.

[0167] In some implementations, the prediction module 42 includes:

[0168] The acquisition unit is used to acquire the historical state parameters of the lost tracking target in multiple historical frames when there is a lost tracking target in the previous frame of the current frame, and obtain multiple historical state parameters.

[0169] The fusion unit is used to perform fusion processing on the multiple historical state parameters to determine the state prediction result of the lost tracking target in the current frame.

[0170] In some implementations, the fusion unit includes:

[0171] The second acquisition subunit is used to acquire the historical detection confidence of the lost tracking target in the multiple historical frames, and obtain multiple historical detection confidences;

[0172] The fusion subunit is used to perform fusion processing on the multiple historical state parameters based on the multiple historical detection confidence levels, and to determine the state prediction result of the lost tracking target in the current frame.

[0173] In some implementations, the fusion subunit includes:

[0174] The second determining subunit is used to determine the fusion coefficient using the historical detection confidence level;

[0175] The third determining subunit is used to perform exponential moving average processing on the historical state parameters based on the fusion coefficient to determine the state prediction result of the lost tracking target in the current frame.

[0176] In some implementations, the acquisition unit includes:

[0177] The detection subunit is used to detect whether a missing flag bit exists in the previous frame;

[0178] The fourth determining subunit is used to determine the lost tracking target based on the loss flag bit when the loss flag bit exists in the previous frame;

[0179] The third acquisition subunit is used to acquire the historical state parameters of the lost tracking target in multiple historical frames, thereby obtaining multiple historical state parameters.

[0180] In this embodiment, the target tracking device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0181] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0182] This invention also provides an electronic device having the above-described features. Figure 6 The target tracking device shown.

[0183] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention, such as... Figure 7As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, memory 54, and at least one communication bus 52. The communication bus 52 is used to enable communication between these components. The communication interface 53 may include a display screen or a keyboard; optionally, the communication interface 53 may also include a standard wired interface or a wireless interface. The memory 54 may be high-speed RAM (Random Access Memory) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 54 may also be at least one storage device located remotely from the aforementioned processor 51. The processor 51 may be combined with... Figure 6 The described apparatus has an application program stored in memory 54, and the processor 51 calls the program code stored in memory 54 to perform any of the above method steps.

[0184] The communication bus 52 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0185] The memory 54 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 54 may also include a combination of the above types of memory.

[0186] The processor 51 can be a central processing unit (CPU), a network processor (NP), or a combination of CPU and NP.

[0187] The processor 51 may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0188] Optionally, memory 54 is also used to store program instructions. Processor 51 can invoke program instructions to implement the target tracking method as shown in any embodiment of this application.

[0189] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the target tracking method in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0190] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A target tracking method characterized by, The method comprises the following steps: obtaining the position of a detected target in a current frame, the detection confidence of the detected target, and the state parameter of a previous frame; performing state prediction on a predicted target in the current frame based on the state parameter of the previous frame to obtain a state prediction result of the predicted target; updating the state prediction result based on the detection confidence to determine a target state prediction result of the predicted target; performing target tracking according to the position of the detected target and the target state prediction result of the predicted target to determine a tracking result; the step of updating the state prediction result based on the detection confidence to determine the target state prediction result of the predicted target comprises the following steps: updating a first noise covariance matrix of the predicted target using the detection confidence to reduce the value of the covariance in the first noise covariance matrix to determine a second noise covariance matrix, wherein the first noise covariance matrix is obtained based on Kalman prediction; performing Kalman update on the state prediction result based on the second noise covariance matrix to determine the target state prediction result of the predicted target; the step of updating the first noise covariance matrix of the predicted target using the detection confidence to determine the second noise covariance matrix comprises the following steps: obtaining the first noise covariance matrix of the predicted target; determining an update coefficient according to the detection confidence, wherein the detection confidence is negatively correlated with the update coefficient; determining the second noise covariance matrix based on the product of the update coefficient and the first noise covariance matrix.

2. The method of claim 1, wherein, the step of performing state prediction on the predicted target in the current frame based on the state parameter of the previous frame to obtain the state prediction result of the predicted target comprises the following steps: when there is a lost tracking target in the previous frame of the current frame, obtaining historical state parameters of the lost tracking target in a plurality of historical frames to obtain a plurality of historical state parameters; performing fusion processing on the plurality of historical state parameters to determine the state prediction result of the lost tracking target in the current frame.

3. The method of claim 2, wherein, the step of performing fusion processing on the plurality of historical state parameters to determine the state prediction result of the lost tracking target in the current frame comprises the following steps: obtaining historical detection confidences of the lost tracking target in the plurality of historical frames to obtain a plurality of historical detection confidences; performing fusion processing on the plurality of historical state parameters based on the plurality of historical detection confidences to determine the state prediction result of the lost tracking target in the current frame.

4. The method of claim 3, wherein, the step of performing fusion processing on the plurality of historical state parameters based on the plurality of historical detection confidences to determine the state prediction result of the lost tracking target in the current frame comprises the following steps: determining a fusion coefficient using the historical detection confidences; performing exponential moving average processing on the historical state parameters based on the fusion coefficient to determine the state prediction result of the lost tracking target in the current frame.

5. The method of claim 2, wherein, the step of obtaining the historical state parameters of the lost tracking target in a plurality of historical frames to obtain a plurality of historical state parameters when there is a lost tracking target in the previous frame of the current frame comprises the following steps: detecting whether there is a loss flag in the previous frame; when the loss flag in the previous frame exists, determining the lost tracking target based on the loss flag; obtaining historical state parameters of the lost tracking target in a plurality of historical frames, to obtain a plurality of historical state parameters.

6. A target tracking device, characterized by, comprising: an acquisition module, configured to acquire a position of a detection target in a current frame, a detection confidence of the detection target, and a state parameter of a previous frame; a prediction module, configured to perform state prediction on a prediction target in the current frame based on the state parameter of the previous frame, to obtain a state prediction result of the prediction target; an update module, configured to update the state prediction result based on the detection confidence, to determine a target state prediction result of the prediction target; a tracking module, configured to perform target tracking according to the position of the detection target and the target state prediction result of the prediction target, to determine a tracking result; the updating of the state prediction result based on the detection confidence to determine the target state prediction result of the prediction target comprises: updating a first noise covariance matrix of the prediction target by using the detection confidence, to determine a second noise covariance matrix by reducing a value of a covariance in the first noise covariance matrix, the first noise covariance matrix being obtained based on Kalman prediction; performing Kalman update on the state prediction result based on the second noise covariance matrix, to determine the target state prediction result of the prediction target; the updating of the first noise covariance matrix of the prediction target by using the detection confidence to determine the second noise covariance matrix comprises: acquiring the first noise covariance matrix of the prediction target; determining an update coefficient according to the detection confidence, the detection confidence being negatively correlated with the update coefficient; determining the second noise covariance matrix based on a product of the update coefficient and the first noise covariance matrix.

7. An electronic device, comprising: comprising: a memory and a processor, which are connected in communication with each other, the memory storing computer instructions, and the processor performing the target tracking method in any one of claims 1-5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to perform the target tracking method in any one of claims 1-5.

Images