Target frame automatic tracking drift judgment method and device, equipment and storage medium

Abstract

The invention discloses a target frame automatic tracking drift judgment method and device, equipment and a storage medium, and relates to the field of image recognition. Extracting a target frame region of two adjacent frames of images, and converting the target frame region into a grey-scale map; angular point detection is carried out on the previous frame grey-scale map, and a first angular point group is extracted; performing optical flow calculation on the current frame grey-scale map based on the previous frame grey-scale map and the first angular point group, and matching a second angular point group; based on the distance and angle information of the successfully matched feature angular point pairs in the angular point group, judging the drift state of the tracking target corresponding to the current frame image; and when it is judged that the tracking target drifts, predicting the target position in the current frame image based on Kalman filtering and re-identifying the tracking target, and determining the real target position in the current frame image by combining a re-identification result and a response diagram obtained after correlation calculation of the tracking template. According to the scheme, feature level drift judgment is realized through corner detection and optical flow matching, effective re-identification is carried out, and the problem of target frame drift is solved.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a method, device, equipment and storage medium for automatically tracking and determining drift of a target frame. Background Art

[0002] Visual tracking algorithms based on correlation filtering are widely used on embedded platforms due to their excellent positioning accuracy and computational efficiency. During tracking, features in the search area of ​​the current frame are extracted and correlated with template features to generate a response map to locate the target's new position. However, due to background interference, occlusion, and other factors, the maximum response generated may deviate from the target's true position, resulting in the current frame's target frame not fully enclosing the target, a phenomenon known as drift.

[0003] Currently, most of the methods used to reduce drift are as follows: 1) Use neural networks to extract features of the template and search area to improve robustness and reduce interference; 2) Determine drift based on the target frame area information. For example, the skewness of the response image is defined in patent CN112509003B, "Method and System for Solving Target Tracking Frame Drift," to determine whether it meets preset conditions. If so, the image within the tracking frame is converted into a binary image, and the center of mass of the region is calculated. The deviation between the center of mass and the target frame center inferred by the tracking algorithm is then used to determine whether the tracking frame should be corrected.

[0004] The above two solutions have the following defects: 1) Neural network methods often require targeted training in different scenarios to obtain appropriate network weights, which is a lot of work; secondly, neural networks are computationally intensive, and deployment on embedded platforms requires sufficient computing power.

[0005] 2) The response map indirectly calculated by correlation filtering is easily affected by background interference, and it is inevitable that there will be multi-peaked or diffuse response maps without drift, resulting in excessive skewness and other indicators. The accuracy of the subsequent binarization centroid calculation is heavily dependent on whether the foreground can be correctly segmented. Summary of the Invention

[0006] The present application provides a method, device, equipment and storage medium for automatically tracking and drifting judgment of a target frame, which has the advantages of improving the accuracy and stability of target tracking and effectively solving the problem of target frame drift.

[0007] On the one hand, the present application provides a method for automatically tracking a target frame and determining drift, the method comprising: Cache two adjacent frames of images and extract the target frame area in the image, and convert the target frame area image into a grayscale image; Perform corner point detection on the previous frame grayscale image to extract the first corner point group within the target frame area; perform optical flow calculation on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and match the second corner point group of the current frame grayscale image; the corner point group contains the target feature corner points of the corresponding grayscale image; Based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group, the drift state of the tracking target corresponding to the current frame image is determined; When it is determined that the tracked target drifts, the target position in the current frame image is predicted based on the Kalman filter and the tracked target is re-identified. The response map after the re-identification result and the tracking template are combined to determine the true target position in the current frame image.

[0008] Specifically, performing corner point detection on the grayscale image of the previous frame to extract the first corner point group within the target frame area includes: Extract the previous frame grayscale image Based on the characteristic value of each pixel, the characteristic corner response value and corner screening threshold of the current frame grayscale image at each pixel are obtained. The formula is as follows:

[0009]

[0010] Among them and Indicates The eigenvalue at the pixel point, Indicates The minimum eigenvalue at the pixel coordinate, that is The characteristic corner response value of the pixel point; Indicates the quality level parameter to be set, Represents the grayscale image of the previous frame The maximum characteristic corner response value in , T represents the corner screening threshold; Filter out the previous frame grayscale image All candidate corner points whose response values ​​are greater than T; Sort the candidate corner points from large to small according to the response value to form a sequence {p1, p2, p3…pi}; Starting from the first candidate corner point p1 in the sequence, calculate the distance from all other candidate feature corner points to p1 d , delete the sequences whose distance is less than the threshold Then, the second candidate corner point is selected, and the distance from the remaining candidate corner points whose response values ​​are smaller than the second candidate corner point to the second candidate corner point is calculated, and so on, the target feature corner point among the candidate feature corner points is determined; all the target feature corner points constitute the first corner point group .

[0011] Specifically, determining the drift state of the tracking target corresponding to the current frame grayscale image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group includes: Determining the total number of characteristic corner points and the number of successful pairings in the first corner point group, and determining a drift state of the tracked target based on the number of successful pairings; wherein, when the number of successful pairings does not exceed the target ratio of the total number of characteristic corner points, it is determined that the tracked target has drifted; When the number of successful pairings exceeds the target ratio of the total number of feature corner points, the distance and angle information values ​​of the feature corner point pairs in the current frame grayscale image and the previous frame grayscale image are calculated; Sort the distance and angle information values ​​of the feature corner pairs in ascending order, select the target ratio data before and after the median, and form the distance array respectively and the angle array ; Based on distance array and the angle array Determine the drift state of the tracked target.

[0012] Specifically, the distance array and the angle array Determine the drift status of the tracked target, including: Calculate the distance array separately and the angle array The mean and standard deviation of If the distance to the mean or standard deviation Exceeds the preset ratio of the target frame or the angle standard deviation Exceeding the threshold , then the drift is determined If true, re-identification is performed; the judgment formula is as follows:

[0013] Among them and Represent the width and height of the target box respectively, and To set the scale; Represents a Boolean operation that determines whether drift is true.

[0014] Specifically, the method of predicting the target position in the grayscale image of the current frame based on Kalman filtering and performing tracking target re-identification includes: When the target frame is confirmed to have drifted, the tracked target is re-identified; the re-identification result re-determines the target frame area and determines the center position as ; Predict the position of the tracking target in the current frame based on Kalman filtering , based on the center position of re-identification and the predicted tracking target position , determine the target distance between the two ; The calculation process is as follows:

[0015] Among them Indicates square root calculation.

[0016] Specifically, combining the re-identification result and the relevant calculated response value of the tracking template to determine the true target position in the current frame image includes: The newly determined target box area is correlated with the tracking template before re-identification to obtain the respective maximum response values ; means as follows:

[0017] Calculate the maximum response value of each Distance to the target Ratio ; Set the maximum ratio The corresponding recognition result is determined as the true target position; the calculation process is as follows:

[0018] Among them and Indicates the The center pixel coordinates of the re-identification results; Representation sample With template The Fourier transform of the kernel correlation between represents element-wise multiplication, is the Fourier transform of the trained filter coefficients, represents the inverse Fourier transform.

[0019] On the other hand, the present application provides a device for automatically tracking and determining drift of a target frame, the device comprising: A conversion module is used to cache two adjacent frames of images and extract the target frame area in the image, and convert the target frame area image into a grayscale image; A corner point calculation module is used to perform corner point detection on the grayscale image of the previous frame and extract a first corner point group within the target frame area; perform optical flow calculation on the grayscale image of the current frame based on the grayscale image of the previous frame and the first corner point group, and match the second corner point group of the grayscale image of the current frame; the corner point group contains the target feature corner points of the corresponding grayscale image; A drift determination module is used to determine the drift state of the tracking target corresponding to the current frame grayscale image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group; The tracking update module is used to predict the target position in the current frame grayscale image based on the Kalman filter and perform tracking target re-identification. It combines the re-identification result and the response map after correlation calculation with the tracking template to determine the true target position in the current frame image.

[0020] On the other hand, the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the target frame automatic tracking drift judgment method described in the above aspect.

[0021] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the target frame automatic tracking drift judgment method described in the above aspect.

[0022] The technical solutions provided by the embodiments of the present application have the following beneficial effects: feature-level drift determination is achieved through corner detection and optical flow matching, and the Kalman filter prediction and re-identification mechanism is combined to effectively distinguish between real drift and interference artifacts, thereby improving the accuracy and stability of target tracking while ensuring computational efficiency. This has the advantages of improving the accuracy and stability of target tracking and effectively solving the problem of target frame drift. 1) It does not require annotation training for different scenarios and has low computing power requirements, taking 13ms on a Hi3559_AV100 A73-core CPU. 2) Directly utilize the appearance information of the target area in two adjacent frames to reduce the error of indirect process calculation; 3) Combine the similarity with the template and the prediction results to filter out the real target. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for automatically tracking and determining drift of a target frame provided in an embodiment of the present application; Figure 2 A schematic diagram of extracting and converting the target frame tracked in the frame image into a grayscale image is shown; Figure 3 Schematic diagram of the positions of the drone and the target frame in the current frame image shown; Figure 4 The algorithm flow chart for extracting the first corner point group is listed; Figure 5 A flow chart for determining the drift state of the target tracked in the current frame grayscale image is shown; Figure 6 The following is a structural block diagram of a target frame automatic tracking drift judgment device provided by an embodiment of the present application; Figure 7 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0025] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0026] In traditional visual tracking algorithms based on correlation filtering, target frame drift is primarily caused by response map interference and insufficient segmentation accuracy. Existing methods rely on morphological features of the response map or binary centroid correction mechanisms, which can easily lead to misjudgment when the target is moving rapidly or in the presence of complex background interference. Multimodality in the response map renders the skewness metric ineffective, and binary segmentation is significantly affected by illumination variations, making it difficult to accurately extract the centroid of the foreground region. These flaws directly impact target positioning accuracy, causing the tracking frame to deviate from its true position and reducing system robustness.

[0027] When faced with the above problems, this application first considers how to improve the accuracy of drift judgment without relying on complex calculations of neural networks. Traditional methods have inherent defects through response graph morphological analysis or binarization centroid correction. The former is easily affected by multi-peak interference, and the latter relies on precise segmentation. To this end, this application attempts to start from the stability of the apparent features of the target area and explore a feature matching mechanism based on spatiotemporal continuity. By analyzing the distribution pattern of corner points in the target frame area of ​​adjacent frames, it is found that the relative position and direction of the corner point group during the target movement have spatiotemporal consistency characteristics. Compared with the global response map, local corner point matching can not only reflect the structural changes of the target itself, but also avoid background noise interference. Further considering the high efficiency of the optical flow method in motion estimation, a dual verification mechanism is constructed in combination with Kalman filter prediction to form a solution that can perform real-time calculations and has anti-interference capabilities.

[0028] Figure 1 Flowchart of the target frame automatic tracking drift determination method provided by an embodiment of the present application, including the following steps: S1, cache two adjacent frames of images and extract the target frame area in the image, and convert the target frame area image into a grayscale image; Caching two adjacent frames of images refers to temporarily storing the image data collected at the current moment and the previous moment in a storage medium. This can be achieved using memory or cache. Extracting the target frame area from the image refers to intercepting the rectangular area containing the tracked object from the complete image. This can be achieved using coordinate interception or image segmentation algorithms, focusing on the local features of the target to reduce background interference. Converting the target frame area image to a grayscale image refers to converting the color image into a single-channel grayscale image. This can be achieved using weighted averaging or brightness extraction algorithms, reducing the complexity of subsequent calculations while preserving the target structural features.

[0029] Figure 2 This diagram shows how to extract and convert a tracked target frame from a frame image into a grayscale image. The left side shows a normal cached frame image, and the right side shows a zoomed-in, converted grayscale image of the target frame area. Two adjacent frames are converted into the grayscale image of the previous frame and the current frame, respectively.

[0030] S2. Perform corner detection on the grayscale image of the previous frame to extract the first corner point group within the target frame area; perform optical flow calculation on the grayscale image of the current frame based on the grayscale image of the previous frame and the first corner point group, and match the second corner point group of the grayscale image of the current frame; Corner detection on the previous frame's grayscale image involves using a feature point detection algorithm to identify pixels with significant gradient changes within the image, known as target feature corners. This can be achieved using the Harris corner detection or Shi-Tomasi algorithm to extract key structural features within the target area. The first corner point cluster is a set of feature points with high response values ​​retained by a screening algorithm. This can be achieved using non-maximum suppression or threshold filtering to ensure a uniform and representative distribution of feature points.

[0031] Corner point extraction for the current frame also aims to select a set of feature points with high response values. However, because the image is composed of two consecutive frames, optical flow matching is used to select the corner points of the current frame. This second corner point group uses pixel motion vectors between adjacent frames to correlate feature points. Specifically, this can be achieved using the Lucas-Kanade or Farneback optical flow algorithms, establishing spatiotemporal correspondences between feature points to reflect the target's motion trajectory.

[0032] S3, based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group, determining the drift state of the tracking target corresponding to the current frame image; Determining the drift state means judging whether the target has deviated from the expected position by counting the motion parameters of the matching feature points in the target frame area of ​​the two frame images. Specifically, this can be achieved by combining the distance mean and standard deviation calculation with the angle variance analysis to quantify the degree of target motion abnormality to avoid subjective threshold setting errors.

[0033] S4. When it is determined that the tracked target drifts, the target position in the current frame image is predicted based on the Kalman filter and the tracked target is re-identified. The response map after the re-identification result and the tracking template are combined to determine the true target position in the current frame image. Determining that the tracking target drifts refers to the target frame tracking failure or offset, such as Figure 3 The positions of the drone and the target box in the current frame are shown. When drift occurs, the target box does not accurately mark the location of the tracked target. Kalman filtering predicts target position using a state estimation model. This is achieved using linear dynamic system modeling and covariance updates to compensate for tracking position deviations caused by occlusion or interference. Figure 3 The marked points in the target box represent the current frame position coordinates predicted based on the Mann filter.

[0034] Tracked target re-identification involves relocating the target within the search area using a feature matching algorithm. This can be achieved using correlation filtering or template matching algorithms. The predicted position is combined with the predicted location to narrow the search range and improve re-identification efficiency. After target re-identification, correlation calculations are performed between the multiple candidate regions identified during re-identification and the pre-reidentification tracking template to generate a frequency domain response map. A similarity metric is then used to assess the degree of match between the candidate regions and the target template, ultimately determining the target bounding box (i.e., the target location). This step can be implemented using normalized cross-correlation or convolution response calculations to quantify the confidence of the recognition results and aid decision-making.

[0035] In a possible implementation, the solution can be implemented through the following steps: First, a continuous video frame sequence is acquired. Two adjacent frames are cached and the target frame regions in these two frames are extracted. The extracted target frame region images are converted to grayscale images to simplify subsequent processing.

[0036] Next, corner detection is performed on the grayscale image of the previous frame. The Harris corner detection algorithm or another modified Shi-Tomasi algorithm is used to extract the first corner cluster within the target frame. Based on the grayscale image of the previous frame and the first corner cluster, optical flow is calculated for the grayscale image of the current frame. Feature point tracking is performed using the Lucas-Kanade optical flow method to match the second corner cluster in the grayscale image of the current frame. These two corner clusters contain the target feature corner points of the corresponding grayscale image.

[0037] Next, the successfully matched feature corner pairs within the corner point cluster are analyzed. The Euclidean distance and angle between these corner pairs are calculated. Based on the statistical information of these distances and angles, the drift state of the tracked target in the current frame's grayscale image is determined. For example, distance and angle thresholds can be set. When the distance or angle change of more than a certain percentage of corner pairs exceeds the threshold, drift is determined.

[0038] If the tracked target is determined to have drifted, a correction mechanism is activated. First, a Kalman filter is used to predict the target's position in the current frame based on historical trajectories. Simultaneously, target re-identification is performed, using methods such as template matching to relocate the target within the search area.

[0039] Finally, the re-identification result is correlated with the tracking template to generate a response map and calculate the response value. Combining the Kalman filter predicted position, the re-identification result position, and its response value, a designed fusion strategy is used to determine the true target position in the current frame. For example, a weighted average of the predicted and re-identified positions can be selected as the final position, with the weights dynamically adjusted based on the response value.

[0040] In summary, this application avoids the possible misjudgment caused by relying on the global response map by directly analyzing the local features of the target area. By adopting a feature matching mechanism based on optical flow, the spatiotemporal continuity of the target motion is fully utilized to improve the accuracy of drift detection. Combining Kalman filter prediction and re-identification results, a position correction mechanism based on multi-source information fusion is constructed, which reduces the computational complexity while ensuring positioning accuracy. This solution does not require a complex neural network structure and avoids the problem of insufficient accuracy caused by binary segmentation. It is suitable for achieving real-time and stable target tracking on embedded platforms with limited computing resources.

[0041] In some embodiments, the present application provides an algorithm for extracting a first corner point group, such as Figure 4 As shown, the specific steps include: Step 401: Extract the previous frame grayscale image Based on the characteristic value of each pixel, the characteristic corner response value and corner screening threshold of the current frame grayscale image at each pixel are calculated; the formula is as follows:

[0042]

[0043] Among them and Indicates The eigenvalue at the pixel point, Indicates The minimum eigenvalue at the pixel coordinate, that is The characteristic corner response value of the pixel point; Indicates the quality level parameter to be set, Represents the grayscale image of the previous frame The maximum characteristic corner response value in , T represents the corner screening threshold; Feature corner response value The calculation is based on the smaller of the two eigenvalues ​​at the pixel point. and The quality level parameter _qLevel is used to dynamically adjust the screening threshold, and is set to 0.01 in this embodiment.

[0044] Step 402: Filter out the previous frame grayscale image All candidate feature corner points whose response values ​​are greater than T; Step 403, sort the candidate corner points from large to small according to the response value to form a sequence {p1, p2, p3...pi}; Step 404: Starting from point p1, calculate the distance from all other candidate feature corner points to p1. d , delete the distance less than the threshold Then select the second candidate corner point, calculate the distance from the remaining candidate corner points whose response values ​​are less than the second candidate corner point to the second candidate corner point, and so on, to determine the target feature corner point among the candidate feature corner points; all target feature corner points form the first corner point group .

[0045] The screening process of candidate feature corners is divided into two stages: the first stage filters out low-quality corners through the corner screening threshold T, and the second stage removes densely distributed redundant corners through the corner distance constraint.

[0046] In this embodiment, the corner distance d The calculation can be done using the Euclidean distance formula, for example, the coordinates of the two corner points are and hour, The distance threshold min_distance can be set to 5 pixels to ensure that there is a reasonable distance between adjacent corner points.

[0047] For example, the first candidate corner point p1 is calculated separately from p2 to pi, assuming that the distance value calculated for p2 is less than the threshold , then point p2 is deleted, and p3 in the updated sequence becomes the second candidate corner point. The distance between p3 and p4 to pi is then calculated, and so on. The advantage of this approach is that each round of iterative calculation can filter out some corner points that do not meet the distance criteria, significantly reducing the number of corner points after the sequence is updated. This is more efficient than calculating the distance between all corner points pairwise.

[0048] Specifically, during corner detection in the previous frame's grayscale image, all pixels within the target bounding box are first traversed, calculating the characteristic corner response value for each pixel. By setting a quality level parameter, the screening threshold can be adapted to the corner intensity distribution of different image scenes. For example, in texture-rich images, the maximum response value is higher, so the threshold T is increased accordingly to avoid retaining too many redundant corner points. In sparsely textured scenes, the threshold T is automatically lowered to ensure a sufficient number of valid corner points are extracted. After the response value screening is completed, the spatial distribution of the remaining candidate corner points is optimized. By calculating the distance between each corner point, corner points with high response values ​​and uniform distribution are prioritized. For example, when the distance between two corner points is less than 5 pixels, only the corner points with the higher response value are retained, and the rest are filtered out. Through this two-stage screening mechanism, the resulting first corner point cluster has both highly discriminative characteristic responses and a reasonable spatial distribution. This effectively avoids the redundant matching calculation caused by densely packed corner points and reduces the background noise introduced by low-quality corner points, providing stable and reliable feature input for subsequent optical flow tracking.

[0049] In the above embodiment, the candidate feature corner points may be redundant due to high density, or the response value of the corner points retained after screening may be insufficient, which affects the stability and tracking accuracy of optical flow matching. To this end, the target feature corner points can be further screened to determine the distance between the corner points. d Less than the distance threshold The candidate feature corner point pairs are filtered out, and the first candidate feature corner point whose feature corner point response value is less than T is filtered out, and the remaining candidate feature corner points are the target feature corner points.

[0050] Candidate feature corner pairs are first screened by calculating the Euclidean distance between each pair of corner points. Adjacent corner points with a distance less than min_distance are excluded. For example, when the coordinates of two corner points are (50, 60) and (55, 65), the distance d = 7.07 pixels. If min_distance = 8 pixels, the corner pair is retained. After elimination, the remaining corner points are sorted from high to low by response value to form an ordered sequence. By continuously calculating the distance between each pair of feature corner points in the sequence and continuously removing feature corner points that do not meet the distance requirement, the remaining target feature corner points are those that meet the requirement.

[0051] This step-by-step screening mechanism ensures that target feature corners meet both spatial distribution uniformity and texture saliency requirements. For example, it retains highly responsive corners with a spacing of at least 8 pixels at the edge of the target frame to avoid corner point stacking in localized areas. This dual-screened corner cluster is used to construct a primary corner cluster, providing a highly discriminative set of feature points for subsequent optical flow matching. For example, in scenes with rapidly moving targets, the filtered corner cluster achieves a 15%-20% increase in matching success rate, effectively reducing drift misjudgments caused by mismatches.

[0052] For the calculation and matching process of the second corner point group, this step is to directly call a more mature method interface, using the optical flow algorithm, combined with the first corner point group and the previous frame grayscale image to match the feature corner points of the current frame grayscale image; judge whether the match is successful and output the target feature corner points that are successfully matched.

[0053] In some embodiments, when the number of successful pairings exceeds the target ratio of the total number of feature corner points, relying solely on the number of pairings cannot effectively distinguish between normal movement caused by changes in the target's posture and abnormal drift caused by background interference, and there is a risk of misjudgment, resulting in the tracking frame correction mechanism being triggered or missed incorrectly. In response to this, the present application further proposes a method for determining the drift state of the current frame grayscale image corresponding to the tracking target based on the distance and angle information of the successfully matched corner point pairs in the two corner point groups. Figure 5 As shown, this solution can be implemented through the following steps: Step 501: determining the total number of characteristic corner points and the number of successful pairings in the first corner point group, and determining the drift state of the tracked target according to the number of successful pairings; The threshold for the ratio of the total number of feature corner points in the first corner point group to the number of successful pairings can be set between 50% and 80%, for example, 70%. If the number of successful pairings does not exceed this ratio, the system is directly judged to be in a drift state. This threshold can be dynamically adjusted based on the target's motion speed.

[0054] Step 502: When the number of successful pairings exceeds the target ratio of the total number of feature corner points, the distance and angle information values ​​of the feature corner point pairs in the current frame grayscale image and the previous frame grayscale image are calculated; For cases where the threshold is exceeded, the Euclidean distance and motion direction angle of the displacement vector of each pair of matching corner points need to be calculated. The angle calculation can use the inverse tangent function to process the coordinate difference.

[0055] Step 503: sort the distance and angle information values ​​of the feature corner pairs in ascending order, select the target ratio data before and after the median, and form the distance array and the angle array ; In one possible implementation, after sorting the motion parameters of all feature corner pairs in ascending order, 10%-30% of the data before and after the median can be intercepted to form a new array, for example, 15% of the data before and after the median can be selected. This data screening mechanism can exclude abnormal motion parameters that are too large or too small, and retain valid data that reflects the motion trend of the target subject. Distance array with the angle array The statistical parameters are calculated using the sliding window method, and the window size is set to 20%-40% of the array length.

[0056] Step 504, based on the distance array and the angle array Determine the drift state of the tracked target.

[0057] In the statistical parameter determination phase, the distance mean threshold can be set to 5%-15% of the target box's short side size, for example, 10%; the distance standard deviation threshold can be set to 3%-8% of the short side size; and the angle standard deviation threshold can be set to 15-35 degrees. Drift determination is triggered when any statistic exceeds the corresponding threshold. Multiple conditions are judged using a logical OR relationship.

[0058] For example, first, determine the total number of feature corner points and the number of successful pairings in the first corner point group. For example, suppose the total number of feature corner points in the first corner point group is 100, and the number of successful pairings is 80. The drift state of the tracked target is determined based on the number of successful pairings. Specifically, set a target ratio of 70%. When the number of successful pairings does not exceed 70% of the total number of feature corner points, the tracked target is considered to have drifted. In this example, 80 is greater than 70, so the next step is determined.

[0059] Secondly, when the number of successful pairings exceeds the target ratio of the total number of feature corner points, the distance and angle information values ​​of the feature corner point pairs in the current frame grayscale image and the previous frame grayscale image are calculated. For example, the Euclidean distance and angle difference of 80 pairs of feature corner points can be calculated.

[0060] Furthermore, the distance and angle information values ​​of the feature corner pairs are sorted in ascending order, and the data of the target ratio before and after the median are selected to form the distance array and the angle array Specifically, we can select the middle 60% of the data, that is, the 16th to 64th data after sorting, to form the distance array and the angle array .

[0061] Finally, based on the distance array and the angle array Determine the drift state of the tracked target. For example, you can calculate The mean and standard deviation of If the distance mean or standard deviation exceeds the preset ratio of the smaller value of the target frame width, or the angle standard deviation exceeds the preset threshold, the tracked target is determined to be drifting.

[0062] In some other embodiments, step 504 may be further implemented by the following formula: Calculate the distance array separately and the angle array The mean and standard deviation of If the distance to the mean or standard deviation Exceeds the preset ratio of the target frame or the angle standard deviation Exceeding the threshold , then the drift is determined If true, re-identification is performed; the judgment formula is as follows:

[0063] Among them and Represent the width and height of the target box respectively, and To set the scale; Represents a Boolean operation that determines whether drift is true.

[0064] For example, the application can set the proportional coefficient and The value range can be set from 0.1 to 0.3, and the specific value is dynamically adjusted according to the target box size. Angle threshold It can be set to 10-30 degrees to distinguish normal deformation from abnormal rotation. During the calculation process, the width and height parameters of the target box w and h The detection results of the current frame are obtained in real time, ensuring that the threshold dynamically adapts to changes in the target size. When any of the three conditions is triggered, the Boolean logic operation immediately returns a true value, triggering the re-identification mechanism.

[0065] Assuming the target box size w* h is 100*80 pixels; scale factor Set to 0.2, Set to 0.15; Angle standard deviation threshold Set to 30°. The drift state is determined as follows: 1) Calculate the mean distance is 25 pixels, and the standard deviation of the distance is 18 pixels, and the angle standard deviation is 35°; 2) Calculate the judgment threshold: distance mean threshold Pixel; distance standard deviation threshold Pixels; 3) Make conditional judgments: , satisfying the first condition; , satisfying the second condition; , satisfying the third condition; 4) Drift is determined because any of the three conditions are met t is true; 5) Finally, based on the judgment results, the system will perform target re-identification operations.

[0066] This solution dynamically adjusts the judgment threshold based on the actual target size, improving its adaptability to targets of varying sizes. Furthermore, through comprehensive judgment based on three dimensions: distance mean, distance standard deviation, and angle standard deviation, it ensures comprehensive judgment criteria while enhancing interference resistance. This multi-metric combined judgment approach effectively reduces the potential for misjudgments and missed detections caused by a single metric or fixed threshold, improving the accuracy and robustness of drift state recognition. Furthermore, this solution can be implemented through simple statistical calculations, with low computational complexity, making it suitable for real-time execution on embedded platforms, thereby enhancing the real-time and stability of target tracking.

[0067] In the above scheme, after determining the drift, it is necessary to re-identify the tracking target, and the re-identification result will re-determine the target frame area and determine the center position as . Based on Kalman filtering, the tracking target position in the current frame is predicted , based on the center position of re-identification and the predicted tracking target position , determine the target distance between the two ; The calculation process is as follows:

[0068] Among them Indicates square root calculation.

[0069] The re-identification process can be implemented using a matching algorithm based on deep features or a color histogram-based similarity comparison method. For example, a ResNet-18 network can be used to extract target appearance features and sort them by similarity. When predicting position using a Kalman filter, the motion model parameters can be set to a constant velocity model, with the process noise covariance matrix Q ranging from [0.1, 1.0] and the observation noise covariance matrix R ranging from [0.5, 2.0]. When calculating target distance, the coordinate difference square operation can use a fast table lookup method to optimize computational efficiency, and the square root calculation can be replaced with an approximate calculation function to reduce computational time.

[0070] The spatial difference between the re-identified center position and the predicted position is quantified using the Euclidean distance parameter, which forms a complementary constraint with the associated response value. For example, when multiple candidate boxes are present in the re-identified result, the distance parameter can be set as a filtering threshold to retain only those with a distance less than 50% of the diagonal length of the target box. The Kalman filter predicts the position by constraining spatial continuity using historical trajectories. The prediction error range can be set to 10%-20% of the target box width.

[0071] Specifically, after detecting that the target frame has drifted, the target detection model is first started to perform a global search on the current frame to obtain multiple candidate target frame areas and their center coordinates. At the same time, the Kalman filter predicts the target position of the current frame based on the motion state of the previous frame and generates predicted coordinates. By calculating the Euclidean distance between the center of each candidate target frame area and the predicted position, a spatial displacement quantification index is formed. This distance parameter and the relevant response values ​​of the candidate target frame and the original template constitute a two-dimensional evaluation system, and normalization is achieved through ratio calculation. For example, when the response value of a candidate frame is 0.8 and the distance is 10 pixels, its ratio coefficient is 0.08, while when the response value of another candidate target frame is 0.6 but the distance is 5 pixels, the ratio coefficient is increased to 0.12. Finally, the candidate target frame corresponding to the maximum ratio is selected as the correction result, which not only ensures the apparent matching degree but also meets the motion continuity requirements.

[0072] Furthermore, by combining the re-identification result with the response map after correlation calculation of the tracking template, the process of determining the true target position in the current frame image can be achieved as follows: A. Calculate the correlation between the newly determined target box area and the tracking template before re-identification to obtain their respective maximum response values ; means as follows:

[0073] B. Calculate the maximum response value of each Distance to the target Ratio ; Set the maximum ratio The corresponding recognition result is determined as the true target position; the calculation process is as follows:

[0074] Among them and Indicates the The center pixel coordinates of the re-identification results; Representation sample With template The Fourier transform of the kernel correlation between represents element-wise multiplication, is the Fourier transform of the trained filter coefficients, represents the inverse Fourier transform.

[0075] Specifically, the similarity between the candidate target bounding box region and the original tracking template is calculated using a kernel-correlated Fourier transform. The features of the sample and template are then rapidly convolved in the Fourier domain to generate a frequency domain response map. An inverse Fourier transform converts the frequency domain response to the spatial domain, obtaining a quantitative value for the degree of match between each candidate region and the template. The maximum response value is extracted as an indicator of the apparent consistency between the candidate region and the template. A normalized ratio metric is constructed by combining the spatial distance between the Kalman filter predicted position and the re-identified candidate center. The distance parameter is calculated using the Euclidean distance formula to suppress candidate regions with excessive spatial offset. The ratio metric combines the apparent match with spatial continuity constraints to eliminate the risk of misjudgment introduced by a single metric. Finally, by comparing the ratio parameters of all candidates, the candidate with the maximum ratio is selected as the true target location, ensuring optimal apparent match and continuous motion trajectory. The trained filter coefficients are learned from a large number of samples, enhancing robustness to target deformation and illumination changes. For example, suppose there are three re-identification results with their maximum response value. are 0.8, 0.7 and 0.9 respectively, corresponding to the target distance The calculated ratios are 10, 8, and 15 pixels. They are 0.08, 0.0875, and 0.06 respectively. Since the second result has the largest ratio, the second re-identification result is selected as the true target location.

[0076] Through the above technical solution, the present application can effectively solve the problem of misjudgment of target position in multi-candidate scenarios. The dual constraints of apparent similarity and spatial distance of the re-identification results and tracking templates are integrated to improve the accuracy of target positioning. The calculation method of kernel correlation and Fourier transform is used to improve the efficiency of feature matching. The introduction of filter coefficients enhances the ability to distinguish between templates and samples, making the response value calculation more resistant to interference. By calculating the ratio of response value to distance, the reliability of apparent feature matching is guaranteed, and the continuity of target motion is controlled, forming a dual verification mechanism for drift phenomenon. This method can more accurately locate the real target in the case of complex background and multi-target interference, reduce drift and misjudgment during tracking, and improve the overall performance and stability of target tracking.

[0077] Figure 6 The following is a structural block diagram of a target frame automatic tracking drift judgment device provided by an embodiment of the present application, which includes: A conversion module 610 is used to cache two adjacent frames of images and extract a target frame area in the image, and convert the target frame area image into a grayscale image; Corner point calculation module 620 is used to perform corner point detection on the grayscale image of the previous frame and extract a first corner point group within the target frame area; perform optical flow calculation on the grayscale image of the current frame based on the grayscale image of the previous frame and the first corner point group, and match the second corner point group of the grayscale image of the current frame; the corner point group contains the target feature corner points of the corresponding grayscale image; A drift determination module 630 is configured to determine the drift state of the tracking target corresponding to the current frame image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group; The tracking update module 640 is used to predict the target position in the current frame image based on the Kalman filter and re-identify the tracked target when it is determined that the tracked target has drifted. The real target position in the current frame image is determined by combining the re-identification result and the response map after correlation calculation with the tracking template.

[0078] It should be noted that the target frame automatic tracking drift judgment device provided in the embodiment of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the target frame automatic tracking drift judgment device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the target frame automatic tracking drift judgment method provided in the above-mentioned method embodiment and the implementation method of the target frame automatic tracking drift judgment device provided in this embodiment belong to the same concept. The specific implementation process of the target frame automatic tracking drift judgment device provided in this embodiment is detailed in the above-mentioned method embodiment and will not be repeated here.

[0079] Figure 7The following is a block diagram of the structure of a computer device provided by an exemplary embodiment of the present application. The computer device is a desktop computer, a laptop computer, a PDA, a cloud server, and the like. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processing units (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0080] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor may be implemented in at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is used to process data while awake, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data while in standby mode. In some embodiments, the processor may integrate a graphics processing unit (GPU), which is responsible for rendering and drawing content displayed on the display. In some embodiments, the processor may also include an artificial intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0081] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method embodiment is implemented. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0082] In some embodiments, the computer device may optionally include a peripheral device interface and at least one peripheral device. The processor, memory, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Specifically, the peripheral device includes at least one of a radio frequency circuit, a display screen, and a keyboard.

[0083] The peripheral device interface can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor and memory. In some embodiments, the processor, memory, and peripheral device interface are integrated on the same chip or circuit board. In other embodiments, any one or two of the processor, memory, and peripheral device interface can be implemented on separate chips or circuit boards, although this embodiment is not limited to this.

[0084] The display screen is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, or any combination thereof. When the display screen is a touch screen, it also has the ability to capture touch signals on or above the surface of the display screen. The touch signals can be input as control signals to a processor for processing. In this case, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen, disposed on the front panel of the computer device; in other embodiments, there can be at least two display screens, disposed on different surfaces of the computer device or in a foldable design; in still other embodiments, the display screen can be a flexible display screen, disposed on a curved or foldable surface of the computer device. Furthermore, the display screen can be configured as a non-rectangular, irregular shape, also known as a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0085] A power supply is used to power various components in computer equipment. The power supply can be AC, DC, disposable batteries, or rechargeable batteries. When the power supply includes a rechargeable battery, it can be wired or wirelessly rechargeable. A wired rechargeable battery is charged via a wired line, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also support fast charging technology.

[0086] Those skilled in the art will understand that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0087] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art will understand that all or part of the processes in the above-mentioned method implementation of the present application can be completed by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the implementation of the above-mentioned methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0088] This specific embodiment is merely an explanation of the present invention and is not intended to limit the present invention. After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed. However, as long as such modifications are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A target frame automatic tracking drift judgment method, characterized in that: The method comprises: Cache two adjacent frames of images and extract the target frame area in the image, and convert the target frame area image into a grayscale image; Perform corner point detection on the previous frame grayscale image to extract the first corner point group within the target frame area; perform optical flow calculation on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and match the second corner point group of the current frame grayscale image; the corner point group contains the target feature corner points of the corresponding grayscale image; Based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group, the drift state of the tracking target corresponding to the current frame image is determined; When it is determined that the tracked target drifts, the target position in the current frame image is predicted based on the Kalman filter and the tracked target is re-identified. The response map after the re-identification result and the tracking template are combined to determine the true target position in the current frame image.

2. The method according to claim 1, characterized in that The performing corner point detection on the grayscale image of the previous frame to extract the first corner point group within the target frame area includes: Extract the previous frame grayscale image Based on the characteristic value of each pixel, the characteristic corner response value and corner screening threshold of the current frame grayscale image at each pixel are obtained. The formula is as follows: Among them and Indicates The eigenvalue at the pixel point, Indicates The minimum eigenvalue at the pixel coordinate, that is The characteristic corner response value of the pixel point; Indicates the quality level parameter to be set, Represents the grayscale image of the previous frame The maximum characteristic corner response value in , T represents the corner screening threshold; Filter out the previous frame grayscale image All candidate corner points whose response values ​​are greater than T; Sort the candidate corner points from large to small according to the response value to form a sequence {p1, p2, p3…pi}; Starting from the first candidate corner point p1 in the sequence, calculate the distance from all other candidate feature corner points to p1 d , delete the sequences whose distance is less than the threshold Then, the second candidate corner point is selected, and the distance from the remaining candidate corner points whose response values ​​are smaller than the second candidate corner point to the second candidate corner point is calculated, and so on, the target feature corner point among the candidate feature corner points is determined; all the target feature corner points constitute the first corner point group .

3. The method according to claim 1, characterized in that The determining of the drift state of the tracking target corresponding to the current frame grayscale image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group includes: Determining the total number of characteristic corner points and the number of successful pairings in the first corner point group, and determining a drift state of the tracked target based on the number of successful pairings; wherein, when the number of successful pairings does not exceed the target ratio of the total number of characteristic corner points, it is determined that the tracked target has drifted; When the number of successful pairings exceeds the target ratio of the total number of feature corner points, the distance and angle information values ​​of the feature corner point pairs in the current frame grayscale image and the previous frame grayscale image are calculated; Sort the distance and angle information values ​​of the feature corner pairs in ascending order, select the target ratio data before and after the median, and form the distance array respectively and the angle array ; Based on distance array and the angle array Determine the drift state of the tracked target.

4. The method according to claim 3, characterized in that The distance array and the angle array Determine the drift status of the tracked target, including: Calculate the distance array separately and the angle array The mean and standard deviation of If the distance to the mean or standard deviation Exceeds the preset ratio of the target frame or the angle standard deviation Exceeding the threshold , then the drift is determined If true, re-identification is performed; the judgment formula is as follows: Among them and Represent the width and height of the target box respectively, and is the preset ratio; Represents a Boolean operation that determines whether drift is true.

5. The method according to any one of claims 1 to 4, characterized in that: The method of predicting the target position in the grayscale image of the current frame based on Kalman filtering and performing tracking target re-identification includes: When the target frame is confirmed to have drifted, the tracked target is re-identified; the re-identification result re-determines the target frame area and determines the center position as ; Predict the position of the tracking target in the current frame based on Kalman filtering , based on the center position of re-identification and the predicted tracking target position , determine the target distance between the two ; The calculation process is as follows: Among them Indicates square root calculation.

6. The method according to claim 5, characterized in that The method combines the re-identification result and the response map obtained by performing correlation calculation on the tracking template to determine the true target position in the current frame image, including: The newly determined target box area is correlated with the tracking template before re-identification to obtain the respective maximum response values ; means as follows: Calculate the maximum response value of each Distance to the target Ratio ; Set the maximum ratio The corresponding recognition result is determined as the true target position; the calculation process is as follows: Among them and Indicates the The center pixel coordinates of the re-identification results; Representation sample With template The Fourier transform of the kernel correlation between represents element-wise multiplication, is the Fourier transform of the trained filter coefficients, represents the inverse Fourier transform.

7. A target frame automatic tracking drift judgment device, characterized in that: The device comprises: A conversion module is used to cache two adjacent frames of images and extract the target frame area in the image, and convert the target frame area image into a grayscale image; A corner point calculation module is used to perform corner point detection on the grayscale image of the previous frame and extract a first corner point group within the target frame area; perform optical flow calculation on the grayscale image of the current frame based on the grayscale image of the previous frame and the first corner point group, and match the second corner point group of the grayscale image of the current frame; the corner point group contains the target feature corner points of the corresponding grayscale image; The drift determination module is used to determine the drift state of the tracking target corresponding to the current frame image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group; The tracking update module is used to predict the target position in the current frame image and re-identify the tracked target based on the Kalman filter when it is determined that the tracked target has drifted. The real target position in the current frame image is determined by combining the re-identification result and the response map after correlation calculation with the tracking template.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the target frame automatic tracking drift judgment method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the target frame automatic tracking drift judgment method as described in any one of claims 1 to 6.

Images