Target frame automatic tracking drift judgment method and device, equipment and storage medium

Abstract

The application discloses a target frame automatic tracking drift judgment method and device, equipment and a storage medium, and relates to the field of image recognition. A target frame region of adjacent two frames of images is extracted, which is converted into a gray image; corner point detection is performed on a previous frame gray image to extract a first corner point group; optical flow calculation is performed on a current frame gray image based on the previous frame gray image and the first corner point group to match a second corner point group; the distance and angle information of matched feature corner point pairs in the corner point group are used to determine the drift state of a tracking target corresponding to the current frame image; when it is determined that the tracking target drifts, the target position in the current frame image is predicted based on Kalman filtering, and the tracking target is re-identified; and the real target position in the current frame image is determined by combining a response map obtained by performing correlation calculation on a re-identification result and a tracking template. According to the scheme, feature level drift determination is achieved through corner point detection and optical flow matching, effective re-identification is performed, and the target frame drift problem is solved.

Description

Technical Field

[0001] This application relates to the field of image recognition, and in particular to a method, apparatus, device, and storage medium for automatically tracking and judging target bounding box drift. Background Technology

[0002] Visual tracking algorithms based on correlation filtering are widely used on embedded platforms due to their good localization accuracy and computational efficiency. During tracking, features of the current frame's search region are extracted and correlated with template features to generate a response map, thus locating the target's new position. However, due to background interference, occlusion, and other factors, the generated maximum response deviates from the target's true position, resulting in the target bounding box in the current frame not completely encompassing the target, a phenomenon known as drift.

[0003] Currently, most methods used to reduce drift are as follows: 1) Use neural networks to extract features from the template and search region to improve robustness and reduce interference; 2) Determine whether drift has occurred based on information from the target bounding box region. For example, in patent CN112509003B, "Method and System for Solving Target Tracking Box Drift," a response map skewness is defined to determine whether it meets preset conditions. If it does, the image within the tracking box region is converted into a binary image, and the centroid of that region is calculated. Then, based on the deviation between the centroid and the target bounding box center predicted by the tracking algorithm, it is determined whether to correct the tracking box.

[0004] The two solutions mentioned above have the following drawbacks: 1) Neural network methods often require targeted training in different scenarios to obtain appropriate network weights, which involves a large workload; secondly, neural networks have a large computational load, and deployment on embedded platforms requires sufficient computing power.

[0005] 2) The response map indirectly calculated by correlation filtering is easily affected by background interference, and it is inevitable that there will be no drift but the response map has multiple peaks or diffuseness, which will cause the skewness and other indicators to exceed the limit; the accuracy of subsequent binarization centroid calculation depends heavily on whether the foreground can be correctly segmented. Summary of the Invention

[0006] This application provides a method, apparatus, device, and storage medium for automatically tracking and judging target bounding boxes, which has the advantages of improving the accuracy and stability of target tracking and effectively solving the target bounding box drift problem.

[0007] On the one hand, this application provides a method for automatically tracking and determining target bounding box drift, the method comprising: Cache two adjacent frames of images and extract the target bounding box region from the images, then convert the target bounding box region image into a grayscale image; Corner detection is performed on the previous frame grayscale image to extract the first corner point group within the target bounding box; optical flow calculation is performed on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and the second corner point group of the current frame grayscale image is matched; the corner point group contains the target feature corner points of the corresponding grayscale image; Based on the distance and angle information of the successfully matched feature corner pairs in the corner group, the drift state of the tracking target corresponding to the current frame image is determined. When the tracking target is determined to have drifted, the target position in the current frame image is predicted based on Kalman filtering and the tracking target is re-identified. The response map after relevant calculations are performed by combining the re-identification result and the tracking template to determine the true target position in the current frame image.

[0008] Specifically, the step of performing corner detection on the previous frame grayscale image and extracting the first group of corner points within the target bounding box includes: Extract the grayscale image of the previous frame Based on the feature values ​​of each pixel, calculate the feature corner response value and corner filtering threshold of the current frame grayscale image at each pixel, using the following formula:

[0009]

[0010] Among them and Indicates in Feature values ​​at a pixel Indicates in The minimum eigenvalue at pixel coordinates, i.e. Feature corner response values ​​of a pixel; This indicates the set quality level parameter. Represents the previous frame grayscale image The response value of the largest feature corner point in the middle, where T represents the corner point filtering threshold; Filter out the previous frame grayscale image Candidate corner points whose response values ​​are greater than T; The candidate corner points are sorted in descending order of response value to form a sequence {p1, p2, p3…pi}; Starting from the first candidate corner point p1 in the sequence, calculate the distance from p1 to all other candidate feature corner points. d Delete sequences whose distance is less than a threshold The first candidate corner point is selected; then the second candidate corner point is selected, and the distance from the remaining candidate corner points with response values ​​less than the second candidate corner point to the second candidate corner point is calculated. This process is repeated to determine the target feature corner point among the candidate feature corner points. All the target feature corner points form the first corner point group. .

[0011] Specifically, determining the drift state of the tracked target corresponding to the current frame grayscale image based on the distance and angle information of successfully matched feature corner pairs in the corner point group includes: Determine the total number of feature corner points and the number of successful pairings in the first corner point group, and determine the drift state of the tracked target based on the number of successful pairings; wherein, when the number of successful pairings does not exceed the target proportion of the total number of feature corner points, it is determined that the tracked target has drifted; When the number of successful pairings exceeds the target proportion of the total number of feature corner points, calculate the distance and angle information values ​​of the feature corner point pairs between the current frame grayscale image and the previous frame grayscale image; The distance and angle information values ​​of the feature corner point pairs are sorted in ascending order, and the data with the target proportions before and after the median are selected to form distance arrays. and angle array ; Based on distance array and angle array Determine the drift state of the target being tracked.

[0012] Specifically, the distance array-based and angle array Determine the drift state of the tracked target, including: Calculate the distance array respectively and angle array The mean and standard deviation; If the distance to the mean or standard deviation Exceeds the preset proportion of the target bounding box, or the standard deviation of the angle. Exceeding the threshold Then drift is determined. If true, perform re-identification; the determination formula is as follows:

[0013] Among them and These represent the width and height of the target bounding box, respectively. and To set the ratio; This represents a Boolean operation that determines whether the drift is true.

[0014] Specifically, the step of predicting the target position in the current frame's grayscale image based on Kalman filtering and performing target re-identification includes: When target bounding box drift is confirmed, the tracked target is re-identified; the re-identification result redetermines the target bounding box region and determines its center position. ; Predicting the position of the tracked target in the current frame based on Kalman filtering. Based on the center position of re-identification and predicted tracking target location Determine the target distance between the two. The calculation process is as follows:

[0015] Among them This indicates the calculation of the square root.

[0016] Specifically, determining the true target location in the current frame image by combining the re-identification result and the relevant calculated response value of the tracking template includes: The newly determined target bounding box region is correlated with the tracking template before re-identification to obtain their respective maximum response values. ; indicates the following:

[0017] Calculate their respective maximum response values Distance from the target ratio ; the maximum ratio The corresponding recognition result is determined as the true target location; the calculation process is as follows:

[0018] Among them and Indicates the first The center pixel coordinates of each recognition result; Indicates sample With template Fourier transform of kernel correlation between them This indicates element-wise multiplication. The Fourier transform of the trained filter coefficients. This represents the inverse Fourier transform.

[0019] On the other hand, this application provides a target bounding box automatic tracking drift determination device, the device comprising: The conversion module is used to cache two adjacent frames of images and extract the target bounding box region from the images, and convert the target bounding box region image into a grayscale image. The corner point calculation module is used to perform corner point detection on the previous frame grayscale image and extract the first corner point group within the target bounding box; perform optical flow calculation on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and match the second corner point group of the current frame grayscale image; the corner point group contains the target feature corner points of the corresponding grayscale image; The drift determination module is used to determine the drift state of the tracking target corresponding to the current frame grayscale image based on the distance and angle information of the successfully matched feature corner point pairs in the corner point group. The tracking update module is used to predict the target position in the current frame grayscale image based on Kalman filtering and to re-identify the tracked target. It combines the re-identification result and the response map after relevant calculations with the tracking template to determine the true target position in the current frame image.

[0020] In another aspect, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the target box automatic tracking drift judgment method described above.

[0021] In another aspect, this application provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the target box automatic tracking drift judgment method described above.

[0022] The beneficial effects of the technical solution provided in this application include at least the following: feature-level drift determination is achieved through corner detection and optical flow matching, and the Kalman filter prediction and re-identification mechanism is combined to effectively distinguish between real drift and interference artifacts. While ensuring computational efficiency, the accuracy and stability of target tracking are improved, which has the advantages of improving the accuracy and stability of target tracking and effectively solving the target box drift problem. 1) No need for annotation training for different scenarios, and the computing power requirement is low, taking 13ms on a Hi3559_AV100 A73 core CPU; 2) Directly utilize the appearance information of the target region in two adjacent frames to reduce the calculation error in the indirect process; 3) Combine the similarity with the template and the prediction results to filter out the real target. Attached Figure Description

[0023] Figure 1 This is a flowchart of the target bounding box automatic tracking drift judgment method provided in the embodiments of this application; Figure 2 This diagram illustrates the extraction and conversion of the tracked target bounding box from a frame image into a grayscale image. Figure 3 The image shown is a schematic diagram illustrating the positions of the drone and the target bounding box in the current frame. Figure 4 The flowchart of the algorithm for extracting the first corner point group is shown; Figure 5 A flowchart is shown to determine the drift state of the target being tracked in the current frame grayscale image; Figure 6 This paper shows a structural block diagram of the target box automatic tracking drift determination device provided in an embodiment of this application; Figure 7 A structural block diagram of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0026] In traditional visual tracking algorithms based on correlation filtering, target bounding box drift is mainly caused by interference in the response map and insufficient segmentation accuracy. Existing methods rely on morphological features of the response map or binarized centroid correction mechanisms, which are prone to misjudgment when the target is moving rapidly or in the presence of complex background interference. The multi-peak phenomenon of the response map causes the skewness index to fail, and binarized segmentation is significantly affected by changes in illumination, making it difficult to accurately extract the centroid of the foreground region. These defects directly affect the target localization accuracy, causing the tracking box to deviate from its true position and reducing the robustness of the system.

[0027] Faced with the aforementioned problems, this application first considers how to improve the accuracy of drift determination without relying on the complex computation of neural networks. Traditional methods, such as response map morphology analysis or binarized centroid correction, have inherent defects; the former is susceptible to multi-peak interference, and the latter relies on precise segmentation. To address this, this application attempts to explore a feature matching mechanism based on spatiotemporal continuity by focusing on the stability of the apparent features of the target region. By analyzing the corner distribution patterns within the target bounding box of adjacent frames, it is found that the relative positions and directions of corner groups during target motion exhibit spatiotemporal consistency. Compared to the global response map, local corner matching can reflect changes in the target's own structure while avoiding background noise interference. Furthermore, considering the efficiency of optical flow in motion estimation, a dual verification mechanism is constructed by combining Kalman filter prediction, forming a solution that is both capable of real-time computation and has anti-interference capabilities.

[0028] Figure 1 This is a flowchart of the target bounding box automatic tracking drift judgment method provided in the embodiments of this application, which includes the following steps: S1. Cache two adjacent frames of images and extract the target bounding box region from the images, and convert the target bounding box region image into a grayscale image; Caching two adjacent frames refers to temporarily storing the image data acquired at the current moment and the previous moment in a storage medium, which can be implemented using memory or a cache. Extracting the target bounding box region from the image means cropping a rectangular region containing the tracked object from the complete image, which can be implemented using coordinate cropping or image segmentation algorithms, focusing on the local features of the target to reduce background interference. Converting the target bounding box region image to grayscale means converting a color image to a single-channel grayscale image, which can be implemented using a weighted average method or a brightness extraction algorithm, reducing the complexity of subsequent calculations while preserving the structural features of the target.

[0029] Figure 2 This diagram illustrates the extraction and conversion of the tracked target bounding box from a frame image into a grayscale image. The left side shows the normally cached frame image, while the right side shows a grayscale image of the target bounding box region that has been cropped, enlarged, and converted. Adjacent frames are converted into the previous frame's grayscale image and the current frame's grayscale image, respectively.

[0030] S2. Perform corner detection on the previous frame grayscale image and extract the first corner point group within the target box area; perform optical flow calculation on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and match the second corner point group of the current frame grayscale image; Corner detection on the previous frame grayscale image refers to identifying pixels with significant gradient changes, i.e., target feature corners, using feature point detection algorithms. Specifically, Harris corner detection or Shi-Tomasi algorithms can be used to extract key structural features within the target region. The first corner group refers to the set of feature points with high response values ​​retained through a filtering algorithm. This can be achieved using non-maximum suppression or threshold filtering methods to ensure uniform distribution and strong representativeness of the feature points.

[0031] The extraction of corner points from the current frame image also aims to select a set of feature points with high response values. However, since the images are two consecutive frames, optical flow matching is used when selecting corner points in the current frame. In other words, the second corner point group uses the pixel motion vectors between adjacent frames to associate feature points. Specifically, the Lucas-Kanade or Farneback optical flow algorithm can be used to establish the spatiotemporal correspondence of feature points to reflect the target's motion trajectory.

[0032] S3. Based on the distance and angle information of the successfully matched feature corner pairs in the corner point group, determine the drift state of the tracking target corresponding to the current frame image; Determining the drift state refers to judging whether the target deviates from the expected position by statistically analyzing the motion parameters of the matching feature points in the target bounding box region of two frame images. Specifically, this can be achieved by combining the calculation of the mean and standard deviation of the distance with the angle variance analysis to quantify the degree of abnormality of the target motion and avoid subjective threshold setting errors.

[0033] S4. When it is determined that the tracked target has drifted, the target position in the current frame image is predicted based on Kalman filtering and the tracked target is re-identified. The response map after relevant calculations is performed by combining the re-identification result and the tracking template to determine the true target position in the current frame image. Determining that the tracked target has drifted refers to situations such as the target bounding box failing to track or an offset occurring, for example... Figure 3 The image shown in the current frame depicts the positions of the drone and the target bounding box. When drift occurs, the target bounding box does not accurately mark the position of the tracked target. Kalman filtering predicts the target position by predicting the target's trajectory through a state estimation model. Specifically, this can be achieved using linear dynamic system modeling and covariance updating to compensate for tracking position deviations caused by occlusion or interference. Figure 3 The marked points in the target box represent the current frame position coordinates predicted based on the Mann filter.

[0034] Target re-identification refers to the process of relocating a target within a search area using feature matching algorithms. Specifically, correlation filtering or template matching algorithms can be employed, combined with predicted location to narrow the search range and improve re-identification efficiency. After target re-identification, correlation calculations are performed between multiple candidate regions identified through re-identification and the original tracking template to generate a frequency domain response map. Then, a similarity metric is used to evaluate the matching degree between the candidate regions and the target template, determining the final target bounding box (i.e., the target location). This step can be implemented using normalized cross-correlation or convolutional response calculations to quantify the reliability of the identification results and aid in decision-making.

[0035] In one possible implementation, this solution can be achieved through the following steps: First, a continuous sequence of video frames is acquired; adjacent frames are buffered, and the bounding box regions in these two frames are extracted. The extracted bounding box regions are then converted to grayscale images to simplify subsequent processing.

[0036] Next, corner detection is performed on the previous frame's grayscale image. The Harris corner detection algorithm or another improved Shi-Tomasi algorithm is used to extract the first group of corner points within the target bounding box. Based on the previous frame's grayscale image and the first group of corner points, optical flow is calculated on the current frame's grayscale image. The Lucas-Kanade optical flow method is used for feature point tracking to match the second group of corner points in the current frame's grayscale image. These two groups of corner points contain the target feature corner points of the corresponding grayscale images.

[0037] Next, analyze the successfully matched feature corner pairs in the corner point group. Calculate the Euclidean distance and included angle between these corner pairs. Based on the statistical information of distance and angle, determine the drift state of the tracked target in the current frame's grayscale image. For example, distance and angle thresholds can be set; when the distance or angle change of more than a certain proportion of corner pairs exceeds the threshold, it is determined that drift has occurred.

[0038] If the tracked target is determined to have drifted, a correction mechanism is initiated. First, a Kalman filter is used to predict the target's position in the current frame based on historical trajectories. Simultaneously, target re-identification is performed, which can be achieved by using methods such as template matching to relocate the target within the search area.

[0039] Finally, the re-identification results are correlated with the tracking template to obtain a response map and calculate the response value. Combining the Kalman filter predicted position, the re-identification result position, and their response value, a designed fusion strategy is used to determine the true target position in the current frame image. For example, a weighted average of the predicted and re-identified positions can be chosen as the final position, with the weights dynamically adjusted based on the response value.

[0040] In summary, this application avoids potential misjudgments caused by relying on global response maps by directly analyzing local features of the target region. Employing an optical flow-based feature matching mechanism fully utilizes the spatiotemporal continuity of target motion, improving drift detection accuracy. Combining Kalman filter prediction and re-identification results, a multi-source information fusion position correction mechanism is constructed, reducing computational complexity while maintaining positioning accuracy. This scheme eliminates the need for complex neural network structures and avoids the accuracy issues caused by binarization segmentation, making it suitable for real-time and stable target tracking on embedded platforms with limited computing resources.

[0041] In some implementations, this application provides an algorithm for extracting the first corner point group, such as... Figure 4 As shown, the specific steps include: Step 401, extract the grayscale image of the previous frame. Based on the feature values ​​of each pixel, calculate the feature corner response value and corner filtering threshold of the current frame's grayscale image at each pixel; the formula is as follows:

[0042]

[0043] Among them and Indicates in Feature values ​​at a pixel Indicates in The minimum eigenvalue at pixel coordinates, i.e. Feature corner response values ​​of a pixel; This indicates the set quality level parameter. Represents the grayscale image of the previous frame. The response value of the largest feature corner point in the middle, where T represents the corner point filtering threshold; Feature corner response value The calculation is based on the smaller of the two feature values ​​at a pixel. and These represent the gradient intensity of the image in the horizontal and vertical directions, respectively. The quality level parameter _qLevel is used to dynamically adjust the filtering threshold; in this embodiment, it is set to 0.01.

[0044] Step 402: Filter out the grayscale image from the previous frame. Candidate feature corner points whose response values ​​are greater than T; Step 403: Sort the candidate corner points in descending order of response value to form the sequence {p1, p2, p3…pi}; Step 404: Starting from point p1, calculate the distance from p1 to all other candidate feature corner points. d Delete distance less than the threshold The first candidate corner point is selected; then the second candidate corner point is selected, and the distance from the remaining candidate corner points with response values ​​less than the second candidate corner point to the second candidate corner point is calculated. This process is repeated to determine the target feature corner points among the candidate feature corner points. All target feature corner points form the first corner point group. .

[0045] The candidate feature corner selection process is divided into two stages: the first stage filters out low-quality corners by using a corner selection threshold T, and the second stage removes densely distributed redundant corners by using corner distance constraints.

[0046] In this embodiment, the corner distance can be used. d The calculation can be performed using the Euclidean distance formula, for example, if the coordinates of the two corner points are respectively... and hour, The distance threshold min_distance can be set to 5 pixels to ensure a reasonable spacing between adjacent corner points.

[0047] For example, the distances from the first candidate corner point p1 to p2 and pi are calculated separately. Assume the calculated distance for p2 is less than the threshold. If p2 is not found, then p3 becomes the second candidate corner point in the updated sequence. Then, the distances between p3 and p4 to p1 are calculated, and so on. The advantage of this approach is that each iteration filters out some corner points that do not meet the distance criteria, significantly reducing the number of points after the sequence is updated. This iterative calculation is more efficient than calculating the distances between all corner points pairwise.

[0048] Specifically, during corner detection in the previous frame's grayscale image, all pixels within the target bounding box are traversed first, and the feature corner response value of each point is calculated. By setting a quality level parameter, the filtering threshold can adapt to the corner intensity distribution of different image scenes. For example, in images with rich texture, the maximum response value is higher, and the threshold T is increased accordingly to avoid retaining too many redundant corners; in scenes with sparse texture, the threshold T is automatically reduced to ensure that a sufficient number of effective corners are extracted. After completing the response value filtering, the spatial distribution of the remaining candidate corners is optimized. By calculating the distance between each pair of corners, corners with high response values ​​and uniform distribution are prioritized for retention. For example, when the distance between two corners is less than 5 pixels, only the corner with the higher response value is retained, and the rest are filtered out. Through the above two-stage filtering mechanism, the final first corner group has both highly discriminative feature responses and a reasonable spatial distribution, effectively avoiding the redundancy of matching calculations caused by dense corners, while reducing background noise interference introduced by low-quality corners, providing stable and reliable feature input for subsequent optical flow tracking.

[0049] In the above embodiments, candidate feature corner points may become redundant due to excessive density, or the retained corner point response values ​​after filtering may be insufficient, affecting the stability of optical flow matching and tracking accuracy. Therefore, target feature corner points can be further filtered by selecting those with higher corner point distances. d Less than the distance threshold The candidate feature corner point pairs are filtered out, and the first candidate feature corner point whose feature corner point response value is less than T is the remaining candidate feature corner point.

[0050] The selection of candidate feature corner point pairs begins by calculating the Euclidean distance between each pair of corner points, eliminating adjacent corner points with a distance less than `min_distance`. For example, when the coordinates of two corner points are (50, 60) and (55, 65), their distance `d` is 7.07 pixels. If `min_distance` is 8 pixels, then the corner point pair is retained. After elimination, the remaining corner points are sorted from highest to lowest response value, forming an ordered sequence. By continuously calculating the distance values ​​between each pair of feature corner points in the sequence and continuously deleting feature corner points that do not meet the distance requirements, the remaining points are the target feature corner points that meet the requirements.

[0051] This step-by-step filtering mechanism ensures that target feature corner points simultaneously meet the requirements of spatial uniformity and texture saliency. For example, it retains high-response corner points with a spacing of more than 8 pixels in the target bounding box edge region, avoiding corner point stacking in local areas. The first corner point group is constructed through the double-filtered corner point group, providing a highly discriminative set of feature points for subsequent optical flow matching. For example, in scenarios with fast target movement, the matching success rate of the filtered corner point group is improved by 15%-20%, effectively reducing drift misjudgment caused by mismatches.

[0052] For the calculation and matching process of the second corner point group, this step directly calls a relatively mature method interface, uses the optical flow algorithm, and combines the first corner point group and the grayscale image of the previous frame to match the feature corner points of the current frame grayscale image; determines whether the match is successful, and outputs the target feature corner points that are successfully matched.

[0053] In some embodiments, when the number of successful pairings exceeds the target proportion of the total number of feature corner points, relying solely on the number of pairings cannot effectively distinguish between normal motion caused by changes in target pose and abnormal drift caused by background interference, posing a risk of misjudgment and leading to the tracking box correction mechanism being incorrectly triggered or missed. To address this, this application further proposes a method for determining the drift state of the tracked target corresponding to the current frame's grayscale image based on the distance and angle information of successfully matched corner point pairs in two corner point groups. Figure 5 As shown, this solution can be implemented through the following steps: Step 501: Determine the total number of feature corner points and the number of successful pairings in the first corner point group, and determine the drift state of the tracked target based on the number of successful pairings; The ratio threshold between the total number of feature corner points in the first corner point group and the number of successfully paired points can be set to 50%-80%, for example, 70%. When the number of successfully paired points does not exceed this ratio, it is directly determined to be a drift state. This threshold can be dynamically adjusted according to the target's movement speed.

[0054] Step 502: When the number of successful pairings exceeds the target proportion of the total number of feature corner points, calculate the distance and angle information values ​​of the feature corner point pairs between the current frame grayscale image and the previous frame grayscale image. For cases exceeding the threshold, it is necessary to calculate the Euclidean distance and motion direction angle of the displacement vector of each pair of matching corner points. The angle calculation can use the arctangent function to handle the coordinate difference.

[0055] Step 503: Sort the distance and angle information values ​​of the feature corner point pairs in ascending order, and select the data with the target proportions before and after the median to form distance arrays. and angle array ; In one possible implementation, after sorting the motion parameters of all feature corner point pairs in ascending order, this step can extract 10%-30% of the data before and after the median to form a new array, for example, selecting 15% of the data before and after the median. This data filtering mechanism can eliminate abnormal motion parameters that are too large or too small, retaining valid data that reflects the movement trend of the target subject. Distance array with angle array The statistical parameters were calculated using the sliding window method, with the window size set to 20%-40% of the array length.

[0056] Step 504, based on the distance array and angle array Determine the drift state of the target being tracked.

[0057] In the statistical parameter determination stage, the distance mean threshold can be set to 5%-15% of the target bounding box's shorter side size, for example, 10%; the distance standard deviation threshold can be set to 3%-8% of the shorter side size; and the angle standard deviation threshold can be set to 15-35 degrees. A drift determination is triggered when any statistical measure exceeds its corresponding threshold, and multiple conditions are judged using a logical OR relationship.

[0058] For example, first, the total number of feature corner points and the number of successful pairings in the first corner point group are determined. For instance, the total number of feature corner points in the first corner point group is 100, and the number of successful pairings is 80. The drift state of the tracked target is determined based on the number of successful pairings. Specifically, the target ratio is set to 70%. When the number of successful pairings does not exceed 70% of the total number of feature corner points, it is determined that the tracked target has drifted. In this example, 80 is greater than 70, so the next step is performed.

[0059] Secondly, when the number of successful pairings exceeds the target proportion of the total number of feature corner points, the distance and angle information values ​​of the feature corner point pairs between the current frame grayscale image and the previous frame grayscale image are calculated. For example, the Euclidean distance and angle difference of 80 pairs of feature corner points can be calculated.

[0060] Furthermore, the distance and angle information values ​​of the feature corner point pairs are sorted in ascending order, and the data with the target proportions before and after the median are selected to form distance arrays. and angle array Specifically, we can select the middle 60% of the data, that is, the 16th to 64th data points after sorting, to form a distance array. and angle array .

[0061] Finally, based on the distance array and angle array Determine the drift state of the tracked target. For example, it can be calculated. The mean and standard deviation, and The standard deviation of the distance means or standard deviation exceeds a preset proportion of the smaller of the target bounding box width and height, or the standard deviation of the angle exceeds a preset threshold, then the tracked target is determined to have drifted.

[0062] In some other embodiments, step 504 can also be further implemented by the following formula: Calculate the distance array separately and angle array The mean and standard deviation; If the distance to the mean or standard deviation Exceeds the preset proportion of the target bounding box, or the standard deviation of the angle. Exceeding the threshold Then drift is determined. If true, perform re-identification; the determination formula is as follows:

[0063] Among them and These represent the width and height of the target bounding box, respectively. and To set the ratio; This represents a Boolean operation that determines whether the drift is true.

[0064] For example, this application may set a scaling factor. and The value can be set from 0.1 to 0.3, and the specific value is dynamically adjusted according to the target bounding box size. Angle threshold. This can be set from 10-30° to distinguish between normal deformation and abnormal rotation. During the calculation process, the width and height parameters of the target bounding box... w and h The detection results are taken from the current frame in real time to ensure that the threshold dynamically adapts to changes in target size. When any one of the three conditions is triggered, the Boolean logic operation immediately returns a true value, triggering the re-identification mechanism.

[0065] Assuming target bounding box size w* h is 100*80 pixels; scaling factor Set it to 0.2. Set to 0.15; Angle standard deviation threshold Set the angle to 30°. The drift state is determined as follows: 1) Calculate the mean distance. 25 pixels, from standard deviation 18 pixels, angular standard deviation It is 35°; 2) Calculate the judgment threshold: Distance to the mean threshold Pixels; Distance standard deviation threshold Pixel; 3) Perform conditional judgment: The first condition is met; The second condition is met; The third condition is met; 4) If any one of the three conditions is met, a drift is determined. t is true; 5) Finally, based on the judgment result, the system will perform target re-identification operation.

[0066] This scheme dynamically adjusts the judgment threshold based on the actual size of the target, improving its adaptability to targets of different sizes. Simultaneously, by comprehensively judging the target based on three dimensions—mean distance, standard deviation distance, and standard deviation angle—it ensures both the coverage of the judgment conditions and enhances its anti-interference capability. This multi-indicator joint judgment method effectively reduces false positives and false negatives that may be caused by a single indicator or fixed threshold, improving the accuracy and robustness of drift state recognition. Furthermore, this scheme can be implemented through simple statistical calculations, has low computational complexity, and is suitable for real-time operation on embedded platforms, thus improving the real-time performance and stability of target tracking.

[0067] In the above scheme, after determining the drift, it is necessary to re-identify the tracked target. The re-identification result redetermines the target bounding box region and determines the center position. Predicting the location of the tracked target in the current frame based on Kalman filtering. Based on the center position of re-identification and predicted tracking target location Determine the target distance between the two. The calculation process is as follows:

[0068] Among them This indicates the calculation of the square root.

[0069] The re-identification process can be implemented using deep feature-based matching algorithms or color histogram-based similarity comparison methods. For example, a ResNet-18 network can be used to extract target appearance features and rank them by similarity. When predicting location using Kalman filtering, the motion model parameters can be set to a constant velocity model, with the process noise covariance matrix Q ranging from [0.1, 1.0] and the observation noise covariance matrix R ranging from [0.5, 2.0]. During target distance calculation, the squared coordinate difference operation can be optimized using a fast lookup table method, and the square root calculation can be replaced with an approximate calculation function to reduce computation time.

[0070] The spatial difference between the re-identified center location and the predicted location is quantified using Euclidean distance, which forms a complementary constraint with the relevant response value. For example, when there are multiple candidate boxes in the re-identification result, the distance parameter can be set as a filtering threshold to retain only candidate results whose distance value is less than 50% of the diagonal length of the target box. The Kalman filter predicts the location by constraining spatial continuity through historical trajectories, and its prediction error range can be set to 10%-20% of the target box width.

[0071] Specifically, after detecting target box drift, the target detection model first performs a global search on the current frame to obtain multiple candidate target box regions and their center coordinates. Simultaneously, the Kalman filter predicts the target position in the current frame based on the motion state of the previous frame, generating predicted coordinates. A spatial displacement quantification index is formed by calculating the Euclidean distance between the center of each candidate target box region and the predicted position. This distance parameter, along with the relevant response values ​​of the candidate target boxes and the original template, constitutes a two-dimensional evaluation system, which is normalized through ratio calculation. For example, when a candidate box has a response value of 0.8 and a distance of 10 pixels, its ratio coefficient is 0.08, while another candidate box has a response value of 0.6 but a distance of 5 pixels, increasing the ratio coefficient to 0.12. Finally, the candidate target box with the largest ratio is selected as the correction result, ensuring both appearance matching and motion continuity requirements.

[0072] Furthermore, the process of determining the true target location in the current frame image by combining the re-identification results and the response map after relevant calculations using the tracking template can be achieved in the following way: A. Perform correlation calculations between the newly determined target bounding box region and the tracking template before re-identification to obtain their respective maximum response values. ; indicates the following:

[0073] B. Calculate the maximum response value for each. Distance from the target ratio ; the maximum ratio The corresponding recognition result is determined as the true target location; the calculation process is as follows:

[0074] Among them and Indicates the first The center pixel coordinates of each recognition result; Indicates sample With template Fourier transform of kernel correlation between them This indicates element-wise multiplication. The Fourier transform of the trained filter coefficients. This represents the inverse Fourier transform.

[0075] Specifically, the similarity between the candidate target bounding box region and the original tracking template is calculated using kernel correlation Fourier transform. The features of the sample and template are then subjected to fast convolution in the Fourier domain to generate a frequency domain response map. An inverse Fourier transform converts the frequency domain response to the spatial domain, obtaining a quantified value of the matching degree between each candidate region and the template. The maximum response value is extracted as the apparent consistency index between the candidate region and the template. Simultaneously, a normalized ratio index is constructed by combining the spatial distance between the Kalman filter predicted position and the re-identification candidate center. The distance parameter is calculated using the Euclidean distance formula to suppress candidate regions with excessive spatial offset. The ratio index calculation combines apparent matching degree with spatial continuity constraints, eliminating the risk of misjudgment that may be introduced by a single index. Finally, by comparing the ratio parameters of all candidates, the candidate with the maximum ratio is selected as the true target position, ensuring optimal apparent matching and continuous motion trajectory. The trained filter coefficients are learned through a large number of samples to enhance robustness to target deformation and illumination changes. For example, assuming there are three re-identification results, their maximum response values... The target distances are 0.8, 0.7, and 0.9 respectively. The values ​​are 10, 8, and 15 pixels respectively. The calculated ratios... The ratios were 0.08, 0.0875, and 0.06, respectively. Since the second result had the largest ratio, it was selected as the true target location.

[0076] Through the above technical solutions, this application effectively solves the problem of target location misjudgment in multi-candidate scenarios. By fusing the re-identification results with the apparent similarity and spatial distance constraints of the tracking template, the accuracy of target localization is improved. The calculation methods of kernel correlation and Fourier transform enhance the efficiency of feature matching. The introduction of filter coefficients enhances the discriminative ability between the template and the sample, making the response value calculation more robust against interference. By calculating the ratio of the response value to the distance, the reliability of apparent feature matching is ensured, and the continuity of target motion is controlled, forming a dual verification mechanism against drift phenomena. This method can more accurately locate the real target in complex backgrounds and with multi-target interference, reducing drift and misjudgment during tracking, and improving the overall performance and stability of target tracking.

[0077] Figure 6 This paper shows a structural block diagram of a target bounding box automatic tracking drift determination device provided in an embodiment of this application. The device includes: The conversion module 610 is used to cache two adjacent frames of images and extract the target bounding box region from the images, and convert the target bounding box region image into a grayscale image. The corner point calculation module 620 is used to perform corner point detection on the previous frame grayscale image and extract the first corner point group within the target box area; perform optical flow calculation on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and match the second corner point group of the current frame grayscale image; the corner point group contains the target feature corner points of the corresponding grayscale image; The drift determination module 630 is used to determine the drift state of the tracking target corresponding to the current frame image based on the distance and angle information of the successfully matched feature corner pairs in the corner point group. The tracking update module 640 is used to predict the target position in the current frame image based on Kalman filtering and to re-identify the tracked target when it is determined that the tracked target has drifted. It then combines the re-identification result with the response map after relevant calculations using the tracking template to determine the true target position in the current frame image.

[0078] It should be noted that the target bounding box automatic tracking drift determination device provided in this embodiment is only an example of the above-described division of functional modules / units. In practical applications, the above functions can be assigned to different functional modules / units as needed, that is, the internal structure of the target bounding box automatic tracking drift determination device can be divided into different functional modules / units to complete all or part of the functions described above. Furthermore, the implementation method of the target bounding box automatic tracking drift determination method provided in the above method embodiments and the implementation method of the target bounding box automatic tracking drift determination device provided in this embodiment belong to the same concept. The specific implementation process of the target bounding box automatic tracking drift determination device provided in this embodiment is detailed in the above method embodiments and will not be repeated here.

[0079] Figure 7This illustration shows a structural block diagram of a computer device provided in an exemplary embodiment of this application. The device includes desktop computers, laptops, handheld computers, and cloud servers. This computer device may include, but is not limited to, a processor and memory. The processor and memory can be connected via a bus or other means. The processor may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, graphics processing units (GPUs), embedded neural network processing units (NPUs) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0080] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0081] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the above embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] In some embodiments, the computer device may also optionally include: a peripheral device interface and at least one peripheral device. The processor, memory, and peripheral device interface can be connected via a bus or signal lines. Each peripheral device can be connected to the peripheral device interface via a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit, a display screen, and a keyboard.

[0083] Peripheral device interfaces can be used to connect at least one I / O (Input / Output) related peripheral device to the processor and memory. In some embodiments, the processor, memory, and peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, memory, and peripheral device interface can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0084] The display screen is used to display the UI (User Interface). This UI can include graphics, text, icons, videos, and any combination thereof. When the display screen is a touch screen, it also has the ability to collect touch signals on or above the surface of the display. These touch signals can be input as control signals to a processor for processing. In this case, the display screen can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen, located on the front panel of the computer device; in other embodiments, there may be at least two display screens, respectively located on different surfaces of the computer device or in a folded design; in still other embodiments, the display screen may be a flexible display screen, located on a curved or folded surface of the computer device. Furthermore, the display screen can be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0085] A power supply is used to power the various components in a computer device. The power supply can be alternating current (AC), direct current (DC), a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is charged via a wired connection, while a wireless rechargeable battery is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0086] Those skilled in the art will understand that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0087] This application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, which, when executed by a processor, implements the methods described in the above-described method embodiments. Those skilled in the art will understand that implementing all or part of the processes in the methods described in the above-described embodiments of this application can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0088] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.

Claims

1. A method for automatically tracking and determining target bounding box drift, characterized in that, The method includes: Cache two adjacent frames of images and extract the target bounding box region from the images, then convert the target bounding box region image into a grayscale image; Corner detection is performed on the previous frame grayscale image to extract the first corner point group within the target bounding box; optical flow calculation is performed on the current frame grayscale image based on the previous frame grayscale image and the first corner point group, and the second corner point group of the current frame grayscale image is matched; the corner point group contains the target feature corner points of the corresponding grayscale image; specifically, the previous frame grayscale image is extracted. Based on the feature values ​​of each pixel, calculate the feature corner response value and corner filtering threshold of the current frame grayscale image at each pixel, using the following formula: Among them and Indicates in Feature values ​​at a pixel Indicates in The minimum eigenvalue at pixel coordinates, i.e. Feature corner response values ​​of a pixel; This indicates the set quality level parameter. Represents the previous frame grayscale image The response value of the largest feature corner point in the middle, where T represents the corner point filtering threshold; Filter out the previous frame grayscale image Candidate corner points whose response values ​​are greater than T; The candidate corner points are sorted in descending order of response value to form a sequence {p1,p2,p3…pi}; Starting from the first candidate corner point p1 in the sequence, calculate the distance from p1 to all other candidate feature corner points. d Delete sequences whose distance is less than a threshold The first candidate corner point is selected; then, the second candidate corner point is selected, and the distance from the remaining candidate corner points with response values ​​less than the second candidate corner point to the second candidate corner point is calculated. This process is repeated to determine the target feature corner point among the candidate feature corner points. All the target feature corner points form the first corner point group. ; Based on the distance and angle information of successfully matched feature corner pairs in the corner point group, the drift state of the tracking target corresponding to the current frame image is determined; specifically, the total number of feature corner points and the number of successfully matched points in the first corner point group are determined, and the drift state of the tracking target is determined according to the number of successfully matched points; wherein, when the number of successfully matched points does not exceed the target proportion of the total number of feature corner points, it is determined that the tracking target has drifted. When the number of successful pairings exceeds the target proportion of the total number of feature corner points, calculate the distance and angle information values ​​of the feature corner point pairs between the current frame grayscale image and the previous frame grayscale image; The distance and angle information values ​​of the feature corner point pairs are sorted in ascending order, and the data with the target proportions before and after the median are selected to form distance arrays. and angle array ; Based on distance array and angle array Determine the drift state of the tracked target; When the tracking target is determined to have drifted, the target position in the current frame image is predicted based on Kalman filtering and the tracking target is re-identified. The response map after relevant calculations are performed by combining the re-identification result and the tracking template to determine the true target position in the current frame image.

2. The method according to claim 1, characterized in that, The distance array and angle array Determine the drift state of the tracked target, including: Calculate the distance array respectively and angle array The mean and standard deviation; If the distance to the mean or standard deviation Exceeds the preset proportion of the target bounding box, or the standard deviation of the angle. Exceeding the threshold Then drift is determined. If true, perform re-identification; the determination formula is as follows: Among them and These represent the width and height of the target bounding box, respectively. and This is a preset ratio; This represents a Boolean operation that determines whether the drift is true.

3. The method according to claim 1 or 2, characterized in that, The method of predicting the target position in the current frame's grayscale image based on Kalman filtering and performing target re-identification includes: When target bounding box drift is confirmed, the tracked target is re-identified; the re-identification result redetermines the target bounding box region and determines its center position. ; Predicting the position of the tracked target in the current frame based on Kalman filtering. Based on the center position of re-identification and predicted tracking target location Determine the target distance between the two. The calculation process is as follows: Among them This indicates the calculation of the square root.

4. The method according to claim 3, characterized in that, The response map, obtained by combining the re-identification result and the tracking template for correlation calculation, determines the true target location in the current frame image, including: The newly determined target bounding box region is correlated with the tracking template before re-identification to obtain their respective maximum response values. ; indicates the following: Calculate their respective maximum response values Distance from the target ratio ; the maximum ratio The corresponding recognition result is determined as the true target location; the calculation process is as follows: Among them and Indicates the first The center pixel coordinates of each recognition result; Indicates sample With template Fourier transform of kernel correlation between them This indicates element-wise multiplication. The Fourier transform of the trained filter coefficients. This represents the inverse Fourier transform.

5. A target bounding box automatic tracking drift judgment device, characterized in that, The device includes: The conversion module is used to cache two adjacent frames of images and extract the target bounding box region from the images, and convert the target bounding box region image into a grayscale image. The corner point calculation module is used to detect corner points on the previous frame grayscale image and extract the first corner point group within the target bounding box region; based on the previous frame grayscale image and the first corner point group, optical flow calculation is performed on the current frame grayscale image, and a second corner point group is matched in the current frame grayscale image; the corner point group contains the target feature corner points of the corresponding grayscale image; specifically, the previous frame grayscale image is extracted. Based on the feature values ​​of each pixel, calculate the feature corner response value and corner filtering threshold of the current frame grayscale image at each pixel, using the following formula: Among them and Indicates in Feature values ​​at a pixel Indicates in The minimum eigenvalue at pixel coordinates, i.e. Feature corner response values ​​of a pixel; This indicates the set quality level parameter. Represents the previous frame grayscale image The response value of the largest feature corner point in the middle, where T represents the corner point filtering threshold; Filter out the previous frame grayscale image Candidate corner points whose response values ​​are greater than T; The candidate corner points are sorted in descending order of response value to form a sequence {p1,p2,p3…pi}; Starting from the first candidate corner point p1 in the sequence, calculate the distance from p1 to all other candidate feature corner points. d Delete sequences whose distance is less than a threshold The first candidate corner point is selected; then, the second candidate corner point is selected, and the distance from the remaining candidate corner points with response values ​​less than the second candidate corner point to the second candidate corner point is calculated. This process is repeated to determine the target feature corner point among the candidate feature corner points. All the target feature corner points form the first corner point group. ; The drift determination module is used to determine the drift state of the tracking target in the current frame image based on the distance and angle information of the successfully matched feature corner pairs in the corner point group; specifically, it determines the total number of feature corner points and the number of successfully matched pairs in the first corner point group, and determines the drift state of the tracking target based on the number of successfully matched pairs; wherein, when the number of successfully matched pairs does not exceed the target proportion of the total number of feature corner points, it is determined that the tracking target has drifted. When the number of successful pairings exceeds the target proportion of the total number of feature corner points, calculate the distance and angle information values ​​of the feature corner point pairs between the current frame grayscale image and the previous frame grayscale image; The distance and angle information values ​​of the feature corner point pairs are sorted in ascending order, and the data before and after the median target ratio are selected to form distance arrays. and angle array ; Based on distance array and angle array Determine the drift state of the tracked target; The tracking update module is used to predict the target position in the current frame image and perform target re-identification based on Kalman filtering when the tracking target is determined to have drifted. It then combines the re-identification result with the response map after relevant calculations using the tracking template to determine the true target position in the current frame image.

6. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the target box automatic tracking drift judgment method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the target box automatic tracking drift judgment method as described in any one of claims 1 to 4.

Images