A real-time tracking method and device for high-speed moving targets based on region expansion

Abstract

The application discloses a kind of real-time tracking method and device of high-speed moving target based on region expansion, and it is related to high-speed target tracking technical field.The method includes: initial target detection, calculating position similarity, calculating motion similarity, fitting parabola and tracking target frame updating step.The real-time tracking method and device of high-speed moving target based on region expansion provided by the application, through the method of model post-processing, the motion similarity and position similarity of current frame and last frame are calculated by expanding target detection frame, and the purpose of tracking is achieved by combining parabola fitting, the tracking of high-speed moving target can be realized, effectively filter mis-tracking target, both avoid the target loss problem caused by light using image processing method, and the purpose of real-time tracking can be achieved, meet industrial requirements.

Description

Technical Field

[0001] This invention relates to the field of high-speed target tracking technology, and in particular to a real-time tracking method and apparatus for high-speed moving targets based on region expansion. Background Technology

[0002] In smart sports applications involving parabolic ball sports, multiple balls often appear on the field. Combined with the potential for false detections by machine vision detection models, accurately identifying the valid ball becomes a major challenge. Furthermore, some ball sports move at extremely high speeds, with significant distances between their positions in consecutive frames, making tracking even more difficult.

[0003] Common solutions in existing technologies include: 1. using high-speed cameras to capture the ball's trajectory; 2. using deep learning models for tracking. While high-speed cameras or deep learning models can solve some problems, in practical applications, these two methods are often impractical due to cost and time constraints. High-speed cameras are expensive and easily affected by environmental factors such as lighting, resulting in unstable tracking; while deep learning models involve large amounts of computational inference, making real-time tracking difficult. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a real-time tracking method and device for high-speed moving targets based on region expansion. The method uses a post-processing algorithm to achieve target tracking, which has low requirements for the camera and achieves good real-time tracking effect, thereby reducing application costs.

[0005] In a first aspect, the present invention provides a real-time tracking method for high-speed moving targets based on region expansion, comprising:

[0006] Initial target detection: Based on the target detection algorithm, the video frames are continuously detected, and an initial tracking target box is determined according to the actual project calibration in at least one detected target box;

[0007] Calculate positional similarity: Expand the target detection box of the current frame to a region with a specified width and height as the IOU comparison region A. n Then, the intersection-union ratio (IoU) with the expanded region B of the tracking target box in the previous frame is calculated, i.e., the positional similarity. r ;

[0008] Motion similarity calculation: Based on the tracking bounding box of the previous frame, the target position in the current frame is predicted using Kalman filtering, and then expanded to a region of specified width and height as the IOU comparison region K. The intersection-union ratio (IoU) with the expanded region B of the tracking bounding box of the previous frame is then calculated, which is the motion similarity IoU. k ;

[0009] Fitting a parabola: Based on the tracking information of the previous N frames, fit the parabolic motion of the sphere, and calculate the predicted output y of the parabola using the actual detected coordinates of the current frame. s With the detected coordinate value y r The difference y o ;

[0010] Tracking bounding box updates: based on positional similarity (iou) r Motion similarity (iou) k and y o Make a comprehensive judgment and update the tracking target bounding box.

[0011] Furthermore, the update of the tracking target bounding box specifically includes:

[0012] If IOU r If there are no elements in the set that are greater than the first set value, the Kalman filter prediction box is directly updated to the tracking target box;

[0013] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value and there is only one detection box, then update the detection box to the tracking target box.

[0014] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value, but there are multiple detection boxes, then the minimum value of y is taken. o The corresponding detection box is updated to the tracking target box;

[0015] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o If the value is less than or equal to the third set value, then the detection box will be updated to the tracking target box;

[0016] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o The third setting will update the tracking target box of the previous frame to the tracking target box of the current frame.

[0017] Furthermore, calculating positional similarity, calculating motion similarity, and fitting the parabola can be performed simultaneously without any order of priority.

[0018] Furthermore, in the steps of calculating positional similarity and calculating motion similarity, expanding the region to a specified width and height specifically includes: taking the center point coordinates of the frame as the center, expanding the width and height to 7 times the original frame width and height to obtain the region with the specified width and height.

[0019] Secondly, the present invention provides a real-time tracking device for high-speed moving targets based on region expansion, comprising:

[0020] The initial target detection module is used to continuously detect video frames based on the target detection algorithm, and determine an initial tracking target box based on the actual project calibration in at least one detected target detection box;

[0021] The position similarity calculation module is used to expand the target detection box of the current frame to a region of specified width and height as the IOU comparison region A. n Then, the intersection-union ratio (IoU) with the expanded region B of the tracking target box in the previous frame is calculated, i.e., the positional similarity. r ;

[0022] The motion similarity calculation module predicts the target position in the current frame using Kalman filtering based on the tracked target box from the previous frame. It then expands this prediction into a region of specified width and height as the IoU comparison region K. Finally, it calculates the intersection-union ratio (IoU) with the expanded region B of the tracked target box from the previous frame. k ;

[0023] The parabola fitting module is used to fit the parabola of the sphere's motion based on the tracking information of the previous N frames. It calculates the parabola prediction output y using the actual detected coordinates of the current frame. s With the detected coordinate value y r The difference y o ;

[0024] The target bounding box update module is used to update the bounding box based on location similarity (ioU). r Motion similarity (iou) k and y o Make a comprehensive judgment and update the tracking target bounding box.

[0025] Furthermore, the tracking target box update module is specifically used to perform the following steps:

[0026] If IOU r If there are no elements in the set that are greater than the first set value, the Kalman filter prediction box is directly updated to the tracking target box;

[0027] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value and there is only one detection box, then update the detection box to the tracking target box.

[0028] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value, but there are multiple detection boxes, then the minimum value of y is taken. o The corresponding detection box is updated to the tracking target box;

[0029] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o If the value is less than or equal to the third set value, then the detection box will be updated to the tracking target box;

[0030] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o The third setting will update the tracking target box of the previous frame to the tracking target box of the current frame.

[0031] Furthermore, the position similarity calculation module, motion similarity calculation module, and parabolic fitting module can be used simultaneously to process the same video frame.

[0032] Furthermore, in the position similarity calculation module and the motion similarity calculation module, expanding the region to a specified width and height specifically includes: taking the center point coordinates of the frame as the center, expanding the width and height to 7 times the original frame width and height to obtain a region with a specified width and height.

[0033] The technical solutions provided in the embodiments of the present invention have at least the following technical effects:

[0034] Based on the output of the target detection model, the post-processing method expands the target detection box to calculate the motion similarity and positional similarity between the current frame and the previous frame, and combines it with parabolic fitting to achieve the tracking purpose. This method can track high-speed moving targets and effectively filter out false targets. It avoids the target loss problem caused by light when using image processing methods, and can achieve real-time tracking, meeting industrial requirements.

[0035] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] Figure 1 This is a flowchart illustrating the overall process of the method in Embodiment 1 of the present invention.

[0038] Figure 2 This is a detailed flowchart of the method in Embodiment 1 of the present invention;

[0039] Figure 3This is a schematic diagram of the initial target detection in Embodiment 1 of the present invention;

[0040] Figure 4 This is a schematic diagram of position similarity calculation in Embodiment 1 of the present invention;

[0041] Figure 5 This is a schematic diagram of motion similarity calculation in Embodiment 1 of the present invention;

[0042] Figure 6 This is a schematic diagram illustrating the calculation of parabolic deviation in Embodiment 1 of the present invention;

[0043] Figure 7 This is a schematic diagram of the device in Embodiment 2 of the present invention. Detailed Implementation

[0044] This invention provides a real-time tracking method and apparatus for high-speed moving targets based on region expansion. It uses a post-processing algorithm to achieve target tracking, which does not require high-performance cameras and achieves good real-time tracking results, thereby reducing application costs.

[0045] The overall concept of the technical solutions in the embodiments of the present invention is as follows:

[0046] Based on the output of the target detection model, this embodiment of the invention uses a post-processing method to expand the target detection box by a certain factor and calculate the motion similarity and position similarity between the current frame and the previous frame to achieve the tracking purpose. At the same time, based on the coordinate information of the tracked target, parabolic coordinates are fitted for filtering out abnormal situations, and the missed detection situations are effectively supplemented without increasing the amount of additional calculation.

[0047] Example 1

[0048] This embodiment provides a real-time tracking method for high-speed moving targets based on region expansion, such as... Figure 1 As shown, it includes:

[0049] Initial target detection: Based on the target detection algorithm, the video frames are continuously detected, and an initial tracking target box is determined according to the actual project calibration in at least one detected target box.

[0050] Calculate positional similarity: Expand the target detection box of the current frame to a region with a specified width and height as the IOU comparison region A. n Then, the intersection-union ratio (IoU) with the expanded region B of the tracking target box in the previous frame is calculated, i.e., the positional similarity. r .

[0051] Motion similarity calculation: Based on the tracking bounding box of the previous frame, the target position in the current frame is predicted using Kalman filtering, and then expanded to a region of specified width and height as the IOU comparison region K. The intersection-union ratio (IoU) with the expanded region B of the tracking bounding box of the previous frame is then calculated, which is the motion similarity IoU. k .

[0052] Fitting a parabola: Based on the tracking information of the previous N frames (e.g., N=5), fit the parabola of the sphere's motion. Calculate the predicted output y of the parabola using the actual detected coordinates of the current frame. s With the detected coordinate value y r The difference y o .

[0053] Tracking bounding box updates: based on positional similarity (iou) r Motion similarity (iou) k and y o Make a comprehensive judgment and update the tracking target bounding box.

[0054] The steps of calculating positional similarity, calculating motion similarity, and fitting the parabola described above are not sequential and can be performed simultaneously.

[0055] In a specific practical example, such as Figure 2 As shown, the specific steps include the following:

[0056] S1. Initial Target Detection: Based on the actual project calibration and the output of the target detection algorithm, determine the initial tracking target bounding box, such as... Figure 3 As shown, the target box in the upper right corner is selected as the initial tracking target box according to the actual project. The initial tracking target box is the comparison benchmark for the post-processing method in the next frame. After the tracking target box in the next frame is updated by the post-processing method, it is used as the comparison benchmark for the post-processing method in the next frame after that.

[0057] S2. Calculate positional similarity: Using the center point coordinates of the detection box as the center, expand the width and height to 7 times the width and height of the detection box to obtain the comparison region A. n Each comparison region A n Calculate IOU with the expanded region B of the tracking target box in the previous frame. r ,like Figure 4 As shown, then output iou. rn :

[0058] iou rn =(A n ∩B) / (A n ∪B)

[0059] Where n is the number of target detection boxes in the current frame, and the intersection-union ratio (IUU) of each target box with the expanded region B of the tracked target box in the previous frame is calculated.

[0060] iourn This is used to determine the positional similarity between the detection box in the current frame and the initial or updated tracking target box in the previous frame, and to quickly filter out detection boxes with small intersections.

[0061] S3. Calculate motion similarity: Based on the target bounding box of the previous frame, predict the position of the next frame using Kalman filtering. Simultaneously, expand the Kalman prediction region by a factor of 7 as the IOU comparison region K (centered on the center point coordinates of the Kalman filter predicted bounding box, expand the width and height to 7 times the size of the detected bounding box to obtain the comparison region K). Figure 5 As shown, the IOU is calculated with the expanded region B of the previous frame, and the IOU is output. k :

[0062] K = Kalman(box) * 7

[0063] iou k This is used to quickly filter out detection boxes whose positions overlap with the tracking target box positions in the previous frame but do not conform to the predicted motion path region.

[0064] S4. Parabola Fitting: Based on the tracking information of the previous 5 frames, fit the parabola of the sphere's motion. Calculate the parabola prediction output y using the actual detected coordinates of the current frame. s With the detected coordinate value y r The difference y o ,like Figure 6 As shown.

[0065] Through the difference y o It can finely filter out detection boxes that are very close to the actual target detection boxes.

[0066] S5. Anomaly detection and target bounding box update:

[0067] 5.1 If iou r If there are no elements greater than 0.7 in the set, it indicates that there are missed detections. Therefore, the Kalman filter prediction box is directly updated to the tracking target box.

[0068] 5.2 If iou r If the set contains elements greater than 0.7, then iou k If the value is ≥0.5 and there is only one detection box, then update the detection box to the tracking target box;

[0069] 5.3 If iou r If the set contains elements greater than 0.7, then iou k If the value is ≥0.5, but there are multiple detection boxes, then the minimum value of y is taken. o The corresponding detection box is updated to the tracking target box;

[0070] 5.4 If iou rIf the set contains elements greater than 0.7, then iou k <0.5, minimum y o If the value is ≤10 pixels, then update the detection box to the tracking target box;

[0071] 5.5, if iou r If the set contains elements greater than 0.7, then iou k <0.5, minimum y o If the value is greater than 10 pixels, the tracking bounding box of the previous frame will be updated to the tracking bounding box of the current frame.

[0072] This embodiment achieves real-time 30FPS tracking using a common industrial camera, significantly reducing application costs. Furthermore, this embodiment compensates for missed detections during target detection; using this method, the missed detection rate is reduced by 1.3%, and the effective ball tracking rate is increased by 10%, meeting the requirements of industrial deployment.

[0073] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0074] Example 2

[0075] This embodiment provides a real-time tracking device for high-speed moving targets based on region expansion, such as... Figure 7 As shown, it includes:

[0076] The initial target detection module is used to continuously detect video frames based on the target detection algorithm, and determine an initial tracking target box based on the actual project calibration in at least one detected target detection box;

[0077] The position similarity calculation module is used to expand the target detection box of the current frame to a region of specified width and height as the IOU comparison region A. n Then, the intersection-union ratio (IoU) with the expanded region B of the tracking target box in the previous frame is calculated, i.e., the positional similarity. r ;

[0078] The motion similarity calculation module predicts the target position in the current frame using Kalman filtering based on the tracked target box from the previous frame. It then expands this prediction into a region of specified width and height as the IoU comparison region K. Finally, it calculates the intersection-union ratio (IoU) with the expanded region B of the tracked target box from the previous frame. k ;

[0079] The parabola fitting module is used to fit the parabola of the sphere's motion based on the tracking information of the previous N frames. It calculates the parabola prediction output y using the actual detected coordinates of the current frame. s With the detected coordinate value y r The difference y o ;

[0080] The target bounding box update module is used to update the bounding box based on location similarity (ioU). r Motion similarity (iou) k and y o Make a comprehensive judgment and update the tracking target bounding box.

[0081] In one possible implementation, the tracking target box update module is specifically used to perform the following steps:

[0082] If IOU r If there are no elements in the set that are greater than the first set value, the Kalman filter prediction box is directly updated to the tracking target box;

[0083] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value and there is only one detection box, then update the detection box to the tracking target box.

[0084] If IOU r If there are elements in the set that are greater than the first set value, then iou k If the value is greater than or equal to the second set value, but there are multiple detection boxes, then the minimum value of y is taken. o The corresponding detection box is updated to the tracking target box;

[0085] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o If the value is less than or equal to the third set value, then the detection box will be updated to the tracking target box;

[0086] If IOU r If there are elements in the set that are greater than the first set value, then iou k <Second setting value, minimum y> o The third setting will update the tracking target box of the previous frame to the tracking target box of the current frame.

[0087] The position similarity calculation module, motion similarity calculation module, and parabolic fitting module can be used simultaneously to process the same video frame.

[0088] In the position similarity calculation module and motion similarity calculation module, expanding the region to a specified width and height specifically includes: taking the center point coordinates of the frame as the center, expanding the width and height to 7 times the original frame width and height to obtain a region with a specified width and height.

[0089] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0090] This invention, based on the output of a target detection model, uses post-processing to expand the target detection box, calculate the motion and positional similarities between the current and previous frames, and combine this with parabolic fitting to achieve tracking. This enables the tracking of high-speed moving targets, effectively filtering out falsely tracked targets. It avoids the target loss problems caused by lighting conditions inherent in image processing methods, while achieving real-time tracking and meeting industrial requirements.

[0091] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for real-time tracking of high-speed moving targets based on region expansion, characterized in that: Comprise: Initial target detection: based on target detection algorithm, continuously detecting video frames, and determining an initial tracking target frame in at least one target detection frame according to actual target calibration; Calculate position similarity: expand the target detection box of the current frame to a region with a specified width and height as the iou comparison region A n Then calculate the intersection over union with the expanded region B of the last frame tracking target box, that is, the position similarity iou r ; Calculate motion similarity: according to the tracking target frame of the last frame, predict the target position of the current frame through Kalman filtering, then expand to a region with specified width and height as the iou comparison region K, and then calculate the intersection over union with the expanded region B of the tracking target frame of the last frame, that is, the motion similarity iou k ; Fitting parabola: based on the tracking information of the previous N frames, a parabola is fitted for the ball movement, and the difference y s between the actual detection coordinate value y r and the parabola prediction output y o is calculated Tracking target frame update: according to position similarity iou r , motion similarity iou k and y o Comprehensive judgment and update tracking target frame; Wherein, the tracking target frame updating specifically comprises: If IOU r If there are no elements in the set that are greater than the first set value, the Kalman filter prediction box is directly updated to the tracking target box; If Iou r There are elements greater than the first set value in the set, Iou k ≥ The second set value, and only one detection box, update the detection box to the tracking target box; If iou r There are elements greater than the first set value in the set, iou k ≥ The second set value, but there are multiple detection boxes, take the minimum y o The corresponding detection box is updated to the tracking target box; If Iou r There are elements greater than the first set value in the set, Iou k < The second set value, minimum y o ≤ The third set value, the detection frame is updated to the tracking target frame; If the IOU r There are elements greater than the first set value in the set, IOU k < The second set value, the minimum y o > The third set value, the last frame tracking target frame is updated to the tracking target frame of the current frame. where iou r The set includes each of the matching regions A n The corresponding iou r .

2. The method of claim 1, wherein: The calculation of position similarity, the calculation of motion similarity and the fitting of parabola can be carried out simultaneously without any priority.

3. The method of claim 1, wherein: In the steps of calculating position similarity and calculating motion similarity, expanding to a region with specified width and height specifically comprises: taking the center point coordinate of the frame as the center, expanding the width and height to 7 times of the original frame width and height to obtain a region with specified width and height.

4. A real-time tracking device for high speed moving targets based on region expansion, characterized in that, Comprise: An initial target detection module, configured to continuously detect video frames based on a target detection algorithm, and determine an initial tracking target frame in at least one target detection frame according to actual target calibration; The position similarity calculation module is configured to expand the target detection frame of the current frame into a region with a specified width and height as an IOU comparison region A n Then, the IOU of the expanded region B of the target frame of the previous frame is calculated, that is, the position similarity IOU r ; The computing motion similarity module is configured to: according to a tracking target frame of a previous frame, predict a target position of a current frame through Kalman filtering, then expand the target position into a region with a specified width and height as an IOU comparison region K, and then calculate an intersection over union between the IOU comparison region K and an expanded region B of a tracking target frame of the previous frame, that is, a motion similarity IOU. k ; a parabolic fitting module, configured to fit a parabolic line of a ball movement based on the previous N frames of tracking information, and calculate a predicted output y of the parabolic line by an actual detected coordinate of a current frame s and the difference y r between the detected coordinate value y o ; The tracking target frame updating module is configured to update the tracking target frame according to the position similarity iou r , the motion similarity iou k , and y o comprehensively judge and update the tracking target frame. Wherein, the tracking target frame updating module is specifically configured to perform the following steps: If Iou r If there is no element greater than the first set value in the set, the Kalman filter prediction frame is directly updated to the tracking target frame. If Iou r There are elements greater than the first set value in the set, Iou k ≥ The second set value, and only one detection box, update the detection box to the tracking target box; If iou r There are elements greater than the first set value in the set, iou k ≥ The second set value, but there are multiple detection boxes, take the minimum y o The corresponding detection box is updated to the tracking target box; If Iou r There are elements greater than the first set value in the set, Iou k < The second set value, minimum y o ≤ The third set value, update the detection frame to the tracking target frame; If the IOU r There are elements greater than the first set value in the set, IOU k < The second set value, the minimum y o > The third set value, the last frame tracking target frame is updated to the tracking target frame of the current frame. where iou r The set includes each of the matching regions A n The corresponding iou r .

5. The apparatus of claim 4, wherein: The position similarity calculation module, the motion similarity calculation module and the parabola fitting module can be used to process the same video frame simultaneously.

6. The apparatus of claim 4, wherein: In the position similarity calculation module and the motion similarity calculation module, expanding to a region with specified width and height specifically comprises: taking the center point coordinate of the frame as the center, expanding the width and height to 7 times of the original frame width and height to obtain a region with specified width and height.

Images