Method, device and storage medium for tracking a moving object

Abstract

The application discloses a motion target tracking method, device and storage medium, and relates to the technical field of computer vision. The motion target tracking method comprises the following steps: when motion target tracking is needed, determining the tracking state of the motion target; based on the tracking state, determining the current tracking detection mode; if the tracking detection mode is the close-up tracking mode, tracking the motion target based on the tracking state. The application detects the motion target through the panoramic detection mode and tracks the motion target through the close-up tracking mode. Since the resolution of the close-up tracking is lower than that of the panoramic detection, and the panoramic detection only detects the motion target but does not track the motion target, the application does not need to track the motion target at high resolution, but can track the motion target at low resolution through the close-up tracking mode, so that the computing power required for detecting the motion target in a large area can be reduced.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to methods, devices and storage media for tracking moving targets. Background Technology

[0002] Moving target tracking is a task that involves locking onto and following the position of a specific target in a continuous sequence of images. Current moving target tracking tasks often require tracking and detecting moving targets over a large area.

[0003] Currently, the common practice is to use equipment to capture a panoramic view of the area to be detected, and then track moving targets within this panoramic view. However, methods that perform detection within a panoramic view require high resolution. When using low-powered equipment, it is impossible to detect moving targets over large areas, resulting in the high computational power required for current methods to detect moving targets over large areas. Summary of the Invention

[0004] The main objective of this application is to provide a moving target tracking method, device, and storage medium, which aims to solve the technical problem of high computing power required for moving target detection over a large area.

[0005] To achieve the above objectives, this application proposes a moving target tracking method, the method comprising:

[0006] When it is necessary to track moving targets, determine the tracking status of the moving targets to be tracked;

[0007] Based on the tracking state, the current tracking detection mode is determined, wherein the tracking detection mode includes a panoramic detection mode and a close-up tracking mode. The panoramic detection mode only detects the moving target, and the resolution of the close-up tracking mode is lower than that of the panoramic detection mode.

[0008] If the tracking detection mode is the close-up tracking mode, then the moving target is tracked based on the tracking state.

[0009] In one embodiment, the tracking state includes a target loss state, and the step of determining the current tracking detection mode based on the tracking state includes:

[0010] If the tracking and detection mode is the panoramic detection mode, when the number of times the moving target is detected reaches a preset number, the position of the moving target at the time of the most recent detection is determined;

[0011] The tracking detection mode is converted to the close-up tracking mode, so as to track the moving target at the position of the moving target in the close-up tracking mode;

[0012] If the tracking detection mode is the close-up tracking mode, and the tracking state is the target loss state for a period of time that reaches a preset loss time threshold, then the tracking detection mode is switched to the panoramic detection mode to detect the moving target in the panoramic detection mode.

[0013] In one embodiment, the tracking state further includes maintaining the tracking state, the moving target includes a first moving target and a second moving target, and the step of tracking the moving target based on the tracking state, if the tracking detection mode is the close-up tracking mode, includes:

[0014] If both the first tracking state of the first moving target and the second tracking state of the second moving target are in the tracking-maintaining state, then the first position of the first moving target is determined based on a preset queue difference method, and the second position of the second moving target is determined based on a preset Kalman tracking method.

[0015] The moving target is tracked based on the first position, the second position, and the center position of the tracking screen during tracking.

[0016] In one embodiment, the step of tracking the moving target based on the first position, the second position, and the center position of the tracking frame during tracking includes:

[0017] Based on the first position and the center position of the image, it is determined whether the first moving target is at the edge of the tracked image;

[0018] If not, the target decision center position for moving target tracking is determined based on the first position, the second position, the center position of the image, and a preset power function.

[0019] Based on the position difference between the target decision center position and the image center position, the tracking position and tracking speed of the moving target are determined;

[0020] The moving target is tracked based on the tracking position and tracking speed.

[0021] In one embodiment, the step of determining the target decision center position for moving target tracking based on the first position, the second position, the center position of the image, and a preset power function if the target is not in the specified position includes:

[0022] Based on the power function, the first position, and the center position of the screen, the horizontal and vertical weights of the first moving target are calculated, and the larger of the horizontal and vertical weights is determined as the decision weight of the first moving target.

[0023] Based on the decision weights, the first position, and the second position, calculate the current decision center position at the current moment;

[0024] The current decision center location is added to a decision center queue consisting of multiple decision center locations, wherein the multiple decision center locations include decision center locations prior to the current time.

[0025] The target decision center location is obtained by calculating the average of the multiple decision center locations and the current decision center location.

[0026] In one embodiment, after determining whether the first moving target is at the edge of the tracked image based on the first position and the center position of the image, the method further includes:

[0027] If it is in the target decision center, the decision weight is set to the maximum value, and the first position is determined as the target decision center.

[0028] Add the target decision center to the decision center queue;

[0029] Based on the position difference between the target decision center position and the image center position, the tracking position and tracking speed of the moving target are determined;

[0030] The moving target is tracked based on the tracking position and tracking speed.

[0031] In one embodiment, if the tracking detection mode is the close-up tracking mode, then the step of tracking the moving target based on the tracking state includes:

[0032] If the first tracking state is the target loss state and the second tracking state is the keep-tracking state, then the loss duration of the first tracking state being the target loss state is determined.

[0033] Based on the duration of the loss, a corresponding target tracking decision is determined;

[0034] Based on the target tracking decision, the moving target is tracked.

[0035] In one embodiment, the second moving target includes a plurality of second moving sub-targets, each second moving sub-target having a corresponding sub-target position, and the step of determining the corresponding target tracking decision based on the loss duration includes:

[0036] If the loss duration is lower than a preset first threshold, the average value of the sub-target locations is calculated, and the average value of the sub-target locations is used as the current decision center location.

[0037] The current decision center location is added to the decision center queue, and the average value of the multiple decision center locations and the current decision center location is calculated to obtain the target decision center location;

[0038] Based on the target decision center location and a preset number of second moving sub-targets, the moving target is tracked, wherein the preset number of second moving sub-targets includes a preset proportion of second moving sub-targets that are closest to the target decision center location;

[0039] If the loss duration is higher than the first threshold and the loss duration is lower than the preset second threshold, then based on the preset adjustment speed, the focal length of the tracking device performing the moving target tracking will be adjusted to the minimum.

[0040] If the duration of loss exceeds the second threshold, the tracking device is adjusted back to the preset position, and the tracking detection mode is changed to the panoramic detection mode to detect the moving target.

[0041] In addition, to achieve the above objectives, this application also proposes a moving target tracking device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the moving target tracking method as described above.

[0042] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the moving target tracking method described above.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] In this embodiment, when moving target tracking is required, the tracking state of the moving target to be tracked is determined, and based on the tracking state, the current tracking detection mode is determined. The tracking detection mode includes a panoramic detection mode and a close-up tracking mode. The panoramic detection mode only detects the moving target, and the resolution of the close-up tracking mode is lower than that of the panoramic detection mode. If the tracking detection mode is the close-up tracking mode, then the moving target is tracked based on the tracking state.

[0045] Current methods for motion target detection in panoramic scenes require high resolution, making it difficult to detect moving targets over large areas with low-powered equipment. This results in high computational demands for large-area motion target detection. This application addresses this issue by using a panoramic detection mode for motion target detection and a close-up tracking mode for motion target tracking. Since the resolution of close-up tracking is lower than that of panoramic detection, and panoramic detection only detects moving targets without tracking them, this application eliminates the need for high-resolution motion target tracking. Instead, it enables motion target tracking at low resolution using the close-up tracking mode. Therefore, this application reduces the computational power required for motion target detection over large areas. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating an embodiment of the moving target tracking method of this application.

[0049] Figure 2 This is a schematic diagram of the first scenario provided in Embodiment 1 of the moving target tracking method of this application;

[0050] Figure 3 This is a schematic diagram of the target tracking process provided in Embodiment 1 of the moving target tracking method of this application;

[0051] Figure 4 This is a flowchart illustrating Embodiment 2 of the moving target tracking method of this application;

[0052] Figure 5 This is a schematic diagram of the lens control process provided in Embodiment 2 of the moving target tracking method of this application;

[0053] Figure 6 This is a schematic diagram of the lens decision-making process provided in Embodiment 2 of the moving target tracking method of this application;

[0054] Figure 7 This is a schematic diagram of the first scene provided in Embodiment 2 of the moving target tracking method of this application;

[0055] Figure 8This is a schematic diagram of the second scenario provided in Embodiment 2 of the moving target tracking method of this application;

[0056] Figure 9 This is a schematic diagram illustrating the relationship between position difference and velocity in Embodiment 2 of the moving target tracking method of this application.

[0057] Figure 10 This is a schematic diagram of the coordinate system provided in Embodiment 2 of the moving target tracking method of this application;

[0058] Figure 11 This is a schematic diagram of the weight decision curve provided in Embodiment 2 of the moving target tracking method of this application;

[0059] Figure 12 This is a flowchart illustrating Embodiment 3 of the moving target tracking method of this application;

[0060] Figure 13 This is a schematic diagram of the device structure of the hardware operating environment involved in the moving target tracking method in the embodiments of this application;

[0061] Figure 14 This is a schematic diagram illustrating the data acquisition consent process involved in the moving target tracking method in this application embodiment.

[0062] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0063] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0064] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0065] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or moving target tracking device capable of performing the above functions. The following description uses a moving target tracking device as an example to illustrate this embodiment and the subsequent embodiments.

[0066] Moving target tracking is a task that involves locking onto and following the position of a specific target in a continuous sequence of images. Current moving target tracking tasks often require tracking and detecting moving targets over a large area.

[0067] Current methods include handcrafted methods based on simple visual features, shape / texture-based methods, motion feature-based methods, classical tracker-based methods, and multi-camera fusion methods. These methods are affected by factors such as lighting variations and color interference. Handcrafted methods based on simple visual features are easily affected by lighting variations and color interference; shape / texture-based methods are computationally intensive and affected by shape deformation, occlusion, and texture interference; motion feature-based methods are affected by camera motion and object motion, and have high computational complexity; classical tracker-based methods rely on initial detection and are limited by the model; and multi-camera fusion methods are costly and complex.

[0068] Current methods typically involve capturing a panoramic view of the area to be detected and then tracking moving targets within that panoramic image. However, methods that perform detection within a panoramic view require high resolution. With low-powered devices, it is impossible to detect moving targets over large areas, resulting in current methods requiring significant computational power for moving target detection over large regions.

[0069] Based on this, embodiments of this application provide a moving target tracking method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the moving target tracking method of this application.

[0070] In this embodiment, the moving target tracking method includes steps S10 to S30:

[0071] Step S10: When it is necessary to perform moving target tracking, determine the tracking status of the moving target to be tracked;

[0072] It should be noted that moving target tracking refers to the process of continuously locating and tracking the trajectory of a specific moving object within a continuous sequence of video frames. In this embodiment, the moving target is a soccer ball, meaning that the soccer ball is used as the moving target for continuous tracking. The tracking state refers to the trackability state of the moving target at the current moment. In this embodiment, the tracking state includes states such as detected and stably tracked, and target lost.

[0073] It is understandable that in the existing technology, due to factors such as changes in lighting, player occlusion, and blurring of fast movement, relying solely on real-time detection can easily lead to interruption or mistracking of the football, resulting in loss of the target in the image or frequent shaking of the gimbal. Therefore, this embodiment first determines the current tracking state of the football to be tracked, so as to determine the subsequent target tracking method of the football based on the current tracking state.

[0074] Step S20: Based on the tracking state, determine the current tracking detection mode, wherein the tracking detection mode includes a panoramic detection mode and a close-up tracking mode. The panoramic detection mode only detects the moving target, and the resolution of the close-up tracking mode is lower than that of the panoramic detection mode.

[0075] It should be noted that the tracking and detection mode refers to the image acquisition and processing strategy adopted by the system in different situations. In this embodiment, it includes two modes: panoramic detection mode and close-up tracking mode. Panoramic detection mode is a high-resolution, wide-field-of-view detection mode, typically implemented with a fixed-focus wide-angle lens, covering the entire football field area. It is used to search for and locate the football over a large area, and is particularly suitable for recapture after the target has been lost. Close-up tracking mode is a lower-resolution, narrower-field-of-view tracking mode, typically implemented with a variable-focus pan-tilt camera. It focuses on the core area of ​​the game and is used for fine-grained tracking and image output of the locked football. Although its resolution is lower than that of panoramic mode, it is sufficient to meet the requirements for clear local imaging.

[0076] It should also be noted that the tracking state includes the current target tracking phase, which includes a target detection phase and a target tracking phase. This embodiment uses different modes for different target tracking phases. If the current tracking phase is the target detection phase, the moving target is detected in panoramic detection mode; if the current tracking phase is the target tracking phase, the detected moving target is tracked in close-up tracking mode. That is, in this embodiment, if in panoramic detection mode, only moving targets are detected, not tracked.

[0077] Understandably, to address the tracking delays or frame drops caused by the limited computing power of existing devices, which prevent the continuous operation of target detection algorithms at high resolutions, this embodiment intelligently switches to the appropriate tracking and detection mode based on the tracking status. When the ball is lost or a large-scale search is required, a panoramic detection mode is activated, using a fixed-focus wide-angle lens to detect the ball across the entire field. When the ball is locked and in an active playing area, the system switches to a lower-resolution close-up tracking mode, focusing on fine-grained local tracking. This significantly reduces the computational load of real-time processing while ensuring tracking continuity, achieving the goal of stably running a high-precision football tracking system on a low-computing-power hardware platform and effectively resolving the contradiction between computing power and accuracy.

[0078] In one feasible implementation, the tracking state includes a target loss state, and the specific implementation of determining the current tracking detection mode based on the tracking state can also be:

[0079] If the tracking detection mode is the panoramic detection mode, when the number of times the moving target is detected reaches a preset number, the position of the moving target at the time of the most recent detection is determined, and the tracking detection mode is switched to the close-up tracking mode to track the moving target at the position of the moving target in the close-up tracking mode. If the tracking detection mode is the close-up tracking mode, and the time when the tracking state is in the target loss state reaches a preset loss time threshold, the tracking detection mode is switched to the panoramic detection mode to detect the moving target in the panoramic detection mode.

[0080] It should be noted that the target loss state refers to the state where no moving target is detected in the current video frame and has not been recaptured after several consecutive frames. In this embodiment, the target loss state can be caused by a player obstructing the view or the soccer ball going out of the camera's range. The preset number of times refers to the cumulative number of times the soccer ball has been successfully detected in panoramic detection mode. For example, if it is detected three times consecutively or cumulatively, it is used to confirm that the target has been stably detected, avoiding triggering mode switching due to accidental false detections. The loss time threshold refers to the time during which the target has not been detected in close-up tracking mode, used to determine whether the current tracking has completely failed and whether it is necessary to switch back to panoramic mode to re-detect.

[0081] Understandably, in existing technologies, frequent target occlusion can cause tracking interruptions, preventing the gimbal from recovering autonomously. Therefore, when the system is in panoramic detection mode and has continuously detected the soccer ball a preset number of times, it considers the target to be stably detected and immediately switches to close-up tracking mode for fine-grained local tracking. However, if the target is lost for more than a set threshold in close-up mode, it is considered a tracking failure, and the system immediately switches back to panoramic mode for a full-field search.

[0082] Specifically, the preset number of detections ensures that close-up tracking is only initiated after the football has been reliably detected multiple times, avoiding invalid gimbal rotation due to single-frame false detections or brief exposure of the ball. The loss time threshold maintains the close-up mode during short periods of player occlusion in football matches, triggering a full-view re-search only when the target cannot be found for an extended period, preventing mode oscillations caused by momentary missed detections. This implementation, through the above steps, effectively avoids system lag or crashes caused by long-term high-load operation of low-computing-power devices while ensuring tracking continuity, thus improving system stability. The scene diagram for target tracking in this embodiment can be found in [reference needed]. Figure 2 The process for target tracking can be referred to Figure 3 .

[0083] Step S30: If the tracking detection mode is the close-up tracking mode, then the moving target is tracked based on the tracking state.

[0084] It is understandable that when the tracking detection mode is the close-up tracking mode, it means that the soccer ball can be stably detected. Therefore, in this embodiment, the moving target is tracked in close-up tracking mode. Furthermore, in close-up mode, this embodiment does not simply map the detected soccer ball pixel coordinates directly to gimbal control commands for tracking. Instead, it determines different tracking methods based on the current tracking state, using different tracking methods to track the target more reasonably and effectively, thereby improving the target tracking effect.

[0085] In summary, this embodiment determines the tracking state of the moving target when it is necessary to track the moving target, and determines the current tracking detection mode based on the tracking state. The tracking detection mode includes a panoramic detection mode and a close-up tracking mode. The panoramic detection mode only detects the moving target, and the resolution of the close-up tracking mode is lower than that of the panoramic detection mode. If the tracking detection mode is the close-up tracking mode, the moving target is tracked based on the tracking state.

[0086] Current methods for motion target detection in panoramic scenes require high resolution, making it difficult to detect moving targets over large areas with low-powered equipment. This results in high computational demands for motion target detection over large areas. This embodiment addresses this issue by using a panoramic detection mode for motion target detection and a close-up tracking mode for motion target tracking. Since the resolution of close-up tracking is lower than that of panoramic detection, and panoramic detection only detects moving targets and does not track them, this embodiment eliminates the need for motion target tracking at high resolution. Instead, it allows for motion target tracking at low resolution using the close-up tracking mode. Therefore, this embodiment reduces the computational power required for motion target detection over large areas.

[0087] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The tracking state further includes maintaining the tracking state, the moving target includes a first moving target and a second moving target, and step S30 of the moving target tracking method further includes steps S31 to S32:

[0088] Step S31: If the first tracking state of the first moving target and the second tracking state of the second moving target are both in the tracking-maintaining state, then the first position of the first moving target is determined based on the preset queue difference method, and the second position of the second moving target is determined based on the preset Kalman tracking method.

[0089] It should be noted that in this embodiment, the first moving target is a soccer ball, and the second moving target is a player. The first tracking state and the second tracking state in this embodiment represent the tracking states of the soccer ball and the player, respectively. Maintaining the tracking state refers to the state in which the target is stably detected over multiple consecutive frames or within a preset time period.

[0090] The queue interpolation method in this embodiment is a linear interpolation method. It inserts several new points between every two adjacent points, making the overall point sequence denser and forming a piecewise linear continuous path, thereby obtaining the position of the football at each moment. As for player tracking, in this embodiment, the Kalman tracking algorithm is used to track and predict the player detection box to obtain the player's tracking box, thereby obtaining the player's position.

[0091] It is understandable that in this embodiment, when both targets are in a state of being tracked, a queue difference-based method is used for the soccer ball, while the Kalman tracking method is used for the group of players, thereby ensuring the accuracy of the soccer ball's position while also taking into account the accuracy of the player movement tracking.

[0092] Specifically, the soccer ball, being a small, high-speed target susceptible to sudden changes due to kicking, is prone to motion blur or partial occlusion, leading to abrupt changes in detection results. Directly using this to control the gimbal would cause camera shake. Therefore, this embodiment employs a queue interpolation method to effectively filter out these sudden changes, outputting a smooth position without sacrificing real-time performance. Furthermore, the movement of the player group exhibits strong continuity. This embodiment uses a Kalman tracking method to improve the accuracy of player position determination based on relevant player movement data. Therefore, through the aforementioned steps, this embodiment ensures both the stability and accuracy of soccer ball position determination.

[0093] Step S32: Track the moving target based on the first position, the second position, and the center position of the tracking screen during tracking.

[0094] It should be noted that the first position refers to the stable coordinates of the soccer ball after processing using the queue interpolation method.

[0095] The second position refers to the player's position predicted using the Kalman tracing method. The center position of the tracked image refers to the center point of the current image captured by the close-up camera.

[0096] It is understandable that existing automatic tracking systems, which focus solely on the soccer ball, are prone to issues such as fragmented footage, players going out of view, or frequent camera shake. Therefore, this embodiment, after acquiring the positions of the soccer ball and players, dynamically calculates the decision center the camera should turn to based on the current center of the frame. By designing an adaptive weighting algorithm, the weight of the soccer ball in the decision is dynamically adjusted according to its offset relative to the frame center, thereby effectively improving the target tracking performance. The close-up control flow of this embodiment can be referred to... Figure 5 The close-up shot decision-making strategy in this embodiment can be referred to Figure 6 .

[0097] In one feasible implementation, the specific implementation of tracking the moving target based on the first position, the second position, and the center position of the tracking screen during tracking can also be:

[0098] Based on the first position and the center position of the screen, it is determined whether the first moving target is at the edge of the tracking screen. If it is not, the target decision center position for tracking the moving target is determined based on the first position, the second position, the center position of the screen and a preset power function. Based on the position difference between the target decision center position and the center position of the screen, the tracking position and tracking speed of the moving target are determined. Based on the tracking position and tracking speed, the moving target is tracked.

[0099] It should be noted that tracking the edges of the image refers to the areas near the perimeter. In this embodiment, the center of the image is used as the origin to divide the image into a central region and an edge region to determine whether the soccer ball is about to leave the effective field of view. The preset power function is a non-linear mathematical function used to dynamically calculate the weight of the soccer ball in the decision center fusion based on its deviation from the center of the image; the greater the deviation, the faster the weight increases. The close-up image in this embodiment can be referenced. Figure 7 , Figure 7 The football is not in the edge area; for close-up shots of the football in the edge area, please refer to... Figure 8 .

[0100] It should also be noted that, in this embodiment, the points in the edge region are points whose horizontal distance from the center of the image exceeds a first edge threshold, or whose vertical distance from the center of the image exceeds a second edge threshold. The first edge threshold and the second edge threshold are different. That is, in this embodiment, determining whether the football is at the edge means determining whether the horizontal distance of the football from the center of the image exceeds the first edge threshold, or whether the vertical distance of the football from the center of the image exceeds the second edge threshold. If so, it indicates that the football is at the edge.

[0101] The target decision center position is the target point that the camera should turn to, calculated by combining the position of the soccer ball, the player's position, and previous frames. The position difference refers to the pixel offset between the target decision center and the current frame center. The tracking position and tracking speed are derived from the position difference, representing the spatial position and corresponding movement speed the gimbal should move to. The relationship between position difference and speed in this embodiment can be found by referring to... Figure 9 The coordinate system in this embodiment can be referred to as Figure 10 The blue coordinates represent the pixel coordinate system, and the red coordinates represent the Cartesian coordinate system.

[0102] Understandably, existing automatic tracking systems often exhibit slow response and are prone to losing target when the football is near the edge of the frame, or experience frequent camera jumps due to player interference in the midfield area. Therefore, this embodiment first determines whether the football is at the edge of the frame during target tracking. If not, it integrates the positions of the football and players with the information of the frame center, dynamically calculates the target decision center using a preset power function, and then determines the tracking position and speed of the gimbal based on the offset of this center from the frame center. This allows for different target tracking methods depending on the target's location, ensuring that key targets are always within the effective field of view. This effectively solves the response lag problem of traditional PID (Proportional-Integral-Derivative Control) systems, improving the sensitivity and stability of the automatic tracking system.

[0103] In one feasible implementation, the specific implementation of determining the target decision center position for moving target tracking based on the first position, the second position, the center position of the image, and a preset power function if the target is not in the specified position can also be:

[0104] Based on the power function, the first position, and the center position of the image, the horizontal and vertical weights of the first moving target are calculated, and the larger of the horizontal and vertical weights is determined as the decision weight of the first moving target. Based on the decision weight, the first position, and the second position, the current decision center position is calculated, and the current decision center position is added to a decision center queue composed of multiple decision center positions, wherein the multiple decision center positions include decision center positions before the current time. The average of the multiple decision center positions and the current decision center position is calculated to obtain the target decision center position.

[0105] It should be noted that the horizontal weight refers to the degree to which the football deviates horizontally from the center of the frame, and the vertical weight refers to the degree to which the football deviates vertically from the center of the frame. The decision weight is the larger of the horizontal and vertical weights, which is used as the unified weight for the final fusion calculation. The current decision center position is the target point to which the camera should turn after weighted fusion based on the decision weights, the football's position, and the player's position in the current frame. The decision center queue is a sliding window queue that stores the decision center positions of multiple historical frames, including the current frame. The target decision center position is the final output position obtained by averaging all positions in the decision center queue, and is used to generate gimbal control commands.

[0106] It should also be noted that the weight calculation formula for the linear version is:

[0107] soccer_weights_horizon=fabs(soccer_ball_x-image_center_x) / (image_center_x / 0.8)

[0108] soccer_weights_vertical=fabs(soccer_ball_y-image_center_y) / (image_center_y / 0.8)

[0109] Soccer_weights=max(soccer_weights_horizon,soccer_weights_horizon)

[0110] Wherein, soccer_weights_horizon is the horizontal weight, soccer_ball_x is the x-coordinate of the soccer ball, image_center_x is the x-coordinate of the image center, soccer_weights_vertical is the vertical weight, soccer_ball_y is the y-coordinate of the soccer ball, image_center_y is the y-coordinate of the image center, and Soccer_weights is the decision weight of the soccer ball.

[0111] The formula for calculating using the power function method is:

[0112] soccer_weights_horizon=5e-06*fabs(soccer_ball_x-image_center_x)^2

[0113] soccer_weights_vertical=2e-06*fabs(soccer_ball_y-image_center_y)^2

[0114] Soccer_weights=max(soccer_weights_horizon,soccer_weights_horizon)

[0115] Y = ax^2, when x = 400, y = 0.8, so a = 5e-06;

[0116] When Y = ax^2, and we take y = 200, we get y = 0.8, and thus a = 2e-06.

[0117] Where image_center_x=480, a=5e-06 are the power function coefficients of the horizontal weights, a=2e-06 are the power function coefficients of the vertical weights, and Y is the power function selected in this embodiment.

[0118] The formula for calculating the decision center in this embodiment of adaptive weighted decision-making is:

[0119] center_x=soccer_weights*soccer_center_x+(1-soccer_weights)*player_average_center_x;

[0120] center_y=soccer_weights*soccer_center_y+(1-soccer_weights)*player_average_center_y;

[0121] Wherein, *soccer_center_x* is the x-coordinate of the center of the soccer ball (the primary moving target), *soccer_center_y* is the y-coordinate of the center of the soccer ball (the primary moving target), *player_average_center_x* is the average x-coordinate of the center ... Figure 11 .

[0122] Understandably, existing target tracking methods often suffer from camera shake and unstable tracking due to sudden changes in the target. In this embodiment, after determining that the soccer ball is not at the edge of the frame, the decision weights in the horizontal and vertical directions are calculated, and the larger value is taken as the unified decision weight. The current decision center is determined by combining the position of the soccer ball and the player, and then it is included in a queue composed of multiple historical decision centers for averaging. This prevents invalid movement caused by slight deviations in a certain direction, and the queue averaging suppresses the influence of instantaneous outliers on camera control. Finally, a stable and continuous tracking trajectory is output, which effectively improves the stability of the automatic tracking system.

[0123] In one feasible implementation, the specific implementation after determining whether the first moving target is at the edge of the tracked image based on the first position and the center position of the image can also be:

[0124] If the first position is determined as the target decision center, the decision weight is set to the maximum value, the target decision center is added to the decision center queue, and the tracking position and tracking speed of the moving target are determined based on the position difference between the target decision center position and the center position of the screen. The moving target is then tracked based on the tracking position and tracking speed.

[0125] It should be noted that in this embodiment, the maximum weight is 1, meaning that the location of the decision center is entirely determined by the position of the soccer ball.

[0126] Understandably, this embodiment, after determining that the football is at the edge of the tracking frame, immediately sets the decision weight to the maximum value and directly uses the first position of the football as the target decision center, skipping the fusion calculation with the player's position, to ensure that the camera responds to the ball's deviation at the fastest speed; then, it adds the decision center to the decision center queue and determines the tracking position and speed of the gimbal based on the position difference between it and the center of the frame, thereby achieving emergency tracking. Under the premise of ensuring that the target is not lost, it effectively solves the problem of untimely tracking of the target in the boundary area in traditional methods and improves the real-time response capability of the system.

[0127] In summary, if both the first tracking state of the first moving target and the second tracking state of the second moving target are in the tracking-maintaining state, the first position of the first moving target is determined based on a preset queue difference method, and the second position of the second moving target is determined based on a preset Kalman tracking method. The moving target is then tracked based on the first position, the second position, and the center position of the tracking screen during tracking.

[0128] In this embodiment, when both targets are being tracked, a queue difference-based method is used for the soccer ball, while a Kalman tracking method is used for the player group. This ensures both accurate soccer ball positioning and accurate player movement tracking. Furthermore, after acquiring the positions of the soccer ball and players, the decision center to which the camera should turn is dynamically calculated based on the current center of the frame. By designing an adaptive weighting algorithm, the weight of the soccer ball in the decision is dynamically adjusted according to its offset relative to the center of the frame, thereby effectively improving the target tracking performance.

[0129] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in Embodiment 1 or Embodiment 2 described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 12 The moving target tracking method further includes steps S30, A31-A33:

[0130] Step A31: If the first tracking state is the target loss state and the second tracking state is the tracking hold state, then determine the loss duration when the first tracking state is the target loss state;

[0131] It should be noted that the loss duration is the length of time elapsed from the moment the football was last reliably detected until the current frame.

[0132] Understandably, this embodiment, under the premise that the football is in a target loss state but the player can still track it stably, begins to accurately record the duration of the loss state, thereby determining whether to maintain close-up tracking or switch back to panoramic mode, and how to continue target tracking. By adopting different tracking methods based on the duration of loss, the target tracking effect is improved.

[0133] Step A32: Based on the lost duration, determine the corresponding target tracking decision;

[0134] It should be noted that target tracking decision refers to the response strategy selected by the system after the target is lost, including the method of subsequent target tracking, whether to reset the device's position, and whether to switch back to panoramic mode.

[0135] In one feasible implementation, the second moving target includes multiple second moving sub-targets, each second moving sub-target having a corresponding sub-target position. The specific implementation of determining the corresponding target tracking decision based on the loss duration can also be:

[0136] If the loss duration is lower than a preset first threshold, the average sub-target position of each sub-target position is calculated, and the average sub-target position is used as the current decision center position. The current decision center position is added to the decision center queue, and the average of the multiple decision center positions and the current decision center position is calculated to obtain the target decision center position. Based on the target decision center position and a preset number of second moving sub-targets, the moving target is tracked. The preset number of second moving sub-targets includes a preset proportion of second moving sub-targets closest to the target decision center position. If the loss duration is higher than the first threshold and lower than a preset second threshold, the focal length of the tracking device for tracking the moving target is adjusted to the minimum based on a preset adjustment speed. If the loss duration is higher than the second threshold, the tracking device is adjusted back to a preset position, and the tracking detection mode is set to the panoramic detection mode to detect the moving target.

[0137] It should be noted that, in this embodiment, the second moving sub-target refers to each player, and the sub-target position refers to the spatial coordinates of each player detected in the current frame. The average sub-target position is the group center point obtained by averaging the positions of all players, representing the overall distribution center of the current players. The first threshold and the second threshold are preset time thresholds used to divide different target loss stages; in this embodiment, the first threshold is 8 seconds and the second threshold is 30 seconds. The preset number of the second moving sub-targets refers to a certain proportion of players closest to the target decision center; in this embodiment, the preset proportion is 20%.

[0138] Understandably, this embodiment, assuming the player group is still trackable, responds in stages based on the duration of target loss. When the loss time is below a first threshold, the center of the player group is used as a temporary decision center, and a smooth target decision center is generated through queue averaging. Simultaneously, only a preset proportion of players closest to this center are selected for local focusing. When the loss time exceeds the first threshold but not the second threshold, the lens focal length is automatically adjusted to the widest angle to expand the search range. When the loss time exceeds the second threshold, the gimbal returns to a preset position and switches back to panoramic detection mode, initiating a full-field search. This embodiment, through the above steps, improves the recovery speed when the target is lost while ensuring target tracking effectiveness.

[0139] Specifically, when the loss duration is less than the first threshold, the system determines that the current target loss is a short-term occlusion. Therefore, instead of immediately abandoning tracking, it calculates the average sub-target position of all second moving sub-targets as the group center and uses it as the current decision center position. It then selects a preset number of players closest to the target decision center position for focused tracking, thereby improving the tracking effect in local scenes.

[0140] When the loss time exceeds the first threshold but is lower than the second threshold, the system judges it as a moderate loss, which may be about to go out of bounds or the pass is interrupted. At this time, the device's focus is adjusted to the minimum to expand the field of view and provide a larger field of view for football detection.

[0141] When the loss duration exceeds the second threshold, the system determines it as a long-term loss, most likely indicating the ball has gone out of bounds or the game has been paused. It immediately executes the gimbal return to its preset position and switches to panoramic detection mode, initiating a full-field scan and re-inspection to ensure no area is missed. Therefore, this embodiment, through the above steps, performs different processing for different target loss scenarios, thereby achieving an efficient and orderly tracking and recovery process on a low-computing-power platform, reducing false positives and improving response speed.

[0142] Step A33: Based on the target tracking decision, track the moving target.

[0143] It should be noted that target tracking decision refers to the response strategy determined based on the duration of loss, including various possibilities such as continuing to track based on the center of the player group, expanding the field of vision to search, or returning to the preset position and switching modes.

[0144] Understandably, current tracking methods suffer from issues such as passive system stagnation or blind searching after the ball is lost, leading to interruptions and slow recovery. This embodiment, however, generates a tiered tracking decision based on the duration of the loss and immediately executes the corresponding tracking action. When a short-term occlusion is detected, the system continuously and smoothly follows the center of the player group as a temporary focus; when a moderate loss is detected, the camera's field of view is automatically expanded to recapture the target; and when a long-term loss is detected, the system actively returns to a preset position and switches to panoramic mode to initiate a full-field scan. This significantly improves the system's recovery capability in complex scenes while ensuring image continuity, effectively solving the problem of slow response after target loss in traditional methods.

[0145] In one embodiment, if the football and player can be detected simultaneously, and the time of loss of the football and player exceeds a third threshold, the device is reset to a preset position; if the time of loss does not exceed the third threshold, the focal length of the device is adjusted to the minimum according to a preset adjustment speed.

[0146] In summary, in this embodiment, if the first tracking state is the target loss state and the second tracking state is the tracking hold state, then the duration of the target loss in the first tracking state is determined, a corresponding target tracking decision is determined based on the duration of the loss, and the moving target is tracked based on the target tracking decision.

[0147] This embodiment, under the premise of confirming that the football is in a target loss state but the player can still track it stably, begins to accurately record the duration of this loss state, thereby determining whether to maintain close-up tracking or switch back to panoramic mode, and how to continue target tracking afterward. When it is determined to be a short-term occlusion, it continuously and smoothly follows the center of the player group as a temporary focus; when it is determined to be a moderate loss, it automatically expands the lens's field of view to recapture the target; when it is determined to be a long-term loss, it actively returns to a preset position and switches to panoramic mode to start a full-field scan, thereby significantly improving the system's recovery capability in complex scenes while ensuring the continuity of the image, effectively solving the problem of slow response after target loss in traditional methods.

[0148] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the moving target tracking method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0149] This application provides a moving target tracking device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the moving target tracking method in Embodiment 1 above.

[0150] The following is for reference. Figure 13 The diagram illustrates a structural schematic suitable for implementing the motion target tracking device in the embodiments of this application. The motion target tracking device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 13 The motion target tracking device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0151] like Figure 13As shown, the moving target tracking device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the moving target tracking device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the moving target tracking device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show moving target tracking devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0152] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0153] The moving target tracking device provided in this application, employing the moving target tracking method described in the above embodiments, can solve the technical problem of high computing power required for moving target detection over large areas. Compared with the prior art, the beneficial effects of the moving target tracking device provided in this application are the same as those of the moving target tracking method described in the above embodiments, and other technical features of this moving target tracking device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0154] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0156] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the moving target tracking method in the above embodiments.

[0157] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0158] The aforementioned computer-readable storage medium may be included in the moving target tracking device; or it may exist independently and not assembled into the moving target tracking device.

[0159] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the moving target tracking device, cause the moving target tracking device to perform the aforementioned moving target tracking method.

[0160] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0162] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0163] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described moving target tracking method, which can solve the technical problem of high computing power required for moving target detection over a large area. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the moving target tracking method provided in the above embodiments, and will not be repeated here.

[0164] All user-related data involved in this application was obtained with the user's permission or consent, as per [reference]. Figure 14 In other words, when this application is applied to a specific product or technology, user permission is required to acquire and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0165] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for tracking moving targets, characterized in that, The method includes: When it is necessary to track moving targets, the tracking state of the moving targets to be tracked is determined, wherein the tracking state includes maintaining the tracking state, and the moving targets include a first moving target and a second moving target; Based on the tracking state, the current tracking detection mode is determined, wherein the tracking detection mode includes a panoramic detection mode and a close-up tracking mode. The panoramic detection mode only detects the moving target, the close-up tracking mode is a tracking mode with a small field of view, and the panoramic detection mode is a detection mode with a large field of view. If the tracking detection mode is the close-up tracking mode, then the moving target is tracked based on the tracking state; Wherein, if the tracking detection mode is the close-up tracking mode, the step of tracking the moving target based on the tracking state includes: If both the first tracking state of the first moving target and the second tracking state of the second moving target are in the tracking-maintaining state, then the first position of the first moving target is determined based on a preset linear interpolation method, and the second position of the second moving target is determined based on a preset Kalman tracking method. The moving target is tracked based on the first position, the second position, and the center position of the tracking screen during tracking. The step of tracking the moving target based on the first position, the second position, and the center position of the tracking screen during tracking includes: Based on the first position and the center position of the image, it is determined whether the first moving target is at the edge of the tracked image; If not, the target decision center position for tracking the moving target is determined based on the first position, the second position, the center position of the image, and a preset power function, wherein the target decision center position is the target point to which the lens should turn; Wherein, if not in the specified position, the step of determining the target decision center position for moving target tracking based on the first position, the second position, the center position of the image, and a preset power function includes: Based on the power function, the first position, and the center position of the screen, the horizontal weight and vertical weight of the first moving target are calculated, and the larger value of the horizontal weight and the vertical weight is determined as the decision weight of the first moving target. The horizontal weight is the degree to which the first moving target deviates from the center position of the screen in the horizontal direction, and the vertical weight is the degree to which the first moving target deviates from the center position of the screen in the vertical direction. Calculate the product of the decision weight and the first position to obtain the first product; calculate the product of (1-decision weight) and the second position to obtain the second product; add the first product and the second product to calculate the current decision center position at the current moment. The current decision center location is added to a decision center queue consisting of multiple decision center locations, wherein the multiple decision center locations include decision center locations prior to the current time. The target decision center location is obtained by calculating the average of the multiple decision center locations and the current decision center location; Based on the position difference between the target decision center position and the image center position, the tracking position and tracking speed of the moving target are determined; The moving target is tracked based on the tracking position and tracking speed.

2. The method as described in claim 1, characterized in that, The tracking status also includes a target loss status, and the step of determining the current tracking detection mode based on the tracking status further includes: If the tracking and detection mode is the panoramic detection mode, when the number of times the moving target is detected reaches a preset number, the position of the moving target at the time of the most recent detection is determined; The tracking detection mode is converted to the close-up tracking mode, so as to track the moving target at the position of the moving target in the close-up tracking mode; If the tracking detection mode is the close-up tracking mode, and the tracking state is the target loss state for a period of time that reaches a preset loss time threshold, then the tracking detection mode is switched to the panoramic detection mode to detect the moving target in the panoramic detection mode.

3. The method as described in claim 1, characterized in that, After the step of determining whether the first moving target is at the edge of the tracked image based on the first position and the center position of the image, the method further includes: If it is in the target decision center, the decision weight is set to the maximum value, and the first position is determined as the target decision center. Add the target decision center to the decision center queue; Based on the position difference between the target decision center position and the image center position, the tracking position and tracking speed of the moving target are determined; The moving target is tracked based on the tracking position and tracking speed.

4. The method as described in claim 1, characterized in that, If the tracking detection mode is the close-up tracking mode, then the step of tracking the moving target based on the tracking state further includes: If the first tracking state is the target loss state and the second tracking state is the keep-tracking state, then the loss duration of the first tracking state being the target loss state is determined. Based on the duration of the loss, a corresponding target tracking decision is determined; Based on the target tracking decision, the moving target is tracked.

5. The method as described in claim 4, characterized in that, The second moving target includes multiple second moving sub-targets, each second moving sub-target having a corresponding sub-target position. The step of determining the corresponding target tracking decision based on the loss duration includes: If the loss duration is lower than a preset first threshold, the average value of the sub-target locations is calculated, and the average value of the sub-target locations is used as the current decision center location. The current decision center location is added to the decision center queue, and the average value of the multiple decision center locations and the current decision center location is calculated to obtain the target decision center location; Based on the target decision center location and a preset number of second moving sub-targets, the moving target is tracked, wherein the preset number of second moving sub-targets includes a preset proportion of second moving sub-targets that are closest to the target decision center location; If the loss duration is higher than the first threshold and the loss duration is lower than the preset second threshold, then based on the preset adjustment speed, the focal length of the tracking device performing the moving target tracking will be adjusted to the minimum. If the duration of loss exceeds the second threshold, the tracking device is adjusted back to the preset position, and the tracking detection mode is changed to the panoramic detection mode to detect the moving target.

6. A moving target tracking device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the moving target tracking method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the moving target tracking method as described in any one of claims 1 to 5.

Images