Sphere Target Tracking Method

Abstract

An embodiment of the present application provides a spherical object tracking method, which is applied to the field of image processing technology. The method includes: after collecting a stadium image, identifying a spherical object and a human object in the stadium image according to the detection result of the stadium image; in response to the spherical object not being identified in the stadium image, expanding the field of view for collecting the stadium image according to a preset field of view expansion rule to obtain a stadium image with an expanded field of view, and searching for the spherical object in the stadium image with the expanded field of view according to a first search area; if the spherical object is still not identified, determining a tracking player object from the identified human objects, and searching for the spherical object according to the tracking player object and a second search area; in response to the spherical object being identified, determining the position of the spherical object. Through the above method, the problems of the ball being lost and the image jitter can be effectively solved, and the spherical object tracking efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method for tracking spherical objects. Background Art

[0002] With the continuous development of computer technology, especially the rise of deep learning, the ability to process and analyze images has been significantly improved, providing a technical basis for the detection of balls such as football in ball games.

[0003] In related technologies, in order to optimize the spherical object tracking effect, the panoramic view of the stadium image is divided into multiple candidate regions, and the candidate regions are determined by the target, and then interpolation transition is performed.

[0004] However, in the above method, when there are situations such as people blocking or the ball flying to a far position, problems such as the ball being lost and the image shaking are likely to occur. Summary of the Invention

[0005] This application provides a method for tracking spherical objects, which can effectively solve the problems of the ball being lost and the image shaking, and improve the efficiency of tracking spherical objects.

[0006] In a first aspect, this application provides a method for tracking spherical objects, including:

[0007] After collecting the stadium image, according to the detection result of the stadium image, identify the spherical object and the human object in the stadium image;

[0008] In response to the spherical object not being identified in the stadium image, according to the preset field of view expansion rule, expand the field of view for collecting the stadium image to obtain the stadium image after the field of view is expanded, and search for the spherical object in the stadium image after the field of view is expanded according to the first search area; wherein, the first search area is determined according to the position where the spherical object was last identified and the first search time;

[0009] If the spherical object is still not identified, determine the tracking player target from the identified human objects, and search for the spherical object according to the tracking player target and the second search area; wherein, the second search area is determined according to the position of the tracking player target and the second search time;

[0010] In response to the spherical object being identified, according to the preset spherical object tracking algorithm, determine the position of the spherical object.

[0011] The sphere target tracking method, device and image processing equipment provided by this application identify sphere targets and human targets by using the detection results of stadium images. In the case of no ball or large ball noise, the sphere target is searched by expanding the search area. When the sphere target is still not found, the player's position is used for tracking. Through the combined tracking of people and balls, the problem of losing the ball (such as the ball being blocked for a long time, or the ball being kicked out of the field, or the noise being too large to recognize the ball) can be effectively solved to achieve efficient tracking of the sphere target; and through the switching tracking of the ball and people, the problem of large fluctuations or stillness in the picture tracking when continuing to use the sphere recognition and tracking method in the case of no ball (such as the cheerleading team taking the field, etc.) can be effectively solved. Brief Description of the Drawings

[0012] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0013] Figure 1 It is a possible application scenario provided by an embodiment of this application;

[0014] Figure 2 It is a schematic flowchart of a sphere target tracking method provided by an embodiment of this application;

[0015] Figure 3 It is a schematic flowchart of another sphere target tracking method provided by an embodiment of this application;

[0016] Figure 4 It is a schematic flowchart of yet another sphere target tracking method provided by an embodiment of this application;

[0017] Figure 5 It is a schematic flowchart of switching from following people to sphere target tracking in an embodiment of this application;

[0018] Figure 6 It is a schematic flowchart of a sphere target tracking method provided by an exemplary embodiment of this application;

[0019] Figure 7 It is a schematic structural diagram of a sphere target tracking device provided by an embodiment of this application;

[0020] Figure 8 It is a schematic structural diagram of the image processing equipment provided by an embodiment of this application.

[0021] Through the above drawings, the clear embodiments of this application have been shown, and there will be more detailed descriptions later. These drawings and text descriptions are not intended to limit the scope of the concept of this application in any way, but to explain the concept of this application to those skilled in the art by referring to specific embodiments. Detailed Description of the Embodiments

[0022] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0023] In a possible application scenario, the sphere target tracking technical solution provided in this embodiment can be applied to a football detection and tracking scenario. Optionally, a wide-angle device can be used in a football game to collect a panoramic video of the stadium. Due to the large angle, the collected images are greatly distorted, or because the targets (balls or people) in the video are too small relative to the entire stadium, in order to facilitate the viewing of the video images, the region of interest can be cropped for video display to improve the viewing experience. Among them, the positioning of the region of interest can be achieved manually or by artificial intelligence. After the region of interest is located, based on the distortion generated by the wide-angle device and the camera installation posture, the cropped video can be further corrected for distortion and posture and then displayed. The entire processing flow can be as Figure 1 shown, including four modules: wide-angle image acquisition, artificial intelligence ball detection, smooth tracking, and image cropping. After the wide-angle image is acquired, artificial intelligence can be used to detect the ball, and adaptive tracking can be achieved through an algorithm. Then, the image correction module is used to correct the image of the cropped region. The present application is applied to the smooth tracking module 101 therein. Among them, the artificial intelligence ball detection module will detect all the balls (including noise balls) and people in the scene, and can number all the balls and people. The same ball moving continuously within a small range will be assigned the same number. When moving significantly or encountering player occlusion, the ball will disappear, and when the ball appears again, the number will change.

[0024] There will be a large number of noise balls during the process of artificial intelligence ball detection and adaptive tracking. The noise balls mainly come from various reasons: 1) The main ball is blocked by the player for a long time, or the main ball flies out of the field to a far position. 2) The volume of the ball is too small, and it becomes even smaller after the artificial intelligence module shrinks the image, making it difficult to detect the target and causing loss. 3) There is no main ball on the field, such as when the cheerleading team takes the field. 4) The artificial intelligence misdetects white football shoes, heads with less hair, or even the lawn with higher brightness as balls. This phenomenon is more likely to occur under low illumination or when the camera is installed far from the field. 5) There are stationary balls on the side of the field, or non-participating players are playing ball, but are captured by the wide-angle camera. 6) The same stadium is divided into multiple areas, and multiple football games are in progress at the same time, and so on.

[0025] In view of the above problems, the embodiments of the present application provide a sphere target tracking method, device and image processing device, which identify sphere targets and human targets based on the detection results of stadium images. In the case of no ball or large ball noise, the sphere target is searched by expanding the search area. When the sphere target still cannot be found, the player position is used for tracking. Through the combined tracking of people and balls, the ball can be effectively prevented from being blocked for a long time, kicked out of the field, or the noise is so large that the ball cannot be recognized, thereby realizing efficient tracking of the sphere. And through the switching tracking method of the ball and people, the problem of large shaking or stillness of the picture tracking due to the lack of a clear tracking target when there is no ball (such as when the cheerleading team takes the field) because the ball is not in the field of view or not detected can be effectively solved. In the case of having a ball, by selecting the main ball with lower noise for smooth tracking, the smoothness of the tracking view of the sphere target is further ensured, and the accuracy of the main ball tracking is improved.

[0026] The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0027] Figure 2 It is a flowchart of the sphere target tracking method provided by the embodiments of the present application. The execution subject of this sphere target tracking method can be an image processing device (the specific form and type of the image processing device are not specifically limited in this embodiment. For example, it can be a terminal device or a server), such as Figure 2 As shown, the method may include the following steps S201-S204:

[0028] Step S201, after collecting the stadium image, identify the sphere target and human target in the stadium image according to the detection result of the stadium image.

[0029] Optionally, the stadium image may be a wide-angle image captured by a wide-angle device and transmitted to the image processing device in real time for detection. The image processing device performs target detection on people and balls in the stadium image according to a preset detection algorithm (such as YOLO algorithm, convolutional neural network CNN, etc.).

[0030] The sphere target is the ball that the players of both sides scramble for during the game, which is different from the balls placed or moving outside the field.

[0031] Step S202: In response to the fact that no sphere target is recognized in the stadium image, according to the preset field of view expansion rule, expand the field of view for collecting the stadium image to obtain the stadium image with the expanded field of view, and search for the sphere target in the stadium image with the expanded field of view according to the first search area; wherein, the first search area is determined according to the position where the sphere target was recognized last time and the first search time.

[0032] In this embodiment, not recognizing the sphere target may mean that in the case of no ball or losing track, the sphere target is not recognized in the current frame, such as not recognizing any sphere, or the recognized sphere has a large amount of noise (as in the following embodiments, the scoring result of the sphere target is lower than the lowest scoring threshold, and this lowest scoring threshold can be determined by those skilled in the art according to a large amount of experimental data or empirical data, or in some embodiments, determined as a situation with large noise according to actual applications or prior data).

[0033] In this embodiment, the field of view expansion rule is a way to gradually expand the field of view to increase the probability of the ball falling into the cropped area. The larger the field of view, the greater the probability of the ball falling into the cropped area. Considering the limitations in the unfolding aspect in this embodiment, there are limitations on the horizontal and vertical cropped areas and they cannot be infinitely expanded. The field of view expansion rule is designed as follows:

[0034] P sx = P osx – a*t1; P ex = P oex + a*t1;

[0035] P sy = P osy – a*t1; P ey = P oey + a*t1;

[0036] In the formula, P sx 、P ex are the starting and ending positions of the unfolded view angle in the x-axis direction respectively, P osx , P oex are the starting and ending positions of the last unfolded view angle in the x-axis direction respectively; similarly, P sy 、P ey 、P osy 、P oey are the corresponding positions of y; a is the speed of field of view expansion, which can be customized, and t1 is the time.

[0037] It is understandable that, in response to the conditions or states on which the operations to be performed depend, when the dependent conditions or states are met, one or more operations to be performed may be real-time or may have a set delay; without special instructions, there is no restriction on the execution order of multiple operations to be performed.

[0038] Next, use the first search area to search for the sphere target in the stadium image with an enlarged field of view. The first search area is determined according to the position of the sphere target recognized last time (i.e., the ball loss position) and the first search time, and the first search time can be adaptively determined by the user according to the actual application or prior data.

[0039] In one example, the position of the sphere target recognized last time includes the horizontal axis coordinate position and the vertical axis coordinate position. The determination method of the first search area can be as follows: determine the horizontal axis retrieval interval that gradually expands with the first search time according to the horizontal axis coordinate position and the sphere speed calculated when the sphere target was recognized last time; determine the vertical axis retrieval interval that gradually expands with the first search time according to the vertical axis coordinate position and the sphere speed calculated when the sphere target was recognized last time; determine the first search area according to the horizontal axis retrieval interval and the vertical axis retrieval interval; where the sphere speed is obtained according to the pixels changed per unit time of the sphere target recognized last time; the first search time is the real-time change time between the starting time when the sphere target was not recognized and the search time threshold.

[0040] In this example, considering that in a short time after the ball is lost, there is a high probability that the ball is still near the loss position. To improve the sphere target tracking efficiency and avoid jitter caused by excessive expansion of the field of view, in this embodiment, the search range is restricted by using the method of determining the search area. The search area gradually expands over time, and the expansion speed of the search area is related to the ball speed, thereby further improving the sphere target tracking efficiency.

[0041] Next, the above horizontal axis retrieval interval and vertical axis retrieval interval will be further introduced. Among them, the determination method of the horizontal axis retrieval interval can be determined by the following method, for example: determine the first horizontal axis boundary of the horizontal axis retrieval interval according to the difference between the horizontal axis coordinate position and the product result of the speed calculated when the sphere target was recognized last time, the first search time, and the preset expansion coefficient; determine the second horizontal axis boundary of the horizontal axis retrieval interval according to the sum of the horizontal axis coordinate position and the product result of the speed calculated when the sphere target was recognized last time, the first search time, and the expansion coefficient; obtain the horizontal axis retrieval interval according to the first horizontal axis boundary and the second horizontal axis boundary.

[0042] In this embodiment, using the speed calculated for the sphere target recognized last time and the first search time, the distance that the sphere target may move within the first search time can be calculated. By multiplying by an expansion factor, the adjustment and correction of this distance can be achieved (for example, the search area needs to be larger than the calculated distance for easier search). Using the current horizontal axis position and the difference between the product of the speed, time, and expansion factor corresponding to the sphere target recognized last time, the first horizontal axis boundary can be calculated. This boundary represents the farthest position that the target sphere may move to the left with the current horizontal axis position as the base point. Similarly, the second horizontal axis boundary, that is, the farthest distance that the target sphere may move to the right. Combining the first horizontal axis boundary and the second horizontal axis boundary can obtain the retrieval interval of the horizontal axis.

[0043] Correspondingly, the determination method of the vertical axis retrieval interval can be determined by the following method. For example: According to the vertical axis coordinate position and the difference between the product result of the speed calculated for the sphere target recognized last time, the first search time, and the preset expansion factor, determine the first vertical axis boundary of the vertical axis retrieval interval; According to the vertical axis coordinate position and the sum of the product result of the speed calculated for the sphere target recognized last time, the first search time, and the expansion factor, determine the second vertical axis boundary of the vertical axis retrieval interval; According to the first vertical axis boundary and the second vertical axis boundary, obtain the vertical axis retrieval interval.

[0044] It can be understood that the calculation principle of the vertical axis retrieval area is similar to that of the above horizontal axis retrieval interval, and no more detailed description will be given here.

[0045] Through the calculation of the above first horizontal axis boundary and the second horizontal axis boundary, by combining historical motion information and the expansion factor, the size and position of the search area can be dynamically adjusted to adapt to the historical motion trend and possible future changes of the sphere target, which helps to improve the success rate of repositioning the sphere target in complex scenarios, especially in the case where the sphere target may accelerate or its path changes. In addition, the flexibility of the expansion factor allows optimization according to specific application requirements, thereby further improving the tracking efficiency or accuracy of the sphere target. In an optional example, the ball speed calculation method is the number of pixels changed per unit time for the sphere target recognized last time (that is, the number of pixels changed per unit time when the historical ball ID remains unchanged), denoted as V. The time is from the moment the ball is lost (that is, the starting time when the sphere target is not recognized, and this time can be determined according to the time interval between the time when the sphere was recognized last time. If the time interval exceeds the duration set by the system for not being able to track the sphere, then it is determined as the ball loss time) to each main ball search frame, that is, the first search time, denoted as t2. The specific calculation method of the search area can be as follows:

[0046] x - 2 *v x*t2 (first horizontal axis boundary) < D x < x + 2 * v x *t2 (second horizontal axis boundary);

[0047] y - 2 * v y *t2 (first vertical axis boundary) < D y < y + 2 * v y *t2 (second vertical axis boundary);

[0048] where D x is the search range in the x - axis direction of the horizontal axis with the previous ball - losing position as the base point, D y is the search range in the y - axis direction of the vertical axis with the previous ball - losing position as the base point, v x is the historical speed of the ball in the x - direction, v y is the historical speed of the ball in the y - direction. Multiplying by 2 means expanding the range at twice the speed (i.e., an optional expansion coefficient, which can be other expansion coefficients in some embodiments and can be adaptively adjusted according to actual applications), effectively reducing the probability of losing the ball.

[0049] Through the above - mentioned method of calculating the search area, by performing target detection within a concentrated area, this localized search reduces the dependence on the entire field of view, thus effectively preventing the phenomenon of jitter caused by rapid switching of the expanded view angle. Moreover, through the above - mentioned precise positioning method of the search area based on the previously recognized spherical target, it is possible to prevent the problem of frequent switching of the expanded view angle among multiple competition areas in the case of multiple sub - area ball games on the same stadium.

[0050] Further exemplarily, considering that the image processing device usually caches future data during the process of spherical target tracking, that is, the possible positions of the spherical object in the predicted future frames. To further improve the efficiency of spherical target tracking, after losing the football, the cached data can be used to help re - locate the football. Specifically, the method provided in this embodiment may further include the following steps: obtaining the cached retrieval frame time according to the position where the target spherical object is predicted to appear in the future frame corresponding to the stadium image after expanding the field of view angle, where the cached retrieval frame time is the time used to predict the appearance of the target spherical object; determining the cached retrieval frame time as the search time threshold.

[0051] Continuing with the above - mentioned method of calculating the search area as an example, for the determination method of the search time threshold, when the search range does not expand to the cached ball, the search time threshold t2 can be determined based on the initial search time. This initial search time threshold is the time corresponding to the start of losing the ball until each main ball search frame. In the further exemplarily scenario where there is future data cached in the system and the search range can be expanded to the cached ball, when it is expanded to the cached frame, t2 can be the time from losing the ball to the cached retrieval frame.

[0052] It is understandable that the future frame, i.e., the detection frame after the current frame, usually caches the future frame according to historical data in practical applications to predict the position of the sphere in the future frame. No further elaboration will be made here.

[0053] By expanding the search range to include the cached frame, using the predicted position of the sphere in the cache to assist in repositioning, and determining the search time threshold by using the cached retrieval frame time to expand the search range, the success rate of the search can be further improved.

[0054] It should be noted that the first search area and the second search area in this embodiment are only used to describe similar objects and have no other special meaning. The first search area and the second search area may be areas with the same range or different ranges, and the same applies to the first search time and the second search time.

[0055] Continue as Figure 2 shown, in step S203, if the sphere target is still not recognized, determine the tracking player target from the recognized human targets, and search for the sphere target according to the tracking player target and the second search area; wherein, the second search area is determined according to the position of the tracking player target and the second search time.

[0056] After searching in the expanded field of view angle and the expanded search area, if the sphere target is still not found, it indicates that the current situation may be a situation without a ball (such as the cheerleading team taking the field), etc. If the sphere search continues based on the above method, the problem of large fluctuations or stillness in tracking may occur in the picture. In this embodiment, by switching the sphere target tracking to the person-following mode, the tracking efficiency of the sphere target is improved, and at the same time, the problem of large fluctuations or stillness in tracking is solved. Optionally, considering that in a dynamic scene, the target may be temporarily undetected due to short-term occlusion, rapid movement, or other reasons. By setting a waiting time, the system can be prevented from immediately switching to the person-following mode due to short-term detection failures, thereby reducing unnecessary mode switching and improving the tracking efficiency of the sphere target. Specifically, when the sphere target (hereinafter referred to as the main ball) is not found through the above search method, a waiting timer is started (the waiting duration can be customized according to experience). When the waiting time is greater than a certain time threshold (such as 30 s), the sphere target tracking is switched to the person-following mode, that is, the tracking player target is determined from the recognized human targets, and the sphere target is recognized according to the tracking player target.

[0057] Exemplarily, when searching for the sphere target according to the tracked player target and the second search area, the position of the tracked player target can be used as the basis point to search for the sphere target within the second search area. It should be noted that in this embodiment, the determination method of the second search area can refer to the determination method of the above-mentioned first search area. By replacing the position and speed of the sphere recognized last time with the position and speed of the tracked player target, the second search area targeted at the tracked player can be calculated. The same applies to the second search time, and related descriptions will not be elaborated further.

[0058] Step S204: In response to recognizing the sphere target, determine the position of the sphere target according to the preset sphere target tracking algorithm.

[0059] Among them, recognizing the sphere target means recognizing the cue ball (such as a sphere that meets the noise conditions and the sphere target scoring result reaches the lowest scoring threshold). When the sphere target is recognized, the smooth sphere target tracking algorithm or other sphere target tracking algorithms (those skilled in the art can adjust or determine the sphere target tracking algorithm according to the actual application) can be used to determine the position of the sphere target to achieve tracking of the sphere target.

[0060] Exemplarily, the above step S204 determines the position of the sphere target according to the preset sphere target tracking algorithm, and can adopt the following method: Determine the smooth position of the sphere target with respect to the current frame according to the position where the sphere target is predicted to appear in the future frames corresponding to the current position of the sphere target; Determine the moving step of the sphere target with respect to the current frame according to the smooth position and the number of future frames; Obtain the position of the sphere target according to the smooth position and the moving step.

[0061] In this example, optionally, the above method of determining the smooth position of the sphere target with respect to the current frame according to the position where the sphere target is predicted to appear in the future frames corresponding to the current position of the sphere target can adopt the following method: Based on the weighted average algorithm, calculate the weighted average value between the positions where the sphere target is predicted to appear in each frame of the future frames corresponding to the current position of the sphere target, and determine the smooth position of the sphere target with respect to the current frame.

[0062] The weighted average algorithm is a method for calculating the average value. By assigning a weight to a data point, the weight reflects the importance of the data point in the calculation of the overall average value. In this embodiment, the weighted average algorithm is used to weight the position of the cue ball using multiple-frame caching, and the position where the future cue ball will be located after multiple frames can be smoothed out, making the smooth position more accurate.

[0063] Among them, the method of smoothing out the position where the future cue ball will be located after multiple frames by weighting the position of the cue ball with multiple-frame caching to obtain the smooth position can be as follows:

[0064] BP = /

[0065] This formula can be used to calculate the weighted average position, that is, the smoothed position BP, where BP(n) is the position of the nth future cache frame. n is also the weight, which reflects the importance of each cache frame in predicting the future position. Optionally, the larger the value of n, the greater the influence of this frame on the future position.

[0066] By taking the weighted average of the positions of the cache frames, the future position of the cue ball can be smoothed, reducing the error caused by single-frame fluctuations. This smoothing process helps to reduce the prediction error caused by single-frame fluctuations or noise.

[0067] Optionally, to determine the movement step of the sphere target with respect to the current frame based on the smoothed position and the number of future frames, the following method can be adopted: Determine the movement step of the sphere target with respect to the current frame according to the ratio between the product of the smoothed position and the preset speed control parameter and the number of future frames.

[0068] It can be understood that the movement step of the current frame, that is, the distance or position adjustment amount that needs to be moved in the current frame, in order to maintain accurate tracking of the sphere target.

[0069] For example, the calculation method of the movement step of the current frame can be as follows:

[0070] BP step =

[0071] In the formula, BP step is the movement step of the current frame, that is, the step that the sphere target needs to move in the current frame. nmax is the number of future frames, that is, the total number of cache frames. c is the parameter for controlling the speed, and this parameter can determine the size of the movement step. A larger value of c means a faster response speed.

[0072] This calculation method ensures that the movement step of the current frame is proportional to the predicted position of the cache frame, thus maintaining a consistent step behavior in the case of different numbers of cache frames, achieving the technical effect of smooth tracking of the sphere target. Furthermore, in each frame, the position of the sphere can be updated according to the calculated movement step. The new position is obtained by adding the current smoothed position and the movement step, thereby realizing smooth tracking of the sphere and reducing screen jitter.

[0073] In an alternative embodiment, the method for determining the above speed control parameter may be as follows: determining the speed control parameter according to the average value of the relative position change amplitude between the current position of the spherical target and the positions of the spherical target in the historical frames; wherein, the speed control parameter is proportional to the average value of the relative position change amplitude.

[0074] In this embodiment, the relative position change amplitude between the positions of the spherical target in the historical frames, that is, the frame difference size between the historical frames, can be used to represent the movement speed of the spherical target. For example, the greater the average value of the relative position change amplitude, the faster the spherical target moves; conversely, the smaller the average value of the relative position change amplitude, the slower the spherical target moves.

[0075] In one example, the relative position change amplitude between the positions of the spherical target in the historical frames can be calculated by the position difference of the spherical target in each pair of consecutive historical frames (i.e., two consecutive historical frames). For example, by obtaining the position coordinates of the spherical target in every two frames and using the position coordinates of the spherical target for difference calculation to obtain the position difference between every two frames, and then accumulating the position differences between every two frames and performing mean calculation, the average value of the relative position change amplitude between the positions of the spherical target in the historical frames can be obtained.

[0076] In another example, the relative position change amplitude between the positions of the spherical target in the historical frames can also be achieved by calculating the pixel value difference of the pixel points at the position of the spherical target for each pair of consecutive historical frames (i.e., two consecutive historical frames), such as by calculating the absolute difference: D(x, y) =| I t (x, y) – I t-1 (x, y)|, where D(x, y) is the pixel value at the position (x, y) in the difference image (i.e., the difference image between each pair of consecutive historical frames), and I t (x, y) and I t-1 (x, y) are the pixel values at the position (x, y) in the current frame and the previous frame respectively. By accumulating the frame differences of all historical frames to obtain an overall frame difference size, such as by summing or averaging all frame differences, a larger frame difference indicates that the spherical target moves faster, and the value of c can be dynamically increased to improve the response speed; if the frame difference is smaller, it indicates that the target moves slower, and the value of c can be dynamically decreased to improve stability.

[0077] Optionally, the speed control parameter c can be dynamically adjusted based on the size of the frame difference through a linear formula, or the speed control parameter can increase proportionally with the frame difference size. This embodiment does not make a special limitation on this.

[0078] By dynamically adjusting the speed control parameters, the smooth spherical target can be adaptively responded to the movement changes of the spherical target during tracking, improving the tracking accuracy. When the spherical target moves faster, the step is increased to increase the response speed, and when the target moves slower, the step is reduced to improve stability. In other words, by using the frame difference information of the historical frame to adjust the speed control parameters, the performance of the target tracking algorithm can be significantly improved, so that it can maintain good tracking effects under different motion conditions.

[0079] It should be noted that, in addition to being determined in the above manner, it can also be customized by the user based on experience.

[0080] It can be seen that using cached frame weighting in the process of spherical target tracking, smoothing the current frame position based on the predicted position of future time points, and then calculating the current moving step by the position of future time points to determine the main ball position can significantly prevent the picture from shaking violently with the ball.

[0081] Figure 3 A flowchart of another ball target tracking method provided in an embodiment of the present application is provided. Based on the above embodiment, this embodiment takes into account that the detection algorithm detects multiple balls (such as misidentified as balls) or the court includes other balls (non-cluster balls, i.e. non-ball targets), in order to further improve the ball target tracking accuracy. In addition to the above steps S201-S204, this embodiment searches for ball targets in the court image after the field of view is expanded according to the first search area in step S202, including:

[0082] Step S2021: In response to not identifying a spherical target in the court image, the field of view used to capture the court image is expanded according to a preset field of view expansion rule to obtain a court image with an expanded field of view.

[0083] It should be noted that the process of expanding the field of view has been introduced in the above embodiments, and the relevant description will not be repeated here.

[0084] Step S2022: If multiple spheres to be identified are searched in the first search area, the sphere target score of each sphere to be identified is calculated according to at least one of the motion index of each sphere to be identified, the sphere confidence, the distance between the sphere position and the last sphere target position identified, and the time when the sphere to be identified is searched, to obtain a sphere target score result; wherein the motion index is determined according to the absolute motion amount and the relative motion amount between relative frames of the sphere to be identified.

[0085] It can be understood that there are multiple spheres to be recognized, that is, multiple spheres detected by the detection algorithm in the image processing device. Usually, there is only one target sphere in the event, that is, the cue ball. In this embodiment, by combining data such as the motion index, sphere confidence, distance between the sphere position and the previous recognized sphere target position, and the time when the sphere to be recognized is searched, noise spheres are denoised to improve the accuracy of sphere recognition and tracking.

[0086] In one implementation, the motion index can be determined in the following way: Determine the absolute motion amount of the sphere to be recognized according to the position difference between the starting position and the ending position of the sphere to be recognized; Determine the relative motion amount of the sphere to be recognized according to the average value of the position differences between the current frame position and the previous frame position for each frame of the sphere to be recognized; Obtain the motion index according to the product of the absolute motion amount and the relative motion amount.

[0087] Optionally, the motion index mindx can be determined according to the absolute motion amount of the sphere to be recognized and the relative motion amount between relative frames, and its calculation method can be as follows:

[0088] mindx = |Ps - Pe| * Σ(|P(n)-P(n-1)|)

[0089] In the formula, P(n) is the position of the current frame, P(n - 1) is the position of the previous frame, Ps is the starting position of the ball, and Pe is the ending position of the ball. Among them, |Ps - Pe| is the absolute motion amount, and |P(n)-P(n-1)| is the relative motion amount between relative frames. It can be seen from the above formula that the motion index is proportional to the absolute motion amount of the sphere target tracking and the relative motion amount between relative frames. It can be understood that the starting position Ps refers to the position where the ball was last successfully detected before the target was lost, and the ending position Pe refers to the position after the ball is redetected. In the case of losing the target, the ending position can be determined by prediction or estimation until the ball is detected again.

[0090] In this embodiment, the absolute motion amount provides overall displacement information, while the relative motion amount between frames provides detailed motion information. By combining the absolute motion amount information and relative motion information of the sphere target to calculate the motion index, a comprehensive evaluation of the motion behavior of the sphere target can be provided, thereby quantifying the motion characteristics of the sphere target and further improving the accuracy of recognizing the sphere target.

[0091] Exemplarily, in step S2022 above, the sphere target score for each sphere to be recognized is calculated respectively based on at least one of the motion index, sphere confidence, distance between the sphere position and the previous recognized sphere target position, and the time when the sphere to be recognized is searched. The following method can be adopted: for each sphere to be recognized, the first credibility information of the sphere to be recognized is determined according to the product between the motion index and the sphere confidence of the sphere to be recognized; the second credibility information of the sphere to be recognized is determined according to the ratio between the first credibility information and the distance between the sphere position of the sphere to be recognized and the previous recognized sphere target position; the sphere target score of the sphere to be recognized is determined according to the ratio between the second credibility information and the time when the sphere to be recognized is searched, so as to obtain the sphere target score for each sphere to be recognized.

[0092] In this example, when there are multiple spheres in the first search area (such as the current frame search area determined by the current frame to determine the search time threshold in the above embodiment, or the cache frame search area determined by the cache frame to determine the search time threshold), the main sphere is selected to improve the accuracy of main sphere recognition. The main sphere is selected using a comprehensive scoring method, and the sphere with the highest comprehensive score is selected as the main sphere. The score is related to the motion index, the confidence of the sphere, and the position of the sphere. The calculation method can be as follows:

[0093] score = (conf * mindx) / diff / t2;

[0094] Where score is the comprehensive score, that is, the sphere target score; conf is the calculated confidence of the sphere (for example, it can be determined according to the position of the stadium area. The confidence of the sphere to be recognized in the auditorium area is low, and the confidence of the sphere to be recognized in the competition area is high), mindx is the motion index of the sphere, diff is the distance (straight-line distance) between the sphere and the lost ball position (that is, the position of the previous recognized sphere target), and t2 is the time difference between the time when the sphere is recognized and the time when the ball is lost. Among them, conf * mindx is the first credibility information, and (conf * mindx) / diff is the second credibility information. It can be seen from the above formula that the higher the confidence of the sphere, the greater the motion index, the closer the distance to the lost ball position, and the smaller the time from the lost ball point, the higher the score of the sphere.

[0095] Step S2023: According to the sphere target score result, select the sphere with the highest score from the multiple spheres to be recognized, and determine the sphere target.

[0096] By using the above method of selecting the cue ball based on the sphere target score, other spheres with high noise detected by the detection algorithm can be removed during the sphere tracking process (for example, the AI misdetects white sneakers, a person with less hair on the head, or even a brightly lit lawn as a ball, which is more likely to occur under low illumination or when the camera is installed far from the stadium, and there are stationary balls on the sidelines of the stadium, or non-playing players are playing but are captured by the camera). Its anti-noise effect is better, so a cue ball with higher accuracy can be selected, thereby improving the sphere target tracking accuracy.

[0097] It should be noted that the process of identifying the sphere target for the stadium image before expanding the field of view angle and the process of identifying the sphere target after following a person are both applicable to the above sphere target identification process, and relevant descriptions will not be elaborated here.

[0098] Figure 4 This is a schematic flowchart of another sphere target tracking method provided by an embodiment of the present application. On the basis of the above embodiment, in this embodiment, the tracking player target and position are determined through single-frame clustering, and the position of the tracking player target is smoothed and the sphere search is performed through multi-frame weighting, which can further improve the accuracy of cue ball recognition and tracking. Specifically, the above step S203 includes the following steps S2031 - S2033.

[0099] Step S2031: If the sphere target has not been recognized yet, determine the position weights of each human target according to the stadium layout information and the positions of each human target, and obtain the human targets to be clustered with position weights greater than the preset weight threshold.

[0100] Among them, the stadium layout information is pre-known information, which may include the size, boundary, key areas (such as goal, penalty area), spectator stands, etc. of the stadium.

[0101] Step S2032: Cluster the human targets to be clustered according to the pre-determined number of clusters to obtain a clustering result; among them, the number of clusters is determined according to the number of competition areas of the stadium.

[0102] Taking a football game as an example, since the players may be relatively scattered, and there may even be two games on a single stadium, in the selection of the initial position of the tracking players, the players are clustered according to the player positions to determine the accurate tracking player target and position. The pre-determined number of clusters can be determined according to the number of competition areas of the stadium. For example, in a football game, there are usually at most two games on the stadium, and moreover, too many centroids have little impact on the selection of the initial position. Two centroids can be selected, that is, two numbers of clusters are determined for clustering the human targets, so as to obtain the clustering results corresponding to the two centroids. The clustering method can adopt K-means clustering.

[0103] Step S2033: Determine a tracked player target according to the clustering result and the position of the ball target identified last time, and search for the ball target according to the tracked player target and the second search area.

[0104] For example, for each clustering result, the distance from the centroid of each cluster to the last ball target position is calculated, and the centroid of the player close to the cue ball position in the last frame is selected as the initial position, and the player corresponding to the centroid of the player (or the player closest to the centroid) is determined as the tracking player target. It can be understood that in cluster analysis, the centroid refers to the center point of a cluster, which can be calculated by the average value of all data points in the cluster (that is, the positions of each human body target).

[0105] Step S2034: searching for a ball target according to the tracked player target and the second search area.

[0106] After determining the target player to be tracked, the player can be tracked as the base point, and the spherical target can be searched in the corresponding second search area by following the person. Through the above single-frame clustering method, the location where the players gather can be quickly located, and the target player to be tracked can be accurately identified, thereby further improving the efficiency and accuracy of spherical tracking and identification.

[0107] In order to further improve the accuracy of the cue ball search process and the smoothness of the picture, combined with Figure 5 As shown, this embodiment combines single-frame clustering and multi-frame weighting to perform player tracking and cue ball search. Specifically, the above step S2034 may include the following steps: determining the transition position of the tracked player according to the initial position of the tracked player identified and the position of the ball target identified last time; obtaining the multi-frame clustering position of the tracked player with respect to the current frame according to the transition position of the tracked player and the position where the tracked player is predicted to appear in the corresponding future frame; searching for the ball target according to the multi-frame clustering position of the tracked player and the second search area.

[0108] As mentioned above, the initial position is the center of mass of the player that is closer to the cue ball position in the last frame. However, since the position of the player is different from the position of the ball, this embodiment uses the following formula to transition the position of the ball in the last frame (i.e., the position where the ball target was recognized last time) to the position of the player:

[0109] P = (BPlast * (maxT – t3) + PPinit * t3) / maxT; (t3 <= maxT)

[0110] Wherein, P is the transitional position of the tracked player, BPlast is the position of the cue ball in the last frame, PPinit is the initial player position found by clustering (i.e., the initial position of the tracked player), t3 is the time, and maxT is the transitional time.

[0111] After losing the ball, directly tracking a fast - moving ball may cause the picture to jitter because the movement of the ball is usually faster and more irregular than that of the player. In this embodiment, by combining the transition from the position of the cue ball recognized last time to the position of the player, the relatively smooth movement of the player can be utilized to smooth the picture change and reduce jitter.

[0112] In subsequent tracking, the retrieval of the cue ball needs to be continuously carried out. If the cue ball cannot be detected, in the multi - frame weighted person - following method, clustering needs to be performed for multiple players between single frames. After clustering, the class closer to the center of the previous tracking is selected as the tracking target. The multi - frame person - following method can be as follows:

[0113] After multi - frame caching and weighting the clustered positions of the players, the position where the player is located after multiple frames is smoothed. PP = / , wherein, PP is the position of the current frame (i.e., the multi - frame clustered position of the tracked player with respect to the current frame), and PP(n) is the position of the tracked player predicted by the nth cached frame. And smooth person - following can be achieved by calculating the step - by - step progress of the current frame: PPstep = , wherein, PPstep is the step that needs to move in the current frame, nmax is the total number of cached frames, and c is a parameter for controlling the speed.

[0114] During the person - following process, if the cue ball is searched, it can be smoothly switched to the position of the cue ball. The smooth transition method is similar to the switching from sphere target tracking to person - following, and the relevant description will not be elaborated here.

[0115] By caching the historical position data of multiple frames, a smooth motion trajectory can be calculated. Compared with single - frame data, this method can better capture the overall trend of the motion, rather than being affected by the noise or anomalies that may exist in a single frame, making the captured motion of the sphere more stable and reducing the picture jitter caused by fast or irregular motion.

[0116] For the convenience of understanding the embodiments of the present application, as Figure 6 shown, the following process is included:

[0117] After the sphere target tracking starts, the sphere target tracking can start at any time. For example, after collecting the stadium images after the start of the game, the sphere target tracking can start, or the sphere target tracking can also start midway;

[0118] Judge whether the cue ball is recognized, that is, whether the sphere target is detected. If the cue ball is recognized, perform the cue ball tracking;

[0119] If the cue ball is not recognized, the expanded field of view angle is enlarged, and the cue ball is searched within the first search area. During the process of searching for the cue ball, the cue ball that meets the requirements (such as selecting the ball with the highest score in the sphere target scoring result) is selected from the multiple recognized spheres, and cue ball tracking is performed;

[0120] The first search area is a gradually expanding area range over time. By means of a waiting count, it is judged whether the cue ball is recognized. If the cue ball is not recognized and the time limit is not exceeded (i.e., the search time threshold), the range of the first search area is enlarged for ball detection; if the cue ball is not recognized and the time limit has been exceeded, the tracking switches from sphere target tracking to person tracking;

[0121] The person tracking method can search for the cue ball by expanding the second search area over time, and select the cue ball that meets the requirements. If the cue ball is recognized, the tracking switches from person tracking to sphere target tracking, and cue ball tracking is performed. Otherwise, continue person tracking and search for the cue ball based on the second search area that expands over time until the cue ball is found and cue ball tracking is performed.

[0122] Through the above technical solutions, by adopting a combined human and ball tracking method, it can effectively prevent the ball from being blocked for a long time, or kicked out of the field, or the noise is so large that the ball cannot be recognized, or there is no ball (such as when the cheerleading team takes the field), and the problem of large fluctuations or stillness in the picture tracking; based on the processing method after losing the ball, the field of view angle is enlarged, the search area is gradually expanded, and comprehensive indicators composed of motion index, confidence level, position relationship, and time relationship are used to select the cue ball, which can effectively reduce the ball loss rate and select a more accurate cue ball; through the calculation method of the search area, it can effectively prevent the rapid switching of the expanded view angle, resulting in jitter, and can prevent the expanded view angle from frequently switching between multiple competition areas in the case of multiple sub-area ball games on the same court; the continuous sphere target tracking or continuous person tracking method uses cache frame weighting to calculate the position at a future time point, and then calculates the current moving step based on the position at the future time point. This method can significantly prevent the picture from shaking violently with the ball. In addition, the weighted clustering method in person tracking effectively reduces the influence of the audience on the sidelines, and improves the accuracy of replacing the cue ball position with the player center when the players are overly dispersed or there are two ball games on the same court.

[0123] Figure 7 It is a schematic structural diagram of a sphere target tracking device provided by an embodiment of the present application, as Figure 7 shown. The device includes a joint recognition module 701, a first search module 702, a second search module 703, and a sphere target tracking module 704, where,

[0124] The combined recognition module 701 is configured to, after collecting the stadium image, recognize the sphere target and the human target in the stadium image according to the detection result of the stadium image;

[0125] The first search module 702 is configured to, in response to the sphere target not being recognized in the stadium image, expand the field of view for collecting the stadium image according to a preset field of view expansion rule to obtain the stadium image with an expanded field of view, and search for the sphere target in the stadium image with the expanded field of view according to a first search area; wherein, the first search area is determined according to the position where the sphere target was recognized last time and the first search time;

[0126] The second search module 703 is configured to, if the sphere target is still not recognized, determine a tracking player target from the recognized human targets, and search for the sphere target according to the tracking player target and a second search area; wherein, the second search area is determined according to the position of the tracking player target and the second search time;

[0127] The sphere target tracking module 704 is configured to, in response to recognizing the sphere target, determine the position of the sphere target according to a preset sphere target tracking algorithm.

[0128] In one embodiment, the position where the sphere target was recognized last time includes the horizontal axis coordinate position and the vertical axis coordinate position. The first search module 702 includes:

[0129] The first interval determination unit is configured to determine a gradually expanding horizontal axis retrieval interval with the first search time according to the horizontal axis coordinate position and the sphere speed calculated when the sphere target was recognized last time;

[0130] The second interval determination unit is configured to determine a gradually expanding vertical axis retrieval interval with the first search time according to the vertical axis coordinate position and the sphere speed calculated when the sphere target was recognized last time;

[0131] The area determination unit is configured to determine the first search area according to the horizontal axis retrieval interval and the vertical axis retrieval interval;

[0132] Wherein, the sphere speed is obtained based on the number of pixels changed per unit time when the sphere target was recognized last time; the first search time is the real-time changing time from the starting time when the sphere target was not recognized to the search time threshold.

[0133] In one embodiment, the first interval determination unit is specifically configured to: determine the first horizontal axis boundary of the horizontal axis search interval according to the difference between the horizontal axis coordinate position and the product result of the speed calculated when the sphere target was last recognized, the first search time, and a preset expansion coefficient; determine the second horizontal axis boundary of the horizontal axis search interval according to the sum of the horizontal axis coordinate position and the product result of the speed calculated when the sphere target was last recognized, the first search time, and the expansion coefficient; and obtain the horizontal axis search interval according to the first horizontal axis boundary and the second horizontal axis boundary.

[0134] In one embodiment, the second interval determination unit is specifically configured to: determine the first vertical axis boundary of the vertical axis search interval according to the difference between the vertical axis coordinate position and the product result of the speed calculated when the sphere target was last recognized, the first search time, and a preset expansion coefficient; determine the second vertical axis boundary of the vertical axis search interval according to the sum of the vertical axis coordinate position and the product result of the speed calculated when the sphere target was last recognized, the first search time, and the expansion coefficient; and obtain the vertical axis search interval according to the first vertical axis boundary and the second vertical axis boundary.

[0135] In one embodiment, the first search module 702 includes:

[0136] A sphere scoring unit, which is configured to calculate the sphere target score of each sphere to be recognized respectively according to at least one of the motion index, sphere confidence, distance between the sphere position and the position of the sphere target recognized last time, and the time when the sphere to be recognized was searched, if multiple spheres to be recognized are searched in the first search area, and obtain a sphere target score result;

[0137] A score selection unit, which is configured to select the sphere with the highest score from the multiple spheres to be recognized according to the sphere target score result, and determine the sphere target;

[0138] Wherein, the motion index is determined according to the absolute motion amount and the relative inter-frame motion amount of the sphere to be recognized.

[0139] In one embodiment, an index determination module for determining the motion index is further included. The index determination module includes: a first motion amount determination unit for determining the absolute motion amount of the sphere to be recognized according to the position difference between the starting position and the ending position of the sphere to be recognized; a second motion amount determination unit for determining the relative motion amount of the sphere to be recognized according to the average value of the position differences between the current frame position and the previous frame position of each frame of the sphere to be recognized; and obtaining the motion index according to the product of the absolute motion amount and the relative motion amount.

[0140] In one embodiment, the sphere scoring unit is specifically configured to, for each sphere to be recognized, determine the first credibility information of the sphere to be recognized according to the product between the motion index and the sphere confidence of the sphere to be recognized; determine the second credibility information of the sphere to be recognized according to the ratio between the first credibility information and the distance between the sphere position of the sphere to be recognized and the position of the sphere target recognized last time; and determine the sphere target score of the sphere to be recognized according to the ratio between the second credibility information and the time when the sphere to be recognized is searched, so as to obtain the sphere target score of each sphere to be recognized.

[0141] In one embodiment, the device further includes:

[0142] A cache time acquisition unit, which is configured to acquire the cache retrieval frame time according to the position where the target sphere is predicted to appear in the future frame corresponding to the stadium image after the field of view angle is enlarged, and the cache retrieval frame time is the time for predicting the appearance of the target sphere;

[0143] A time threshold determination unit, which is configured to determine the cache retrieval frame time as the search time threshold.

[0144] In one embodiment, the second search module 703 includes:

[0145] A weight acquisition unit, which is configured to determine the position weights of each human target according to the stadium layout information and the positions of each human target, and acquire the human targets to be clustered whose position weights are greater than a preset weight threshold;

[0146] A single-frame clustering unit, which is configured to cluster the human targets to be clustered according to a pre-determined number of clusters to obtain a clustering result; wherein, the number of clusters is determined according to the number of competition areas in the stadium;

[0147] A player determination unit, which is configured to determine the tracked player target according to the clustering result and the position of the sphere target recognized last time.

[0148] In one embodiment, the second search module 703 includes:

[0149] A transition unit, which is configured to determine the transition position of the tracked player according to the initial position of the tracked player recognized and the position of the sphere target recognized last time;

[0150] A multi-frame clustering unit, which is configured to obtain the multi-frame clustering position of the tracked player with respect to the current frame according to the transition position of the tracked player and the position where the tracked player is predicted to appear in the corresponding future frame;

[0151] A search unit, which is configured to search for a sphere target according to the multi-frame clustering positions of the tracked player and the second search area.

[0152] In one embodiment, the sphere target tracking module 704 includes:

[0153] A smoothing unit, which is configured to determine the smoothed position of the sphere target with respect to the current frame according to the position where the sphere target is predicted to appear in the future frames corresponding to the current position of the sphere target;

[0154] A step determination unit, which is configured to determine the movement step of the sphere target with respect to the current frame according to the smoothed position and the number of future frames;

[0155] A tracking unit, which is configured to obtain the position of the sphere target according to the smoothed position and the movement step.

[0156] In one embodiment, the smoothing unit is specifically configured to calculate the weighted average value between the positions where the sphere target is predicted to appear in each of the future frames corresponding to the current position of the sphere target based on a weighted average algorithm, and determine the smoothed position of the sphere target with respect to the current frame; and / or, the determining the movement step of the sphere target with respect to the current frame according to the smoothed position and the number of future frames includes: determining the movement step of the sphere target with respect to the current frame according to the ratio between the product of the smoothed position and a preset speed control parameter and the number of future frames.

[0157] In one embodiment, the step determination unit is specifically configured to determine a speed control parameter according to the average value of the relative position change amplitudes between the positions of the sphere target in the historical frames corresponding to the current position of the sphere target; wherein, the speed control parameter is proportional to the average value of the relative position change amplitudes.

[0158] The sphere target tracking device provided in the above embodiments can be used to execute the sphere target tracking method in any of the foregoing method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0159] Figure 8 A kind of image processing device provided in the embodiments of the present application, such as Figure 8 shown, this image processing device includes: a processor 802, and a memory 801, a display 803 communicatively connected to the processor;

[0160] The memory 801 stores computer execution instructions;

[0161] The processor 802 executes the computer-executable instructions stored in the memory 801 to implement the sphere target tracking method provided in any one of the above first aspects. The display 803 is used to display the image after image processing.

[0162] The image processing device provided in the above embodiments can be used to execute the sphere target tracking method in any of the foregoing method embodiments. The implementation principles and technical effects are similar and will not be elaborated here.

[0163] The embodiment of the present application further provides a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are executed by a processor, they are used to implement the sphere target tracking method provided in the above first aspect.

[0164] For the above computer-readable storage medium, the above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0165] Optionally, the readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0166] The computer-readable storage medium provided in the above embodiments can be used to execute the sphere target tracking method in any of the foregoing method embodiments. The implementation principles and technical effects are similar and will not be elaborated here.

[0167] The embodiment of the present application further provides a computer program product. The computer program product includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solutions provided in any of the foregoing method embodiments can be implemented.

[0168] In this application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the relationship between associated objects and indicates that three relationships can exist. For example, A and / or B can represent the cases of A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after; in a formula, the character " / " represents a "division" relationship between the associated objects before and after. "At least one (item)" or a similar expression refers to any combination of these items, including any combination of a single item or multiple items. For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0169] It can be understood that the various numerical numbers involved in the embodiments of this application are only for the convenience of description and are not used to limit the scope of the embodiments of this application. In the embodiments of this application, the magnitude of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0170] Those skilled in the art will readily conceive of other embodiments of this application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common general knowledge or conventional technical means in this technical field that are not disclosed in this application. The specification and examples are only considered exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0171] It should be understood that this application is not limited to the exact structures that have been described and shown in the drawings above, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. A sphere target tracking method, characterized in that, Including: After collecting the stadium image, according to the detection result of the stadium image, identifying the sphere target and the human target in the stadium image; In response to the sphere target not being identified in the stadium image, according to the preset field of view expansion rule, expanding the field of view for collecting the stadium image to obtain the stadium image with the expanded field of view, and searching for the sphere target in the stadium image with the expanded field of view according to the first search area; wherein, the first search area is determined according to the position of the sphere target identified last time and the first search time; the first search time is the real-time changing time from the starting time when the sphere target is not identified to the search time threshold; If the sphere target is still not identified, after a preset duration, determining the tracking player target from the identified human targets, and searching for the sphere target according to the tracking player target and the second search area; wherein, the second search area is determined according to the position of the tracking player target and the second search time; the tracking player target is determined by clustering the human targets in the current frame, the tracking player target is the human target with the smallest distance from the initial position, and the initial position is the position of the centroid of the cluster with the smallest distance from the position of the sphere target identified last time; In response to identifying the sphere target, according to the preset sphere target tracking algorithm, determining the position of the sphere target.

2. The method according to claim 1, wherein The position of the sphere target identified last time includes the horizontal axis coordinate position and the vertical axis coordinate position, and the determination method of the first search area includes: According to the horizontal axis coordinate position and the sphere speed calculated when the sphere target was identified last time, determining the horizontal axis retrieval interval that gradually expands with the first search time; According to the vertical axis coordinate position and the sphere speed calculated when the sphere target was identified last time, determining the vertical axis retrieval interval that gradually expands with the first search time; According to the horizontal axis retrieval interval and the vertical axis retrieval interval, determining the first search area; Wherein, the sphere speed is obtained according to the pixels changed per unit time of the sphere target identified last time.

3. The method according to claim 2, characterized in that According to the horizontal axis coordinate position and the sphere speed calculated when the sphere target was identified last time, determining the horizontal axis retrieval interval that gradually expands with the first search time, including: According to the horizontal axis coordinate position, and the difference between the speed calculated when the sphere target was identified last time and the product result of the first search time and the preset expansion coefficient, determining the first horizontal axis boundary of the horizontal axis retrieval interval; According to the horizontal axis coordinate position, and the sum of the speed calculated when the sphere target was identified last time and the product result of the first search time and the expansion coefficient, determining the second horizontal axis boundary of the horizontal axis retrieval interval; According to the first horizontal axis boundary and the second horizontal axis boundary, obtaining the horizontal axis retrieval interval.

4. The method according to claim 2, wherein According to the vertical axis coordinate position and the sphere speed calculated when the sphere target was identified last time, determining the vertical axis retrieval interval that gradually expands with the first search time, including: Determine the first vertical axis boundary of the vertical axis search interval based on the difference between the vertical axis coordinate position and the product result of the speed calculated when the spherical target was last recognized, the first search time, and the preset expansion coefficient; Determine the second vertical axis boundary of the vertical axis search interval based on the sum of the vertical axis coordinate position and the product result of the speed calculated when the spherical target was last recognized, the first search time, and the expansion coefficient; Obtain the vertical axis search interval according to the first vertical axis boundary and the second vertical axis boundary.

5. The method according to any one of claims 1-4, characterized in that, The searching for the spherical target in the stadium image after expanding the field of view angle according to the first search area includes: If multiple spheres to be recognized are found in the first search area, calculate the spherical target score of each sphere to be recognized respectively according to at least one of the motion index, sphere confidence, distance between the sphere position and the position of the spherical target recognized last time, and the time when the sphere to be recognized was found, to obtain the spherical target score result; Select the sphere with the highest score from the multiple spheres to be recognized according to the spherical target score result to determine the spherical target; Wherein, the motion index is determined according to the absolute motion amount and the relative inter-frame relative motion amount of the sphere to be recognized.

6. The method according to claim 5, wherein The determination method of the motion index includes: Determine the absolute motion amount of the sphere to be recognized according to the position difference between the starting position and the ending position of the sphere to be recognized; Determine the relative motion amount of the sphere to be recognized according to the average value of the position differences between the current frame position and the previous frame position of each frame of the sphere to be recognized; Obtain the motion index according to the product of the absolute motion amount and the relative motion amount.

7. The method according to claim 5, wherein The calculating the spherical target score of each sphere to be recognized respectively according to at least one of the motion index, sphere confidence, distance between the sphere position and the position of the spherical target recognized last time, and the time when the sphere to be recognized was found includes: For each sphere to be recognized, determine the first credibility information of the sphere to be recognized according to the product of the motion index and the sphere confidence of the sphere to be recognized; Determine the second credibility information of the sphere to be recognized according to the ratio of the first credibility information to the distance between the sphere position of the sphere to be recognized and the position of the spherical target recognized last time; Determine the spherical target score of the sphere to be recognized according to the ratio of the second credibility information to the time when the sphere to be recognized was found, so as to obtain the spherical target score of each sphere to be recognized.

8. The method according to any one of claims 2-4, characterized in that, It further includes: Obtain the cached retrieval frame time according to the position where the target sphere is predicted to appear in the stadium image corresponding to the future frame after expanding the field of view angle, and the cached retrieval frame time is the time used to predict the appearance of the target sphere; Determine the cached retrieval frame time as the search time threshold.

9. The method according to claim 1, characterized in that, The determining the tracking player target from the recognized human targets includes: Determine the position weights of each human target according to the stadium layout information and the positions where each human target is located, and obtain the human targets to be clustered whose position weights are greater than the preset weight threshold; Cluster the human targets to be clustered according to a pre-determined number of clusters to obtain a clustering result; wherein, the number of clusters is determined according to the number of competition areas in the stadium. Determine the tracking player target according to the clustering result and the position of the sphere target recognized last time.

10. The method according to claim 1 or 9, characterized in that The searching for the sphere target according to the tracking player target and the second search area includes: Determine the transition position of the tracking player according to the initial position of the recognized tracking player and the position of the sphere target recognized last time. Obtain the multi-frame clustering positions of the tracking player for the current frame according to the transition position of the tracking player and the predicted positions of the tracking player in the corresponding future frames. Search for the sphere target according to the multi-frame clustering positions of the tracking player and the second search area.

11. The method according to claim 1, wherein The determining the position of the sphere target according to the pre-set sphere target tracking algorithm includes: Determine the smooth position of the sphere target for the current frame according to the predicted positions of the sphere target in the future frames corresponding to the current position of the sphere target. Determine the moving step of the sphere target for the current frame according to the smooth position and the number of future frames. Obtain the position of the sphere target according to the smooth position and the moving step.

12. The method according to claim 11, wherein The determining the smooth position of the sphere target for the current frame according to the predicted positions of the sphere target in the future frames corresponding to the current position of the sphere target includes: Based on the weighted average algorithm, calculate the weighted average between the current position of the sphere target and the predicted positions of the sphere target in each frame of the future frames to determine the smooth position of the sphere target for the current frame. And / or, the determining the moving step of the sphere target for the current frame according to the smooth position and the number of future frames includes: Determine the moving step of the sphere target for the current frame according to the ratio between the product of the smooth position and the pre-set speed control parameter and the number of future frames.

13. The method according to claim 12, wherein The determining method of the speed control parameter includes: Determine the speed control parameter according to the average value of the relative position change amplitudes between the current position of the sphere target and the positions of the sphere target in the historical frames. Wherein, the speed control parameter is proportional to the average value of the relative position change amplitudes.

Images