Sphere target tracking method

Abstract

The embodiment of the invention provides a sphere target tracking method, which is applied to the technical field of image processing. The method comprises the steps that after a court image is collected, a ball target and a human body target in the court image are recognized according to a detection result of the court image; in response to the situation that a ball target is not recognized in the court image, expanding a field angle used for collecting the court image according to a preset field angle expansion rule so as to obtain the court image after the field angle is expanded, and searching the ball target in the court image after the field angle is expanded according to the first search area; if the ball target is still not recognized, a tracking player target is determined from the recognized human body targets, and the ball target is searched according to the tracking player target and the second search area; in response to identifying the spherical target, a location of the spherical target is determined. According to the method, the problems of ball losing and picture jittering can be effectively solved, and the ball target tracking efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a spherical target tracking method. Background Art

[0002] With the continuous development of computer technology, especially the rise of deep learning, the ability to process and analyze images has been significantly improved, providing a technical basis for the detection of ball games such as football.

[0003] In the related art, in order to optimize the ball tracking effect, the panoramic view of the court image is divided into multiple candidate areas, and the candidate areas are determined by targets, and then interpolation transition is performed.

[0004] However, the above method is prone to problems such as the ball being lost and the image being jittery when a person blocks the ball or the ball flies far away. Summary of the invention

[0005] The present application provides a spherical target tracking method, which can effectively solve the problems of ball loss and image jitter, and improve the efficiency of spherical target tracking.

[0006] In a first aspect, the present application provides a spherical target tracking method, comprising:

[0007] After collecting the court image, identifying the ball target and the human target in the court image according to the detection result of the court image;

[0008] In response to the failure to identify the spherical target in the court image, the field of view used to capture the court image is expanded according to a preset field of view expansion rule to obtain the court image after the field of view is expanded, and the spherical target in the court image after the field of view is expanded is searched according to a first search area; wherein the first search area is determined according to the position where the spherical target was identified last time and the first search time;

[0009] If the spherical target is still not identified, a tracking player target is determined from the identified human targets, and the spherical target is searched based on the tracking player target and a second search area; wherein the second search area is determined based on the position of the tracking player target and a second search time;

[0010] In response to identifying the spherical target, the position of the spherical target is determined according to a preset spherical target tracking algorithm.

[0011] The ball target tracking method, device and image processing equipment provided by the present application use the detection results of the court image to identify the ball target and the human target. When there is no ball or the ball noise is large, the ball target is searched by expanding the search area, and when the ball target is still not found, the player position is used for tracking. By jointly tracking the person and the ball, the problem of losing the ball (such as the ball is blocked for a long time, or the ball is kicked out of the court, or the noise is so large that the ball cannot be identified) can be effectively solved to achieve efficient tracking of the ball target; and by switching the ball and the person tracking, the problem of large shaking or stillness of the picture tracking can be effectively solved when there is no ball (such as the cheerleading team on the court, etc.) and the ball recognition and tracking method is continued. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0013] Figure 1 A possible application scenario provided for the embodiment of the present application;

[0014] Figure 2 A schematic diagram of a flow chart of a spherical target tracking method provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of a flow chart of another spherical target tracking method provided in an embodiment of the present application;

[0016] Figure 4 A schematic diagram of a flow chart of another spherical target tracking method provided in an embodiment of the present application;

[0017] Figure 5 This is a schematic diagram of the process of switching from human to spherical target tracking in an embodiment of the present application;

[0018] Figure 6 A schematic diagram of a flow chart of a spherical target tracking method provided by an exemplary embodiment of the present application;

[0019] Figure 7 A schematic diagram of the structure of a spherical target tracking device provided in an embodiment of the present application;

[0020] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

[0021] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0022] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0023] In a possible application scenario, the spherical target tracking technical solution provided in this embodiment can be applied to a football detection and tracking scenario. Optionally, a football match can use a wide-angle device to capture a panoramic video of the stadium. Due to the large angle, the captured image is greatly distorted, or because the target (ball or person) in the video is too small relative to the entire stadium, in order to facilitate the viewing of the video image, the region of interest can be cut out for video display to improve the viewing experience, wherein the positioning of the region of interest can be achieved manually or by artificial intelligence. After the region of interest is positioned, based on the distortion caused by the wide-angle device and the camera installation posture, the cut-out video can be further displayed after distortion and posture correction. The entire processing flow can be as follows: Figure 1 As shown, it includes four modules: wide-angle image acquisition, artificial intelligence ball detection, smooth tracking, and image extraction. After acquiring the wide-angle image, artificial intelligence ball detection can be used, and adaptive tracking can be achieved through the algorithm. Then, the image correction module is used to correct the image of the extracted area. The present application is applied to the smooth tracking module 101. Among them, the artificial intelligence ball detection module will detect all balls (including noise balls) and people in the scene, and can number all balls and people. The same ball will be assigned the same number when it moves continuously within a small range. When moving significantly or encountering a player's occlusion, the ball will disappear, and when the ball appears again, the number will switch.

[0024] There will be a large number of noise balls in the process of detecting the ball and adaptively tracking it through artificial intelligence. The noise balls mainly come from many reasons: 1) The main ball is blocked by the players for a long time, or the main ball flies far outside the court. 2) The ball is too small, and the artificial intelligence module becomes smaller after reducing the image, making it difficult to detect the target, resulting in loss. 3) There is a situation where there is no main ball on the court, such as when the cheerleading team is on the court. 4) Artificial intelligence mistakenly detects white sneakers, human heads with less hair, and even brighter grass as balls. This phenomenon is more likely to occur in low light or when the camera is installed far away from the court. 5) There is a stationary ball on the side of the court, or there are non-participating players playing, but they are captured by the wide-angle camera. 6) The same court is divided into multiple areas, and multiple games are going on at the same time, etc.

[0025] In view of the above problems, the embodiments of the present application provide a ball target tracking method, device and image processing equipment, which use the detection results of the court image to identify the ball target and the human target. In the case of no ball or large ball noise, the ball target is searched by expanding the search area, and when the ball target is still not found, the player position is used for tracking. By tracking the person and the ball together, it can effectively prevent the ball from being blocked for a long time, or the ball from being kicked out of the court, or the noise is so loud that the ball cannot be identified, thereby achieving efficient tracking of the ball. In addition, by switching the tracking of the ball and the person, it can effectively solve the problem of large shaking or stillness of the picture tracking due to the lack of a clear tracking target when there is no ball (such as the cheerleading team on the court, etc.) because the ball is not in the field of view or is not detected. In the case of a ball, by selecting the main ball with lower noise for smooth tracking, the stability of the expansion angle of the ball target tracking is further guaranteed, and the accuracy of the main ball tracking is improved.

[0026] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0027] Figure 2 The flowchart of the spherical target tracking method provided in the embodiment of the present application is shown in FIG. 1 . The spherical target tracking method may be executed by an image processing device (the specific form and type of the image processing device are not specifically limited in the present embodiment, for example, it may be a terminal device or a server), such as Figure 2 As shown, the method may include the following steps S201-S204:

[0028] Step S201: After collecting a court image, according to the detection result of the court image, identify the ball target and the human target in the court image.

[0029] Optionally, the stadium image can be a wide-angle image taken by a wide-angle device and transmitted to an image processing device in real time for detection. The image processing device performs target detection on people and balls in the stadium image according to a preset detection algorithm (such as YOLO algorithm, convolutional neural network CNN, etc.).

[0030] Among them, the ball target is the ball that the players of both teams compete for in the game, which is different from the ball placed or moving outside the field.

[0031] Step S202: In response to not identifying a spherical target in the court image, the field of view used to capture the court image is expanded according to a preset field of view expansion rule to obtain a court image after the field of view is expanded, and the court image after the field of view is expanded is searched for the spherical target in a first search area; wherein the first search area is determined according to a position where the spherical target was last identified and a first search time.

[0032] In this embodiment, failure to identify the spherical target may be a case where there is no ball or the ball is lost, the current frame does not identify the spherical target, such as no sphere is identified, or the identified sphere has large noise (for example, in the embodiments below, the spherical target scoring result is lower than the minimum scoring threshold, which can be determined by technicians in this field based on a large amount of experimental data or empirical data, or in some embodiments, it is determined as a case with large noise based on actual applications or prior data).

[0033] In this embodiment, the field of view angle expansion rule is a method of gradually expanding the field of view angle to increase the probability of the ball falling into the clipping area. The larger the field of view angle, the greater the probability of the ball falling into the clipping area. This embodiment takes into account that due to the limitation of expansion, the horizontal and vertical clipping areas are limited and cannot be expanded infinitely. The field of view angle expansion rule is designed as follows:

[0034] P sx =P osx – a*t1; P ex =P oex + a*t1;

[0035] P sy =P osy – a*t1; P ey =P oey + a*t1;

[0036] Where P sx , P ex are the starting and ending positions of the expanded view in the x-axis direction, P osx , P oex are the starting and ending positions of the last expansion of the viewing angle in the x-axis direction; similarly, P sy , P ey , P osy , P oey are the positions corresponding to y; a is the speed of field angle expansion, which can be adjusted by user, and t1 is the time.

[0037] It can be understood that in response to the conditions or states on which the executed operations depend, when the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0038] Next, the first search area is used to search the court image after the field of view is expanded to search for the spherical target. The first search area is determined based on the position where the spherical target was last identified (i.e., the position where the ball was lost) and the first search time. The first search time can be adaptively determined by the user based on actual applications or prior data.

[0039] In one example, the position of the last identified spherical target includes a horizontal axis coordinate position and a vertical axis coordinate position, and the method for determining the first search area can be as follows: determine a horizontal axis search interval that gradually expands with the first search time based on the horizontal axis coordinate position and the spherical speed calculated when the spherical target was last identified; determine a vertical axis search interval that gradually expands with the first search time based on the vertical axis coordinate position and the spherical speed calculated when the spherical target was last identified; determine the first search area based on the horizontal axis search interval and the vertical axis search interval; wherein the spherical speed is obtained based on the pixel change per unit time when the spherical target was last identified; and the first search time is the real-time change time from the start time when the spherical target is not identified to the search time threshold.

[0040] In this example, considering that after the ball is lost, there is a high probability that the ball is still near the lost position within a short period of time, in order to improve the tracking efficiency of the spherical target and avoid jitter caused by excessive expansion of the field of view, this embodiment limits the search range by using a method for determining the search area. The search area gradually expands over time, and the speed at which the search area expands is related to the ball speed, thereby further improving the tracking efficiency of the spherical target.

[0041] Next, the horizontal axis search interval and the vertical axis search interval are further introduced. The horizontal axis search interval can be determined by the following methods: determining the first horizontal axis boundary of the horizontal axis search interval according to the horizontal axis coordinate position and the difference between the speed calculated when the spherical target was recognized last time and the product result of the first search time and the preset expansion coefficient; determining the second horizontal axis boundary of the horizontal axis search interval according to the horizontal axis coordinate position and the sum of the speed calculated when the spherical target was recognized last time and the product result of the first search time and the expansion coefficient; obtaining the horizontal axis search interval according to the first horizontal axis boundary and the second horizontal axis boundary.

[0042] In this embodiment, the speed calculated when the spherical target was recognized last time and the first search time can be used to calculate the distance that the spherical target may move within the first search time. By multiplying by the expansion coefficient, the distance can be adjusted and corrected (for example, the search area needs to be larger than the calculated distance to facilitate the search). The current horizontal axis position and the difference between the speed, time and expansion coefficient corresponding to the last recognition of the spherical target can be used to calculate the first horizontal axis boundary. This boundary represents the farthest position that the target sphere may move to the left with the current horizontal axis position as the base point. Similarly, the second horizontal axis boundary, that is, the farthest distance that the target sphere may move to the right. The search interval of the horizontal axis can be obtained by combining the first horizontal axis boundary and the second horizontal axis boundary.

[0043] Correspondingly, the method for determining the vertical axis search interval can be determined in the following manner, such as: determining the first vertical axis boundary of the vertical axis search interval based on the vertical axis coordinate position and the difference between the speed calculated when the spherical target was last identified and the product of the first search time and the preset expansion coefficient; determining the second vertical axis boundary of the vertical axis search interval based on the vertical axis coordinate position and the sum of the speed calculated when the spherical target was last identified and the product of the first search time and the expansion coefficient; obtaining the vertical axis search interval based on the first vertical axis boundary and the second vertical axis boundary.

[0044] It can be understood that the calculation principle of the vertical axis search area is similar to that of the above-mentioned horizontal axis search interval, and the relevant description will not be further introduced here.

[0045] By combining the calculation of the first horizontal axis boundary and the second horizontal axis boundary with the historical motion information and the expansion coefficient, the size and position of the search area can be dynamically adjusted to adapt to the historical motion trend and possible future changes of the spherical target, which helps to improve the success rate of relocating the spherical target in complex scenes, especially when the spherical target may accelerate or the path changes. In addition, the flexibility of the expansion coefficient allows optimization according to specific application requirements, thereby further improving the efficiency or accuracy of spherical target tracking. In an optional example, the ball speed calculation method is the pixel change per unit time for the last identified spherical target (that is, the pixel change per unit time when the historical ball ID remains unchanged), which is recorded as V. The time is from the start of the ball loss (that is, the start time when the spherical target is not identified, which can be determined based on the time interval between the last time the spherical target was identified. If the time interval exceeds the duration of the failure to track the spherical target set by the system, it is determined as the ball loss time) to each main ball search frame, that is, the first search time, recorded as t2, and the specific search area calculation method can be as follows:

[0046] x - 2 *v x*t2 (first horizontal axis boundary) < D x < x + 2 *v x * t2 (second horizontal axis boundary);

[0047] y - 2 *v y *t2 (first vertical axis boundary) < D y < y + 2 *v y * t2 (second vertical axis boundary);

[0048] Where D x is the search interval in the x direction of the horizontal axis with the last lost ball position as the base point, D y The search interval in the y direction of the vertical axis is based on the last lost ball position, v x is the historical velocity of the ball in the x direction, v y is the historical speed of the ball in the y direction, and multiplying it by 2 means expanding the interval by 2 times the speed (that is, an optional expansion factor, which can also be other expansion factors in some embodiments and can be adaptively adjusted according to actual applications), effectively reducing the probability of losing the ball.

[0049] By calculating the search area as described above and detecting targets in a concentrated area, this localized search reduces reliance on the entire field of view, thereby effectively preventing rapid switching of the expanded viewing angle and causing jitter. Furthermore, by accurately positioning the search area based on the last identified spherical target, it is possible to prevent the problem of frequent switching of the expanded viewing angle between multiple competition areas in the case of multiple regional games on the same court.

[0050] As a further example, considering that the image processing device usually caches future data during the ball target tracking process, that is, the possible ball position predicted in the future frame, in order to further improve the ball target tracking efficiency, the cached data can be used to help relocate the ball after the ball is lost. Specifically, the method provided in this embodiment can also include the following steps: according to the position of the target ball predicted to appear in the future frame corresponding to the court image after the expanded field of view, the cache retrieval frame time is obtained, and the cache retrieval frame time is used to predict the time of the target ball appearing; the cache retrieval frame time is determined as the search time threshold.

[0051] Continuing with the calculation method of the above search area as an example, the method for determining the search time threshold is that when the search range does not expand the cached ball, the search time threshold t2 can be determined based on the initial search time. The initial search time threshold is the time from the ball loss to each main ball search frame. For further example scenarios, for future data cached in the system, the search range can be expanded to the cached ball. When expanded to the cached frame, t2 can be the time from the ball loss to the cached retrieval frame.

[0052] It is understandable that the future frame is the detection frame after the current frame. In practical applications, the future frame is usually cached according to historical data to predict the position of the sphere in the future frame, and the relevant description will not be repeated here.

[0053] The success rate of the search can be further improved by expanding the search range to include cache frames, using the predicted sphere position in the cache to assist in repositioning, and using the cache retrieval frame time to determine the search time threshold.

[0054] It should be noted that the first search area in this embodiment and the second search area described later are only used to describe similar objects and have no other special meanings. The first search area and the second search area may be areas of the same range or areas of different ranges. The same applies to the first search time and the second search time.

[0055] Continue as Figure 2 As shown, step S203, if the spherical target is still not identified, determine the tracking player target from the identified human targets, and search the spherical target based on the tracking player target and the second search area; wherein the second search area is determined based on the position of the tracking player target and the second search time.

[0056] After searching with the expanded field of view and the expanded search area, if the spherical target is still not found, it means that there may be no ball (such as the cheerleading team on the field) at present. If the spherical search is continued based on the above method, the screen may have the problem of tracking large shaking or stillness. This embodiment switches the spherical target tracking to the following person mode, thereby improving the spherical target tracking efficiency and solving the problem of tracking large shaking or stillness. Optionally, considering that in a dynamic scene, the target may be temporarily undetectable due to short-term occlusion, rapid movement or other reasons. By setting a waiting time, the system can be prevented from switching to the following person mode immediately due to a short-term detection failure, thereby reducing unnecessary mode switching and improving the spherical target tracking efficiency. Specifically, when the spherical target (hereinafter referred to as the main ball) is not searched through the above search method, a waiting time is performed (the waiting time can be customized according to experience). When the waiting time is greater than a certain time threshold (such as 30s), the spherical target tracking is switched to following the person, that is, the tracking player target is determined from the identified human target, and the spherical target is identified according to the tracking player target.

[0057] For example, according to the tracking player target and the second search area, the ball target can be searched for in the second search area by using the position of the tracking player target as the base point. It should be noted that the method for determining the second search area in this embodiment can refer to the method for determining the first search area, and the position and speed of the ball identified last time can be replaced with the position and speed of the tracking player target, so that the second search area with the tracking player as the target can be calculated. The second search time is the same, and the relevant description will not be repeated.

[0058] Step S204: In response to identifying the spherical target, determining the position of the spherical target according to a preset spherical target tracking algorithm.

[0059] Among them, identifying the spherical target means identifying the main ball (such as a sphere that meets the noise condition, and the spherical target scoring result reaches the minimum scoring threshold). When the spherical target is identified, a smooth spherical target tracking algorithm or other spherical target tracking algorithm (technical personnel in this field can adjust or determine the spherical target tracking algorithm in combination with actual applications) can be used to determine the position of the spherical target to achieve tracking of the spherical target.

[0060] Exemplarily, the above-mentioned step S204 determines the position of the spherical target according to a preset spherical target tracking algorithm, and can be carried out in the following manner: according to the position where the spherical target is predicted to appear in the future frame corresponding to the current position of the spherical target, the smooth position of the spherical target with respect to the current frame is determined; according to the smoothed position and the number of future frames, the moving step of the spherical target with respect to the current frame is determined; according to the smoothed position and the moving step, the position of the spherical target is obtained.

[0061] In this example, optionally, the above-mentioned determination of the smooth position of the spherical target with respect to the current frame based on the position of the spherical target predicted to appear in the future frame corresponding to the current position of the spherical target can be carried out in the following manner: based on a weighted average algorithm, a weighted average value between the position of the spherical target predicted to appear in each frame in the future frame corresponding to the current position of the spherical target is calculated to determine the smooth position of the spherical target with respect to the current frame.

[0062] The weighted average algorithm is a method for calculating the average value by assigning a weight to a data point, which reflects the importance of the data point in the overall average value calculation. This embodiment uses the weighted average algorithm and multi-frame buffer to weight the position of the cue ball, which can smooth out the position of the cue ball in the future after multiple frames, making the smoothed position more accurate.

[0063] Among them, by caching the cue ball position weighted by multiple frames, the future position of the cue ball after multiple frames is smoothed to obtain a smooth position, which can be as follows:

[0064] BP = /

[0065] This formula can be used to calculate the weighted average position, also known as the smoothed position BP, where BP(n) is the position of the nth cached frame in the future. n is also known as the weight, which reflects the importance of each cached frame in predicting the future position. Optionally, the larger the value of n, the greater the impact of the frame on the future position.

[0066] By taking a weighted average of the positions of the cached frames, the future position of the cue ball can be smoothed out, reducing the error caused by single-frame fluctuations. This smoothing process helps reduce prediction errors caused by single-frame fluctuations or noise.

[0067] Optionally, the above-mentioned determination of the movement step of the spherical target with respect to the current frame based on the smooth position and the number of future frames can be carried out in the following manner: the movement step of the spherical target with respect to the current frame is determined based on the ratio between the product of the smooth position and a preset speed control parameter and the number of future frames.

[0068] It can be understood that the movement step of the current frame is the distance or position adjustment amount that needs to be moved in the current frame in order to maintain accurate tracking of the spherical target.

[0069] For example, the movement step of the current frame can be calculated as follows:

[0070] BP step =

[0071] In the formula, BP step is the moving step of the current frame, that is, the step that the spherical target needs to move in the current frame, nmax is the number of future frames, that is, the total number of cached frames, and c is a parameter for controlling the speed. This parameter can determine the size of the moving step. A larger c value means a faster response speed.

[0072] This calculation method achieves the technical effect of smooth sphere target tracking by ensuring that the motion step of the current frame is proportional to the predicted position of the cached frame, thereby maintaining a consistent step behavior under different numbers of cached frames. Then, in each frame, the position of the sphere can be updated according to the calculated motion step. The new position is obtained by adding the motion step to the current smooth position, thereby achieving smooth tracking of the sphere and reducing image jitter.

[0073] In an optional embodiment, the speed control parameter may be determined as follows: the speed control parameter is determined based on the average value of the relative position change between the current position of the spherical target and the spherical target position in the historical frame; wherein the speed control parameter is proportional to the average value of the relative position change.

[0074] In this embodiment, the relative position change amplitude between the positions of the spherical target in the historical frames, that is, the frame difference between the historical frames, can be used to indicate the speed of the movement of the spherical target. For example, the larger the average value of the relative position change amplitude, the faster the spherical target moves; conversely, the smaller the average value of the relative position change amplitude, the smaller the speed of the spherical target moves.

[0075] In one example, the relative position change amplitude between the positions of the spherical targets in the historical frames can be calculated by the position difference of the spherical targets in each pair of consecutive historical frames (i.e., two consecutive historical frames), such as by obtaining the position coordinates of the spherical targets in every two frames, and using the position coordinates of the spherical targets to perform difference calculation to obtain the position difference between every two frames, and then by accumulating the position difference between every two frames and performing average calculation, the average value of the relative position change amplitude between the positions of the spherical targets in the historical frames can be obtained.

[0076] In another exemplary embodiment, the relative position change amplitude between the positions of the spherical target in the historical frames can also be achieved by calculating the pixel value difference of the pixel points at the position of the spherical target for each pair of consecutive historical frames (i.e., two consecutive historical frames), such as by calculating the absolute difference: D(x, y) = | I t (x, y) – I t-1 (x, y)|, where D(x, y) ) is the pixel value at position (x, y) in the difference image (i.e., the difference image between each pair of consecutive historical frames), and I t (x, y) and I t-1 (x, y) are the pixel values ​​at position (x, y) of the current frame and the previous frame respectively. By accumulating the frame differences of all historical frames, a total frame difference size can be obtained, such as by summing or averaging all frame difference values. A larger frame difference indicates that the spherical target moves faster, and the value of c can be dynamically increased to improve the response speed; if the frame difference is small, it indicates that the target moves slowly, and the value of c can be dynamically reduced to improve stability.

[0077] Optionally, the speed control parameter c may be dynamically adjusted based on the size of the frame difference through a linear formula, or the speed control parameter may be increased year-on-year based on the size of the frame difference, which is not particularly limited in this embodiment.

[0078] By dynamically adjusting the speed control parameters, the smooth spherical target can be adaptively responded to the movement changes of the spherical target during tracking, improving the tracking accuracy. When the spherical target moves faster, the step is increased to increase the response speed, and when the target moves slower, the step is reduced to improve stability. In other words, by using the frame difference information of the historical frame to adjust the speed control parameters, the performance of the target tracking algorithm can be significantly improved, so that it can maintain good tracking effects under different motion conditions.

[0079] It should be noted that, in addition to being determined in the above manner, it can also be customized by the user based on experience.

[0080] It can be seen that using cached frame weighting in the process of spherical target tracking, smoothing the current frame position based on the predicted position of future time points, and then calculating the current moving step by the position of future time points to determine the main ball position can significantly prevent the picture from shaking violently with the ball.

[0081] Figure 3 A flowchart of another ball target tracking method provided in an embodiment of the present application is provided. Based on the above embodiment, this embodiment takes into account that the detection algorithm detects multiple balls (such as misidentified as balls) or the court includes other balls (non-cluster balls, i.e. non-ball targets), in order to further improve the ball target tracking accuracy. In addition to the above steps S201-S204, this embodiment searches for ball targets in the court image after the field of view is expanded according to the first search area in step S202, including:

[0082] Step S2021: In response to not identifying a spherical target in the court image, the field of view used to capture the court image is expanded according to a preset field of view expansion rule to obtain a court image with an expanded field of view.

[0083] It should be noted that the process of expanding the field of view has been introduced in the above embodiments, and the relevant description will not be repeated here.

[0084] Step S2022: If multiple spheres to be identified are searched in the first search area, the sphere target score of each sphere to be identified is calculated according to at least one of the motion index of each sphere to be identified, the sphere confidence, the distance between the sphere position and the last sphere target position identified, and the time when the sphere to be identified is searched, to obtain a sphere target score result; wherein the motion index is determined according to the absolute motion amount and the relative motion amount between relative frames of the sphere to be identified.

[0085] It can be understood that there are multiple balls to be identified, that is, multiple balls detected by the detection algorithm in the image processing device, and there is usually only one target ball in the game, that is, the main ball. This embodiment combines the motion index of the ball to be identified, the ball confidence, the distance between the ball position and the last identified target position of the ball, and the time when the ball to be identified is searched, etc., to denoise the noise ball to improve the accuracy of ball recognition and tracking.

[0086] In one embodiment, the motion index can be determined in the following manner: the absolute motion amount of the sphere to be identified is determined based on the position difference between the starting position and the end position of the sphere to be identified; the relative motion amount of the sphere to be identified is determined based on the average value of the position difference between the current frame position and the previous frame position of the sphere to be identified for each frame; and the motion index is obtained based on the product of the absolute motion amount and the relative motion amount.

[0087] Optionally, the motion index mindx can be determined based on the absolute motion amount of the sphere to be identified and the relative motion amount between relative frames, and the calculation method thereof can be as follows:

[0088] mindx = |Ps - Pe| * Σ(|P(n)-P(n-1)|)

[0089] In the formula, P(n) is the position of the current frame, P(n-1) is the position of the previous frame, Ps is the starting position of the ball, and Pe is the end position of the ball. Among them, |Ps - Pe| is the absolute motion, and |P(n)-P(n-1)| is the relative motion between relative frames. It can be seen from the above formula that the absolute motion of the motion index spherical target tracking is proportional to the relative motion between relative frames. It can be understood that the starting position Ps refers to the position of the ball that was successfully detected for the last time before the target was lost, and the end position Pe refers to the position after the ball was re-detected. In the case of a lost target, the end position can be determined by prediction or estimation until the ball is detected again.

[0090] In this embodiment, the absolute motion provides overall displacement information, while the relative inter-frame motion provides detailed motion information. By combining the absolute motion information and relative motion information of the spherical target to calculate the motion index, a comprehensive evaluation of the motion behavior of the spherical target can be provided, thereby quantifying the motion characteristics of the spherical target and improving the accuracy of identifying the spherical target.

[0091] Exemplarily, the above-mentioned step S2022 calculates the sphere target score of each sphere to be identified according to at least one of the motion index of each sphere to be identified, the sphere confidence, the distance between the sphere position and the last identified sphere target position, and the time when the sphere to be identified is searched. The following method can be adopted: for each sphere to be identified, the first credibility information of the sphere to be identified is determined according to the product of the motion index of the sphere to be identified and the sphere confidence; the second credibility information of the sphere to be identified is determined according to the ratio of the first credibility information and the distance between the sphere position of the sphere to be identified and the last identified sphere target position; the sphere target score of the sphere to be identified is determined according to the ratio of the second credibility information and the time when the sphere to be identified is searched, so as to obtain the sphere target score of each sphere to be identified.

[0092] In this example, there are multiple balls in the first search area (such as the current frame search area with the search time threshold determined by the current frame or the cache frame search area with the search time threshold determined by the cache frame as exemplified in the above embodiment), and the cue ball is selected to improve the accuracy of cue ball recognition. The cue ball selection uses a comprehensive scoring method, and the ball with the highest comprehensive score is selected as the cue ball. The score is related to the motion index, the confidence of the ball, and the position of the ball. The calculation method can be as follows:

[0093] score = (conf * mindx) / diff / t2;

[0094] Among them, score is the comprehensive score, that is, the ball target score; conf is the confidence of the calculated ball (for example, it can be determined according to the location of the stadium area, the confidence of the ball to be identified in the audience area is low, and the confidence of the ball to be identified in the competition area is high), mindx is the movement index of the ball, diff is the distance (straight-line distance) between the ball and the lost ball position (that is, the position of the last identified ball target), and t2 is the time difference between the time the ball is identified and the time the ball is lost. Among them, conf * mindx is the first credibility information, (conf * mindx) / diff is the second credibility information. From the above formula, it can be seen that the higher the confidence of the ball, the greater the motion index, the closer to the lost ball position, the shorter the time to the lost ball point, and the higher the score of the ball.

[0095] Step S2023: According to the spherical target scoring result, a sphere with the highest score is selected from the multiple spheres to be identified to determine the spherical target.

[0096] By selecting the cue ball based on the above-mentioned ball target scoring method, the ball tracking process can remove other balls with high noise identified by the detection algorithm (such as white sneakers, human heads with less hair, and even brighter lawns mistakenly detected as balls by artificial intelligence. This phenomenon is more likely to occur under low illumination or when the camera is installed far away from the court, as well as when there are stationary balls on the side of the court, or when non-participating players are playing but are captured by the camera). Its anti-noise effect is better, thereby selecting a cue ball with higher accuracy, thereby improving the accuracy of ball target tracking.

[0097] It should be noted that the process of identifying the spherical target in the court image before expanding the field of view angle and the process of identifying the spherical target after following a person are both applicable to the above-mentioned spherical target identification process, and the relevant explanations will not be repeated here.

[0098] Figure 4 A flowchart of another ball target tracking method provided by an embodiment of the present application is provided. Based on the above embodiment, this embodiment determines the tracking player target and position by clustering a single frame, and smoothes the position of the tracking player target and performs ball search by weighting multiple frames, which can further improve the accuracy of cue ball recognition and tracking. Specifically, the above step S203 includes the following steps S2031-S2033.

[0099] Step S2031: If the spherical target is still not identified, the position weight of each human target is determined according to the layout information of the court and the position of each human target, and the human targets to be clustered whose position weight is greater than a preset weight threshold are obtained.

[0100] The stadium layout information is pre-known information, which may include the size, boundaries, key areas (such as goals, penalty areas), and spectator seats of the stadium.

[0101] Step S2032: clustering the human targets to be clustered according to a predetermined number of clusters to obtain clustering results; wherein the number of clusters is determined according to the number of competition areas of the stadium.

[0102] Taking a football match as an example, since the players may be relatively scattered, and there may even be two matches on a single field, the players are clustered according to their positions in order to accurately determine the target and position of the players to be tracked. The number of clusters to be determined can be determined based on the number of competition areas in the field. For example, in a football match, there are usually only two matches on the field at most. Moreover, too many centroids do not have much effect on the selection of the initial position. Two centroids can be selected, that is, two cluster numbers are determined, and human targets are clustered to obtain the clustering results corresponding to the two centroids. The clustering method can use K-means clustering.

[0103] Step S2033: Determine a tracked player target according to the clustering result and the position of the ball target identified last time, and search for the ball target according to the tracked player target and the second search area.

[0104] For example, for each clustering result, the distance from the centroid of each cluster to the last ball target position is calculated, and the centroid of the player close to the cue ball position in the last frame is selected as the initial position, and the player corresponding to the centroid of the player (or the player closest to the centroid) is determined as the tracking player target. It can be understood that in cluster analysis, the centroid refers to the center point of a cluster, which can be calculated by the average value of all data points in the cluster (that is, the positions of each human body target).

[0105] Step S2034: searching for a ball target according to the tracked player target and the second search area.

[0106] After determining the target player to be tracked, the player can be tracked as the base point, and the spherical target can be searched in the corresponding second search area by following the person. Through the above single-frame clustering method, the location where the players gather can be quickly located, and the target player to be tracked can be accurately identified, thereby further improving the efficiency and accuracy of spherical tracking and identification.

[0107] In order to further improve the accuracy of the cue ball search process and the smoothness of the picture, combined with Figure 5 As shown, this embodiment combines single-frame clustering and multi-frame weighting to perform player tracking and cue ball search. Specifically, the above step S2034 may include the following steps: determining the transition position of the tracked player according to the initial position of the tracked player identified and the position of the ball target identified last time; obtaining the multi-frame clustering position of the tracked player with respect to the current frame according to the transition position of the tracked player and the position where the tracked player is predicted to appear in the corresponding future frame; searching for the ball target according to the multi-frame clustering position of the tracked player and the second search area.

[0108] As mentioned above, the initial position is the center of mass of the player that is closer to the cue ball position in the last frame. However, since the position of the player is different from the position of the ball, this embodiment uses the following formula to transition the position of the ball in the last frame (i.e., the position where the ball target was recognized last time) to the position of the player:

[0109] P = (BPlast * (maxT – t3) + PPinit * t3) / maxT; (t3 <= maxT)

[0110] Where P is the transition position of the tracked player, BPlast is the position of the main ball in the last frame, PPinit is the initial player position found by clustering (i.e., the initial position of the tracked player), t3 is the time, and maxT is the transition time.

[0111] After losing the ball, directly tracking the fast-moving ball may cause image jitter because the movement of the ball is usually faster and more irregular than that of the player. This embodiment combines the last recognition of the transition from the main ball position to the player's position to use the player's relatively stable movement to smooth the image changes and reduce jitter.

[0112] In the subsequent tracking, the cue ball needs to be continuously searched. If the cue ball cannot be detected, multiple players in a single frame need to be clustered through multi-frame weighted tracking. After clustering, the class that is closer to the last tracking center is selected as the tracking target. The multi-frame tracking method can be as follows:

[0113] After multi-frame cache, players are clustered and their positions are weighted to smooth out the positions of players after multiple frames. PP = / , where PP is the current frame position (i.e., the multi-frame clustered position of the tracked player with respect to the current frame), and PP(n) is the position of the tracked player predicted in the nth cache frame. And the current frame can be stepped forward to achieve smooth tracking: PPstep = , where PPstep is the step required for the current frame to move, nmax is the total number of cached frames, and c is the parameter for controlling the speed.

[0114] During the process of following a person, if the main ball is found, the position of the main ball can be smoothly switched to. The smooth transition method is similar to switching from spherical target tracking to following a person, and the relevant instructions will not be repeated here.

[0115] By caching the historical position data of multiple frames, a smooth motion trajectory can be calculated. Compared with single-frame data, this method can better capture the overall trend of the movement, rather than being affected by the noise or anomalies that may exist in a single frame, making the captured movement of the ball more stable and reducing the image jitter caused by fast or irregular movement.

[0116] To facilitate understanding of the embodiments of the present application, Figure 6 As shown, the following process is included:

[0117] After the ball target tracking starts, the ball target tracking can be started at any time, for example, after the game starts by collecting the court image, or the ball target tracking can be started in the middle of the game;

[0118] Determine whether the cue ball is recognized, that is, whether the spherical target is detected. If the cue ball is recognized, perform cue ball tracking;

[0119] If the cue ball is not identified, the expanded field of view angle is expanded, and the cue ball is searched in the first search area. In the process of searching for the cue ball, a cue ball that meets the requirements (such as selecting the ball with the highest score in the ball target scoring result) is selected from the multiple identified balls, and the cue ball is tracked;

[0120] The first search area is an area range that gradually expands over time. By performing a waiting count, it is determined whether the cue ball is recognized. If the cue ball is not recognized and the timeout has not occurred (i.e., the search time threshold), the first search area range is expanded to detect the ball; if the cue ball is not recognized and the timeout has occurred, the ball target tracking is switched to the person tracking.

[0121] The following person method can search for the cue ball by expanding the second search area over time, and select the cue ball that meets the requirements. If the cue ball is identified, switch from following the person to spherical target tracking, and track the cue ball. Otherwise, continue to follow the person and search for the cue ball based on the second search area that expands over time, until the cue ball is found, and track the cue ball.

[0122] Through the above technical scheme, the joint tracking method of people and balls can be adopted to effectively prevent the problem of large shaking or stillness of the picture tracking when the ball is blocked for a long time, or the ball is kicked out of the court, or the noise is so loud that the ball cannot be recognized, or there is no ball (such as the cheerleading team is on the court); based on the processing method after the ball is lost, the field of view is expanded, the search area is gradually expanded, and the main ball is selected using comprehensive indicators such as motion index, confidence, position relationship, and time relationship, which can effectively reduce the ball loss rate and select a more accurate main ball; by calculating the search area, it can effectively prevent the rapid switching of the expanded perspective and cause jitter, and it can prevent the expanded perspective from frequently switching between multiple competition areas in the case of multiple regional games on the same court; continuous spherical target tracking or continuous following method uses cache frame weighting to calculate the position of future time points, and then calculates the current moving step by the position of future time points. This method can significantly prevent the picture from shaking violently with the ball. In addition, the weighted clustering method with philosophies effectively reduces the influence of spectators on the sidelines and improves the accuracy of replacing the cue ball position with the player center when players are dispersed in transition or when there are two games on the same court.

[0123] Figure 7 is a schematic diagram of the structure of a spherical target tracking device provided in an embodiment of the present application, such as Figure 7 As shown, the device includes a joint identification module 701, a first search module 702, a second search module 703 and a spherical target tracking module 704, wherein:

[0124] A joint recognition module 701 is configured to recognize a ball target and a human target in the court image according to a detection result of the court image after the court image is acquired;

[0125] A first search module 702 is configured to, in response to the failure to identify the ball target in the court image, expand the field of view used to acquire the court image according to a preset field of view expansion rule to obtain the court image after the field of view is expanded, and search for the ball target in the court image after the field of view is expanded according to a first search area; wherein the first search area is determined according to the position where the ball target was identified last time and the first search time;

[0126] The second search module 703 is configured to determine a tracked player target from the identified human targets if the spherical target is still not identified, and search for the spherical target based on the tracked player target and a second search area; wherein the second search area is determined based on the position of the tracked player target and a second search time;

[0127] The spherical target tracking module 704 is configured to determine the position of the spherical target according to a preset spherical target tracking algorithm in response to identifying the spherical target.

[0128] In one implementation, the position of the spherical target recognized last time includes a horizontal axis coordinate position and a vertical axis coordinate position, and the first search module 702 includes:

[0129] A first interval determination unit is configured to determine a horizontal axis search interval that is gradually expanded with a first search time according to the horizontal axis coordinate position and the spherical body speed calculated when the spherical body target was recognized last time;

[0130] A second interval determination unit is configured to determine a longitudinal axis search interval that is gradually expanded with the first search time according to the longitudinal axis coordinate position and the spherical body speed calculated when the spherical body target was recognized last time;

[0131] an area determination unit, configured to determine the first search area according to the horizontal axis search interval and the vertical axis search interval;

[0132] The ball speed is obtained according to the pixel change per unit time when the ball target was recognized last time; the first search time is the real-time change time from the start time when the ball target is not recognized to the search time threshold.

[0133] In one embodiment, the first interval determination unit is specifically configured to: determine the first horizontal axis boundary of the horizontal axis search interval based on the horizontal axis coordinate position and the difference between the speed calculated when the spherical target was last identified and the product result of the first search time and a preset expansion coefficient; determine the second horizontal axis boundary of the horizontal axis search interval based on the horizontal axis coordinate position and the sum of the speed calculated when the spherical target was last identified and the product result of the first search time and the expansion coefficient; obtain the horizontal axis search interval based on the first horizontal axis boundary and the second horizontal axis boundary.

[0134] In one embodiment, the second interval determination unit is specifically configured to: determine the first longitudinal axis boundary of the longitudinal axis search interval according to the longitudinal axis coordinate position and the difference between the speed calculated when the spherical target was last identified and the product result of the first search time and a preset expansion coefficient; determine the second longitudinal axis boundary of the longitudinal axis search interval according to the longitudinal axis coordinate position and the sum of the speed calculated when the spherical target was last identified and the product result of the first search time and the expansion coefficient; obtain the longitudinal axis search interval according to the first longitudinal axis boundary and the second longitudinal axis boundary.

[0135] In one implementation, the first search module 702 includes:

[0136] a sphere scoring unit, configured to calculate a sphere target score for each sphere to be identified according to at least one of a motion index of each sphere to be identified, a sphere confidence, a distance between the sphere position and the last sphere target position identified, and a time when the sphere to be identified is searched, if multiple spheres to be identified are searched in the first search area, to obtain a sphere target scoring result;

[0137] A scoring selection unit, configured to select a sphere with the highest score from the plurality of spheres to be identified according to the sphere target scoring result, and determine the sphere target;

[0138] The motion index is determined based on the absolute motion amount and the relative motion amount between frames of the sphere to be identified.

[0139] In one embodiment, it also includes an index determination module for determining the motion index, and the index determination module includes: a first motion determination unit, used to determine the absolute motion of the sphere to be identified based on the position difference between the starting position and the end position of the sphere to be identified; a second motion determination unit, used to determine the relative motion of the sphere to be identified based on the average value of the position difference between the current frame position and the previous frame position of the sphere to be identified for each frame; the motion index is obtained according to the product of the absolute motion and the relative motion.

[0140] In one embodiment, the sphere scoring unit is specifically configured to determine, for each sphere to be identified, first credibility information of the sphere to be identified based on the product of the motion index of the sphere to be identified and the sphere confidence; determine second credibility information of the sphere to be identified based on the ratio of the first credibility information and the distance between the sphere position of the sphere to be identified and the last identified sphere target position; determine the sphere target score of the sphere to be identified based on the ratio of the second credibility information and the time of searching for the sphere to be identified, so as to obtain a sphere target score for each sphere to be identified.

[0141] In one embodiment, the device further comprises:

[0142] A cache time acquisition unit, configured to acquire a cache retrieval frame time according to a position of a target ball predicted to appear in a future frame corresponding to the court image after the expanded field of view, wherein the cache retrieval frame time is a time for predicting the appearance of the target ball;

[0143] A time threshold determination unit is configured to determine the cache retrieval frame time as the search time threshold.

[0144] In one implementation, the second search module 703 includes:

[0145] A weight acquisition unit, which is configured to determine the position weight of each human target according to the layout information of the court and the position of each human target, and acquire the human targets to be clustered whose position weight is greater than a preset weight threshold;

[0146] A single-frame clustering unit is configured to cluster the human targets to be clustered according to a predetermined number of clusters to obtain a clustering result; wherein the number of clusters is determined according to the number of competition areas of the stadium;

[0147] The player determination unit is configured to determine the player target to be tracked based on the clustering result and the position of the spherical target identified last time.

[0148] In one implementation, the second search module 703 includes:

[0149] a transition unit, configured to determine a transition position of the tracked player according to the identified initial position of the tracked player and the position of the ball target identified last time;

[0150] A multi-frame clustering unit, configured to obtain a multi-frame clustering position of the tracked player with respect to a current frame according to a transition position of the tracked player and a position of the tracked player predicted to appear in a corresponding future frame;

[0151] A searching unit is configured to search for a spherical target according to the multi-frame clustered position of the tracked player and the second searching area.

[0152] In one implementation, the spherical target tracking module 704 includes:

[0153] a smoothing unit configured to determine a smoothed position of the spherical target with respect to a current frame according to a position of the spherical target predicted to appear in a future frame corresponding to the current position of the spherical target;

[0154] a step determination unit, configured to determine a movement step of the spherical object with respect to a current frame according to the smoothed position and the number of future frames;

[0155] A tracking unit is configured to obtain the position of the spherical target according to the smoothed position and the moving step.

[0156] In one embodiment, the smoothing unit is specifically configured to calculate the weighted average value between the current position of the spherical target and the position of the spherical target predicted to appear in each frame in the future frames based on a weighted average algorithm to determine the smoothed position of the spherical target with respect to the current frame; and / or, determining the moving step of the spherical target with respect to the current frame based on the smoothed position and the number of future frames, including: determining the moving step of the spherical target with respect to the current frame based on the ratio between the product of the smoothed position and a preset speed control parameter and the number of future frames.

[0157] In one embodiment, the step determination unit is specifically configured to determine the speed control parameter based on the average value of the relative position change amplitude between the current position of the spherical target and the spherical target position in the historical frame; wherein the speed control parameter is proportional to the average value of the relative position change amplitude.

[0158] The spherical target tracking device provided in the above embodiment can be used to execute the spherical target tracking method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0159] Figure 8 An image processing device provided in an embodiment of the present application, such as Figure 8 As shown, the image processing device includes: a processor 802, a memory 801 and a display 803 which are communicatively connected to the processor;

[0160] The memory 801 stores computer-executable instructions;

[0161] The processor 802 executes the computer-executable instructions stored in the memory 801 to implement the spherical object tracking method provided in any one of the first aspects. The display 803 is used to display the image after image processing.

[0162] The image processing device provided in the above embodiment can be used to execute the spherical target tracking method in any of the above method embodiments. Its implementation principle and technical effect are similar and will not be repeated here.

[0163] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the spherical target tracking method provided in the first aspect as described above.

[0164] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0165] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0166] The computer-readable storage medium provided in the above embodiment can be used to execute the spherical target tracking method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0167] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution provided by any of the above method embodiments can be implemented.

[0168] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0169] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0170] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0171] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A spherical target tracking method, characterized in that: include: After collecting the court image, identifying the ball target and the human target in the court image according to the detection result of the court image; In response to the failure to identify the spherical target in the court image, the field of view used to capture the court image is expanded according to a preset field of view expansion rule to obtain the court image after the field of view is expanded, and the spherical target in the court image after the field of view is expanded is searched according to a first search area; wherein the first search area is determined according to the position where the spherical target was identified last time and the first search time; If the spherical target is still not identified, a tracking player target is determined from the identified human targets, and the spherical target is searched based on the tracking player target and a second search area; wherein the second search area is determined based on the position of the tracking player target and a second search time; In response to identifying the spherical target, the position of the spherical target is determined according to a preset spherical target tracking algorithm.

2. The method according to claim 1, characterized in that The position of the spherical target recognized last time includes a horizontal axis coordinate position and a vertical axis coordinate position, and the method for determining the first search area includes: Determine a horizontal axis search interval that is gradually expanded with the first search time according to the horizontal axis coordinate position and the spherical speed calculated when the spherical target was recognized last time; Determine a longitudinal search interval that is gradually expanded with the first search time according to the longitudinal coordinate position and the spherical speed calculated when the spherical target was recognized last time; Determining the first search area according to the horizontal axis search interval and the vertical axis search interval; The ball speed is obtained according to the pixel change per unit time when the ball target was recognized last time; the first search time is the real-time change time from the start time when the ball target is not recognized to the search time threshold.

3. The method according to claim 2, characterized in that According to the horizontal axis coordinate position and the spherical speed calculated when the spherical target was recognized last time, determining a horizontal axis search interval that is gradually expanded with the first search time, including: Determine a first horizontal axis boundary of the horizontal axis search interval according to the horizontal axis coordinate position and the difference between the speed calculated when the spherical target was recognized last time and the product of the first search time and the preset expansion coefficient; Determine a second horizontal axis boundary of the horizontal axis search interval according to the horizontal axis coordinate position and the sum of the speed calculated when the spherical target was recognized last time and the product of the first search time and the expansion coefficient; The horizontal axis search interval is obtained according to the first horizontal axis boundary and the second horizontal axis boundary.

4. The method according to claim 2, characterized in that: According to the longitudinal axis coordinate position and the spherical speed calculated when the spherical target was recognized last time, determining a longitudinal axis search interval that is gradually expanded with the first search time, including: Determine a first longitudinal axis boundary of the longitudinal axis search interval according to the longitudinal axis coordinate position and the difference between the speed calculated when the spherical target was recognized last time and the product of the first search time and the preset expansion coefficient; Determine a second longitudinal axis boundary of the longitudinal axis search interval according to the longitudinal axis coordinate position and the sum of the speed calculated when the spherical target was recognized last time and the product of the first search time and the expansion coefficient; The vertical axis search interval is obtained according to the first vertical axis boundary and the second vertical axis boundary.

5. The method according to any one of claims 1 to 4, characterized in that: The step of searching for the spherical target in the court image after the field of view is expanded according to the first search area includes: If multiple spheres to be identified are searched in the first search area, then according to at least one of the motion index of each sphere to be identified, the sphere confidence, the distance between the sphere position and the last sphere target position identified, and the time when the sphere to be identified is searched, the sphere target score of each sphere to be identified is calculated to obtain a sphere target score result; According to the spherical target scoring result, a sphere with the highest score is selected from the multiple spheres to be identified to determine the spherical target; The motion index is determined based on the absolute motion amount and the relative motion amount between relative frames of the sphere to be identified.

6. The method according to claim 5, characterized in that The method for determining the motion index includes: Determining the absolute movement amount of the sphere to be identified according to the position difference between the starting position and the ending position of the sphere to be identified; Determining the relative motion amount of the sphere to be identified according to an average value of the position difference between the current frame position and the previous frame position of the sphere to be identified for each frame; The motion index is obtained according to the product of the absolute motion amount and the relative motion amount.

7. The method according to claim 5, characterized in that The step of calculating the ball target score of each ball to be identified according to at least one of the motion index of each ball to be identified, the ball confidence, the distance between the ball position and the last identified ball target position, and the time of searching for the ball to be identified comprises: For each sphere to be identified, determining first credibility information of the sphere to be identified according to the product of the motion index of the sphere to be identified and the sphere confidence; Determine the second credibility information of the sphere to be identified according to the ratio of the first credibility information to the distance between the sphere position of the sphere to be identified and the last identified sphere target position; The sphere target score of the sphere to be identified is determined according to the ratio between the second credibility information and the time for searching the sphere to be identified, so as to obtain the sphere target score of each sphere to be identified.

8. The method according to any one of claims 2 to 4, characterized in that: Also includes: Obtaining a cache retrieval frame time according to the predicted position of the target ball in the future frame corresponding to the court image after the expanded field of view, wherein the cache retrieval frame time is the time used to predict the appearance of the target ball; The cache retrieval frame time is determined as the search time threshold.

9. The method according to claim 1, characterized in that: Determining the tracking player target from the identified human target includes: According to the layout information of the stadium and the location of each human target, the position weight of each human target is determined, and the human targets to be clustered whose position weight is greater than a preset weight threshold are obtained; Clustering the human targets to be clustered according to a predetermined number of clusters to obtain a clustering result; wherein the number of clusters is determined according to the number of competition areas of the stadium; The player target to be tracked is determined according to the clustering result and the position of the spherical target recognized last time.

10. The method according to claim 1 or 9, characterized in that: The searching for the ball target according to the tracking player target and the second search area comprises: Determining a transition position of the tracked player according to the identified initial position of the tracked player and the position of the spherical target identified last time; Obtaining a multi-frame clustered position of the tracked player with respect to the current frame according to the transition position of the tracked player and the position of the tracked player predicted to appear in the corresponding future frame; The spherical target is searched for according to the multi-frame clustered position of the tracked player and the second search area.

11. The method according to claim 1, characterized in that: Determining the position of the spherical target according to a preset spherical target tracking algorithm includes: Determine the smooth position of the spherical target with respect to the current frame according to the position of the spherical target predicted to appear in the future frame corresponding to the current position of the spherical target; Determining the movement step of the spherical target with respect to the current frame according to the smoothed position and the number of future frames; The position of the spherical target is obtained according to the smoothed position and the moving step.

12. The method according to claim 11, characterized in that The step of determining the smooth position of the spherical target with respect to the current frame according to the position of the spherical target predicted to appear in the future frame corresponding to the current position of the spherical target comprises: Based on a weighted average algorithm, a weighted average value between the current position of the spherical target and the predicted position of the spherical target in each frame in the future is calculated to determine the smooth position of the spherical target with respect to the current frame; And / or, determining the movement step of the spherical target with respect to the current frame according to the smoothed position and the number of future frames, comprises: The moving step of the spherical target with respect to the current frame is determined according to the ratio between the product of the smooth position and the preset speed control parameter and the number of the future frames.

13. The method according to claim 12, characterized in that The speed control parameter is determined by: Determine the speed control parameter according to the average value of the relative position change amplitude between the current position of the spherical target and the position of the spherical target in the historical frame; The speed control parameter is proportional to the average value of the relative position change amplitude.

Images