Target tracking method, device, equipment and storage medium

Abstract

The present invention discloses a target tracking method, device, equipment and storage medium, which relates to the field of computer vision technology. The method includes: obtaining a tracking range and a moving target within the tracking range in a video frame image; determining a tracking target based on the position information of the moving target and the position information of the tracking range to obtain a first tracking target; performing motion trajectory prediction based on the current video frame image and the next video frame image to track the first tracking target in real time or stop tracking; after stopping tracking of the first tracking target, triggering continued tracking, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain a second tracking target, and tracking the second tracking target in real time. The present invention solves the problem of low tracking accuracy in the target tracking method of the prior art when tracking athletes in a sports field, and achieves the effect of improving tracking accuracy and preventing invalid tracking.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target tracking method, device, equipment and storage medium. Background Art

[0002] Object tracking technology has long been a hot topic in the field of computer vision, with broad application prospects in many areas. Currently, object tracking technology has also been introduced in intelligent video surveillance, such as monitoring athletes in sports competitions. This is especially true in competitive sports such as football, basketball, and ice hockey, where monitoring the movement of each athlete is crucial.

[0003] Current methods for tracking athletes in sports fields can easily misidentify spectators as athletes, leading to target tracking of spectators. Furthermore, tracking targets can be lost when athletes move too fast or are crowded together. Therefore, existing target tracking methods suffer from low tracking accuracy. Summary of the Invention

[0004] The main purpose of the present invention is to provide a target tracking method, device, equipment and storage medium, aiming to solve the technical problem of low tracking accuracy when tracking athletes in a sports field in the target tracking method of the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a target tracking method, the method comprising:

[0007] Acquire the tracking range and the moving target within the tracking range in the video frame image;

[0008] Determine a tracking target according to the position information of the moving target and the position information of the tracking range, and obtain a first tracking target;

[0009] Performing motion trajectory prediction based on the current video frame image and the next video frame image to track the first tracking target in real time or stop tracking;

[0010] After stopping tracking of the first tracking target, continuing tracking is triggered, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain a second tracking target, and tracking the second tracking target in real time.

[0011] Optionally, in the target tracking method, the step of acquiring the moving target within the tracking range specifically includes:

[0012] Utilizing the target detection model, detection is performed within the tracking range to obtain the moving target and its position information.

[0013] Optionally, in the target tracking method, the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain the first tracking target specifically includes:

[0014] Obtaining a center point of the tracking range according to the position information of the tracking range;

[0015] Obtaining a distance ratio corresponding to the moving target according to a distance from a first vertex of the moving target detection frame to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation;

[0016] determining whether the distance ratio is less than a preset threshold, wherein the preset threshold is obtained based on the aspect ratio of the tracking range;

[0017] If the distance ratio is less than a preset threshold, the moving target is determined to be the first tracking target.

[0018] Optionally, in the target tracking method, after the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain the first tracking target, the method further includes:

[0019] Classifying the first tracking targets to obtain first tracking targets of different categories;

[0020] The tracking frames of the first tracking targets of different categories are displayed separately in the video frame image.

[0021] Optionally, in the target tracking method, the step of classifying the first tracking target to obtain first tracking targets of different categories specifically includes:

[0022] Enlarging the tracking frame of the first tracking target by a preset multiple to obtain an enlarged tracking frame;

[0023] Cut out the first tracking target from the expanded tracking frame to obtain the cut-out target;

[0024] The extracted target is input into a target classification model, and the category corresponding to the extracted target is output to obtain first tracking targets of different categories.

[0025] Optionally, in the target tracking method, the step of performing motion trajectory prediction based on the current video frame image and the next video frame image to perform real-time tracking or stop tracking of the first tracking target specifically includes:

[0026] Obtaining a motion offset according to a center point of a tracking frame of the first tracking target in a current video frame image and a center point of a tracking frame of the first tracking target in a next video frame image;

[0027] Determining whether the motion offset is within a preset threshold range and whether the distance ratio corresponding to the first tracking target is less than a preset threshold value;

[0028] If the motion offset is within a preset threshold range and the distance ratio corresponding to the first tracking target is less than a preset threshold value, determining that the motion trajectory prediction result of the first tracking target is that the first tracking target is moving normally within the tracking range, and performing real-time tracking of the first tracking target;

[0029] If the motion offset is within a preset threshold range, but the distance ratio corresponding to the first tracking target is not less than a preset threshold value, determining that the motion trajectory prediction result of the first tracking target is that the first tracking target is not within the tracking range, and stopping tracking of the first tracking target;

[0030] If the motion offset is not within a preset threshold range, but the distance ratio corresponding to the first tracking target is less than a preset threshold value, it is determined that the motion trajectory prediction result of the first tracking target is that the first tracking target is tracked abnormally within the tracking range, and tracking of the first tracking target is stopped.

[0031] Optionally, in the above target tracking method, the step of triggering continued tracking specifically includes:

[0032] Identify special events that occur within the tracking range and the corresponding event duration;

[0033] When it is recognized that the special event ends or the event duration is ended, it is determined whether the current timing duration is within a preset duration range, and whether the number of first tracking targets within the tracking range reaches a preset target number;

[0034] If the current timing duration is within the preset duration range, and the number of first tracking targets within the tracking range reaches the preset target number, it is determined that the continued tracking trigger condition is met, and continued tracking is triggered;

[0035] If the current timing duration is within the preset duration range, but the number of first tracking targets within the tracking range does not reach the preset target number, the process returns to the step of identifying special events occurring within the tracking range and continues to monitor the special events.

[0036] In a second aspect, the present invention provides a target tracking device, comprising:

[0037] A tracking range acquisition module, used to acquire the tracking range in the video frame image and the moving target within the tracking range;

[0038] A tracking target determining module, configured to determine a tracking target based on the position information of the moving target and the position information of the tracking range, and obtain a first tracking target;

[0039] A real-time tracking module, configured to predict a motion trajectory based on a current video frame image and a next video frame image, so as to track the first tracking target in real time or stop tracking;

[0040] The associated tracking module is used to trigger continued tracking after stopping tracking of the first tracking target, return to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and track the second tracking target in real time.

[0041] In a third aspect, the present invention provides a target tracking device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the target tracking method as described above is implemented.

[0042] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program can be executed by one or more processors to implement the target tracking method as described above.

[0043] The above one or more technical solutions provided by the present invention may have the following advantages or at least achieve the following technical effects:

[0044] The present invention proposes a target tracking method, device, equipment, and storage medium. By acquiring a tracking range and a moving target within the tracking range, the method identifies a sports field and the people within the sports field. A first tracking target is then determined based on the position information of the moving target and the position information of the tracking range, identifying the athletes within the sports field to prevent spectators from being mistakenly identified as athletes. Then, based on the current video frame image and the next video frame image, a motion trajectory is predicted, and the athletes are tracked or stopped in real time. This prevents spectators from being mistakenly tracked and wastes system computing power. Tracking of athletes leaving the sports field can also be stopped in a timely manner to prevent an increase in system computing power and waste of system resources. Furthermore, by triggering continued tracking, a second tracking target is obtained, and the second tracking target is tracked in real time to prevent the tracking target from being lost. The present invention screens eligible moving targets and only performs real-time tracking after the tracking target is determined, thereby improving tracking accuracy and preventing invalid tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these provided drawings without paying any creative work.

[0046] Figure 1 Schematic diagram of the flow of the first embodiment of the target tracking method of the present invention;

[0047] Figure 2 This is a schematic diagram of the hardware structure of the target tracking device involved in the present invention;

[0048] Figure 3 This is a diagram showing the effect of tracking athletes using a tracking method in the prior art;

[0049] Figure 4 Another effect diagram of tracking athletes using the existing tracking method;

[0050] Figure 5 Schematic diagram of the flow of the second embodiment of the target tracking method of the present invention;

[0051] Figure 6 This is a diagram showing the effect of obtaining the tracking range in step S10 in the second embodiment of the target tracking method of the present invention;

[0052] Figure 7 This is a diagram showing the effect of determining the tracking target in step S30 in the second embodiment of the target tracking method of the present invention;

[0053] Figure 8 for Figure 7 A simplified schematic diagram of

[0054] Figure 9 Schematic diagram of the functional modules of the first embodiment of the target tracking device of the present invention.

[0055] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0057] It should be noted that in the embodiments of the present invention, all directional indications (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0058] In the present invention, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "include..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element. In addition, in the present invention, unless otherwise clearly specified and defined, the terms "connect", "fixed" and the like should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements.

[0059] In the present invention, if there are descriptions involving "first," "second," etc., such descriptions are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the present invention, the use of suffixes such as "module," "component," or "unit" to indicate elements is merely to facilitate the description of the present invention and does not have any specific meaning in itself. Therefore, "module," "component," or "unit" may be used interchangeably.

[0060] Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances. Furthermore, the technical solutions of the various embodiments may be combined with each other, but this must be based on the fact that they can be implemented by those skilled in the art. If the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0061] An analysis of existing technologies reveals that the current method for tracking athletes in sports fields generally involves acquiring the first frame of the video, performing target detection, and obtaining the tracking target and its bounding box; then acquiring the image features and motion features of each tracking target, calculating the similarity, and determining the probability that two tracking targets belong to the same target; finally, associating the tracking targets, assigning a digital ID to each tracking target, and performing tracking display. The tracking effect is as follows: Figure 3 Shown and Figure 4 As shown. Figure 3 It can be seen that when tracking athletes, existing tracking methods are prone to misidentifying spectators as athletes and tracking them as well, which can easily lead to increased system computing power and waste of resources. Figure 4 It can be seen that due to the fast movement speed of athletes in sports events, they are sometimes crowded together. At this time, it is difficult to distinguish the team to which the athletes belong, and the tracking target is easily lost. The target identification and tracking can only be restarted after the tracking target is offset. However, the data of the target will not be synchronized during this period, which may lead to errors in the subsequent monitoring data of the target.

[0062] In view of the technical problem that the target tracking method in the prior art has low tracking accuracy when tracking athletes in a sports field, the present invention provides a target tracking method, the overall concept of which is as follows:

[0063] Acquire a tracking range in a video frame image and a moving target within the tracking range; determine a tracking target based on the position information of the moving target and the position information of the tracking range to obtain a first tracking target; perform motion trajectory prediction based on the current video frame image and the next video frame image to track the first tracking target in real time or stop tracking; when tracking of the first tracking target is stopped, trigger continued tracking, return to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and track the second tracking target in real time.

[0064] Through the above technical solution, the tracking range and the moving target within the tracking range are obtained, and the sports field and the people in the sports field are identified; then, based on the position information of the moving target and the position information of the tracking range, the first tracking target is determined, and the athletes in the sports field are identified to prevent the audience from being mistakenly identified as athletes; then, based on the current video frame image and the next video frame image, the motion trajectory is predicted, and the athletes are tracked in real time or stopped, which can prevent the audience from being mistakenly tracked and wasting system computing power, and can also stop tracking the athletes who leave the sports field in time to prevent increasing the system computing power and wasting system resources; and further, by triggering continued tracking, a second tracking target is obtained, and the second tracking target is tracked in real time to prevent the tracking target from being lost. The present invention screens the moving targets that meet the conditions, and then tracks them in real time after determining the tracking target, thereby improving tracking accuracy and preventing invalid tracking.

[0065] Example 1

[0066] Reference Figure 1 A flow chart of the present invention is provided, which proposes a first embodiment of the target tracking method of the present invention, and the target tracking method is applied to a target tracking device.

[0067] The target tracking device refers to a terminal device or network device that can achieve network connection, which can be a terminal device such as a mobile phone, computer, tablet computer, portable computer, or a network device such as a server or cloud platform.

[0068] like Figure 2 FIG. 1 is a schematic diagram of the hardware structure of a target tracking device, wherein the device may include: a processor 1001 , such as a CPU (Central Processing Unit), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 .

[0069] Those skilled in the art will understand that Figure 2 The hardware structure shown in the figure does not constitute a limitation on the target tracking device of the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0070] Specifically, the communication bus 1002 is used to implement connection and communication between these components;

[0071] The user interface 1003 is used to connect to the client and perform data communication with the client. The user interface 1003 may include an output unit, such as a display screen, an input unit, such as a keyboard, and optionally, the user interface 1003 may also include other input / output interfaces, such as a standard wired interface or a wireless interface.

[0072] The network interface 1004 is used to connect to the backend server and perform data communication with the backend server. The network interface 1004 may include an input / output interface, such as a standard wired interface or a wireless interface, such as a Wi-Fi interface.

[0073] The memory 1005 is used to store various types of data. These data may include, for example, instructions for any application or method in the target tracking device, as well as application-related data. The memory 1005 may be a high-speed RAM memory or a stable memory such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the processor 1001.

[0074] For details, please refer to Figure 2 , the memory 1005 may include an operating system, a network communication module, a user interface module and a computer program, wherein the network communication module is mainly used to connect to the server and perform data communication with the server;

[0075] The processor 1001 is configured to call the computer program stored in the memory 1005 and perform the following operations:

[0076] Acquire the tracking range and the moving target within the tracking range in the video frame image;

[0077] Determine a tracking target according to the position information of the moving target and the position information of the tracking range, and obtain a first tracking target;

[0078] Performing motion trajectory prediction based on the current video frame image and the next video frame image to track the first tracking target in real time or stop tracking;

[0079] After stopping tracking of the first tracking target, continuing tracking is triggered, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain a second tracking target, and tracking the second tracking target in real time.

[0080] Based on the above target tracking device, the following Figure 1 The target tracking method of this embodiment is described in detail in the flowchart shown in FIG. The method may include the following steps:

[0081] Step S10: Acquire the tracking range in the video frame image and the moving target within the tracking range.

[0082] Specifically, first obtain the video stream, mark the tracking range in the video frame image of the video stream according to the shape characteristics of the sports field, and obtain the position information of the tracking range, that is, identify the sports field; then, use the target detection model to detect the moving target within the tracking range. For example, in a sports field, there are not only athletes, but also billboards, sports equipment and other items. These items need to be excluded to ensure that the identified moving targets are people, that is, identify the people in the sports field.

[0083] Step S30: determining a tracking target according to the position information of the moving target and the position information of the tracking range, and obtaining a first tracking target.

[0084] Specifically, in the video frame image, after the tracking range is identified, the tracking range can be framed using the corresponding detection frame. After the moving target is identified, the moving target can also be framed using the target frame. Calculate the center point of the framed tracking range. Based on the framed moving target and the tracking range, as well as the position information of the moving target and the tracking range, calculate the distance from the vertex of the target frame to the center point, and the distance from the vertex of the detection frame in the same orientation as the vertex of the target frame to the center point. Determine whether the ratio of these two distances is less than a preset value. If so, it means that the moving target in the target frame is the tracking target, that is, the athlete in the stadium has been identified, preventing the audience from being mistakenly identified as an athlete. Otherwise, it means that the moving target may be a referee or scorer, etc. Non-athletes within the tracking range will not be tracked subsequently. According to this method, all athletes in the stadium can be obtained, that is, the first tracking target within the tracking range.

[0085] Step S50: performing motion trajectory prediction based on the current video frame image and the next video frame image to perform real-time tracking or stop tracking of the first tracking target.

[0086] Specifically, after identifying the first tracking target, there is no need to identify the tracking range and the first tracking target for the next video frame image. The first tracking target of the current video frame image is directly associated with the tracking target of the next video frame image to obtain a predicted tracking frame of the tracking target, that is, a motion trajectory prediction is performed. Then, the motion offset between the tracking frame of the first tracking target in the current video frame image and the predicted tracking frame in the next video frame image is calculated to determine whether the tracking target corresponding to the predicted frame is the same as the first tracking target marked in the current video frame image. In this way, the first tracking target is continuously tracked, that is, real-time tracking of the first tracking target is achieved, which can prevent the audience from being tracked by mistake and wasting system computing power.

[0087] When an athlete needs to leave the field, such as when a player is being replaced, the first tracking target's position, predicted based on the trajectory, is determined to be within the tracking range. If not, this indicates the athlete has left the field, and tracking of that athlete, and therefore the first tracking target, is stopped. This immediate cessation of tracking for athletes leaving the field prevents increased system computational complexity and waste of system resources.

[0088] Step S70: After stopping tracking of the first tracking target, triggering continued tracking, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and tracking the second tracking target in real time.

[0089] Specifically, for the situation of stopping tracking, it is judged whether the triggering condition for continuing tracking is met. If so, then there is no need to identify the tracking range again. It is only necessary to return to step S30 and determine the tracking target according to the moving target in the tracking range to obtain the second tracking target. Then, recirculating steps S50 and S70, the situation of tracking target loss will not occur like this. That is, when replacing an athlete, when the previous athlete leaves the court, tracking is stopped. When the substitute athlete enters the court, the trigger continues tracking, determines the substitute athlete and the corresponding tracking frame, and tracks it in real time to prevent the tracking target from being lost.

[0090] The target tracking method provided by this embodiment identifies the sports field and the people in the sports field by obtaining the tracking range and the moving targets within the tracking range; then, based on the position information of the moving targets and the position information of the tracking range, determines the first tracking target and identifies the athletes in the sports field, thereby preventing spectators from being mistakenly identified as athletes; then, based on the current video frame image and the next video frame image, predicts the motion trajectory and tracks or stops the athletes in real time, thereby preventing spectators from being mistakenly tracked and wasting system computing power; and can also promptly stop tracking athletes who leave the sports field, thereby preventing an increase in system computing power and wasting system resources; further, by triggering continued tracking, obtains a second tracking target and tracks the second tracking target in real time, thereby preventing the tracking target from being lost. The present invention screens the moving targets that meet the conditions and performs real-time tracking after determining the tracking target, thereby improving tracking accuracy and preventing invalid tracking.

[0091] Example 2

[0092] Based on the same invention concept, Figure 5 , a second embodiment of the target tracking method of the present invention is proposed, and the target tracking method is applied to a target tracking device.

[0093] The following combination Figure 5The target tracking method of this embodiment is described in detail in the flowchart shown in FIG. The method may include the following steps:

[0094] Step S10: Acquire the tracking range in the video frame image and the moving target within the tracking range.

[0095] Specifically, step S10 may include:

[0096] Step S11: Acquire a video frame image;

[0097] Step S12: performing range recognition based on the video frame image to obtain the tracking range and position information in the video frame image.

[0098] Specifically, the sports field in the video frame image is identified, and according to the shape characteristics of the sports field, the corresponding tracking range is obtained by using the circumscribed polygon of the sports field, such as a rectangle or an ellipse. The detection frame of the tracking range in the video frame image is also obtained, and the position information of the four vertices of the rectangle corresponding to the minimum circumscribed rectangle or the minimum inscribed ellipse of the tracking range is recorded, such as coordinate information. Figure 6 The figure shows the effect diagram of the tracking range obtained in this embodiment. In the figure, the sports field is a rectangle. After the sports field in the image is recognized, the rectangular tracking range shown in the figure is obtained.

[0099] Step S13: Utilizing the target detection model, detection is performed within the tracking range to obtain the moving target and its position information.

[0100] Specifically, after obtaining the moving target, the detection frame corresponding to the moving target in the video frame image is also obtained. After determining the sports venue, it is also necessary to eliminate the interference of obstacles in the sports venue, such as sports equipment, billboards and other items, identify the people in it, including all people such as athletes, scorers, referees, etc. within the tracking range, obtain the moving target, and obtain the position information of the moving target, such as coordinate information. Among them, the target detection model is actually an object recognition model. The model can be built with a lightweight model module and trained using the existing deep learning recognition algorithm. Here, the trained recognition model can be directly used, that is, the video frame image marked with the tracking range detection frame is input into the target detection model, and all the moving targets within the tracking range in the video frame image are output, that is, the video frame image with the detection frames of all people marked is obtained.

[0101] Step S30: determining a tracking target according to the position information of the moving target and the position information of the tracking range, and obtaining a first tracking target.

[0102] In order to track the athletes in real time and prevent the target from being lost, the athletes are detected within the tracking range obtained in step S10.

[0103] Specifically, step S30 may include:

[0104] Step S31: obtaining the center point of the tracking range according to the position information of the tracking range;

[0105] Step S32: Obtaining a distance ratio corresponding to the moving target based on a distance from a first vertex of the moving target detection frame to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation;

[0106] Step S33: determining whether the distance ratio is less than a preset threshold, wherein the preset threshold is obtained based on the aspect ratio of the tracking range;

[0107] Step S34: If the distance ratio is less than a preset threshold, the moving target is determined to be a first tracking target.

[0108] In this embodiment, Figure 6 The video frame image is used to detect the moving target and determine the tracking target. The obtained effect is shown in the figure below. Figure 7 For easy viewing, Figure 7 The tracking range and moving target in the video frame image are simplified to obtain Figure 8 The simplified schematic diagram shown in the figure, I represents the detection box of the moving target, and S represents the detection box of the tracking range.

[0109] According to the above steps, the center point O of the sports field is obtained. Then, the position information of the upper left corner vertex of I and the position information of the upper left corner vertex of S are used to determine whether the moving target is a tracking target. Specifically, based on the distance l from the upper left corner vertex of I to the center point O and the distance L from the upper left corner vertex of S to the center point O, the distance ratio corresponding to the moving target is obtained; then, it is determined whether the distance ratio is less than a preset value. The preset value is obtained based on the aspect ratio of the tracking range. Because the tracking range may be larger than the actual sports field, a preset threshold value can be set based on actual conditions to narrow the detection area and obtain an ROI (Region of Interest) area, which can improve the detection speed. If the calculated distance ratio is less than the preset threshold value, it indicates that the moving target is within the ROI area and the moving target is an athlete located in the actual sports field. If the calculated distance ratio is not less than the preset threshold value, it indicates that the moving target is not an athlete and does not need to be tracked subsequently.

[0110] Step S40: classifying the first tracking target so as to distinguish and display them in the video frame image.

[0111] Specifically, after identifying all athletes in the sports field, the existing method is to directly mark all athletes uniformly. This embodiment classifies the athletes and distinguishes and displays the corresponding detection frames on the video frame image, making it easier for monitoring personnel to better track the target.

[0112] Specifically, step S40 may include:

[0113] Step S41: classify the first tracking targets to obtain first tracking targets of different categories.

[0114] More specifically, step S41 may include:

[0115] Step S41.1: Enlarging the tracking frame of the first tracking target by a preset multiple to obtain an enlarged tracking frame;

[0116] Step S41.2: extracting the first tracking target from the expanded tracking frame to obtain the extracted target;

[0117] Step S41.3: Input the extracted target into a target classification model, output the category corresponding to the extracted target, and obtain first tracking targets of different categories.

[0118] Step S42: displaying the tracking frames of the first tracking targets of different categories in the video frame image.

[0119] In the present embodiment, the first tracking targets of different categories promptly belong to the athletes of different teams. The tracking frame of the first tracking target is expanded, such as, on the original basis, expanded 1.4 times, the circumscribed rectangular frame of the original tracking target is expanded into the tracking frame after the expansion that the tracking target is fully contained. Then, based on this tracking frame, the character image of the athlete in the frame is extracted, the extracted character image is input into the target classification model, and the class of the tracking target is obtained, that is, the team to which the athlete belongs. After the categories of all first tracking targets are identified and classified, the first tracking targets with different categories are obtained, that is, the athletes belonging to different teams are obtained, and then the tracking frames of these tracking targets are distinguished and displayed, that is, the athletes of different teams are represented with different colors. The athletes belonging to a team are marked with the tracking frame of the same color. For antagonistic competition events, the athletes of the same team can be better tracked, and the athletes of another team can be distinguished to prevent tracking from being lost. When there is the overlapping situation of the tracking frames of the athletes in the video frame image, the difference of the teams can be distinguished according to the identity information of the athletes.

[0120] Step S50: performing motion trajectory prediction based on the current video frame image and the next video frame image to perform real-time tracking or stop tracking of the first tracking target.

[0121] Specifically, the player position information and player classification information in the first video frame image can be used to associate the corresponding player in the next video frame image, so as to perform motion trajectory prediction, thereby tracking the player in real time or stopping tracking.

[0122] Specifically, step S50 may include:

[0123] Step S51: obtaining a motion offset according to the center point of the tracking frame of the first tracking target in the current video frame image and the center point of the tracking frame of the first tracking target in the next video frame image.

[0124] The motion offset is calculated based on the distance between the center point of the player tracking frame in the current video frame image and the center point of the corresponding player tracking frame in the next video frame image.

[0125] Step S52: determining whether the motion offset is within a preset threshold range, and whether the distance ratio corresponding to the first tracking target is less than a preset threshold value.

[0126] The preset threshold range of the motion offset can be determined according to actual conditions and for different types of motion.

[0127] Step S53: If the motion offset is within a preset threshold range and the distance ratio corresponding to the first tracking target is less than a preset threshold value, it is determined that the motion trajectory prediction result of the first tracking target is that the first tracking target is moving normally within the tracking range, and the first tracking target is tracked in real time.

[0128] Specifically, the distance ratio here is the ratio of the distance between the first vertex of the tracking frame of the first tracking target and the center point of the tracking range to the distance between the second vertex of the tracking range, located in the same orientation as the first vertex, and the center point of the tracking range. If the motion offset is within a preset threshold, it indicates that the athlete in the current video frame and the athlete in the next video frame are indeed the same athlete, and tracking is normal. If the distance ratio corresponding to the athlete is less than the preset threshold, it indicates that the athlete is moving within the tracking range. In other words, a motion trajectory prediction result is obtained for the first tracking target moving normally within the tracking range, and accordingly, the first tracking target can be tracked in real time.

[0129] Step S54: If the motion offset is within the preset threshold range, but the distance ratio corresponding to the first tracking target is not less than the preset threshold value, it is determined that the motion trajectory prediction result of the first tracking target is that the first tracking target is not within the tracking range, and tracking of the first tracking target is stopped.

[0130] Specifically, tracking can be stopped only when the first tracking target is out of the tracking range after multiple consecutive predictions of the motion trajectory. This can avoid situations such as temporarily leaving the playing field to shoot a free throw, which would cause the tracking of the athlete to be stopped. The number of such multiple times can be set according to the specific sport category.

[0131] Step S55: If the motion offset is not within the preset threshold range, but the distance ratio corresponding to the first tracking target is less than the preset threshold value, it is determined that the motion trajectory prediction result of the first tracking target is that the first tracking target is abnormally tracked within the tracking range, and tracking of the first tracking target is stopped.

[0132] Specifically, if the motion offset is not within the preset threshold range, it indicates that the tracking frame may be misaligned, or the tracking targets tracked by the tracking frame may be inconsistent. However, if the athlete corresponding to the tracking frame is still within the field, it indicates that the current situation is a tracking anomaly. To prevent continuous tracking errors or affecting the tracking of other tracking targets, stop tracking the tracking target in time.

[0133] Specifically, if the motion offset is not within a preset threshold range and the distance ratio corresponding to the first tracking target is not less than a preset threshold value, then there may be a tracking anomaly or the athlete has left the stadium, and tracking of the first tracking target is also stopped.

[0134] Step S70: After stopping tracking of the first tracking target, triggering continued tracking, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and tracking the second tracking target in real time.

[0135] During a game, the main reason for stopping athlete tracking is the occurrence of some special events, such as halftime, active timeout, and athlete substitution. When these special events are over and the athletes return to the court, the traditional method requires re-execution of the entire tracking method. This method does not require this. The tracking stop of this method is in a paused state. When the special event is over, it is only necessary to return to the step of determining the athlete, and there is no need to execute steps such as identifying the sports field, saving computing power and system resources.

[0136] Specifically, step S70 may include:

[0137] Step S71: After stopping tracking the first tracking target, identifying special events occurring within the tracking range and corresponding event durations.

[0138] All competitions involve a duration, and this duration can be used to trigger continued tracking. Special events with time limits, such as halftime breaks and voluntary timeouts, can be directly determined to have ended based on their corresponding duration. For emergencies like player substitutions, these special events can be manually monitored to ensure they have concluded.

[0139] Step S72: When it is recognized that the special event ends or the event duration timing ends, it is determined whether the current timing duration is within a preset duration range and whether the number of first tracking targets within the tracking range reaches a preset target number.

[0140] The preset duration range varies based on the sport, generally set based on the regular match duration or the allowed extra time. It's a range of values, with a minimum of, for example, the regular match duration and a maximum of the combined regular match duration and the allowed extra time. When the current special event ends or halftime arrives, the match is determined to be over by determining whether the duration of the game has been within the preset duration range. Furthermore, the number of athletes on the field is determined to have reached the preset target number, which is determined based on the specific sport.

[0141] Step S73: If the current timing duration is within the preset duration range and the number of first tracking targets within the tracking range reaches the preset target number, it is determined that the continue tracking trigger condition is met, and the continue tracking is triggered. The process returns to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain the second tracking target, and the second tracking target is tracked in real time.

[0142] If the current timing duration is within the preset duration range, it means that the game has not ended, and the number of athletes in the stadium meets the set number, which means that all athletes are in the stadium, that is, the substitute players have entered the field. At this time, the continue tracking trigger condition is met, triggering the continue tracking, returning to step S30, and continuing to determine the tracking target, that is, obtaining the second tracking target, and realizing real-time tracking of the substitute athletes.

[0143] Step S74: If the current timing duration is within the preset duration range, but the number of first tracking targets within the tracking range does not reach the preset target number, return to the step of identifying special events occurring within the tracking range and continue to monitor the special events.

[0144] If the current timed duration is within the preset duration, it means the game is not over, but the number of players in the stadium does not meet the set number, which means not all players are in the stadium, that is, the substitutes may not be on the court, or there may be other special times. At this time, the process can return to step S71 to continue monitoring special events.

[0145] Specifically, if the current timing duration exceeds the preset duration range, tracking is terminated regardless of whether the number of first tracking targets reaches the preset target number, that is, there is no need to trigger further tracking.

[0146] The target tracking method provided by this embodiment classifies first tracking targets and displays tracking frames for different categories of first tracking targets separately, facilitating better tracking of athletes from different teams. It also predicts motion trajectories based on the current and next video frames, promptly stopping tracking for athletes who leave the field or experience tracking anomalies to avoid wasting computing power. Furthermore, during a match, when the number of athletes on the field reaches a set number, it triggers continued tracking to prevent loss of the tracked target. This embodiment can more accurately identify athletes on the field, provides strong real-time performance, and quickly track targets.

[0147] Example 3

[0148] Based on the same invention concept, Figure 9 , a first embodiment of a target tracking device of the present invention is proposed, which can be a virtual device applied to a target tracking device.

[0149] The following combination Figure 9 The functional module diagram shown in FIG. 1 is a diagram that describes in detail the target tracking device provided by this embodiment. The device may include:

[0150] A tracking range acquisition module, used to acquire the tracking range in the video frame image and the moving target within the tracking range;

[0151] A tracking target determining module, configured to determine a tracking target based on the position information of the moving target and the position information of the tracking range, and obtain a first tracking target;

[0152] A real-time tracking module, configured to predict a motion trajectory based on a current video frame image and a next video frame image, so as to track the first tracking target in real time or stop tracking;

[0153] The associated tracking module is used to trigger continued tracking after stopping tracking of the first tracking target, return to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and track the second tracking target in real time.

[0154] Furthermore, the tracking range acquisition module may include:

[0155] The moving target detection unit is used to detect the moving target and its position information within the tracking range using the target detection model.

[0156] Furthermore, the tracking target determination module may include:

[0157] a center point acquiring unit, configured to acquire the center point of the tracking range according to the position information of the tracking range;

[0158] a distance ratio obtaining unit, configured to obtain a distance ratio corresponding to the moving target based on a distance from a first vertex of the moving target detection frame to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation;

[0159] a first determining unit, configured to determine whether the distance ratio is less than a preset threshold, wherein the preset threshold is obtained based on the aspect ratio of the tracking range;

[0160] The target determination unit is configured to determine that the moving target is a first tracking target if the distance ratio is less than a preset threshold value.

[0161] Furthermore, the device may further include:

[0162] a target classification module, configured to classify the first tracking target to obtain first tracking targets of different categories;

[0163] The distinguishing display module is used to distinguish and display the tracking frames of the first tracking targets of different categories in the video frame image.

[0164] Furthermore, the target classification module may include:

[0165] a tracking frame processing unit, configured to enlarge the tracking frame of the first tracking target by a preset multiple to obtain an enlarged tracking frame;

[0166] a target extraction unit, configured to extract the first tracking target from the expanded tracking frame to obtain the extracted target;

[0167] The classification unit is used to input the extracted target into a target classification model, output the category corresponding to the extracted target, and obtain first tracking targets of different categories.

[0168] Furthermore, the real-time tracking module may include:

[0169] an offset calculation unit, configured to obtain a motion offset based on a center point of a tracking frame of the first tracking target in a current video frame image and a center point of a tracking frame of the first tracking target in a next video frame image;

[0170] a second judging unit, configured to judge whether the motion offset is within a preset threshold range, and whether the distance ratio corresponding to the first tracking target is less than a preset threshold value;

[0171] a real-time tracking unit, configured to, if the motion offset is within a preset threshold range and the distance ratio corresponding to the first tracking target is less than a preset threshold value, determine that a motion trajectory prediction result of the first tracking target indicates that the first tracking target is moving normally within the tracking range, and perform real-time tracking of the first tracking target;

[0172] a stopping tracking unit, configured to, if the motion offset is within a preset threshold range but the distance ratio corresponding to the first tracking target is not less than a preset threshold value, determine that a motion trajectory prediction result of the first tracking target indicates that the first tracking target is outside the tracking range, and stop tracking the first tracking target;

[0173] and a tracking abnormality unit, configured to determine, if the motion offset is not within a preset threshold range but the distance ratio corresponding to the first tracking target is less than a preset threshold value, that the motion trajectory prediction result of the first tracking target is a tracking abnormality of the first tracking target within the tracking range, and stop tracking the first tracking target.

[0174] Furthermore, the association tracking module may include:

[0175] A special event identification unit, used to identify special events occurring within the tracking range and the corresponding event duration;

[0176] a trigger condition judgment unit, configured to, upon recognizing that the special event has ended or the event duration has ended, determine whether the current timing duration is within a preset duration range, and whether the number of first tracking targets within the tracking range has reached a preset target number;

[0177] a continue tracking triggering unit, configured to determine that a continue tracking triggering condition is satisfied and trigger continue tracking if the current timing duration is within a preset duration range and the number of first tracking targets within the tracking range reaches a preset target number;

[0178] The continuous event monitoring unit is used to return to the step of identifying special events occurring within the tracking range and continuously monitor the special events if the current timing duration is within the preset duration range but the number of first tracking targets within the tracking range has not reached the preset target number.

[0179] It should be noted that the functions that can be realized by each module in the target tracking device provided in this embodiment and the corresponding technical effects achieved can refer to the description of the specific implementation methods in each embodiment of the target tracking method of the present invention. For the sake of brevity of the description, they will not be repeated here.

[0180] Example 4

[0181] Based on the same inventive concept, this embodiment provides a target tracking device, which may include a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, all or part of the steps of each embodiment of the target tracking method of the present invention are implemented.

[0182] Specifically, the target tracking device refers to a terminal device or network device that can achieve network connection, which can be a terminal device such as a mobile phone, computer, tablet computer, portable computer, or a network device such as a server or cloud platform.

[0183] It can be understood that the device may further include a communication bus, a user interface and a network interface.

[0184] The communication bus is used to realize the connection and communication between these components.

[0185] The user interface is used to connect to the client and communicate data with the client. The user interface may include an output unit, such as a display screen, and an input unit, such as a keyboard. Optionally, the user interface may also include other input / output interfaces, such as a standard wired interface and a wireless interface.

[0186] The network interface is used to connect to the backend server and perform data communication with the backend server. The network interface may include an input / output interface, such as a standard wired interface, or a wireless interface, such as a Wi-Fi interface.

[0187] The memory is used to store various types of data, which may include, for example, instructions for any application or method in the target tracking device, as well as application-related data. The memory may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Optionally, the memory may also be a storage device independent of the processor.

[0188] The processor is used to call the computer program stored in the memory and execute the target tracking method as described above. The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic component, and is used to execute all or part of the steps of each embodiment of the target tracking method as described above.

[0189] Example 5

[0190] Based on the same inventive concept, this embodiment provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, an App application store, etc., wherein a computer program is stored on the storage medium, and the computer program can be executed by one or more processors. When the computer program is executed by the processor, all or part of the steps of each embodiment of the target tracking method of the present invention can be implemented.

[0191] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0192] The above descriptions are merely optional embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by utilizing the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly or indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A target tracking method, characterized in that: The method comprises: Acquire the tracking range and the moving target within the tracking range in the video frame image; Determine a tracking target according to the position information of the moving target and the position information of the tracking range to obtain a first tracking target; Performing motion trajectory prediction based on the current video frame image and the next video frame image to track the first tracking target in real time or stop tracking; After stopping tracking of the first tracking target, if a trigger condition for continuing tracking is met, continuing tracking is triggered, returning to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, to obtain a second tracking target, and tracking the second tracking target in real time; The step of determining a tracking target based on the position information of the moving target and the position information of the tracking range to obtain a first tracking target specifically includes: Obtaining a center point of the tracking range according to the position information of the tracking range; Obtaining a distance ratio according to a distance from a first vertex of the detection frame of the moving target to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation; If the distance ratio is less than a preset threshold, the moving target is determined to be the first tracking target; wherein the preset threshold is obtained based on the aspect ratio of the tracking range.

2. The target tracking method according to claim 1, wherein: The step of acquiring the moving target within the tracking range specifically includes: Utilizing the target detection model, detection is performed within the tracking range to obtain the moving target and its position information.

3. The target tracking method according to claim 1, wherein: The step of determining the first tracking target based on the distance from the first vertex of the detection frame of the moving target to the center point and the distance from the second vertex of the tracking range to the center point specifically includes: Obtaining a distance ratio corresponding to the moving target according to a distance from a first vertex of a detection frame of the moving target to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation; determining whether the distance ratio is less than a preset threshold, wherein the preset threshold is obtained based on the aspect ratio of the tracking range; If the distance ratio is less than a preset threshold, the moving target is determined to be the first tracking target.

4. The target tracking method according to claim 1, wherein: After the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range to obtain the first tracking target, the method further includes: Classifying the first tracking targets to obtain first tracking targets of different categories; The tracking frames of the first tracking targets of different categories are displayed separately in the video frame image.

5. The target tracking method according to claim 4, wherein: The step of classifying the first tracking targets to obtain first tracking targets of different categories specifically includes: Enlarging the tracking frame of the first tracking target by a preset multiple to obtain an enlarged tracking frame; Cut out the first tracking target from the expanded tracking frame to obtain the cut-out target; The extracted target is input into a target classification model, and the category corresponding to the extracted target is output to obtain first tracking targets of different categories.

6. The target tracking method according to claim 1, wherein: The step of performing motion trajectory prediction based on the current video frame image and the next video frame image to perform real-time tracking or stop tracking of the first tracking target specifically includes: Obtaining a motion offset according to a center point of a tracking frame of the first tracking target in a current video frame image and a center point of a tracking frame of the first tracking target in a next video frame image; determining whether the motion offset is within a preset threshold range and whether a distance ratio corresponding to the first tracking target is less than a preset threshold value; wherein the distance ratio is a ratio of a distance between a first vertex of a tracking frame of the first tracking target and a center point of a tracking range and a distance between a second vertex of the tracking range and the center point of the tracking range, the second vertex being located at the same orientation as the first vertex; If the motion offset is within a preset threshold range and the distance ratio corresponding to the first tracking target is less than a preset threshold value, determining that the motion trajectory prediction result of the first tracking target is that the first tracking target is moving normally within the tracking range, and performing real-time tracking of the first tracking target; If the motion offset is within a preset threshold range, but the distance ratio corresponding to the first tracking target is not less than a preset threshold value, determining that the motion trajectory prediction result of the first tracking target is that the first tracking target is not within the tracking range, and stopping tracking of the first tracking target; If the motion offset is not within a preset threshold range, but the distance ratio corresponding to the first tracking target is less than a preset threshold value, it is determined that the motion trajectory prediction result of the first tracking target is that the first tracking target is tracked abnormally within the tracking range, and tracking of the first tracking target is stopped.

7. The target tracking method according to claim 1, wherein: If the triggering condition for continuing tracking is met, the step of triggering continuing tracking specifically includes: Identify special events that occur within the tracking range and the corresponding event duration; When it is recognized that the special event ends or the event duration is ended, it is determined whether the current timing duration is within a preset duration range, and whether the number of first tracking targets within the tracking range reaches a preset target number; If the current timing duration is within the preset duration range, and the number of first tracking targets within the tracking range reaches the preset target number, it is determined that the trigger condition for continuing tracking is met, and continuing tracking is triggered; If the current timing duration is within the preset duration range, but the number of first tracking targets within the tracking range does not reach the preset target number, the process returns to the step of identifying special events occurring within the tracking range and continues to monitor the special events.

8. A target tracking device, characterized in that: The device comprises: A tracking range acquisition module, used to acquire the tracking range in the video frame image and the moving target within the tracking range; a tracking target determining module, configured to determine a tracking target according to the position information of the moving target and the position information of the tracking range, and obtain a first tracking target; A real-time tracking module, configured to predict a motion trajectory based on a current video frame image and a next video frame image, so as to track the first tracking target in real time or stop tracking; an associated tracking module configured to, after stopping tracking of the first tracking target, trigger the continuation of tracking if a trigger condition for continuing tracking is met, return to the step of determining the tracking target based on the position information of the moving target and the position information of the tracking range, obtain a second tracking target, and track the second tracking target in real time; The tracking target determination module includes: a center point acquiring unit, configured to acquire the center point of the tracking range according to the position information of the tracking range; a distance ratio acquiring unit, configured to acquire a distance ratio based on a distance from a first vertex of a detection frame of the moving target to the center point and a distance from a second vertex of the tracking range to the center point, wherein the first vertex and the second vertex are located in the same orientation; A target determination unit is configured to determine that the moving target is the first tracking target if the distance ratio is less than a preset threshold value; wherein the preset threshold value is obtained based on the aspect ratio of the tracking range.

9. A target tracking device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the target tracking method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: The computer program can be executed by one or more processors to implement the target tracking method according to any one of claims 1 to 7.

Images