Screen-separated ball game video analysis method and device

Abstract

The invention provides a separation net ball game video analysis method and device, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and computer vision. The method comprises the following steps: segmenting a live broadcast video stream of a net-separated ball game according to score rounds to obtain at least one live broadcast video clip; performing frame-by-frame detection on any live video clip to obtain a detection position of the target sphere in each live video frame in the live video clip; checking the detection positions of the target sphere in the live broadcast video frames based on a fitting result obtained by performing trajectory fitting on the detection positions of the target sphere according to a time sequence to obtain checking positions of the target sphere in the live broadcast video frames; and based on the verification position of the target ball in each live video frame, obtaining at least one of a ball hitting action analysis result and a ball hitting route analysis result for the target ball in the live video clip.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of deep learning and computer vision technology, and more particularly to a method and device for analyzing tennis sports videos. Background Art

[0002] With the continuous improvement of the competitive level of tennis sports, coaches and athletes are paying more and more attention to analyzing athletes' technical movements and tactical applications during training or competition. Timely and reasonable correction and optimization of athletes' technical movements and tactical applications based on the analysis results will help improve and enrich athletes' competitive level. However, the analysis of competition and training videos is mostly done manually, which is inefficient and prone to errors. Summary of the Invention

[0003] The embodiments of the present disclosure provide a method for analyzing tennis sports videos, a device for analyzing tennis sports videos, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, an embodiment of the present disclosure proposes a method for analyzing videos of tennis sports, including: dividing a live video stream of tennis sports according to scores and rounds to obtain at least one live video segment; performing frame-by-frame detection on any live video segment to obtain a detection position of a target sphere in each live video frame in the live video segment; based on a fitting result of trajectory fitting of each detection position of the target sphere in chronological order, verifying the detection position of the target sphere in each live video frame to obtain a verified position of the target sphere in each live video frame; based on the verified position of the target sphere in each live video frame, obtaining at least one of a hitting action analysis result and a hitting line analysis result for the target sphere in the live video segment.

[0005] In a second aspect, the disclosed embodiments propose a video analysis device for tennis sports, comprising: a segmentation module, a detection module, a verification module, and a processing module. The segmentation module is configured to segment the live video stream of tennis sports according to the score and rounds to obtain at least one live video segment; the detection module is configured to perform frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame in the live video segment; the verification module is configured to verify the detection position of the target ball in each live video frame based on the fitting results of the trajectory fitting of each detection position of the target ball in chronological order to obtain the verification position of the target ball in each live video frame; and the processing module is configured to obtain at least one of the batting action analysis result and the batting line analysis result for the target ball in the live video segment based on the verification position of the target ball in each live video frame.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the method for analyzing tennis sports videos as described in any of the above implementation methods when executing the instructions.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the method for analyzing tennis sports videos described in any of the above implementations when executed.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the method for analyzing tennis sports videos as described in any of the above implementations.

[0009] According to the video analysis scheme for tennis sports provided by the embodiment of the present disclosure, analysis can be performed based on the live video stream of tennis sports generated in real time. Specifically, the live video stream can be segmented according to the score round to obtain live video segments corresponding to each score round, and then the live video segments can be used as the granularity or unit for frame-by-frame detection to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using a trajectory fitting method. In this way, at least the accuracy and efficiency of the position detection of the target ball in each live video frame can be improved. Then, based on the verified position of the target ball in each live video frame, when analyzing and identifying the hitting action and / or hitting line of the target ball, not only can the efficiency of real-time analysis of the live video be improved, thereby reducing the delay in obtaining the corresponding analysis results, but also the accuracy of the analysis of the hitting action and / or hitting line used by the athlete in the tennis sports can be improved.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 is an exemplary system architecture in which the present disclosure may be applied; Figure 2A flowchart of a method for analyzing tennis sports videos provided by an embodiment of the present disclosure; Figure 3 A flowchart of another method for analyzing tennis sports videos provided by an embodiment of the present disclosure; Figure 4 A schematic diagram of a tennis sports video analysis system in an application scenario provided by an embodiment of the present disclosure; Figure 5 A structural block diagram of a tennis sports video analysis device provided by an embodiment of the present disclosure; Figure 6 A schematic structural diagram of an electronic device suitable for executing a method for analyzing tennis sports videos provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0013] It should be pointed out that in the technical solutions disclosed herein, the collection, acquisition, storage, processing, transmission, provision, disclosure and application of user personal information (such as sports video information, etc.) are all carried out with the user's knowledge and explicit authorization, comply with the relevant laws and regulations, and do not violate public order and good morals.

[0014] With the continuous improvement of the competitive level of tennis sports (such as badminton, tennis, and table tennis), coaches and athletes are increasingly focusing on analyzing the technical movements and tactical application of athletes during training or competition. This is especially true for young athletes, whose technical movements and tactical application are still immature. Promptly and appropriately correcting and optimizing athletes' technical movements and tactical application based on analysis results can help improve and enrich their competitive level, training methods, and approaches. However, current analysis of match and training videos is mostly done manually, which is inefficient and prone to errors. Therefore, further improvement is needed in the practical application of computer technology and software systems to efficiently analyze the technical and tactical aspects of tennis sports.

[0015] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the disclosed tennis video analysis solution may be applied.

[0016] like Figure 1 As shown, the system architecture 100 may include an anchor client 101 , a live broadcast server 102 , and a user client 103 .

[0017] Network 104 is used to provide a transmission link between the host client 101 and the live broadcast server 102. Network 104 can include various wired and wireless transmission links. Network 105 is used to provide a transmission link between the live broadcast server 102 and the user client 103. Network 105 can include various wired and wireless transmission links.

[0018] Users of host client 101 (e.g., organizers or broadcasters of a tennis match) can use video capture devices such as cameras and microphones to capture live images and audio in real time, recording them in real time to generate a live video stream. Host client 101 can send the recorded live video stream to live streaming server 102. Live streaming server 102 receives the live video stream sent by host client 101 and sends it to user client 103. After receiving the live video stream, user client 103 can play the live video.

[0019] Various applications for realizing information communication between the anchor client 101, the live broadcast server 102, and the user client 103 can be installed, such as video recording applications, video analysis applications, instant messaging applications, etc.

[0020] The host client 101, live streaming server 102, and user client 103 can be either hardware or software. When the host client 101 is hardware, it can be any electronic device with a display screen, a sound receiving device, etc., including but not limited to video recording devices. When the host client 101 is software, it can be installed in any of the electronic devices listed above. It can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here. When the live streaming server 102 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the live streaming server 102 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here. When the user client 103 is hardware, it can be any electronic device with a display screen, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When the user client 103 is software, it can be installed in any of the electronic devices listed above. It can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here.

[0021] The live streaming server 102 can provide various services through various built-in applications. For example, an application related to video analysis of netball sports can be provided. When running this application, the live streaming server 102 can achieve the following effects: it can analyze the live video stream of the netball sports generated in real time. Specifically, the live video stream can be segmented by score round to obtain live video segments corresponding to each score round. Then, frame-by-frame detection can be performed using the live video segments as the granularity or unit to obtain the detected position of the target ball in each live video. Furthermore, the detected position of the target ball in each live video frame can be verified, specifically by using a trajectory fitting method. This can at least improve the accuracy and efficiency of the target ball position detection in each live video frame. Then, based on the verified position of the target ball in each live video frame, when analyzing and identifying the target ball's hitting action and / or hitting path, not only can the efficiency of the real-time analysis of the live video be improved, thereby reducing the delay in obtaining the corresponding analysis results, but also the accuracy of the analysis of the hitting action and / or hitting path used by the athlete in the netball sports can be improved.

[0022] Because video analysis based on live video streams requires significant computing resources and significant computational power, the tennis sports video analysis methods provided in the subsequent embodiments of this disclosure are generally performed by a live streaming server 102, which possesses significant computing power and resources. Accordingly, the tennis sports video analysis device is generally also located within the live streaming server 102. However, it should also be noted that if the host client 101 also possesses sufficient computing power and resources, the host client 101 can also utilize a live video analysis application installed thereon to perform the aforementioned operations delegated to the live streaming server 102, thereby outputting the same results as the live streaming server 102. In particular, in the presence of multiple terminal devices with varying computing capabilities, if the live video analysis application determines that the terminal device it is on possesses significant computing power and sufficient remaining computing resources, it can delegate the aforementioned operations to the terminal device, thereby appropriately alleviating the computational burden on the live streaming server 102. Accordingly, the tennis sports video analysis device can also be located within the host client 101. In this case, the exemplary system architecture 100 may also exclude the live streaming server 102 and the network 104.

[0023] It should be understood that Figure 1 The number of host clients, live broadcast servers, and user clients in the embodiment is merely illustrative. Any number of host clients, live broadcast servers, and user clients may be provided as needed.

[0024] Please refer to Figure 2 , Figure 2This is a flow chart of a method for analyzing tennis sports videos provided by an embodiment of the present disclosure, wherein process 200 includes the following steps: Step 201: Segment a live video stream of a tennis sport according to scores and rounds to obtain at least one live video segment.

[0025] This step is intended to be performed by the above-mentioned execution subject (for example Figure 1 The live broadcast server 102 shown in the figure divides the live video stream of the tennis sport received in real time into scores and rounds to determine at least one (i.e. one or more) live video segments for detection, which start from the entry into the serving phase and end with the end of a score round.

[0026] In some optional implementations of the disclosed embodiments, the execution entity can detect, based on the received live video stream, whether the athletes (or players) of the two opposing teams in the tennis game are respectively in the serving area and the receiving area, and whether the ball has not been detected moving across the net or is stationary for multiple consecutive frames. Specifically, if it is detected that each athlete is in the serving area and the receiving area, and if the ball has not been detected moving across the net or is stationary for multiple consecutive frames, then it can be determined that the tennis game has entered the serving phase. Furthermore, after determining that the tennis game has entered the time period corresponding to the serving phase, the execution entity can determine that a scoring round, i.e., the current scoring round, has ended in response to determining that no hits have occurred for multiple consecutive frames, or that the ball has been stationary for multiple consecutive frames, or that the ball has not been detected moving across the net for multiple consecutive frames. In some optional implementations of the embodiments of the present disclosure, a target detection algorithm based on a YOLO (You Only Look Once) deep neural network can be used to detect the position of the athlete. Specifically, the midpoint of the lower boundary of the bounding box in the detection result can be used as the position of the athlete.

[0027] In some optional implementations of the embodiments of the present disclosure, the above-mentioned tennis-related sports may include badminton, tennis, or table tennis, etc., which can be either singles or doubles sports.

[0028] Step 202: Detect any live video segment frame by frame to obtain the detection position of the target sphere in each live video frame in the live video segment.

[0029] In this embodiment, a real-time frame-by-frame detection can be performed on any live video segment obtained by segmenting the live video stream of tennis sports according to the score round to obtain the detection result within a score round. Specifically, the detection result may include but is not limited to the position of the detected target ball in each live video frame in any live video segment, that is, the above-mentioned detection position. In some optional implementation methods of the disclosed embodiment, a pre-trained ball detection model can be used to identify each live video frame to obtain the position of the target ball in each live video frame. Among them, a ball position sample including, for example, an image and position label of a ball can be constructed based on historical tennis sports videos, and then a preset deep learning network (such as a deep neural network (DNN)) can be trained based on this to obtain the ball detection model.

[0030] It should be noted that the above-mentioned operation of performing frame-by-frame detection on the live video segment corresponding to any score round can be understood as the operation of performing frame-by-frame detection on any live video segment obtained by segmenting according to the score round. Therefore, when the live video stream of the above-mentioned tennis sports is segmented according to the score round to obtain multiple live video segments, the operation of performing frame-by-frame detection on each live video segment can be performed.

[0031] Step 203: Based on the fitting results of the trajectory fitting of each detection position of the target sphere in time sequence, the detection position of the target sphere in each live video frame is verified to obtain the verified position of the target sphere in each live video frame.

[0032] In this embodiment, considering that the ball in the scoring round usually flies at a fast speed and is a small target, and the shape of the ball appearing in the live video frame image is prone to irregularity, it is difficult to achieve high accuracy in the false detection rate and / or missed detection rate of the detection results of the intelligent tracking of the ball. Therefore, in order to improve the accuracy of the position detection of the ball, the detection position of the target ball detected frame by frame in each live video frame of the live video clip can be fitted in time sequence, such as performing quadratic curve fitting, to obtain the corresponding fitting results. Then, based on the fitting results, the detection position of the target ball in each live video frame can be verified to obtain the corresponding verification position.

[0033] Step 204: Based on the verified position of the target ball in each live video frame, at least one of a hitting action analysis result and a hitting line analysis result for the target ball in the live video segment is obtained.

[0034] In this embodiment, after completing the verification of the position of the target ball in each live video frame and obtaining its final detection position, i.e., the verification position, in each live video frame, a technical and tactical analysis of the corresponding score round can be achieved based on the verified position of the target ball, which may specifically include but is not limited to obtaining the analysis results of the corresponding hitting action and / or the hitting line analysis results for the target ball.

[0035] In some optional implementations of the disclosed embodiments, in response to different tennis sports, the corresponding hitting actions and / or hitting routes may be different. For example, if the tennis sports are badminton, the hitting actions may include but are not limited to rubbing, pushing, diving, flat blocking, receiving, high-flying, smashing, etc., and the hitting routes may include but are not limited to straight lines, diagonal lines, and middle lines in the front and back courts; if the tennis sports are tennis, the hitting actions may include but are not limited to flat shots, topspin, backspin, flat serves, slice serves, net volleys, high-pressure shots, etc., and the hitting routes may include but are not limited to deep diagonal lines, shallow straight lines, volley straight lines, high-pressure diagonal lines, etc.; if the tennis sports are table tennis, the hitting actions may include but are not limited to short swings, long splits, lobs, loops, slices, smashes, and side cuts, and the hitting routes may include but are not limited to forehand diagonal lines, backhand straight lines, short ball routes near the net, and long ball routes from the baseline, etc.

[0036] The video analysis method for tennis sports provided by the embodiment of the present disclosure can be used as a basis for analysis based on the live video stream of tennis sports generated in real time. Specifically, the live video stream can be segmented according to the score round to obtain live video segments corresponding to each score round, and then the live video segments can be used as the granularity or unit for frame-by-frame detection to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using a trajectory fitting method. In this way, at least the accuracy and efficiency of the position detection of the target ball in each live video frame can be improved. Then, based on the verified position of the target ball in each live video frame, when analyzing and identifying the hitting action and / or hitting line of the target ball, not only can the efficiency of real-time analysis of the live video be improved, thereby reducing the delay in obtaining the corresponding analysis results, but also the accuracy of the analysis of the hitting action and / or hitting line used by the athlete in the tennis sports can be improved.

[0037] Please refer to Figure 3 , Figure 3 This is a flow chart of another method for analyzing tennis sports videos provided by an embodiment of the present disclosure, wherein process 300 includes the following steps: Step 301: Segment a live video stream of a tennis sport according to scores and rounds to obtain at least one live video segment.

[0038] Step 302: Detect any live video segment frame by frame to obtain the detection position of the target sphere in each live video frame in the live video segment.

[0039] Step 303: Based on the fitting results of the trajectory fitting of each detection position of the target sphere in time sequence, the detection position of the target sphere in each live video frame is verified to obtain the verified position of the target sphere in each live video frame.

[0040] Step 304: Based on the verified position of the target ball in each live video frame, at least one of a hitting action analysis result and a hitting line analysis result for the target ball in the live video segment is obtained.

[0041] The above steps 301-304 are similar to the following Figure 2 Steps 201-204 shown are consistent. For the same content, please refer to the corresponding part of the previous embodiment and will not be repeated here.

[0042] Step 305: Input the strategy question and answer information corresponding to tennis sports, as well as at least one of the batting action analysis results and the batting line analysis results, into a tactical strategy analysis model that has been pre-trained based on a large language model and a knowledge base corresponding to tennis sports, and output tactical strategies for tennis sports.

[0043] In this embodiment, after analyzing a live video stream of a tennis sport generated in real time to obtain accurate analysis results of the hitting action and / or hitting path for the target ball, these analysis results can be input into a large tactical strategy analysis model pre-trained based on a large language model and a knowledge base corresponding to the tennis sport. The large model then outputs tactical strategies for the tennis sport in real time in the form of a dialogue, which is used to provide feedback to athletes and / or coaches. The solutions of the disclosed embodiments not only enable efficient and accurate analysis of live video streams of tennis sports, but also provide optimized tactical strategies or suggestions in real time.

[0044] Furthermore, in some optional implementations of the disclosed embodiments, the aforementioned tactical strategies for tennis sports include textual strategy information and / or video strategy information. The video strategy information may include example videos of standard hitting motions and / or standard hitting paths. In this way, the tactical strategies can be presented in a variety of ways to meet the needs of different users. Furthermore, the aforementioned tactical strategies for tennis sports can be visualized to enhance the user experience.

[0045] Furthermore, in some optional implementations of the disclosed embodiments, the aforementioned knowledge base corresponding to tennis sports can be constructed based on standard sports literature and / or standard sports video data corresponding to these ball sports. Specifically, this standard sports literature and / or standard sports video data can be data annotated by professionals. Furthermore, a large language model can be used as a foundation in conjunction with the aforementioned knowledge base to train a large model for tactical and strategic analysis corresponding to tennis sports. Furthermore, by employing Low-Rank Adaptation (LoRA) technology, newly added professional knowledge, professional data, and coach feedback can be rapidly learned and iterated, creating a "data flywheel" effect through continuous incremental learning.

[0046] Furthermore, in some optional implementations of the disclosed embodiments, in the process of training a large model for tactical strategy analysis based on a knowledge base corresponding to tennis-related sports, the standard motion video data in the knowledge base can be analyzed based on a deep learning network. For example, the motion of key points of the athlete's posture can be analyzed for continuous multi-frame video data to obtain at least one of the detection results of the standard joint angle change, the standard joint speed detection result, the standard angular velocity change, and the standard muscle force or force change corresponding to the hitting action against the target ball during the movement. Specifically, each detection result can be represented in the form of a change curve. During the specific analysis, the above-mentioned standard motion video data can be used as a reference video stream for the tennis-related sports, and the following contents can be included: (1) Based on the reference video stream, basic input data for inputting into a preset deep learning network is obtained; wherein the basic input data includes a plurality of consecutive reference video frames (e.g., three consecutive frames) in the reference video stream, key points of the athlete's posture in each reference video frame, and coordinate information (e.g., three-dimensional coordinate information) of the key points of the athlete's posture in each reference video frame; wherein the preset deep learning network may include, but is not limited to, a residual network (Residual Network, ResNet), a point cloud deep learning network (PointNet), and a multilayer perceptron (Multilayer Perceptron, MLP).

[0047] (2) Input the basic input data into the above-mentioned ResNet, PointNet and MLP, extract the feature vectors corresponding to multiple different moments respectively, and concatenate the feature vectors of different types corresponding to the same moment to obtain the comprehensive feature vector corresponding to each moment; among them, ResNet can be used to extract image features, PointNet can be used to extract three-dimensional spatial coordinate features, and MLP can be used to extract three-dimensional spatial angle features.

[0048] (3) The comprehensive feature vectors corresponding to multiple consecutive moments are input into a long short-term memory (LSTM) network to extract the timing information, and then output at least one of the detection results of the above-mentioned standard joint angle change, standard joint speed detection result, standard angular velocity change detection result, and standard muscle force or force change detection result.

[0049] In some optional implementations of the embodiments of the present disclosure, in the above Figure 2 or Figure 3 On the basis of the corresponding embodiment, before the above step 203 or step 303, it can also include performing frame-by-frame detection on any live video segment determined from the live video stream according to the score round, and eliminating abnormal detection results from the obtained detection results, thereby achieving a preprocessing of the frame-by-frame detection results. In this way, on this basis, the position test of the fitting results based on trajectory fitting is performed. On the one hand, it can improve the efficiency of the position test of the target ball in each live video frame, and on the other hand, it helps to improve the accuracy of the analysis of the hitting action and / or hitting line adopted for the target ball. Specifically, the above preprocessing process of eliminating abnormal detection results can include the following contents: In response to determining the second distance between the detection position of the target sphere in each live video frame and the detection position in the previous live video frame of each live video frame, based on the size relationship between the second distance and the second distance threshold, the detection position of the target sphere in each live video frame is filtered to obtain the filtered detection position. In other words, the distance calculation can be performed on the detection positions corresponding to each of two adjacent live video frames in the live video segment of the target sphere, and based on the size relationship between the obtained distance, i.e., the second distance, and the pre-set distance threshold, i.e., the second distance threshold, the detection position of the target sphere in the live video frame in the live video segment is filtered. Furthermore, the above-mentioned step 203 or step 303 can be specifically implemented as follows: based on the fitting results of the trajectory fitting of each filtered detection position of the target sphere in chronological order, each filtered detection position of the target sphere is verified to obtain the verification position of the target sphere in each live video. Among them, the value of the second distance threshold can be set according to the type of tennis sports, that is, the value of the second distance threshold can be different depending on the type of tennis sports, and can be set according to specific needs, and is not specifically limited here.

[0050] Furthermore, in some optional implementations of the embodiments of the present disclosure, the above-mentioned step of filtering the detection position of the target sphere in each live video frame based on the size relationship between the second distance and the second distance threshold to obtain the filtered detection position can be specifically performed as follows: in response to the second distance being greater than the second distance threshold, performing an abnormal detection result removal operation on the detection position of the target sphere in each live video frame; in response to the abnormal detection result removal operation being performed on all detection positions of the target sphere in the live video segment, the remaining detection positions are used as the filtered detection positions.

[0051] In this embodiment, when it is determined that the distance between the detection positions corresponding to the target sphere in two adjacent live video frames exceeds a pre-set distance threshold, namely a second distance threshold, it can be considered an abnormal detection result and needs to be removed. After filtering all the detection positions of the target sphere obtained by frame-by-frame detection based on the live video clip, the remaining detection positions that have not been removed are used as the filtered detection positions, and trajectory fitting can be further performed in chronological order based on them. In this way, the screening and removal of abnormal detection results by setting a distance threshold is achieved efficiently and reliably.

[0052] In some optional implementations of any embodiment of the present disclosure, the above-mentioned step 203 or step 303 can be specifically performed as follows: based on the fitting result, determine the fitting position of the target sphere in each live video frame; in response to the first distance between the detection position and the fitting position of the target sphere in each live video frame being greater than a first distance threshold, use the fitting position as the verification position of the target sphere in each live video frame; in response to the first distance being less than or equal to the first distance threshold, use the detection position as the verification position of the target sphere in each live video frame.

[0053] In this embodiment, the position of the target sphere in each live video frame of the live video segment can be determined as its fitting position based on the fitting result obtained by performing trajectory fitting. Specifically, the detection result, i.e., the detected position of the target sphere in each live video frame, can be secondary processed based on the relationship between the distance between the target sphere's corresponding detected position in each live video frame and its corresponding fitting position and a pre-set distance threshold, i.e., a first distance threshold. In this way, while achieving efficient and reliable position verification, the accuracy of the position detection of the target sphere in each live video frame can be further improved. Specifically, if the distance between the detected position and the fitting position of the target sphere corresponding to each live video frame, i.e., the first distance, is greater than the first distance threshold, then the detection result indicates an anomaly. To ensure the accuracy of the above analysis results, the more accurate fitting position can be used as the final position of the target sphere in the corresponding live video frame; otherwise, the detection result indicates that the completion and replacement are relatively accurate and no completion or replacement is required. The value of the first distance threshold can be set according to the type of tennis sport. That is, the value of the first distance threshold can vary depending on the type of tennis sport and can be set according to specific needs. It is not specifically limited here.

[0054] In some optional implementations of any embodiment of the present disclosure, the above-mentioned tennis sports video analysis method may also include the following contents: in response to detecting a target sphere in multiple preceding live video frames before each live video frame, performing trajectory fitting in time sequence based on the detection position of the target sphere in each preceding live video frame to obtain a fitting result; in response to not detecting the target sphere in multiple preceding live video frames, performing trajectory fitting based on the detection position of the target sphere in multiple subsequent live video frames after each live video frame, and performing trajectory fitting in time sequence to obtain a fitting result.

[0055] In this embodiment, for each real-time live video frame in the live video clip, the selection range of the detection results for trajectory fitting can be determined based on whether the target sphere is detected in multiple preceding live video frames that are chronologically preceding any or each current real-time live video frame. While ensuring the reliability of trajectory fitting, the efficiency of trajectory fitting can be maximized. Specifically, if the presence of the target sphere can be detected in multiple preceding live video frames before the current live video frame, the multiple preceding live video frames before the current live video frame can be cached in chronological order. If the presence of the target sphere cannot be detected in multiple live video frames before the current live video frame, the multiple subsequent live video frames after the current live video frame can be cached in chronological order, and then trajectory fitting can be performed based on the detection position of the target sphere in the cached multiple live video frames. In some optional implementations of the embodiments of the present disclosure, the above-mentioned multiple preceding live video frames or multiple subsequent live video frames can be multiple continuous live video frames.

[0056] In some optional implementations of the embodiments of the present disclosure, taking into account the processing capabilities of the execution entity and the user's experience of real-time analysis of the live video stream, the duration granularity of the cache of the above-mentioned live video frames can be single-second or half-second, and the number of the above-mentioned multiple preceding or subsequent live video frames cached can be set to ranging from a dozen frames to dozens of frames according to specific needs.

[0057] In a specific example, any of the above-mentioned positions can be represented by two-dimensional coordinates. The detected position of the target sphere (such as a badminton) in any live video frame in the above-mentioned live video clip can be represented as p(x,y), and the detected position pl(x,y) of the target sphere in the first L (an integer greater than or equal to 2) consecutive live video frames of the current live video frame is cached. First, the distance between the position p(x,y) corresponding to the current live video frame and the position pl(x,y) in the live video frame before and adjacent to it, that is, the previous live video frame, is detected (corresponding to the second distance in the above-mentioned embodiment). When the distance is greater than the distance threshold s(p1) (corresponding to the second distance threshold in the above-mentioned embodiment), it is considered that the detection result is abnormal and needs to be eliminated or removed. Furthermore, after completing the above-mentioned elimination and screening of abnormal detection results for all live video frames in the live video segment, a quadratic curve fitting can be performed for each live video frame using the detection position data from m (an integer greater than or equal to 2) frames preceding the current live video frame. Based on the fitting curve, a current fitting position p(x,y)' is determined. If the distance between the current detection position p(x,y) and the current fitting position p(x,y)' (corresponding to the first distance in the above embodiment) is greater than s(p2) (corresponding to the first distance threshold in the above embodiment), the detection result is considered abnormal and should be eliminated. The fitting position p(x,y)' determined based on the quadratic curve fitting replaces the detection position p(x,y) of the current live video frame as the sphere position for the current frame (corresponding to the verification position in the above embodiment). Furthermore, if the target sphere in the m frames preceding each current live video frame is not visible, the detection position data from the m frames following each current live video frame is cached, and the same quadratic curve fitting results are used to verify and complete the detection result of the current live video frame.

[0058] In some optional implementations of any embodiment of the present disclosure, the above-mentioned step 204 or step 304 can be specifically performed as follows: determining the hitting frame based on the result of curve fitting of the verification position of the target ball in each live video frame; obtaining at least one of the hitting action analysis result and the hitting line analysis result based on the hitting frame, the verification position of the target ball in each live video frame and the posture key point detection result of the target athlete in the live video clip in each live video frame.

[0059] In this embodiment, curve fitting can be performed based on the verified position of the target ball in each live video frame (for example, quadratic curve fitting can be performed based on the vertical coordinate in the coordinates used to represent the position and the time), and each hitting frame in the live video segment can be determined based on the obtained fitting curve, wherein the type of hitting frame in the live video segment can be different according to the type of tennis sport. For example, for badminton, the hitting frame can be a live video frame containing the moment when the players of each party swing the racket to hit the ball, while for tennis or table tennis, in addition to the live video frame containing the moment when the players of each party swing the racket to hit the ball, the hitting frame can also be a live video frame containing the moment when the ball touches the ground. At the same time, the detection results obtained by performing frame-by-frame detection on each live video clip may include, in addition to the position of the target ball in each live video frame, the detection results of the key points of the posture of the target athlete in each live video frame. Furthermore, based on the determined hitting frames, the verified position of the target ball in each live video frame, and the key points of the posture of the target athlete in each live video frame, an accurate analysis of the hitting action and / or the hitting route formed by the target athlete against the target ball in each scoring round can be achieved.

[0060] Furthermore, in some optional implementations of the embodiments of the present disclosure, in the step of obtaining at least one of a batting action analysis result and a batting route analysis result based on the batting frame, the verification position of the target ball in each live video frame, and the posture key point detection result of the target player in each live video frame, a plurality of live video frames, such as N (an integer greater than or equal to 2), can be determined forward and backward in chronological order based on each batting frame. At this time, (2 N+1) live video frames, then we can further The verification position of each live video frame in N+1) live video frames and the target athlete in the (2 The posture key point detection results in each of the N+1) live video frames can realize accurate analysis of the hitting action and / or hitting line implemented by the target athlete against the target ball in each scoring round.

[0061] In some optional implementations of the embodiments of the present disclosure, the above-mentioned posture key point detection results include but are not limited to at least one of the detection results of the joint angle changes during the athlete's movement, the joint speed detection results, the angular velocity change detection results and the muscle force or force change detection results.

[0062] In some optional implementations of the embodiments of the present disclosure, on the basis of any of the above-mentioned embodiments, the above-mentioned net-type tennis sports video analysis method may further include the following contents: based on the target reference object in the live video frame of the sports field corresponding to the net-type sports, pre-position marking is performed to obtain a reference position, and based on the position detection of the target reference object on the sports field in each live video frame of the live video clip, a detection position corresponding to the target reference object is obtained; based on the size relationship between the distance between the reference position and the detection position corresponding to the target reference object in each live video frame and a third distance threshold, each live video frame is filtered to obtain a valid live video frame in the live video clip; and any live video clip is detected frame by frame to obtain the detection position of the target sphere in each live video frame in the live video clip, including: performing frame by frame detection on the valid live video frames in any live video clip to obtain the detection position of the target sphere in each valid live video frame.

[0063] In this embodiment, the position of a target reference object on the playing field in a video frame containing a tennis-related sport (e.g., a video frame in a video stream transmitted before the sport begins but after the camera begins operating) can be pre-annotated as a reference position. Based on the target reference object on the playing field in each live video frame of the live video segment, intelligent detection can be performed, for example, using a pre-trained detection model based on a deep learning network to obtain a detection position corresponding to the target reference object. Furthermore, for each live video frame, the live video frames contained in the live video segment can be filtered based on the relationship between the reference position and the real-time detected detection position corresponding to the target reference object and a pre-set distance threshold, i.e., a third distance threshold, to obtain valid live video frames for frame-by-frame detection. By filtering out invalid live video frames in the live video segment, the quality of the live video frames used for intelligent detection can be improved, thereby further improving the accuracy of the analysis of the target player's hitting action and / or hitting path formed against the target ball during each scoring round. Among them, the value of the third distance threshold can be set according to the type of tennis sports, that is, the value of the third distance threshold can be different depending on the type of tennis sports, and can be set according to specific needs, and is not specifically limited here.

[0064] In some optional implementations of the embodiments of the present disclosure, the target reference objects include but are not limited to at least two of the end lines, side lines, service lines, center lines, and net posts in a sports field. Exemplarily, the reference positions may include first-class labeled positions obtained by labeling the positions of all intersections of the end lines and side lines of the sports field in a live video frame, or second-class labeled positions obtained by labeling the positions where the net posts intersect with the sports field. Accordingly, the detection positions corresponding to the target reference objects obtained by intelligent detection may include first-class detection positions obtained by intelligent detection of the positions of all intersections of the end lines and side lines of the sports field in a live video frame, or second-class detection positions obtained by intelligent detection of the positions where the net posts intersect with the sports field.

[0065] Furthermore, in some optional implementations of the embodiments of the present disclosure, the above-mentioned step of filtering each live video frame based on the size relationship between the distance between the reference position in each live video frame and the detection position corresponding to the target reference object and the third distance threshold to obtain valid live video frames in the live video clip can be specifically implemented as follows: in response to the distance between the reference position in each live video frame and the detection position corresponding to the target reference object of the same category as the reference position is greater than the preset third distance threshold, the live video frame can be considered invalid and can be removed, and the valid live video frame is retained for subsequent processing and analysis.

[0066] In one specific example, any of the aforementioned locations can be represented by two-dimensional coordinates. The corner locations where the end lines and sidelines of the playing field intersect, as well as the locations where the net posts intersect the playing field, are pre-annotated in the live video frame image as reference locations court(x,y) and net(x,y), respectively. A deep learning network, such as a Region-based Convolutional Neural Network (RCNN), detects the playing field in the live video frame image. If the difference between the intersection location court_n(x,y) in the field detection result and the reference location court(x,y) is greater than a threshold court_th(x,y) (corresponding to the third distance threshold), or if the difference between the net_n(x,y) in the net position detection result and the reference location net(x,y) is greater than a threshold net_th(x,y) (corresponding to the third distance threshold), the live video frame data is deemed invalid and the invalid frame data is removed, retaining the valid data for subsequent processing and analysis.

[0067] Furthermore, in some optional implementations of the embodiments of the present disclosure, the following contents may also be included: marking the intersections between all the constituent lines of the sports field in each live video frame, and obtaining a mapping transformation matrix between the sports field in each live video frame and the standard sports field corresponding to the tennis sports through homography transformation. Furthermore, based on the mapping transformation matrix, the position of the target ball in the above-mentioned hitting frame can be mapped to the standard sports field to obtain the position of the target player's hitting point. Furthermore, based on the position of the hitting point, the target player in the above-mentioned hitting frame (2 The posture key point detection results in each live video frame of N+1) live video frames and the verification position of the target ball in each live video frame can achieve a more accurate analysis of the hitting action and / or hitting line formed by the target athlete against the target ball in each scoring round, thereby realizing technical and tactical analysis.

[0068] Furthermore, in some optional implementations of any embodiment of the present disclosure, the detected position of the athlete can also be mapped to a standard sports field based on the mapping transformation matrix, so as to facilitate accurate analysis of whether the athlete is in the serving area or the receiving area, as well as the hitting action, hitting line, etc.

[0069] Furthermore, in some optional implementations of any embodiment of the present disclosure, in order to facilitate querying and managing the above-mentioned analysis results of the hitting action and / or the hitting route for the target ball, the hitting action analysis results and / or the hitting route analysis results can be counted by live broadcast time according to single score rounds, multiple score rounds, or single match rounds, where each match round can include multiple score rounds, and displayed in a visual form, such as by drawing a chart, which can then be sent to a display device for display via a visual feedback node. The above-mentioned statistical results can be stored in a database table.

[0070] Furthermore, in some optional implementations of any embodiment of the present disclosure, a live video stream analyzed using the above-mentioned video analysis solution is saved, and the video time corresponding to the above-mentioned hitting action analysis results and / or hitting line analysis results is indexed, and the indexing results are visualized to facilitate coaches, athletes, etc. to view the corresponding video segments.

[0071] It should be noted that the tennis sports video analysis method described in any of the above embodiments is not only applicable to the application scenario of analyzing the above-mentioned live video stream, but can also be applied to pre-recorded game broadcast video streams and / or training video streams.

[0072] To deepen understanding, this disclosure also provides a specific implementation solution in combination with a specific application scenario, see Figure 4 The system 400 for implementing the tennis video analysis method shown in FIG. 4 may include a video capture device 401, a tactical analysis module 402, a tactical strategy analysis module 403, and a visualization display device 404. The video capture device 401 may be deployed in Figure 1 On the host client 101 side shown, the tactical analysis module 402 and the tactical strategy analysis module 403 can be deployed Figure 1 On the live broadcast server 102 side shown, the visual display device 404 can be deployed on Figure 1 The user client 103 side is shown.

[0073] The video capture device 401 can be used to capture a video stream of a net-type sport, such as a live video stream. Specifically, the video capture device 401 can include a high-definition camera. The high-definition camera can be placed directly behind the playing field, shooting from above and below, with the camera's direction as perpendicular as possible to the end line of the playing field. The video viewing angle is fixed, while ensuring clarity, illustratively, at no less than 30 frames per second (FPS). The tactical analysis module 402 can be used to perform specific analysis based on the received live video stream, obtaining at least one of an analysis of the ball's hitting motion and a result of the ball's hitting trajectory. The tactical strategy analysis module 403 can be used to output, based on the analysis results output by the tactical analysis module 402, a tactical strategy for the net-type sport in real time, in the form of a dialogue, using a large model, for feedback to athletes and / or coaches. In this embodiment, the analysis scheme implemented by the tactical analysis module 402 and the tactical strategy analysis module 403 can be implemented by referring to the corresponding content of the net-type sport video analysis method described in any of the above embodiments, and will not be further described here. Visual display device 404 can be used to display the above analysis results and tactical strategies. Furthermore, in this embodiment, the Robot Operating System (ROS2) can be used to provide a node management and communication framework, and ROS nodes can be created for corresponding functional modules in the system (such as the above tactical analysis module 402 and tactical strategy analysis module 403).

[0074] Through the solution provided by the embodiments of the present disclosure, a set of automatic analysis solutions for sports video techniques and tactics based on athlete detection, posture and movement recognition, ball detection algorithms, and technical analysis large models is realized, which can improve the accuracy and efficiency of technical and tactical data analysis and recording. Specifically, by using advanced visual and artificial intelligence technologies, it is possible to accurately capture and analyze effective rounds in sports videos, quickly and accurately analyze the technical and tactical application performance of athletes on both sides, and ensure that every technical link can be accurately analyzed. Furthermore, based on professional technical data and a large model base training, a large model for analyzing tennis sports techniques can be used, and based on the video analysis results, targeted optimization strategies and suggestions can be given for teenagers' ball technical movements and actual combat applications.

[0075] Furthermore, automated performance recording and real-time analysis reduce reliance on manual operations and specialized knowledge, thereby improving the consistency, objectivity, and accuracy of data collection. Furthermore, the efficiency of technical and tactical analysis can be significantly improved. By rapidly processing, analyzing, and compiling statistical data, and leveraging large models, coaches and athletes can be provided with effective, intelligent, and real-time feedback in a question-and-answer format, thereby optimizing training and competition strategies while also reducing the risk of injury. This is not only crucial for optimizing athlete training and competition strategies, but also provides coaches with more detailed and reliable data support, enabling targeted training and competition preparation, as well as improving on-the-spot tactics.

[0076] The tennis video analysis solution implemented in any of the above-mentioned embodiments can be applied not only in professional competitive sports, helping coaches and athletes analyze and improve their performance in competition and training, but also in schools and mass sports, providing technical support for amateur competition and training analysis. Furthermore, it can be integrated into sports technology products, such as intelligent sports performance analysis software and smart fitness equipment, to provide users with scientific and accurate sports analysis services and enhance the user experience.

[0077] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a tennis sports video analysis device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices, such as servers.

[0078] like Figure 5As shown, the apparatus 500 for analyzing tennis sports video according to the embodiment of the present disclosure may include: a segmentation module 501, a detection module 502, a verification module 503, and a processing module 504. The segmentation module 501 is configured to segment the live video stream of the tennis sports according to the score and rounds to obtain at least one live video segment; the detection module 502 is configured to perform frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame in the live video segment; the verification module 503 is configured to verify the detection position of the target ball in each live video frame based on the fitting results of the trajectory fitting of each detection position of the target ball in chronological order to obtain the verified position of the target ball in each live video frame; and the processing module 504 is configured to obtain at least one of a hitting action analysis result and a hitting route analysis result for the target ball in the live video segment based on the verified position of the target ball in each live video frame.

[0079] In the embodiment of the present disclosure, the specific processing of the segmentation module 501, the detection module 502, the verification module 503 and the processing module 504 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiment are not repeated here.

[0080] In some optional implementations of the embodiments of the present disclosure, the above-mentioned verification module 503 is further configured to: determine the fitting position of the target sphere in each live video frame based on the fitting result; in response to the first distance between the detection position and the fitting position of the target sphere in each live video frame being greater than the first distance threshold, use the fitting position as the verification position of the target sphere in each live video frame; in response to the first distance being less than or equal to the first distance threshold, use the detection position as the verification position of the target sphere in each live video frame.

[0081] In some optional implementations of the embodiments of the present disclosure, the above-mentioned tennis sports video analysis device 500 may further include a fitting module (not shown in the figure), which is configured to: in response to the detection of a target sphere in multiple preceding live video frames before each live video frame, perform trajectory fitting in time sequence based on the detection position of the target sphere in each preceding live video frame to obtain a fitting result; in response to the failure to detect the target sphere in multiple preceding live video frames, perform trajectory fitting based on the detection position of the target sphere in multiple subsequent live video frames after each live video frame, and perform trajectory fitting in time sequence to obtain a fitting result.

[0082] In some optional implementations of the embodiments of the present disclosure, the above-mentioned tennis sports video analysis device 500 may also include a first screening module (not shown in the figure), which is configured to: in response to determining the second distance between the detection position of the target sphere in each live video frame and the detection position in the previous live video frame of each live video frame, filter the detection position of the target sphere in each live video frame based on the size relationship between the second distance and the second distance threshold to obtain the filtered detection position; and the above-mentioned verification module 503 is further configured to: based on the fitting results of the trajectory fitting of each filtered detection position of the target sphere in chronological order, verify each filtered detection position of the target sphere to obtain the verification position of the target sphere in each live video.

[0083] In some optional implementations of the embodiments of the present disclosure, the above-mentioned first screening module is further configured to: in response to the second distance being greater than the second distance threshold, perform an abnormal detection result removal operation on the detection position of the target sphere in each live video frame; in response to completing the abnormal detection result removal operation on all detection positions of the target sphere in the live video segment, use the remaining detection positions as the screened detection positions.

[0084] In some optional implementations of the embodiments of the present disclosure, the above-mentioned processing module 504 is further configured to: determine the hitting frame based on the results of curve fitting of the verification position of the target ball in each live video frame; obtain at least one of the hitting action analysis results and the hitting line analysis results based on the hitting frame, the verification position of the target ball in each live video frame and the posture key point detection results of the target athlete in the live video clip in each live video frame.

[0085] In some optional implementations of the embodiments of the present disclosure, the above-mentioned tennis sports video analysis device 500 may further include a determination module (not shown in the figure), which is configured to: input strategy question and answer information corresponding to tennis sports, and at least one of the batting action analysis results and the batting line analysis results into a tactical strategy analysis big model that is pre-trained based on a big language model and a knowledge base corresponding to tennis sports, and output tactical strategies for tennis sports.

[0086] In some optional implementations of the embodiments of the present disclosure, the tactical strategy for tennis sports includes strategy text information and / or strategy video information.

[0087] In some optional implementations of the embodiments of the present disclosure, the above-mentioned net-type tennis sports video analysis device 500 also includes a second screening module (not shown in the figure), which is configured to: pre-mark the position of the target reference object in the live video frame of the sports field corresponding to the net-type sports to obtain a reference position, and perform position detection on the target reference object on the sports field in each live video frame of the live video clip to obtain a detection position corresponding to the target reference object; based on the relationship between the distance between the reference position and the detection position corresponding to the target reference object in each live video frame and a third distance threshold, filter each live video frame to obtain a valid live video frame in the live video clip; and the above-mentioned detection module 502 is further configured to: perform frame-by-frame detection on the valid live video frames in any live video clip to obtain the detection position of the target ball in each valid live video frame.

[0088] In some optional implementations of the embodiments of the present disclosure, the target reference objects include at least two of the end lines, side lines, service lines, center lines, and net posts in the sports field.

[0089] This embodiment, as an apparatus embodiment corresponding to the above-described method embodiment, can be used to analyze a live video stream of a tennis game generated in real time. Specifically, the live video stream can be segmented by score round to obtain live video segments corresponding to each score round. Frame-by-frame detection can then be performed using the live video segments as the granularity or unit to obtain the detected position of the target ball in each live video. Furthermore, the detected position of the target ball in each live video frame can be verified, specifically by using a trajectory fitting method. This can at least improve the accuracy and efficiency of detecting the position of the target ball in each live video frame. Furthermore, when analyzing and identifying the hitting action and / or hitting path of the target ball based on the verified position of the target ball in each live video frame, not only can the efficiency of real-time analysis of the live video be improved, thereby reducing the delay in obtaining the corresponding analysis results, but it can also improve the accuracy of the analysis of the hitting action and / or hitting path used by the athlete in the tennis game.

[0090] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the tennis sports video analysis method described in any of the above embodiments when executing.

[0091] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the tennis video analysis method described in any of the above embodiments when executed.

[0092] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product including a computer program, which, when executed by a processor, can implement the method for analyzing tennis sports videos described in any of the above embodiments.

[0093] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] like Figure 6 As shown, electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of electronic device 600. Computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0095] Multiple components in the electronic device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0096] The computing unit 601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the tennis video analysis method. For example, in some embodiments, the tennis video analysis method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the tennis video analysis method described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the tennis sports video analysis method in any other appropriate manner (eg, by means of firmware).

[0097] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0101] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0102] A computer system may include clients and servers. These clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server can be a cloud server, a server in a distributed system, or a server integrated with a blockchain. A cloud server, also known as a cloud computing server or cloud host, is a host product within the cloud computing service ecosystem. It addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.

[0103] According to the technical solution for realizing video analysis of tennis-type sports in the embodiment of the present disclosure, the live video stream of the tennis-type sports generated in real time can be used as the basis for analysis. Specifically, the live video stream can be segmented according to the score round to obtain live video segments corresponding to each score round, and then the live video segments can be used as the granularity or unit for frame-by-frame detection to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using a trajectory fitting method. In this way, at least the accuracy and efficiency of the position detection of the target ball in each live video frame can be improved. Then, based on the verified position of the target ball in each live video frame, when analyzing and identifying the hitting action and / or hitting line of the target ball, not only can the efficiency of real-time analysis of the live video be improved, thereby reducing the delay in obtaining the corresponding analysis results, but also the accuracy of the analysis of the hitting action and / or hitting line etc. adopted by the athlete in the tennis-type sports can be improved.

[0104] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0105] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for analyzing tennis sports videos, comprising: Segmenting a live video stream of a tennis game according to scores and rounds to obtain at least one live video segment; Performing frame-by-frame detection on any of the live video segments to obtain a detection position of the target sphere in each live video frame in the live video segment; Based on the fitting results of the trajectory fitting of each detected position of the target sphere in chronological order, the detected position of the target sphere in each live video frame is verified to obtain the verified position of the target sphere in each live video frame; Based on the verified position of the target ball in each live video frame, at least one of a hitting action analysis result and a hitting line analysis result for the target ball in the live video segment is obtained.

2. The method according to claim 1, wherein The method of verifying the detected position of the target sphere in each live video frame based on the fitting results of the trajectory fitting of each detected position of the target sphere in chronological order to obtain the verified position of the target sphere in each live video frame includes: Based on the fitting results, determining the fitting position of the target sphere in each live video frame; In response to a first distance between a detected position and a fitted position of the target sphere in each live video frame being greater than a first distance threshold, using the fitted position as a verification position of the target sphere in each live video frame; In response to the first distance being less than or equal to the first distance threshold, the detected position is used as a verification position of the target sphere in each live video frame.

3. The method according to claim 1, further comprising: In response to detecting the target sphere in a plurality of preceding live video frames before each live video frame, performing trajectory fitting in chronological order based on the detected positions of the target sphere in each of the preceding live video frames to obtain the fitting result; In response to the target sphere not being detected in the multiple preceding live video frames, trajectory fitting is performed based on the detection position of the target sphere in multiple subsequent live video frames after each live video frame, and the trajectory fitting is performed in chronological order to obtain the fitting result.

4. The method according to claim 1, further comprising: In response to determining a second distance between a detection position of the target sphere in each live video frame and a detection position in a previous live video frame of each live video frame, filtering the detection position of the target sphere in each live video frame based on a magnitude relationship between the second distance and a second distance threshold to obtain a filtered detection position; and The method of verifying the detected position of the target sphere in each live video frame based on the fitting results of the trajectory fitting of each detected position of the target sphere in chronological order to obtain the verified position of the target sphere in each live video frame includes: Based on the fitting results of trajectory fitting of each filtered detection position of the target sphere in chronological order, each filtered detection position of the target sphere is verified to obtain the inspection position of the target sphere in each live video.

5. The method according to claim 4, wherein The filtering process of the detection position of the target sphere in each video frame to obtain the filtered detection position includes: In response to the second distance being greater than the second distance threshold, performing an abnormal detection result removal operation on the detection position of the target sphere in each live video frame; In response to performing the abnormal detection result removal operation on all detection positions of the target sphere in the live video segment, the remaining detection positions are used as filtered detection positions.

6. The method according to claim 1, wherein The obtaining, based on the verified position of the target ball in each live video frame, at least one of a hitting action analysis result and a hitting path analysis result for the target ball in the live video segment, comprises: Determining a hitting frame based on a result of curve fitting of a check position of the target ball in each live video frame; Based on the hitting frame, the verification position of the target ball in each live video frame and the posture key point detection results of the target athlete in the live video clip in each live video frame, at least one of the hitting action analysis result and the hitting line analysis result is obtained.

7. The method according to claim 1, further comprising: The strategy question and answer information corresponding to the tennis sports, and at least one of the batting action analysis results and the batting line analysis results are input into a tactical strategy analysis large model that has been pre-trained based on a large language model and a knowledge base corresponding to the tennis sports, and the tactical strategy for the tennis sports is output.

8. The method according to claim 7, wherein: The tactical strategy for the tennis sport includes strategy text information and / or strategy video information.

9. The method according to claim 1, further comprising: Pre-marking a target reference object in a live video frame of a sports field corresponding to the netted sports to obtain a reference position, and detecting a position of the target reference object on the sports field in each live video frame of the live video segment to obtain a detection position corresponding to the target reference object; Based on a magnitude relationship between a distance between the reference position and the detection position corresponding to the target reference object in each live video frame and a third distance threshold, filtering each live video frame to obtain a valid live video frame in the live video segment; as well as The performing frame-by-frame detection on any of the live video segments to obtain the detection position of the target sphere in each live video frame in the live video segment includes: performing frame-by-frame detection on the valid live video frames in any of the live video segments to obtain the detection position of the target sphere in each valid live video frame.

10. The method according to claim 9, wherein: The target reference objects include at least two of the end lines, side lines, service lines, center lines, and net posts in the sports field.

11. A tennis sports video analysis device comprising: a segmentation module configured to segment the live video stream of the tennis sport according to the score and round, thereby obtaining at least one live video segment; a detection module configured to perform frame-by-frame detection on any of the live video segments to obtain a detection position of the target sphere in each live video frame in the live video segment; a verification module configured to verify the detected position of the target sphere in each live video frame based on a fitting result of trajectory fitting performed on each detected position of the target sphere in chronological order, thereby obtaining a verified position of the target sphere in each live video frame; The processing module is configured to obtain at least one of a hitting action analysis result and a hitting line analysis result for the target ball in the live video segment based on the verification position of the target ball in each live video frame.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the tennis video analysis method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the tennis video analysis method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method for analyzing a video of tennis sports according to any one of claims 1 to 10.

Images