Screening method and device for netball video analysis

Abstract

The present disclosure provides a kind of screen net ball game video analysis method and device, it is related to artificial intelligence technical field, specifically related to deep learning and computer vision technical field.The method comprises: according to score round, the live video stream of screen net ball game is divided, at least one live video segment is obtained;Any live video segment is detected frame by frame, and the detection position of target ball in each live video frame in live video segment is obtained;Based on the fitting result of trajectory fitting of each detection position of target ball in time sequence, the detection position of target ball in each live video frame is verified, and the verification position of target ball in each live video frame is obtained;Based on the verification position of target ball in each live video frame, at least one of the analysis result of hitting action and the analysis result of hitting line for target ball in live video segment is obtained.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of deep learning and computer vision technology, and in particular to a method and apparatus for analyzing tennis-like sports videos. Background Technology

[0002] With the continuous improvement of tennis competition, coaches and athletes are paying more and more attention to analyzing athletes' technical movements and tactical applications in training or competition. Timely and reasonable correction and optimization of athletes' technical movements and tactical applications based on the analysis results can help improve and enrich athletes' competitive level. However, at present, the analysis of competition and training videos is mostly done manually, which is inefficient and prone to errors. Summary of the Invention

[0003] This disclosure provides a method, device, electronic device, computer-readable storage medium, and computer program product for analyzing video of tennis ball sports.

[0004] In a first aspect, embodiments of this disclosure propose a video analysis method for tennis-like sports, comprising: segmenting a live video stream of a tennis-like sports according to the score and rallies to obtain at least one live video segment; performing frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame of the live video segment; verifying the detection position of the target ball in each live video frame based on the fitting result of trajectory fitting of each detection position of the target ball in chronological order to obtain the verified position of the target ball in each live video frame; and obtaining at least one of the analysis results of the hitting action and the hitting trajectory analysis results for the target ball in the live video segment based on the verified position of the target ball in each live video frame.

[0005] Secondly, embodiments of this disclosure propose a video analysis device for tennis-like sports, comprising: a segmentation module, a detection module, a verification module, and a processing module. The segmentation module is configured to segment a live video stream of a tennis-like sport according to the score and number of rallies, obtaining at least one live video segment. The detection module is configured to perform frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame within the segment. The verification module is configured to verify the detection position of the target ball in each live video frame based on the fitting results of trajectory fitting of each detection position of the target ball in chronological order, obtaining the verified position of the target ball in each live video frame. The processing module is configured to, based on the verified position of the target ball in each live video frame, obtain at least one of the following: an analysis result of the hitting action of the target ball in the live video segment and an analysis result of the hitting trajectory.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the tennis ball sports video analysis method described in any of the above implementations.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the tennis ball sports video analysis method described in any of the above implementations when executed.

[0008] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the tennis ball sports video analysis method described in any of the above implementations.

[0009] According to the tennis ball sports video analysis scheme provided in this disclosure, analysis can be performed based on the real-time generated live video stream of tennis ball sports. Specifically, the live video stream can be segmented according to the score rounds to obtain live video segments corresponding to each score round. Then, frame-by-frame detection can be performed at the granularity or unit of the live video segments to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using trajectory fitting. This can at least improve the accuracy and efficiency of the target ball position detection in each live video frame. Therefore, when analyzing and identifying the hitting action and / or hitting trajectory of the target ball based on the verified position of the target ball in each live video frame, it can not only improve the efficiency of real-time analysis of the live video and reduce the latency of obtaining the corresponding analysis results, but also improve the accuracy of the analysis of the hitting action and / or hitting trajectory used by the athlete in the tennis ball sports.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture to which this disclosure can be applied; Figure 2A flowchart of a video analysis method for tennis ball sports provided in this embodiment of the present disclosure; Figure 3 A flowchart of another method for analyzing tennis ball sports videos provided in this embodiment of the present disclosure; Figure 4 A schematic diagram of a video analysis system for tennis ball sports in an application scenario provided by an embodiment of this disclosure; Figure 5 A structural block diagram of a tennis sports video analysis device provided in this embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device suitable for performing a video analysis method for tennis ball sports, provided as an embodiment of this disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0013] It should be noted that the collection, acquisition, storage, processing, transmission, provision, disclosure, and application of user personal information (such as sports video information) involved in the technical solution disclosed herein are all carried out with the user's knowledge and explicit authorization, comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0014] With the continuous improvement of competitive levels in tennis-related sports (such as badminton, tennis, and table tennis), the analysis of athletes' technical movements and tactical applications during training and competition is receiving increasing attention from coaches and athletes. This is especially important for young athletes whose technical movements and tactical applications are not yet fully developed. Timely and reasonable correction and optimization of athletes' technical movements and tactical applications based on analysis results can help improve and enrich their competitive level, training methods, and approaches. However, currently, the analysis of match and training videos is mostly done manually, which is inefficient and prone to errors. Therefore, further improvements are needed in the practice of using computer technology and software systems to efficiently analyze the techniques and tactics of tennis-related sports.

[0015] Figure 1 An exemplary system architecture 100 is shown, which can be applied to an embodiment of the tennis video analysis scheme disclosed herein.

[0016] like Figure 1 As shown, the system architecture 100 may include a broadcaster client 101, a live streaming server 102, and a user client 103.

[0017] Network 104 is a medium used to provide a transmission link between the broadcast client 101 and the live streaming server 102. Network 104 may include various wired and wireless transmission links. Network 105 is a medium used to provide a transmission link between the live streaming server 102 and the user client 103. Network 105 may include various wired and wireless transmission links.

[0018] Users of the broadcast client 101 (such as organizers or broadcasters of tennis matches) can use video capture devices, such as cameras and microphones, to capture live images and audio in real time, generating a live video stream. The broadcast client 101 can then send the recorded live video stream to the live streaming server 102. The live streaming server 102 can receive the live video stream from the broadcast client 101 and send it to the user client 103. Upon receiving the live video stream, the user client 103 can play it.

[0019] Various applications for enabling information communication between the broadcaster client 101, the live streaming server 102, and the user client 103 can be installed. These applications include video recording applications, video analysis applications, and instant messaging applications.

[0020] The broadcast client 101, live streaming server 102, and user client 103 can be hardware or software. When the broadcast client 101 is hardware, it can be various electronic devices with a display screen, audio receiver, etc., including but not limited to video recording equipment. When the broadcast client 101 is software, it can be installed on the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When the live streaming server 102 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the live streaming server 102 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When the user client 103 is hardware, it can be various electronic devices with a display screen, including but not limited to smartphones, tablets, laptops, and desktop computers. When the user client 103 is software, it can be installed on the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0021] The live streaming server 102 can provide various services through its built-in applications. Taking the application for video analysis of net-play sports as an example, when running this application, the live streaming server 102 can achieve the following effects: It can analyze the live video stream of net-play sports generated in real time. Specifically, it can segment the live video stream according to the score and rounds to obtain live video segments corresponding to each score and round. Then, it can perform frame-by-frame detection at the granularity or unit of the live video segments to obtain the detection position of the target ball in each live video. Furthermore, it can verify the detection position of the target ball in each live video frame, specifically by using trajectory fitting. This can at least improve the accuracy and efficiency of detecting the target ball's position in each live video frame. Therefore, based on the verified position of the target ball in each live video frame, when analyzing and identifying the hitting action and / or hitting trajectory of the target ball, it can not only improve the efficiency of real-time analysis of the live video, thereby reducing the latency of obtaining the corresponding analysis results, but also improve the accuracy of the analysis of the hitting action and / or hitting trajectory used by the athlete in the net-play sports.

[0022] Since video analysis based on live video streams requires significant computing resources and capabilities, the tennis ball video analysis methods provided in subsequent embodiments of this disclosure are generally executed by a live streaming server 102 with strong computing power and abundant resources. Correspondingly, the tennis ball video analysis device is also typically located within the live streaming server 102. However, it should also be noted that when the broadcast client 101 also possesses sufficient computing power and resources, it can also perform the aforementioned calculations performed by the live streaming server 102 through its installed live video analysis application, thereby outputting the same results as the live streaming server 102. Especially when multiple terminal devices with different computing capabilities exist simultaneously, if the live video analysis application determines that the terminal device has strong computing power and sufficient remaining computing resources, it can allow the terminal device to perform the aforementioned calculations, thereby appropriately reducing the computing pressure on the live streaming server 102. Accordingly, the tennis ball video analysis device can also be located within the broadcast client 101. In this case, the exemplary system architecture 100 may not include the live streaming server 102 and the network 104.

[0023] It should be understood that Figure 1 The number of broadcaster clients, live streaming servers, and user clients shown is merely illustrative. Depending on implementation needs, any number of broadcaster clients, live streaming servers, and user clients can be included.

[0024] Please refer to Figure 2 , Figure 2A flowchart of a video analysis method for tennis ball games provided in this disclosure embodiment, wherein process 200 includes the following steps: Step 201: Divide the live video stream of tennis matches into segments according to the score and rounds to obtain at least one live video segment.

[0025] This step is intended for the aforementioned implementing entity of the tennis sports video analysis method (e.g., Figure 1 The live streaming server 102 shown divides the live video stream of tennis ball games into segments according to the score rallies to identify at least one (i.e., one or more) live video segments for detection that begin at the start of the serve and end at the end of a score rally.

[0026] In some optional implementations of the embodiments of this disclosure, the execution entity may detect, based on the received live video stream, whether the athletes (or players) of both sides in the tennis ball match are respectively in the service area and the receiving area, and whether the movement of the ball across the net or the ball is stationary has not been detected for several consecutive frames. Specifically, if it is detected that each athlete is in the service area and the receiving area, and the movement of the ball across the net or the ball is stationary for several consecutive frames, then it can be determined that the tennis ball match has entered the serving phase. Further, after determining the time period corresponding to the tennis ball match entering the serving phase, in response to determining that no ball has been hit for several consecutive frames, or the ball is stationary for several consecutive frames, or the movement of the ball across the net has not been detected for several consecutive frames, then a score round, i.e., the current score round, can be determined to have ended. In some optional implementations of the embodiments of this disclosure, the position of the athlete can be detected using a target detection algorithm based on the YOLO (You Only Look Once) deep neural network. Specifically, the midpoint of the lower boundary of the bounding box in the detection result can be taken as the position of the athlete.

[0027] In some optional implementations of the embodiments of this disclosure, the aforementioned tennis-related sports may include badminton, tennis, or table tennis, and may be either singles or doubles sports.

[0028] Step 202: Perform frame-by-frame detection on any live video segment to obtain the detection position of the target sphere in each live video frame of the live video segment.

[0029] In this embodiment, real-time frame-by-frame detection can be performed on any live video segment obtained by dividing the live video stream of tennis matches into score rounds to obtain the detection result within one score round. Specifically, the detection result may include, but is not limited to, the position of the detected target ball in each live video frame of any live video segment, i.e., the aforementioned detection position. In some optional implementations of this disclosure, a pre-trained ball detection model can be used to identify each live video frame to obtain the position of the target ball in each live video frame. Specifically, a ball position sample, including images and position labels of the ball, can be constructed based on historical tennis match videos, and then a preset deep learning network (such as a deep neural network (DNN)) can be trained based on this sample to obtain the ball detection model.

[0030] It should be noted that the above-mentioned operation of performing frame-by-frame detection on the live video segment corresponding to any score round can be understood as performing frame-by-frame detection on any live video segment obtained by dividing it according to the score round. Therefore, when the live video stream of the above-mentioned tennis ball game is divided into multiple live video segments according to the score round, the operation of performing frame-by-frame detection on each live video segment can be performed.

[0031] Step 203: Based on the fitting results of trajectory fitting of each detection position of the target sphere in time sequence, the detection position of the target sphere in each live video frame is verified to obtain the verified position of the target sphere in each live video frame.

[0032] In this embodiment, considering that the spheres in the scoring rounds typically fly at high speeds, are small targets, and their shapes appearing in live video frames are often irregular, the false detection rate and / or false negative rate of intelligent sphere tracking detection results are difficult to achieve high accuracy. Therefore, to improve the accuracy of sphere position detection, the detected positions of the target sphere in each live video frame of the live video segment, detected frame by frame, can be fitted with trajectories in chronological order, such as performing quadratic curve fitting, to obtain the corresponding fitting results. These fitting results can then be used to verify the detected positions of the target sphere in each live video frame, yielding the corresponding verification positions.

[0033] Step 204: Based on the verification position of the target ball in each live video frame, obtain at least one of the following: the analysis result of the hitting action against the target ball and the analysis result of the hitting trajectory in the live video segment.

[0034] In this embodiment, after verifying the position of the target ball in each live video frame and obtaining its final detected position, i.e., the verified position, in each live video frame, the technical and tactical analysis of the corresponding score round can be realized based on the verified position of the target ball. Specifically, this may include, but is not limited to, obtaining the analysis results of the corresponding hitting action and / or the analysis results of the hitting trajectory for the target ball.

[0035] In some optional implementations of the embodiments of this disclosure, the corresponding hitting actions and / or hitting trajectories may differ depending on the type of tennis ball sport. For example, if the tennis ball sport is badminton, the hitting actions may include, but are not limited to, drop shots, push shots, smashes, flat drives, receiving smashes, high clears, and smashes, and the hitting trajectories may include, but are not limited to, straight lines, diagonal lines, and middle lines in the foreground and backcourt; if the tennis ball sport is tennis, the hitting actions may include, but are not limited to, flat shots, topspin shots, backspin shots, flat serves, slice serves, volleys at the net, and overhead smashes, and the hitting trajectories may include, but are not limited to, deep diagonal lines, shallow straight lines, volleys down the line, and overhead smash diagonal lines; if the tennis ball sport is table tennis, the hitting actions may include, but are not limited to, short pushes, long slices, flicks, topspins, slices, smashes, and sidespins, and the hitting trajectories may include, but are not limited to, forehand diagonal lines, backhand straight lines, short ball trajectories near the net, and long ball trajectories at the baseline.

[0036] The tennis ball video analysis method provided in this embodiment can analyze live video streams of tennis ball games generated in real time. Specifically, the live video stream can be segmented according to the score rounds to obtain live video segments corresponding to each score round. Then, frame-by-frame detection can be performed at the granularity or unit of the live video segments to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using trajectory fitting. This can at least improve the accuracy and efficiency of the target ball position detection in each live video frame. Therefore, when analyzing and identifying the hitting action and / or hitting trajectory of the target ball based on the verified position of the target ball in each live video frame, it can not only improve the efficiency of real-time analysis of the live video and reduce the latency of obtaining the corresponding analysis results, but also improve the accuracy of the analysis of the hitting action and / or hitting trajectory used by the athlete in the tennis ball game.

[0037] Please refer to Figure 3 , Figure 3 A flowchart of another method for analyzing tennis ball sports videos provided in this disclosure embodiment, wherein process 300 includes the following steps: Step 301: Divide the live video stream of tennis matches into segments according to the score and rounds to obtain at least one live video segment.

[0038] Step 302: Perform frame-by-frame detection on any live video segment to obtain the detection position of the target sphere in each live video frame of the live video segment.

[0039] Step 303: Based on the fitting results of trajectory fitting of each detection position of the target sphere in time sequence, the detection position of the target sphere in each live video frame is verified to obtain the verified position of the target sphere in each live video frame.

[0040] Step 304: Based on the verification position of the target ball in each live video frame, obtain at least one of the following: the analysis result of the hitting action against the target ball and the analysis result of the hitting trajectory in the live video segment.

[0041] Steps 301-304 above are similar to... Figure 2 The steps 201-204 shown are the same. For the same parts, please refer to the corresponding parts of the previous embodiment. They will not be repeated here.

[0042] Step 305: Input at least one of the strategy question-and-answer information corresponding to tennis-like sports, as well as the analysis results of hitting actions and the analysis results of hitting trajectories, into a large tactical strategy analysis model pre-trained based on a large language model and a knowledge base corresponding to tennis-like sports, and output tactical strategies for tennis-like sports.

[0043] In this embodiment, the live video stream of a tennis ball game is analyzed in real time to obtain accurate analysis results of the hitting action and / or trajectory for the target ball. These analysis results can then be input into a large-scale tactical strategy analysis model pre-trained based on a large language model and a knowledge base corresponding to tennis ball games. The large-scale model then outputs tactical strategies for the tennis ball game in real time in a dialogue format, providing feedback to athletes and / or coaches. The solution of this embodiment not only enables efficient and accurate analysis of live video streams of tennis ball games but also provides optimized tactical strategies or suggestions in real time.

[0044] Furthermore, in some optional implementations of the embodiments of this disclosure, the tactical strategies for tennis-like sports described above include strategy text information and / or strategy video information. The strategy video information may include example videos of standard hitting actions and / or standard hitting trajectories. Thus, the tactical strategies can be displayed in diverse ways to meet different user needs. Furthermore, the tactical strategies for tennis-like sports described above can be visualized to enhance the user experience.

[0045] Furthermore, in some optional implementations of this disclosure, the knowledge base corresponding to tennis-related sports can be constructed based on standard sports literature and / or standard sports video data corresponding to the sport. Specifically, the standard sports literature and / or standard sports video data can be data annotated by professionals. Further, a large language model can be used as a foundation, combined with the aforementioned knowledge base, to train a large-scale tactical strategy analysis model corresponding to tennis-related sports. Furthermore, by employing Low-Rank Adaptation (LoRA) technology, the newly added professional knowledge, professional data, and coach feedback can be rapidly learned and iterated, forming a "data flywheel" effect through continuous incremental learning.

[0046] Furthermore, in some optional implementations of this disclosure, during the training of a large-scale tactical strategy analysis model based on a knowledge base corresponding to tennis ball sports, a deep learning network can be used to analyze standard motion video data in the knowledge base. For example, this involves analyzing the motion of key points in the athlete's posture across multiple consecutive frames of video data to obtain at least one of the following: detection results of standard joint angle changes, standard joint velocity changes, standard angular velocity changes, and standard muscle force or stress changes during the hitting action against the target ball. Specifically, each detection result can be represented by a change curve. In specific analysis, the aforementioned standard motion video data can be used as a reference video stream for the tennis ball sports and may include the following: (1) Based on the reference video stream, obtain the basic input data for inputting the preset deep learning network; wherein, the basic input data includes multiple consecutive reference video frames (e.g., three consecutive frames) in the reference video stream, the athlete's pose key points in each reference video frame, and the coordinate information (e.g., three-dimensional coordinate information) of the athlete's pose key points in each reference video frame; wherein, the preset deep learning network may include, but is not limited to, a residual network (ResNet), a point cloud deep learning network (PointNet), and a multilayer perceptron (MLP).

[0047] (2) Input the basic input data into ResNet, PointNet and MLP as described above, extract the feature vectors corresponding to multiple different times respectively, and concatenate the feature vectors of different types corresponding to the same time to obtain the comprehensive feature vector corresponding to each time; among them, ResNet can be used to extract image features, PointNet can be used to extract three-dimensional spatial coordinate features and MLP can be used to extract three-dimensional spatial angle features.

[0048] (3) Input the comprehensive feature vectors corresponding to multiple consecutive time points into a Long Short-Term Memory (LSTM) network to extract temporal information, and then output at least one of the following: detection results of standard joint angle changes, standard joint velocity changes, standard angular velocity changes, and standard muscle force or force changes.

[0049] In some optional implementations of the embodiments of this disclosure, in the above... Figure 2 or Figure 3 Based on the corresponding embodiments, before step 203 or step 303 above, it may further include performing frame-by-frame detection on any live video segment determined from the live video stream according to the score rounds, and removing abnormal detection results from the obtained detection results, thereby achieving a preprocessing of the frame-by-frame detection results. Thus, based on this, performing the position verification of the above-mentioned trajectory fitting results can, on the one hand, improve the efficiency of position verification of the target ball in each live video frame, and on the other hand, help improve the accuracy of analysis of the hitting action and / or hitting trajectory used against the target ball. Specifically, the above-mentioned preprocessing process for removing abnormal detection results may include the following: In response to determining the second distance between the detection position of the target sphere in each live video frame and the detection position in the previous live video frame, the detection positions of the target sphere in each live video frame are filtered based on the relationship between the second distance and a second distance threshold, resulting in filtered detection positions. That is, the distance between the detection positions of the target sphere in each pair of adjacent live video frames in a live video segment can be calculated, and the detection positions of the target sphere in the live video frames of the segment can be filtered based on the relationship between the obtained distance (the second distance) and a pre-set distance threshold (the second distance threshold). Further, step 203 or step 303 can be specifically implemented as follows: based on the fitting results of trajectory fitting of each filtered detection position of the target sphere in chronological order, each filtered detection position of the target sphere is verified to obtain the verified position of the target sphere in each live video. The value of the second distance threshold can be set according to the type of tennis ball sport. That is, the value of the second distance threshold can vary depending on the type of tennis ball sport and can be set according to specific needs. No specific limitation is made here.

[0050] Furthermore, in some optional implementations of the embodiments of this disclosure, the step of filtering the detection positions of the target sphere in each live video frame based on the relationship between the second distance and the second distance threshold to obtain the filtered detection positions can be specifically executed as follows: in response to the second distance being greater than the second distance threshold, performing an abnormal detection result removal operation on the detection positions of the target sphere in each live video frame; in response to the completion of the abnormal detection result removal operation on all detection positions of the target sphere in the live video segment, using the remaining detection positions as the filtered detection positions.

[0051] In this embodiment, when the distance between the detection positions of the target sphere in two adjacent live video frames exceeds a preset distance threshold (i.e., a second distance threshold), it can be considered an abnormal detection result and needs to be removed. After filtering all detection positions of the target sphere obtained by frame-by-frame detection based on the live video segment, the remaining detection positions are used as the filtered detection positions, and trajectory fitting can be performed in chronological order based on these positions. Thus, the filtering and removal of abnormal detection results is achieved by setting a distance threshold, which is both efficient and reliable.

[0052] In some optional implementations of any embodiment of this disclosure, step 203 or step 303 may be specifically executed as follows: based on the fitting result, determine the fitting position of the target sphere in each live video frame; in response to a first distance between the detected position and the fitting position of the target sphere in each live video frame being greater than a first distance threshold, use the fitting position as the verification position of the target sphere in each live video frame; in response to a first distance being less than or equal to the first distance threshold, use the detected position as the verification position of the target sphere in each live video frame.

[0053] In this embodiment, the position of the target sphere in each live video frame of the live video segment can be determined based on the fitting result obtained from trajectory fitting as its fitted position. Specifically, secondary processing of the detection result, i.e., the detection position of the target sphere in each live video frame, can be completed based on the relationship between the distance between the detection position and the corresponding fitted position of the target sphere in each live video frame and a pre-set distance threshold, i.e., a first distance threshold. In this way, on the basis of achieving efficient and reliable position verification, the accuracy of the target sphere's position detection in each live video frame can be further improved. Specifically, if the distance between the detection position and the fitted position of the target sphere in each live video frame, i.e., the first distance, is greater than the first distance threshold, it indicates that the detection result is abnormal. To ensure the accuracy of the above analysis results, the more accurate fitted position can be taken as the final position of the target sphere in the corresponding live video frame; otherwise, it indicates that the detection result is relatively accurate and no completion or replacement is required. The value of the first distance threshold can be set according to the type of tennis ball sport, i.e., the value of the first distance threshold can vary depending on the type of tennis ball sport and can be set according to specific needs, without being specifically limited here.

[0054] In some optional implementations of any embodiment of this disclosure, the above-described tennis ball sports video analysis method may further include the following: in response to detecting a target ball in multiple preceding live video frames before each live video frame, performing trajectory fitting in chronological order based on the detection position of the target ball in each preceding live video frame to obtain a fitting result; in response to not detecting a target ball in multiple preceding live video frames, performing trajectory fitting in chronological order based on the detection position of the target ball in multiple subsequent live video frames after each live video frame to obtain a fitting result.

[0055] In this embodiment, for each real-time live video frame in a live video segment, the selection range of detection results for trajectory fitting can be determined based on whether a target sphere is detected in multiple preceding live video frames that are chronologically preceding any or every current real-time live video frame. This ensures the reliability of trajectory fitting while maximizing its efficiency. Specifically, if the target sphere is detected in multiple preceding live video frames, these frames can be cached chronologically. If the target sphere is not detected in these frames, subsequent live video frames can be cached chronologically, and trajectory fitting can be performed based on the detection position of the target sphere in the cached frames. In some optional implementations of this embodiment, the multiple preceding or subsequent live video frames can be consecutive live video frames.

[0056] In some optional implementations of the embodiments of this disclosure, taking into account the processing capabilities of the executing entity and the user's experience of real-time analysis of the live video stream, the granularity of the buffering duration of the aforementioned live video frames can be at the single-second or half-second level, and the number of the aforementioned multiple preceding or subsequent live video frames can be set to a dozen to several dozen frames according to specific needs.

[0057] In a specific example, any of the aforementioned positions can be represented by two-dimensional coordinates. The detection position of the target sphere (e.g., a badminton shuttlecock) in any live video frame of the aforementioned live video segment can be represented as p(x,y). The detection positions pl(x,y) of the target sphere in the previous L (integers greater than or equal to 2) consecutive live video frames of the current live video frame are cached. First, the distance between the position p(x,y) corresponding to the current live video frame and the position pl(x,y) in the preceding live video frame (corresponding to the second distance in the above embodiment) is detected. When this distance is greater than the distance threshold s(p1) (corresponding to the second distance threshold in the above embodiment), the detection result is considered abnormal and needs to be removed. Furthermore, after filtering out the anomaly detection results for all live video frames in the aforementioned live video segment, for each live video frame, quadratic curve fitting can be performed using the detection position data in m (integers greater than or equal to 2) frames preceding the current live video frame. Based on the fitted curve, the current fitted position p(x,y)' is determined. If the distance between the current detection position p(x,y) and the current fitted position p(x,y)' (corresponding to the first distance in the above embodiment) is greater than s(p2) (corresponding to the first distance threshold in the above embodiment), then the detection result is considered abnormal and needs to be removed. The fitted position p(x,y)' determined based on the quadratic curve fitting is used to replace the detection position p(x,y) of the current live video frame as the sphere position of the current frame (corresponding to the verification position in the above embodiment). Furthermore, when the target sphere in the m frames preceding each current live video frame is not visible, the detection position data in the m frames following each current live video frame is cached, and the same quadratic curve fitting result is used to verify and complete the detection result of the current live video frame.

[0058] In some optional implementations of any embodiment of this disclosure, step 204 or step 304 may be specifically performed as follows: determining the hitting frame based on the curve fitting result of the verification position of the target ball in each live video frame; and obtaining at least one of the hitting action analysis result and the hitting trajectory analysis result based on the hitting frame, the verification position of the target ball in each live video frame, and the detection result of the key points of the target athlete's posture in each live video frame in the live video segment.

[0059] In this embodiment, curve fitting can be performed based on the verified position of the target ball in each live video frame (e.g., quadratic curve fitting based on the ordinate and time in the coordinate system used to represent the position), and each hitting frame in the live video segment can be determined based on the obtained fitted curve. The type of hitting frame in the live video segment can vary depending on the type of tennis sport. For example, for badminton, the hitting frame can be a live video frame that includes the moment when each player swings their racket to hit the ball, while for tennis or table tennis, in addition to the live video frame that includes the moment when each player swings their racket to hit the ball, the hitting frame can also be a live video frame that includes the moment when the ball touches the ground. Meanwhile, the detection results obtained by performing frame-by-frame detection on each live video segment can include not only the position of the target ball in each live video frame, but also the detection results of the target athlete's posture key points in each live video frame. Furthermore, based on the determined hitting frame, the verified position of the target ball in each live video frame, and the detection results of the target athlete's posture key points in each live video frame, it is possible to accurately analyze the hitting action and / or hitting trajectory performed by the target athlete against the target ball in each score round.

[0060] Furthermore, in some optional implementations of this disclosure, in the step of obtaining at least one of the following based on the hitting frame, the verification position of the target ball in each live video frame, and the detection results of the target athlete's posture key points in each live video frame, the hitting action analysis result and the hitting trajectory analysis result can be obtained. Specifically, multiple live video frames can be determined forward and backward in chronological order, for example, N (an integer greater than or equal to 2) frames, using each hitting frame as a base point. In this case, (2... If there are N+1) live video frames, then further analysis can be performed based on the target sphere in the (2) The verification position and the target athlete in each of the N+1) live video frames are in the (2) The system detects the posture key points in each of the N+1 live video frames, enabling accurate analysis of the hitting actions and / or hitting trajectories performed by the target athlete against the target ball in each score round.

[0061] In some optional implementations of the embodiments of this disclosure, the above-mentioned posture key point detection results include, but are not limited to, at least one of the following: detection results of joint angle changes, joint velocity changes, angular velocity changes, and muscle force or force changes during the athlete's movement.

[0062] In some optional implementations of the embodiments of this disclosure, based on any of the above embodiments, the above-described tennis-playing video analysis method may further include the following: obtaining a reference position by pre-marking the target reference object in a live video frame containing the sports field corresponding to the tennis-playing sport; and detecting the position of the target reference object on the sports field in each live video frame of the live video segment to obtain a detection position corresponding to the target reference object; filtering each live video frame based on the relationship between the distance between the reference position and the detection position corresponding to the target reference object in each live video frame and a third distance threshold to obtain valid live video frames in the live video segment; and performing frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame of the live video segment, including: performing frame-by-frame detection on the valid live video frames in any live video segment to obtain the detection position of the target ball in each valid live video frame.

[0063] In this embodiment, the target reference objects on the sports field in video frames containing the sports field corresponding to tennis (e.g., video frames in the video stream received before the start of the sport or after the shooting equipment starts running) can be pre-marked as reference positions. Intelligent detection is then performed on the target reference objects on the sports field in each live video frame of the live video segment, for example, using a detection model pre-trained based on a deep learning network to obtain the detection position corresponding to the target reference object. Furthermore, for each live video frame, the live video frames contained in the live video segment can be filtered based on the relationship between the distance between the reference position and the real-time detected detection position corresponding to the target reference object, and a pre-set distance threshold, i.e., a third distance threshold, to obtain valid live video frames for frame-by-frame detection. Thus, by filtering out invalid live video frames in the live video segment, the quality of the live video frames used for intelligent detection can be improved, thereby further improving the accuracy of the analysis of the hitting actions and / or hitting trajectories performed by the target athlete against the target ball in each score rally. The value of the third distance threshold can be set according to the type of tennis ball game. That is, the value of the third distance threshold can vary depending on the type of tennis ball game and can be set according to specific needs. No specific limitation is made here.

[0064] In some optional implementations of this disclosure, the target reference object includes, but is not limited to, at least two of the end lines, sidelines, service lines, center lines, and net posts in the sports field. For example, the reference position may include a first type of marked position obtained by marking all intersections of the end lines and sidelines of the sports field in a live video frame, or a second type of marked position obtained by marking the intersections of the net posts and the sports field. Correspondingly, the detection position corresponding to the target reference object obtained through intelligent detection may include a first type of detection position obtained by intelligent detection of all intersections of the end lines and sidelines of the sports field in a live video frame, or a second type of detection position obtained by intelligent detection of the intersections of the net posts and the sports field.

[0065] Furthermore, in some optional implementations of the embodiments of this disclosure, the step of filtering each live video frame based on the relationship between the distance between the reference position in each live video frame and the detection position corresponding to the target reference object and the third distance threshold, to obtain valid live video frames in the live video segment, can be specifically implemented as follows: in response to the distance between the reference position in each live video frame and the detection position corresponding to the target reference object of the same category being greater than the preset third distance threshold, the live video frame can be considered invalid and thus removed, retaining the valid live video frames for subsequent processing and analysis.

[0066] In a specific example, any of the aforementioned locations can be represented in two-dimensional coordinates. All corner points where the end lines and sidelines of the sports field intersect, and the locations where the net posts intersect with the sports field, are pre-labeled in the live video frame image and used as reference locations court(x,y) and net(x,y), respectively. A region-based convolutional neural network (RCNN) detects the sports field in the live video frame image. If the difference between the intersection location court_n(x,y) and the reference location court(x,y) in the field detection result is greater than the threshold court_th(x,y) (corresponding to the third distance threshold mentioned above), or the difference between the net_n(x,y) and the reference location net(x,y) in the net location detection result is greater than the threshold net_th(x,y) (corresponding to the third distance threshold mentioned above), then the live video frame data is considered invalid. Invalid frame data is removed, and valid data is retained for subsequent processing and analysis.

[0067] Furthermore, in some optional implementations of this disclosure, the following may also be included: marking the intersections between all the constituent lines of the sports field in each live video frame, and obtaining a mapping transformation matrix between the sports field in each live video frame and the standard sports field corresponding to tennis ball games through homography transformation. Further, based on this mapping transformation matrix, the position of the target ball in the aforementioned hitting frame can be mapped to the standard sports field to obtain the position of the target athlete's hitting point. Further, based on the position of the hitting point and the target athlete's position in the aforementioned (2) including the hitting frame... By analyzing the posture key point detection results in each of the N+1) live video frames and the verification position of the target ball in each live video frame, a more accurate analysis of the hitting actions and / or hitting trajectories performed by the target athlete against the target ball in each score round can be achieved, thereby enabling the analysis of tactics and techniques.

[0068] Furthermore, in some optional implementations of any embodiment of this disclosure, the detected athlete's position can also be mapped to a standard sports field based on the mapping transformation matrix, which facilitates accurate analysis of whether the athlete is in the service area or the receiving area, as well as the hitting action and the hitting trajectory.

[0069] Furthermore, in some optional implementations of any embodiment of this disclosure, to facilitate querying and management of the above-mentioned analysis results of the hitting action and / or the hitting trajectory for the target ball, the analysis can be performed in single-round, multi-round, or single-player round or multi-player round formats. Each round may include multiple rounds. The analysis results of the hitting action and / or the hitting trajectory are statistically analyzed according to the live broadcast time and displayed in a visual form, such as by drawing charts. These results can then be sent to a display device for display via a visual feedback node. The statistical results can be stored in a database table.

[0070] Furthermore, in some optional implementations of any embodiment of this disclosure, the live video stream analyzed using the above-described video analysis scheme is saved, and the video time corresponding to the above-described ball-hitting action analysis results and / or ball-hitting trajectory analysis results is indexed, and the indexing results are visualized so that coaches and athletes can view the corresponding video segments.

[0071] It should be noted that the tennis video analysis method described in any of the above embodiments is applicable not only to the application scenario of analyzing the above live video stream, but also to pre-recorded match broadcast video streams and / or training video streams.

[0072] To enhance understanding, this disclosure also provides a specific implementation scheme based on a particular application scenario. Please refer to the example below. Figure 4 The system 400 shown for implementing a video analysis method for tennis ball games may include a video acquisition device 401, a tactical analysis module 402, a tactical strategy analysis module 403, and a visualization display device 404. The video acquisition device 401 can be deployed in... Figure 1 On the broadcast client 101 side shown, the tactical analysis module 402 and the tactical strategy analysis module 403 can be deployed on... Figure 1 On the side of the live streaming server 102 shown, the visualization display device 404 can be deployed on... Figure 1 The user client 103 side is shown.

[0073] The aforementioned video acquisition device 401 can be used to acquire video streams of tennis-like sports, such as live video streams. Specifically, the video acquisition device 401 may include a high-definition camera, and can be positioned directly behind the sports field, shooting from above, with the lens direction as perpendicular as possible to the end lines of the sports field, fixing the video viewing angle position, and ensuring the clarity of the shot, for example, not less than 30 frames per second (FPS). The tactical analysis module 402 can be used to perform specific analysis based on the received live video stream, obtaining at least one of the following: ball-hitting action analysis results and ball-hitting trajectory analysis results. The tactical strategy analysis module 403 can be used to output, in real-time, a large model in the form of a dialogue, tactical strategies for the tennis-like sports, based on the analysis results output by the tactical analysis module 402, for feedback to athletes and / or coaches. In this embodiment, the analysis scheme implemented by the tactical analysis module 402 and the tactical strategy analysis module 403 can be implemented with reference to the corresponding content in the tennis-like sports video analysis method described in any of the above embodiments, and will not be repeated here. The visualization device 404 can be used to display the above analysis results and tactical strategies. Further, in this embodiment, a Robot Operating System (ROS2) can be used to provide a node management and communication framework, and ROS nodes can be created for the corresponding functional modules in the system (such as the aforementioned tactical analysis module 402 and tactical strategy analysis module 403).

[0074] The solution provided in this disclosure implements an automated sports video tactical analysis scheme that integrates athlete detection, posture and movement recognition, ball detection algorithms, and a large-scale technical analysis model. This improves the accuracy and efficiency of tactical data analysis and recording. Specifically, by using advanced vision and artificial intelligence technologies, it can accurately capture and analyze effective rallies in sports videos, quickly and accurately analyze the tactical application performance of both athletes, ensuring that every technical aspect is precisely analyzed. Furthermore, based on professional technical data and a large-scale model base, a large-scale model for analyzing tennis techniques can be trained, and targeted optimization strategies and suggestions can be provided based on video analysis results, such as for the ball techniques and practical applications of teenagers.

[0075] Furthermore, by automating the recording and real-time analysis of athletic performance, reliance on manual operation and specialized knowledge is reduced, thereby improving the consistency, objectivity, and accuracy of data collection. Moreover, it significantly enhances the efficiency of tactical analysis by rapidly processing, analyzing, and statistically analyzing data, and providing coaches and athletes with effective, real-time, question-and-answer-style intelligence through large-scale models. This optimizes training and competition strategies while reducing the risk of athlete injuries. This is crucial not only for optimizing athlete training and competition strategies but also for providing coaches with more detailed and reliable data support, helping athletes to conduct targeted training, prepare for competitions, and improve their tactics during matches.

[0076] The tennis video analysis solution implemented in any of the above embodiments can not only be applied to professional competitive sports, helping coaches and athletes analyze and improve their performance on the field and in training, but also to school sports and mass sports activities, providing technical support for enthusiasts' competition and training analysis. Furthermore, it can be integrated into sports technology products, such as intelligent sports performance analysis software and intelligent fitness equipment, providing users with scientific and accurate sports analysis services and enhancing the user experience.

[0077] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a tennis ball sports video analysis device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices, such as servers.

[0078] like Figure 5As shown, the tennis ball sports video analysis device 500 of this embodiment may include: a segmentation module 501, a detection module 502, a verification module 503, and a processing module 504. The segmentation module 501 is configured to segment the live video stream of tennis ball sports according to the score and rallies, obtaining at least one live video segment. The detection module 502 is configured to perform frame-by-frame detection on any live video segment to obtain the detection position of the target ball in each live video frame within the live video segment. The verification module 503 is configured to verify the detection position of the target ball in each live video frame based on the fitting results of trajectory fitting of each detection position of the target ball in chronological order, obtaining the verified position of the target ball in each live video frame. The processing module 504 is configured to obtain at least one of the following based on the verified position of the target ball in each live video frame: an analysis result of the hitting action of the target ball and an analysis result of the hitting trajectory in the live video segment.

[0079] In this embodiment, the specific processing of the segmentation module 501, detection module 502, verification module 503, and processing module 504 in the tennis sports video analysis device 500, and the resulting technical effects, can be found in the following references: Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiments will not be repeated here.

[0080] In some optional implementations of the embodiments of this disclosure, the above-mentioned verification module 503 is further configured to: determine the fitting position of the target sphere in each live video frame based on the fitting result; in response to a first distance between the detection position and the fitting position of the target sphere in each live video frame being greater than a first distance threshold, use the fitting position as the verification position of the target sphere in each live video frame; in response to a first distance being less than or equal to the first distance threshold, use the detection position as the verification position of the target sphere in each live video frame.

[0081] In some optional implementations of the embodiments of this disclosure, the above-mentioned tennis ball sports video analysis device 500 may further include a fitting module (not shown in the figure), configured to: in response to detecting a target ball in a plurality of preceding live video frames before each live video frame, perform trajectory fitting based on the detection position of the target ball in each preceding live video frame in chronological order to obtain a fitting result; in response to not detecting a target ball in a plurality of preceding live video frames, perform trajectory fitting based on the detection position of the target ball in a plurality of subsequent live video frames after each live video frame, perform trajectory fitting in chronological order to obtain a fitting result.

[0082] In some optional implementations of the embodiments of this disclosure, the above-mentioned tennis ball sports video analysis device 500 may further include a first filtering module (not shown in the figure), configured to: in response to determining a second distance between the detection position of the target ball in each live video frame and the detection position in the previous live video frame of each live video frame, filter the detection positions of the target ball in each live video frame based on the relationship between the second distance and a second distance threshold to obtain filtered detection positions; and the above-mentioned verification module 503 is further configured to: verify each filtered detection position of the target ball based on the fitting result of trajectory fitting of each filtered detection position of the target ball in chronological order to obtain the verification position of the target ball in each live video.

[0083] In some optional implementations of the embodiments of this disclosure, the first filtering module is further configured to: in response to the second distance being greater than the second distance threshold, perform an abnormal detection result removal operation on the detection positions of the target sphere in each live video frame; in response to the completion of the abnormal detection result removal operation on all detection positions of the target sphere in the live video segment, use the remaining detection positions as the filtered detection positions.

[0084] In some optional implementations of the embodiments of this disclosure, the processing module 504 is further configured to: determine the hitting frame based on the curve fitting result of the verification position of the target ball in each live video frame; and obtain at least one of the hitting action analysis result and the hitting trajectory analysis result based on the hitting frame, the verification position of the target ball in each live video frame, and the detection result of the posture key points of the target athlete in each live video frame in the live video segment.

[0085] In some optional implementations of the embodiments of this disclosure, the above-mentioned video analysis device 500 for tennis-like sports may further include a determining module (not shown in the figure), which is configured to: input at least one of the strategy question-and-answer information corresponding to tennis-like sports, as well as the analysis results of hitting actions and the analysis results of hitting trajectories, into a large tactical strategy analysis model pre-trained based on a large language model and a knowledge base corresponding to tennis-like sports, and output a tactical strategy for tennis-like sports.

[0086] In some optional implementations of the embodiments of this disclosure, the tactical strategies for tennis include strategy text information and / or strategy video information.

[0087] In some optional implementations of the embodiments of this disclosure, the above-mentioned tennis ball sports video analysis device 500 further includes a second filtering module (not shown in the figure), which is configured to: pre-mark the reference position based on the target reference object in the live video frame containing the sports field corresponding to the tennis ball sports, and perform position detection on the target reference object on the sports field in each live video frame of the live video segment to obtain the detection position corresponding to the target reference object; filter each live video frame based on the relationship between the distance between the reference position and the detection position corresponding to the target reference object in each live video frame and a third distance threshold to obtain the valid live video frames in the live video segment; and the above-mentioned detection module 502 is further configured to: perform frame-by-frame detection on the valid live video frames in any live video segment to obtain the detection position of the target ball in each valid live video frame.

[0088] In some optional implementations of the embodiments of this disclosure, the target reference objects mentioned above include at least two of the end lines, sidelines, service lines, center lines, and net posts in the sports field.

[0089] This embodiment exists as a device embodiment corresponding to the above method embodiment. It can analyze the live video stream of tennis-like sports generated in real time. Specifically, the live video stream can be segmented according to the score rounds to obtain live video segments corresponding to each score round. Then, frame-by-frame detection can be performed at the granularity or unit of the live video segments to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using trajectory fitting. In this way, at least the accuracy and efficiency of the target ball position detection in each live video frame can be improved. Therefore, when analyzing and identifying the hitting action and / or hitting trajectory of the target ball based on the verified position of the target ball in each live video frame, it can not only improve the efficiency of real-time analysis of the live video and reduce the latency of obtaining the corresponding analysis results, but also improve the accuracy of the analysis of the hitting action and / or hitting trajectory used by the athlete in the tennis-like sports.

[0090] According to embodiments of this disclosure, this disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the tennis ball sports video analysis method described in any of the above embodiments.

[0091] According to embodiments of this disclosure, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the tennis ball sports video analysis method described in any of the above embodiments when executed.

[0092] According to embodiments of this disclosure, this disclosure also provides a computer program product including a computer program that, when executed by a processor, can implement the tennis ball sports video analysis method described in any of the above embodiments.

[0093] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0095] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the tennis ball sports video analysis method. For example, in some embodiments, the tennis ball sports video analysis method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the tennis ball sports video analysis method described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform a tennis ball sports video analysis method by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology. Cloud servers, also known as cloud computing servers or cloud hosts, are a hosting product within the cloud computing service ecosystem, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak business scalability.

[0103] According to the technical solution for video analysis of tennis-type sports according to embodiments of this disclosure, analysis can be performed based on live video streams of tennis-type sports generated in real time. Specifically, the live video stream can be segmented according to the score rounds to obtain live video segments corresponding to each score round. Then, frame-by-frame detection can be performed at the granularity or unit of the live video segments to obtain the detection position of the target ball in each live video. Furthermore, the detection position of the target ball in each live video frame can be verified, specifically by using trajectory fitting. This can at least improve the accuracy and efficiency of the target ball position detection in each live video frame. Therefore, when analyzing and identifying the hitting action and / or hitting trajectory of the target ball based on the verified position of the target ball in each live video frame, it can not only improve the efficiency of real-time analysis of the live video and reduce the latency of obtaining the corresponding analysis results, but also improve the accuracy of the analysis of the hitting action and / or hitting trajectory used by the athlete in the tennis-type sports.

[0104] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0105] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for analyzing tennis-related sports videos, comprising: For live video streams of tennis matches, divide them according to the score of each round to obtain at least one live video segment; Frame-by-frame detection is performed on any of the live video segments to obtain the detection position of the target sphere in each live video frame of the live video segment. Based on the fitting results of trajectory fitting of each detection position of the target sphere in chronological order, the detection positions of the target sphere in each live video frame are verified to obtain the verified positions of the target sphere in each live video frame. This includes: determining the fitted position of the target sphere in each live video frame based on the fitting results; in response to a first distance between the detection position and the fitted position of the target sphere in each live video frame being greater than a first distance threshold, using the fitted position as the verified position of the target sphere in each live video frame; and in response to a first distance being less than or equal to the first distance threshold, using the detection position as the verified position of the target sphere in each live video frame. Based on the verification position of the target ball in each live video frame, at least one of the following is obtained: the analysis result of the hitting action against the target ball and the analysis result of the hitting trajectory in the live video segment.

2. The method according to claim 1, further comprising: In response to the detection of the target sphere in multiple preceding live video frames before each live video frame, the trajectory is fitted in chronological order based on the detection position of the target sphere in each preceding live video frame to obtain the fitting result; In response to the absence of detection of the target sphere in the plurality of preceding live video frames, trajectory fitting is performed based on the detection position of the target sphere in a plurality of subsequent live video frames after each live video frame, and the trajectory fitting is performed in chronological order to obtain the fitting result.

3. The method according to claim 1, further comprising: In response to determining a second distance between the detection position of the target sphere in each live video frame and its detection position in the previous live video frame, the detection positions of the target sphere in each live video frame are filtered based on the relationship between the second distance and a second distance threshold, resulting in filtered detection positions; and The method of verifying the detection position of the target sphere in each live video frame based on the fitting results of trajectory fitting of each detection position of the target sphere in chronological order, and obtaining the verified position of the target sphere in each live video frame, includes: Based on the fitting results of trajectory fitting of each filtered detection position of the target sphere in chronological order, the each filtered detection position of the target sphere is verified to obtain the verification position of the target sphere in each live video.

4. The method according to claim 3, wherein, The step of filtering the detection positions of the target sphere in each live video frame to obtain the filtered detection positions includes: In response to the second distance being greater than the second distance threshold, an abnormal detection result removal operation is performed on the detection position of the target sphere in each live video frame; In response to performing the abnormal detection result removal operation on all detection positions of the target sphere in the live video segment, the remaining detection positions are used as the filtered detection positions.

5. The method according to claim 1, wherein, The step of obtaining at least one of the following based on the verification position of the target ball in each live video frame, namely the analysis result of the hitting action against the target ball and the analysis result of the hitting trajectory in the live video segment, includes: The hitting frame is determined based on the curve fitting results of the test position of the target ball in each live video frame. Based on the hitting frame, the verified position of the target ball in each live video frame, and the detection results of the key points of the target athlete's posture in each live video frame, at least one of the hitting action analysis results and the hitting trajectory analysis results is obtained.

6. The method according to claim 1, further comprising: The strategy question-and-answer information corresponding to the tennis ball game, as well as at least one of the hitting action analysis results and hitting trajectory analysis results, are input into a large tactical strategy analysis model pre-trained based on a large language model and a knowledge base corresponding to the tennis ball game, and the model outputs a tactical strategy for the tennis ball game.

7. The method according to claim 6, wherein, The tactical strategies for the tennis ball sport include strategy text information and / or strategy video information.

8. The method according to claim 1, further comprising: Based on the target reference object in the live video frame containing the sports field corresponding to the net-type sports, the reference position is obtained by pre-marking the position, and the position of the target reference object on the sports field in each live video frame of the live video segment is detected to obtain the detection position corresponding to the target reference object. Based on the relationship between the distance between the reference position and the detection position corresponding to the target reference object in each live video frame and the third distance threshold, each live video frame is filtered to obtain the effective live video frames in the live video segment. as well as The step of performing frame-by-frame detection on any of the live video segments to obtain the detection position of the target sphere in each live video frame of the live video segment includes: performing frame-by-frame detection on each valid live video frame of any of the live video segments to obtain the detection position of the target sphere in each valid live video frame.

9. The method according to claim 8, wherein, The target reference objects include at least two of the following in the sports field: end line, sideline, service line, center line, and net posts.

10. A video analysis device for tennis sports, comprising: The segmentation module is configured to segment the live video stream of tennis matches according to the score of each round, resulting in at least one live video segment. The detection module is configured to perform frame-by-frame detection on any of the live video segments to obtain the detection position of the target sphere in each live video frame of the live video segment; The verification module is configured to verify the detection position of the target sphere in each live video frame based on the fitting result of trajectory fitting of each detection position of the target sphere in chronological order, and obtain the verification position of the target sphere in each live video frame. The processing module is configured to obtain at least one of the following based on the verification position of the target ball in each live video frame: the analysis result of the hitting action against the target ball and the analysis result of the hitting trajectory in the live video segment. The verification module is further configured to: determine the fitted position of the target sphere in each live video frame based on the fitting result; in response to a first distance between the detected position and the fitted position of the target sphere in each live video frame being greater than a first distance threshold, use the fitted position as the verification position of the target sphere in each live video frame; and in response to a first distance being less than or equal to the first distance threshold, use the detected position as the verification position of the target sphere in each live video frame.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the tennis ball sports video analysis method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the tennis video analysis method according to any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the tennis ball sports video analysis method according to any one of claims 1-9.

Images