Methods, equipment, and storage media for generating special effects in ball sports videos

Abstract

This application discloses a method, device, and storage medium for generating special effects in ball sports videos, relating to the field of video processing technology. The method includes: acquiring a ball sports video stream, the video stream containing a ball; performing ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame; determining the first position coordinates of the ball in the ball-hitting frame as the ball-hitting position coordinates in the ball-hitting frame; generating a ball-hitting effect layer based on the ball-hitting position coordinates and target ball-hitting effect data; and compositing the ball-hitting effect layer with the ball sports video stream to generate a first special effects video with ball-hitting effects. This solves the technical problem of low generation efficiency and difficulty in real-time application of existing special effects generation methods for ball sports videos, thereby improving processing efficiency to the real-time level while ensuring the accuracy of core tracking and ball-hitting event recognition.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, device, and storage medium for generating special effects in ball sports videos. Background Technology

[0002] In the current technological context, there are two main technical approaches to adding dynamic effects to videos of ball sports such as tennis. The first approach is a manual or semi-manual post-production compositing method based on professional video editing software (such as Adobe Premiere Pro and After Effects). This method relies entirely on manual operation, requiring post-production editors to manually draw the trajectory of the tennis ball frame by frame or through keyframe animation, and then select special effects materials from the material library for precise alignment, size adjustment, and timing matching to achieve visual effects such as particle trails and ball-hitting light effects. Although the effects are exquisite, the production process is extremely time-consuming and has a high professional threshold, making it unsuitable for the needs of rapid, batch processing. The second category is automatic generation methods based on computer vision algorithms. These methods use deep learning models (such as the YOLO series and DeepSORT) to automatically detect and track spheres in videos, and then overlay preset effects onto the screen based on the obtained sphere coordinate data. Although this method achieves automated processing, it usually requires the use of highly complex deep learning models and relies on high-performance graphics processing units (GPUs) for inference. This results in high hardware costs and significant processing latency, making it difficult to achieve real-time or near-real-time effect generation on ordinary hardware. Therefore, it cannot be applied to lightweight or real-time-critical scenarios such as live streaming and real-time interaction.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a method, device, and storage medium for generating special effects in ball sports videos, aiming to solve the technical problems of low generation efficiency and difficulty in real-time application of existing special effects generation methods for ball sports videos.

[0005] To achieve the above objectives, this application proposes a method for generating special effects in ball sports videos, the method comprising:

[0006] Acquire a video stream of ball sports, wherein the video stream contains a ball;

[0007] Perform ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame in the video frame and the first position coordinates of the ball in the ball-hitting frame;

[0008] The first position coordinates of the ball in the hitting frame are determined as the hitting position coordinates in the hitting frame. Based on the hitting position coordinates and the target hitting effect data, a hitting effect layer is generated.

[0009] The ball-hitting effect layer is combined with the ball-playing video stream to generate a first effect video with ball-hitting effects.

[0010] In one embodiment, the ball sports video stream further includes a player and a racket, and the step of detecting ball-hitting events in each video frame of the ball sports video stream to obtain the hitting frame and the first position coordinates of the ball in the hitting frame includes:

[0011] The sphere is tracked and detected to generate the first position coordinates of the sphere in each video frame of the ball motion video stream and the motion speed of the sphere in each video frame;

[0012] Target detection is performed on the racket to generate the second position coordinates of the racket in each of the video frames;

[0013] The player's hitting posture is detected, and the hitting posture detection results of the player in each of the video frames are generated. The hitting posture detection results are used to indicate whether the player's posture is a hitting posture.

[0014] The hitting frames in the ball game video stream are determined based on the first position coordinates, the second position coordinates, the movement speed, and the hitting posture detection results of each video frame.

[0015] In one embodiment, the step of determining the hitting frame in the ball game video stream based on the first position coordinates, the second position coordinates, the movement speed, and the hitting posture detection result of each of the video frames includes:

[0016] Based on the motion speed of each video frame, calculate the change in the motion speed of the sphere between each video frame and the previous video frame;

[0017] For each video frame, calculate the spatial distance between the first position coordinate and the second position coordinate of the video frame;

[0018] For each video frame, determine whether the hitting posture detection result indicates that the player's posture is a hitting posture;

[0019] If the absolute value of the change in motion speed is greater than a preset speed change threshold, the spatial distance is less than a preset distance threshold, and the ball-hitting posture detection result is yes, then the video frame is determined as the ball-hitting frame.

[0020] In one embodiment, before the step of generating the shot effect layer based on the shot position coordinates and the target shot effect data, the method further includes:

[0021] In response to the user's selection of special effects data, the target hitting special effects data is determined from multiple candidate hitting special effects data, wherein the target hitting special effects data is a video clip of hitting special effects containing a transparency channel;

[0022] The step of generating a shot effect layer based on the shot position coordinates and target shot effect data includes:

[0023] Using the ball-hitting position coordinates as the central anchor point, each frame of the ball-hitting effect video clip is sequentially superimposed onto the continuous video frames starting from the ball-hitting frame in the ball sports video stream to form the ball-hitting effect layer.

[0024] In one embodiment, the step of compositing the ball-hitting effect layer with the ball game video stream to generate a first effect video with ball-hitting effects includes:

[0025] Using an Alpha channel blending algorithm, the ball-hitting effect layer is blended and superimposed with the corresponding video frame in the ball game video stream to obtain the first effect video with the ball-hitting effect.

[0026] In one embodiment, before the step of compositing the ball-hitting effect layer with the ball game video stream, the method further includes:

[0027] The video frames with odd frame numbers in the ball sports video stream are taken as the first target frames. The players in each first target frame are detected based on the target detection algorithm, and the bounding box coordinates of the first player corresponding to each first target frame are generated.

[0028] The video frames with even-numbered frame sequences in the ball sports video stream are taken as the second target frames. Based on the first player bounding box coordinates corresponding to each first target frame, the second player bounding box coordinates corresponding to each second target frame are calculated by linear interpolation.

[0029] Based on the player's first player bounding box coordinates or second player bounding box coordinates in each of the video frames, a player frame effect layer is generated.

[0030] The player frame effect layer, the ball-hitting effect layer, and the ball-playing video stream are combined to generate a second effect video with player frame effects and ball-hitting effects.

[0031] In one embodiment, the step of generating a player frame effect layer based on the player's first player bounding box coordinates or the second player bounding box coordinates in each of the video frames includes:

[0032] For video frames with odd frame numbers, a special effect graphic of a preset shape is drawn with the midpoint of the bottom edge of the corresponding first player bounding box as the geometric center, the length of the bottom edge of the first player bounding box as the major axis, and one-third of the length of the bottom edge of the first player bounding box as the minor axis, with the minor axis perpendicular to the major axis.

[0033] For video frames with even frame numbers, the special effect graphic of the preset shape is drawn with the midpoint of the bottom edge of the corresponding second player bounding box as the geometric center, the length of the bottom edge of the second player bounding box as the major axis, and one-third of the length of the bottom edge of the second player bounding box as the minor axis, with the minor axis perpendicular to the major axis.

[0034] Combine the special effects graphics corresponding to all video frames into the player frame special effects layer.

[0035] In one embodiment, before the step of compositing the player frame effect layer, the ball-hitting effect layer, and the ball game video stream, the method further includes:

[0036] In response to the user's selection, select the target effect image from the candidate effect images;

[0037] Using the first position coordinates of the sphere in each of the video frames as the center anchor point, the target effect image is sequentially superimposed onto each of the video frames in the ball motion video stream to form a sphere trajectory effect layer;

[0038] The player frame effect layer, the ball trajectory effect layer, and the shot effect layer are combined with the ball game video stream to generate a third effect video with the player frame effect, the ball trajectory effect, and the shot effect.

[0039] Furthermore, to achieve the above objectives, this application also proposes a device for generating special effects for ball sports videos, the device comprising:

[0040] The video acquisition module is used to acquire video streams of ball sports, wherein the video streams of ball sports include a ball;

[0041] The ball-hitting event detection module is used to perform ball-hitting event detection on each video frame in the ball sports video stream, and obtain the ball-hitting frame in the video frame and the first position coordinates of the ball in the ball-hitting frame;

[0042] The special effects generation module is used to determine the first position coordinates of the ball in the hitting frame as the hitting position coordinates in the hitting frame, and generate a hitting effect layer based on the hitting position coordinates and target hitting effect data;

[0043] The compositing module is used to combine the ball-hitting effect layer with the ball-playing video stream to generate a first effect video with ball-hitting effects.

[0044] In addition, to achieve the above objectives, this application also proposes a device for generating special effects for ball sports videos. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the ball sports video special effects generation method described above.

[0045] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the ball sports video effects generation method described above.

[0046] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the ball sports video effects generation method described above.

[0047] The present application proposes one or more technical solutions, which have at least the following technical effects: First, it acquires a video stream containing a ball in motion; second, it performs ball-hitting event detection on each video frame in the video stream to obtain the ball-hitting frame and the first position coordinates of the ball within the ball-hitting frame, replacing the manual frame-by-frame judgment and trajectory drawing of existing manual solutions. Then, it determines the first position coordinates of the ball within the ball-hitting frame as the ball-hitting position coordinates in the ball-hitting frame, and generates a ball-hitting effect layer based on the ball-hitting position coordinates and target ball-hitting effect data. Finally, it synthesizes the ball-hitting effect layer with the video stream to generate a first-effect video with ball-hitting effects. This solves the technical problem of low generation efficiency and difficulty in real-time application of existing methods for generating effects for ball-motion videos. While ensuring the accuracy of core tracking and ball-hitting event recognition, it improves processing efficiency to real-time levels, enabling ordinary users to easily create professional-looking effect videos for use in live streaming, mobile devices, and a wide range of amateur scenarios. The method for generating special effects in ball sports videos provided in this application first directly acquires the ball sports video stream. An algorithm automatically detects hitting events in each video frame of the video stream, accurately identifying the hitting frame and the ball's first position coordinates within that frame. This replaces the tedious manual process of drawing trajectories frame by frame and determining the hitting moment, saving significant time and greatly improving generation efficiency. Simultaneously, using the ball's first position coordinates directly as the hitting position coordinates, preset target hitting effect data is called to generate a hitting effect layer covering only a local area. Finally, this hitting effect layer is overlaid onto the corresponding frame of the original ball sports video stream, leaving the remaining frames unchanged, generating a first-effect video with the hitting effect. This simplifies the compositing process, quickly responds to video stream processing needs, and enables fast, automatic, and visually reasonable special effects generation on ordinary computing devices, meeting the requirements of real-time or near-real-time application scenarios. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating an embodiment of the method for generating special effects in ball sports videos according to this application.

[0051] Figure 2A schematic diagram illustrating the initial stage of a ball-hitting effect provided in this application;

[0052] Figure 3 A schematic diagram illustrating the peak phase of a shot effect provided in this application;

[0053] Figure 4 A schematic diagram illustrating the dissipation phase of a ball-hitting effect provided in this application;

[0054] Figure 5 A flowchart illustrating another method for generating video effects for ball sports provided in this application;

[0055] Figure 6 A schematic diagram of a special effects graphic including an ellipse, provided for this application;

[0056] Figure 7 A flowchart illustrating another method for generating special effects for ball sports videos provided in this application;

[0057] Figure 8 A schematic diagram of a video frame of a third special effect video with a sphere trajectory effect overlaid with a circle effect provided in this application;

[0058] Figure 9 A schematic diagram of a video frame of another third effect video provided in this application, which is overlaid with a sphere trajectory effect in the style of a circle;

[0059] Figure 10 A schematic diagram of a video frame of a third special effect video with a sphere trajectory effect overlaid with a bright light effect provided in this application;

[0060] Figure 11 A schematic diagram of a video frame of another third effect video provided in this application, which is overlaid with a sphere trajectory effect with a bright light effect style;

[0061] Figure 12 This is a schematic diagram of the module structure of the ball sports video effects generation device according to an embodiment of this application;

[0062] Figure 13 This is a schematic diagram of the device structure of the hardware operating environment involved in the method for generating special effects for ball sports videos in this application. Detailed Implementation

[0063] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0064] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0065] The main solution of this application embodiment is as follows: acquire a ball sports video stream, the ball sports video stream containing a ball; perform ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame; determine the first position coordinates of the ball in the ball-hitting frame as the ball-hitting position coordinates in the ball-hitting frame; generate a ball-hitting effect layer based on the ball-hitting position coordinates and target ball-hitting effect data; and synthesize the ball-hitting effect layer with the ball sports video stream to generate a first effect video with ball-hitting effects.

[0066] In this embodiment, for ease of description, the following description will focus on a ball sports video effects generation system.

[0067] There are two main technical approaches to adding dynamic effects to videos of ball sports such as tennis. The first approach is a manual or semi-manual post-production compositing method based on professional video editing software (such as Adobe Premiere Pro and After Effects). This method relies entirely on manual operation, requiring post-production editors to manually draw the trajectory of the tennis ball frame by frame or through keyframe animation. After selecting special effects materials from the material library, the selected special effects materials are aligned with the manually drawn trajectory, and their position, size, appearance, and disappearance time are adjusted frame by frame to match the moments of the tennis ball hitting and landing in the video, in order to achieve visual effects such as particle trails and hitting light effects. Although the effects are exquisite, the production process is extremely time-consuming and has a high professional threshold, which cannot meet the needs of rapid and batch processing. The second category is automatic generation methods based on computer vision algorithms. These methods automatically detect and track spheres and players in videos using trained deep learning models (such as the YOLO series and DeepSORT), and continue tracking in subsequent frames to obtain the two-dimensional bounding boxes or center point coordinates of the spheres and players in each frame. Based on the obtained sphere coordinate data, preset effects are superimposed on the screen. Although this method achieves automated processing, it usually requires the use of highly complex deep learning models. When processing high frame rate and high resolution motion videos, the computational load of frame-by-frame analysis is huge, making it impossible to achieve real-time or near real-time effect generation. It usually requires minutes or even longer background processing time and relies on high-performance GPUs for inference. Therefore, it suffers from high hardware costs and significant processing latency, making it difficult to achieve real-time or near real-time effect generation on ordinary hardware. Consequently, it cannot be applied to lightweight or real-time-critical scenarios such as live streaming and real-time interaction.

[0068] This application provides a solution that first directly acquires a video stream of ball sports, and then automatically detects hitting events in each video frame of the video stream using an algorithm. This accurately identifies the hitting frame and the ball's first position coordinates within that frame, replacing the tedious manual process of drawing trajectories frame by frame and determining the hitting moment, saving significant time and greatly improving generation efficiency. Simultaneously, using the ball's first position coordinates directly as the hitting position coordinates, preset target hitting effect data is called to generate a hitting effect layer that only covers a local area. Finally, this hitting effect layer is overlaid onto the corresponding frame of the original ball sports video stream, leaving the remaining frames unchanged, generating a first-effect video with the hitting effect. This simplifies the compositing process, quickly responds to video stream processing needs, and enables fast, automatic, and visually plausible effect generation on ordinary computing devices, meeting the requirements of real-time or near-real-time application scenarios.

[0069] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a ball sports video effects generation device capable of achieving the above functions. The following description uses a ball sports video effects generation system as an example to illustrate this embodiment and the subsequent embodiments.

[0070] Based on this, embodiments of this application provide a method for generating special effects in ball sports videos, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method for generating special effects for ball sports videos according to this application.

[0071] In this embodiment, the method for generating special effects for ball sports videos includes steps 101-104:

[0072] Step 101: Obtain the video stream of ball sports, which includes the ball itself;

[0073] Step 102: Perform ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame.

[0074] Specifically, a ball sports video stream can refer to a continuous sequence of video frames read from input ports such as cameras, local files, and network live streaming sources (e.g., live streams of tennis and badminton matches, sports videos shot by mobile phones). It contains core elements such as the ball, player, racket, and court, serving as the raw data carrier for special effects generation, and supports real-time or offline modes. A video frame is the basic unit of a ball sports video stream; each frame is a static image, played continuously at a fixed frame rate (e.g., 30 frames / second) to form dynamic video. The video frame is the smallest processing unit for detecting ball-hitting events. Ball-hitting event detection refers to the process of automatically identifying the key event of "ball contact with racket" in the video stream through algorithms. Its core is the precise location of the "moment of impact" (hitting frame) and the "hitting position" (ball coordinates), replacing manual judgment. The hitting frame is a specific video frame in the ball sports video stream that captures the exact moment of "ball contact with racket," serving as the key time node for triggering hitting effects and determining the timing of the effects' overlay. The first position coordinate is the two-dimensional pixel coordinate (x, y) of the center of the ball in the hitting frame, which accurately corresponds to the spatial position of the ball when it comes into contact with the racket, and is the core positioning benchmark for the hitting effect.

[0075] In some embodiments, after system startup, a real-time video stream of ball sports can be acquired by turning on a camera, and video parameters such as resolution and frame rate of the video stream can be obtained. Then, a recording file is created to store the original video stream. If no stop processing command is triggered, the video stream is read frame by frame. After each frame is acquired, it is cached in a queue for use in subsequent detection processes. Based on this, the system performs ball-hitting event detection on each cached video frame: a lightweight object detection model can be used to identify the ball frame by frame and track its trajectory. Combining multimodal features such as racket position, player posture, and ball speed changes, the system comprehensively determines whether the current video frame is a hitting frame. If it is determined to be a hitting frame, the center coordinates of the ball in that frame are extracted as the first position coordinates. This achieves automatic identification and positioning of key hitting moments in the ball sports video stream, providing a "temporal reference" (hitting frame) and a "spatial reference" (first position coordinates) for subsequent special effects generation. This ensures that the hitting effects and the actual hitting action are accurately matched in time and position, avoiding effect misalignment. It also eliminates the tedious operation of manually finding the hit time frame by frame and manually marking the hit position in the existing manual solution, greatly reducing the professional threshold and time cost, and improving the efficiency of special effects generation.

[0076] Optionally, the ball sports video stream also includes players and rackets. The steps of performing hit event detection on each video frame in the ball sports video stream to obtain the hit frame and the first position coordinates of the ball in the hit frame include:

[0077] Track and detect the sphere to generate the first position coordinates of the sphere in each video frame of the ball motion video stream and the motion speed in each video frame;

[0078] Target detection is performed on the racket to generate the second position coordinates of the racket in each video frame;

[0079] The player's hitting posture is detected, and the hitting posture detection results of the player in each video frame are generated. The hitting posture detection results are used to indicate whether the player's posture is a hitting posture.

[0080] Based on the detection results of the first position coordinates, second position coordinates, motion speed, and hitting posture of each video frame, the hitting frames in the ball game video stream are determined.

[0081] Specifically, ball tracking and detection refers to the algorithmic process of continuously identifying and locating ball targets in a video stream of ball sports, outputting the ball's position and motion state in each video frame, which forms the basis for obtaining spatial and motion information related to the shot. The first position coordinates are the two-dimensional pixel coordinates of the ball's center in a single frame of video, serving as the core data for locating the ball's spatial position. The motion speed is the ball's movement rate calculated based on the difference in ball positions across consecutive video frames, combined with the video frame rate, reflecting the dynamic characteristics of the ball's motion state. Racket target detection is the process of single-frame identification and location of the racket target in a video stream of ball sports, outputting the racket's spatial coordinates in each video frame, i.e., the second position coordinates. The second position coordinates are the two-dimensional pixel coordinates of the racket's bounding box center (or key area) in a single frame of video, used to determine the spatial distance between the racket and the ball. Player hitting posture detection refers to the process of determining whether the player is in the action of swinging the racket by analyzing the shape of the player's bounding box or the position of key points (such as the shoulder and elbow). The hitting posture detection result is a binary result used to identify whether a player's posture is a "hitting posture" or not, and serves as an auxiliary basis for filtering out non-hitting scenarios.

[0082] For example, ball tracking and detection can be performed on each video frame in a ball sports video stream. A lightweight object detection model with full integer quantization (such as YOLOv8n) can be used to identify the ball frame by frame, outputting the two-dimensional pixel coordinates of the ball's center in each video frame (i.e., the first position coordinates). Simultaneously, the pixel difference between the first position coordinates of the ball in adjacent video frames is calculated, and combined with the video frame rate (e.g., 30 frames / second), the ball's motion speed in each frame is obtained, thus capturing the dynamic changes in the ball's motion state. Then, the same lightweight object detection model is reused to extend the "racket" detection category, identifying the racket frame by frame and outputting the two-dimensional pixel coordinates of the racket's bounding box center (i.e., the second position coordinates). At the same time, a lightweight pose estimation network is used to extract key points from the player, and based on geometric features such as elbow angle, arm extension direction, and torso orientation, it is determined whether the player is currently in a swinging posture, generating a "yes / no" swing posture detection result. The lightweight pose estimation network can be MoveNet Thunder. Subsequently, the four types of information—first position coordinates, second position coordinates, motion speed, and hitting posture detection results—are spatiotemporally aligned and comprehensively judged according to preset fusion rules to determine the hitting frame in the ball game video stream. This collaborative judgment of multi-dimensional features replaces the tedious manual frame-by-frame judgment of the hitting moment, avoiding the risk of misjudgment from single-dimensional detection and accurately locking the temporal sequence (hitting frame) and spatial position (first position coordinates) of the hitting instant. This provides a precise benchmark for the temporal alignment and spatial positioning of subsequent hitting effects, ensuring a natural fit between the effects and the actual hitting action.

[0083] Optionally, the steps for determining the hitting frames in the ball game video stream based on the first position coordinates, second position coordinates, motion speed, and hitting posture detection results of each video frame include:

[0084] Based on the motion speed of each video frame, calculate the change in the motion speed of the sphere between each video frame and the previous video frame.

[0085] For each video frame, calculate the spatial distance between the first and second position coordinates of the video frame;

[0086] For each video frame, determine whether the hitting posture detection result indicates that the player's posture is a hitting posture;

[0087] If the absolute value of the change in motion speed is greater than the preset speed change threshold, the spatial distance is less than the preset distance threshold, and the ball-hitting posture detection result is yes, the video frame is determined as the ball-hitting frame.

[0088] Specifically, the change in motion speed refers to the absolute value of the difference between the ball's motion speed in the current video frame and the ball's motion speed in the previous video frame. It is a core indicator for quantifying the degree of abrupt change in the ball's motion state. At the moment of impact, the ball experiences a significant sudden change in speed due to the force exerted by the racket, resulting in a substantial increase in the change in motion speed. The preset speed change threshold is a critical value obtained through training with sample data from ball sports scenarios such as tennis or badminton. It is used to define the difference between "abrupt change in ball speed caused by impact" and "speed fluctuations during normal flight," serving as a benchmark for determining whether the ball is in a hitting state. Spatial distance refers to the two-dimensional pixel straight-line distance between the ball's first position coordinates and the racket's second position coordinates within the same video frame, directly reflecting the spatial proximity of the ball and racket. The preset distance threshold is a critical value obtained through training with sample data from ball sports scenarios. It is used to define the spatial states of "ball and racket in contact / near contact" and "ball and racket separated," serving as a benchmark for determining the spatial characteristics of hitting behavior.

[0089] For example, for each video frame in a ball game video stream, the ball's velocity in that frame and the previous frame is retrieved. By calculating the difference in velocity between two adjacent frames and taking its absolute value, the change in the ball's velocity between the current and previous video frames is obtained, thus capturing the abrupt velocity change caused by the racket's force at the moment of impact. Secondly, the two-dimensional coordinate straight-line distance calculation formula (…) can be used. ,in( , () represents the coordinates of the first position of the sphere. , (where is the second position coordinate of the racket). The spatial distance between the ball and the racket within the same video frame is calculated to quantify their spatial proximity. Next, for each video frame, the previously generated hitting posture detection result is called to determine whether the player's posture in that video frame is a hitting posture, filtering out non-hitting scenarios such as racket swings. Finally, preset speed change thresholds and preset distance thresholds, trained using ball sports scene sample data, are retrieved to perform multi-condition collaborative verification on each video frame: only when the absolute value of the speed change in that video frame is greater than the preset speed change threshold, the spatial distance between the ball and the racket is less than the preset distance threshold, and the hitting posture detection result is "yes," is that video frame determined to be a hitting frame. By cross-validating three conditions—sudden changes in motion speed, proximity of the ball and racket, and the player's hitting posture—non-hitting scenarios are precisely filtered out (such as speed changes caused by the ball landing, hitting postures during a racket swing, and proximity states where the ball and racket are not in contact). This avoids misjudgments caused by single-condition judgments and ensures that only the video frame corresponding to the instant the ball is subjected to the force of the racket is marked as a hitting frame. This provides an accurate and reliable benchmark for the timing alignment of subsequent hitting effects (synchronized with the hitting frames), ensuring visual consistency between the effects and the actual hitting action.

[0090] Step 103: Determine the first position coordinates of the ball in the hitting frame as the hitting position coordinates in the hitting frame, and generate a hitting effect layer based on the hitting position coordinates and the target hitting effect data.

[0091] Specifically, the hitting position coordinates, or the first position coordinates of the ball in the hitting frame, are the precise spatial positioning points where the ball contacts the racket at the moment of impact, serving as the core spatial reference for overlaying hitting effects. The target hitting effect data consists of pre-configured or user-selected hitting effect resources, which can be background-free video clips with an alpha channel (e.g., 1-3 seconds in length, corresponding to 30-90 frames). Users can change the target hitting effect data by modifying configuration parameters; it is the basic material for generating effects. Different hitting effect data produce different visual effects; refer to [reference / reference]. Figures 2 to 4 , Figure 2 This is a diagram illustrating the initial stage of a shot effect. Figure 3 This is a diagram illustrating the peak phase of a shot effect. Figure 4 This is a diagram illustrating the dissipation phase of a shot effect. Figures 2 to 4 It is a process in which a sphere of light grows from small to large and then back to small, and its brightness increases from weak to strong and then back to weak, with a process of expansion and then dissipation.

[0092] In some embodiments, the first position coordinates of the ball (i.e., the center pixel coordinates) detected in the hitting frame are first directly determined as the hitting position coordinates, serving as the spatial reference point for special effects rendering. Subsequently, target hitting effect data is loaded according to user selection or preset configuration. Using the hitting position coordinates as the central anchor point, the special effects material (target hitting effect data) is scaled at the original ratio or adaptively and drawn frame by frame onto a transparent layer with the same resolution as the original ball sports video stream. This forms a hitting effect layer that contains visual elements only at the moment of hitting and in several subsequent video frames, achieving precise alignment between the special effects and the hitting point, automatically generating special effects, laying the foundation for efficient compositing in the future, and supporting flexible changes to the special effects style, thus laying the foundation for efficient compositing in the future.

[0093] Optionally, before generating the shot effect layer based on the shot position coordinates and target shot effect data, the following steps may also be included:

[0094] In response to the user's selection of special effects data, the target shot effect data is determined from multiple candidate shot effect data. The target shot effect data is a shot effect video clip that includes an alpha channel.

[0095] Specifically, the effect data selection operation refers to the user's action of choosing a shot effect from multiple preset options through a graphical interface (such as clicking, swiping, or drop-down menus). Candidate shot effect data consists of various shot effect resource libraries built into the system or provided in the cloud, such as "flame explosion," "blue halo," and "particle splash," each of which is an independent resource. The target shot effect data is the effect resource finally selected by the user, serving as the visual template for the shot effect used in this processing. Shot effect video clips containing an alpha channel refer to video files with an alpha channel. These video clips can be in Web Media format, Apple ProRes 4444 format, or Portable Network Graphics (PNG) format, with a transparent background, retaining only the main effect element, making it easy to overlay onto any background.

[0096] For example, before generating the shot effect layer, the system can provide a visual interactive interface to display multiple pre-stored candidate shot effect data, and simultaneously present dynamic previews of various candidate shot effect data, making it convenient for users to select intuitively. When a user selects a specific effect by clicking on the preview image, checking the list, or other operations, the system responds to the selection operation in real time, loads the corresponding effect resource from the local resource package or remote server, and verifies whether it is a video clip containing a transparency channel (e.g., checking whether the alpha channel exists). After confirmation, the resource is identified as the target shot effect data and stored in a temporary access directory, providing accurate and compliant material support for subsequent generation of effect layers based on the shot position coordinates, and achieving rapid adaptation to personalized effect requirements.

[0097] Optionally, the steps for generating a shot effect layer based on the shot position coordinates and target shot effect data include:

[0098] Using the ball-hitting position coordinates as the central anchor point, each frame of the ball-hitting effect video clip is sequentially superimposed onto the continuous video frames starting from the ball-hitting frame in the ball sports video stream, forming a ball-hitting effect layer.

[0099] Specifically, consecutive video frames starting from the hit frame refer to a time window that extends for N frames (e.g., 30 frames) from the determined hit frame, corresponding to the duration of the special effects playback.

[0100] For example, the system uses the ball-hitting position coordinates (x, y) determined in the hitting frame as the center anchor point and loads the target hitting effect data (hitting effect video clip). Then, starting from the hitting frame (let's call it frame t), the system sequentially takes the 1st frame, the 2nd frame, ... the Nth frame of the hitting effect video clip, and establishes a one-to-one correspondence between the frame order of the hitting effect video clip and the consecutive video frames in the ball sports video stream starting from the hitting frame—the 1st frame of the hitting effect video clip is precisely aligned with the hitting frame, the 2nd frame is aligned with the next frame after the hitting frame, and so on, ensuring that the timing of the effects is synchronized with the video progress after the ball hit. During the overlay process, the transparent channel of the hitting effect video clip is used to shield invalid background areas to avoid obscuring key details such as the ball and the player in the original ball sports video stream. Finally, all the aligned and overlaid effect frames are combined in their original timing to form an independent hitting effect layer, which can be directly used for compositing with the original video stream. By employing a combination of spatial anchoring and temporal alignment in the overlay logic, the system ensures that the hitting effects and the actual hitting action are precisely aligned spatially and presented sequentially in time, preventing effect misalignment or temporal disconnection. Simultaneously, generating independent hitting effect layers preserves the original video data while flexibly adapting to the compositing needs with other layers, simplifying subsequent processes. Furthermore, the use of alpha channels allows for a natural blending of the effects with the original video, enhancing the visual experience.

[0101] Step 104: Combine the hitting effect layer with the ball movement video stream to generate the first effect video with hitting effects.

[0102] Specifically, the hitting effect layer is precisely aligned with the ball sports video stream frame by frame to ensure that the effect frame matches the corresponding video frame (hitting frame and subsequent consecutive frames). At the same time, a semi-transparent overlay algorithm is used to overlay the hitting effect layer onto the corresponding video frame. Finally, all overlaid frame sequences are encoded and encapsulated according to the resolution and frame rate parameters of the original ball sports video stream to generate a first effect video with hitting effects and a playback rhythm consistent with the original video, thus achieving an organic fusion of the hitting effect and the original video stream.

[0103] Optionally, the steps of compositing the hit effect layer with the ball movement video stream to generate a first effects video with hit effects include:

[0104] Using the Alpha channel blending algorithm, the ball-hitting effect layer is blended and superimposed with the corresponding video frames in the ball movement video stream to obtain the first effect video with ball-hitting effects.

[0105] Specifically, the Alpha channel blending algorithm is an image processing algorithm that uses the image's Alpha channel (the channel that stores transparency information) to achieve pixel fusion between the foreground (the ball-hitting effect layer) and the background (the video stream of ball sports). The Alpha value (0-255) controls the transparency of the foreground. The higher the Alpha value, the clearer the effect, and the lower the value, the more transparent it is, which can achieve a natural transition between the effect and the original video.

[0106] For example, firstly, the frame timing consistency between the ball-hitting effect layer and the ball game video stream is verified to ensure that each frame of the ball-hitting effect layer precisely matches the corresponding ball-hitting frame and subsequent consecutive frames in the ball game video stream, without frame misalignment or timing discrepancies. Then, a lightweight alpha channel blending algorithm is invoked to perform a blending operation on each pair of matched effect frames and video frames—based on the alpha channel data of the effect frame, the foreground (effect) weight and background (original video) weight of each pixel are calculated to ensure that the effect is clearly presented without obscuring key details in the original video. In the Alpha channel blending process, the Alpha channel blending algorithm automatically performs weighted summation of the pixel RGB values ​​of the effect frames and video frames to generate fused pixel data, achieving a natural transition between the effect and the original video. Finally, after all corresponding frames have been blended and superimposed, the blended frame sequence is uniformly encoded and encapsulated according to the original resolution, frame rate, encoding format, and other parameters of the ball sports video stream, generating a first effect video with the same playback rhythm and picture quality as the original video, and containing a precise ball-hitting effect, which can be used directly without manual subsequent adjustments.

[0107] Based on the ball sports video effects generation method provided in this application, a video stream containing a ball is obtained. Hitting events are detected in each video frame of the video stream to obtain the hitting frame and the first position coordinates of the ball within the hitting frame, replacing the manual frame-by-frame judgment and trajectory drawing of existing methods. Next, the first position coordinates of the ball within the hitting frame are determined as the hitting position coordinates in the hitting frame. Based on the hitting position coordinates and target hitting effect data, a hitting effect layer is generated. Finally, the hitting effect layer is synthesized with the ball sports video stream to generate a first effects video with hitting effects. This solves the technical problem of low generation efficiency and difficulty in real-time application of existing ball sports video effects generation methods. While ensuring the accuracy of core tracking and hitting event recognition, the processing efficiency is improved to real-time level, allowing ordinary users to easily create professional-looking effects videos for application in live streaming, mobile devices, and a wide range of amateur scenarios.

[0108] In some embodiments, reference Figure 5 Before the step of compositing the ball-hitting effect layer with the ball movement video stream, the following steps are also included:

[0109] Step 501: Take the video frames with odd frame numbers in the ball sports video stream as the first target frames, perform target detection on the players in each first target frame based on the target detection algorithm, and generate the bounding box coordinates of the first player corresponding to each first target frame;

[0110] Step 502: In the video stream of ball sports, the video frames with even frame numbers are taken as the second target frames. Based on the first player bounding box coordinates corresponding to each first target frame, the second player bounding box coordinates corresponding to each second target frame are calculated by linear interpolation.

[0111] Step 503: Generate a player frame effect layer based on the player's first or second player bounding box coordinates in each video frame;

[0112] Step 504: Composite the player frame effect layer, the ball hitting effect layer, and the ball sports video stream to generate a second effect video with player frame effects and ball hitting effects.

[0113] Specifically, the target detection algorithm can be a lightweight target detection model adapted to edge devices, such as the quantized YOLOv8n. After full integer quantization, it has the fewest parameters and is used to quickly identify the player's position in the first target frame, outputting the bounding box coordinates. The first player bounding box coordinates are the two-dimensional rectangular coordinates of the player in the first target frame, which may include the x / y coordinates of the top left corner and the width and height of the box. The first player bounding box coordinates are the core data describing the player's spatial position in the image. The second player bounding box coordinates are the rectangular coordinates of the player in the second target frame obtained through linear interpolation, and are consistent with the format of the first player bounding box coordinates. The player box effect layer is an independent image layer formed by drawing specific shapes (such as ellipses, rectangles, etc.) based on the player bounding box coordinates of each frame. It is synchronized with the original video stream in time and can be overlaid with other effect layers. The second effect video is a video that integrates the original ball movement footage, player box effects, and shot effects, combining position markers (player box effects) and shot highlight effects, providing richer visual information.

[0114] As an example, the video frames in the ball game video stream are filtered by frame number. Video frames with odd-numbered frame numbers are designated as the first target frames. A lightweight and optimized target detection algorithm (such as the quantized YOLOv8n) is used to identify player targets in the first target frames frame by frame, and the bounding box coordinates of the first player corresponding to each first target frame are output. Subsequently, video frames with even-numbered frame numbers are designated as the second target frames. The bounding box coordinates of the first player in the two adjacent first target frames of each second target frame are extracted. Based on a linear interpolation algorithm, coordinate weights are assigned according to the frame number interval to calculate the bounding box coordinates of the second player corresponding to each second target frame, ensuring that the player position data of the entire frame sequence is continuous and without gaps. Next, based on the first or second player bounding box coordinates of all video frames, player frame graphics of a specific style are drawn according to preset rules, such as rectangular marker frames and elliptical positioning frames on the soles of feet. Transparency channels are added to the graphics to avoid obscuring the original video image. These are combined in frame sequence to form independent player frame effect layers. Finally, the player frame effect layer, the generated ball-hitting effect layer, and the ball movement video stream are precisely aligned in frame time. The Alpha channel mixing algorithm is used to perform weighted fusion operations on the three types of frame data to ensure visual naturalness. After all corresponding frames are mixed, they are encapsulated according to the resolution, frame rate, and encoding format of the original video to generate a second effect video that includes both player position markers and ball-hitting highlight effects.

[0115] In some embodiments, the step of generating a player frame effect layer based on the player's first or second player bounding box coordinates in each video frame includes:

[0116] For video frames with odd frame numbers, draw a special effect graphic of a preset shape with the midpoint of the bottom edge of the corresponding first player's bounding box as the geometric center, the length of the bottom edge of the first player's bounding box as the major axis, and one-third of the length of the bottom edge of the first player's bounding box as the minor axis, with the minor axis perpendicular to the major axis.

[0117] For video frames with even frame numbers, draw a special effect graphic of a preset shape with the midpoint of the bottom edge of the corresponding second player's bounding box as the geometric center, the length of the bottom edge of the second player's bounding box as the major axis, and one-third of the length of the bottom edge of the second player's bounding box as the minor axis, with the minor axis perpendicular to the major axis.

[0118] Combine the special effects graphics corresponding to all video frames into a player frame effect layer.

[0119] Specifically, the midpoint of the bottom edge is the center pixel coordinate of the bottom edge of the player's bounding box, serving as the geometric center of the effect graphic to ensure the graphic is anchored in the player's foot area, conforming to the movement trajectory. The major and minor axes define the two principal axes of the effect graphic. The major axis is equal to the length of the bottom edge of the player's bounding box, and the minor axis is one-third of the bottom edge length and perpendicular to the major axis. Predefined shapes are predefined effect graphic styles, such as ellipses and rectangles, used for visually identifying players. (Refer to...) Figure 6 , Figure 6 It contains elliptical special effects graphics.

[0120] As an example, for video frames with odd frame numbers in a ball game video stream, first parse the pixel coordinates of the left and right endpoints of the bottom edge of the first player's bounding box (let's call them (x1, y1) and (x2, y2), which are the ordinates of the bottom edge of the bounding box). Calculate (x1+x2) / 2 to get the midpoint of the bottom edge, and determine this point as the geometric center of the special effects graphic. At the same time, extract the length of the bottom edge of the first player's bounding box (|x2-x1|), and use it directly as the major axis of the special effects graphic. Calculate the minor axis by "bottom edge length × 1 / 3", and set the minor axis to be perpendicular to the major axis (i.e., the major axis is parallel to the horizontal direction of the video frame, and the minor axis is perpendicular to the horizontal direction). Then, draw the special effects graphic according to a preset shape (such as an ellipse). The edges of the special effects graphic can be softly feathered to improve visual comfort. For video frames with even frame numbers, the drawing logic for odd frames is completely reused: the midpoint of the bottom edge is calculated based on the coordinates of the second player's bounding box as the geometric center, with the length of its bottom edge as the major axis and one-third of the bottom edge as the minor axis, and the axis directions are kept perpendicular, to draw a preset shape effect graphic consistent with the odd frame, ensuring that the effects of odd and even frames are completely consistent in size and style, avoiding visual breaks during movement. Finally, the effect graphics drawn for all video frames (including odd and even frames) are sorted sequentially according to the frame time sequence of the original video stream, and an alpha channel with preset transparency is added to each graphic (to avoid obscuring core elements such as players and balls in the original image). After frame sequence verification to ensure there are no misalignments, they are combined to form an independent player frame effect layer. The time sequence of this player frame effect layer is completely synchronized with the original video stream and can be directly used for subsequent mixing and compositing operations with the ball-hitting effect layer and the original video stream. By using the midpoint of the bottom edge as the anchor point and combining it with an adaptive axis length design, the generated special effects graphics are more in line with the actual standing posture of a player. At the same time, relying on the odd and even frame differentiation processing strategy, the computational overhead is effectively reduced while ensuring continuous display of the entire frame, making it suitable for mobile devices or real-time video enhancement application scenarios.

[0121] In some embodiments, reference Figure 7 Before the step of compositing the player frame effect layer, the shot effect layer, and the ball movement video stream, the following steps are also included:

[0122] Step 701: In response to the user's selection operation, select the target effect image from the candidate effect images;

[0123] Step 702: Using the first position coordinates of the sphere in each video frame as the center anchor point, the target effect image is sequentially superimposed onto each video frame in the ball motion video stream to form a sphere trajectory effect layer.

[0124] Step 703: Combine the player frame effect layer, ball trajectory effect layer, and shot effect layer with the ball sports video stream to generate a third effect video with player frame effect, ball trajectory effect, and shot effect.

[0125] Specifically, the user selection operation refers to the user's action of selecting an effect style through the system's visual interactive interface (such as an effect preview list or drop-down menu), which is the interactive trigger condition determined by the target effect image. Candidate effect images are a diverse collection of pre-stored visual materials of ball trajectories, including different styles such as circles, lights, and dashed lines. The ball trajectory effect layer is an independent image layer formed by overlaying the target effect image frame by frame onto the corresponding video frame's ball position, arranged sequentially by frame, which can intuitively present the ball's movement path. The third effect video is the final video product that integrates the original ball movement video stream, player frame effect layer, shot effect layer, and ball trajectory effect layer, featuring three effects: player frame effect (continuous marker), ball trajectory effect (full-time follow), and shot effect (short-term highlight), providing rich visual information. (Reference) Figure 8 and Figure 9 , Figure 8 and Figure 9 For video frames of a third effect video with a sphere trajectory effect overlaid with a circle effect, refer to Figure 10 and Figure 11 , Figure 10 and Figure 11 The video frames are the third effect video with a sphere trajectory effect overlaid with a bright light effect.

[0126] As an example, in response to user interaction (i.e., user selection), the system selects a target effect image from a pre-defined library of candidate effect images (such as glowing dots, flame icons, circles, particle trails, etc., PNG images with alpha channels). Then, based on the first position coordinates of the sphere in each video frame, and using these coordinates as the center anchor point to ensure the trajectory perfectly matches the sphere's actual movement path, the target effect image is scaled at its original ratio or adaptively and superimposed frame-by-frame onto a transparent canvas with the same resolution as the original video. This forms a sphere trajectory effect layer covering the entire video stream duration. This layer displays only a lightweight graphic at the sphere's current position in each frame, resulting in a smooth motion trajectory during continuous playback. Finally, in the compositing stage, the original ball movement video stream, the player frame effect layer (identifying player positions), the sphere trajectory effect layer (following the ball's movement), and the shot effect layer (highlighting the moment of impact) are layered with alpha blending according to rendering priority, with the shot effect layer placed on top to ensure visual prominence. All synthesized frames are encoded and output to generate a third-effect video that simultaneously includes player identification, ball trajectory, and shot highlight. By adding ball trajectory effects, it complements player frame effects and shot effects, intuitively presenting the ball's movement path in the video stream of ball sports. Furthermore, it allows users to choose ball trajectory styles to adapt to the visual needs of different scenarios, enhancing the flexibility and practicality of the ball trajectory effects.

[0127] In some embodiments, prior to the step of compositing the ball-hitting effect layer with the ball sports video stream, the method further includes:

[0128] In response to the user's selection, select the target effect image from the candidate effect images;

[0129] Using the first position coordinates of the sphere in each video frame as the central anchor point, the target effect image is sequentially superimposed onto each video frame in the sphere's motion video stream to form a sphere trajectory effect layer;

[0130] The ball trajectory effect layer and the ball hit effect layer are composited with the ball movement video stream to generate a fourth effect video with ball trajectory and ball hit effects.

[0131] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for generating special effects for ball sports videos in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0132] This application also provides a device for generating video special effects for ball sports. Please refer to [link / reference]. Figure 12 The device for generating special effects for ball sports videos includes:

[0133] The video acquisition module 1201 is used to acquire video streams of ball sports, which include a ball.

[0134] The ball-hitting event detection module 1202 is used to detect ball-hitting events in each video frame of the ball sports video stream, and obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame.

[0135] The special effects generation module 1203 is used to determine the first position coordinates of the ball in the hitting frame as the hitting position coordinates in the hitting frame, and generate a hitting effect layer based on the hitting position coordinates and the target hitting effect data;

[0136] Compositing module 1204 is used to composite the ball-hitting effect layer with the ball movement video stream to generate a first effect video with ball-hitting effects.

[0137] The ball sports video special effects generation device provided in this application, employing the ball sports video special effects generation method in the above embodiments, can solve the technical problem of low generation efficiency and difficulty in real-time application of existing ball sports video special effects generation methods. Compared with the prior art, the beneficial effects of the ball sports video special effects generation device provided in this application are the same as those of the ball sports video special effects generation method provided in the above embodiments, and other technical features in the ball sports video special effects generation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0138] This application provides a ball sports video special effects generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the ball sports video special effects generation method in the above embodiment 1.

[0139] The following is for reference. Figure 13 The diagram illustrates a structural schematic of a ball sports video effects generation device suitable for implementing embodiments of this application. The ball sports video effects generation device in this application may include, but is not limited to, mobile terminals such as laptops, tablets (Portable Application Description, PADs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 13 The ball sports video effects generation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 13As shown, the ball sports video effects generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the ball sports video effects generation device. The processing unit 1001, the ROM 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the ball sports video effects generation device to communicate wirelessly or wiredly with other devices to exchange data. Although a ball sports video effects generation device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0141] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0142] The ball sports video effects generation device provided in this application, employing the ball sports video effects generation method described in the above embodiments, can solve the technical problem of low generation efficiency and difficulty in real-time application of existing ball sports video effects generation methods. Compared with the prior art, the beneficial effects of the ball sports video effects generation device provided in this application are the same as those of the ball sports video effects generation method provided in the above embodiments, and other technical features of this ball sports video effects generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0144] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0145] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the ball sports video effects generation method in the above embodiments.

[0146] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0147] The aforementioned computer-readable storage medium may be included in the ball sports video effects generation device; or it may exist independently and not assembled into the ball sports video effects generation device.

[0148] The aforementioned computer-readable storage medium carries one or more programs. When the one or more programs are executed by the ball sports video effects generation device, the ball sports video effects generation device causes the following: it acquires a ball sports video stream, the ball sports video stream containing a ball; it performs ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame; it determines the first position coordinates of the ball in the ball-hitting frame as the ball-hitting position coordinates in the ball-hitting frame; based on the ball-hitting position coordinates and target ball-hitting effect data, it generates a ball-hitting effect layer; and it composites the ball-hitting effect layer with the ball sports video stream to generate a first effects video with ball-hitting effects.

[0149] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0151] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0152] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for generating special effects for ball sports videos. This solves the technical problem in the prior art where the special effects generation methods for ball sports videos have low generation efficiency and are difficult to apply in real time. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the ball sports video special effects generation method provided in the above embodiments, and will not be repeated here.

[0153] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for generating special effects for ball sports videos.

[0154] The computer program product provided in this application can solve the technical problem that the existing methods for generating special effects for ball sports videos have low generation efficiency and are difficult to apply in real time. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the ball sports video special effects generation method provided in the above embodiments, and will not be repeated here.

[0155] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating special effects in ball sports videos, characterized in that, The method for generating special effects for ball sports videos includes: Acquire a video stream of a ball sport, the video stream containing a ball, a player, and a racket; Perform ball-hitting event detection on each video frame in the ball sports video stream to obtain the ball-hitting frame in the video frame and the first position coordinates of the ball in the ball-hitting frame; The first position coordinates of the ball in the hitting frame are determined as the hitting position coordinates in the hitting frame. Based on the hitting position coordinates and the target hitting effect data, a hitting effect layer is generated. The video frames with odd frame numbers in the ball sports video stream are taken as the first target frames. The players in each first target frame are detected based on the target detection algorithm, and the bounding box coordinates of the first player corresponding to each first target frame are generated. The video frames with even-numbered frame sequences in the ball sports video stream are taken as the second target frames. Based on the first player bounding box coordinates corresponding to each first target frame, the second player bounding box coordinates corresponding to each second target frame are calculated by linear interpolation. For video frames with odd frame numbers, a special effect graphic of a preset shape is drawn with the midpoint of the bottom edge of the corresponding first player bounding box as the geometric center, the length of the bottom edge of the first player bounding box as the major axis, and one-third of the length of the bottom edge of the first player bounding box as the minor axis, with the minor axis perpendicular to the major axis. For video frames with even frame numbers, the special effect graphic of the preset shape is drawn with the midpoint of the bottom edge of the corresponding second player bounding box as the geometric center, the length of the bottom edge of the second player bounding box as the major axis, and one-third of the length of the bottom edge of the second player bounding box as the minor axis, with the minor axis perpendicular to the major axis. Combine the special effects graphics corresponding to all video frames into a player frame effect layer; The player frame effect layer, the ball-hitting effect layer, and the ball game video stream are combined to generate a second effect video with player frame effect and ball-hitting effect; The ball-hitting effect layer is combined with the ball-playing video stream to generate a first effect video with ball-hitting effects; Prior to the step of generating the hitting effect layer based on the hitting position coordinates and the target hitting effect data, the method further includes: In response to the user's selection of special effects data, the target hitting special effects data is determined from multiple candidate hitting special effects data, wherein the target hitting special effects data is a video clip of hitting special effects containing a transparency channel; The step of generating a shot effect layer based on the shot position coordinates and target shot effect data includes: Using the ball-hitting position coordinates as the central anchor point, each frame of the ball-hitting effect video clip is sequentially superimposed onto the continuous video frames starting from the ball-hitting frame in the ball sports video stream to form the ball-hitting effect layer.

2. The method for generating special effects for ball sports videos as described in claim 1, characterized in that, The step of detecting ball-hitting events in each video frame of the ball sports video stream to obtain the ball-hitting frame and the first position coordinates of the ball in the ball-hitting frame includes: The sphere is tracked and detected to generate the first position coordinates of the sphere in each video frame of the ball motion video stream and the motion speed of the sphere in each video frame; Target detection is performed on the racket to generate the second position coordinates of the racket in each of the video frames; The player's hitting posture is detected, and the hitting posture detection results of the player in each of the video frames are generated. The hitting posture detection results are used to indicate whether the player's posture is a hitting posture. The hitting frames in the ball game video stream are determined based on the first position coordinates, the second position coordinates, the movement speed, and the hitting posture detection results of each video frame.

3. The method for generating special effects for ball sports videos as described in claim 2, characterized in that, The step of determining the hitting frame in the ball game video stream based on the first position coordinates, the second position coordinates, the movement speed, and the hitting posture detection results of each video frame includes: Based on the motion speed of each video frame, calculate the change in the motion speed of the sphere between each video frame and the previous video frame; For each video frame, calculate the spatial distance between the first position coordinate and the second position coordinate of the video frame; For each video frame, determine whether the hitting posture detection result indicates that the player's posture is a hitting posture; If the absolute value of the change in motion speed is greater than a preset speed change threshold, the spatial distance is less than a preset distance threshold, and the ball-hitting posture detection result is yes, then the video frame is determined as the ball-hitting frame.

4. The method for generating special effects for ball sports videos as described in claim 1, characterized in that, The step of combining the ball-hitting effect layer with the ball sports video stream to generate a first effect video with ball-hitting effects includes: Using an Alpha channel blending algorithm, the ball-hitting effect layer is blended and superimposed with the corresponding video frame in the ball game video stream to obtain the first effect video with the ball-hitting effect.

5. The method for generating special effects for ball sports videos as described in claim 1, characterized in that, Before the step of compositing the player frame effect layer, the ball-hitting effect layer, and the ball game video stream, the method further includes: In response to the user's selection, select the target effect image from the candidate effect images; Using the first position coordinates of the sphere in each of the video frames as the center anchor point, the target effect image is sequentially superimposed onto each of the video frames in the ball motion video stream to form a sphere trajectory effect layer; The player frame effect layer, the ball trajectory effect layer, and the shot effect layer are combined with the ball game video stream to generate a third effect video with the player frame effect, the ball trajectory effect, and the shot effect.

6. A device for generating special effects for ball sports videos, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for generating video effects for ball sports as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for generating special effects for ball sports videos as described in any one of claims 1 to 5.

Images