Method and system for action intelligence analysis

Abstract

This application discloses a method and system for intelligent motion analysis, belonging to the field of artificial intelligence technology. The method includes: acquiring user videos and template videos; detecting key points of the human skeleton in both videos using a lightweight detection model to obtain dual-channel labeled videos with coordinate information of the key points; performing temporal and spatial alignment on the dual-channel labeled videos; processing the aligned dual-channel labeled videos according to a selected comparison mode to generate standardized comparison videos; and inputting the standardized comparison videos and selected prompt word templates into a pre-trained multimodal large model to obtain analysis, explanation, and improvement suggestions. The lightweight detection model achieves accurate and efficient extraction of motion features and standardized video comparison, while the large model performs professional analysis and natural language explanation of the multimodal videos, improving detection accuracy, analysis efficiency, and the professionalism of the explanation.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and specifically relates to a method and system for intelligent motion analysis. Background Technology

[0002] Badminton demands a high degree of precision and coordination in movements. Precise movement analysis, standardized comparison, and professional instruction are core elements for improving training effectiveness. Traditional badminton movement analysis relies on manual observation by coaches, which suffers from strong subjectivity, inconsistent evaluation standards, and an inability to quantify the results. Furthermore, professional coaching is costly and difficult to implement in real-time and consistently.

[0003] Existing badminton motion analysis systems are mostly based on a single small model to achieve posture detection and modeling. Although they can complete basic motion simulation, they can only provide visualization results and cannot output professional and detailed motion analysis explanations and targeted improvement suggestions. Trainees find it difficult to understand the core issues and correction methods of motion deviations. At the same time, the single small model has limited feature extraction and analysis capabilities in complex scenarios, and its accuracy in recognizing motion details and judging deviations is insufficient, which cannot meet the needs of refined training.

[0004] Therefore, there is an urgent need for a motion intelligent analysis system to improve detection accuracy, analysis efficiency, and the professionalism of explanation. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a motion intelligent analysis method and system to improve detection accuracy, analysis efficiency, and the professionalism of explanation.

[0006] A motion intelligence analysis method, comprising: Obtain the user video to be analyzed, template video, comparison mode, and selected prompt word template; A lightweight detection model is used to detect key points of the human skeleton in both the user video and the template video, resulting in user videos and template videos with detection annotations, wherein the detection annotations include the coordinates of the skeleton key points. The user video and the template video with detection annotations are subjected to temporal and spatial alignment to obtain aligned dual-channel labeled video. Based on the acquired comparison pattern, the aligned dual-channel labeled video is processed to generate a standardized comparison video; The standardized comparison video and the selected prompt word template are input into the pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user's video.

[0007] Furthermore, the motion intelligent analysis method is applied to the analysis of ball-related movements that require the use of a racket; the detection annotation also includes: racket position; After detecting human skeleton key point information in both the user video and the template video using a lightweight detection model to obtain user videos and template videos with detection annotations, and before performing temporal and spatial alignment processing on the user videos and template videos with detection annotations to obtain aligned dual-channel labeled videos, the method further includes: A lightweight detection model is used to detect racket position information in both the user video and the template video, resulting in user videos and template videos with detection annotations.

[0008] Furthermore, the lightweight detection model includes a pose detection model and an object detection model; Human skeleton key point information is detected in both the user video and the template video using a lightweight detection model, including: A sampling strategy is adopted to sample the frame images of the user video and the template video every N frames to obtain sampled frame images, where N is an integer greater than or equal to 2; The pose detection model is used to detect key points of the human skeleton in the sampled frame images of the user video and the sampled frame images of the template video, respectively. A lightweight detection model is used to detect racket position information in both the user video and the template video, including: The target detection model is used to perform racket target recognition on the sampled frame images of the user video and the sampled frame images of the template video, respectively.

[0009] Furthermore, after detecting key points of the human skeleton and racket position information in both user videos and template videos using a lightweight detection model to obtain user videos and template videos with detection annotations, before performing temporal and spatial alignment processing on the user videos and template videos with detection annotations, the following steps are also included: Detecting the user's hand holding the camera based on user videos with detection annotations; Determine if the user's hand holding the clapper matches the hand holding the clapper in the template video; Temporal and spatial alignment processing is performed on user videos and template videos with detection annotations, including: When the user's hand holding the camera matches the hand holding the camera in the template video, temporal and spatial alignment processing is performed on the user video with detection labels and the template video with detection labels. When the user's hand holding the camera does not match the hand holding the camera in the template video, the template video is mirrored, and temporal and spatial alignment is performed on the user video with detection labels and the mirrored template video with detection labels.

[0010] Furthermore, user hand detection is performed based on user videos with detection annotations, including: Extract the racket center coordinates, wrist key point coordinates, and shoulder key point coordinates from user videos with detection annotations, and calculate the distances from the racket center to the left wrist key point and the right wrist key point. Based on the distance from the center of the racket to the key points of the left and right wrists, the racket-holding hand side is initially determined. By combining the coordinates of key points on the shoulder, the initially determined racket-holding hand side is calibrated to obtain the user's racket-holding type.

[0011] Furthermore, based on the distances from the center of the racket to the key points of the left and right wrists, the preliminary determination of the racket-holding hand side includes: Calculate the Euclidean distance from the center of the racket to the key points of the left and right wrists and calculate the variance for each. The side with the smaller variance is used to initially determine the racket-holding hand side.

[0012] Furthermore, temporal and spatial alignment processing is performed on user videos with detection annotations and template videos with detection annotations, including: When the comparison mode is the intuitive comparison mode, the frame rate of the template video with detection labels is stretched to match the frame rate of the user video with detection labels, and the playback speed of the user video is adjusted to the preset slow motion speed to achieve time sequence alignment of the first and last frames. When the comparison mode is the overlap comparison mode, the frame number of the template video with detection labels is stretched to match the frame number of the user video with detection labels, and the playback speed of the user video is adjusted to the preset slow motion speed to achieve temporal alignment of the first and last frames. Based on the shoulder node, the shoulder spacing ratio of the temporally aligned user video and the template video is calculated. The temporally aligned template video is scaled based on the shoulder spacing ratio, and spatial posture alignment is achieved through shoulder coordinate smoothing and translation transformation.

[0013] Furthermore, the comparison modes include intuitive comparison mode and overlapping comparison mode; The aligned dual-channel labeled video is processed according to the acquired comparison pattern to generate a standardized comparison video, including: If it is an intuitive comparison mode, the aligned dual-channel labeled video will be split into left and right screens and spliced ​​together to generate an independent synchronous comparison video; If the overlay comparison mode is selected, the user video is sharpened, the template video is Gaussian blurred, and the processed template video is overlaid on the processed user video with a preset transparency to generate a fused comparison video.

[0014] Furthermore, the prompt word template is a user-defined template, which is used to specify the focus of the analysis. The focus of the analysis includes at least one of the following: differences in posture, trajectory, timing or normativity of the ankle joint, legs, waist and abdomen, arms, and racket posture.

[0015] A motion intelligence analysis device, comprising: The video acquisition module is used to acquire user videos to be analyzed, template videos, comparison patterns, and selected prompt word templates; The detection module is used to detect human skeleton key point information in both the user video and the template video using a lightweight detection model, so as to obtain the user video with detection annotation and the template video with detection annotation, wherein the detection annotation includes the coordinates of the skeleton key points; The video processing module is used to perform temporal and spatial alignment processing on user videos and template videos with detection annotations to obtain aligned dual-channel labeled videos. Based on the acquired comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos. The large model inference module is used to input the standardized comparison video and the selected prompt word template into the pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user video.

[0016] An intelligent motion analysis system includes a front-end interaction module, a lightweight model detection library, a video processing module, a multimodal large model, and a prompt word template library; The front-end interaction module is used for user video upload, standard action video selection, comparison mode selection, prompt word template selection, and to integrate and display comparison videos and analysis and explanation suggestions. The lightweight model detection library is used to perform human skeleton key point detection on the user video and the selected template video to obtain user video with detection annotations and template video with detection annotations, wherein the detection annotations include the coordinates of skeleton key points; The video processing module is used to perform temporal and spatial alignment processing on user videos with detection annotations and template videos with detection annotations to obtain aligned dual-channel labeled videos; according to the selected comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos; The multimodal large model is used to receive standardized comparison videos and selected prompt word templates, perform inference, and output analysis, explanations, and improvement suggestions for the user's videos. The prompt word template library is used to store and provide prompt word templates.

[0017] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0019] Compared with the prior art, this application has the following advantages: 1. The solution proposed in this application integrates the advantages of small and large models for intelligent motion analysis. It achieves accurate and efficient extraction of motion features and standardized video comparison through the small model, while completing professional analysis and natural language explanation of multimodal videos through the large model. Compared with single model analysis, it improves accuracy, analysis efficiency and professionalism of explanation, and provides an integrated intelligent analysis and guidance solution for badminton training.

[0020] 2. The solution proposed in this application integrates the advantages of MediaPipe and YOLOv8 small models in pose recognition and object detection with the capabilities of Qwen3-VL multimodal large models in video understanding and natural language analysis and explanation. This enables quantitative comparison, precise analysis, professional explanation, and targeted improvement suggestions for action videos. At the same time, through customized prompt word templates, it meets the explanation needs of different training scenarios, significantly improving the accuracy and professionalism of action video analysis. This provides integrated intelligent technical support for autonomous training, teaching guidance, and competition review of ball sports such as badminton and dance movements.

[0021] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A system architecture diagram according to an embodiment of this application is shown; Figure 2 A flowchart of a badminton motion intelligent analysis method according to an embodiment of this application is shown; Figure 3 A flowchart illustrating video slow-motion, stretching, and alignment processing according to an embodiment of this application is shown; Figure 4 A schematic diagram of skeleton key points and racket detection according to an embodiment of this application is shown; Figure 5 This document shows a visual comparison of the video output effect according to an embodiment of this application; Figure 6 The image shows the effect of overlay comparison video output according to an embodiment of this application; Figure 7 An example diagram of a prompt word template according to an embodiment of this application is shown; Figure 8 The diagram shows the output results according to an embodiment of this application; Figure 9 A schematic diagram of the structure of a badminton motion intelligent analysis device according to an embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The system of this invention adopts a "front-end and back-end separation + large and small model fusion" architecture, including four core parts: front-end interaction module, lightweight model (i.e. small model) detection library, video processing module, multimodal large model and prompt word template library. Through a two-step process of "small model feature extraction and video comparison → large model multimodal analysis and explanation", it realizes intelligent analysis of the entire process of action.

[0026] The front-end interaction module is used for user video upload, standard action video selection, comparison mode selection, prompt word template selection, and to integrate and display comparison videos and analysis and explanation suggestions. The lightweight model detection library is used to perform human skeleton key point detection on the user video and the selected template video to obtain user video with detection annotations and template video with detection annotations, wherein the detection annotations include the coordinates of skeleton key points; The video processing module is used to perform temporal and spatial alignment processing on user videos with detection annotations and template videos with detection annotations to obtain aligned dual-channel labeled videos; according to the selected comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos; The multimodal large model is used to receive standardized comparison videos and selected prompt word templates, perform inference, and output analysis, explanations, and improvement suggestions for the user's videos. The prompt word template library is used to store and provide prompt word templates.

[0027] The technical solution of this application can be applied to various motion analysis scenarios that require human motion capture, posture recognition, trajectory quantification, and standard comparison, covering multiple types of motion: including competitive movements of various ball sports such as basketball, football, badminton, and table tennis; dance performance movements such as street dance, classical dance, and folk dance; and can also be extended to professional sports movements such as track and field, gymnastics, ice and snow sports, water sports, and martial arts; fitness and shaping movements such as yoga, Pilates, and strength training; and rehabilitation medical limb movements such as postoperative rehabilitation, posture correction, elderly balance training, and children's sensory integration training; and is also suitable for physical performance movements such as opera movements, model catwalks, acrobatics, and stage play performances. In addition, it can also be used for human-machine operation actions such as industrial assembly line assembly, special operation control, vehicle driving, and VR somatosensory interaction; human behavior actions such as pedestrian gait, falls, and climbing; daily life limb actions such as infants learning to walk and daily living; and special professional action scenarios such as military and police tactical training, parachuting, delicate surgical operations, and horse racing gait recognition. By quantitatively analyzing the human joint angles, movement trajectories, force application sequences, and action standard in different scenarios, it can achieve functions such as action error correction, ability assessment, teaching guidance, risk warning, and posture correction.

[0028] The following example, using badminton motion analysis as an example, illustrates the solution of this application embodiment.

[0029] Figure 1 According to the system architecture diagram of this application embodiment, it is divided into a front-end and back-end separation architecture. The front-end is responsible for video uploading, comparison mode selection, prompt word template selection, result display and interaction; the back-end is the core processing layer, which integrates a small model detection library, video processing module, multimodal large model, prompt word template library, model file, template video library, temporary file storage, etc.; the data flow is as follows: user uploads video → small model completes detection and comparison video generation → comparison video inputs large model → calls customized prompt word template → large model outputs analysis and explanation suggestions → front-end visualization display, forming a complete closed loop of "detection-comparison-analysis-explanation".

[0030] Comparison Figure 1The complete business data flow is described as follows: Users submit local videos via drag-and-drop or click upload in the front-end video upload module. The files are first stored in the temporary storage area of ​​the data layer. Then, the required comparison mode is selected through the comparison mode selection module, and the video processing interface of the interface routing layer forwards the video to the file management module. The file management module pushes the user-uploaded video to the video processing pipeline, which completes the entire task orchestration process between the standard action template and the user-uploaded video. A lightweight small model first performs action detection and generates a comparison video, which is then sent to a large model to match and call a custom prompt word template. Finally, the large model outputs professional action analysis and guidance suggestions, which are fed back to the front-end layer. The front-end layer's result display module integrates the output of the large model with the comparison video for visualization rendering, thus completing the "detection-comparison-analysis-explanation" closed loop.

[0031] The following is combined Figure 2 The overall system architecture described above explains the intelligent analysis method for badminton movements in this application: Step 101: Obtain the user video to be analyzed, template video, comparison mode, and selected prompt word template.

[0032] Step 101 is data acquisition, which can also include data preprocessing. Step 101 uses user-uploaded badminton move MP4 videos and standard move template MP4 videos as initial input. OpenCV is used to extract frames from both videos, unify resolution and encoding formats, perform image noise reduction, and automatically remove blurry, overexposed, and distorted frames. The final output is a preprocessed standardized user video frame sequence and a standardized template video frame sequence, providing a stable and consistent input foundation for subsequent model detection. The comparison mode allows selection of video templates and comparison modes, such as jump smash templates and backhand transition templates. Comparison modes include direct comparison and overlapping comparison.

[0033] Step 102: Detect key points of the human skeleton and racket position information in the user video using a lightweight detection model, and perform human skeleton key point detection on the template video to obtain user video with detection annotations and template video with detection annotations.

[0034] The purpose of step 102 is to detect the skeleton and racket. The detection annotations for both user videos and template videos are the coordinates of key points of the skeleton and the position information of the racket. Specifically, the lightweight detection model includes a pose detection model for skeleton detection and a target detection model for racket recognition; At this point, step 102 takes the preprocessed dual-channel video frame sequence output from step 101 as input, and simultaneously loads the MediaPipe Pose Landmarker Lite model and the YOLOv8s model. It adopts a sampling strategy every N frames (e.g., 3 frames) to perform human skeleton key point detection (e.g., 17 points) and racket target recognition and localization in parallel. Finally, it outputs user video detection results and template video detection results containing the coordinates of skeleton key points and racket position information, thus completing the accurate extraction of the core visual features of badminton movements.

[0035] Step 103: Detect the user's hand holding the camera based on the user video with detection annotations; The user video detection result output in step 102 is used as the input for this step 103. The coordinates of the racket center and the coordinates of the two wrist key points are extracted from the user video detection result. The distance from the racket center to the left wrist key point and the right wrist key point is calculated. Based on the distance from the racket center to the left wrist key point and the right wrist key point, the racket-holding hand side is determined.

[0036] Specifically, the coordinates of the racket center and wrist key points can be extracted from the first few valid frames of the user's video detection results. The Euclidean distance from the racket center to the key points of the left and right wrists is calculated, and the variance is calculated for each. The side with the smaller variance is determined as the racket-holding hand side, providing a basis for subsequent video alignment. Here, the racket-holding hand side refers to the racket-holding type, which is divided into left-handed and right-handed rackets.

[0037] Step 104: Determine if the user's hand holding the clapper matches the hand holding the clapper in the template video. If not, proceed to step 105; if yes, proceed to step 106. Step 105: Mirror the template video; then proceed to step 106; Step 106: Perform temporal and spatial alignment processing on the user video with detection annotation and the template video with detection annotation, and then proceed to step 107; If you jump from step 105 to execute step 106, then step 106 is to perform temporal and spatial alignment processing on the user video with detection annotations and the mirrored template video with detection annotations.

[0038] The specific processing procedure for step 106 is as follows: When the comparison mode is the intuitive comparison mode, the frame rate of the template video with detection labels is stretched to match the frame rate of the user video with detection labels, and the playback speed of the user video is adjusted to the preset slow motion speed to achieve time sequence alignment of the first and last frames. When the obtained comparison mode is the overlapping comparison mode, the frame number of the template video with detection labels is stretched to match the frame number of the user video with detection labels, and the playback speed of the user video is adjusted to a preset slow motion speed to achieve temporal alignment of the first and last frames; taking the shoulder node as the reference, the shoulder spacing ratio of the temporally aligned user video and the template video is calculated, and the temporally aligned template video is scaled based on the shoulder spacing ratio, and spatial posture alignment is achieved through shoulder coordinate smoothing and translation transformation.

[0039] In short, step 106 involves taking the user video and template video with detection annotations as input, and using temporal stretching processing to match the frame number with the user video to achieve temporal alignment of the first and last frames. Temporal stretching processing refers to mapping the frame sequence of the input video to the target frame number / duration through uniform resampling along the time axis and linear inter-frame interpolation, thereby achieving temporal stretching of the video and ensuring strict alignment of the first and last frames.

[0040] Step 107: Based on the acquired comparison pattern, process the aligned dual-channel labeled video to generate a standardized comparison video; Step 107 takes the dual-channel labeled video, which has been aligned temporally and spatially in step 106 above, as input, and performs processing according to the obtained comparison mode. In the intuitive comparison mode, the two videos are spliced ​​together side by side to generate an independent synchronous comparison video. Figure 5 A visual comparison of the video output is provided. In the overlay comparison mode, the user's video is sharpened to enhance details, and the template video is Gaussian blurred to blur the background before being overlaid and blended with a preset transparency (e.g., 0.7) to generate a shoulder-aligned blended comparison video. The final output is a standardized comparison video conforming to the Qwen3-VL input specification, as shown below. Figure 6 The overlay comparison video output image shown is the standardized comparison video image, which can be directly used as input for the subsequent multimodal large model analysis module, completing the seamless connection between small model processing and large model analysis.

[0041] Figure 3 The flowcharts for video slow-motion, stretching and alignment, and video blending are provided, and the specific processing steps are as follows: Using user videos and template videos with detection annotations as input, the user video is slowed down to 0.2x speed and the frame rate is interpolated (e.g., 30 frames per second) to obtain the user slow-motion video. The template video is temporally stretched to match the frame rate (e.g., uniformly set to 30 frames per second) to the user video to achieve temporal alignment of the first and last frames. Then, the shoulder distance ratio is calculated based on the shoulder node to complete the scaling. The spatial posture is accurately aligned through shoulder coordinate smoothing, translation offset, and radial transformation. Finally, a dual-channel annotated video with temporal synchronization and spatial alignment is output. The user slow-motion video is sharpened, and the template video is blurred. The sharpened and blurred videos are then fused by weighted superposition, followed by brightness adjustment, to obtain the fused video.

[0042] Through steps 101 to 107 above, data acquisition and preprocessing, skeleton and racket detection, automatic detection of the racket-holding hand, video temporal and spatial processing, and comparison video generation were carried out, and finally, motion detection and comparison video generation based on a small model were completed.

[0043] After step 107 and before inputting the pre-trained multimodal large model, the generated standardized comparison video can be output and its playback controlled synchronously. At this time, the user can also view the comparison results and download the video.

[0044] Step 108: Input the standardized comparison video and the selected prompt word template into the pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user video.

[0045] The following explanation uses the Qwen3-VL multimodal large model as an example to illustrate step 108.

[0046] In step 108, Qwen3-VL receives intuitively compared dual videos or overlapping and fused videos generated by the small model to complete multimodal video understanding and professional analysis of badminton movements, specifically including: (1) Video input adaptation The comparison videos were converted to the required format according to the Qwen3-VL model input requirements to ensure the complete transmission of video frame sequences and annotation information, so that the large model could accurately identify features such as skeleton annotations, racket positions, and motion deviations in the videos.

[0047] (2) Model inference optimization For badminton scenarios, the inference parameters of the Qwen3-VL model are fine-tuned to improve the model's ability to understand badminton motion features (such as swing trajectory, joint angle, hitting point, and footwork), and to enhance the accuracy of comparative analysis between motion deviations and standard motions.

[0048] Step 109: Integrate and present the output from Step 107, along with the analysis, explanation, and improvement suggestions for the user video.

[0049] It should be noted that the core innovation of this application lies in the fusion method of large models such as Qwen3-VL and small models such as MediaPipe and YOLOv8. The small model extracts and standardizes badminton-specific features from the original video, transforming the unstructured original action video into a structured comparison video with detection annotations. The comparison video is then input into the large model, solving the problem of inaccurate feature extraction and analysis deviating from professional requirements when the large model directly analyzes the original video. At the same time, the efficient detection of the small model saves computing power for the analysis of the large model, improving the overall processing efficiency of the system. This achieves the complementary advantages of "small model accurately extracting features and large model performing professional analysis", significantly improving the accuracy of video analysis.

[0050] In addition, this application provides a customized cue word template library. This library is used to build basic cue word templates, on which users can customize the cue word content, freely specify the focus of analysis, and guide the large model to specifically analyze the degree of difference between their own movements and standard movements. Users can focus on the differences in posture, trajectory, timing, and standardization of single or multiple body parts such as the ankle, legs, waist and abdomen, arms, and racket posture. The core design principle of the templates is to combine the annotation features of the small model to guide the large model to focus on the user-specified body parts and movement dimensions for analysis. This template library includes: (1) Basic prompt word template It provides a general analysis framework on which users can add, delete, or modify prompts, specify the body parts and movement segments to be analyzed, and clarify the requirements for comparison and the level of detail in the explanation.

[0051] (2) Professional development and expansion Users can customize prompts to enhance comparative analysis of details such as swing trajectory, relative joint positions, hitting point, body coordination, and footwork, outputting quantitative differences in specified areas, reasons for deviations, and targeted improvement suggestions. Alternatively, users can directly use pre-organized prompt schemes categorized by key areas as needed. Figure 7 A sample image of a prompt word template is provided.

[0052] Finally, the large-scale model analyzes and explains the results, generating and outputting them. Based on the selected prompt word template, the large-scale model performs multimodal analysis on the comparison video and generates motion analysis explanations and improvement suggestions in natural language format. The content includes: overall motion evaluation, core deviation points (combined with skeleton / racket annotation features), analysis of the causes of deviations, targeted improvement methods, training suggestions, etc. The system integrates the comparison video and the text explanation suggestions generated by the large-scale model, displaying them in a unified front-end interface. It also supports downloading and saving the comparison video and explanation text, facilitating repeated viewing and training by trainees. Figure 8 A diagram showing the output results of the analysis and explanation of the large model is provided.

[0053] The beneficial effects of this application are as follows: (1) The integration and innovation of large and small models greatly improves the accuracy of analysis. This innovative approach combines the Qwen3-VL large model with MediaPipe and YOLOv8 small models. The small model enables accurate and efficient extraction of badminton-specific features and standardized video annotation, solving the feature recognition bias problem caused by the large model directly analyzing the original video. The large model leverages its multimodal understanding and professional analysis capabilities. The two complement each other, resulting in a significant improvement in the accuracy and professionalism of video analysis compared to a single model.

[0054] (2) The detection of the racket-holding hand is accurate and the movement alignment is highly precise. By combining the racket-to-wrist distance variance algorithm with shoulder coordinate calibration, the racket-holding hand can be automatically and accurately determined. Based on the shoulder node, the temporal and spatial dual alignment algorithm ensures accurate matching between the user and the template action, providing high-quality standardized input for large model analysis.

[0055] (3) Multi-faceted explanations to adapt to different training scenarios We have built a customized prompt template library. By changing the content of the templates, we can output analysis and explanations with different focuses, such as basic introduction, professional improvement, competition review, and single specialization, to meet the training needs of different groups such as beginners, intermediate trainees, and professional athletes.

[0056] (4) Achieve integrated intelligent guidance to fill the gap in professional coaches. This system integrates quantitative comparison of movements, visual feedback, professional analysis and explanation, and targeted improvement suggestions. Without the need for real-time guidance from professional coaches, trainees can independently complete movement analysis and correction, reducing training costs and achieving routine intelligent training guidance.

[0057] (5) Easy to use and burden-free, with high processing efficiency The entire analysis process is non-contact, requiring only video footage captured by ordinary camera equipment to complete the entire analysis process without the need for wearing any sensors. The small model employs an inter-frame sampling strategy, which significantly reduces the amount of computation. Combined with inference optimization of the large model, it ensures the overall processing efficiency of the system and enables rapid analysis and feedback.

[0058] (6) The combination of results visualization and text makes guidance more intuitive. The integrated presentation of comparative videos and natural language explanations and suggestions generated by large models allows trainees to visually see movement deviations through video visualizations and understand the core of the deviations and correction methods through professional text, thereby improving training effectiveness.

[0059] The technical solutions adopted in the embodiments of the present invention will be described below in conjunction with specific system hardware configurations.

[0060] 1. System Hardware and Data Input The system allows users to capture badminton action videos using a regular camera, mobile phone, or action camera. Users can then upload their MP4-formatted videos along with corresponding template videos through a web-based front-end interface. The system supports simultaneous uploading of two videos. The front-end also provides options for comparison modes (intuitive comparison / overlapping comparison) and prompt word templates (basic introduction / professional improvement / match review / single specialty), allowing users to choose according to their training needs.

[0061] 2. Video preprocessing After receiving the uploaded video, the backend extracts video frames using the OpenCV library, unifies the video resolution and encoding format, and performs image noise reduction using Gaussian filtering to remove abnormal frames such as blurry, overexposed, and distorted frames, ensuring stable input quality for subsequent small model detection. The preprocessed video frames are stored in time sequence to prepare for subsequent detection.

[0062] 3. Skeleton and racket detection based on small models The backend loads a small model detection library and synchronously calls the MediaPipe Pose Landmarker Lite model and the YOLOv8s model. It performs detection on preprocessed video frames using a sampling strategy every 3 frames. The skeleton keypoint and racket detection results are as follows: Figure 4 As shown, the specific process is as follows: (1) The MediaPipe model completes the detection of 17 key points of the human skeleton, outputs the coordinates of each joint point, and automatically completes the skeleton connection labeling. The user video is labeled as a red skeleton, and the template video is labeled as a blue skeleton. (2) The YOLOv8s model completes the racket target detection and outputs the racket center coordinates and bounding box, which can be represented by red lines to achieve accurate racket position labeling.

[0063] (3) The detection results are fed back in real time. If an abnormality occurs (such as missing key points of the skeleton or unrecognized racket), the system will automatically resample and detect to ensure the integrity of the detection.

[0064] 4. Automatic detection and calibration of the racket holder From the first 10 valid detection samples of the video, the coordinates of the racket center and the key points of the left and right wrists are extracted. The Euclidean distance from the racket center to both wrists is calculated. The variance of the distance values ​​on the same side is calculated. The side with the smaller variance is determined as the racket-holding hand side (the smaller the variance, the more stable the positional relationship between the racket and the wrist). Then, the coordinates of the key points of both shoulders are combined for calibration. If there is a deviation between the initial judgment result and the shoulder posture, the system automatically corrects it. Finally, the user's racket holding type (left hand / right hand) is output, and the result is synchronized to the subsequent video alignment module.

[0065] 5. Intelligent alignment of video temporal and spatial sequences (1) Timing alignment A red skeleton and racket annotation are added to the user's video, and the video playback speed is adjusted to 0.2x. A blue skeleton and racket annotation are added to the template video. The total number of frames in the template video is stretched to be exactly the same as that in the user's video using a time-stretching algorithm to achieve time synchronization of the first and last frames. At the same time, both videos are interpolated to 30 frames per second to ensure smooth playback. If the racket-holding hand is different from that in the template video, the system automatically mirrors the template video to adapt to the user's racket-holding habits.

[0066] (2) Spatial alignment The distance between the shoulder keypoints of the user video and the template video is calculated to obtain the spatial scaling ratio, and the template skeleton is scaled proportionally. The shoulder coordinates of consecutive frames are smoothed by a shoulder tracker to eliminate jitter and abnormal jumps. Based on the center point of the shoulders, the shoulder coordinates of the user and the template are accurately aligned by translation transformation to achieve a high degree of matching of the action space posture.

[0067] 6. Standardized comparison video generation Based on the comparison mode selected by the user, a corresponding standardized comparison video is generated to provide input for large model analysis. (1) Intuitive comparison mode The user video, after temporal and spatial processing, is spliced ​​with the template video in a split-screen format to generate two independently annotated videos, ensuring synchronized playback of the two videos. Figure 5 A visual comparison of the video output effect according to an embodiment of this application is shown.

[0068] (2) Overlapping comparison mode The user video is sharpened to enhance limb and joint details, while the template video is Gaussian blurred to blur the background and highlight skeletal features. The template video is then overlaid on the user video with a 0.7 opacity to generate a shoulder-aligned, blended, annotated video that clearly shows the location of motion deviations. Figure 6 An overlay comparison video output effect diagram is shown according to an embodiment of this application.

[0069] In addition, the generated comparison videos are formatted according to the input requirements of the Qwen3-VL model to ensure the complete transmission of video information.

[0070] 7. Multimodal large model inference with fusion of large and small models (1) Video input The standardized comparison video is input into the Qwen3-VL multimodal large model. The model automatically identifies features such as skeleton annotation, racket position, action sequence, and spatial deviation in the video. Relying on the professional annotation of the small model, it accurately focuses on the core analysis dimensions of badminton actions.

[0071] (2) Calling prompt word template Based on the prompt word template selected by the user, the system retrieves the corresponding template from the prompt word template library and inputs the template content and comparison video into the large model simultaneously, guiding the large model to perform analysis according to the preset focus. Figure 7 An example diagram of a prompt word template according to an embodiment of this application is shown.

[0072] (3) Model reasoning The inference parameters of the Qwen3-VL model were fine-tuned, and badminton sports features were set as the core focus dimension. After the model completed multimodal video understanding, it generated action analysis explanations and improvement suggestions in natural language format based on the prompt word template requirements. The content included overall action evaluation, core deviation points, reasons for deviation, correction methods, and training suggestions.

[0073] 8. Results Integration and Visualization Output The comparison videos generated by the small model and the textual explanations and suggestions output by the large model are integrated and displayed in a unified manner through the front-end interface: (1) The comparison video provides functions such as synchronized playback, pause, and progress dragging, making it convenient for trainees to view the details of the movements; (2) The text explanation should be presented according to the analysis focus of the user-defined prompt words, and the differences in movement and improvement methods of the user-specified parts (such as ankle, leg, waist and abdomen, arm, racket posture) should be presented first. The core deviation points and improvement methods should be marked in bold to improve readability. Figure 8 The diagram shows the output results according to an embodiment of this application.

[0074] (3) The front end supports the download of comparison videos and the copying and saving of explanatory texts. Trainees can save the results for repeated training and review.

[0075] 9. Model Optimization and System Stability Improvement (1) Small model optimization The MediaPipe model can use lightweight pre-trained weights to ensure real-time detection; the YOLOv8s model is fine-tuned on a self-made badminton-specific dataset to improve racket recognition accuracy in complex scenarios (such as low light or fast movements); and the computational load can be reduced and processing efficiency improved by using frame-by-frame sampling and outlier removal.

[0076] (2) Large model optimization For badminton scenarios, the Qwen3-VL model was fine-tuned by adding badminton terminology and motion feature databases to enhance the model's understanding of badminton movements; the inference speed was optimized to ensure real-time analysis.

[0077] (3) Overall system optimization Post-processing strategies such as sliding window mean smoothing and coordinate interpolation can be used to reduce detection jitter of key points in small model skeletons; semantic optimization of the output results of large models can be performed to ensure the fluency and professionalism of the explanatory text; the system is equipped with an anomaly handling mechanism, which automatically prompts users and provides solutions in case of video upload failure, model detection anomalies, etc., thereby improving the overall stability of the system.

[0078] Based on the same inventive concept, this application also provides a badminton motion intelligent analysis device corresponding to the above method, such as... Figure 9 As shown, it includes: The video acquisition module is used to acquire user videos to be analyzed, template videos, comparison patterns, and selected prompt word templates; The detection module is used to detect the human skeleton key points and racket position information of both the user video and the template video using a lightweight detection model, so as to obtain the user video with detection annotations and the template video with detection annotations, wherein the detection annotations are the coordinates of the skeleton key points and the racket position. The video processing module is used to perform temporal and spatial alignment processing on user videos and template videos with detection annotations to obtain aligned dual-channel labeled videos. Based on the acquired comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos. The large model inference module is used to input the standardized comparison video and the selected prompt word template into the pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user video.

[0079] Based on the same inventive concept as the above disclosure, this disclosure also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the above-described method. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of readable storage media include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0080] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0081] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for intelligent motion analysis, characterized in that, include: Obtain the user video to be analyzed, template video, comparison mode, and selected prompt word template; A lightweight detection model is used to detect key points of the human skeleton in both the user video and the template video, resulting in user videos and template videos with detection annotations, wherein the detection annotations include the coordinates of the skeleton key points. The user video and the template video with detection annotations are subjected to temporal and spatial alignment to obtain aligned dual-channel labeled video. Based on the acquired comparison pattern, the aligned dual-channel labeled video is processed to generate a standardized comparison video; The standardized comparison video and the selected prompt word template are input into a pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user video.

2. The method according to claim 1, characterized in that, The intelligent motion analysis method is applied to the analysis of ball-playing motions that require the use of a racket; the detection annotation also includes: racket position; After detecting human skeleton key point information in both the user video and the template video using a lightweight detection model to obtain user videos and template videos with detection annotations, and before performing temporal and spatial alignment processing on the user videos and template videos with detection annotations to obtain aligned dual-channel labeled videos, the method further includes: A lightweight detection model is used to detect racket position information in both the user video and the template video, resulting in user videos and template videos with detection annotations.

3. The method according to claim 2, characterized in that, The lightweight detection model includes a pose detection model and a target detection model; Human skeleton key point information is detected in both the user video and the template video using a lightweight detection model, including: A sampling strategy is adopted to sample the frame images of the user video and the template video every N frames to obtain sampled frame images, where N is an integer greater than or equal to 2; The pose detection model is used to detect key points of the human skeleton in the sampled frame images of the user video and the sampled frame images of the template video, respectively. A lightweight detection model is used to detect racket position information in both the user video and the template video, including: The target detection model is used to perform racket target recognition on the sampled frame images of the user video and the sampled frame images of the template video, respectively.

4. The method according to claim 2, characterized in that, After detecting racket position information in both the user video and the template video using a lightweight detection model to obtain user videos and template videos with detection annotations, before performing temporal and spatial alignment processing on the user videos and template videos with detection annotations, the process further includes: Detect the user's hand holding the camera based on the user video with detection annotations; Determine if the user's hand holding the clapper matches the hand holding the clapper in the template video; Temporal and spatial alignment processing is performed on user videos and template videos with detection annotations, including: When the user's hand holding the camera matches the hand holding the camera in the template video, temporal and spatial alignment processing is performed on the user video with detection labels and the template video with detection labels. When the user's hand holding the camera does not match the hand holding the camera in the template video, the template video is mirrored, and temporal and spatial alignment is performed on the user video with detection labels and the mirrored template video with detection labels.

5. The method according to claim 4, characterized in that, Detecting the user's hand holding the camera based on the user video with detection annotations includes: Extract the racket center coordinates, wrist key point coordinates, and shoulder key point coordinates from the user video with detection annotations, and calculate the distance from the racket center to the left wrist key point and the right wrist key point; Based on the distance from the center of the racket to the key points of the left and right wrists, the racket-holding hand side is initially determined. By combining the coordinates of key points on the shoulder, the initially determined racket-holding hand side is calibrated to obtain the user's racket-holding type.

6. The method according to claim 5, characterized in that, Based on the distances from the center of the racket to the key points of the left and right wrists, the initial determination of the racket-holding hand side includes: Calculate the Euclidean distance from the center of the racket to the key points of the left and right wrists and calculate the variance for each. The side with the smaller variance is used to initially determine the racket-holding hand side.

7. The method according to claim 1, characterized in that, Temporal and spatial alignment processing is performed on user videos and template videos with detection annotations, including: When the comparison mode is the intuitive comparison mode, the frame rate of the template video with detection labels is stretched to match the frame rate of the user video with detection labels, and the playback speed of the user video is adjusted to the preset slow motion speed to achieve time sequence alignment of the first and last frames. When the obtained comparison mode is the overlapping comparison mode, the frame number of the template video with detection labels is stretched to match the frame number of the user video with detection labels, and the playback speed of the user video is adjusted to a preset slow motion speed to achieve temporal alignment of the first and last frames; taking the shoulder node as the reference, the shoulder spacing ratio of the temporally aligned user video and the template video is calculated, and the temporally aligned template video is scaled based on the shoulder spacing ratio, and spatial posture alignment is achieved through shoulder coordinate smoothing and translation transformation.

8. The method according to claim 1, characterized in that, The comparison modes include intuitive comparison mode and overlapping comparison mode; The step of processing the aligned dual-channel labeled video according to the acquired comparison pattern to generate a standardized comparison video includes: If it is an intuitive comparison mode, the aligned dual-channel labeled video will be split into left and right screens and spliced ​​together to generate an independent synchronous comparison video; If the overlay comparison mode is selected, the user video is sharpened, the template video is Gaussian blurred, and the processed template video is overlaid on the processed user video with a preset transparency to generate a fused comparison video.

9. The method according to claim 2, characterized in that, The prompt word template is a user-defined template, which is used to specify the focus of the analysis. The focus of the analysis includes at least one of the following: differences in posture, trajectory, timing or normativity of ankle joint, leg, waist and abdomen, arm, and racket posture.

10. A motion intelligent analysis device, characterized in that, include: The video acquisition module is used to acquire user videos to be analyzed, template videos, comparison patterns, and selected prompt word templates; The detection module is used to detect human skeleton key point information in both the user video and the template video using a lightweight detection model, so as to obtain the user video with detection annotation and the template video with detection annotation, wherein the detection annotation includes the coordinates of the skeleton key points; The video processing module is used to perform temporal and spatial alignment processing on user videos and template videos with detection annotations to obtain aligned dual-channel labeled videos. Based on the acquired comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos. The large model inference module is used to input the standardized comparison video and the selected prompt word template into the pre-trained multimodal large model to obtain analysis, explanation and improvement suggestions for the user video.

11. A motion intelligent analysis system, characterized in that, Includes a front-end interaction module, a lightweight model detection library, a video processing module, a multimodal large model, and a prompt word template library; The front-end interaction module is used for user video upload, standard action video selection, comparison mode selection, prompt word template selection, and to integrate and display comparison videos and analysis and explanation suggestions. The lightweight model detection library is used to perform human skeleton key point detection on the user video and the selected template video to obtain user video with detection annotations and template video with detection annotations, wherein the detection annotations include the coordinates of skeleton key points; The video processing module is used to perform temporal and spatial alignment processing on user videos with detection annotations and template videos with detection annotations to obtain aligned dual-channel labeled videos; according to the selected comparison mode, the aligned dual-channel labeled videos are processed to generate standardized comparison videos; The multimodal large model is used to receive standardized comparison videos and selected prompt word templates, perform inference, and output analysis, explanations, and improvement suggestions for the user's videos. The prompt word template library is used to store and provide prompt word templates.

12. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

Images