Athletic performance analysis and guidance method and system based on multi-modal data fusion

Abstract

The invention discloses an athletic performance analysis and guidance method based on multi-modal data fusion. The method comprises the following steps: acquiring multi-modal physiological and athletic data, related to athletic performance, of a user; processing the multi-modal physiological and motion data to extract multi-dimensional motion features, the multi-dimensional motion features including muscle activity features, kinematics features and dynamics features; based on the multi-dimensional motion features, performing fusion analysis and prediction through an artificial intelligence model, and generating an analysis result including future motion state prediction and / or motion quality evaluation; and based on the analysis result, generating and outputting guidance information for the user. According to the method, multi-modal data processing, feature fusion, model prediction and guidance generation are integrated, absolute dependence of exercise analysis on personal experience of coaches is reduced, and professional exercise performance analysis and guidance can be popularized and applied in wider scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and data fusion technology, specifically to a method and system for motion performance analysis and guidance based on multimodal data fusion. Background Technology

[0002] With the popularization of national fitness and the development of competitive sports, people have placed higher demands on the scientific nature of sports training. Traditional sports performance analysis often relies on coaches' experience and observation or single sensor data (such as using only accelerometers or heart rate monitors). This method has obvious limitations: First, subjective observation cannot quantify intrinsic biological signals such as muscle activation state and joint stress, making it difficult to accurately locate technical defects; second, single-dimensional data cannot comprehensively reflect the true state of complex movements, resulting in biased analysis results; finally, traditional analysis methods are usually lagging, unable to predict future trends and potential risks of movements, leading to untimely guidance and insufficient ability to prevent injuries.

[0003] Therefore, existing technologies lack a comprehensive solution that can make full use of multi-source information to achieve a comprehensive understanding of the underlying mechanisms, from current situation analysis to future prediction, and provide real-time personalized guidance. Summary of the Invention

[0004] To overcome the above-mentioned technical deficiencies, the present invention provides a method and system for motion performance analysis and guidance based on multimodal data fusion.

[0005] To solve the above problems, the present invention is implemented according to the following technical solution:

[0006] In a first aspect, the present invention provides a method for analyzing and guiding sports performance based on multimodal data fusion, characterized by comprising the following steps: collecting multimodal physiological and sports data related to user sports performance; processing the multimodal physiological and sports data to extract multidimensional sports features, including muscle activity features, kinematic features, and dynamic features; performing fusion analysis and prediction based on the multidimensional sports features using an artificial intelligence model to generate analysis results including predictions of future movement states and / or assessments of movement quality; and generating and outputting guidance information for the user based on the analysis results.

[0007] In conjunction with the first aspect, the present invention provides a first specific implementation of the first aspect. Specifically, the acquisition of multimodal physiological and motor data related to user and motor performance includes the following steps: acquiring joint point temporal data obtained based on motion sequence image processing, wherein the joint point temporal data includes 2D or 3D joint point temporal data; acquiring electromyographic signals acquired based on flexible sensors, wherein the electromyographic signals characterize the activation state of the target muscle group during movement; and acquiring plantar pressure distribution data acquired based on pressure sensing units, wherein the plantar pressure distribution data includes static pressure values ​​and dynamic pressure change sequences.

[0008] In conjunction with the first aspect, the present invention provides a second specific implementation of the first aspect, specifically characterized in that the processing of the multimodal physiological and motor data to extract multidimensional motion features includes the following steps: processing the acquired joint point temporal data, electromyographic signals, and plantar pressure distribution data to form a multimodal dataset with a unified timestamp; preprocessing the original data of each modality in the multimodal dataset, the preprocessing including filtering and enhancing the electromyographic signals, smoothing and denoising the joint point temporal data, and standardizing the plantar pressure distribution data; based on a preset standard movement template, using the joint point temporal data and / or plantar pressure distribution data, dividing the user's continuous motion flow into multiple movement stages; and extracting the multidimensional motion features from the data of different modalities within each divided movement stage.

[0009] In conjunction with the first aspect, the present invention provides a third specific implementation of the first aspect. Specifically, the multidimensional motion features include at least: muscle activity features extracted from the electromyographic signals, the muscle activity features including muscle activation intensity, activation timing, and fatigue index; kinematic features extracted from the joint timing data, the kinematic features including joint angle, angular velocity, joint coupling relationship, and center of mass trajectory; and dynamic features extracted from the plantar pressure distribution data, the dynamic features including pressure center trajectory, peak impact force, and pressure propagation velocity.

[0010] In conjunction with the first aspect, this invention provides a fourth specific implementation of the first aspect. Specifically, the step of performing fusion analysis and prediction based on the multi-dimensional motion features using an artificial intelligence model to generate an analysis result including future action state prediction and / or action quality assessment specifically includes the following steps: inputting the multi-dimensional motion features into a multi-channel deep spatiotemporal graph neural network; learning the spatial relationships between human joints through the spatial graph convolutional layer in the multi-channel deep spatiotemporal graph neural network, learning the dynamic evolution pattern of each joint point through the temporal convolutional layer, dynamically interacting and weighted fusing the muscle activity features, kinematic features, and dynamic features to generate a deeply fused spatiotemporal feature representation; predicting the user's key joint angles through a regression prediction branch based on the deeply fused spatiotemporal feature representation; evaluating the quality of the user's action completion through a classification evaluation branch based on the deeply fused spatiotemporal feature representation; and generating an analysis result by combining the predicted joint angle values ​​and the quality level.

[0011] In conjunction with the first aspect, the present invention provides a fifth specific implementation of the first aspect. Specifically, the step of generating and outputting guidance information for the user based on the analysis results includes the following steps: making a judgment based on the predicted joint angle value and quality level, combined with real-time feature data; triggering guidance at the corresponding level when at least one of the following conditions is met simultaneously: the predicted key joint angle deviates from a preset angle threshold; related muscle activity features show underactivation, overactivation, or temporal abnormalities; plantar dynamics features show that the movement stability is lower than a preset standard; and generating corresponding prompts and interactive instructions according to the triggered guidance level.

[0012] In conjunction with the first aspect, the present invention provides a sixth specific implementation of the first aspect. Specifically, the guidance levels include at least: Level 1 prompts, corresponding to a slight deviation in the predicted action, generating visual or audio prompt instructions; Level 2 prompts, corresponding to a larger deviation in the predicted action, generating enhanced visual prompts and specific voice prompt instructions; and Level 3 prompts, corresponding to a significant deviation in the predicted action from the standard and the existence of risk, generating warning visual prompts and adjustment suggestion instructions.

[0013] In conjunction with the first aspect, the present invention provides a seventh specific implementation of the first aspect. Specifically, the generation of corresponding prompt content and interaction instructions includes the following steps: comparing the user's real-time or predicted skeletal posture with the standard action skeletal posture and marking the differences; generating a muscle activation heatmap based on real-time muscle activity data and marking abnormal muscle states with icons; and displaying the predicted joint angle, quality rating, and text improvement suggestions in the data panel.

[0014] In conjunction with the first aspect, the present invention provides an eighth specific implementation of the first aspect, which specifically includes: generating an analysis report based on historical data, the analysis report including a movement quality score, a risk trend graph, and a guidance event record; selecting corresponding training movements from a preset training movement library based on the improvement directions in the analysis report to generate a personalized training plan; constructing a personal movement feature model based on the user's body feature data, historical movement feature parameters, and personalized model configuration; and continuously updating and optimizing the personal movement feature model based on the user's subsequent training data.

[0015] Secondly, the present invention also provides a motion performance analysis and guidance system based on multimodal data fusion, comprising: a data acquisition module for acquiring multimodal physiological and motion data related to user motion performance; a feature extraction module for processing the multimodal physiological and motion data to extract multidimensional motion features, the multidimensional motion features including muscle activity features, kinematic features, and dynamic features; an analysis and prediction module for performing fusion analysis and prediction based on the multidimensional motion features using an artificial intelligence model to generate analysis results including future motion state prediction and / or motion quality assessment; and a guidance output module for generating and outputting guidance information for the user based on the analysis results.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] By comprehensively collecting and fusing multi-dimensional features such as muscle activity (e.g., electromyographic signals), kinematics (e.g., joint angles, velocity), and dynamics (e.g., force, ground reaction force), a complete and systematic analysis of the entire chain from neuromuscular control to external movement performance is achieved. This overcomes the limitations of traditional single-dimensional analysis, enabling a deeper revelation of the underlying physiological and biomechanical mechanisms of technical movements, providing a solid foundation for precise guidance.

[0018] By using artificial intelligence models to fuse and predict multi-dimensional features, it is possible to generate predictions about future movement states (such as predicting potential movement deformities or injury risks caused by fatigue). This allows the system to move beyond the delayed analysis of completed movements and proactively warn of risks and anticipate problems, thus providing a crucial time window for timely intervention of guidance information and prevention of sports injuries.

[0019] Based on the guidance information generated from deep fusion analysis (including movement quality assessment and future performance prediction), the system can pinpoint a user's unique technical weaknesses, strength imbalances, or compensatory patterns, and provide customized corrective training, strength enhancement, or fatigue recovery programs. This significantly improves the scientific rigor, relevance, and efficiency of training, while effectively ensuring user safety through risk warnings.

[0020] This method integrates multimodal data processing, feature fusion, model prediction, and guidance generation, greatly reducing the absolute dependence of sports analysis on the coach's personal experience. This enables high-level, professional-grade sports performance analysis and guidance to be popularized and applied in a wider range of scenarios (such as daily fitness, amateur training, and rehabilitation exercises). Attached Figure Description

[0021] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a motion performance analysis and guidance method based on multimodal data fusion according to the present invention. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] like Figure 1 As shown, this invention provides a method and system for motion performance analysis and guidance based on multimodal data fusion.

[0025] Example 1

[0026] like Figure 1 As shown, a method for analyzing and guiding sports performance based on multimodal data fusion includes the following steps: collecting multimodal physiological and sports data related to user sports performance; processing the multimodal physiological and sports data to extract multidimensional sports features, including muscle activity features, kinematic features, and dynamic features; performing fusion analysis and prediction based on the multidimensional sports features using an artificial intelligence model to generate analysis results that include predictions of future movement states and / or assessments of movement quality; and generating and outputting guidance information for the user based on the analysis results.

[0027] Compared with existing technologies, the multimodal data fusion-based method for motion performance analysis and guidance provided by this invention has the following significant advantages: By collecting and fusing multimodal physiological and motion data, it can comprehensively extract features from multiple dimensions such as muscle activity, kinematics, and dynamics, overcoming the limitations of single data source analysis. This allows the evaluation of motion performance to go beyond surface movements and deeply reveal its underlying physiological and mechanical causes, resulting in more comprehensive and accurate analysis results. Through the fusion analysis and prediction of multi-dimensional features using artificial intelligence models, predictive information about future motion states can be generated. This not only enables the assessment of the user's current state but also allows for the forward-looking prediction of motion trends and potential motion risks, providing a crucial early intervention window for injury prevention and technology optimization. The guidance information generated based on the results (including predictions and assessments) after deep fusion analysis ensures the scientific rigor and relevance of the guidance. This method achieves a closed loop from "data acquisition" to "guidance output," providing users with real-time, online, personalized feedback, greatly improving training efficiency and the speed of learning motor skills.

[0028] In a preferred embodiment, S100: Collect multimodal physiological and motor data related to user and motor performance.

[0029] Specifically, the multimodal physiological and motor data may include, but is not limited to: electromyographic signals acquired by surface electromyography (sEMG) sensors; acceleration and angular velocity data acquired by inertial measurement units (IMUs) to calculate kinematic parameters of various body segments; ground reaction force data acquired by pressure testing platforms or foot pressure pads; and precise joint angle and trajectory data obtained by optical motion capture systems or computer vision algorithms.

[0030] In a preferred embodiment, S200: The multimodal physiological and motion data are processed to extract multidimensional motion features, which include muscle activity features, kinematic features and dynamic features.

[0031] The multidimensional motion features include:

[0032] Muscle activity characteristics: such as muscle activation sequence, activation degree, muscle fatigue index, etc.

[0033] Kinematic characteristics: such as joint angles, angular velocity, angular acceleration, limb displacement, velocity, and range of motion;

[0034] Dynamic characteristics: such as peak ground reaction force, impulse, joint torque, power, etc.

[0035] In a preferred embodiment, S300: Based on the multi-dimensional motion features, a fusion analysis and prediction are performed using an artificial intelligence model to generate analysis results that include predictions of future motion states and / or motion quality assessments.

[0036] Specifically, based on the aforementioned multi-dimensional motion features, a pre-trained artificial intelligence model is used for fusion analysis and prediction. The artificial intelligence model can be a deep learning model (such as LSTM, Transformer, Graph Neural Network GNN, etc.) or a hybrid model that integrates multiple algorithms, capable of generating analysis results that include predictions of future motion states (such as the next joint posture and center of gravity trajectory) and / or motion quality assessments (such as technical motion scores, abnormal motion pattern identification, and injury risk levels).

[0037] In a preferred embodiment, S400: Based on the analysis results, generate and output guidance information for the user.

[0038] Specifically, the guidance information can be specific voice prompts (such as "Knee adduction, please correct it"), visual feedback (such as highlighting the parts that need attention in a virtual human body model on the screen), tactile feedback (such as vibration from a smart device), or training suggestions in text form. The guidance information emphasizes real-time, targeted, and actionable characteristics.

[0039] In a preferred embodiment of the present invention, the method is specifically described using a basketball jump shot as an example:

[0040] Athletes wear smart clothing and insoles integrated with flexible sensors. The clothing incorporates sensors at corresponding locations on major joints (such as the ankle, knee, hip, shoulder, and elbow) and major muscle groups in the lower limbs (such as the quadriceps and hamstrings); simultaneously, the insoles have built-in pressure sensing units. The system simultaneously collects electromyographic signals during takeoff, kinematic data from various body segments, and plantar pressure distribution data.

[0041] The collected raw data were fused and feature extracted. The activation sequence and integrated electromyography (EMG) values ​​of the lower limb muscle groups were analyzed from the EMG signals. Based on flexible sensor data, kinematic parameters such as the angles and angular velocities of the hip, knee, and ankle joints in the sagittal and frontal planes were calculated. Combined with insole pressure data, the dynamic characteristics such as the trajectory of the plantar pressure center, impulse, and peak force during takeoff and landing were analyzed.

[0042] The extracted multi-dimensional features are input into a trained LSTM neural network model. The model performs fusion analysis and temporal prediction on these features, and outputs comprehensive analysis results: for example, it predicts that under the current force exertion pattern, the knee joint has a high risk of valgus at the moment of landing; at the same time, it assesses that there is an abnormal movement pattern of "premature activation of the quadriceps and insufficient hamstring coordination" in this jump, and gives a movement quality score of 85.

[0043] Based on the analysis results, voice instructions were immediately given to the athlete via Bluetooth headset: "Pay attention to aligning your knees with your toes when landing, and feel the power coming from your hips when jumping." At the same time, the movement video was played back on a tablet, and the ideal joint alignment and power sequence were displayed with animation overlay for visual comparison and correction.

[0044] In a preferred embodiment, the collection of multimodal physiological and motor data related to user performance specifically includes the following steps:

[0045] S101: Obtain joint timing data based on motion sequence image processing, wherein the joint timing data includes 2D or 3D joint timing data.

[0046] Specifically, a sequence of images of the user during movement is continuously acquired using deployed visual sensors (such as RGB cameras, depth cameras, or high-speed infrared motion capture systems). These images are then processed for human posture estimation and joint point recognition, extracting the positional information of the user's major joint points and arranging them chronologically to generate temporal joint point data. This data can include joint point coordinates in a two-dimensional (2D) plane, or three-dimensional (3D) spatial joint point coordinates obtained through multi-view visual information fusion and depth information calculation, thereby accurately describing the continuous trajectory of the user's limb posture changes during movement.

[0047] S102: Acquire electromyographic signals based on flexible sensors, wherein the electromyographic signals characterize the activation state of the target muscle group during movement.

[0048] Specifically, one or more flexible wearable electromyography (EMG) sensors, attached to the user's skin, collect bioelectrical signals, i.e., EMG signals, generated by the target muscle group during exercise. These EMG signals can accurately characterize the activation state, activation sequence, and activation intensity of a specific muscle or muscle group during exercise in real time, providing a data foundation for analyzing muscle function and neural control strategies.

[0049] S103: Acquire plantar pressure distribution data based on pressure sensing unit, wherein the plantar pressure distribution data includes static pressure values ​​and dynamic pressure change sequences.

[0050] Specifically, pressure sensor arrays integrated into insoles or pressure plates collect pressure information about the user's foot when it contacts the support surface. The foot pressure distribution data includes static pressure values ​​when the user is standing still, and dynamic pressure change sequences over time in different areas of the foot during dynamic movements such as walking, running, and jumping. This data can be used to analyze the user's balance, gait characteristics, and foot load distribution.

[0051] In a preferred embodiment, the processing of the multimodal physiological and motion data to extract multidimensional motion features specifically includes the following steps:

[0052] S201: Process the acquired joint time series data, electromyographic signals and plantar pressure distribution data to form a multimodal dataset with a unified timestamp.

[0053] Specifically, asynchronous raw data streams from different sensors are received, including joint time-series data, electromyography signals, and plantar pressure distribution data. Using hardware synchronization signals or software timestamp interpolation algorithms, the data from all modalities are aligned to a unified timeline, forming a synchronized multimodal dataset with a unified timestamp. This dataset provides the foundation for subsequent cross-modal joint analysis and feature fusion.

[0054] S202: Preprocess the original data of each modality in the multimodal dataset. The preprocessing includes filtering and enhancing electromyographic signals, smoothing and denoising joint time series data, and standardizing plantar pressure distribution data.

[0055] Specifically, the raw data of each modality in the multimodal dataset are preprocessed to improve data quality and prepare for feature extraction. The preprocessing includes at least:

[0056] Filtering and enhancement of electromyographic signals: Bandpass filters are used to remove power frequency interference and motion artifacts, and the signals are enhanced to clearly characterize the muscle activation state through full-wave rectification and smoothing (such as linear envelope).

[0057] Smoothing and denoising of key point time series data: Kalman filtering, low-pass filtering or moving average algorithm is applied to smooth the key point coordinate sequence to suppress noise caused by image recognition error or slight body shaking, so as to obtain a smoother and more accurate motion trajectory.

[0058] Standardization of plantar pressure distribution data: The absolute pressure values ​​are normalized based on the user's weight or the maximum range of a single sensor to eliminate the influence of individual and sensor differences, making the data comparable between different users and different tests.

[0059] S203: Based on a preset standard motion template, using joint timing data and / or plantar pressure distribution data, the user's continuous motion flow is divided into multiple motion stages.

[0060] Specifically, a standard movement template is pre-stored, which defines the spatiotemporal or dynamic characteristics of each key movement phase (such as squatting, standing up; support phase, swing phase) of the target movement (such as squatting, standing up; support phase, swing phase). By dynamically time-warping or rule-based matching of real-time collected joint point time-series data (such as knee joint angle, hip joint velocity) and / or plantar pressure distribution data (such as pressure center trajectory, total pressure value) with the standard movement template, the user's continuous motion flow is automatically divided into multiple movement phases with clear biomechanical significance.

[0061] S204: Within each of the divided action stages, extract the multi-dimensional motion features from data of different modalities.

[0062] Within the defined time windows of each action phase, the multi-dimensional motion features are extracted from data of different modalities. These features collectively constitute a comprehensive description of the user's motion performance.

[0063] Extract kinematic features from joint time-series data: For example, during the "squatting" phase of a squat, extract the maximum flexion angle, range of motion, movement speed / acceleration, and trunk forward tilt angle of the hip, knee, and ankle joints.

[0064] Extracting muscle activity features from electromyography (EMG) signals: For example, during the "hold phase" of running, extracting the activation onset time, activation duration, average amplitude, and integrated EMG values ​​of specific muscles (such as the rectus femoris and gastrocnemius).

[0065] Extracting dynamic features from plantar pressure distribution data: For example, at the moment of "heel strike" in gait, extracting the peak impact force, the trajectory of the pressure center in the heel area, the peak pressure and arrival time of each zone of the plantar surface, etc.

[0066] In a preferred embodiment, the multidimensional motion features include at least:

[0067] Muscle activity features extracted from the electromyographic signals include muscle activation intensity, activation sequence, and fatigue index.

[0068] Extracted from preprocessed electromyographic signals, these features are used to quantify the functional state of the target muscle group during movement. Specifically, the muscle activity characteristics include:

[0069] Muscle activation intensity: Characterizes the strength of muscle contraction, which can be calculated by the amplitude of electromyographic signals (such as peak, average or root mean square value).

[0070] Activation sequence: Characterizes the specific moments or relative order in which different muscles begin to activate and cease activation during a movement cycle, used to analyze the coordination between muscles.

[0071] Fatigue index: Characterizes the change in muscle endurance during sustained exercise, and can be estimated by the decreasing trend of the median frequency or the increasing trend of the amplitude in the electromyographic signal spectrum.

[0072] The kinematic features extracted from the joint time-series data include joint angles, angular velocities, joint coupling relationships, and center-of-mass trajectories.

[0073] Extracted from smoothed and denoised joint time-series data, these kinematic features describe the motion patterns, speeds, and coordination of the body and its various parts. Specifically, these kinematic features include:

[0074] Joint angle: The angle formed between adjacent segments of the body, which is a basic parameter for describing movement posture (such as the knee flexion angle).

[0075] Joint angular velocity: the rate of change of joint angle over time, used to assess the speed and explosive power of a movement.

[0076] Joint coupling relationship: describes the coordination pattern of two or more joints during movement, such as the phase relationship or proportional relationship of the angle changes of the hip and knee joints.

[0077] Center of mass trajectory: The path of the user's total center of mass in space, which is a key indicator for assessing overall balance and stability.

[0078] The dynamic features extracted from the plantar pressure distribution data include the pressure center trajectory, peak impact force, and pressure propagation velocity.

[0079] Extracted from standardized plantar pressure distribution data, these dynamic characteristics reflect the interaction forces and their distribution between the person and the support surface during movement. Specifically, these characteristics include:

[0080] Pressure center trajectory: The path along which the point of application of the resultant vertical reaction force moves over time on the plantar support surface, used to analyze balance control and gait stability.

[0081] Peak impact force: The maximum vertical reaction force generated at the moment the foot contacts the ground, which is highly correlated with the risk of sports injury.

[0082] Pressure propagation velocity: The rate at which plantar pressure propagates from the heel to the forefoot (or between specific areas), used to assess the foot's rolling mechanism and cushioning effectiveness.

[0083] In a preferred embodiment, the step of performing fusion analysis and prediction based on the multi-dimensional motion features using an artificial intelligence model to generate analysis results including future motion state prediction and / or motion quality assessment specifically includes the following steps:

[0084] S301: Input the multi-dimensional motion features into a multi-channel deep spatiotemporal graph neural network.

[0085] Specifically, muscle activity characteristics Kinematic characteristics and dynamic characteristics The vectors are concatenated and vectorized to form a multi-channel feature vector. For a containing Each time step A sequence of key points, whose input can be constructed into a three-dimensional tensor. ,in It is the number of channels for the features at each node, which integrates information from all dimensions such as electromyography, joint angle, and plantar pressure.

[0086] S302: The spatial relationships between human joints are learned through the spatial graph convolutional layer in the multi-channel deep spatiotemporal graph neural network, and the dynamic evolution pattern of each joint is learned through the temporal convolutional layer. The muscle activity features, kinematic features and dynamic features are dynamically interacted and weighted and fused to generate a deep fused spatiotemporal feature representation.

[0087] Specifically, the human skeleton is defined as a diagram. ,in yes A set of key points It is a collection of skeletal connections. It is the adjacency matrix of the graph, encoding the natural connectivity relationships between joints. Learnable adjacency matrices can be further utilized. To capture potential spatial dependencies that go beyond physical connections.

[0088] Graph convolution operation: The following approximate implementation of spectral graph convolution is used to capture spatial features: in, It is the first The input features of the layer It is an adjacency matrix with self-connections. It is an identity matrix. yes degree matrix ( ), used for normalization. This is the trainable weight matrix for this layer. It is a non-linear activation function (such as ReLU). This operation allows the features of each key point to aggregate information from its neighboring nodes.

[0089] Temporal dynamic modeling (temporal convolutional layer): After spatial convolution, a one-dimensional temporal convolutional layer is used to model along the time dimension. For a length of... The convolution kernel has the following output: ;in, and These are the weights and biases of the convolution kernel. It is the output of the spatial graph convolution. This operation can learn the dynamic evolution patterns of the key points.

[0090] Multimodal feature interaction and weighted fusion:

[0091] A cross-modal attention mechanism is introduced in the middle layer of the network. This mechanism is applied to feature subsets from different modalities. (like , , ), calculate its attention weights : ; ;in, It is the global average of all modal features. They are shared linear transformation weights. It is an attention vector. This indicates a splicing operation. It refers to the number of modalities. Ultimately, the features after weighted fusion... This process enables dynamic interaction and weighting between modalities, allowing the network to focus on signals most relevant to the current motion task.

[0092] After processing by multiple spatiotemporal convolutional blocks and fusion modules, the network finally generates a deeply fused spatiotemporal feature representation. This representation comprehensively contains core information about the user's movement status.

[0093] S303: Based on the spatiotemporal feature representation of the deep fusion, the user's key joint angles are predicted through a regression prediction branch.

[0094] Specifically, based on The regression prediction branch of the network (usually composed of fully connected layers) is used to predict the future. Key joint angles after time ;in, and These are the weights and biases of the regression branch. This branch minimizes the predicted value. From the perspective of the real future Mean square error between To conduct training.

[0095] S304: Based on the spatiotemporal feature representation of the deep fusion, the quality of user action completion is evaluated through a classification evaluation branch.

[0096] Specifically, specifically, based on The classification evaluation branch of the network (usually composed of a fully connected layer and a Softmax function) is used to evaluate the quality of the user's completed or currently performed actions. ;in, It is a probability vector, representing that the action belongs to The probability of each quality level (e.g., "Excellent", "Good", "Acceptable", "Risk Exists"). Evaluation Results This branch minimizes the cross-entropy loss. To conduct training, among which It is the one-hot encoding of the real label.

[0097] S305: Combine predicted joint angles with quality grades to generate analysis results.

[0098] Specifically, the predicted joint angles output by the regression prediction branch are combined. The quality level output by the classification evaluation branch and its probability distribution This generates a structured analysis result. This result not only provides quantitative predictions of future movement states to warn of abnormal postures or potential risks, but also offers a qualitative assessment and confidence level of the current movement performance quality. Together, these provide accurate and reliable data for generating personalized, forward-looking movement guidance.

[0099] In a preferred embodiment, generating and outputting guidance information for the user based on the analysis results specifically includes the following steps:

[0100] Based on the predicted joint angle and quality level, and combined with real-time feature data, a judgment is made; when at least one of the following conditions is met simultaneously, the corresponding level of guidance is triggered: the predicted key joint angle deviates from the preset angle threshold; the related muscle activity features show underactivation, overactivation, or temporal abnormalities; the plantar dynamics features show that the movement stability is lower than the preset standard; according to the triggered guidance level, corresponding prompts and interactive instructions are generated.

[0101] Specifically, it receives joint angle prediction values ​​from the regression prediction branch. and the quality level of the classification and evaluation branches It combines real-time muscle activity characteristics and plantar dynamics data, and makes real-time judgments based on predefined expert rules and safety thresholds.

[0102] The system will trigger the corresponding level of guidance information when at least one of the following conditions is met:

[0103] Condition 1: Warning of Abnormal Movement Posture - The predicted future key joint angles (such as knee flexion angle) deviate from the preset safe or standard angle threshold range for this movement. For example, predicting that the knee joint will valgus in the next moment during a squat ( ).

[0104] Condition 2: Muscle dysfunction identification - Muscle activity characteristics related to the target movement show underactivation (e.g., gluteus medius amplitude below the threshold), overactivation (e.g., excessive tension in the rectus femoris leading to synergist dominance), or timing abnormalities (e.g., delayed hamstring activation, which does not match the timing of the quadriceps).

[0105] Condition 3: Insufficient dynamic stability - Plantar dynamic characteristics indicate that movement stability is lower than the preset standard. For example, the swing amplitude of the pressure center trajectory exceeds the stable range, or the pressure propagation speed during the single-leg support phase is abnormal, indicating a decline in balance control ability.

[0106] In a preferred embodiment, the guidance level includes at least:

[0107] Level 1 prompts correspond to slight deviations in the predicted action, generating visual or audio prompts.

[0108] Secondary prompts are generated for cases where the predicted action has a large deviation, providing enhanced visual cues and specific voice prompts.

[0109] Level 3 prompts correspond to situations where the predicted action deviates significantly from the standard and poses a risk, generating warning visual prompts and adjustment suggestions.

[0110] Specifically, based on the severity, combination, and quality level of the above conditions. This triggers guidance at the corresponding level. The guidance levels include at least:

[0111] Level 1 Guidance (Informative): Triggered when a minor deviation or potential risk is detected. For example, a quality level of "Acceptable" but a forecast indicates a deteriorating attitude.

[0112] Level 2 guidance (corrective): Triggered when a clear error or moderate risk is detected. For example, a quality level of "risk exists" and a critical angle has exceeded the threshold.

[0113] Level 3 guidance (warning): Triggered immediately when a major error or high risk that could lead to injury is detected. For example, predicting a severe abnormality in joint angle accompanied by an excessively high peak impact force.

[0114] In a preferred embodiment, generating the corresponding prompt content and interaction instructions specifically includes the following steps:

[0115] The system compares the user's real-time or predicted skeletal pose with the standard motion skeletal pose and marks the differences.

[0116] A muscle activation heatmap is generated based on real-time muscle activity data, and abnormal muscle states are marked with icons.

[0117] The data panel displays predicted joint angles, quality ratings, and text improvement suggestions.

[0118] Based on the triggered guidance level and the specific exception type, the system generates corresponding prompts and interactive instructions from a pre-set guidance corpus. The prompts are specific, actionable guidance statements, such as:

[0119] Regarding angle deviation: "Please make sure your knees and toes are pointing in the same direction, and avoid knee valgus."

[0120] For insufficient muscle activation: "Concentrate your mind and feel your glutes working."

[0121] Regarding the lack of stability: "Please slow down your movements and shift your center of gravity slightly backward."

[0122] Interactive commands are used to control user terminal devices to achieve multimodal interaction, for example:

[0123] Visual instructions: Highlight abnormal joints on AR glasses or screen interface, or display a demonstration of the correct motion animation.

[0124] Auditory instructions: The above prompts are read aloud via text-to-speech (TTS), or specific prompt sounds are played.

[0125] Tactile commands: Tactile cues are applied to specific body parts (such as the locations of under-activated muscles) via the vibration motor of a wearable device.

[0126] In a preferred embodiment, S500: Generate an analysis report based on historical data, the analysis report including action quality scores, risk trend graphs, and guidance event records.

[0127] Specifically, the system persistently stores the multimodal data, extracted features, analysis results, and guidance events generated during each user training session, forming a personal historical database. Based on this, the system generates an analysis report periodically (e.g., weekly or monthly) or on demand, which includes at least:

[0128] Movement quality rating: The quality level (such as "excellent", "good" etc.) of all training movements within the period is statistically analyzed, the average score and trend score (month-on-month increase / decrease) are calculated, and the distribution of the user's strengths and weaknesses in different types of movements (such as strength, stability, coordination) is visualized in the form of radar charts, etc.

[0129] Risk Trend Chart: The chart statistically analyzes and visualizes the abnormal posture, abnormal muscle function, and insufficient stability detected during the period in a time series, forming a trend line of risk event frequency and severity, which intuitively shows the dynamic changes in the user's injury risk.

[0130] Guidance Event Log: Lists the main corrective instructions triggered within the cycle and their frequency of occurrence, helping users and coaches review the most common error patterns.

[0131] S600: Based on the improvement directions in the analysis report, select the corresponding training actions from the preset training action library to generate a personalized training plan;

[0132] Specifically, based on the weaknesses and improvement directions identified in the analysis report, targeted training actions are intelligently selected from a preset training action library to generate a personalized training plan.

[0133] Selection logic: For example, if the report shows that the user has poor knee stability and insufficient gluteus medius activation, the system will prioritize exercises such as "shell pose" and "side-lying leg raise" that aim to strengthen the hip abductor muscles. If the report shows uneven foot pressure distribution, foot function training exercises such as "heel raises" and "towel gripping" will be selected.

[0134] The generated plan clearly specifies the combination of training movements, number of sets, number of repetitions, rest intervals, and other elements, forming a complete training program.

[0135] S700: Constructs a personal motion feature model based on the user's body feature data, historical motion feature parameters, and personalized model configuration.

[0136] Specifically, in order to more accurately describe the individual characteristics of users, the system constructs a personal motion characteristic model based on the following data:

[0137] User's physical characteristics data, such as height, weight, limb length, and history of injuries or illnesses.

[0138] Historical motion characteristic parameters: stable characteristics extracted from historical data that represent a user's personal motion patterns, such as typical joint range of motion, dominant side muscle activation patterns, and center of pressure preference in static standing posture.

[0139] Personalized model configuration: Using transfer learning or meta-learning techniques, based on a general AI model, key parameters of the model are fine-tuned using the user's personal data to better suit the user's unique needs. This process can be formalized as follows: ;in, These are the basic parameters of the pre-trained model. , It is the user's personal data and tags. It is a regularization coefficient used to prevent overfitting and ensure that the model retains general knowledge while being personalized.

[0140] S800: Continuously update and optimize the personal motion feature model based on the user's subsequent training data.

[0141] Specifically, the personal motion feature model is not static; the system continuously updates and optimizes the model based on the user's subsequent training data, either through online learning or periodic batch learning. This enables the model to:

[0142] Tracking user skill progress: As users' strength, flexibility, and coordination improve through training, the model can adjust its criteria for judging "standard movements" and "abnormal risks" in a timely manner.

[0143] Adapting to short-term fluctuations in the user's state: Taking into account fluctuations in the user's state due to fatigue, recovery, etc., makes the assessment and guidance more robust.

[0144] Achieving true personalized evolution: As data continues to accumulate, the model increasingly resembles the user's "digital motion twin," providing increasingly accurate assessments, predictions, and guidance.

[0145] Example 2

[0146] A motion performance analysis and guidance system based on multimodal data fusion includes:

[0147] The data acquisition module is used to collect multimodal physiological and motor data related to user performance.

[0148] This module integrates multiple sensors, including:

[0149] Physiological electrical signal sensors: such as surface electromyography sensors, used to collect electrical signals generated by muscle activity.

[0150] Kinematic sensors, such as inertial measurement units, are used to collect acceleration, angular velocity, and attitude data of various parts of the body; and / or optical motion capture systems.

[0151] Dynamic sensors, such as force tables and pressure insoles, are used to collect data on ground reaction forces and pressure distribution.

[0152] The feature extraction module is used to process the multimodal physiological and motion data to extract multidimensional motion features, including muscle activity features, kinematic features and dynamic features.

[0153] This module extracts characteristic indicators with clear physical meaning from the raw data using signal processing algorithms, including:

[0154] Muscle activity characteristics: such as muscle activation timing, integrated electromyography value, median frequency, etc.

[0155] Kinematic characteristics: such as joint angles, angular velocity, angular acceleration, movement trajectory, speed, rhythm, etc.

[0156] Dynamic characteristics: such as peak force, impulse, pressure center trajectory, force generation rate, etc.

[0157] The analysis and prediction module is used to perform fusion analysis and prediction based on the multi-dimensional motion features through an artificial intelligence model, and generate analysis results that include predictions of future motion states and / or motion quality assessments.

[0158] This module is the core of the system, and its functions include:

[0159] Multimodal feature fusion: Heterogeneous features from different sensors are aligned and fused within a unified spatiotemporal framework to construct a comprehensive user state vector. Action quality assessment: Based on the fused features, a trained classification or regression model (such as Support Vector Machine, Random Forest, or Deep Learning Network) is used to quantitatively score the technical accuracy, cost-effectiveness, and efficiency of the current action. Future action state prediction: Using temporal models (such as Long Short-Term Memory Network LSTM or Gated Recurrent Unit GRU), based on current and historical feature sequences, the system predicts the user's potential state in the near future, such as action distortion, muscle fatigue, and the resulting risk of injury.

[0160] The guidance output module is used to generate and output guidance information for users based on the analysis results.

[0161] This module transforms the abstract results output by the analysis and prediction module into concrete, actionable guidance plans, which are then delivered to users or coaches in real-time or near real-time through visual interfaces (such as tablets, augmented reality AR glasses), voice prompts, or haptic feedback (such as vibrations from smart bracelets). Guidance information may include movement correction suggestions, strength training plans, rest interval reminders, or injury risk warnings.

[0162] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Therefore, any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for sports performance analysis and guidance based on multi-modal data fusion, characterized in that, The method comprises the following steps: Collecting multi-modal physiological and motion data related to the user's motion performance; Processing the multi-modal physiological and motion data to extract multi-dimensional motion features, including muscle activity features, kinematic features, and kinetic features; Based on the multi-dimensional motion features, performing fusion analysis and prediction through an artificial intelligence model to generate analysis results containing future motion state prediction and / or motion quality assessment; Based on the analysis results, generating and outputting guidance information for the user.

2. The method according to claim 1, wherein, The collection of multi-modal physiological and motion data related to the user's motion performance specifically includes the following steps: Obtaining joint time series data based on motion sequence image processing, including 2D or 3D joint time series data; Obtaining electromyographic signals collected based on flexible sensors, which represent the activation state of target muscle groups during the motion process; Obtaining plantar pressure distribution data collected based on pressure sensing units, including static pressure values and dynamic pressure change sequences. 3.The method of claim 1, wherein, The processing of the multi-modal physiological and motion data to extract multi-dimensional motion features specifically includes the following steps: Processing the obtained joint time series data, electromyographic signals, and plantar pressure distribution data to form a multi-modal data set with unified timestamps; Preprocessing each modality of raw data in the multi-modal data set, including filtering and enhancing electromyographic signals, smoothing and denoising joint time series data, and standardizing plantar pressure distribution data; Based on a pre-set standard motion template, dividing the user's continuous motion stream into multiple action stages using joint time series data and / or plantar pressure distribution data; In each action stage, the multi-dimensional motion features are extracted from different modalities of data.

4. The method of claim 3, wherein, The multi-dimensional motion features at least include: Muscle activity features extracted from the electromyographic signals, including muscle activation intensity, activation timing, and fatigue indicators; Kinematic features extracted from the joint time series data, including joint angles, angular velocities, joint coupling relationships, and center of mass trajectories; Kinetic features extracted from the plantar pressure distribution data, including pressure center trajectories, impact force peaks, and pressure propagation speeds.

5. The method of claim 1, wherein, The fusion analysis and prediction through an artificial intelligence model based on the multi-dimensional motion features to generate analysis results containing future motion state prediction and / or motion quality assessment specifically includes the following steps: Inputting the multi-dimensional motion features into a multi-channel deep spatio-temporal graph neural network; Learning the spatial relationships between human joints through the spatial graph convolution layer in the multi-channel deep spatio-temporal graph neural network, and learning the dynamic evolution patterns of each joint through the temporal convolution layer, the muscle activity features, kinematic features, and kinetic features are dynamically interacted and weighted fused to generate a deep fused spatio-temporal feature representation; Based on the deep fused spatio-temporal feature representation, predicting the user's key joint angles through a regression prediction branch; Based on the deep fusion spatio-temporal feature representation, the completion quality of the user action is evaluated by a classification evaluation branch to perform a level evaluation; The joint angle prediction value and the quality level are integrated to generate an analysis result.

6. The method of claim 1, wherein, Based on the analysis result, guidance information for the user is generated and output, specifically including the following steps: Based on the joint angle prediction value and the quality level, real-time feature data is combined for judgment; When at least one of the following conditions is met, guidance of the corresponding level is triggered: The predicted key joint angle deviates from the preset angle threshold; The related muscle activity feature shows insufficient activation, excessive activation, or timing abnormalities; The plantar dynamics feature shows that the motion stability is lower than the preset standard; According to the triggered guidance level, corresponding prompt content and interaction instructions are generated.

7. The method of claim 6, wherein the method further comprises: The guidance level at least includes: Level one prompt, corresponding to a slight deviation in the predicted action, generates visual or audio prompt instructions; Level two prompt, corresponding to a larger deviation in the predicted action, generates enhanced visual prompts and specific voice prompt instructions; Level three prompt, corresponding to a significant deviation of the predicted action from the standard and a risk, generates warning visual prompts and adjustment suggestions.

8. The method of claim 6, wherein the method further comprises: The generation of corresponding prompt content and interaction instructions specifically includes the following steps: Compare the user's real-time or predicted skeletal posture with the standard action skeletal posture, and label the difference parts; Generate a muscle activation heat map based on real-time muscle activity data, and label abnormal muscle states with icons; In the data panel, display the predicted joint angle, quality rating, and text improvement suggestions. 9.The sports performance analysis and guidance method based on multi-modal data fusion according to claim 1, characterized in that, Also includes: Generate an analysis report based on historical data, including action quality score, risk trend chart, and guidance event record; Based on the improvement direction in the analysis report, select corresponding training actions from the preset training action library to generate a personalized training plan; Based on the user's physical feature data, historical motion feature parameters, and model personalization configuration, build a personal motion feature model; According to the user's subsequent training data, continuously update and optimize the personal motion feature model.

10. A sports performance analysis and coaching system based on multi-modal data fusion, characterized in that, It includes: A data acquisition module for acquiring multi-modal physiological and motion data related to user performance; A feature extraction module for processing the multi-modal physiological and motion data to extract multi-dimensional motion features, including muscle activity features, kinematic features, and dynamic features; An analysis and prediction module for fusion analysis and prediction based on the multi-dimensional motion features through an artificial intelligence model to generate an analysis result containing future action state prediction and / or action quality evaluation; A guidance output module for generating and outputting guidance information for the user based on the analysis result.

Images