Intelligent motion posture correction and risk warning system based on multi-modal data fusion

Abstract

The application discloses an intelligent motion posture correction and risk early warning system based on multi-modal data fusion and belongs to the technical field of computer vision and motion analysis. In view of the problems of static risk assessment, black-box decision process and lack of explainability of correction guidance existing in the existing motion monitoring system, the application comprises a multi-modal data acquisition module, a physical-physiological coupling perception module, a dynamic risk field evaluation module and an explainable decision feedback module. A coupling dynamic model is formed by constructing a causal correlation network of physical characteristics and physiological signals; a time-varying risk field is generated according to factors such as motion state and individual fatigue attention, and dynamic safety range adjustment is performed; physical abnormalities are located based on the risk field, physiological causes are traced back, and explainable feedback instructions containing action adjustment and muscle control suggestions are generated.

Description

Technical Field

[0001] This invention discloses an intelligent motion posture correction and risk warning system based on multimodal data fusion, belonging to the field of computer vision and motion analysis technology. Background Technology

[0002] Currently, motion posture capture and correction systems based on visual sensors have been widely used in sports training, rehabilitation medicine, and other fields. Existing technologies, such as CN120604979A, have achieved the monitoring of muscle thermal gradients and joint trajectories during movement by fusing infrared thermal imaging, RGB-D images, and bioelectric sensor data. However, their early warning mechanism relies on a preset dual-threshold model, triggering an alarm when the data exceeds the threshold. This fails to deeply analyze the intrinsic relationship between posture deviations and changes in physiological signals, resulting in early warning results that lack the ability to trace the causes of injury and are difficult to provide targeted corrective guidance.

[0003] Regarding risk assessment methods, CN121095835A discloses a sports risk classification and early warning system that integrates video, heart rate, and facial expression features to comprehensively judge the exerciser's state through multi-dimensional data. However, this scheme simply weights and fuses different modal features, and the risk assessment model is based on a static rule base. It cannot dynamically adjust the safety range according to the exerciser's real-time physiological state and fatigue accumulation. When the exerciser is in a fatigued state, the initially set risk threshold is still used, which may result in missed or false alarms, affecting the accuracy and reliability of the early warning system.

[0004] Regarding decision-making feedback mechanisms, CN121122572A constructs a motion monitoring system architecture that includes an AI decision-making layer and a multi-dimensional feedback layer, mentioning feedback forms such as transcutaneous electrical nerve stimulation feedback and augmented reality guidance. However, this patent only discloses the composition framework of functional modules and does not detail how the core early warning model transforms the multimodal fusion results into interpretable information that users can understand. The feedback instructions are mostly preset general prompts, lacking personalized adjustments based on individual physiological characteristics and exercise habits. This results in exercisers only being aware of the superficial symptoms of postural errors, making it difficult to understand the root cause of the errors and the correct adjustment methods.

[0005] In summary, there is an urgent need for an intelligent motion posture correction and risk warning system based on multimodal data fusion to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent motion posture correction and risk warning system based on multimodal data fusion. This system includes a multimodal data acquisition module, a physical-physiological coupling sensing module, a dynamic risk field assessment module, and an interpretable decision feedback module. The multimodal data acquisition module is used to simultaneously acquire the athlete's posture images, inertial data, electromyographic signals, and thermal imaging data. The physical-physiological coupling sensing module is used to perform joint feature extraction on the aforementioned multimodal data and construct a causal correlation network between physical features and physiological signals to form a physical-physiological coupling dynamic model. The dynamic risk field assessment module is used to generate a time-varying risk field based on the coupled dynamic model, combined with individual fatigue, attention, and environmental factors, and to dynamically adjust the safety range. The interpretable decision feedback module is used to locate the physical characteristics of the risk source based on the time-varying risk field and trace its physiological causes to generate interpretable feedback instructions containing suggestions for movement adjustment and muscle control. This invention achieves a technological leap from static threshold alarms to dynamic risk assessment, and from black-box decision-making to interpretable guidance.

[0007] The objective of this invention can be achieved through the following technical solutions: The intelligent motion posture correction and risk warning system based on multimodal data fusion includes a multimodal data acquisition module, a physical-physiological coupling sensing module, a dynamic risk field assessment module, and an interpretable decision feedback module. The multimodal data acquisition module is used to synchronously acquire the athlete's posture and physiological signals through multiple sensors to obtain raw multimodal data. The physical-physiological coupling sensing module is used to perform data fusion and feature extraction on athletes using the original multimodal data, and to construct a physical-physiological causal relationship network and a physical-physiological coupling dynamic model. The dynamic risk field assessment module is used to construct a dynamic risk field based on the physical-physiological coupling dynamic model, considering the athlete's current exercise state and individual fatigue, attention, and environmental factors, and to generate a time-varying risk field. The interpretable decision feedback module is used to generate interpretable risk tracing and personalized correction instructions for athletes based on the time-varying risk field, and to create interpretable feedback instructions.

[0008] Preferably, the multimodal data acquisition module includes an attitude acquisition unit: The system acquires raw posture image data by using a visual sensor to capture images of the athlete's posture; and acquires raw motion trajectory data by using an inertial sensor to capture motion trajectory data of the athlete's posture.

[0009] Preferably, the multimodal data acquisition module includes a physiological acquisition unit: The exercise involves acquiring electrical signals from the athlete's muscle activity using a bioelectric sensor to obtain raw electromyographic data, and acquiring raw thermal imaging data by using an infrared thermal imaging sensor to obtain thermal imaging data of the athlete's body surface temperature.

[0010] Preferably, the physical-physiological coupling sensing module includes a physical feature extraction unit: Based on the original multimodal data, the skeletal points of the athlete are extracted to obtain physical motion features; the joint angles of the athlete are calculated according to the original multimodal data to obtain physical joint features.

[0011] Preferably, the physical-physiological coupling sensing module includes a physiological feature extraction unit: Based on the original multimodal data, time-frequency domain analysis is performed on the electromyography signals of the athletes to obtain physiological fatigue characteristics; and temperature field analysis is performed on the thermal imaging data of the athletes using the original multimodal data to obtain physiological metabolic characteristics.

[0012] Preferably, the physical-physiological coupling sensing module includes a causal association building unit: The physical-physiological coupling sensing module includes a causal association construction unit, which is used to extract physical and physiological features of the athlete using the original multimodal data and construct a causal association network of musculoskeletal dynamics and physiological signals to form the physical-physiological coupling dynamic model. By performing skeletal point detection on posture images using computer vision algorithms, joint coordinates and motion trajectories are obtained; precise joint angles are calculated using inertial data; muscle activation level and fatigue index are extracted using surface electromyography signals; muscle metabolic heat distribution is obtained using infrared thermography; based on time-aligned multimodal features, a causal correlation network between physical features and physiological signals is constructed using structural equation modeling, revealing the causal influence path of muscle fatigue on joint displacement and the transmission mechanism of abnormal metabolic heat on injury risk; through quantitative analysis of causal strength and network parameter learning, a dynamic model representing the intrinsic coupling relationship between physical movement and physiological state is finally formed, providing a comprehensive motion representation that integrates biomechanics and physiological response for risk assessment.

[0013] Preferably, the dynamic risk field assessment module includes a motion state assessment unit: The current motion trajectory of the athlete is calculated in real time using the physical-physiological coupled dynamic model to obtain the current kinematic state; the current physiological indicators of the athlete are monitored in real time using the physical-physiological coupled dynamic model to obtain the current physiological state.

[0014] Preferably, the dynamic risk field assessment module includes a risk field construction unit: The dynamic risk field assessment module includes a risk field construction unit, which is used to calculate the risk field of the athlete based on the current kinematic state and individual fatigue, attention and environmental factors, and generate a time-varying risk field. Based on the current physiological state and individual fatigue, attention, and environmental factors, the system dynamically adjusts the safety range for the athlete, correcting the time-varying risk field. It receives kinematic parameters such as joint coordinates, angles, and angular velocities, while simultaneously collecting variables such as fatigue index obtained from heart rate variability, focus level obtained from facial recognition, physical fatigue level, and environmental friction coefficient. Using an artificial potential field method, it calculates the risk potential energy distribution in the movement space, generating a risk field centered on the athlete, with peak potential energy corresponding to high-risk injury areas. Based on physiological states such as muscle fatigue index and abnormal soft tissue temperature, it dynamically adjusts the safety boundary, shrinking the safety domain of corresponding joints when fatigue accumulates and expanding the high-risk area when focus decreases. Through dynamic calibration of the risk field parameters, it ultimately generates a time-varying risk field that responds in real-time to physiological changes and environmental conditions, achieving a leap from static threshold alarms to dynamic risk assessment.

[0015] Preferably, the interpretable decision feedback module includes a risk tracing unit: The interpretable decision feedback module includes a risk tracing unit, which is used to locate the risk source of the athlete based on the time-varying risk field and obtain the physical characteristics of the risk source. By tracing the causes of risk in athletes through the time-varying risk field, the physiological characteristics of the risk sources are obtained. Spatial gradient analysis and extreme point detection are performed on the potential energy distribution of the risk field to locate the joint region and movement phase with the highest risk potential energy, obtaining the spatial location and posture characteristics of the risk sources, including kinematic parameters such as the name of the joint that causes high risk, the direction and amplitude of angular deviation, and abnormal trajectory patterns. Based on the causal relationship network in the physical-physiological coupling dynamic model, the causes of risk are traced back, and the physical characteristics of the risk sources are used as target variables to propagate backward along the causal path to identify key physiological factors that lead to physical abnormalities, including deep indicators such as the fatigue index of the dominant muscle group that causes joint displacement, abnormal muscle activation timing, and the degree of local metabolic heat accumulation. Through joint analysis of physical abnormalities and physiological causes, a risk source feature set containing both physical phenomena and physiological mechanisms is finally obtained, providing an interpretable diagnostic basis for personalized correction strategies.

[0016] Preferably, the interpretable decision feedback module includes an instruction generation unit: The interpretable decision feedback module includes an instruction generation unit, which is used to match the athlete with a personalized correction strategy based on the physical characteristics and physiological characteristics of the risk source, and generate personalized correction instructions. Based on the personalized correction instructions, multimodal feedback encoding is applied to the athlete to construct interpretable feedback instructions. A knowledge base storing biomechanical principles and rehabilitation experience is accessed, and risk source characteristics are matched with knowledge base rules in a multidimensional way. Taking into account the athlete's historical data and physical condition, the optimal correction strategy is selected, generating dual guidance instructions that include movement adjustment requirements and muscle control suggestions. Based on the athlete's environment and terminal capabilities, the instructions are multimodal feedback encoded, transforming them into augmented reality visual guidance lines, voice prompts, and wearable device tactile vibration signals. Through cross-modal mapping and synchronous encoding of the personalized correction instructions, interpretable feedback instructions in multiple forms, including visual, auditory, and tactile senses, are finally constructed, achieving both cognitive and executive correction effects.

[0017] The beneficial effects of this invention are: This invention constructs a causal correlation network between physical features and physiological signals, enabling in-depth exploration of the intrinsic relationship between movement posture deviations and physiological factors such as muscle fatigue and metabolic abnormalities. This allows the system to not only perceive posture deviations but also trace the physiological root causes of these deviations, significantly improving the depth of analysis and diagnostic accuracy of sports injury risks.

[0018] This invention constructs a time-varying risk field by introducing individual fatigue, attention, and environmental factors, and dynamically adjusts the safety range based on the athlete's real-time physiological state. This enables risk assessment to adaptively evolve with fatigue accumulation, changes in focus, and changes in environmental conditions, effectively overcoming the shortcomings of missed and false alarms in static threshold alarms, and significantly improving the dynamic adaptability and accuracy of risk warning.

[0019] This invention generates interpretable feedback instructions that include movement adjustment requirements and muscle control suggestions by locating risk sources and tracing physiological causes. It also implements visual, auditory, and tactile multimodal coding, enabling athletes to not only perceive the appearance of postural errors but also understand the root causes and correct adjustment methods, significantly improving the cognitive depth and execution effect of exercise posture correction. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the architecture of the intelligent motion posture correction and risk warning system based on multimodal data fusion of the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Example: Figure 1 As shown, the intelligent motion posture correction and risk warning system based on multimodal data fusion includes a multimodal data acquisition module, a physical-physiological coupling sensing module, a dynamic risk field assessment module, and an interpretable decision feedback module. The multimodal data acquisition module is used to synchronously acquire the athlete's posture and physiological signals through multiple sensors to obtain raw multimodal data. The physical-physiological coupling sensing module is used to perform data fusion and feature extraction on athletes using the original multimodal data, and to construct a physical-physiological causal relationship network and a physical-physiological coupling dynamic model. The dynamic risk field assessment module is used to construct a dynamic risk field based on the physical-physiological coupling dynamic model, considering the athlete's current exercise state and individual fatigue, attention, and environmental factors, and to generate a time-varying risk field. The interpretable decision feedback module is used to generate interpretable risk tracing and personalized correction instructions for athletes based on the time-varying risk field, and to create interpretable feedback instructions.

[0023] In this embodiment, the exerciser's posture and physiological signals are simultaneously acquired via multiple sensors to obtain raw multimodal data. The specific implementation method is as follows: The multimodal data acquisition module is responsible for high-precision synchronous acquisition of the athlete's posture information and physiological signals, providing basic data support for subsequent coupled sensing and risk assessment. This module consists of a posture acquisition unit and a physiological acquisition unit, deployed respectively in the athlete's surrounding environment and on the wearable device. The posture acquisition unit employs a fusion of visual and inertial sensors. The visual sensor uses two high-frame-rate industrial cameras to form a binocular stereo vision system, positioned 2.5 meters directly in front of the athlete. The camera intrinsic parameter matrix K, distortion coefficient vector D, and binocular system rotation matrix R and translation vector T are pre-acquired using a checkerboard calibration method. The inertial sensor uses a nine-axis inertial measurement unit, including a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, with a sampling frequency set to 200Hz. It is fixed to the athlete's lumbosacral region and key positions on both thighs and calves using elastic straps to capture limb movement trajectories. The physiological data acquisition unit includes a bioelectric sensor and an infrared thermal imaging sensor. The bioelectric sensor uses surface electromyography (EMG) electrodes, attached to the center of the target muscle belly with a 20mm electrode spacing, and a reference electrode placed at a bony prominence. The infrared thermal imaging sensor uses an uncooled microbolometer focal plane array, mounted 2 meters to the side of the athlete, with the lens optical axis perpendicular to the athlete's coronal plane. Blackbody calibration ensures accurate temperature measurement. To ensure temporal consistency of multimodal data, all sensors are connected to the same synchronous trigger controller, achieving microsecond-level synchronous acquisition through hardware trigger signals. The rising edge of the visual sensor's trigger pulse simultaneously activates the inertial sensor's data buffer and the bioelectric sensor's sampling window. The infrared thermal imaging sensor uses frame synchronization mode and is aligned with the visual sensor. The acquired raw multimodal data is uploaded in real-time to a local workstation via a high-speed data transmission interface for storage and preliminary processing.

[0024] In the posture acquisition unit, the pose images of the mover acquired by the vision sensor need to undergo stereo matching and 3D reconstruction to obtain the 3D spatial coordinates of the joint points. Let the pixel coordinates of a certain joint point in the left camera image be... The pixel coordinates of the matching point in the right camera image are Based on the principle of binocular visual triangulation, the three-dimensional coordinates of this point are... : ,in, , denoted by , where is the equivalent focal length of the camera in the x and y directions, and b is the baseline length of the binocular system, obtained from the camera calibration parameters. The raw accelerometer data is acquired by the inertial sensor. gyroscope data The rotation matrix of the sensor coordinate system relative to the world coordinate system needs to be obtained through quaternion attitude calculation. The Mahony complementary filtering algorithm is used to fuse accelerometer and gyroscope data, and the filtered attitude quaternion is then obtained. Iterative updates are performed using the formula: In the formula, To represent quaternion multiplication, The angular velocity vector after accelerometer correction is expressed as follows: ,in The angular velocity measured by the gyroscope. The error vector between the accelerometer observation and the projection of the gravity direction. , These are the proportional-integral coefficients. The calculated attitude quaternions can be used to transform accelerometer data to the world coordinate system, and then the motion trajectory of the joints can be obtained through quadratic integration. The visual 3D coordinates and the inertial trajectory are fused using a Kalman filter. The visual coordinates are used as observations to correct for inertial integral drift, and the final output is accurate raw motion trajectory data, including the 3D position time sequence of each joint. And the corresponding linear velocity and angular velocity information.

[0025] In the physiological acquisition unit, the raw electromyographic signals acquired by the bioelectric sensor are microvolt-level voltage time-series data, denoted as... , where i represents the i-th channel, corresponding to a specific muscle. Since electromyography (EMG) signals are susceptible to power frequency interference, motion artifacts, and baseline drift, pre-filtering is necessary. A fourth-order Butterworth bandpass filter is used to remove motion artifacts below 20Hz and high-frequency noise above 500Hz, while a 50Hz notch filter is used to eliminate power frequency interference. The filtered EMG signal is used to extract muscle activation features, where the root mean square (RMS) is a commonly used indicator of muscle exertion intensity. In the formula, T is the length of the sliding window, typically taken as 100ms to 200ms to accommodate the dynamic characteristics of muscle contraction. Besides RMS, the median frequency of the electromyographic signal can also be calculated to assess muscle fatigue. The raw data acquired by the infrared thermal imaging sensor is the digital grayscale value output by the detector, which needs to be converted into temperature values ​​after non-uniformity correction and temperature calibration. The radiative transfer model established using the blackbody radiation law determines the grayscale value of each pixel. Surface temperature at corresponding point The linear relationship satisfied is: In the formula, These are calibration coefficients, obtained from two-point or multi-point temperature calibration experiments. The corrected thermal imaging temperature matrix. It can reflect the changes in local muscle temperature distribution during exercise, providing a basis for metabolic activity analysis. The raw multimodal data output by the multimodal data acquisition module includes: posture image data—binocular image sequences, raw motion trajectory data—joint 3D coordinates and inertial data, raw electromyography data—signals before multi-channel filtering, and raw thermal imaging data—temperature matrix. All data are encapsulated into data frames with a unified timestamp and transmitted to the physical-physiological coupling sensing module via a fiber optic network, laying the foundation for subsequent feature extraction and causal correlation construction.

[0026] In this embodiment, data fusion and feature extraction are performed on the athletes using the original multimodal data, and a physical-physiological causal relationship network is constructed to build a physical-physiological coupled dynamic model. The specific implementation method is as follows: The physical-physiological coupling sensing module is responsible for performing deep feature extraction and causal correlation modeling on the raw multimodal data output by the multimodal data acquisition module, constructing a dynamic model that can characterize the intrinsic coupling relationship between physical motion and physiological state. This module consists of a physical feature extraction unit, a physiological feature extraction unit, and a causal correlation construction unit connected sequentially. The physical feature extraction unit receives posture image data and raw motion trajectory data from the raw multimodal data, and extracts the physical motion features and physical joint features of the athlete using computer vision algorithms and kinematic calculation methods. The physiological feature extraction unit receives raw electromyography data and raw thermal imaging data, and extracts the physiological fatigue features and physiological metabolic features of the athlete using time-frequency domain analysis and temperature field analysis methods. The causal correlation construction unit takes the time-aligned physical and physiological features as input, uses structural equation modeling to construct a causal correlation network between physical features and physiological signals, estimates the causal path coefficients and learns the network parameters, and finally outputs a physical-physiological coupling dynamic model, providing a comprehensive motion representation integrating biomechanics and physiological response for subsequent dynamic risk assessment.

[0027] In the physical feature extraction unit, human skeleton point detection is first performed on the binocular vision image sequence. A deep learning-based pose estimation network is used to locate the joints of the mover in each frame of the image. After obtaining the two-dimensional pixel coordinates, the three-dimensional spatial coordinates of the joints are reconstructed by combining the binocular vision triangulation principle. Let the three-dimensional coordinates of the j-th joint at time t be... The physical motion characteristics include the three-dimensional position time sequence of each joint. Linear velocity vector and linear acceleration vector The velocity and acceleration are obtained by numerically differentiating the position time sequence. Simultaneously, the physical feature extraction unit accurately calculates the joint angles based on the attitude quaternions obtained from the inertial sensor and the three-dimensional coordinates reconstructed from vision. Taking the knee joint angle as an example, let the coordinates of the thigh joint be... The coordinates of the center point of the knee joint are The coordinates of the center point of the ankle joint are Then the knee joint angle From thigh vector With calf vector Calculated using the vector dot product formula: Similarly, the flexion-extension angles, abduction-inversion angles, and rotation angles of major joints such as the hip, ankle, shoulder, and elbow can be obtained, forming a set of physical joint characteristics. Where J is the total number of joints. The physical motion features and physical joint features output by the physical feature extraction unit together form the physical feature vector. This vector dimension can reach tens to hundreds of dimensions, comprehensively depicting the instantaneous motion state of the athlete.

[0028] The physiological feature extraction unit receives raw electromyography (EMG) data and raw thermal imaging data, and performs time-frequency domain analysis on the multi-channel surface EMG signals to extract physiological fatigue features. First, the filtered EMG signals... Perform a sliding window Fourier transform to obtain the power spectral density function for each window. ,in For frequency variables. A typical manifestation of muscle fatigue is a shift in the power spectrum towards lower frequencies; therefore, the median frequency is an effective indicator for quantifying the degree of fatigue, defined as the frequency value that divides the power spectrum into two parts with equal energy: In the formula, The median frequency of the electromyographic signal in channel i at time t gradually decreases with accumulated fatigue. In addition to the median frequency, the root mean square (RMS) value can also be extracted. Characterizes muscle exertion intensity and integrated electromyography (EMG) value. Characterizes the total amount of work done by muscles. The physiological fatigue characteristic vector is denoted as... Where M is the number of electromyography channels. For infrared thermal imaging data, the physiological feature extraction unit extracts the thermal imaging temperature matrix. Perform temperature field analysis to extract the temperature statistical characteristics of the muscle region of interest. Let the region of interest corresponding to the m-th muscle be... The average temperature in this region With temperature variance : , In the formula, The total number of pixels in the region of interest. Increased muscle metabolic activity during movement leads to a rise in local temperature; therefore, the mean temperature reflects the level of metabolic activity, while the temperature variance characterizes the uniformity of heat distribution. The physiological metabolic feature vector is denoted as... The physiological feature vector is the final output of the physiological feature extraction unit. Composed of fatigue characteristics and metabolic characteristics: .

[0029] Causal correlation building blocks receive physical feature vectors With physiological characteristic vector First, time alignment and normalization are performed on the two types of features to ensure that all feature components are on the same dimension and time reference. Then, a structural equation model is used to construct a causal relationship network between physical features and physiological signals. The structural equation model consists of two parts: a measurement model and a structural model. The measurement model describes the relationship between latent variables and observed variables, while the structural model describes the causal paths between latent variables. In this embodiment, kinematic parameters such as joint angles and joint angular velocities are used as observed variables, and muscle fatigue index and metabolic heat distribution parameters are used as endogenous latent variables to construct a path graph describing the causal influence of muscle fatigue on joint angle offset. Let the latent variable vector be... The observed variable vector is Structural equation model: In the formula, B is the path coefficient matrix between endogenous latent variables. Let be the influence coefficient matrix of exogenous latent variables on endogenous latent variables, and let x be the vector of exogenous latent variables. The structural model residual vector; This is the factor loading matrix, describing the relationship between observed variables and latent variables. To measure the residual vector of the measurement model, maximum likelihood estimation or Bayesian estimation is performed on multiple sets of time series data to obtain the estimated values ​​and significance levels of each path coefficient, thereby quantifying the causal influence of muscle fatigue on joint displacement and the transmission path coefficients of metabolic thermal abnormalities on injury risk. The strength of causal association can be expressed using path coefficients. This indicates that the subscript i corresponds to the dependent variable and the subscript j corresponds to the independent variable. The larger the absolute value, the more significant the causal influence. Through iterative learning and model optimization of the causal network, a physical-physiological coupling dynamic model is ultimately formed. This model takes physical and physiological features as input and outputs the causal path coefficient matrix B between each physical and physiological variable. The coupled dynamic model provides a comprehensive motion representation that integrates biomechanical and physiological responses for the subsequent dynamic risk field assessment module, enabling risk assessment to make in-depth inferences based on causal relationships rather than simple correlation analysis.

[0030] In this embodiment, a time-varying risk field is constructed based on the physical-physiological coupling dynamic model to analyze the athlete's current movement state and individual fatigue, attention, and environmental factors, thereby generating the risk field. The specific implementation method is as follows: The dynamic risk field assessment module is responsible for real-time analysis of the physical-physiological coupled dynamic model output by the physical-physiological coupled sensing module. Combining the athlete's current movement state, physiological state, and individual fatigue, attention, and environmental factors, it constructs a dynamically evolving, time-varying risk field, achieving a technological leap from static threshold alarms to dynamic risk assessment. This module consists of a movement state assessment unit and a risk field construction unit connected sequentially. The movement state assessment unit receives the physical-physiological coupled dynamic model and real-time acquired multimodal data, performs real-time calculation and monitoring of the athlete's current movement trajectory and physiological indicators, and outputs the current kinematic and physiological states. The risk field construction unit, based on the current kinematic and physiological states and individual fatigue, attention, and environmental factors, uses the artificial potential field method to calculate the risk potential energy distribution in the movement space, generating an initial risk field. It then dynamically adjusts the safety boundary according to physiological state and environmental factors, ultimately outputting a real-time updated time-varying risk field, providing quantitative evidence for subsequent interpretable decision feedback.

[0031] The motion state assessment unit first acquires real-time raw data from the multimodal data acquisition module, including the current frame image acquired by the visual sensor, acceleration and angular velocity data output by the inertial sensor, surface electromyography signals, and infrared thermal imaging data. Using the feature extraction network in the pre-constructed physical-physiological coupled dynamic model, physical and physiological features are quickly extracted from the current frame data. Specifically, a lightweight pose estimation network is used to obtain the three-dimensional coordinates of each joint at the current moment. And calculate the instantaneous velocity based on the position difference between adjacent frames. With acceleration The current joint angle is calculated using joint coordinates. For example, the knee joint angle is determined by the thigh vector. With calf vector Calculate using the formula: In the formula, , The current set of kinematic states is denoted as... Simultaneously, bandpass filtering and sliding window processing are applied to the real-time electromyography (EMG) signals to extract the root mean square (RMS) value at the current moment. With median frequency Extract the average temperature of the region of interest from the thermal imaging data. With variance To obtain the current physiological state set The motion state assessment unit outputs the above states to the risk field construction unit in real time.

[0032] The risk field construction unit first collects individual fatigue and attention environmental factors, including the mental fatigue index obtained through heart rate variability analysis. Attention score obtained through facial expression recognition The physical fatigue level is obtained by integrating exercise duration and intensity. and the environmental ground friction coefficient obtained through external sensors. With instrument stability parameters After normalizing the above factors, an environmental state vector is formed. Subsequently, an artificial potential field method was used to construct a risk field centered on the current position of the mover. The motion space was discretized into a three-dimensional grid, with each grid point... Risk potential energy It consists of two parts: one part is the gravitational potential energy from the target posture, guiding the athlete towards the standard posture; the other part is the repulsive potential energy from the current deviation and physiological abnormality, representing the risk of injury. To simplify the calculation, this embodiment directly defines the risk potential energy function based on the current kinematic and physiological states: In the formula, the first term is the gravitational term. Let i be the reference position of the i-th joint in the standard pose. The first term represents the weighting coefficient for the corresponding joint; the second term represents the risk repulsion term. For the first The risk contribution generated by a risk source at location r This is a dynamically adjusted coefficient. In practical applications, the risk sources are mainly concentrated around the athlete's body; therefore, the risk field can be simplified as the superposition of potential energy centered on each joint. Focusing on the combined effects of joint angle deviation and muscle fatigue, the risk potential energy of each joint is defined as follows: In the formula, For the current joint angle, For reference angle, The maximum allowable deviation range; The dynamic weights related to fatigue are determined by the muscle fatigue index, such as... ,in This is the magnification factor; The spatial scale representing the scope of risk impact. Overall risk field. It is obtained by superimposing the risk potential energy of all joints: The potential energy peak region in the risk field corresponds to the current high-risk damage region, and the potential energy gradient direction indicates the direction of the fastest increase in risk.

[0033] After the initial risk field is constructed, the risk field construction unit dynamically adjusts the safety boundaries based on the current physiological state and environmental factors. The safety boundaries are defined as the permissible range of motion for each joint, with initial values ​​provided by a standard posture database. When the physiological state indicates that muscle fatigue has accumulated beyond a threshold, the safety range of the corresponding joint should contract, i.e., decrease. When concentration decreases, the high-risk area should be expanded, i.e., increased. or improve The dynamic adjustment rule can be expressed as: , In the formula, , To adjust the coefficient. Simultaneously, the environmental friction coefficient... When the value is too low, the base value of the overall risk potential energy can be increased. This can be achieved by adjusting the risk field parameters. After dynamic calibration is implemented, the risk potential energy distribution is recalculated to generate a time-varying risk field that can respond in real time to physiological changes and environmental conditions. This risk field, centered on the athlete, evolves dynamically over time, with its potential energy value directly reflecting the degree of injury risk under the current exercise posture and physiological state. The risk field construction unit outputs the final generated time-varying risk field to the interpretability decision feedback module, providing a quantitative basis for subsequent risk tracing and personalized correction.

[0034] In this embodiment, interpretable risk tracing and personalized correction instructions are generated for athletes based on the time-varying risk field, and interpretable feedback instructions are created. The specific implementation method is as follows: The interpretable decision feedback module is responsible for in-depth analysis of the time-varying risk field output by the dynamic risk field assessment module. Combining this with the causal relationship network in the physical-physiological coupling dynamic model, it achieves precise location of risk sources and traces their physiological causes. Based on the characteristics of the risk sources, it generates personalized and interpretable corrective instructions, which are then conveyed to the athlete through multimodal feedback. This module consists of a risk tracing unit and an instruction generation unit connected sequentially. The risk tracing unit receives the time-varying risk field. The system performs spatial gradient analysis and extreme point detection on the risk potential energy distribution to locate the joint region and movement phase with the highest risk potential energy, thereby obtaining the physical characteristics of the risk source. Simultaneously, based on the causal relationship network in the physical-physiological coupling dynamic model, the physical characteristics of the risk source are used as the target variable and propagated backward along the causal path to identify the key physiological factors leading to physical abnormalities, thus obtaining the physiological characteristics of the risk source. The instruction generation unit, based on the physical and physiological characteristics of the risk source, accesses a personalized correction strategy knowledge base for multidimensional matching, generating personalized correction instructions that include movement adjustment requirements and muscle control suggestions. These instructions are then multimodal feedback encoded, ultimately constructing interpretable feedback instructions in multiple forms, including visual, auditory, and tactile feedback. These instructions are then transmitted to augmented reality display devices, voice broadcasting devices, and wearable vibration devices, providing exercisers with corrective guidance at both cognitive and executive levels.

[0035] The risk tracing unit first examines the time-varying risk field. Perform spatial gradient analysis to calculate the risk potential energy gradient vector. The gradient direction indicates the direction of the fastest increase in risk, and the gradient magnitude... It reflects the drastic nature of risk changes. By detecting extreme points in the gradient field, the local maxima of the risk potential energy are located. This point corresponds to the spatial location with the highest risk of damage at the current moment. The current joint positions of the athlete Perform nearest neighbor matching to determine the joint index to which the risk source belongs. Simultaneously, the current kinematic parameters of the joint are analyzed, including joint angles. angular velocity and angle deviation The set of physical characteristics of the risk source is denoted as follows: [Information on electromyography and thermal imaging characteristics of the muscle group corresponding to the joint]. .

[0036] After identifying the risk source, the risk tracing unit traces the causes of the risk based on the causal relationship network in the physical-physiological coupling dynamic model. This causal relationship network is represented by a structural equation model and includes the causal path coefficient matrix B between physical and physiological variables. The joint angle deviation in the physical characteristics of the risk source. As the target variable, backpropagation is performed along the causal path to calculate the contribution of each physiological variable to the physical abnormality. Let the physiological variable vector be... This includes muscle fatigue index, muscle activation timing parameters, and metabolic heat accumulation, with the physical variable vector being... This includes joint angular deviations, angular velocities, etc. According to the structural equation model, physical variables and physiological variables satisfy a linear relationship. ,in This is a causal influence coefficient matrix. For a specific physical variable... Its sensitivity to various physiological variables can be determined by the partial derivatives. To quantify the contribution of various physiological factors to the current physical anomaly, a contribution index is introduced. , defined as the product of the degree of deviation of the physiological variable from its normal range and the causality coefficient: In the formula, This represents the real-time value of the m-th physiological variable at the current moment. This is the reference value for this physiological variable. Standard deviation, used for normalization. Contribution. A positive value indicates that the physiological factor promotes the occurrence of physical abnormalities; the larger the absolute value, the more significant the impact. The contribution of each physiological variable is ranked, and the top K physiological factors with the highest contribution are selected as the physiological characteristics of the risk source, denoted as [missing information]. Each feature is accompanied by its contribution value and the current measured value. The risk tracing unit ultimately outputs a risk source feature set containing both physical phenomena and physiological mechanisms. This provides an interpretable diagnostic basis for the generation of subsequent personalized correction instructions.

[0037] The instruction generation unit receives the risk source feature set. The system accesses a personalized correction strategy knowledge base for multi-dimensional matching. This knowledge base stores a set of correction rules built upon principles of exercise biomechanics and experience in exercise rehabilitation. Each rule includes preconditions and corresponding correction suggestions. The matching process employs multi-dimensional similarity calculation, comprehensively considering the degree of matching between risk source characteristics and rule preconditions, as well as personalized factors such as the athlete's historical data and physical condition. Let there be L rules in the knowledge base, and the... The preconditions of each rule are represented as feature vectors. Each element corresponds to a feature dimension. Current risk source features Can be converted into feature vectors of the same dimension Match degree Calculated by the formula: In the formula, For the first The weight coefficients of each feature dimension can be dynamically adjusted based on historical feedback data. For similarity functions, an indicator function is used for categorical features, and Gaussian similarity is used for numerical features. Choose the rule with the highest matching degree. The system extracts the corresponding correction suggestions for the rule and generates personalized correction instructions. These instructions contain dual guidance: a physical level instruction describing specific movement adjustment requirements, such as "reducing the knee valgus angle" or "increasing the hip flexion range"; and a physiological level instruction describing corresponding muscle control suggestions, such as "relaxing the vastus lateralis, activating the vastus medialis" or "strengthening the gluteus medius muscle activation awareness." The instruction generation unit then performs multimodal feedback encoding on the instructions based on the exerciser's environment and the capabilities of the terminal device. Assume that currently available feedback channels include augmented reality display devices, voice broadcasting devices, and wearable vibration devices. The instruction encoding process maps the abstract instruction content to specific control signals for each channel. For the visual channel, the direction and amplitude of joint angle adjustment are transformed into three-dimensional guide lines in the augmented reality scene. The starting point of the lines is the current joint position, and the ending point is the target joint position. The line color dynamically changes according to the risk level. The coordinates of the endpoints of the guide lines are... : In the formula, This is the current joint position. This represents the target position under standard orientation. The value is the angle deviation, and L is the display length scaling factor for the virtual guide line. For the auditory channel, muscle control suggestions are converted into speech synthesis signals, using a pre-recorded professional guidance voice library or real-time text-to-speech technology to generate the broadcast content. For the tactile channel, the joint and muscle information requiring prompts is mapped to vibration signals at specific frequencies and locations, with the vibration frequency... Inversely proportional to the muscle fatigue index, the more severe the fatigue, the lower the vibration frequency to attract attention. The vibration location corresponds to the wearable vibration unit near the joint that needs adjustment. By performing cross-modal mapping and synchronous encoding on personalized corrective instructions, interpretable feedback instructions in multiple forms, including visual, auditory, and tactile, are finally constructed and presented to the exerciser in real time through various terminal devices, achieving corrective effects at both the cognitive and executive levels.

[0038] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0039] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent motion posture correction and risk warning system based on multimodal data fusion, characterized in that, It includes a multimodal data acquisition module, a physical-physiological coupling sensing module, a dynamic risk field assessment module, and an interpretable decision feedback module. The multimodal data acquisition module is used to synchronously acquire the athlete's posture and physiological signals through multiple sensors to obtain raw multimodal data. The physical-physiological coupling sensing module is used to perform data fusion and feature extraction on athletes using the original multimodal data, and to construct a physical-physiological causal relationship network and a physical-physiological coupling dynamic model. The dynamic risk field assessment module is used to construct a dynamic risk field based on the physical-physiological coupling dynamic model, considering the athlete's current exercise state and individual fatigue, attention, and environmental factors, and to generate a time-varying risk field. The interpretable decision feedback module is used to generate interpretable risk tracing and personalized correction instructions for athletes based on the time-varying risk field, and to create interpretable feedback instructions.

2. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The multimodal data acquisition module includes an attitude acquisition unit: The system acquires raw posture image data by using a visual sensor to capture images of the athlete's posture; and acquires raw motion trajectory data by using an inertial sensor to capture motion trajectory data of the athlete's posture.

3. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The multimodal data acquisition module includes a physiological acquisition unit: The exercise involves acquiring electrical signals from the athlete's muscle activity using a bioelectric sensor to obtain raw electromyographic data, and acquiring raw thermal imaging data by using an infrared thermal imaging sensor to obtain thermal imaging data of the athlete's body surface temperature.

4. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The physical-physiological coupling sensing module includes a physical feature extraction unit: Based on the original multimodal data, the skeletal points of the athlete are extracted to obtain physical motion features; the joint angles of the athlete are calculated according to the original multimodal data to obtain physical joint features.

5. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The physical-physiological coupled sensing module includes a physiological feature extraction unit: Based on the original multimodal data, time-frequency domain analysis was performed on the electromyographic signals of the athletes to obtain physiological fatigue characteristics; Physiological metabolic characteristics are obtained by performing temperature field analysis on the thermal imaging data of the athlete using the original multimodal data.

6. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The physical-physiological coupling sensing module includes a causal association construction unit: The physical-physiological coupling sensing module includes a causal association construction unit, which is used to extract physical and physiological features of the athlete using the original multimodal data and construct a causal association network of musculoskeletal dynamics and physiological signals to form the physical-physiological coupling dynamic model. By performing skeletal point detection on posture images using computer vision algorithms, joint coordinates and motion trajectories are obtained; precise joint angles are calculated using inertial data; muscle activation level and fatigue index are extracted using surface electromyography signals; muscle metabolic heat distribution is obtained using infrared thermography; based on time-aligned multimodal features, a causal correlation network between physical features and physiological signals is constructed using structural equation modeling, revealing the causal influence path of muscle fatigue on joint displacement and the transmission mechanism of abnormal metabolic heat on injury risk; through quantitative analysis of causal strength and network parameter learning, a dynamic model representing the intrinsic coupling relationship between physical movement and physiological state is finally formed, providing a comprehensive motion representation that integrates biomechanics and physiological response for risk assessment.

7. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 1, characterized in that, The dynamic risk field assessment module includes a motion state assessment unit: The current motion trajectory of the athlete is calculated in real time using the physical-physiological coupled dynamic model to obtain the current kinematic state; the current physiological indicators of the athlete are monitored in real time using the physical-physiological coupled dynamic model to obtain the current physiological state.

8. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 7, characterized in that, The dynamic risk field assessment module includes a risk field construction unit: The dynamic risk field assessment module includes a risk field construction unit, which is used to calculate the risk field of the athlete based on the current kinematic state and individual fatigue, attention and environmental factors, and generate a time-varying risk field. Based on the current physiological state and individual fatigue, attention, and environmental factors, the exerciser's safety range is dynamically adjusted, and the time-varying risk field is corrected. Kinematic parameters such as joint coordinates, angles, and angular velocities are received, and variables such as fatigue index obtained from heart rate variability, focus obtained from facial recognition, physical fatigue level, and environmental friction coefficient are collected. The artificial potential field method is used to calculate the risk potential energy distribution in the movement space, generating a risk field centered on the athlete, with the peak potential energy corresponding to the high-risk area of ​​injury. The safety boundary is dynamically adjusted based on physiological states such as muscle fatigue index and abnormal soft tissue temperature. When fatigue accumulates, the safety domain shrinks for the corresponding joint range of motion, and the high-risk area expands when concentration decreases. By dynamically calibrating the risk field parameters, a time-varying risk field that responds to physiological changes and environmental conditions in real time is finally generated, realizing the leap from static threshold alarm to dynamic risk assessment.

9. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 8, characterized in that, The interpretable decision feedback module includes a risk tracing unit: The interpretable decision feedback module includes a risk tracing unit, which is used to locate the risk source of the athlete based on the time-varying risk field and obtain the physical characteristics of the risk source. By tracing the causes of risk in athletes through the time-varying risk field, the physiological characteristics of the risk sources are obtained; Spatial gradient analysis and extreme point detection are performed on the potential energy distribution of the risk field to locate the joint region and motion phase with the highest risk potential energy, and obtain the spatial location and posture characteristics of the risk source, including kinematic parameters such as the name of the joint that causes high risk, the direction and amplitude of angular deviation, and abnormal trajectory patterns. The causes of risk are traced back based on the causal relationship network in the physical-physiological coupling dynamic model. The physical characteristics of the risk source are used as the target variable and propagated backward along the causal path to identify the key physiological factors that cause physical abnormalities, including deep indicators such as the fatigue index of the dominant muscle group that causes joint displacement, abnormal muscle activation timing, and the degree of local metabolic heat accumulation. By conducting a joint analysis of physical anomalies and physiological causes, a risk source feature set containing both physical phenomena and physiological mechanisms is obtained, providing an interpretable diagnostic basis for personalized correction strategies.

10. The intelligent motion posture correction and risk warning system based on multimodal data fusion according to claim 9, characterized in that, The interpretable decision feedback module includes an instruction generation unit: The interpretable decision feedback module includes an instruction generation unit, which is used to match the athlete with a personalized correction strategy based on the physical characteristics and physiological characteristics of the risk source, and generate personalized correction instructions. Based on the personalized correction instructions, multimodal feedback coding is performed on the athlete to construct interpretable feedback instructions; The system accesses a knowledge base of personalized correction strategies that stores biomechanical principles and rehabilitation experience. It performs multidimensional matching of risk source characteristics with knowledge base rules, comprehensively considers the athlete's historical data and physical condition, selects the optimal correction strategy, and generates dual guidance instructions that include movement adjustment requirements and muscle control suggestions. Based on the athlete's environment and terminal capabilities, the instructions are multimodal feedback encoded and transformed into augmented reality visual guidance lines, voice prompts, and wearable device tactile vibration signals. By performing cross-modal mapping and synchronous encoding on personalized correction instructions, the system ultimately constructs interpretable feedback instructions in multiple forms, including visual, auditory, and tactile, to achieve dual cognitive and executive correction effects.

Images