A swimming athlete waist and back injury early warning system and method based on multi-modal time sequence information fusion

Abstract

This invention belongs to the field of sports science and sports medicine, and discloses a swimmer's back injury early warning system and method based on multimodal temporal information fusion. It includes a data acquisition module for simultaneously acquiring temporal data of the swimmer's kinematics, electromyography, and biomechanics; a data processing and feature extraction module connected to the data acquisition module for preprocessing and aligning the acquired raw data, and extracting kinematic, electromyographic, and biomechanical features to form a multimodal feature sequence; and an injury risk early warning model module storing a pre-trained multimodal temporal fusion injury risk early warning model, receiving the multimodal feature sequence output from the data processing and feature extraction module, and calculating and outputting the injury risk probability value and risk level. This system features multi-dimensional fusion, temporal dynamic modeling, personalized early warning, high precision, and real-time performance.

Description

Technical Field

[0001] This invention relates to the fields of sports science and sports medicine, specifically to a swimmer back injury early warning system and method based on multimodal temporal information fusion. Background Technology

[0002] Swimming, as a high-intensity, high-repetition competitive sport, easily subjectes the back muscles (especially the lumbar spine and deep stabilizing muscles) to complex biomechanical loads. Athletes specializing in butterfly, breaststroke, and other similar strokes often experience severe back injuries such as lower back pain, herniated discs, and pars interarticularis fractures due to underwater trunk twisting, wave-like movements, and high-frequency repetition during long-term training, impacting their athletic performance and careers.

[0003] Currently, the prevention and control of lower back injuries mainly relies on coaching experience, athletes' subjective feedback, and regular medical examinations, which suffers from problems such as strong subjectivity, significant lag, and a lack of quantitative indicators. Although some studies have attempted to introduce technologies such as inertial measurement units (IMU), surface electromyography (sEMG), and optical motion capture, most of them are single or simple combinations of modalities, making it difficult to achieve high-precision synchronous acquisition and deep fusion of multi-dimensional information, let alone build a real-time dynamic personalized injury risk assessment model. Summary of the Invention

[0004] To address the issues raised in the background section regarding the reliance on subjective judgment for swimmers' lower back injury risk, the lack of real-time, dynamic, and quantitative assessment methods, the fact that existing technologies mostly rely on single-modal data acquisition and cannot fully reflect the complex injury mechanisms involving the interaction of "movement-neuromuscular-biomechanics," and the lack of intelligent systems for personalized risk warning, this invention provides a swimmer lower back injury early warning system based on multimodal temporal information fusion.

[0005] A swimmer's back injury early warning system based on multimodal temporal information fusion includes a data acquisition module for simultaneously acquiring swimmer's kinematic temporal data, electromyographic temporal data, and biomechanical temporal data. The data processing and feature extraction module, connected to the data acquisition module, is used to preprocess and align the acquired raw data, and extract kinematic features, electromyographic features and biomechanical features to form a multimodal feature sequence. The damage risk warning model module stores a pre-trained multimodal temporal fusion damage risk warning model, receives the multimodal feature sequence output by the data processing and feature extraction module, and calculates and outputs the damage risk probability value and risk level. The early warning feedback module, connected to the injury risk early warning model module, is used to generate early warning signals based on the risk level and issue early warnings to athletes and / or coaching teams through at least one feedback channel.

[0006] Preferably, the feedback channel is one or more of a wearable vibrator, an audio device, or a graphical user interface.

[0007] Preferably, the data acquisition module includes: Optical motion capture unit and / or inertial measurement unit for acquiring motion data; Surface electromyography (EMG) acquisition unit, used to acquire electromyographic data; The computational unit is used to calculate or estimate biomechanical data based on kinematic data and personalized musculoskeletal models.

[0008] As a preferred option, the system adopts a "device-edge-cloud" collaborative architecture; Edge-side: Includes lightweight wearable sensing devices integrated on the athlete's body, including at least IMU and sEMG sensors, for real-time acquisition of core data; Edge side: Edge computing devices deployed in training venues receive data from the end side, run damage risk early warning models, and achieve low-latency real-time risk assessment and preliminary early warning; Cloud: Receives data from the edge, performs storage, in-depth analysis, and iterative model training, and distributes updated models to the edge. It also provides coaches and researchers with a platform for macro-level data visualization and long-term risk trend analysis.

[0009] As a preferred embodiment, the end-side subsystem hardware consists of: a lightweight, breathable, elastic sports belt; the belt is embedded with: Three 9-axis IMU modules are located in the middle of the lower back and on both sides of the lumbar muscles, respectively; A 4-channel wireless sEMG acquisition module with electrode pads connected to a waist belt via a magnetic interface, which are then attached to the erector spinae and external oblique muscles. The main control MCU is responsible for sensor driving, preliminary data filtering, and Bluetooth transmission. Software function: The firmware is responsible for collecting data at a sampling rate of 100Hz (IMU) and 1000Hz (sEMG), packaging it, and sending it to the edge gateway in real time via Bluetooth Low Energy 5.2; Edge subsystem hardware configuration: A waterproof, portable chassis containing: an edge computing module: NVIDIA Jetson Orin Nano.

[0010] Communication gateway: supports Bluetooth, Wi-Fi 6 and 4G; switch and power module.

[0011] Software features: Runs Ubuntu system and multiple Docker containers. Data receiving container: Receives Bluetooth data streams from multiple belts, performs time alignment and buffering; Feature extraction container: Runs feature extraction algorithms to transform the raw data stream into a feature sequence in real time; Real-time inference container: Loads and runs the multimodal temporal fusion damage risk warning Lite model to perform real-time risk assessment on the input feature sequence; Early warning decision and forwarding container: Based on the risk level, it generates control instructions and early warning messages, and uploads the early warning events and compressed feature data to the cloud via Wi-Fi / 4G.

[0012] The cloud subsystem includes infrastructure: deployed on public or private servers, including cloud servers, object storage, databases, and GPU computing instances; • Service Module: Data Access and Storage Service: Receives data uploaded from the edge gateways of various training venues and persistently stores it in a time-series database and object storage; Model training and iteration service: Regularly retrain and optimize the full-scale multimodal time-series fusion damage risk warning model using accumulated new data, and automatically distribute the performance-enhanced lightweight model to each edge gateway.

[0013] Preferably, the above-mentioned early warning system for lower back injuries in swimmers based on multimodal temporal information fusion is adopted, including the following steps: S1. During the training process of swimmers, multimodal time-series data are collected simultaneously; the time-series data includes kinematic time-series data, electromyographic time-series data, and biomechanical time-series data. S2. Extract and preprocess multimodal features; construct a multimodal feature sequence for model input; Kinematic feature extraction: From kinematic time-series data, features including but not limited to lumbar spine range of motion, joint range of motion, peak angular velocity, peak angular acceleration, motion asymmetry index, and inter-joint coordination are extracted to form a kinematic feature sequence; Electromyography feature extraction: After filtering, rectifying and other preprocessing of the raw sEMG signal, extract features including but not limited to root mean square value, integral electromyography value, average power frequency, median frequency and its slope, muscle activation timing, and muscle coordination mode features based on nonnegative matrix factorization to form an electromyography feature sequence. Biomechanical feature extraction: From biomechanical time-series data, features including but not limited to peak compressive force, peak anterior and posterior shear force, peak lateral shear force, peak muscle force, and extensor / flexor torque ratio of lumbar segments are extracted to form a biomechanical feature sequence; S3. Construct and train a multimodal temporal fusion damage risk early warning model; Construct a multimodal temporal fusion neural network model, which includes at least: Modality-specific encoding subnetworks: receive motion feature sequences, electromyography feature sequences, and biomechanical feature sequences respectively; each subnetwork contains a recurrent neural network layer or a self-attention mechanism layer for capturing temporal dependencies; Cross-modal feature fusion module: Receives high-level feature representations output from each modality-specific coding sub-network, and achieves deep fusion of multimodal information through cross-attention mechanism, feature concatenation, weighted summation, or gated fusion methods; this module can dynamically learn the contribution weights of different modal features to the current damage risk assessment; Risk prediction output layer: Receives the fused feature representation, passes through a fully connected layer and activation function, and outputs a probability value representing the risk of lower back injury occurring in the current or future specific time period.

[0014] The multimodal time-series fusion injury risk warning model is trained using a historically collected multimodal swimming training data sample set labeled with whether injury has occurred. By optimizing the loss function, the model can accurately learn the mapping relationship from multimodal time-series features to injury risk. S4: Real-time damage risk assessment and early warning; Multimodal time-series data, acquired in real-time or near real-time and preprocessed, are input into a trained multimodal time-series fusion damage risk early warning model. The model calculates and outputs real-time damage risk probability values. Based on preset risk thresholds, risk probability values ​​are divided into different risk levels; when a risk level reaches or exceeds a warning threshold, a warning signal is generated. The warning signal is fed back to relevant personnel in at least one of the following ways: through tactile, auditory, or visual feedback to athletes via wearable devices; or through wireless transmission to the terminal devices of coaches or team doctors for graphical display and alarm.

[0015] Preferably, wearing the customized equipment in step S1 includes: Optical marker kit: High-brightness waterproof infrared LED markers are attached to a total of 28 key anatomical points on the athlete's body surface; 8 high-speed infrared optical motion capture cameras are deployed around the pool and underwater to form a three-dimensional measurement field and track the three-dimensional coordinates of the markers in real time. Surface electromyography (EMG) acquisition kit: Uses wireless waterproof surface EMG sensors. Electrodes are attached to the erector spinae, rectus abdominis, external oblique, and latissimus dorsi muscles on both sides of the athlete, for a total of 8 channels. The sensors acquire raw EMG signals at a sampling rate of 2000Hz. Inertial Measurement Unit: Two high-precision 9-axis IMU sensors are fixed to the athlete's waist and sacrum to collect acceleration, angular velocity and magnetic field data, assist in motion tracking and provide independent kinematic information; All acquisition devices achieve hardware-level synchronization through TTL pulse signals emitted by the central synchronization controller, ensuring that the timestamps of multi-source data are aligned and the synchronization error is controlled within 5ms. As a preferred approach, multimodal feature extraction and preprocessing are performed. Kinematic data processing: Filtering the coordinates of optical markers, filling in missing values, and calculating the angles of various joints in the human body; Electromyography (EMG) data processing: The raw sEMG signal was bandpass filtered, filtered with a 50Hz notch, rectified, and then low-pass filtered at 6Hz to obtain a linear envelope. The root mean square value, average power frequency, and the linear fitting slope of the average power frequency over time were calculated for each channel in each action cycle. Non-negative matrix decomposition was performed on the EMG envelope signals of the 8 channels to extract 3-4 muscle synergistic modules and their activation curves. The stability of the synergistic module weight vector (correlation coefficient with the template) and the temporal consistency of the activation curves were calculated. An EMG feature sequence was formed. Biomechanical data processing: Based on optical motion capture data, a personalized whole-body musculoskeletal model of the subject was established and scaled in the open-source biomechanical software OpenSim; combined with ground reaction force data collected by the force table, inverse dynamics analysis was performed to calculate the joint torque and force of the L4 / L5 and L5 / S1 joints; and the muscle force of the target muscle was estimated. The peak compressive force, peak anterior shear force, peak posterior shear force, and peak muscle force of the major lumbar and back muscle groups were extracted from the results for each movement cycle.

[0016] As a preferred option, the construction and training of the multimodal temporal fusion damage risk early warning model includes: The multimodal temporal fusion damage risk early warning model includes the following layers: Input layer and embedding layer: Receives three feature sequences F_kin, F_emg, and F_mech. Each modality feature is mapped to a unified hidden space dimension D through three independent fully connected embedding layers, yielding embedding representations E_kin, E_emg, and E_mech. Modality-specific timing coding layer: Action coding branch: Input E_kin into a bidirectional LSTM layer to capture local temporal patterns; input E_kin into a Transformer encoder layer to capture global dependencies of long-distance joint motions within the motion cycle using a self-attention mechanism; concatenate the final hidden state of the LSTM with the output of the Transformer encoder to obtain the high-level action coding H_kin. Electromyography coding branch: symmetrical to the motor branch structure, E_emg is processed to obtain the high-level electromyography coding H_emg; Biomechanical encoding branch: Since biomechanical characteristics usually change slowly, E_mech is directly encoded using a fully connected layer to obtain H_mech; Cross-modal feature fusion module: Use H_kin as the query, and concatenate H_emg and H_mech as the key and value. Calculate attention weights: Attention(Q, K, V) = softmax((Q * W_q) * (K * W_k)^T / sqrt(d_k)) * (V * W_v); Where W_q, W_k, W_v are learnable parameter matrices, and d_k is the dimension of the key vector; The attention mechanism enables the model to dynamically focus on the muscle activity abnormalities (H_emg) and abnormal mechanical loads (H_mech) most relevant to the current abnormal movement pattern (represented by H_kin). The fused features are represented as F_fused; Finally, H_kin and F_fused are concatenated to obtain the final multimodal fusion feature H_final; Risk prediction output layer: H_final is passed through a global max pooling layer to aggregate time dimension information, then input into a two-layer fully connected network, and finally outputs the risk probability P_risk ∈ [0,1] through a sigmoid activation function; Model Training: Historical training data is collected, and data segments are labeled by sports medicine experts according to predefined injury criteria. A combination of binary cross-entropy loss and Focal Loss is used to address class imbalance. The Adam optimizer is employed to train and validate the model on data divided into training, validation, and test sets until the model's performance on the validation set stabilizes. The final model should achieve performance metrics of AUC > 0.85, sensitivity > 80%, and specificity > 85% on the independent test set.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention, for the first time, deeply integrates temporal data from three core modalities—kinematics, electromyography, and biomechanics—through multi-dimensional information fusion, comprehensively characterizing the complex causes of swimming-related lower back injuries. The assessment dimensions are more comprehensive, and the early warning basis is more scientific. Employing a deep learning model capable of processing temporal data, it effectively captures the dynamic evolution of injury risk with the movement cycle and training process, achieving a leap from "static snapshot" to "dynamic cinematic" assessment.

[0018] High Precision and Real-Time Performance: Through an optimized model architecture and edge-cloud collaborative computing, the system achieves real-time risk assessment and feedback with millisecond to second-level latency during training, while ensuring accurate early warnings, thus significantly advancing the intervention window. The model can adaptively learn the unique movement patterns and physiological characteristics of different athletes, and the output risk assessment results are highly individualized, providing precise data-driven decision support for personalized training adjustments and injury prevention. Revolutionary Proactive Prevention Model: This transforms injury control from the traditional passive "pain-response" model to a proactive "risk warning-proactive intervention" model, potentially significantly reducing the incidence and severity of back injuries in swimmers, protecting athlete health, and improving training efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart of the overall process for a method for early warning of lower back injuries in swimmers based on multimodal temporal information fusion, provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the multimodal time-series data synchronous acquisition step (S1) in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of a network architecture of the multimodal temporal fusion damage risk warning model (MTF-Net) in an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of a specific implementation of the cross-modal feature fusion module (based on cross attention) in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the "end-edge-cloud" collaborative architecture of the swimmer back injury early warning system based on multimodal temporal information fusion provided in this embodiment of the invention. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Example

[0025] A swimmer's back injury early warning system based on multimodal temporal information fusion includes a data acquisition module for simultaneously acquiring swimmer's kinematic temporal data, electromyographic temporal data, and biomechanical temporal data. The data processing and feature extraction module, connected to the data acquisition module, is used to preprocess and align the acquired raw data, and extract kinematic features, electromyographic features and biomechanical features to form a multimodal feature sequence. The damage risk warning model module stores a pre-trained multimodal temporal fusion damage risk warning model, receives the multimodal feature sequence output by the data processing and feature extraction module, and calculates and outputs the damage risk probability value and risk level. The early warning feedback module, connected to the injury risk early warning model module, is used to generate early warning signals based on the risk level and issue early warnings to athletes and / or coaching teams through at least one feedback channel.

[0026] Preferably, the feedback channel is one or more of a wearable vibrator, an audio device, or a graphical user interface.

[0027] The specific data acquisition modules include: Optical motion capture unit and / or inertial measurement unit for acquiring motion data; Surface electromyography (EMG) acquisition unit, used to acquire electromyographic data; The computational unit is used to calculate or estimate biomechanical data based on kinematic data and personalized musculoskeletal models.

[0028] The system adopts a "device-edge-cloud" collaborative architecture; Edge-side: Includes lightweight wearable sensing devices integrated on the athlete's body, including at least IMU and sEMG sensors, for real-time acquisition of core data; Edge side: Edge computing devices deployed in training venues receive data from the end side, run damage risk early warning models, and achieve low-latency real-time risk assessment and preliminary early warning; Cloud: Receives data from the edge, performs storage, in-depth analysis, and iterative model training, and distributes updated models to the edge. It also provides coaches and researchers with a platform for macro-level data visualization and long-term risk trend analysis.

[0029] In terms of specific hardware, the edge subsystem (wearable data collection terminal) Hardware components: A lightweight, breathable, elastic sports belt. Embedded in the belt: Three 9-axis IMU modules are located in the middle of the lower back (L3 level) and on both sides of the lumbar muscles.

[0030] The 4-channel wireless sEMG acquisition module has electrodes that connect to the belt via a magnetic interface and are attached to the erector spinae and external oblique muscles, respectively.

[0031] The main control MCU (such as Nordic nRF52840) is responsible for sensor driving, preliminary data filtering, and Bluetooth transmission.

[0032] Vibration feedback motor and rechargeable lithium battery.

[0033] Software function: The firmware program is responsible for collecting data at a sampling rate of 100Hz (IMU) and 1000Hz (sEMG), packaging it, and sending it to the edge gateway in real time via Bluetooth Low Energy 5.2.

[0034] Edge subsystem (intelligent analytics gateway) Hardware components: A waterproof portable chassis, containing: Edge computing module: NVIDIA Jetson Orin Nano.

[0035] Communication gateway: Supports Bluetooth, Wi-Fi 6 and 4G.

[0036] Switches and power modules.

[0037] Software features: Runs Ubuntu system and multiple Docker containers. Data receiving container: Receives Bluetooth data streams from multiple belts, performs time alignment, and buffers them.

[0038] Feature extraction container: Runs feature extraction algorithms to transform the raw data stream into a feature sequence in real time.

[0039] Real-time inference container: Loads and runs the MTF-Net Lite model to perform real-time risk assessment on the input feature sequence.

[0040] o Warning Decision and Forwarding Container: Based on the risk level, it generates control commands (reverse control of belt vibration via Bluetooth) and warning messages, and uploads the warning events and compressed feature data to the cloud via Wi-Fi / 4G.

[0041] Local UI container (optional): Provides a local web interface for on-site coaches to view real-time status.

[0042] Cloud-based subsystem (data management and analysis platform) Infrastructure: Deployed on public clouds (such as Alibaba Cloud) or private servers, including cloud servers, object storage, databases, and GPU computing instances.

[0043] Service modules: Data access and storage services: Receive data uploaded from edge gateways of various training venues and persistently store it in time-series databases and object storage.

[0044] Model training and iteration service: Regularly (e.g., weekly) retrain and optimize the full MTF-Net model using accumulated new data, and automatically distribute the performance-enhanced lightweight model to each edge gateway.

[0045] Data analysis and visualization services: Provides a RESTful API and web frontend. Coaches and researchers can log in to the platform via a browser to view: Team Dashboard: Real-time risk status (red, yellow, green lights) for all athletes.

[0046] Individual details page: Historical risk trend chart of a single athlete, replay of high-risk action segments (combined with synchronized video), electromyography activation chart, and personalized training suggestion report.

[0047] Statistical Analysis Page: Correlation analysis of team injury risk with training load and cycle arrangement.

[0048] User Management and Access Control: Manage accounts and access permissions for different roles (head coach, specialist coaches, team doctors, researchers).

[0049] In another embodiment, based on the above system, a method for early warning of lower back injuries in swimmers based on multimodal temporal information fusion is provided, including the following steps:

[0050] During swimmer training, the following three types of time-series data are collected simultaneously: Kinematic timing data: Waterproof optical markers or inertial measurement units (IMUs) deployed on key bony landmarks on the athlete's body surface are used to collect data on the three-dimensional spatial motion trajectory, angle, angular velocity, and angular acceleration of the athlete's trunk and limb joints during swimming.

[0051] Electromyography timing data: Raw signals of electrophysiological activity of the relevant muscle groups during swimming were collected by waterproof surface electromyography (sEMG) sensors attached to the skin surface of the athlete's lower back muscles.

[0052] Biomechanical time series data: Based on the aforementioned kinematic time series data and the athlete's personalized musculoskeletal model, the joint contact force, shear force, compressive force, and muscle force data of key muscles in the target lumbar spine segments (such as L4 / L5, L5 / S1) are obtained through inverse dynamics calculation or machine learning model estimation.

[0053] The synchronous acquisition is achieved through hardware synchronous triggering or high-precision timestamp alignment, ensuring that the time synchronization error between multimodal data is less than 10 milliseconds.

[0054] The raw time-series data collected in S1 is preprocessed and features are extracted to construct a multimodal feature sequence for model input: Kinematic feature extraction: From kinematic time-series data, extract features including but not limited to lumbar spine range of motion (flexion, extension, lateral flexion, rotation), joint range of motion, peak angular velocity, peak angular acceleration, motion asymmetry index, and inter-joint coordination (such as continuous relative phase) to form a kinematic feature sequence.

[0055] Electromyographic feature extraction: After preprocessing the raw sEMG signal by filtering and rectification, extract features including but not limited to root mean square value, integral electromyographic value, average power frequency, median frequency and its slope (fatigue index), muscle activation timing, and muscle synergistic pattern features based on nonnegative matrix factorization to form an electromyographic feature sequence.

[0056] Biomechanical feature extraction: From biomechanical time-series data, features including but not limited to peak lumbar segment compressive force, peak anterior and posterior shear force, peak lateral shear force, peak muscle force, and extensor / flexor torque ratio are extracted to form a biomechanical feature sequence.

[0057] Construct a multimodal temporal fusion neural network model, which includes at least: Modality-specific encoding subnetworks: These receive action feature sequences, electromyography feature sequences, and biomechanical feature sequences, respectively. Each subnetwork contains a recurrent neural network layer (such as LSTM or GRU) or a self-attention mechanism layer (such as a Transformer encoder layer) to capture temporal dependencies.

[0058] The cross-modal feature fusion module receives high-level feature representations from the dedicated coding sub-networks of each modality and achieves deep fusion of multimodal information through cross-attention mechanisms, feature concatenation, weighted summation, or gated fusion methods. This module can dynamically learn the contribution weights of different modal features to the current damage risk assessment.

[0059] Risk prediction output layer: Receives the fused feature representation, passes through a fully connected layer and an activation function (such as Sigmoid), and outputs a probability value (between 0 and 1) representing the risk of lower back injury occurring in the current or future specific time period.

[0060] The multimodal temporal fusion injury risk warning model is trained using a historically collected multimodal swimming training data sample set labeled with whether injury has occurred. By optimizing the loss function (such as binary cross-entropy loss or Focal Loss), the model can accurately learn the mapping relationship from multimodal temporal features to injury risk.

[0061] Multimodal time-series data (corresponding to a sliding time window) acquired in real-time or near real-time and preprocessed are input into a trained multimodal time-series fusion damage risk warning model. The model calculates and outputs real-time damage risk probability values.

[0062] Based on a preset risk threshold (which can be dynamically adjusted), the risk probability value is divided into different risk levels (such as low risk, medium risk, and high risk). When the risk level reaches or exceeds the warning threshold, a warning signal is generated.

[0063] The warning signal is fed back to relevant personnel in at least one of the following ways: through tactile (vibration), auditory (prompt sound) or visual (light) feedback to athletes via wearable devices; or through wireless transmission to the terminal devices of coaches or team doctors for graphical display and alarm.

[0064] The specific implementation details of steps S1 to S4 are as follows:

[0065] like Figure 2 As shown, this step is performed in a standard swimming pool environment. Athletes wear custom-made equipment: Optical Marker Kit: High-brightness, waterproof infrared LED markers are affixed to 28 key anatomical points on the athlete's body (including the head, neck, shoulders, elbows, wrists, chest, waist, pelvis, hips, knees, and ankles). Eight high-speed infrared optical motion capture cameras (such as OptiTrack Prime 41, 120Hz) are deployed around the pool and underwater to create a three-dimensional measurement field, tracking the three-dimensional coordinates of the markers in real time.

[0066] Surface electromyography (EMG) acquisition kit: Utilizes a wireless, waterproof surface EMG sensor (such as the Delsys Trigno Avanti). Electrodes are attached to the erector spinae (L3 level), rectus abdominis, external oblique, and latissimus dorsi muscles on both sides of the athlete, totaling eight channels. The sensor acquires raw EMG signals at a sampling rate of 2000 Hz.

[0067] Inertial Measurement Unit: Two high-precision 9-axis IMU sensors (such as Xsens MTw Awinda, 100Hz) are fixed at the athlete's waist (T12-L1 level) and sacrum to collect acceleration, angular velocity and magnetic field data, assist in motion tracking and provide independent kinematic information.

[0068] All acquisition devices achieve hardware-level synchronization through TTL pulse signals emitted by the central synchronization controller, ensuring that the timestamps of multi-source data are aligned and the synchronization error is controlled within 5ms.

[0069] Process the raw data collected by S101: 1. Kinematic Data Processing: The coordinates of optical marker points are filtered (6Hz Butterworth low-pass filter) and missing values ​​are filled. The angles of various joints in the human body are calculated. This embodiment focuses on the flexion-extension angle, lateral flexion angle, and rotation angle (Euler angles, ZXY sequence) of the lumbar spine local coordinate system relative to the pelvic coordinate system. The first derivative (angular velocity) and second derivative (angular acceleration) of these angles are also calculated. For symmetrical movements (such as leg kicks), the asymmetry index (AI) of amplitude and velocity is calculated. Feature values ​​of the above parameters, such as peak value, mean, range, and variance, are extracted within each swimming stroke cycle (such as one arm stroke), forming a kinematic feature sequence of length T, F_kin ∈ R^(T×N_kin), where N_kin is the feature dimension (e.g., 20 dimensions).

[0070] 2. Electromyography (EMG) Data Processing: The raw sEMG signals were subjected to 20-450Hz bandpass filtering, 50Hz notch filtering, and full-wave rectification, followed by 6Hz low-pass filtering to obtain a linear envelope. The root mean square value, average power frequency, and the slope of the linear fit of the average power frequency over time (as a local muscle fatigue index) for each channel in each movement cycle were calculated. Furthermore, nonnegative matrix factorization was performed on the EMG envelope signals of the 8 channels to extract 3-4 muscle synergistic modules and their activation curves. The stability of the synergistic module weight vector (correlation coefficient with the template) and the temporal consistency of the activation curves were calculated. An EMG feature sequence F_emg ∈ R^(T×N_emg) (e.g., 15-dimensional) was formed.

[0071] 3. Biomechanical Data Processing: First, based on optical motion capture data, a personalized full-body musculoskeletal model of the subject is established and scaled in the open-source biomechanical software OpenSim. Combined with ground reaction force data collected from a force table (installed on the starting platform), inverse dynamics analysis is performed to calculate the joint torques and forces of the L4 / L5 and L5 / S1 joints. Furthermore, a static optimization algorithm is used to estimate the muscle forces of the target muscles. From the results, the peak compressive force, peak anterior shear force, peak posterior shear force of the lumbar spine segments, and the peak muscle forces of the major lumbar and back muscle groups are extracted for each movement cycle. This forms a biomechanical feature sequence F_mech ∈ R^(T×N_mech) (e.g., 10-dimensional). To improve real-time performance, this step can also use a pre-trained lightweight neural network model to directly and quickly estimate biomechanical features from kinematic features and anthropometric parameters.

[0072] like Figure 3 As shown, the MTF-Net model constructed in this embodiment includes the following layers: 1. Input Layer and Embedding Layer: Receives three feature sequences F_kin, F_emg, and F_mech. Each modality feature is mapped to a unified hidden space dimension D (e.g., D=64) through three independent fully connected embedding layers, resulting in embedding representations E_kin, E_emg, and E_mech.

[0073] 2. Modality-specific timing coding layer: The motion coding branch: E_kin is input into a bidirectional LSTM layer (32 hidden units) to capture local temporal patterns. Simultaneously, E_kin is input into a Transformer encoder layer (4 heads, 128 feedforward dimensions) to capture global dependencies of long-distance joint movements within the motion cycle using a self-attention mechanism. The final hidden state of the LSTM and the output of the Transformer encoder (averaged along the time dimension) are concatenated to obtain the high-level motion coding H_kin.

[0074] Electromyography coding branch: symmetrical to the motor branch structure, processing E_emg yields the high-level electromyography coding H_emg.

[0075] Biomechanical encoding branch: Since biomechanical characteristics usually change slowly, E_mech is directly encoded using a fully connected layer to obtain H_mech.

[0076] Cross-modal feature fusion module (e.g.) Figure 4 This is one of the core innovations of the model. This embodiment uses a fusion mechanism based on cross-attention.

[0077] Use H_kin as the query, and concatenate H_emg and H_mech as the key and value.

[0078] Calculate the attention weights: Attention(Q, K, V) = softmax((Q * W_q) * (K * W_k)^T / sqrt(d_k)) * (V * W_v). Where W_q, W_k, and W_v are the learnable parameter matrices, and d_k is the dimension of the key vector.

[0079] The attention mechanism enables the model to dynamically focus on the muscle activity abnormalities (H_emg) and abnormal mechanical loads (H_mech) most relevant to the current abnormal movement pattern (represented by H_kin). The fused features are represented as F_fused.

[0080] Finally, H_kin and F_fused are concatenated to obtain the final multimodal fusion feature H_final.

[0081] Risk prediction output layer: Pass H_final through a global max pooling layer to aggregate the time dimension information, then input it into a two-layer fully connected network (with ReLU activation and Dropout in the middle), and finally output the risk probability P_risk ∈ [0, 1] through a Sigmoid activation function.

[0082] Model training: Collect historical training data, and label the data segments by sports medicine experts according to predefined injury criteria (such as low back pain causing training interruption) (1 indicates injury occurrence or high risk, 0 indicates safety). Use a combination of binary cross-entropy loss function and Focal Loss to address the class imbalance problem. Adopt the Adam optimizer to train and validate on the data divided into training set, validation set and test set until the performance of the model on the validation set tends to be stable. The final model should achieve performance metrics of AUC>0.85, sensitivity>80%, and specificity>85% on the independent test set.

[0083] In the training application stage: 1. Real-time data stream: Athletes wear a simplified version of wearable devices (only IMU and sEMG) to collect data in real time with a sliding window (such as a 5-second window, 1-second step). After data preprocessing and feature extraction, a feature sequence isomorphic to that during training is formed.

[0084] 2. Edge computing: The feature sequence is sent to an edge computing device (such as NVIDIA Jetson Orin Nano) deployed by the poolside. A lightweight MTF-Net Lite model optimized by pruning and quantization runs on this device, which can complete one inference within 200ms and output the risk probability P_risk of the current window.

[0085] 3. Risk grading and warning: Set two thresholds: Th_low = 0.3, Th_high = 0.7. If P_risk < Th_low, it is determined as low risk; if Th_low ≤ P_risk < Th_high, it is determined as medium risk; if P_risk ≥ Th_high, it is determined as high risk.

[0086] 4. Multi-level feedback: High risk: Immediately trigger the strong vibration motor integrated in the athlete's waistband and play a voice prompt "Pay attention to your waist posture" through the bone conduction headphones. At the same time, the warning information (including risk level, risk probability, main risk features) is pushed to the APP on the poolside coach's handheld tablet in real time, and the interface is highlighted in red and an alarm sound is issued.

[0087] Medium risk: The belt emits a mild, intermittent vibration as a warning. The athlete's icon on the coach's terminal turns yellow, and a corresponding marker appears on the risk trend graph.

[0088] Low risk: No tactile or auditory feedback. The instructor's terminal displays in green.

[0089] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention, for the first time, deeply integrates temporal data from three core modalities—kinematics, electromyography, and biomechanics—through multi-dimensional information fusion, comprehensively characterizing the complex causes of swimming-related lower back injuries. The assessment dimensions are more comprehensive, and the early warning basis is more scientific. Employing a deep learning model capable of processing temporal data, it effectively captures the dynamic evolution of injury risk with the movement cycle and training process, achieving a leap from "static snapshot" to "dynamic cinematic" assessment.

[0090] High Precision and Real-Time Performance: Through an optimized model architecture and edge-cloud collaborative computing, the system achieves real-time risk assessment and feedback with millisecond to second-level latency during training, while ensuring accurate early warnings, thus significantly advancing the intervention window. The model can adaptively learn the unique movement patterns and physiological characteristics of different athletes, and the output risk assessment results are highly individualized, providing precise data-driven decision support for personalized training adjustments and injury prevention. Revolutionary Proactive Prevention Model: This transforms injury control from the traditional passive "pain-response" model to a proactive "risk warning-proactive intervention" model, potentially significantly reducing the incidence and severity of back injuries in swimmers, protecting athlete health, and improving training efficiency.

Claims

1. A swimmer's lumbar and back injury early warning system based on multimodal temporal information fusion, characterized in that: It includes a data acquisition module for simultaneously acquiring the kinematic timing data, electromyographic timing data, and biomechanical timing data of swimmers; The data processing and feature extraction module, connected to the data acquisition module, is used to preprocess and align the acquired raw data, and extract kinematic features, electromyographic features and biomechanical features to form a multimodal feature sequence. The damage risk warning model module stores a pre-trained multimodal temporal fusion damage risk warning model, receives the multimodal feature sequence output by the data processing and feature extraction module, and calculates and outputs the damage risk probability value and risk level. The early warning feedback module, connected to the injury risk early warning model module, is used to generate early warning signals based on the risk level and issue early warnings to athletes and / or coaching teams through at least one feedback channel.

2. The early warning system for lower back injuries in swimmers based on multimodal temporal information fusion according to claim 1, characterized in that: The feedback channel can be one or more of wearable vibrators, audio devices, and graphical user interfaces.

3. The early warning system for lower back injuries in swimmers based on multimodal temporal information fusion according to claim 1, characterized in that: The data acquisition module includes: Optical motion capture unit and / or inertial measurement unit for acquiring motion data; Surface electromyography (EMG) acquisition unit, used to acquire electromyographic data; The computational unit is used to calculate or estimate biomechanical data based on kinematic data and personalized musculoskeletal models.

4. The early warning system for lower back injuries in swimmers based on multimodal temporal information fusion according to claim 1, characterized in that: The system adopts a collaborative architecture of "end-edge-cloud"; • Edge-side: Includes lightweight wearable sensing devices integrated on the athlete's body, including at least IMU and sEMG sensors, for real-time acquisition of core data; • Edge side: Edge computing devices deployed in training venues receive data from the end side, run damage risk early warning models, and achieve low-latency real-time risk assessment and preliminary early warning; Cloud: Receives data from the edge, performs storage, in-depth analysis, and iterative model training, and distributes updated models to the edge. It also provides coaches and researchers with a platform for macro-level data visualization and long-term risk trend analysis.

5. A swimmer back injury early warning system based on multimodal temporal information fusion according to claim 1, characterized in that: End-side subsystem hardware components: a lightweight, breathable, elastic sports belt; embedded in the belt: Three 9-axis IMU modules are located in the middle of the lower back and on both sides of the lumbar muscles, respectively; A 4-channel wireless sEMG acquisition module with electrode pads connected to a waist belt via a magnetic interface, which are then attached to the erector spinae and external oblique muscles. The main control MCU is responsible for sensor driving, preliminary data filtering, and Bluetooth transmission. Software function: The firmware is responsible for collecting data at a sampling rate of 100Hz (IMU) and 1000Hz (sEMG), packaging it, and sending it to the edge gateway in real time via Bluetooth Low Energy 5.2; Edge subsystem hardware configuration: A waterproof portable chassis containing: an edge computing module: NVIDIA Jetson Orin Nano; Communication gateway: supports Bluetooth, Wi-Fi 6 and 4G; switch and power module; Software features: Runs Ubuntu system and multiple Docker containers. Data receiving container: Receives Bluetooth data streams from multiple belts, performs time alignment and buffering; Feature extraction container: Runs feature extraction algorithms to transform the raw data stream into a feature sequence in real time; Real-time inference container: Loads and runs the multimodal temporal fusion damage risk warning Lite model to perform real-time risk assessment on the input feature sequence; Early warning decision and forwarding container: Based on the risk level, it generates control instructions and early warning messages, and uploads the early warning events and compressed feature data to the cloud via Wi-Fi / 4G; The cloud subsystem includes infrastructure: deployed on public or private servers, including cloud servers, object storage, databases, and GPU computing instances; • Service Module: Data Access and Storage Service: Receives data uploaded from the edge gateways of various training venues and persistently stores it in a time-series database and object storage; Model training and iteration service: Regularly retrain and optimize the full-scale multimodal time-series fusion damage risk warning model using accumulated new data, and automatically distribute the performance-enhanced lightweight model to each edge gateway.

6. A method for early warning of lumbar and back injuries in swimmers based on multimodal temporal information fusion, characterized in that: A swimmer back injury early warning system based on multimodal temporal information fusion, as described in any one of claims 1 to 5, includes the following steps: S1. During the training process of swimmers, multimodal time-series data are collected simultaneously; the time-series data includes kinematic time-series data, electromyographic time-series data, and biomechanical time-series data; S2. Extract and preprocess multimodal features; construct a multimodal feature sequence for model input; Kinematic feature extraction: From kinematic time-series data, features including but not limited to lumbar spine range of motion, joint range of motion, peak angular velocity, peak angular acceleration, motion asymmetry index, and inter-joint coordination are extracted to form a kinematic feature sequence; Electromyography feature extraction: After filtering, rectifying and other preprocessing of the raw sEMG signal, extract features including but not limited to root mean square value, integral electromyography value, average power frequency, median frequency and its slope, muscle activation timing, and muscle coordination mode features based on nonnegative matrix factorization to form an electromyography feature sequence. Biomechanical feature extraction: From biomechanical time-series data, features including but not limited to peak compressive force, peak anterior and posterior shear force, peak lateral shear force, peak muscle force, and extensor / flexor torque ratio of lumbar segments are extracted to form a biomechanical feature sequence; S3. Construct and train a multimodal temporal fusion damage risk early warning model; Construct a multimodal temporal fusion neural network model, which includes at least: Modality-specific encoding subnetworks: receive motion feature sequences, electromyography feature sequences, and biomechanical feature sequences respectively; each subnetwork contains a recurrent neural network layer or a self-attention mechanism layer for capturing temporal dependencies; Cross-modal feature fusion module: Receives high-level feature representations output from each modality-specific coding sub-network, and achieves deep fusion of multimodal information through cross-attention mechanism, feature concatenation, weighted summation, or gated fusion methods; this module can dynamically learn the contribution weights of different modal features to the current damage risk assessment; Risk prediction output layer: Receives the fused feature representation, passes through a fully connected layer and an activation function (such as Sigmoid), and outputs a probability value representing the risk of lower back injury occurring in the current or future specific time period; The multimodal time-series fusion injury risk warning model is trained using a historically collected multimodal swimming training data sample set labeled with whether injury has occurred. By optimizing the loss function, the model can accurately learn the mapping relationship from multimodal time-series features to injury risk. S4: Real-time damage risk assessment and early warning; Multimodal time-series data, acquired in real-time or near real-time and preprocessed, are input into a trained multimodal time-series fusion damage risk early warning model. The model calculates and outputs real-time damage risk probability values. Based on preset risk thresholds, risk probability values ​​are divided into different risk levels; when a risk level reaches or exceeds a warning threshold, a warning signal is generated. The warning signal is fed back to relevant personnel in at least one of the following ways: through tactile, auditory, or visual feedback to athletes via wearable devices; or through wireless transmission to the terminal devices of coaches or team doctors for graphical display and alarm.

7. The method for early warning of lumbar and back injuries in swimmers based on multimodal temporal information fusion according to claim 6, characterized in that: Wearing custom equipment in step S1 includes: Optical marker kit: High-brightness waterproof infrared LED markers are attached to a total of 28 key anatomical points on the athlete's body surface; 8 high-speed infrared optical motion capture cameras are deployed around the pool and underwater to form a three-dimensional measurement field and track the three-dimensional coordinates of the markers in real time. Surface electromyography (EMG) acquisition kit: Uses wireless waterproof surface EMG sensors. Electrodes are attached to the erector spinae, rectus abdominis, external oblique, and latissimus dorsi muscles on both sides of the athlete, for a total of 8 channels. The sensors acquire raw EMG signals at a sampling rate of 2000Hz. Inertial Measurement Unit: Two high-precision 9-axis IMU sensors are fixed to the athlete's waist and sacrum to collect acceleration, angular velocity and magnetic field data, assist in motion tracking and provide independent kinematic information; All acquisition devices achieve hardware-level synchronization through TTL pulse signals emitted by the central synchronization controller, ensuring that the timestamps of multi-source data are aligned and the synchronization error is controlled within 5ms.

8. A method for early warning of lumbar and back injuries in swimmers based on multimodal temporal information fusion according to claim 7, characterized in that: Multimodal feature extraction and preprocessing; Kinematic data processing: Filtering the coordinates of optical markers, filling in missing values, and calculating the angles of various joints in the human body; Electromyography (EMG) data processing: The raw sEMG signal was bandpass filtered, filtered with a 50Hz notch, rectified, and then low-pass filtered at 6Hz to obtain a linear envelope. The root mean square value, average power frequency, and the linear fitting slope of the average power frequency over time were calculated for each channel in each action cycle. Non-negative matrix decomposition was performed on the EMG envelope signals of the 8 channels to extract 3-4 muscle synergistic modules and their activation curves. The stability of the synergistic module weight vector (correlation coefficient with the template) and the temporal consistency of the activation curves were calculated. An EMG feature sequence was formed. Biomechanical data processing: Based on optical motion capture data, a personalized whole-body musculoskeletal model of the subject was established and scaled in the open-source biomechanical software OpenSim; combined with ground reaction force data collected by the force table, inverse dynamics analysis was performed to calculate the joint torque and force of the L4 / L5 and L5 / S1 joints; and the muscle force of the target muscle was estimated. The peak compressive force, peak anterior shear force, peak posterior shear force, and peak muscle force of the major lumbar and back muscle groups were extracted from the results for each movement cycle.

9. A method for early warning of lumbar and back injuries in swimmers based on multimodal temporal information fusion according to claim 8, characterized in that: The construction and training of a multimodal temporal fusion damage risk early warning model includes: The multimodal temporal fusion damage risk early warning model includes the following layers: Input layer and embedding layer: Receives three feature sequences F_kin, F_emg, and F_mech. Each modality feature is mapped to a unified hidden space dimension D through three independent fully connected embedding layers, yielding embedding representations E_kin, E_emg, and E_mech. Modality-specific timing coding layer: Action coding branch: Input E_kin into a bidirectional LSTM layer to capture local temporal patterns; input E_kin into a Transformer encoder layer to capture global dependencies of long-distance joint motions within the motion cycle using a self-attention mechanism; concatenate the final hidden state of the LSTM with the output of the Transformer encoder to obtain the high-level action coding H_kin. Electromyography coding branch: symmetrical to the motor branch structure, E_emg is processed to obtain the high-level electromyography coding H_emg; Biomechanical encoding branch: Since biomechanical characteristics usually change slowly, E_mech is directly encoded using a fully connected layer to obtain H_mech; Cross-modal feature fusion module: Use H_kin as the query, and concatenate H_emg and H_mech as the key and value. Calculate attention weights: Attention(Q, K, V) = softmax((Q * W_q) * (K * W_k)^T / sqrt(d_k)) * (V * W_v); Where W_q, W_k, W_v are learnable parameter matrices, and d_k is the dimension of the key vector; The attention mechanism enables the model to dynamically focus on the muscle activity abnormalities (H_emg) and abnormal mechanical loads (H_mech) most relevant to the current abnormal movement pattern (represented by H_kin). The fused features are represented as F_fused; Finally, H_kin and F_fused are concatenated to obtain the final multimodal fusion feature H_final; Risk prediction output layer: H_final is passed through a global max pooling layer to aggregate time dimension information, then input into a two-layer fully connected network, and finally outputs the risk probability P_risk ∈ [0, 1] through a sigmoid activation function; Model Training: Historical training data is collected, and data segments are labeled by sports medicine experts according to predefined injury criteria. A combination of binary cross-entropy loss and Focal Loss is used to address class imbalance. The Adam optimizer is employed to train and validate the model on data divided into training, validation, and test sets until the model's performance on the validation set stabilizes. The final model should achieve performance metrics of AUC > 0.85, sensitivity > 80%, and specificity > 85% on the independent test set.

Images