Body feeling evaluation method and system based on electroencephalogram-electromyographic signal fusion and dynamic interaction modeling

Abstract

The invention discloses a body feeling evaluation method based on electroencephalogram-electromyographic signal fusion and dynamic interaction modeling. The body feeling evaluation method comprises the steps that EEG signals and EMG signals in the lower limb movement process of a subject are synchronously collected; carrying out band-pass filtering, artifact removal and wavelet transform processing on the acquired signals, extracting multi-channel time-frequency features, and forming a preprocessing feature matrix; fusing the time-frequency features of the EEG signal and the EMG signal, constructing a multi-modal feature set, and compressing feature dimensions by adopting a sparse coding method; inputting the compressed feature sequence into a neural network model combining a long short-term memory network and an attention mechanism, and carrying out dynamic interaction modeling; and an interaction index sequence is generated based on model output, and an interaction matrix is constructed through a sliding window and Gaussian kernel smoothing processing, so that visualization of brain-muscle interaction strength and dynamic quantification of a proprioceptive function are realized. The invention further provides a system for implementing the method. The method is high in objectivity, high in feature extraction precision and excellent in dynamic modeling capability.

Description

Technical Field

[0001] This invention belongs to the field of biomedical signal processing and rehabilitation engineering technology, specifically relating to a dynamic interactive modeling method and system that integrates electroencephalogram (EEG) and electromyogram (EMG) signals. Background Technology

[0002] Proprioception, as the neural basis of human motor coordination, integrates signals from peripheral receptors in the skin, joints, and muscles to construct a real-time perception network of limb movement status and spatial position. This neural information processing mechanism plays a crucial role in optimizing motor coordination, regulating dynamic balance, and maintaining postural stability. Accurate and real-time assessment of proprioceptive function is of great significance for the formulation of neurorehabilitation intervention strategies and the tracking of their effects. Currently used clinical assessment methods, such as passive motion threshold testing, joint position reproduction methods, and active motion recognition methods, mainly rely on subjective scale scoring, which suffers from problems such as strong subjectivity, low quantitative accuracy, and lack of dynamic response capability, making it difficult to fully reveal the neurophysiological mechanisms and changes of proprioception.

[0003] EEG signals can reflect the real-time neural activity state of the cerebral cortex, while EMG signals characterize the electrophysiological responses triggered by muscle contraction. With the continuous development of multimodal sensing technology and biosignal processing algorithms, methods integrating physiological signals such as EEG and EMG provide a new pathway for the objective assessment of proprioception. By simultaneously acquiring EEG and EMG signals, the complete process of peripheral motor information transmission to the central nervous system and the induction of cortical responses can be depicted. This brain-muscle neural interaction model provides important evidence for revealing the physiological mechanisms of proprioception and lays the foundation for the dynamic assessment of its functional state.

[0004] While EEG and EMG signal fusion technology offers a novel approach to the objective assessment of proprioception, most current methods primarily focus on extracting specific time points or static statistical features, lacking systematic modeling of the temporal dependencies and dynamic interaction patterns between EEG and EMG signals. Existing assessment methods, such as Granger causality analysis, coherence analysis, and transfer entropy methods based on EEG-EMG synchronicity, while revealing the information flow between brain and muscle signals to some extent, often rely on static features, have limited modeling capabilities, and struggle to meet the continuous assessment needs of complex dynamic motion scenarios. For example, the motion state recognition method based on EEG coherence analysis proposed in patent CN110942456A does not address the dynamic response of EMG signals; CN113076182A relies on EMG to predict movement intentions, lacking integrated analysis of the central nervous system. Although these methods have exploratory value in early research, they fail to comprehensively reflect the temporal evolution and interaction characteristics of proprioceptive function.

[0005] On the other hand, current multimodal brain-electromyography fusion systems generally suffer from technical limitations: firstly, they lack nonlinear dynamic modeling capabilities, making it difficult to capture deep temporal dependencies between high-dimensional features; secondly, they lack high-resolution visualization methods for interaction intensity, hindering physician understanding and application; and thirdly, most systems employ offline computation processes, failing to support real-time data processing and clinical feedback. These problems directly result in current assessment results lacking timeliness, stability, and clinical interpretability, limiting their widespread application in rehabilitation medicine.

[0006] Therefore, there is an urgent need for a proprioceptive assessment method and system that integrates EEG and EMG signals, possesses dynamic interactive modeling capabilities, temporal visualization analysis capabilities, and real-time response capabilities. This system should be able to accurately capture the time-varying interaction between brain and muscle signals, dynamically reflect the evolution of proprioceptive function, and thus provide scientific, precise, and real-time support for functional assessment, efficacy tracking, and personalized intervention in clinical rehabilitation. Summary of the Invention

[0007] To overcome the problems of high subjectivity, low quantification accuracy, and lack of dynamic response capability in existing technologies for proprioceptive assessment, this invention proposes a proprioceptive assessment method and system based on EEG-EMG signal fusion and dynamic interaction modeling. This method simultaneously acquires EEG and EMG signals during lower limb movement of the subject, constructs a multimodal feature set of brain-muscle signal coupling, and combines a long short-term memory network (LSTM) and attention mechanism to construct a dynamic interaction modeling network. It then extracts and quantifies the dynamic interaction intensity between EEG and EMG, achieving quantitative assessment and visual tracking of proprioceptive function.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] A proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling includes the following steps:

[0010] Step 1: Simultaneously acquire EEG and EMG signals during the subject's movement using a multimodal acquisition module, and construct the original multi-channel dataset.

[0011] Step 2: Preprocess the raw data, including bandpass filtering, artifact removal and wavelet transform, to extract multi-channel time-frequency features and form a preprocessed dataset.

[0012] Step 3: Channel dimension fusion is performed on the preprocessed EEG and EMG features, and sparse autoencoder is used to reduce the dimensionality of the multimodal features to construct a low-dimensional multimodal feature set.

[0013] Step 4: Using the multimodal feature set obtained in Step 3 as input, construct and train a deep neural network model combining LSTM and attention mechanism to model the dynamic interaction between EEG and EMG signals.

[0014] Step 5: Based on the model output trained in Step 4, generate a sequence of brain-muscle interaction intensity indicators, and visualize the indicators through sliding window and Gaussian kernel smoothing. Finally, it is used for quantitative assessment and temporal tracking of proprioceptive function.

[0015] Furthermore, in step 1, the sampling frequency of both EEG and EMG signals is 1000Hz. Specifically, the EEG signal is collected using 16-channel electrodes deployed in the motor sensory cortex, covering the central area and part of the parietal lobe, to acquire neural activity in the cerebral cortex; the EMG signal is collected using 8-channel surface electrodes to acquire electrophysiological signals from the major muscle groups of both lower limbs of the subject.

[0016] Preferably, in step 1, the subject sits with arms crossed and feet naturally placed on the foot pedals. During the exercise, the subject focuses on the white cross mark in the center of the screen to avoid interference from eye movement, blinking, swallowing, and other actions, thus ensuring the stability and effectiveness of the signal during the acquisition period.

[0017] Furthermore, in step 2, the EEG signal is first rereferenced using bilateral mastoid processes as reference electrodes and then subjected to 0.5–30Hz bandpass filtering. Incremental independent component analysis (ORICA) is then used to remove physiological interferences such as eye movement and blinking myogenic artifacts. The preprocessed EEG signal is then subjected to wavelet transform to extract its multi-channel, multi-band time-frequency features. In the EMG signal processing flow, a Fast Fourier Transform (FFT) is first used to determine if 50Hz power frequency interference exists. If it does, a notch filter centered at 50Hz with a bandwidth of 2Hz is used to suppress it. Next, a 20–250Hz bandpass filter is used to remove low-frequency noise. Finally, wavelet transform is performed on the EMG signal to extract its multi-channel, multi-band time-frequency features.

[0018] Furthermore, in step 3, the time-frequency features of EEG and EMG extracted in step 2 are concatenated along the channel dimension to construct a fused multimodal feature set. This feature set, after normalization, is input into a sparse autoencoder for dimensionality reduction and sparse coding to extract low-dimensional key features for subsequent modeling and analysis. The optimization objective function of the sparse coding is:

[0019]

[0020] Where X is the fusion feature matrix, D is the dictionary matrix, θ is the encoding vector, and λ is the sparse regularization coefficient.

[0021] In step 4, the low-dimensional sparse multimodal feature sequence obtained in step 3 is used as input to construct a deep neural network model based on LSTM and an attention mechanism. This model uses the LSTM structure to extract the temporal dependencies in the sequence features and uses the attention mechanism to identify key interaction moments, focusing on feature segments that significantly affect changes in brain-muscle interaction. This enables the modeling of the dynamic coupling relationship between EEG and EMG signals, constructing a brain-muscle interaction representation structure with nonlinear modeling capabilities. The weighting technique of the attention mechanism is as follows:

[0022]

[0023] Where Q is the query vector, K t The characteristic representation of time t;

[0024] The model is obtained through offline training and deployed in the system for online inference. It supports periodic model parameter updates and incremental learning mechanisms to ensure that the system has real-time response capabilities and individualized adaptation capabilities.

[0025] In step 5, the low-dimensional sparse feature sequence extracted in step 3 is input into the neural network model trained in step 4, and a brain-muscle dynamic interaction index sequence is generated through forward propagation. Subsequently, the index sequence is smoothed by a combination of sliding window and Gaussian kernel function to reduce short-term fluctuation interference and improve the stability and interpretability of the interaction index. The smoothing calculation method can be expressed as follows:

[0026]

[0027] Among them, I t+τ σ is the original interaction index within the sliding window; σ is the standard deviation of the Gaussian kernel, which determines the smoothing intensity; L is the length of one side of the window, and the total window length is 2L+1.

[0028] In step 5, the smoothed brain-muscle interaction index sequence is constructed into an interaction matrix reflecting the relationship between different feature dimensions, and the average interaction intensity between EEG and EMG signals is visualized by heatmap. The sliding window analysis method is used to perform time series analysis on the interaction intensity index to reveal the dynamic trend of interaction intensity changes over time, thereby realizing the visual tracking of the evolution process of proprioceptive functional state.

[0029] A proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling includes a multimodal signal acquisition module, a data preprocessing module, a feature compression module, a neural network modeling module, and an index generation and visualization module.

[0030] The multimodal signal acquisition module is used to simultaneously acquire EEG and EMG signals wirelessly, supporting continuous data recording of subjects in motion, and transmitting the acquired data to the host computer in real time for processing and analysis via Wi-Fi communication.

[0031] The data preprocessing module is used to perform filtering, rereference, artifact removal and wavelet time-frequency transformation processing involved in steps 2 and 3, and to extract multi-channel, multi-band fusion time-frequency features of EEG and EMG signals;

[0032] The feature compression module is used to normalize the fused multimodal feature set and use a sparse autoencoder structure to reduce its dimensionality, thereby compressing the feature dimension and improving modeling efficiency and generalization ability.

[0033] The neural network modeling module is used to build and run a deep neural network model based on Long Short-Term Memory (LSTM) and attention mechanisms, enabling dynamic interactive modeling of multimodal feature sequences and supporting real-time online inference. This module adopts a decoupled structure for offline training and online inference, possessing periodic parameter updates and incremental learning capabilities, adapting to dynamic modeling and individual difference adjustment needs; simultaneously, it can run on GPU-optimized computing architectures, supporting edge deployment, low-power operation, and batch inference tasks.

[0034] The indicator generation and visualization module has sliding window analysis and heatmap output functions, which are used to continuously track the temporal evolution of brain-muscle interaction intensity, and realize the visualization and quantitative analysis of proprioceptive function state.

[0035] The beneficial effects of this invention are mainly reflected in:

[0036] 1. By simultaneously acquiring EEG and EMG signals and integrating advanced algorithms such as wavelet transform, sparse autoencoder, and deep neural network, a multimodal fusion modeling framework for brain-muscle signals is constructed. This framework can accurately characterize the dynamic interaction between brain and muscle, overcome the shortcomings of traditional proprioceptive assessment methods, such as strong subjectivity and low quantification accuracy, and realize objective quantitative assessment of proprioceptive function with physiological basis.

[0037] 2. By constructing a deep learning structure that combines LSTM and attention mechanisms, it can effectively extract temporal dependencies in physiological signals, adaptively identify key interaction moments, and exhibit good robustness and generalization ability in dynamic motion states, making it suitable for proprioceptive function assessment in diverse scenarios.

[0038] 3. By introducing a sliding window and Gaussian kernel function smoothing strategy, combined with dynamic interaction matrix and heatmap visualization methods, the temporal evolution of brain-muscle interaction intensity can be clearly presented, providing quantitative support for clinical rehabilitation assessment, assisting doctors in formulating personalized intervention strategies, and significantly improving the practicality and clinical guidance value of the assessment tool. Attached Figure Description

[0039] Figure 1 A flowchart of the lower limb motor proprioception assessment method provided by the present invention;

[0040] Figure 2 The locations of the EEG sensor (a) and EMG sensor (b) of the present invention are shown, wherein 1 is the rectus femoris muscle, 2 is the biceps femoris muscle, 3 is the gastrocnemius muscle, and 4 is the tibialis anterior muscle.

[0041] Figure 3 This is a schematic diagram of the structure of the lower limb motor proprioception assessment system provided by the present invention;

[0042] Figure 4The EEG acquisition interface diagram of the lower limb motor proprioception assessment system provided by the present invention is shown.

[0043] Figure 5 An electromyography (EMG) acquisition interface diagram of the lower limb motor proprioception assessment system provided by the present invention;

[0044] Figure 6 The average interaction weight map of the lower limb motor proprioceptive assessment feature dimension provided by this invention. Detailed Implementation

[0045] The present invention will now be further described with reference to the accompanying drawings.

[0046] Reference Figures 1-6 A proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling includes the following steps:

[0047] Step 1: Simultaneous acquisition of EEG and EMG signals, as follows:

[0048] EEG and EMG signals were collected simultaneously during the flexion and extension movements of the lower limbs of the subjects. The sampling frequency was set to 1000Hz, and the collection time for each experiment was 90 seconds.

[0049] like Figure 2 As shown in (a), the EEG signal was generated using a 16-channel Ag-AgCl electrode, deployed according to the international 10-20 system, covering the central area, the motor and sensory cortex, and part of the parietal lobe. The selected channels included C1, C2, C3, C4, C5, C6, Cz, CP3, CP4, P3, P4, Pz, FC3, FC4, TP9, and TP10, in order to comprehensively capture cortical neural activity related to lower limb movement.

[0050] like Figure 2 As shown in (b), the EMG signal was obtained by placing surface electrodes on four major muscle groups of the lower limb: rectus femoris, biceps femoris, gastrocnemius and tibialis anterior, symmetrically arranged on both sides, with a total of 8 channels, to synchronously record the electrical activity information of muscle groups during the flexion and extension movements of the lower limb.

[0051] During the experiment, the subjects sat with their arms crossed and their feet naturally placed on the foot pedals. They focused on the white cross mark in the center of the screen during the flexion and extension movements of their lower limbs, avoiding distractions such as eye movement, blinking, and swallowing to ensure the quality of the collected signal.

[0052] Step 2: Signal preprocessing and time-frequency feature extraction, as follows:

[0053] The acquired EEG and EMG signals are preprocessed online through an embedded signal processing module, including filtering, rereference, artifact removal, and time-frequency transformation.

[0054] The EEG signal is first bandpass filtered from 0.5 to 30 Hz to retain the dominant frequency components relevant to the motion task. Then, a rereference operation is performed using the TP9 and TP10 electrodes as reference channels. The artifact removal process employs incremental independent component analysis (ORICA) or its lightweight variant to dynamically identify and remove artifacts such as eye movements, blinks, and electromyography within a sliding time window. The cleaned EEG signal is then subjected to online Morlet wavelet transform to extract multi-channel, multi-band time-frequency features, continuously outputting a high-dimensional EEG feature matrix.

[0055] The EMG signal processing procedure includes: First, analyzing the signal spectrum using Fast Fourier Transform (FFT) to identify the presence of significant 50Hz power frequency interference. If this frequency component is detected, a notch filter centered at 50Hz with a bandwidth of 2Hz is used to suppress it. Subsequently, a third-order bandpass filter based on Butterworth design is used to perform online bandpass filtering of the EMG signal from 20-250Hz to remove low-frequency motion artifacts and high-frequency noise. After filtering, wavelet transform is also used to extract multi-channel, multi-band EMG time-frequency features, forming a high-dimensional EMG feature matrix. The entire preprocessing workflow is implemented using a streaming processing structure to ensure high stability and low latency continuous feature extraction during acquisition.

[0056] Step 3: Real-time feature fusion and encoding compression, as follows:

[0057] The system synchronously concatenates the multi-channel time-frequency features of the EEG and EMG signals extracted in step 2 along the channel dimension to construct a fused feature tensor with the shape [channel × time × frequency]. The channel dimension is arranged in the order of "EEG channel first, EMG channel second" to maintain a clear division of the signal source structure. This tensor is continuously updated during data acquisition and converted into a two-dimensional matrix with the structure [time × (number of channels × number of frequencies)] through streaming reconstruction to adapt to the input requirements of subsequent neural network models.

[0058] To improve the numerical stability and generalization ability of the model, the system integrates a unified normalization and feature compression module. The fused feature matrix is ​​first normalized to map all feature values ​​to the [0,1] interval, and then input into the sparse autoencoder model deployed in the system kernel for dimensionality reduction and encoding.

[0059] The sparse autoencoder has undergone parameter optimization during the training phase. Its hidden layer node count is set to 32 by default, and can be flexibly adjusted between 20 and 40 depending on the fusion feature dimension to balance feature compression effectiveness and computational efficiency. The sparsity regularization parameter λ is set to 0.1, the decoding function uses a pure linear function, and GPU-accelerated training is supported. This model achieves effective extraction and compression of brain-muscle coupling features by minimizing reconstruction error and constraining activation sparsity.

[0060] During the system deployment phase, the autoencoder model operates in a lightweight forward inference mode, which can process the fused feature stream in real time and stably output a sparse feature sequence with a shape of [time×32] as the standard input to the LSTM-attention mechanism neural network modeling module.

[0061] Step 4: Brain-muscle dynamic interaction modeling, as follows:

[0062] The low-dimensional sparse multimodal feature sequence obtained in step 3 is used as the input to the neural network model. The input data shape is [time × 32], where each time step corresponds to a fused sparse feature vector.

[0063] In the network structure, a sequence input layer is first set as a 32-dimensional temporal feature vector; then an LSTM layer is connected to extract the temporal dependencies and dynamic evolution patterns in the feature sequence. The LSTM layer is set to output complete sequence information (i.e., "OutputMode" is "sequence") to preserve the dynamic state representation at each time point.

[0064] Following the LSTM output, a custom attention layer is embedded. This layer employs a functional architecture, specifically additive attention, which calculates the importance weights of features at each time step to focus on key interaction moments, thus adaptively weighting the temporal features. The attention output is further connected to fully connected layers and regression layers to generate continuous brain-muscle interaction metrics.

[0065] The neural network model is trained using the `trainNetwork` function in MATLAB, with mean squared error (MSE) as the loss function. GPU acceleration is supported to improve training efficiency. After training, the model can be exported to intermediate representation formats such as ONNX, and can be deployed on PyTorch, TensorFlow, or edge device platforms, achieving cross-platform inference compatibility. The current version is deployed in the evaluation system with a lightweight inference architecture to perform online real-time modeling tasks.

[0066] Meanwhile, the model supports periodic parameter updates and incremental learning mechanisms to adapt to the proprioceptive assessment needs of different individuals, scenarios and states, significantly enhancing the modeling robustness and generalization ability of the system.

[0067] Ultimately, the brain-muscle interaction index sequence output by the model is presented in time series form, providing core data support for subsequent dynamic visualization analysis and proprioceptive function state tracking.

[0068] Step 5: Generation and visualization analysis of interactive metrics, as follows:

[0069] The low-dimensional sparse feature sequence extracted in step 3 is input in real time into the LSTM-attention mechanism neural network model trained in step 4, and a brain-muscle dynamic interaction index sequence is generated through forward propagation. This index is output in the form of a continuous time series to quantify the interaction strength between EEG and EMG signals at each time point.

[0070] To improve the temporal stability and visualization continuity of the indicators, the system employs a smoothing strategy combining a sliding window and a Gaussian kernel function to dynamically filter the original interactive indicator sequence. Specifically, using a window length of 2L+1 units, a weighted moving average operation is applied at each central time point t, with the weight distribution satisfying the form of a Gaussian kernel function.

[0071]

[0072] Among them, I t+τ : Represents the original interaction index; σ: Gaussian kernel standard deviation, controlling the smoothing intensity; τ is the relative window displacement.

[0073] The smoothed index sequence is further constructed into a two-dimensional interaction matrix to represent the average coupling strength between each feature dimension. Here, "feature dimension" mainly refers to the 32-dimensional low-dimensional sparse features after encoding and compression, which can be understood as the fused channel-frequency composite feature representation. This matrix is ​​dynamically visualized in the form of a heatmap, enhancing its ability to perceive temporal evolution characteristics and spatial structure.

[0074] Ultimately, the system-generated brain-muscle dynamic interaction indicators and their visualization atlas enable real-time and continuous monitoring of the subject's proprioceptive function during movement, exhibiting high timeliness and dynamic response capabilities. The system employs a modular integrated design, supporting a closed-loop process of signal acquisition, feature extraction, interactive modeling, and result visualization. This enhances the automation and system integration of proprioceptive assessment, providing clinicians with efficient, quantitative, and interpretable support for rehabilitation decision-making.

[0075] A proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling includes a multimodal signal acquisition module, a data preprocessing module, a feature compression module, a neural network modeling module, and an index generation and visualization module.

[0076] The multimodal signal acquisition module is used to simultaneously acquire EEG and EMG signals wirelessly, supporting continuous data recording of subjects in motion, and transmitting the acquired data to the host computer in real time for processing and analysis via Wi-Fi communication.

[0077] The data preprocessing module is used to perform filtering, rereference, artifact removal and wavelet time-frequency transformation processing involved in steps 2 and 3, and to extract multi-channel, multi-band fusion time-frequency features of EEG and EMG signals;

[0078] The feature compression module is used to normalize the fused multimodal feature set and use a sparse autoencoder structure to reduce its dimensionality, thereby compressing the feature dimension and improving modeling efficiency and generalization ability.

[0079] The neural network modeling module is used to build and run a deep neural network model based on Long Short-Term Memory (LSTM) and attention mechanisms, enabling dynamic interactive modeling of multimodal feature sequences and supporting real-time online inference. This module adopts a decoupled structure for offline training and online inference, possessing periodic parameter updates and incremental learning capabilities, adapting to dynamic modeling and individual difference adjustment needs; simultaneously, it can run on GPU-optimized computing architectures, supporting edge deployment, low-power operation, and batch inference tasks.

[0080] The indicator generation and visualization module has sliding window analysis and heatmap output functions, which are used to continuously track the temporal evolution of brain-muscle interaction intensity, and realize the visualization and quantitative analysis of proprioceptive function state.

[0081] This embodiment describes the operation process from the user interaction level and the functional modules implemented by the system technology from two dimensions.

[0082] The user operation process is as follows:

[0083] Step 1: Experimental preparation and synchronous signal acquisition.

[0084] Before the experiment began, the researchers used the graphical interface of the host computer for the multimodal signal acquisition module. Figure 4 and Figure 5 Enter the subject's basic information (ID, gender, experimental conditions) in the input field and start the data acquisition module. The subject sits with arms crossed, feet on the foot pedals, and looks at the white cross mark in the center of the screen to begin lower limb flexion and extension exercises.

[0085] During the experiment, the system synchronously acquired EEG and EMG signals in real time via Wi-Fi wireless communication at a sampling frequency of 1000Hz, with each experiment lasting 90 seconds. The system interface displayed 16-channel EEG waveforms and 8-channel EMG spectrum graphs in real time, providing device connection status prompts and data recording management functions to ensure signal quality and the integrity of experimental records.

[0086] Step 2: Real-time data preprocessing and feature extraction.

[0087] During signal acquisition, the data preprocessing module performs real-time filtering, rereference, artifact removal, and time-frequency feature extraction on EEG and EMG signals. EEG signals undergo 0.5–30Hz bandpass filtering, TP9 and TP10 rereference, and incremental independent component analysis (ORICA) to remove artifacts before online Morlet wavelet time-frequency transformation. Similarly, EMG signals undergo FFT analysis, 50Hz notch filtering, and 20–250Hz bandpass filtering before real-time wavelet time-frequency transformation to ensure high-quality real-time data output.

[0088] Step 3: Real-time feature fusion and encoding compression.

[0089] The real-time acquired EEG and EMG signal features are synchronously fused along the channel dimension to construct a fused feature tensor (EEG channel first, EMG channel second). The fused feature matrix is ​​normalized and then input into a sparse autoencoder in real time for feature compression and dimensionality reduction, stably outputting a low-dimensional sparse feature sequence for subsequent real-time modeling.

[0090] Step 4: Real-time modeling with dynamic interaction.

[0091] The low-dimensional sparse sequence after feature compression is input in real time into a neural network model composed of LSTM and additive attention mechanism for online inference, generating brain-muscle interaction index sequences in real time. The model supports offline training, online real-time inference and periodic parameter updates, and has cross-platform inference compatibility. It can be deployed on PyTorch, TensorFlow or edge computing platforms to improve real-time performance and system generalization performance.

[0092] Step 5: Real-time generation and visualization analysis of interactive metrics.

[0093] The model's real-time output sequence of brain-muscle interaction indicators is smoothed in real-time using a sliding window and a Gaussian kernel function to construct a dynamic interaction matrix. A heatmap is then used to display the dynamic trend of brain-muscle interaction intensity over time. The resulting real-time interaction indicators and visualizations provide clinicians with intuitive and efficient quantitative assessment references and personalized rehabilitation decision-making support.

[0094] The functional modules of the proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling are as follows:

[0095] Multimodal signal acquisition module: Enables wireless real-time synchronous acquisition of EEG and EMG signals, real-time data display and recording, and provides stable and convenient remote interaction and management functions;

[0096] Data preprocessing module: Embedded real-time filtering, rereference, artifact removal, and time-frequency feature extraction algorithms to ensure real-time and efficient signal processing capabilities;

[0097] Feature compression module: integrates a sparse autoencoder model, with real-time normalization and feature dimensionality reduction compression functions, improving feature extraction efficiency and data generalization performance;

[0098] Neural Network Modeling Module: Constructs a real-time dynamic interactive model based on LSTM and additive attention mechanism, supports online inference and periodic parameter updates, and is compatible with GPU and edge computing deployment;

[0099] Indicator generation and visualization module: Enables real-time generation, smoothing, and real-time visualization of brain-muscle interaction indicators, providing intuitive interactive dynamic analysis and clinical decision support.

[0100] The above modules work together to form a highly efficient and integrated real-time proprioceptive assessment system, which significantly improves the real-time nature, automation, and ease of operation of clinical assessment.

[0101] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling, characterized in that, The method includes the following steps: Step 1: Simultaneously acquire EEG and EMG signals during the subject's movement using a multimodal acquisition module, and construct the original multi-channel dataset. Step 2: Preprocess the raw data, including bandpass filtering, artifact removal and wavelet transform, to extract multi-channel time-frequency features and form a preprocessed dataset. Step 3: Channel dimension fusion is performed on the preprocessed EEG and EMG features, and sparse autoencoder is used to reduce the dimensionality of the multimodal features to construct a low-dimensional multimodal feature set. Step 4: Using the multimodal feature set obtained in Step 3 as input, construct and train a deep neural network model combining LSTM and attention mechanism to model the dynamic interaction between EEG and EMG signals. Step 5: Based on the model output trained in Step 4, generate a sequence of brain-muscle interaction intensity indicators, and visualize the indicators through sliding window and Gaussian kernel smoothing. Finally, it is used for quantitative assessment and temporal tracking of proprioceptive function.

2. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1, characterized in that, In step 1, the sampling frequency of both EEG and EMG signals is 1000Hz. EEG acquisition uses 16-channel electrodes covering the motor sensory cortex area, while EMG acquisition uses 4-channel electrodes on each side to acquire signals of the main muscle groups of the lower limbs.

3. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 1, the subject sits with arms crossed and feet naturally placed on the foot pedal. During the exercise, the subject focuses on the white cross mark in the center of the screen to avoid interference from eye movement, blinking, or swallowing, thus ensuring the quality of the acquired signal.

4. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 2, the EEG signal is first rereferenced using bilateral mastoids as reference electrodes and then bandpass filtered from 0.5 to 30 Hz; subsequently, incremental independent component analysis (ORICA) is used to remove physiological interference. The preprocessed EEG signal is then subjected to wavelet transform to extract its multi-channel and multi-band time-frequency features. In the EMG signal processing flow, the presence of 50Hz power frequency interference is first determined by Fast Fourier Transform (FFT). If it exists, a notch filter with a bandwidth of 2Hz centered at 50Hz is used to suppress it. Then, a 20-250Hz bandpass filter is used to remove low-frequency noise. Finally, the EMG signal is subjected to wavelet transform to extract its multi-channel and multi-band time-frequency features.

5. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 3, the EEG and EMG time-frequency features extracted in step 2 are concatenated along the channel dimension to construct a fused multimodal feature set. After normalization, the feature set is input into a sparse autoencoder for dimensionality reduction and sparse coding to extract low-dimensional key features for subsequent modeling.

6. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 4, the low-dimensional sparse multimodal feature sequence extracted in step 3 is used as input to construct a deep neural network model that combines LSTM and attention mechanism. LSTM is used to extract time-dependent features in the sequence, and attention mechanism is used to identify key interaction moments and focus on feature segments that contribute significantly to changes in brain-muscle interaction. The model is obtained through offline training and deployed in the system for online inference, supporting periodic model parameter updates and incremental learning.

7. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 5, the low-dimensional sparse feature sequence extracted in step 3 is input into the neural network model trained in step 4. The brain muscle dynamic interaction index sequence is generated through forward propagation, and the sequence is smoothed by sliding window and Gaussian kernel to reduce short-term fluctuation interference and improve the stability and interpretability of the interaction index.

8. The proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1 or 2, characterized in that, In step 5, the smoothed brain-muscle interaction index sequence is constructed into an interaction matrix reflecting the relationship between different feature dimensions, and the average interaction intensity between EEG and EMG signals is visualized by heatmap. The sliding window analysis method is used to perform time series analysis on the interaction intensity index to reveal the dynamic trend of interaction intensity changes over time, thereby realizing the visual tracking of the evolution process of proprioceptive functional state.

9. A system implementing the proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling as described in claim 1, characterized in that, The system includes: The multimodal signal acquisition module is used to simultaneously acquire EEG and EMG signals wirelessly, supporting continuous recording during motion. The data preprocessing module is used to filter, rereference, remove artifacts, and perform time-frequency analysis on the acquired signals, and extract multi-channel and multi-band fused time-frequency features. The feature compression module is used to normalize the fused multimodal feature set and reduce its dimensionality through sparse coding, thereby improving modeling efficiency. The neural network modeling module, based on LSTM and attention mechanisms, enables dynamic interactive modeling of feature sequences and supports real-time online inference. The indicator generation and visualization module is used to generate brain-muscle interaction indicator sequences, and to construct an interaction matrix by combining sliding windows and smoothing processing, and to display the time sequence visualization through heatmaps and other methods.

10. The system as described in claim 9, characterized in that, The multimodal signal acquisition module transmits the acquired EEG and EMG signals to the host computer in real time via Wi-Fi communication for data processing and analysis; the neural network modeling module adopts an offline training and online inference separation structure, supports periodic parameter updates and incremental learning, and is suitable for real-time dynamic modeling and individual adaptive adjustment scenarios. The neural network modeling module is configured to run a GPU-optimized deep learning model, supporting batch inference and online evaluation. It is suitable for low-power deployment scenarios and supports operation on edge computing platforms. The indicator generation and visualization module has sliding window analysis and heatmap output functions, which are used to continuously track the temporal evolution of proprioceptive functional states.

Images