Proprioception evaluation method and system based on electroencephalogram-electromyogram signal fusion and dynamic interaction modeling

Abstract

A proprioceptive evaluation method based on electroencephalogram-electromyogram signal fusion and dynamic interaction modeling, EEG signals and EMG signals of a subject during lower limb movement are synchronously collected; the collected signals are subjected to band pass filtering, artifact removal and wavelet transform processing, multi-channel time-frequency features are extracted, and a pretreatment feature matrix is formed; the time-frequency features of the EEG signals and the EMG signals are fused, a multi-modal feature set is constructed, and a sparse coding method is used to compress the feature dimension; the compressed feature sequence is input into a neural network model combining a long short-term memory network and an attention mechanism, dynamic interaction modeling is performed; an interaction index sequence is generated based on the model output, and an interaction matrix is constructed through sliding window and Gaussian kernel smoothing processing, so that the visualization of brain-muscle interaction intensity and the dynamic quantification of proprioceptive function are realized. The application also provides a system for implementing the method. The application has high objectivity, high feature extraction accuracy and excellent dynamic modeling capability.

Description

Technical Field

[0001] This invention belongs to the field of biomedical signal processing and rehabilitation engineering technology, specifically relating to a dynamic interactive modeling method and system that integrates electroencephalogram (EEG) and electromyogram (EMG) signals. Background Technology

[0002] Proprioception, as the neural basis of human motor coordination, integrates signals from peripheral receptors in the skin, joints, and muscles to construct a real-time perception network of limb movement status and spatial position. This neural information processing mechanism plays a crucial role in optimizing motor coordination, regulating dynamic balance, and maintaining postural stability. Accurate and real-time assessment of proprioceptive function is of great significance for the formulation of neurorehabilitation intervention strategies and the tracking of their effects. Currently used clinical assessment methods, such as passive motion threshold testing, joint position reproduction methods, and active motion recognition methods, mainly rely on subjective scale scoring, which suffers from problems such as strong subjectivity, low quantitative accuracy, and lack of dynamic response capability, making it difficult to fully reveal the neurophysiological mechanisms and changes of proprioception.

[0003] EEG signals can reflect the real-time neural activity state of the cerebral cortex, while EMG signals characterize the electrophysiological responses triggered by muscle contraction. With the continuous development of multimodal sensing technology and biosignal processing algorithms, methods integrating physiological signals such as EEG and EMG provide a new pathway for the objective assessment of proprioception. By simultaneously acquiring EEG and EMG signals, the complete process of peripheral motor information transmission to the central nervous system and the induction of cortical responses can be depicted. This brain-muscle neural interaction model provides important evidence for revealing the physiological mechanisms of proprioception and lays the foundation for the dynamic assessment of its functional state.

[0004] While EEG and EMG signal fusion technology offers a novel approach to the objective assessment of proprioception, most current methods primarily focus on extracting specific time points or static statistical features, lacking systematic modeling of the temporal dependencies and dynamic interaction patterns between EEG and EMG signals. Existing assessment methods, such as Granger causality analysis, coherence analysis, and transfer entropy based on EEG-EMG synchronicity, while revealing the information flow between brain and muscle signals to some extent, often rely on static features, have limited modeling capabilities, and struggle to meet the continuous assessment needs in complex dynamic motion scenarios. Although these methods have exploratory value in early research, they fail to comprehensively reflect the temporal evolution and interaction characteristics of proprioceptive function.

[0005] On the other hand, current multimodal brain-electromyography fusion systems generally suffer from technical limitations: firstly, they lack nonlinear dynamic modeling capabilities, making it difficult to capture deep temporal dependencies between high-dimensional features; secondly, they lack high-resolution visualization methods for interaction intensity, hindering physician understanding and application; and thirdly, most systems employ offline computation processes, failing to support real-time data processing and clinical feedback. These problems directly result in the current assessment results lacking timeliness, stability, and clinical interpretability, limiting their widespread application in rehabilitation medicine.

[0006] Therefore, there is an urgent need for a proprioceptive assessment method and system that integrates EEG and EMG signals, possesses dynamic interactive modeling capabilities, temporal visualization analysis capabilities, and real-time response capabilities. This system should be able to accurately capture the time-varying interaction between brain and muscle signals, dynamically reflect the evolution of proprioceptive function, and thus provide scientific, precise, and real-time support for functional assessment, efficacy tracking, and personalized intervention in the clinical rehabilitation process. Summary of the Invention

[0007] To overcome the problems of high subjectivity, low quantification accuracy, and lack of dynamic response capability in existing technologies for proprioceptive assessment, this invention proposes a proprioceptive assessment method and system based on EEG-EMG signal fusion and dynamic interaction modeling. This method simultaneously acquires EEG and EMG signals during lower limb movement of the subject, constructs a multimodal feature set of brain-muscle signal coupling, and combines a long short-term memory network (LSTM) and attention mechanism to construct a dynamic interaction modeling network. It then extracts and quantifies the dynamic interaction intensity between EEG and EMG, achieving quantitative assessment and visual tracking of proprioceptive function.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] A proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling includes the following steps:

[0010] Step 1: Simultaneously acquire EEG and EMG signals during the subject's movement using a multimodal acquisition module, and construct the original multi-channel dataset.

[0011] Step 2: Preprocess the raw data, including bandpass filtering, artifact removal and wavelet transform, to extract multi-channel time-frequency features and form a preprocessed dataset.

[0012] Step 3: Channel dimension fusion is performed on the preprocessed EEG and EMG features, and sparse autoencoder is used to reduce the dimensionality of the multimodal features to construct a low-dimensional multimodal feature set.

[0013] Step 4: Using the multimodal feature set obtained in Step 3 as input, construct and train a deep neural network model combining LSTM and attention mechanism to model the dynamic interaction between EEG and EMG signals.

[0014] Step 5: Based on the model output trained in Step 4, generate a sequence of brain-muscle interaction intensity indicators, and visualize the indicators through sliding window and Gaussian kernel smoothing. Finally, it is used for quantitative assessment and temporal tracking of proprioceptive function.

[0015] Furthermore, in step 1, the sampling frequency of both EEG and EMG signals is 1000Hz. Specifically, the EEG signal is collected using 16-channel electrodes deployed in the motor sensory cortex, covering the central area and part of the parietal lobe, to acquire neural activity in the cerebral cortex; the EMG signal is collected using 8-channel surface electrodes to acquire electrophysiological signals from the major muscle groups of both lower limbs of the subject.

[0016] Preferably, in step 1, the subject sits with arms crossed and feet naturally placed on the foot pedals. During the exercise, the subject focuses on the white cross mark in the center of the screen to avoid interference from eye movement, blinking, swallowing, and other actions, thus ensuring the stability and effectiveness of the signal during the acquisition period.

[0017] Furthermore, in step 2, the EEG signal is first rereferenced using bilateral mastoid processes as reference electrodes and then bandpass filtered at 0.5–30 Hz. Incremental Independent Component Analysis (ORICA) is then used to remove physiological interferences such as eye movement and blinking myogenic artifacts. The preprocessed EEG signal is then subjected to wavelet transform to extract its multi-channel, multi-band time-frequency features. In the EMG signal processing flow, a Fast Fourier Transform (FFT) is first used to determine if 50 Hz power frequency interference exists. If it does, a notch filter centered at 50 Hz with a bandwidth of 2 Hz is used to suppress it. Then, a 20–250 Hz bandpass filter is used to remove low-frequency noise. Finally, wavelet transform is performed on the EMG signal to extract its multi-channel, multi-band time-frequency features.

[0018] Furthermore, in step 3, the time-frequency features of EEG and EMG extracted in step 2 are concatenated along the channel dimension to construct a fused multimodal feature set. This feature set, after normalization, is input into a sparse autoencoder for dimensionality reduction and sparse coding to extract low-dimensional key features for subsequent modeling and analysis. The optimization objective function of the sparse coding is:

[0019] ;

[0020] in, X To fuse the feature matrix, D It is a dictionary matrix. θ For encoding vectors, λ is the sparse regularization coefficient.

[0021] In step 4, the low-dimensional sparse multimodal feature sequence obtained in step 3 is used as input to construct a deep neural network model based on LSTM and an attention mechanism. This model uses the LSTM structure to extract the temporal dependencies in the sequence features and uses the attention mechanism to identify key interaction moments, focusing on feature segments that significantly affect brain-muscle interaction changes. This enables the modeling of the dynamic coupling relationship between EEG and EMG signals, constructing a brain-muscle interaction representation structure with nonlinear modeling capabilities. The weight calculation method for the attention mechanism is as follows:

[0022] ;

[0023] in, Q For query vector, K t For a moment t Feature representation;

[0024] The model is obtained through offline training and deployed in the system for online inference. It supports periodic model parameter updates and incremental learning mechanisms to ensure that the system has real-time response capabilities and individualized adaptation capabilities.

[0025] In step 5, the low-dimensional sparse feature sequence extracted in step 3 is input into the neural network model trained in step 4, and a brain-muscle dynamic interaction index sequence is generated through forward propagation. Subsequently, the index sequence is smoothed by a combination of sliding window and Gaussian kernel function to reduce short-term fluctuation interference and improve the stability and interpretability of the interaction index. The smoothing calculation method can be expressed as follows:

[0026] ;

[0027] in, I t+τ The original interaction index is within the sliding window; σ is the standard deviation of the Gaussian kernel, which determines the smoothing intensity. L The length of one side of the window is 2L+1, and the total window length is 2L+1.

[0028] In step 5, the smoothed brain-muscle interaction index sequence is constructed into an interaction matrix reflecting the relationship between different feature dimensions, and the average interaction intensity between EEG and EMG signals is visualized by heatmap. The sliding window analysis method is used to perform time series analysis on the interaction intensity index to reveal the dynamic trend of interaction intensity changes over time, thereby realizing the visual tracking of the evolution process of proprioceptive functional state.

[0029] A proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling includes a multimodal signal acquisition module, a data preprocessing module, a feature compression module, a neural network modeling module, and an index generation and visualization module.

[0030] The multimodal signal acquisition module is used to simultaneously acquire EEG and EMG signals wirelessly, supporting continuous data recording of subjects in motion, and transmitting the acquired data to the host computer in real time for processing and analysis via Wi-Fi communication.

[0031] The data preprocessing module is used to perform filtering, rereference, artifact removal and wavelet time-frequency transformation processing involved in steps 2 and 3, and to extract multi-channel, multi-band fusion time-frequency features of EEG and EMG signals;

[0032] The feature compression module is used to normalize the fused multimodal feature set and use a sparse autoencoder structure to reduce its dimensionality, thereby compressing the feature dimension and improving modeling efficiency and generalization ability.

[0033] The neural network modeling module is used to build and run a deep neural network model based on Long Short-Term Memory (LSTM) and attention mechanisms, enabling dynamic interactive modeling of multimodal feature sequences and supporting real-time online inference. This module adopts a decoupled structure for offline training and online inference, possessing periodic parameter updates and incremental learning capabilities, adapting to dynamic modeling and individual difference adjustment needs; simultaneously, it can run on GPU-optimized computing architectures, supporting edge deployment, low-power operation, and batch inference tasks.

[0034] The indicator generation and visualization module has sliding window analysis and heatmap output functions, which are used to continuously track the temporal evolution of brain-muscle interaction intensity, and realize the visualization and quantitative analysis of proprioceptive function state.

[0035] The beneficial effects of this invention are mainly reflected in:

[0036] By simultaneously acquiring EEG and EMG signals and integrating advanced algorithms such as wavelet transform, sparse autoencoder, and deep neural network, a multimodal fusion modeling framework for brain-muscle signals is constructed. This framework can accurately characterize the dynamic interaction between brain and muscle, overcome the shortcomings of traditional proprioceptive assessment methods such as strong subjectivity and low quantification accuracy, and realize an objective quantitative assessment of proprioceptive function with physiological basis.

[0037] By constructing a deep learning structure that combines LSTM and attention mechanisms, it is possible to effectively extract temporal dependencies in physiological signals, adaptively identify key interaction moments, and demonstrate good robustness and generalization ability in dynamic motion states, making it suitable for proprioceptive function assessment in diverse scenarios.

[0038] By introducing a sliding window and Gaussian kernel function smoothing strategy, combined with dynamic interaction matrix and heatmap visualization methods, the temporal evolution of brain-muscle interaction intensity can be clearly presented, providing quantitative support for clinical rehabilitation assessment, assisting doctors in formulating personalized intervention strategies, and significantly improving the practicality and clinical guidance value of the assessment tool. Attached Figure Description

[0039] Figure 1 A flowchart of the lower limb motor proprioception assessment method provided by the present invention;

[0040] Figure 2 The diagram shows the placement of the EEG sensor and EMG sensor of the present invention. (a) shows the placement of the EEG sensor of the present invention, and (b) shows the placement of the EMG sensor. In the diagram, 1 is the rectus femoris muscle, 2 is the biceps femoris muscle, 3 is the gastrocnemius muscle, and 4 is the tibialis anterior muscle.

[0041] Figure 3 This is a schematic diagram of the structure of the lower limb motor proprioception assessment system provided by the present invention;

[0042] Figure 4 The EEG acquisition interface diagram of the lower limb motor proprioception assessment system provided by the present invention is shown.

[0043] Figure 5 An electromyography (EMG) acquisition interface diagram of the lower limb motor proprioception assessment system provided by the present invention;

[0044] Figure 6 The average interaction weight map of the lower limb motor proprioceptive assessment feature dimension provided by this invention. Detailed Implementation

[0045] The present invention will now be further described with reference to the accompanying drawings.

[0046] Reference Figures 1-6 A proprioceptive assessment method based on EEG-EMG signal fusion and dynamic interaction modeling includes the following steps:

[0047] Step 1: Simultaneous acquisition of EEG and EMG signals, as follows:

[0048] EEG and EMG signals were simultaneously collected from the subjects during lower limb flexion and extension movements. The sampling frequency was set to 1000 Hz, and the duration of each experiment was 90 seconds.

[0049] like Figure 2As shown in (a), the EEG signal was generated using a 16-channel Ag-AgCl electrode, which was deployed according to the international 10-20 system, covering the central area, the motor and sensory cortex, and part of the parietal lobe. The selected channels included C1, C2, C3, C4, C5, C6, Cz, CP3, CP4, P3, P4, Pz, FC3, FC4, TP9, and TP10, in order to comprehensively capture cortical neural activity related to lower limb movement.

[0050] like Figure 2 As shown in (b), the EMG signal was obtained by using surface electrodes placed on four major muscle groups of the lower limb: rectus femoris, biceps femoris, gastrocnemius and tibialis anterior, arranged symmetrically on the left and right sides, with a total of 8 channels, to synchronously record the electrical activity information of muscle groups during the flexion and extension movements of the lower limb.

[0051] During the experiment, the subjects sat with their arms crossed and their feet naturally placed on the foot pedals. They focused on the white cross mark in the center of the screen during the flexion and extension movements of their lower limbs, avoiding distractions such as eye movement, blinking, and swallowing to ensure the quality of the collected signal.

[0052] Step 2: Signal preprocessing and time-frequency feature extraction, as follows:

[0053] The acquired EEG and EMG signals are preprocessed online through an embedded signal processing module, including filtering, rereference, artifact removal, and time-frequency transformation.

[0054] The EEG signal is first bandpass filtered from 0.5 to 30 Hz to retain the dominant frequency components relevant to the motion task. Then, a rereference operation is performed using the TP9 and TP10 electrodes as reference channels. The artifact removal process employs incremental independent component analysis (ORICA) or its lightweight variant to dynamically identify and remove artifacts such as eye movements, blinks, and electromyography within a sliding time window. The cleaned EEG signal is then subjected to online Morlet wavelet transform to extract multi-channel, multi-band time-frequency features, continuously outputting a high-dimensional EEG feature matrix.

[0055] The EMG signal processing procedure includes: First, analyzing the signal spectrum using Fast Fourier Transform (FFT) to identify the presence of significant 50 Hz power frequency interference. If this frequency component is detected, a notch filter centered at 50 Hz with a bandwidth of 2 Hz is used to suppress it. Subsequently, a third-order bandpass filter based on Butterworth design is used to perform online bandpass filtering of the EMG signal from 20-250 Hz to remove low-frequency motion artifacts and high-frequency noise. After filtering, wavelet transform is also used to extract multi-channel, multi-band EMG time-frequency features, forming a high-dimensional EMG feature matrix. The entire preprocessing flow is implemented using a streaming processing structure to ensure high stability and low latency continuous feature extraction during acquisition.

[0056] Step 3: Real-time feature fusion and encoding compression, as follows:

[0057] The system synchronously concatenates the multi-channel time-frequency features of the EEG and EMG signals extracted in step 2 along the channel dimension to construct a fused feature tensor with the shape [channel × time × frequency]. The channel dimension is arranged in the order of "EEG channel first, EMG channel second" to maintain a clear division of the signal source structure. This tensor is continuously updated during data acquisition and converted into a two-dimensional matrix with the structure [time × (number of channels × number of frequencies)] through streaming reconstruction to adapt to the input requirements of subsequent neural network models.

[0058] To improve the numerical stability and generalization ability of the model, the system integrates a unified normalization and feature compression module. The fused feature matrix is ​​first normalized to map all feature values ​​to the [0, 1] interval, and then input into the sparse autoencoder model deployed in the system kernel for dimensionality reduction and encoding.

[0059] The sparse autoencoder has undergone parameter optimization during the training phase. Its hidden layer node count is set to 32 by default, and can be flexibly adjusted between 20 and 40 depending on the fusion feature dimension to balance feature compression effectiveness and computational efficiency. The sparsity regularization parameter λ is set to 0.1, the decoding function uses a pure linear function (purelin), and GPU-accelerated training is supported. This model achieves effective extraction and compression of brain-muscle coupling features by minimizing reconstruction error and constraining activation sparsity.

[0060] During the system deployment phase, the autoencoder model operates in a lightweight forward inference mode, which can process the fused feature stream in real time and stably output a sparse feature sequence with a shape of [time×32] as the standard input to the LSTM-attention mechanism neural network modeling module.

[0061] Step 4: Brain-muscle dynamic interaction modeling, as follows:

[0062] The low-dimensional sparse multimodal feature sequence obtained in step 3 is used as the input to the neural network model. The input data shape is [time × 32], where each time step corresponds to a fused sparse feature vector.

[0063] In the network structure, a sequence input layer is first set as a 32-dimensional temporal feature vector; then an LSTM layer is connected to extract the temporal dependencies and dynamic evolution patterns in the feature sequence. The LSTM layer is set to output complete sequence information (i.e., "OutputMode" is "sequence") to preserve the dynamic state representation at each time point.

[0064] Following the LSTM output, a custom attention layer is embedded. This layer employs a functional architecture, specifically additive attention, which calculates the importance weights of features at each time step to focus on key interaction moments, thus adaptively weighting the temporal features. The attention output is further connected to fully connected layers and regression layers to generate continuous brain-muscle interaction metrics.

[0065] The neural network model is trained using the `trainNetwork` function in MATLAB, with mean squared error (MSE) as the loss function. GPU acceleration is supported to improve training efficiency. After training, the model can be exported to intermediate representation formats such as ONNX, and deployed on PyTorch, TensorFlow, or edge device platforms, achieving cross-platform inference compatibility. The current version is deployed in the evaluation system with a lightweight inference architecture to perform online real-time modeling tasks.

[0066] Meanwhile, the model supports periodic parameter updates and incremental learning mechanisms to adapt to the proprioceptive assessment needs of different individuals, scenarios and states, significantly enhancing the modeling robustness and generalization ability of the system.

[0067] Ultimately, the brain-muscle interaction index sequence output by the model is presented in time series form, providing core data support for subsequent dynamic visualization analysis and proprioceptive function state tracking.

[0068] Step 5: Generation and visualization analysis of interactive metrics, as follows:

[0069] The low-dimensional sparse feature sequence extracted in step 3 is input in real time into the LSTM-attention mechanism neural network model trained in step 4, and a brain-muscle dynamic interaction index sequence is generated through forward propagation. This index is output in the form of a continuous time series to quantify the interaction strength between EEG and EMG signals at each time point.

[0070] To improve the temporal stability and visualization continuity of the indicators, the system employs a smoothing strategy combining a sliding window and a Gaussian kernel function to dynamically filter the original interactive indicator sequence. Specifically, the method involves using a window length of 2... L +1 is the unit, at each central time point t The weighted moving average operation is applied, and the weight distribution satisfies the Gaussian kernel function form:

[0071] ;

[0072] in, I t+τ: Represents the original interaction index; σ: Gaussian kernel standard deviation, controlling the smoothing intensity; τ This represents the relative window displacement.

[0073] The smoothed index sequence is further constructed into a two-dimensional interaction matrix to represent the average coupling strength between each feature dimension. Here, "feature dimension" mainly refers to the 32-dimensional low-dimensional sparse features after encoding and compression, which can be understood as the fused channel-frequency composite feature representation. This matrix is ​​dynamically visualized in the form of a heatmap, enhancing its ability to perceive temporal evolution characteristics and spatial structure.

[0074] Ultimately, the system-generated brain-muscle dynamic interaction indicators and their visualization atlas enable real-time and continuous monitoring of the subject's proprioceptive function during movement, exhibiting high timeliness and dynamic response capabilities. The system employs a modular integrated design, supporting a closed-loop process of signal acquisition, feature extraction, interactive modeling, and result visualization. This enhances the automation and system integration of proprioceptive assessment, providing clinicians with efficient, quantitative, and interpretable support for rehabilitation decision-making.

[0075] A proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling includes a multimodal signal acquisition module, a data preprocessing module, a feature compression module, a neural network modeling module, and an index generation and visualization module.

[0076] The multimodal signal acquisition module is used to simultaneously acquire EEG and EMG signals wirelessly, supporting continuous data recording of subjects in motion, and transmitting the acquired data to the host computer in real time for processing and analysis via Wi-Fi communication.

[0077] The data preprocessing module is used to perform filtering, rereference, artifact removal and wavelet time-frequency transformation processing involved in steps 2 and 3, and to extract multi-channel, multi-band fusion time-frequency features of EEG and EMG signals;

[0078] The feature compression module is used to normalize the fused multimodal feature set and use a sparse autoencoder structure to reduce its dimensionality, thereby compressing the feature dimension and improving modeling efficiency and generalization ability.

[0079] The neural network modeling module is used to build and run a deep neural network model based on Long Short-Term Memory (LSTM) and attention mechanisms, enabling dynamic interactive modeling of multimodal feature sequences and supporting real-time online inference. This module adopts a decoupled structure for offline training and online inference, possessing periodic parameter updates and incremental learning capabilities, adapting to dynamic modeling and individual difference adjustment needs; simultaneously, it can run on GPU-optimized computing architectures, supporting edge deployment, low-power operation, and batch inference tasks.

[0080] The indicator generation and visualization module has sliding window analysis and heatmap output functions, which are used to continuously track the temporal evolution of brain-muscle interaction intensity, and realize the visualization and quantitative analysis of proprioceptive function state.

[0081] This embodiment describes the operation process from the user interaction level and the functional modules implemented by the system technology from two dimensions.

[0082] The user operation process is as follows:

[0083] Step 1: Experimental preparation and synchronous signal acquisition.

[0084] Before the experiment began, the researchers used the graphical interface of the host computer for the multimodal signal acquisition module ( Figure 4 and Figure 5 Enter the subject's basic information (ID, gender, experimental conditions) in the input field and start the data acquisition module. The subject sits with arms crossed, feet on the foot pedals, and looks at the white cross mark in the center of the screen to begin lower limb flexion and extension exercises.

[0085] During the experiment, the system synchronously acquired EEG and EMG signals in real time via Wi-Fi wireless communication at a sampling frequency of 1000 Hz, with each experiment lasting 90 seconds. The system interface displayed 16-channel EEG waveforms and 8-channel EMG spectrograms in real time, providing device connection status prompts and data recording management functions to ensure signal quality and the integrity of experimental records.

[0086] Step 2: Real-time data preprocessing and feature extraction.

[0087] During signal acquisition, the data preprocessing module performs real-time filtering, rereference, artifact removal, and time-frequency feature extraction on EEG and EMG signals. EEG signals undergo 0.5–30 Hz bandpass filtering, TP9 and TP10 rereference, and incremental independent component analysis (ORICA) to remove artifacts before online Morlet wavelet time-frequency transformation. EMG signals undergo FFT analysis, 50 Hz notch filtering, and 20–250 Hz bandpass filtering, followed by real-time wavelet time-frequency transformation to ensure high-quality real-time data output.

[0088] Step 3: Real-time feature fusion and encoding compression.

[0089] The real-time acquired EEG and EMG signal features are synchronously fused along the channel dimension to construct a fused feature tensor (EEG channel first, EMG channel second). The fused feature matrix is ​​normalized and then input into a sparse autoencoder in real time for feature compression and dimensionality reduction, stably outputting a low-dimensional sparse feature sequence for subsequent real-time modeling.

[0090] Step 4: Real-time modeling with dynamic interaction.

[0091] The low-dimensional sparse sequence after feature compression is input in real time into a neural network model composed of LSTM and additive attention mechanism for online inference, generating brain-muscle interaction index sequences in real time. The model supports offline training, online real-time inference and periodic parameter updates, and has cross-platform inference compatibility. It can be deployed on PyTorch, TensorFlow or edge computing platforms to improve real-time performance and system generalization performance.

[0092] Step 5: Real-time generation and visualization analysis of interactive metrics.

[0093] The model's real-time output sequence of brain-muscle interaction indicators is smoothed in real-time using a sliding window and a Gaussian kernel function to construct a dynamic interaction matrix. A heatmap is then used to display the dynamic trend of brain-muscle interaction intensity over time. The resulting real-time interaction indicators and visualizations provide clinicians with intuitive and efficient quantitative assessment references and personalized rehabilitation decision-making support.

[0094] The functional modules of the proprioceptive assessment system based on EEG-EMG signal fusion and dynamic interaction modeling are as follows:

[0095] Multimodal signal acquisition module: Enables wireless real-time synchronous acquisition of EEG and EMG signals, real-time data display and recording, and provides stable and convenient remote interaction and management functions;

[0096] Data preprocessing module: Embedded real-time filtering, rereference, artifact removal, and time-frequency feature extraction algorithms to ensure real-time and efficient signal processing capabilities;

[0097] Feature compression module: integrates a sparse autoencoder model, with real-time normalization and feature dimensionality reduction compression functions, improving feature extraction efficiency and data generalization performance;

[0098] Neural Network Modeling Module: Constructs a real-time dynamic interactive model based on LSTM and additive attention mechanism, supports online inference and periodic parameter updates, and is compatible with GPU and edge computing deployment;

[0099] Indicator generation and visualization module: Enables real-time generation, smoothing, and real-time visualization of brain-muscle interaction indicators, providing intuitive interactive dynamic analysis and clinical decision support.

[0100] The above modules work together to form a highly efficient and integrated real-time proprioceptive assessment system, which significantly improves the real-time nature, automation, and ease of operation of clinical assessment.

[0101] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A proprioceptive evaluation method based on electroencephalography-electromyography signal fusion and dynamic interaction modeling, characterized in that, The method comprises the following steps: Step 1, synchronously collecting the electroencephalogram (EEG) and electromyogram (EMG) signals of a subject during movement by a multi-modal acquisition module, and constructing an original multi-channel data set; Step 2, preprocessing the original data, including band-pass filtering, artifact removal and wavelet transform steps, extracting multi-channel time-frequency features, and forming a preprocessed data set; Step 3, fusing the channel dimensions of the preprocessed EEG and EMG features, and using a sparse autoencoder to reduce the dimension of the multi-modal features, to construct a low-dimensional multi-modal feature set; Step 4, taking the multi-modal feature set obtained in step 3 as input, constructing and training a deep neural network model combining LSTM and attention mechanism, to model the dynamic interaction between the EEG and EMG signals; Step 5, based on the output of the model trained in step 4, generating a brain-muscle interaction intensity index sequence, and visualizing the index through a sliding window and Gaussian kernel smoothing process, to finally quantitatively evaluate and time-series track the proprioceptive function.

2. The somatosensory evaluation method based on electroencephalography-electromyography fusion and dynamic interaction modeling according to claim 1, wherein, In step 1, the sampling frequency of the EEG signal and the EMG signal is 1000 Hz, the EEG acquisition uses 16-channel electrodes to cover the motor sensory cortex area, and the EMG acquisition uses 4-channel electrodes on both sides to collect the signals of the main muscle groups of the lower limbs.

3. The somatosensory evaluation method based on electroencephaloelectromyoelectric signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 1, the subject adopts a sitting posture, holds arms, and naturally places feet on the footboard, and looks at the white cross mark in the center of the screen during movement to avoid visual drift, blinking, and swallowing actions that interfere with signal quality.

4. The somatosensory evaluation method based on electroencephaloelectromyoelectric signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 2, the EEG signal is first re-referenced using bilateral mastoid processes and band-pass filtered at 0.5-30 Hz; then, it is processed using incremental independent component analysis (ORICA) to remove physiological interference; The preprocessed EEG signal is then wavelet transformed to extract its multi-channel, multi-frequency time-frequency features; in the EMG signal processing flow, first, the presence of 50 Hz power frequency interference is determined by fast Fourier transform (FFT), and if present, a notch filter centered at 50 Hz with a bandwidth of 2 Hz is used to suppress it; then, 20-250 Hz band-pass filtering is used to remove low-frequency noise, and finally, the EMG signal is wavelet transformed to extract its multi-channel, multi-frequency time-frequency features.

5. The somatosensory evaluation method based on electroencephaloelectromyoelectric signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 3, the EEG and EMG time-frequency features extracted in step 2 are concatenated in the channel dimension to construct a fused multi-modal feature set; after normalization, the feature set is input into a sparse autoencoder for dimension reduction and sparse coding to extract low-dimensional key features for subsequent modeling.

6. The proprioceptive assessment method based on electroencephaloelectromyographic signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 4, the low-dimensional sparse multi-modal feature sequence extracted in step 3 is taken as input to construct a deep neural network model combining LSTM and attention mechanism, which uses LSTM to extract time-dependent features in the sequence and combines attention mechanism to identify key interaction times and focus on feature segments that significantly contribute to brain-muscle interaction changes; The model is obtained through offline training and deployed in the system for online inference, supporting periodic model parameter updates and incremental learning.

7. The somatosensory evaluation method based on electroencephaloelectromyoelectric signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 5, the low-dimensional sparse feature sequence extracted in step 3 is input into the neural network model trained in step 4, and a brain-muscle dynamic interaction index sequence is generated by forward propagation. The sequence is subjected to sliding window and Gaussian kernel smoothing to weaken short-term fluctuation interference and improve the stability and interpretability of the interaction index.

8. The somatosensory evaluation method based on electroencephaloelectromyoelectric signal fusion and dynamic interaction modeling according to claim 1 or 2, characterized in that, In step 5, the smoothed brain-muscle interaction index sequence is constructed into an interaction matrix reflecting the relationship between different feature dimensions, and the average interaction intensity between EEG and EMG signals is visualized by a heat map. The sliding window analysis method is used to analyze the interaction intensity index in time sequence, revealing the dynamic trend of the interaction intensity over time, thereby realizing the visual tracking of the evolution process of the proprioceptive function state.

9. The system based on the somatosensory evaluation method of the electroencephalogram-electromyogram fusion and dynamic interaction modeling according to claim 1, characterized in that, The system comprises: A multi-modal signal acquisition module for synchronously acquiring EEG signals and EMG signals through wireless means, supporting continuous recording under motion state; A data preprocessing module for filtering, re-referencing, artifact removal and time-frequency analysis of the acquired signals, extracting multi-channel, multi-band fusion time-frequency features; A feature compression module for normalizing the fused multi-modal feature set and reducing dimension through sparse coding to improve modeling efficiency; A neural network modeling module based on LSTM and attention mechanism for dynamic interaction modeling of feature sequences, supporting real-time online inference; An index generation and visualization module for generating brain-muscle interaction index sequences, constructing interaction matrices combined with sliding window and smoothing, and performing time series visualization through heat maps and other means.

10. The system of claim 9, wherein, The multi-modal signal acquisition module transmits the acquired EEG and EMG signals to the host computer in real time through Wi-Fi communication for data processing and analysis. The neural network modeling module adopts a separate structure for offline training and online inference, supports periodic parameter updating and incremental learning, and is suitable for real-time dynamic modeling and individual adaptive adjustment scenarios. The neural network modeling module is configured to run a GPU-optimized deep learning model, supports batch inference and online evaluation, is suitable for low-power deployment scenarios, and supports edge computing platform operation. The index generation and visualization module has sliding window analysis and heat map output functions for continuous tracking of the time series evolution of the proprioceptive function state.

Images