A cross-band neuromuscular coupling analysis method based on VMD and MSCNN

Abstract

The present invention proposes a cross-band neuromuscular coupling analysis method based on VMD and MSCNN, which relates to the field of biological signal processing technology. The method first decomposes EEG and sEMG signals by VMD, and uses MSCNN to extract the time-frequency characteristics of each frequency band; then, the coupling analysis of neuromuscular signals in different frequency bands is performed by using kernelized generalized partial directional coherence (gPDC), and the complex interaction mechanism between cross-bands is deeply explored. This method can not only capture the causal relationship and coupling strength within the same frequency band, but also reveal the nonlinear coupling relationship across frequency bands, overcoming the limitations of traditional signal processing methods in analyzing cross-band interaction mechanisms and the shortcomings faced by same-band coupling analysis. In addition, this method makes full use of the advantages of deep learning models in extracting complex coupling relationships, thereby providing a more comprehensive and accurate analysis perspective for the study of neurorehabilitation assessment and neuroregulatory mechanisms.

Description

Technical Field

[0001] The present invention relates to the technical field of biological signal processing, and in particular to a cross-band neuromuscular coupling analysis method based on VMD and MSCNN. Background Art

[0002] Previous studies on movement mechanisms have mainly focused on the information transmission and interaction patterns within the central nervous system (CNS) and the peripheral nervous system (PNS), and have rarely explored the dynamic feedback and information transmission between the two systems. However, with the continuous advancement of physiological signal acquisition and analysis technology, more and more studies have found that there is a significant synchronization phenomenon between the brain and muscles during movement. This synchronization phenomenon refers to the coordination of brain neural activity (especially EEG) and peripheral muscle activity (surface electromyography, sEMG) in the time domain or frequency domain, reflecting the coupling mechanism between the brain and muscles during movement control.

[0003] This synchronization phenomenon is of great significance in the study of motor control, coordinated movements, and neurorehabilitation. The coupling between the brain and muscles can not only reflect the brain's transmission of motor commands, but also reveal the counteraction of peripheral feedback on the regulation of the central nervous system during neurorehabilitation. However, most existing neuromuscular coupling analysis methods are still limited to coupling in the same frequency band, that is, they only focus on the coupling relationship between EEG and sEMG signals in the same frequency range.

[0004] Same-band coupling methods, such as wavelet packet decomposition, empirical mode decomposition (EMD), Granger causality, coherence analysis, etc., although these traditional methods have certain coupling analysis capabilities in the frequency domain, they are often limited to linear assumptions, limited frequency resolution, and inability to process non-stationary signals. They cannot effectively reveal the complex information transmission and feedback mechanisms between the brain and muscles in dynamic, nonlinear cross-bands. In actual physiological processes, EEG and sEMG signals are usually distributed in different frequency ranges, with EEG mostly located in low frequencies (such as 1-40Hz), while sEMG usually appears in a wider frequency band (20-300Hz). These differences make it difficult to capture the complex interactive relationship between different frequency components in the neuromuscular system by relying solely on same-band coupling methods.

[0005] With the increasing demand for neurorehabilitation and the deepening understanding of motor control mechanisms, the importance of cross-band coupling analysis has become increasingly prominent. Cross-band coupling can not only reveal the regulation of muscle activity at different frequencies by the central nervous system, but also reveal the feedback regulation mechanism of the peripheral nervous system on brain neural activity. This analysis also plays a key role in revealing the neural regulation reconstruction of stroke patients after motor control function damage.

[0006] In summary, how to analyze the cross-frequency band coupling relationship is a problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0007] The technical solution of the present invention to solve the above technical problems is to provide a cross-band neuromuscular coupling analysis method based on VMD and MSCNN, comprising the following steps:

[0008] Step 1: Acquire synchronously collected electroencephalogram (EEG) and surface electromyography (sEMG) signals;

[0009] Step 2: Preprocess the synchronously collected EEG and sEMG signals, apply VMD to decompose the signals, decompose the EEG signal x(t) and sEMG signal y(t) into multiple modes, make each mode have band-limited characteristics in a specific frequency band, and separate the frequencies between the modes from each other;

[0010] Step 3: Design the MSCNN feature extraction framework to extract time-frequency features from the EEG and sEMG signals after variational modal decomposition. The MSCNN feature extraction framework includes at least three groups of convolutional layers. The first group of convolutional layers uses three parallel convolutional kernels for multi-scale feature extraction. The second group of convolutional layers consists of one convolutional layer. The third group of convolutional layers includes multiple convolutional layers in series. Compress the high-dimensional feature vector to the required number of channels, and perform feature fusion on the small-scale, medium-scale and large-scale features extracted by multi-scale calculation.

[0011] Step 4: Construct a cross-modal multivariate autoregression model (MVAR), combine the feature vectors of EEG and sEMG to form the overall input signal matrix, and estimate the coefficient matrix A(k) of the MVAR model;

[0012] Step 5: Calculate the partial directional coherence (PDC) and introduce the Gaussian kernel to expand the PDC into gPDC for analyzing the nonlinear coupling relationship between EEG and sEMG modalities;

[0013] Step 6: According to the changes of gPDC values ​​at different frequencies, identify the coupling relationship between different modal features of EEG and sEMG.

[0014] Furthermore, the step 2 comprises the following steps:

[0015] Optimization objectives and formula derivation:

[0016] Assuming s(t) is the input signal (which can be x(t) or y(t)), the goal of VMD is to decompose s(t) into K modal functions u k (t), each mode function corresponds to a center frequency ω k ;

[0017] The optimization objective function of VMD is:

[0018]

[0019] The objective function is to minimize the modal function u k The bandwidth of (t) is used to make its spectrum as concentrated as possible at its center frequency ω k Nearby, ensuring that the sum of all modes can reconstruct the original signal;

[0020] Add constraints and Lagrange multipliers:

[0021] Introducing constraints:

[0022]

[0023] By using the Lagrange multiplier method, the constraints are added to the optimization objective to form the Lagrange function:

[0024]

[0025] Where λ(t) is the Lagrange multiplier, α is the penalty parameter for controlling the modal bandwidth, and <·,·> represents the inner product;

[0026] Iterative solution:

[0027] The alternating direction multiplier method is used for iterative solution, including:

[0028] Set the number of modes K; set the initial value of the center frequency Initialize modal function and the Lagrange multiplier λ 0 (t);

[0029] For each mode u k (t), under the condition of fixing other modes and center frequencies, the update formula is:

[0030]

[0031] in, is the Fourier transform of the signal s(t); is the mode u i Fourier transform of (t); is the Fourier transform of the Lagrange multiplier λ(t); IFT is the inverse Fourier transform;

[0032] Update the center frequency ω of each mode k :

[0033]

[0034] Determine the center frequency of each mode through weighted frequency estimation;

[0035] Update the Lagrange multiplier λ(t):

[0036]

[0037] Among them, τ is the step size, which is used to adjust the update speed of the Lagrange multiplier;

[0038] By detecting the change in the modal function If it is less than the set threshold, it is judged whether it has converged. If it has converged, the iteration is stopped.

[0039] Furthermore, the step 2 comprises:

[0040] The VMD process is applied to the EEG signal x(t) and the sEMG signal y(t), specifically including:

[0041] Signal setting: For EEG signal x(t), select the modality number K x , set the penalty parameter α x and the initial center frequency

[0042] For the sEMG signal y(t), select the modal number K y , set the penalty parameter α y and the initial center frequency

[0043] Perform VMD decomposition: Perform VMD on the EEG signal x(t) to obtain K x Mode Each mode represents the component of the EEG signal in different frequency bands;

[0044] Perform VMD on the sEMG signal y(t) and get K y Mode

[0045] Save each modal function u x,k (t) and u y,k (t) and the corresponding center frequency ω x,k Red Omega y,k .

[0046] Furthermore, in the step 3 package, the first group of convolutional layers uses three parallel convolution kernels, the number of convolution kernels is 128, the convolution kernel sizes are 3, 5 and 7 respectively, the step length is 1, and no padding; each convolutional layer will perform three operations in sequence: convolution operation, batch normalization and activation function GELU;

[0047] After the output of the first set of convolutional layers, downsampling is performed through a maximum pooling layer with a pooling kernel size of 2 and a stride of 2;

[0048] Entering the second set of convolutional layers, the second set of convolutional layers consists of one convolutional layer with 64 convolution kernels, a convolution kernel size of 3, a stride of 1, and no padding;

[0049] After a maximum pooling layer, the pooling kernel size is also 2, the step size is 2, and downsampling is performed again;

[0050] The third group of convolutional layers includes a plurality of convolutional layers connected in series;

[0051] Among them, the convolution calculation of the first row is no longer involved. The convolution calculation of the second row has two convolution layers in series. The number of convolution kernels is 64, the size is 3, and the step size is 1. Finally, it passes through a convolution layer with 64 convolution kernels, 1 convolution kernel size, and step size 1 for feature integration. The convolution calculation of the third row has three convolution layers in series. The number of convolution kernels of the first two are 64, the size of convolution kernel is 3, and the step size is 1. The number of convolution kernels of the last convolution kernel is 64, the size of convolution kernel is 1, and the step size is 1 for integrating larger scale features. The results of the three rows of convolution calculations are all passed through a convolution layer with 32 convolution kernels, 1 convolution kernel size, and step size 1, without padding. The small-scale, medium-scale, and large-scale features extracted by multi-scale calculations are feature fused.

[0052] Furthermore, the step 4 comprises:

[0053] Construct a multivariate autoregressive model (MVAR) of EEG and sEMG modalities to combine the features of EEG and sEMG to form a joint input signal;

[0054] Combine the EEG and sEMG feature vectors to form an overall input signal matrix:

[0055]

[0056] in, is the EEG modality, is the sEMG modality K y , Z(t) is the joint signal composed of the modal features of EEG and sEMG at time t; for the joint signal Z(t)Z(t), a multivariate autoregression model (MVAR) is constructed:

[0057]

[0058] Where: A(k) is the autoregressive coefficient matrix; p is the order of the autoregressive model; E(t) is the residual vector; by fitting Z(t), the coefficient matrix A(k) of the MVAR model is estimated.

[0059] Furthermore, in step 5,

[0060] Partial Directional Coherence (PDC) is defined as:

[0061]

[0062] Among them, A ij (f) is the coefficient of the MVAR model in the frequency domain, which represents the influence from the jth signal to the ith signal at frequency f;

[0063] The Gaussian kernel function is defined as:

[0064]

[0065] Among them: σ is the parameter of the kernel function, which controls the width of the kernel function; ||X i -X j || is the Euclidean distance, which is used to measure the distance between the feature vectors of the EEG modality and the sEMG modality;

[0066] In the frequency domain, using the extended form of Gaussian kernel, PDC is generalized to the generalized partial directional coherence (gPDC) of Gaussian kernel:

[0067]

[0068] Among them, K(A ij (f),A ii (f)) is the result of applying the Gaussian kernel function to the frequency domain coefficients of the MVAR model;

[0069] gPDC captures the nonlinear coupling relationship between EEG and sEMG modalities through a Gaussian kernel.

[0070] The present invention proposes a method combining variational mode decomposition (VMD) and multi-scale convolutional neural network (MS-CNN). The method first decomposes the EEG and sEMG signals by VMD, and uses MSCNN to extract the time-frequency features of each frequency band. Subsequently, the coupling analysis of nerve-muscle signals in different frequency bands is performed using kernelized generalized partial directional coherence (gPDC), and the complex interaction mechanism across frequency bands is deeply explored. This method can not only capture the causal relationship and coupling strength within the same frequency band, but also reveal the nonlinear coupling relationship across frequency bands, overcoming the limitations of traditional signal processing methods in analyzing cross-frequency band interaction mechanisms and the shortcomings faced by same-frequency band coupling analysis. In addition, this method makes full use of the advantages of deep learning models in extracting complex coupling relationships, thereby providing a more comprehensive and accurate analysis perspective for the study of neurorehabilitation assessment and neuroregulatory mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0072] Figure 1 This is a flowchart of the steps of the cross-band neuromuscular coupling analysis method based on VMD and MSCNN according to the present invention;

[0073] Figure 2 This is the decomposition result diagram of EEG and EMG after VMD according to the present invention;

[0074] Figure 3 This is the MSCNN network architecture diagram of the present invention;

[0075] Figure 4 This is a structural diagram of the EEG and sEMG cross-modal coupling analysis described in the present invention. DETAILED DESCRIPTION

[0076] The present invention proposes a cross-band neuromuscular coupling analysis method based on VMD and MSCNN, aiming to design a method that can realize coupling analysis of EEG and sEMG signals across frequency bands, and proposes an analysis method for feature extraction using VMD and MSCNN.

[0077] The cross-band neuromuscular coupling analysis method based on VMD and MSCNN proposed by the present invention will be described below in a specific embodiment:

[0078] In the technical solution of this embodiment, Figure 1 As shown, a cross-band neuromuscular coupling analysis method based on VMD and MSCNN includes the following steps:

[0079] Step 1: Acquire synchronously collected electroencephalogram (EEG) and surface electromyography (sEMG) signals;

[0080] Specifically, data is collected at a sampling frequency of 1000 Hz, and the start and end time of the action are marked with timestamps. The collected EEG and sEMG signals are divided according to the preparation stage, movement execution stage, and rest stage, and corresponding labels are added according to different stages to ensure that the nerve and muscle activities in each stage can be fully reflected.

[0081] Step 2: Preprocess the synchronously collected EEG and sEMG signals, apply VMD to decompose the signals, decompose the EEG signal x(t) and sEMG signal y(t) into multiple modes, make each mode have band-limited characteristics in a specific frequency band, and separate the frequencies between the modes from each other;

[0082] Specifically, VMD is an adaptive, quasi-orthogonal, non-recursive signal decomposition method that can decompose a signal composed of multiple components into several components with limited bandwidth. Each component revolves around its specific center frequency and has a band-limited characteristic. VMD determines the center frequency and bandwidth of each component by iteratively searching for the optimal variational model, thereby achieving effective separation of the components.

[0083] Compared with recursive decomposition algorithms such as EMD, VMD overcomes the shortcomings of modal aliasing and endpoint effects, and has better noise robustness and noise reduction effects. The essence of VMD is to convert the modal estimation problem into a variational problem, and adaptively decompose the signal into multiple intrinsic mode functions (IMFs) by searching for the optimal solution under the constraints in the frequency domain. Each IMF reflects the characteristics of the signal in different frequency bands. Specifically, after the EEG and sEMG signals are decomposed by VMD, multiple modes can be obtained, each of which has good band-limited characteristics in a specific frequency band, and the frequencies between the modes are separated from each other. Through this decomposition method, the complex structure of EEG and sEMG signals can be displayed, which is convenient for subsequent feature extraction and analysis.

[0084] Step 3: Design the MSCNN feature extraction framework to extract time-frequency features from the EEG and sEMG signals after variational modal decomposition. The MSCNN feature extraction framework includes at least three groups of convolutional layers. The first group of convolutional layers uses three parallel convolutional kernels for multi-scale feature extraction. The second group of convolutional layers consists of one convolutional layer. The third group of convolutional layers includes multiple convolutional layers in series. Compress the high-dimensional feature vector to the required number of channels, and perform feature fusion on the small-scale, medium-scale and large-scale features extracted by multi-scale calculation.

[0085] The present invention designs a multi-scale convolutional neural network framework for extracting time-frequency features from EEG and sEMG signals after variational modal decomposition. The framework makes full use of the parallel characteristics of convolution kernels of different scales and can extract information of multiple scales at the same level, thereby enhancing the representation ability of signal features. The design of the network structure enables MSCNN to provide accurate and effective feature extraction when processing complex neuromuscular signals, providing rich input for subsequent coupling analysis.

[0086] The MSCNN feature extraction structure is as follows Figure 3As shown (the specific parameters of the structure in the present invention are a setting with better results after experiments. The MSCNN network structure parameters include but are not limited to this parameter setting. The parameters mentioned here specifically refer to: the number of channels of the convolution kernel (filter), the size of the convolution kernel, the step size; the size and step size of the maximum pooling)

[0087] VMD is used to decompose the EEG signal into modes, and feature vectors are extracted from each mode. The sEMG signal is also decomposed into modes and feature vectors are extracted.

[0088] Step 4: Construct a cross-modal multivariate autoregression model (MVAR), combine the feature vectors of EEG and sEMG to form the overall input signal matrix, and estimate the coefficient matrix A(k) of the MVAR model;

[0089] Step 5: Calculate the partial directional coherence (PDC) and introduce the Gaussian kernel to expand the PDC into gPDC for analyzing the nonlinear coupling relationship between EEG and sEMG modalities;

[0090] Step 6: According to the changes of gPDC values ​​at different frequencies, identify the coupling relationship between different modal features of EEG and sEMG.

[0091] Furthermore, the step 2 comprises the following steps:

[0092] Optimization objectives and formula derivation:

[0093] Assuming s(t) is the input signal (which can be x(t) or y(t)), the goal of VMD is to decompose s(t) into K modal functions u k (t), each mode function corresponds to a center frequency ω k ;

[0094] The optimization objective function of VMD is:

[0095]

[0096] The objective function is to minimize the modal function u k The bandwidth of (t) is used to make its spectrum as concentrated as possible at its center frequency ω k Nearby, ensuring that the sum of all modes can reconstruct the original signal;

[0097] Add constraints and Lagrange multipliers:

[0098] The introduced constraints are:

[0099]

[0100] By using the Lagrange multiplier method, the constraints are added to the optimization objective to form the Lagrange function:

[0101]

[0102] Where λ(t) is the Lagrange multiplier, α is the penalty parameter for controlling the modal bandwidth, and <·,·> represents the inner product;

[0103] Iterative solution:

[0104] The alternating direction multiplier method is used for iterative solution, including:

[0105] Set the number of modes K; set the initial value of the center frequency Initialize modal function and the Lagrange multiplier λ 0 (t);

[0106] For each mode u k (t), under the condition of fixing other modes and center frequencies, the update formula is:

[0107]

[0108] in, is the Fourier transform of the signal s(t); is the mode u i Fourier transform of (t); is the Fourier transform of the Lagrange multiplier λ(t); IFT is the inverse Fourier transform;

[0109] Update the center frequency ω of each mode k :

[0110]

[0111] Determine the center frequency of each mode through weighted frequency estimation; make its spectrum as concentrated as possible on ω k nearby.

[0112] Update the Lagrange multiplier λ(t):

[0113]

[0114] Among them, τ is the step size, which is used to adjust the update speed of the Lagrange multiplier;

[0115] By detecting the change in the modal function If it is less than the set threshold, it is judged whether it has converged. If it has converged, the iteration is stopped.

[0116] Furthermore, the step 2 comprises:

[0117] The VMD process is applied to the EEG signal x(t) and the sEMG signal y(t), specifically including:

[0118] Signal setting: For EEG signal x(t), select the modality number K x , set the penalty parameter α x and the initial center frequency

[0119] For the sEMG signal y(t), select the modal number K y , set the penalty parameter α y and the initial center frequency

[0120] Perform VMD decomposition: Perform VMD on the EEG signal x(t) to obtain K x Mode Each mode represents the component of the EEG signal in different frequency bands;

[0121] Perform VMD on the sEMG signal y(t) and get K y Mode

[0122] Save each modal function u x,k (t) and u y,k (t) and the corresponding center frequency ω x,k Red Omega y,k .

[0123] Furthermore, in the step 3 package, the first group of convolutional layers uses three parallel convolution kernels, the number of convolution kernels is 128, the convolution kernel sizes are 3, 5 and 7 respectively, the step length is 1, and no padding; each convolutional layer will perform three operations in sequence: convolution operation, batch normalization and activation function GELU;

[0124] After the output of the first set of convolutional layers, downsampling is performed through a maximum pooling layer with a pooling kernel size of 2 and a stride of 2;

[0125] Entering the second set of convolutional layers, the second set of convolutional layers consists of one convolutional layer with 64 convolution kernels, a convolution kernel size of 3, a stride of 1, and no padding;

[0126] After a maximum pooling layer, the pooling kernel size is also 2, the step size is 2, and downsampling is performed again;

[0127] The third group of convolutional layers includes a plurality of convolutional layers connected in series;

[0128] Among them, the convolution calculation of the first row is no longer involved. The convolution calculation of the second row has two convolution layers in series. The number of convolution kernels is 64, the size is 3, and the step size is 1. Finally, it passes through a convolution layer with 64 convolution kernels, 1 convolution kernel size, and step size 1 for feature integration. The convolution calculation of the third row has three convolution layers in series. The number of convolution kernels of the first two are 64, the size of convolution kernel is 3, and the step size is 1. The number of convolution kernels of the last convolution kernel is 64, the size of convolution kernel is 1, and the step size is 1 for integrating larger scale features. The results of the three rows of convolution calculations are all passed through a convolution layer with 32 convolution kernels, 1 convolution kernel size, and step size 1, without padding. The small-scale, medium-scale, and large-scale features extracted by multi-scale calculations are feature fused.

[0129] Specifically, the input signal data is synchronously collected EEG and sEMG signals, which are processed by zero-mean normalization to ensure the validity and stability of the data. Next, multi-scale feature extraction is performed through the first set of convolutional layers.

[0130] Furthermore, if Figure 4 As shown, step 4 includes:

[0131] Construct a multivariate autoregressive model (MVAR) of EEG and sEMG modalities to combine the features of EEG and sEMG to form a joint input signal;

[0132] Combine the EEG and sEMG feature vectors to form an overall input signal matrix:

[0133]

[0134] in, is the EEG modality, is the sEMG modality K y , Z(t) is the joint signal composed of the modal features of EEG and sEMG at time t; for the joint signal Z(t), a multivariate autoregression model (MVAR) is constructed:

[0135]

[0136] Where: A(k) is the autoregressive coefficient matrix; p is the order of the autoregressive model; E(t) is the residual vector; by fitting Z(t), the coefficient matrix A(k) of the MVAR model is estimated.

[0137] Furthermore, in step 5,

[0138] Partial Directional Coherence (PDC) is defined as:

[0139]

[0140] Among them, Aij (f) is the coefficient of the MVAR model in the frequency domain, which represents the influence from the jth signal to the ith signal at frequency f;

[0141] The Gaussian kernel function is defined as:

[0142]

[0143] Among them: σ is the parameter of the kernel function, which controls the width of the kernel function; ||X i -X j || is the Euclidean distance, which is used to measure the distance between the feature vectors of the EEG modality and the sEMG modality;

[0144] In the frequency domain, using the extended form of Gaussian kernel, PDC is generalized to the generalized partial directional coherence (gPDC) of Gaussian kernel:

[0145]

[0146] Among them, K(A ij (f),A ii (f)) is the result of applying the Gaussian kernel function to the frequency domain coefficients of the MVAR model;

[0147] gPDC captures the nonlinear coupling relationship between EEG and sEMG modalities through a Gaussian kernel.

[0148] Specifically, the gPDC matrix gPDC(f) can describe the coupling relationship between the EEG modality and the sEMG modality, and the specific structure is as follows:

[0149] gPDC(f)=[gPDC EEG→EMG ikB EMG→EEG ];

[0150] Among them: gPDC EEG→EMG Represents the coupling relationship from EEG modality to sEMG modality;

[0151] ikB EMG→EEG Represents the coupling relationship between the sEMG modality and the EEG modality.

[0152] The present invention proposes a cross-band neuromuscular coupling analysis method based on the combination of VMD and MS-CNN. Through the multimodal decomposition and time-frequency feature extraction of EEG and sEMG signals, the coupling relationship and information interaction between signals in different frequency bands are effectively revealed. This method makes full use of the frequency band decomposition ability of VMD and the multi-scale feature extraction advantage of MS-CNN, breaks through the limitations of traditional same-band analysis, and realizes cross-band signal coupling analysis. Combined with the kernelized generalized partial directional coherence method, the coupling characteristics of EEG and sEMG signals in different frequency bands and their information flow laws are further explored. In order to prove the rationality and innovation of the proposed patent algorithm, the experiment is designed to obtain EEG signals at leads C3, C4, F3 and F4 through the joint EEG-sEMG acquisition of two action paradigms of arm flexion and rest of volunteers, and use surface electromyography sensors to collect sEMG signals at the anterior deltoid, middle deltoid, biceps, triceps and forearm finger extensor. By analyzing the EEG and sEMG signals at rest and in action, VMD was used to decompose the EEG and EMG of each channel into four modes, which were input into MSCNN for feature extraction. The gPDC values ​​of EEG and sEMG at different frequencies were obtained. The coupling relationship between the different modal features of EEG and sEMG can be identified, and the difference in cross-frequency coupling between resting and action states can be significantly distinguished. It was found that during the cross-frequency band experiment, as the grip strength increased, the high-frequency signals between EEG and sEMG would show stronger synchronous coupling, indicating that the electrical activity of the brain and the electrical activity of the muscles are more closely connected.

[0153] On the other hand, compared with other traditional convolutional network structures, the MSCNN model proposed in this paper combines the advantages of multi-scale parallel convolution and serial convolution. In particular, on the basis of parallel multi-scale convolution kernels, multiple layers of serial convolution are added to further refine feature extraction and feature integration. The first group of convolution layers uses three parallel convolution kernels, and different convolution kernel sizes can simultaneously capture different local scale features in the signal. This design can process information of multiple scales in parallel in the same network layer, thereby significantly improving the sensitivity to signals of different frequencies and time scales. This is different from the way that serial convolution layers extract features step by step. It can capture richer feature information in the early stage and reduce information loss.

[0154] In summary, the present invention can realize the coupling analysis of EEG and sEMG signals across frequency bands, and the proposed method and structure for feature extraction using VMD and MSCNN are innovative, providing a more comprehensive and accurate analysis perspective for neurorehabilitation assessment.

[0155] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A cross-band neuromuscular coupling analysis method based on VMD and MSCNN, characterized in that: The following steps are involved: Step 1: Acquire synchronously collected EEG and sEMG signals; Step 2: Preprocess the synchronously collected EEG and sEMG signals, apply VMD to decompose the signals, decompose the EEG signal x(t) and sEMG signal y(t) into multiple modes, make each mode have band-limited characteristics in a specific frequency band, and separate the frequencies between the modes from each other; Step 3: Design the MSCNN feature extraction framework to extract time-frequency features from the EEG and sEMG signals after variational modal decomposition. The MSCNN feature extraction framework includes at least three groups of convolutional layers. The first group of convolutional layers uses three parallel convolutional kernels for multi-scale feature extraction. The second group of convolutional layers consists of one convolutional layer. The third group of convolutional layers includes multiple convolutional layers in series. The small-scale, medium-scale and large-scale features extracted by multi-scale calculations are fused. Step 4: Construct a cross-modal multivariate autoregressive model, combine the feature vectors of EEG and sEMG to form the overall input signal matrix, and estimate the coefficient matrix A(k) of the MVAR model; Step 5: Calculate the partial directional coherence and introduce the Gaussian kernel to expand the partial directional coherence into gPDC, which is used to analyze the nonlinear coupling relationship between EEG and sEMG modalities; Step 6: According to the changes of gPDC values ​​at different frequencies, identify the coupling relationship between different modal features of EEG and sEMG.

2. The method according to claim 1, characterized in that The step 2 comprises the following steps: Optimization objectives and formula derivation: Assuming s(t) is the input signal, the goal of VMD is to decompose s(t) into K modal functions u k (t), each mode function corresponds to a center frequency ω k ; The optimization objective function of VMD is: The objective function is to minimize the modal function u k The bandwidth of (t) is used to focus its spectrum on its center frequency ω k Nearby, ensuring that the sum of all modes can reconstruct the original signal; Add constraints and Lagrange multipliers, and introduce constraints as follows: By using the Lagrange multiplier method, the constraints are added to the optimization objective to form the Lagrange function: Where λ(t) is the Lagrange multiplier, α is the penalty parameter for controlling the modal bandwidth, and <·,·> represents the inner product; Iterative solution: The alternating direction multiplier method is used for iterative solution, including: Set the number of modes K; set the initial value of the center frequency Initialize modal function and the Lagrange multiplier λ 0 (t); For each mode u k (t), under the condition of fixing other modes and center frequencies, the update formula is: in, is the Fourier transform of the signal s(t); is the mode u i Fourier transform of (t); is the Fourier transform of the Lagrange multiplier λ(t); IFT is the inverse Fourier transform; Update the center frequency ω of each mode k : Determine the center frequency of each mode through weighted frequency estimation; Update the Lagrange multiplier λ(t): Among them, τ is the step size, which is used to adjust the update speed of the Lagrange multiplier; By detecting the change in the modal function If it is less than the set threshold, it is judged whether it has converged. If it has converged, the iteration is stopped.

3. The method according to claim 2, characterized in that The step 2 comprises: The VMD process is applied to the EEG signal x(t) and the sEMG signal y(t), specifically including: Signal setting: For EEG signal x(t), select the modality number K x , set the penalty parameter α x and the initial center frequency For the sEMG signal y(t), select the modal number K y , set the penalty parameter α y and the initial center frequency Perform VMD decomposition: Perform VMD on the EEG signal x(t) to obtain K x Mode Each mode represents the component of the EEG signal in different frequency bands; Perform VMD on the sEMG signal y(t) and get K y Mode Save each modal function u x,k (t) and u y,k (t) and the corresponding center frequency ω x,k Red Omega y,k .

4. The method according to claim 1, characterized in that In the step 3 package, the first group of convolutional layers uses three parallel convolution kernels, the number of convolution kernels is 128, the convolution kernel sizes are 3, 5 and 7 respectively, the step length is 1, and no padding; each convolutional layer will perform three operations in sequence: convolution operation, batch normalization and activation function GELU; Down-sample through a maximum pooling layer with a pooling kernel size of 2 and a stride of 2; enter the second set of convolutional layers, which consists of one convolutional layer with 64 convolution kernels, a convolution kernel size of 3, a stride of 1, and no padding; Through a maximum pooling layer, the pooling kernel size is also 2, the step size is 2, and the sampling is performed again; The third group of convolutional layers includes multiple convolutional layers connected in series; among them, the convolution calculation of the first row does not participate, the convolution calculation of the second row has two convolutional layers connected in series, the number of convolution kernels is 64, the size is 3, and the step size is 1, and finally passes through a convolutional layer with 64 convolution kernels, 1 convolution kernel size, and step size 1 for integrating features; the convolution calculation of the third row has three convolutional layers connected in series, the first two have 64 convolution kernels, 3 convolution kernel size, and step size 1, and the last convolution kernel has 64 convolution kernels, 1 convolution kernel size, and step size 1 for integrating larger scale features, and the convolution calculation results of the three rows are all passed through a convolutional layer with 32 convolution kernels, 1 convolution kernel size, and step size 1, without padding; the small-scale, medium-scale, and large-scale features extracted by multi-scale calculations are feature fused.

5. The method according to claim 1, characterized in that: The step 4 comprises: Construct a multivariate autoregressive model of EEG and sEMG modalities to combine the features of EEG and sEMG to form a joint input signal; Combine the EEG and sEMG feature vectors to form an overall input signal matrix: in, is the EEG modality, is the sEMG modality K y , Z(t) is the joint signal composed of the modal features of EEG and sEMG at time t; for the joint signal Z(t), a multivariate autoregressive model is constructed: Where: A(k) is the autoregressive coefficient matrix; p is the order of the autoregressive model; E(t) is the residual vector; by fitting Z(t), the coefficient matrix A(k) of the MVAR model is estimated.

6. The method according to claim 5, characterized in that In step 5, The partial directional coherence is defined as: Among them, A ij (f) is the coefficient of the MVAR model in the frequency domain, which represents the influence from the jth signal to the ith signal at frequency f; The Gaussian kernel function is defined as: Among them: σ is the parameter of the kernel function, which controls the width of the kernel function; ||X i -X j || is the Euclidean distance, which is used to measure the distance between the feature vectors of the EEG modality and the sEMG modality; In the frequency domain, using the extended form of Gaussian kernel, PDC is generalized to the generalized partial directional coherence of Gaussian kernel: Among them, K(A ij (f),A ii (f)) is the result of applying the Gaussian kernel function to the frequency domain coefficients of the MVAR model; gPDC captures the nonlinear coupling relationship between EEG and sEMG modalities through a Gaussian kernel.

Images