A method for EEG recognition of natural hand movements based on time-frequency multi-layer brain network

Abstract

The present invention discloses a method for EEG recognition of natural hand movements based on a time-frequency multi-layer brain network, comprising: (1) collecting multi-channel EEG signals of natural hand movements; (2) preprocessing the multi-channel EEG signals to extract the delta wave, theta wave, alpha wave, beta wave, and gamma wave at each time point; (3) constructing a time-domain multi-layer brain network using a wSAR model; (4) calculating a frequency-domain multi-layer brain network using cross-frequency coupling; (5) combining and standardizing the time-domain and frequency-domain multi-layer brain networks; (6) calculating the multi-layer network metric and the decomposed super-adjacency matrix of the multi-layer brain network, and inputting them into a two-layer graph convolutional network (GCN) to fuse artificial, shallow, and deep features for classification. The present invention proposes a new time-frequency fusion multi-layer brain network using the wSAR model and cross-frequency coupling (CFC), and uses two layers of GCN to comprehensively extract the feature information of the multi-layer brain network, which is conducive to revealing the neurophysiological mechanism of the brain and improving the decoding accuracy of natural movement EEG.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram (EEG) signal processing, and in particular relates to an EEG recognition method for natural hand movements based on a time-frequency multi-layer brain network. Background Art

[0002] A brain-computer interface (BCI) is a method of controlling external devices through EEG signals. It collects and analyzes EEG signals, and decodes them into control instructions that can be recognized by the machine, thereby achieving control of external devices. Currently, brain-computer interfaces are widely used in the fields of medical rehabilitation and neuroengineering. In the study of EEG signals of natural hand movements, the sophistication of human hand movements and external interference lead to the complexity of natural movements. Many natural movements require the coordinated work of multiple brain regions to be realized. Traditional EEG feature extraction methods only consider the individual features of some brain regions, which makes it difficult for traditional EEG feature extraction methods to extract more effective features.

[0003] Given the growing demand for joint analysis of multiple brain regions, brain network analysis has become a hot topic in brain-computer interfaces. Brain network analysis can explore neural connections in the brain at both macroscopic and microscopic scales, and can also uncover distinct connections between individual brain regions at the structural, functional, or causal-effect levels. In recent years, brain network analysis has primarily focused on functional and causal-effect connectivity.

[0004] Although brain network analysis has shown strong vitality, the information represented by a single-layer brain network only reflects the relationship between specific times or specific frequencies, and the resulting feature information is still very limited. Therefore, multi-layer brain network analysis is necessary in current research. Due to the ambiguity of the definition of multi-layer brain networks, most current multi-layer brain network analysis studies have ignored the importance of inter-layer connections, resulting in information loss in multi-layer brain networks. In addition, many previous studies have only considered multi-layer brain networks in the frequency domain or dynamic brain networks in the time domain, while ignoring multi-layer brain networks in the time domain, which fails to fully utilize multi-layer brain network information. Summary of the Invention

[0005] To solve the above problems, the present invention discloses a method for EEG recognition of natural hand movements based on a time-frequency multi-layer brain network. This method proposes a new possibility for inter-layer connection between time domain and frequency domain multi-layer brain networks, and also provides a new method for comprehensive analysis of natural hand movement EEG.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network, comprising the following steps:

[0008] (1) Collecting multi-channel EEG signals of natural hand movements

[0009] (2) Preprocess the multi-channel EEG signals and use a sliding time window to extract the EEG signals at each time point, and extract the delta wave, theta wave, alpha wave, beta wave, and gamma wave at each time point.

[0010] (3) Data at each time point are processed, using electrodes as nodes and directional transfer functions (DTFs) as intra-layer connections to construct a time-domain multilayer brain network. Specifically, the time-domain inter-layer connections are calculated as follows: A weighted quadratic autoregressive (wSAR) model is constructed: Based on the assumption that closer temporal distances have a greater impact, a weighted term is added to the AR model, and the constant term is processed. The improved wSAR model parameters are calculated as inter-layer connections.

[0011] (4) The data of each frequency band are processed to calculate the cross-frequency inter-layer coupling. The calculation method of the frequency domain inter-layer connection is as follows: perform Hilbert transform on the EEG signal of each frequency, extract the amplitude and phase information, and calculate the cross-frequency coupling (CFC) between each frequency. Here, the average value of the phase-amplitude coupling (PAC), phase-phase coupling (PPC) and amplitude-amplitude coupling (AAC) indicators is selected as the final inter-layer connection.

[0012] (5) The time-frequency multilayer brain network constructed by (3) and (4) is combined into a 3D standard form: M = (V, E, ω, l), where V represents the point set, E represents the edge set, ω represents the size of the edge, and 1 represents the position mark of the point. This standard form abstracts the brain area into the z-axis, time as the x-axis, and frequency as the y-axis, that is, compressing the single-layer network into one dimension. As the number of layers increases, the dimensionality of the multilayer brain network increases.

[0013] (6) Calculate the standard form of multi-layer brain network metrics in (5), including common multi-layer degree, eigenvector centrality, and temporal flexibility unique to multi-layer brain networks, for neural analysis of natural action decoding. Finally, calculate the adjacency matrix and input this standard form into a graph convolutional network (GCN) for classification and recognition.

[0014] Furthermore, step (2) includes:

[0015] (2.1) The EEG signal was low-pass filtered at 100 Hz, outliers were removed, and linear interpolation was used to supplement the data.

[0016] (2.2) Use the ICA blind source separation algorithm to remove other bioelectric components, and then perform segmentation, baseline correction and public average reference.

[0017] (2.3) The processed EEG signal is filtered using a 4th-order Butterworth IIR filter at 0.3-3 Hz (δ wave), 4-8 Hz (θ wave), 8-13 Hz (α wave), 13-30 Hz (β wave), and 30-45 Hz (γ wave). The EEG signal of each frequency is then sampled using a time window of 1 s and a step size of 100 ms to obtain the signal value at the corresponding time point.

[0018] Furthermore, step (3) includes:

[0019] (3.1) Select the EEG signal of a certain channel in a certain frequency band and construct a wSAR model for the EEG signal:

[0020]

[0021] in represents the EEG signal value of the jth channel at time i, represents the model parameter value of channel j, ω j represents the constant term, which we assume here as the initial value of the influence of each channel, ε represents the noise, the model order is m, and k is the weighting value. In order to avoid the k value being too large, which will cause the subsequent time term to lose its meaning, we set k to arrive At the same time, in order to consider the optimal order, the initial value of m is set to 1 here.

[0022] (3.2) The wSAR model formula is rewritten as follows:

[0023]

[0024] Using the least squares method, let ε 2 to a j The partial derivative of is equal to 0, which minimizes the residual sum of squares.

[0025] In order to make the formula expression simpler, the wSAR model is recorded in matrix form below.

[0026]

[0027] Where N is the total number of time points, and the least squares estimate of the parameter matrix A is

[0028]

[0029] where X j is the signal matrix of the past time in the wSAR model, Y j is the signal matrix of the future time in the wSAR model.

[0030] (3.3) The EEG signals of all channels in this frequency band are calculated by (3.2) to obtain all the parameter matrices A j According to the actual situation, when there is no action intention and action execution, each channel should have the same initial value for the action response, so it is necessary to align the initial values ​​of each channel. This method takes each parameter matrix A j The constant term ω in j The mean of ω avg ,Right now

[0031]

[0032] Where M represents the total number of channels, ω j is the constant term of each channel parameter matrix. Then ω avg Substitute back the original AR model formula of all channels and repeat step (3.2) to obtain the final parameter matrix B j .

[0033] (3.4) Take the order m from 1 to 10, repeat the steps (3.2) and (3.3), and calculate the autocovariance function σ1, σ2, ..., σ of each order sample. 10 According to the final prediction error (FPE) criterion, calculate the order that makes FPE reach the minimum, and select the parameter matrix B of this order j .

[0034] (3.5) Multiply both sides of the AR model equation for all channels by a time lag term And perform z-transform to obtain the transfer function H of the system ij (f) represents the transfer relationship from channel i to channel j. Then normalize it to get the final directional transfer function (DTF)

[0035]

[0036] Among them H im (f) represents the transfer function from channel i to channel m.

[0037] (3.6) All the signals of all frequency bands are calculated through steps (3.2)-(3.5), and finally a time-domain multi-layer brain network is constructed: the electrodes are used as nodes; a threshold of 60% is taken to remove false connections, and the final obtained As an intra-layer connection, that is, the connection from node i to node j in the same layer is Take a 60% threshold to remove false connections and transform the parameter matrix B j As an inter-layer connection, the connection from node i in layer k to node i in layer 1 in frequency band j is b m+1-|k-l| .

[0038] Furthermore, step (4) includes:

[0039] (4.1) Perform Hilbert transform on the EEG signals of each frequency at all time points to obtain the amplitude and phase values ​​at each time point.

[0040] (4.2) Calculate the phase-amplitude coupling (PAC), phase-phase coupling (PPC), and amplitude-amplitude coupling (AAC) between any two frequencies:

[0041]

[0042] Where n is the total number of time points in the EEG data, represents the amplitude of the high-frequency band H at time i, represents the phase of the low-frequency band L at time i.

[0043]

[0044] in and They represent the phase values ​​of frequency bands a and b at time i respectively. The imaginary part and sgn function are taken to reduce the influence of volume conduction effect and common source.

[0045] AAC=Corr(A i , A j )

[0046] Among them, Corr() represents the Pearson correlation coefficient, A i With A j Represent the amplitudes of frequency bands i and j respectively.

[0047] (4.3) Calculate the average value of PAC, PPC, and AAC as the cross-frequency coupling value CFC, take a threshold of 60%, remove false connections, and use the CFC value as the inter-layer coupling between corresponding frequencies to form the frequency domain part of the multi-layer brain network.

[0048] Furthermore, step (5) specifically includes:

[0049] The obtained multi-layer brain network in the time-frequency domain is written in a standardized form: M = (V, E, ω, 1), where the brain area is the z-axis, time is the x-axis, and frequency is the y-axis. Where V is a point set, including the point labels, such as ("v1", ..., "v N ”); E is an edge set, including the label of the edge, such as (“e 12 ”,……,”e N(N-1) ”); ω is the connection value of the edge, such as (ω(v i , v j), ...); 1 is the location label of point V, which is used to indicate the location of the point, so L is a two-dimensional matrix, such as (("v1", 1, 1, 1), ..., ("v 25 ”, 2, 5, 6), …), where since the present invention has three dimensions, the three numbers after the point number are the coordinates of the point. The reason for standardizing the multi-layer brain network is firstly to facilitate visualization and retrieval, and secondly to facilitate the unified processing of subsequent metric calculations and GCN.

[0050] Furthermore, step (6) specifically includes:

[0051] (6.1) Calculating the metrics of time-frequency multilayer brain networks: multilayer degree d M (i), eigenvector centrality The node flexibility F is used for neurophysiological analysis. At the same time, the manually extracted features are input into the last layer of GCN for feature weighted fusion.

[0052]

[0053]

[0054]

[0055]

[0056] where d M (i) is v i The total node degree of , a(i, j, k) is the element in the adjacency tensor of the time-frequency multilayer brain network; is the adjacency tensor of the multi-layer brain network, is the eigenvector of the tensor corresponding to the eigenvalue λ; f i is the number of changes of a node community at each time point, N is the total number of changes, and F j For node v j The node flexibility; Q is a modular function used for community detection, where A ijs is the adjacency matrix of layer s, P ijs is the associated zero matrix, γ s is the resolution parameter, ω isk is the connection value from node i in layer s to node i in layer k, c ir with c is is to set the parameters, δ() is the Dirac function.

[0057] (6.2) The time-frequency multilayer brain network is input into the GCN, and the input node signal value is the degree of freedom of each node; the input adjacency matrix is ​​the decomposed super-adjacency matrix of the time-frequency multilayer brain network. In particular, the processing method of the time-frequency multilayer brain network adjacency matrix is ​​as follows: Since the adjacency matrix of the multilayer brain network is a three-dimensional tensor, it is difficult to calculate due to its high dimension. In order to reduce the amount of calculation, inspired by the Cartesian product of the graph, the time-frequency domain multilayer brain network is decomposed into two-directional sub-networks, and the original three-dimensional adjacency matrix is ​​decomposed into two orthogonal two-dimensional super-adjacency matrices as the adjacency matrix input of the GCN, and finally feature fusion is performed.

[0058] (6.3) The GCN structure consists of two GCN layers and one feature classification layer. To fully utilize the information of the time-frequency multi-layer brain network, the two GCN layers are divided into shallow and deep layers. All images are output once after passing through the shallow GCN and once again after passing through the deep GCN. Finally, the artificial, shallow, and deep features are fused to obtain the final classification result.

[0059] The beneficial effects of the present invention are:

[0060] (1) The present invention uses a time-frequency multilayer brain network method to construct a functional connectivity model of the brain. Compared with traditional methods and general brain networks, this method describes the functional connectivity of the brain during natural hand movements from a more comprehensive perspective and explains the mechanism of brain work from a new perspective.

[0061] (2) This paper emphasizes the importance of interlayer connections in multi-layer brain networks, and proposes a new interlayer connection calculation method using the wSAR model and hybrid cross-frequency coupling, which improves the interpretability of interlayer connections and increases the information validity of multi-layer brain networks, providing a new method for improving the decoding of natural hand movements.

[0062] (3) The present invention standardizes the time-frequency multi-layer brain network, which can intuitively show the connection mode of the brain functional network and is conducive to revealing the neural operation mechanism of the brain.

[0063] (4) The present invention reduces the dimensionality of the adjacency matrix of the time-frequency multi-layer brain network, reduces the amount of calculation, and uses the improved GCN to simultaneously consider the artificial, shallow, and deep features of the multi-layer brain network. While more comprehensively utilizing the multi-layer brain network information, the accuracy of natural hand movement decoding is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is the main flow chart of the EEG recognition method for natural hand movements based on a time-frequency multi-layer brain network of the present invention.

[0065] Figure 2This is a flowchart of constructing a time-domain multi-layer brain network in the EEG recognition method of natural hand movements based on a time-frequency multi-layer brain network of the present invention.

[0066] Figure 3 This is a standardized diagram of the time-frequency multi-layer brain network of the natural hand movement EEG recognition method based on the time-frequency multi-layer brain network of the present invention.

[0067] Figure 4 This is the GCN structure diagram of the natural hand movement EEG recognition method based on the time-frequency multi-layer brain network of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0069] like Figure 1 As shown, the present invention proposes a natural hand movement EEG recognition method based on a time-frequency multi-layer brain network, which specifically includes the following steps:

[0070] (1) Collect multi-channel EEG signals of natural hand movements;

[0071] (2) Preprocess the multi-channel EEG signals and use a sliding time window to extract the EEG signals at each time point, and extract the delta wave, theta wave, alpha wave, beta wave, and gamma wave at each time point.

[0072] Specifically, step (2) includes:

[0073] (2.1) The EEG signal was low-pass filtered at 100 Hz, outliers were removed, and linear interpolation was used to supplement the data.

[0074] (2.2) Use the ICA blind source separation algorithm to remove other bioelectric components, and then perform segmentation, baseline correction and public average reference.

[0075] (2.3) The processed EEG signal was filtered using a 4th-order Butterworth IIR filter at 0.3–3 Hz (delta wave), 4–8 Hz (theta wave), 8–13 Hz (alpha wave), 13–30 Hz (beta wave), and 30–45 Hz (gamma wave). The EEG signal at each frequency was then sampled using a time window of 1 s and a step size of 100 ms to obtain the signal value at the corresponding time point.

[0076] (3) The time-point data are processed, and a time-domain multilayer brain network is constructed, using electrodes as nodes, directed transfer functions (DTFs) as intra-layer connections, and weighted quadratic autoregressive (wSAR) model parameters as inter-layer connections. Specifically, the time-domain inter-layer connections are calculated as follows: Constructing the wSAR model: Based on the assumption that closer temporal distances have a greater impact, weighted terms are added to the AR model, and the constant terms are processed. The improved wSAR model parameters are calculated as inter-layer connections.

[0077] Specifically, if Figure 2 As shown, step (3) includes:

[0078] (3.1) Select the EEG signal of a certain channel in a certain frequency band and construct a wSAR model for the EEG signal:

[0079]

[0080] in represents the EEG signal value of the jth channel at time i, represents the model parameter value of channel j, ω j represents the constant term, which we assume here as the initial value of the influence of each channel, ε represents the noise, the model order is m, and k is the weighting value. In order to avoid the k value being too large, which will cause the subsequent time term to lose its meaning, we set k to arrive At the same time, in order to consider the optimal order, the initial value of m is set to 1 here.

[0081] (3.2) The wSAR model formula is rewritten as follows:

[0082]

[0083] Using the least squares method, let ε 2 to a j The partial derivative of is equal to 0, which minimizes the residual sum of squares.

[0084] remember

[0085]

[0086] Where N is the total number of time points, and the least squares estimate of the parameter matrix A is

[0087]

[0088] where X j is the signal matrix of the past time in the wSAR model, Y j is the signal matrix of the future time in the wSAR model.

[0089] (3.3) The EEG signals of all channels in this frequency band are calculated by (3.2) to obtain all the parameter matrices A j According to the actual situation, when there is no action intention and action execution, each channel should have the same initial value for the action response, so it is necessary to align the initial values ​​of each channel. This method takes each parameter matrix A j The constant term ω in j The mean of ω avg ,Right now

[0090]

[0091] Where M represents the total number of channels, ω j is the constant term of each channel parameter matrix. Then ω avg Substitute back the original AR model formula of all channels and repeat step (3.2) to obtain the final parameter matrix B j .

[0092] (3.4) Take the order m from 1 to 10, repeat the steps (3.2) and (3.3), and calculate the autocovariance function σ1, σ2, ..., σ of each order sample. 10 According to the final prediction error (FPE) criterion, calculate the order that makes FPE reach the minimum, and select the parameter matrix B of this order j .

[0093] (3.5) Multiply both sides of the AR model equation for all channels by a time lag term And perform z-transform to obtain the transfer function H of the system ij (f) represents the transfer relationship from channel i to channel j. Then normalize it to get the final directional transfer function (DTF)

[0094]

[0095] Among them H im (f) represents the transfer function from channel i to channel m.

[0096] (3.6) All the signals of all frequency bands are calculated through steps (3.2)-(3.5), and finally a time-domain multi-layer brain network is constructed: the electrodes are used as nodes; a threshold of 60% is taken to remove false connections, and the final obtained As an intra-layer connection, that is, the connection from node i to node j in the same layer is Take a 60% threshold to remove false connections and transform the parameter matrix B j As an inter-layer connection, the connection from node i in layer k to node i in layer 1 in frequency band j is

[0097] (4) Process the frequency band data and calculate the cross-frequency inter-layer coupling. The calculation method of the frequency domain inter-layer connection is as follows: perform Hilbert transform on the EEG signal of each frequency, extract the amplitude and phase information, and calculate the cross-frequency coupling (CFC) between each frequency. Here, the average value of the phase-amplitude coupling (PAC), phase-phase coupling (PPC) and amplitude-amplitude coupling (AAC) indicators is selected as the final inter-layer connection.

[0098] Specifically, step (4) includes:

[0099] (4.1) Perform Hilbert transform on the EEG signals of each frequency at all time points to obtain the amplitude and phase values ​​at each time point.

[0100] (4.2) Calculate the phase-amplitude coupling (PAC), phase-phase coupling (PPC), and amplitude-amplitude coupling (AAC) between any two frequencies:

[0101]

[0102] Where n is the total number of time points in the EEG data, represents the amplitude of the high-frequency band H at time i, represents the phase of the low-frequency band L at time i.

[0103]

[0104] in and They represent the phase values ​​of frequency bands a and b at time i respectively. The imaginary part and sgn function are taken to reduce the influence of volume conduction effect and common source.

[0105] AAC=Corr(A i , A j )

[0106] Among them, Corr() represents the Pearson correlation coefficient, A i With A j Represent the amplitudes of frequency bands i and j respectively.

[0107] (4.3) Calculate the average value of PAC, PPC, and AAC as the cross-frequency coupling value CFC, take a threshold of 60%, remove false connections, and use the CFC value as the inter-layer coupling between corresponding frequencies to form the frequency domain part of the multi-layer brain network.

[0108] (5) The time-frequency multilayer brain network constructed by (3) and (4) is combined into a 3D standard form: M = (V, E, ω, 1), where V represents the point set, E represents the edge set, ω represents the size of the edge, and l represents the position mark of the point. This standard form abstracts the brain area into the z-axis, time as the x-axis, and frequency as the y-axis, that is, compressing the single-layer network into one dimension. As the number of layers increases, the dimensionality of the multilayer brain network increases.

[0109] Specifically, if Figure 3 As shown, step (5) specifically includes:

[0110] The obtained multi-layer brain network in the time-frequency domain is written into a standardized form: M = (V, E, ω, 1), with the brain area as the z-axis, time as the x-axis, and frequency as the y-axis, as shown in the attached figure. Figure 3 As shown. Where V is a point set, including the point labels, such as ("v1", ..., "v N ”); E is an edge set, including the label of the edge, such as (“e 12 ”,……,”e N(N-1) ”); ω is the connection value of the edge, such as (ω(v i , v i ), ...); 1 is the location label of point V, which is used to indicate the location of the point, so L is a two-dimensional matrix, such as (("v1", 1, 1, 1), ..., ("v 25 ”, 2, 5, 6), …), where since the present invention has three dimensions, the three numbers after the point number are the coordinates of the point. The reason for standardizing the multi-layer brain network is firstly to facilitate visualization and retrieval, and secondly to facilitate the unified processing of subsequent metric calculations and GCN.

[0111] (6) Calculate the standard form of multi-layer brain network metrics in (5), including common multi-layer degree, eigenvector centrality, and temporal flexibility unique to multi-layer brain networks, for neural analysis of natural action decoding. Finally, calculate the adjacency matrix and input this standard form into a graph convolutional network (GCN) for classification and recognition.

[0112] Specifically, if Figure 4 As shown, step (6) specifically includes:

[0113] (6.1) Calculating the metrics of time-frequency multilayer brain networks: multilayer degree d M (i), eigenvector centrality The node flexibility F is used for neurophysiological analysis. At the same time, the manually extracted features are input into the last layer of GCN for feature weighted fusion.

[0114]

[0115]

[0116]

[0117]

[0118] where d M (i) is v i The total node degree of , α(i, j, k) is the element in the adjacency tensor of the time-frequency multilayer brain network; is the adjacency tensor of the multi-layer brain network, is the eigenvector of the tensor corresponding to the eigenvalue λ; f i is the number of changes of a node community at each time point, N is the total number of changes, and F i For node v j The node flexibility; Q is a modular function used for community detection, where A ijs is the adjacency matrix of layer s, P ijs is the associated zero matrix, γ s is the resolution parameter, ω isk is the connection value from node i in layer s to node i in layer k, c ir with c is is to set the parameters, δ() is the Dirac function.

[0119] (6.2) The time-frequency multilayer brain network is input into the GCN, and the input node signal value is the degree of freedom of each node; the input adjacency matrix is ​​the decomposed super-adjacency matrix of the time-frequency multilayer brain network. In particular, the processing method of the time-frequency multilayer brain network adjacency matrix is ​​as follows: Since the adjacency matrix of the multilayer brain network is a three-dimensional tensor, it is difficult to calculate due to its high dimension. In order to reduce the amount of calculation, inspired by the Cartesian product of the graph, the time-frequency domain multilayer brain network is decomposed into two-directional sub-networks, and the original three-dimensional adjacency matrix is ​​decomposed into two orthogonal two-dimensional super-adjacency matrices as the adjacency matrix input of the GCN, and finally feature fusion is performed.

[0120] (6.3) The GCN structure consists of two GCN layers and one feature classification layer. To fully utilize the information of the time-frequency multi-layer brain network, the two GCN layers are divided into shallow and deep layers. All images are output once after passing through the shallow GCN and once again after passing through the deep GCN. Finally, the artificial, shallow, and deep features are fused to obtain the final classification result.

[0121] It should be noted that the above content merely illustrates the technical idea of ​​the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.

Claims

1. A method for EEG recognition of natural hand movements based on a time-frequency multi-layer brain network, characterized by: The following steps are involved: (1) Collect multi-channel EEG signals of natural hand movements; (2) Preprocess the multi-channel EEG signals and use a sliding time window to extract the EEG signals at each time point, and extract the delta wave, theta wave, alpha wave, beta wave, and gamma wave at each time point; (3) The data at each time point are processed, and a time-domain multilayer brain network is constructed, with electrodes as nodes, DTF as intra-layer connections, and wSAR model parameters as inter-layer connections. The calculation method for time-domain inter-layer connections is as follows: Based on the assumption that the closer the time distance, the greater the impact, a weighted term is added to the traditional AR model, and the constant term is processed at the same time to establish the wSAR model, and the wSAR model parameters are calculated as inter-layer connections; (4) Process the data of each frequency band and calculate the cross-frequency inter-layer coupling. The calculation method of frequency domain inter-layer connection is as follows: perform Hilbert transform on the EEG signal of each frequency, extract the amplitude and phase information, calculate the cross-frequency coupling CFC between each two frequencies, and select the average value of phase-amplitude coupling PAC, phase-phase coupling PPC and amplitude-amplitude coupling AAC as the final inter-layer connection; (5) The time-frequency multilayer brain network constructed by (3) and (4) is combined into a three-dimensional standard form: M = (V, E, ω, l), where V represents the point set, E represents the edge set, ω represents the size of the edge, and 1 represents the position mark of the point; this standard form abstracts the brain area into the z-axis, time as the x-axis, and frequency as the y-axis, that is, compressing the single-layer network into one dimension. As the dimension of the number of layers increases, the dimension of the multilayer brain network increases; (6) Calculate the standard form of multi-layer brain network metrics in (5), including the common multi-layer degree, eigenvector centrality, and temporal flexibility unique to multi-layer brain networks, for neural analysis of natural action decoding; finally, calculate the adjacency matrix and input this standard form into GCN for classification and recognition.

2. The method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network according to claim 1, characterized in that: Step (2) includes: (2.1) The EEG signal was low-pass filtered at 100 Hz, outliers were removed, and data were supplemented using linear interpolation; (2.2) Use ICA blind source separation algorithm to remove other bioelectric components, then perform segmentation, baseline correction and public average reference; (2.3) The processed EEG signal was filtered using a 4th-order Butterworth IIR filter at 0.3–3 Hz, 4–8 Hz, 8–13 Hz, 13–30 Hz, and 30–45 Hz. The EEG signal at each frequency was then sampled using a time window of 1 s and a step size of 100 ms to obtain the signal value at the corresponding time point.

3. The method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network according to claim 1, characterized in that: Step (3) includes: (3.1) Select the EEG signal of a certain channel in a certain frequency band and construct a wSAR model for the EEG signal: in represents the EEG signal value of the jth channel at time i, represents the model parameter value of channel j, ω j Represents the constant term, which is assumed to be the initial value of the influence of each channel, ε represents noise, the model order is m, and k is the weighting value; in order to avoid the k value being too large, which will cause the subsequent time term to lose its meaning, k is set to arrive The average value of ; At the same time, in order to consider the optimal order, the initial value of m is set to 1 here; (3.2) The wSAR model formula is rewritten as follows: Using the least squares method, let ε 2 to a j The partial derivative is equal to 0, which minimizes the residual sum of squares; In order to make the formula expression simpler, the wSAR model is recorded in matrix form below. Where N is the total number of time points, the parameter matrix A j The least squares estimate of (3.3) The EEG signals of all channels in this frequency band are calculated by (3.2) to obtain all the parameter matrices A j ; Take each parameter matrix A j The constant term ω in j The mean of ω avg ,Right now Where M represents the total number of channels, ω j is the constant term of each channel parameter matrix; then ω avg Substitute back the original AR model formula of all channels and repeat step (3.2) to obtain the final parameter matrix B j ; (3.4) Take the order m from 1 to 10, repeat the steps (3.2) and (3.3), and calculate the autocovariance function σ1, σ2, ..., σ of each order sample. 10 According to the final prediction error FPE criterion, calculate the order that makes FPE reach the minimum, and select the parameter matrix B of this order j ; (3.5) Multiply both sides of the AR model equation for all channels by a time lag term And perform z-transform to obtain the transfer function H of the system kj (f) represents the transfer relationship from channel k to channel j, which is then normalized to obtain the final directional transfer function Among them H km (f) represents the transfer function from channel k to channel m, (3.6) All the signals of all frequency bands are calculated through steps (3.2)-(3.5), and finally a time-domain multi-layer brain network is constructed: the electrodes are used as nodes; a threshold of 60% is taken to remove false connections, and the final obtained As an intra-layer connection, that is, the connection from channel k to channel j in the same layer is Take a 60% threshold to remove false connections and transform the parameter matrix B j As an inter-layer connection, the connection from node i in layer o to node i in layer l is b m+1-|o-l| .

4. The method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network according to claim 1, characterized in that: Step (4) includes: (4.1) Perform Hilbert transform on the EEG signals of each selected frequency at all time points to obtain the amplitude and phase values ​​at each time point; (4.2) Calculate the phase-amplitude coupling PAC, phase-phase coupling PPC, and amplitude-amplitude coupling AAC between any two frequencies respectively: Where n is the total number of time points in the EEG data, represents the amplitude of the high-frequency band H at time i, represents the phase of the low-frequency band L at time i; in and Represent the phase values ​​of frequency bands a and b at time i respectively. The imaginary part and sgn function are taken to reduce the influence of volume conduction effect and common source. AAC=Corr(A α ,A β ) Among them, Corr() represents the Pearson correlation coefficient, A α With A β Represent the amplitudes of frequency bands α and β respectively, (4.3) Calculate the average value of PAC, PPC, and AAC as the cross-frequency coupling value CFC, take a threshold of 60%, remove false connections, and use the CFC value as the inter-layer coupling between corresponding frequencies to form the frequency domain part of the multi-layer brain network.

5. The method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network according to claim 1, characterized in that: Step (5) specifically includes: The obtained time-frequency domain multi-layer brain network is written in a standardized form: M = (V, E, ω, l), where the brain area is the z-axis, time is the x-axis, and frequency is the y-axis; V is the point set containing the point labels, E is the edge set containing the edge labels, ω is the connection value of the edge, and 1 is the position label of point V, which is used to indicate the location of the point. Therefore, l is a two-dimensional matrix with three dimensions, and the three numbers after the point label are the coordinates of the point.

6. The method for recognizing natural hand movements using EEG based on a time-frequency multilayer brain network according to claim 1, characterized in that: Step (6) specifically includes: (6.1) Calculating the metrics of time-frequency multilayer brain networks: multilayer degree d M (i), eigenvector centrality Node flexibility F is used for neurophysiological analysis. At the same time, the manually extracted features are input into the last layer of GCN for feature weighted fusion. where d M (i) is v i The total node degree of , a(i, j, o) is the element in the adjacency tensor of the time-frequency multilayer brain network; is the adjacency tensor of the multi-layer brain network, is the eigenvector of the tensor corresponding to the eigenvalue λ; f i is the number of changes of a node community at each time point, N is the total number of changes, and F j For node v j The node flexibility; Q is a modular function used for community detection, where A ijs is the adjacency matrix of layer s, P ijs is the associated zero matrix, γ s is the resolution parameter, ω iso is the connection value from node i in layer s to node i in layer o, c ir with c is is to set the parameters, δ() is the Dirac function; (6.2) The time-frequency multilayer brain network is input into GCN, and the input node signal value is the degree of freedom of each node; the input adjacency matrix is ​​the decomposed super-adjacency matrix of the time-frequency multilayer brain network; the processing method of the time-frequency multilayer brain network adjacency matrix is ​​as follows: Since the adjacency matrix of the multilayer brain network is a three-dimensional tensor, it has a high dimension and is difficult to calculate; in order to reduce the amount of calculation, inspired by the Cartesian product of the graph, the time-frequency domain multilayer brain network is decomposed into two-directional sub-networks, and the original three-dimensional adjacency matrix is ​​decomposed into two orthogonal two-dimensional super-adjacency matrices as the adjacency matrix input of GCN, and finally feature fusion is performed; (6.3) The structure of GCN consists of two GCN layers and one feature classification layer. In order to make full use of the information of the time-frequency multi-layer brain network, the two GCN layers are divided into shallow and deep layers. All images output features once after passing through the shallow GCN and output features again after passing through the deep GCN. Finally, the artificial, shallow and deep features are fused to obtain the final classification result.

Images