Auditory Spatial Attention Decoding Method Based on Complex EEG Signal Transformation

Abstract

The application discloses an auditory spatial attention decoding method and device based on multi-channel electroencephalogram signal complex network conversion, and the method comprises the following steps: converting each channel electroencephalogram time sequence into a visible graph by using a visible graph complex network construction method; obtaining the network topological structure similarity of the visible graph corresponding to each channel electroencephalogram time sequence by using a similarity calculation method based on an adjacency matrix; generating the network graph feature representation of the multi-channel electroencephalogram signal according to the network topological structure similarity calculation result of the visible graph; and taking the network graph feature representation of the multi-channel electroencephalogram time sequence as the input of a machine learning model, so as to realize the auditory spatial attention decoding based on the multi-channel electroencephalogram signal complex network conversion. The application can realize high-precision auditory spatial attention decoding in a traditional machine learning model.

Description

Technical Field

[0001] This invention belongs to the field of complex network graph signal processing of electroencephalogram (EEG) signals, and specifically relates to an auditory spatial attention decoding method and device based on the transformation of complex network graphs of multi-channel EEG signals. Background Technology

[0002] Electroencephalogram (EEG) signals are electrical signals generated by neurons in the brain during their activity. By recording the firing activity of neurons in the brain, EEG signals can be used to study the patterns of neuronal firing. Using the time-series information in EEG signals, complex EEG networks can be constructed. The network topology and characteristic states reflect the generation and propagation of EEG signals. Using complex network theory and pattern recognition techniques, decoded information about brain activity can be extracted. Analyzing complex EEG networks can provide a deeper understanding and explanation of cognitive states in the brain, greatly promoting research and analysis of the fundamental mechanisms of brain function and neurological diseases. Therefore, EEG time-series and complex network methods can complement and be cross-applied to jointly explore and deeply analyze brain function and behavior.

[0003] The brain is an extremely complex biological system, and describing its dynamic behavior from EEG signal time series becomes challenging given this complexity. Traditional time series analysis methods also struggle to handle such highly complex nonlinear systems. Over the past decade, complex networks have received significant attention in EEG signal analysis for characterizing the brain's complex systems. Recently, several novel complex network construction methods have been proposed to map univariate / multivariate EEG time series onto complex networks. Existing research indicates that these methods hold great potential for applications such as characterizing important properties of complex dynamic systems.

[0004] Existing auditory spatial attention decoding typically employs one of two approaches. Approach 1: Directly utilizing multi-channel EEG signals as input to a deep neural network model, and employing deep learning methods to learn representations from the EEG signals to achieve auditory attention decoding. Approach 2: Utilizing the frequency domain characteristics of EEG signals, such as alpha band signals, and deep learning models, to construct an auditory attention decoding framework.

[0005] Electroencephalogram (EEG) signals are neural electrical activity responses formed by highly nonlinear coupling of a large number of nerve cells, exhibiting deterministic chaotic characteristics. The interactions between neurons in EEG activity exhibit nonlinear behavior and display complex, irregular dynamic patterns in time and space. Existing auditory spatial attention decoding primarily utilizes EEG signals or various specific sub-band signals in conjunction with representation learning to design various decoding frameworks. This decoding framework does not fully consider the spatial domain and nonlinear dynamics of EEG signals, and therefore cannot achieve higher-precision auditory attention decoding. Summary of the Invention

[0006] According to a first aspect of the present disclosure, an auditory spatial attention decoding method based on complex network transformation of multi-channel electroencephalogram (EEG) signals is provided, the method comprising the following steps:

[0007] Using a visible graph complex network construction method, the EEG time series of each channel is converted into a visible graph;

[0008] By using a similarity calculation method based on adjacency matrix, the network topology similarity of the visible graphs corresponding to the EEG time series of each channel is obtained;

[0009] Based on the network topology similarity calculation results of different visible graphs, a network graph feature representation of multi-channel EEG signals is generated.

[0010] By using the network graph feature representation of multichannel EEG time series as input to a machine learning model, auditory spatial attention decoding based on multichannel EEG signal network graphs is achieved.

[0011] In one embodiment, the method for constructing a visible graph complex network to convert a multi-channel EEG time series into a visible graph specifically includes:

[0012] Input a single-channel EEG time series s of length n i S = {s1, s2, s3…s} n}, s i This represents the data at the i-th point in the EEG time series;

[0013] Based on the time index i of the EEG data, the set of nodes V of the visible graph G is directly defined, and V is denoted as {i|i=1,2,...n};

[0014] Based on the EEG time series data values ​​corresponding to arbitrary time indices i and j (j > i), determine whether the nodes in the visible graph G satisfy the visibility relationship to determine the edges of the visible graph. Specifically:

[0015] If time indices i and j are adjacent, then nodes i and j in graph G are connected by an edge.

[0016] For any time index k, the EEG time series data values ​​satisfy the following relationship: If i < k < j (i ≤ 1, j ≤ n) and j > i + 1, then it can be seen that there is an edge connecting nodes i and j in graph G.

[0017] Based on the edge calculation results of the visible graph G, the visible graph transformation of the single-channel EEG time series is realized, G = (V, E).

[0018] In one embodiment, the multichannel EEG time series is converted into a visible graph by using a proximity method or ordinal-to-network method instead of the visible graph complex network construction method.

[0019] In one embodiment, the method of obtaining the network topology similarity of different visible graphs using a similarity calculation method based on the adjacency matrix specifically includes:

[0020] Let adj_M_1 and adj_M_2 represent the adjacency matrices of any two visible graphs. Both adj_M_1 and adj_M_2 are N*N two-dimensional matrices composed of elements 0 or 1. When a matrix element is 1, it means that there is an edge between the corresponding node pairs.

[0021] For the adjacency matrices adj_M_1 and adj_M_2, traverse the matrix row index a from 1 to N and the matrix column index b from 1 to N, and determine whether each element adj_M_1(a,b) and adj_M_2(a,b) of the adjacency matrix is ​​1.

[0022] After traversing all matrix rows a and matrix columns b, count the number of elements in the same position of adjacency matrices adj_M_1 and adj_M_2 that are all 1, denoted as SD. SD represents the similarity of the topological structures of two visible graph networks.

[0023] In one embodiment, generating a network graph feature representation of a multi-channel EEG signal based on the network topology similarity calculation results of different visible graphs specifically includes:

[0024] For C-channel EEG signals, the visible graph complex network construction method is used to convert them into C visible graphs;

[0025] Using the network topology similarity calculation method of visible graphs, the similarity SD(e,v) of the network topology of any two visible graphs is obtained. The channel indices e and v range from 1 to C. SD(e,v) represents the similarity of the network topology of the visible graphs of EEG channels e and v.

[0026] By forming a similarity matrix from all SD(e,v), the network graph SD feature representation of the C-channel EEG signal is realized.

[0027] In one embodiment, the machine learning model is any one of the following: Naive Bayes model, neural network model, support vector machine, and random forest model.

[0028] According to a second aspect of the present disclosure, an auditory spatial attention decoding device based on multi-channel EEG signal complex network conversion is provided, the device comprising:

[0029] The visible graph acquisition module is used to convert the EEG time series of each channel into visible graphs using the visible graph complex network construction method;

[0030] The similarity acquisition module is used to obtain the network topology similarity of the visible graphs corresponding to the EEG time series of each channel using a similarity calculation method based on the adjacency matrix.

[0031] The feature representation acquisition module is used to generate network graph feature representations of multi-channel EEG signals based on the network topology similarity calculation results of different visible graphs.

[0032] The auditory spatial attention decoding module is used to take the network graph feature representation of multi-channel EEG time series as input to the machine learning model to realize auditory spatial attention decoding based on multi-channel EEG signal network graph.

[0033] According to a third aspect of the present disclosure, an auditory spatial attention decoding device based on multi-channel EEG signal complex network conversion is provided. The device includes: an EEG signal acquisition module, an EEG signal set creation module, and a processor. The EEG signal acquisition module is used for EEG data acquisition. The EEG signal set creation module filters and preprocesses the EEG data acquired by the EEG signal acquisition module, then segments and slices it to generate EEG signal samples, which are then transmitted to the processor. The processor calls stored program instructions to cause the device to execute the above-described auditory spatial attention decoding method steps based on multi-channel EEG signal complex network conversion.

[0034] According to a fourth aspect of the present disclosure, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the above-described steps of the auditory spatial attention decoding method based on complex network conversion of multichannel electroencephalogram signals.

[0035] This invention provides an auditory spatial attention decoding method and device based on complex network conversion of multi-channel EEG signals. From the perspective of complex network analysis of multi-channel EEG signals, it uses a time-series complex network construction method to convert the time series of EEG signals into a visible graph. Using the network topology similarity calculation of the visible graph, the multi-channel EEG signals are converted into a feature representation of a network graph, thereby realizing a more accurate auditory attention decoding method that can be achieved in traditional machine learning models.

[0036] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0038] Figure 1 This is a schematic flowchart of the auditory spatial attention decoding method based on the complex network conversion of multi-channel EEG signals in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the process of converting multi-channel EEG time series into visible graphs using the visible graph complex network construction method in this embodiment of the invention;

[0040] Figure 3 This is a diagram illustrating the network graph feature representation of multi-channel EEG signals in an embodiment of the present invention.

[0041] Figure 4 This is a structural diagram of the auditory spatial attention decoding device based on the complex network conversion of multi-channel EEG signals in an embodiment of the present invention;

[0042] Figure 5 This is a structural diagram of the auditory spatial attention decoding device based on the complex network conversion of multi-channel EEG signals in an embodiment of the present invention;

[0043] Figure 6 This is an architectural diagram of an auditory spatial attention decoding device based on the transformation of complex multi-channel EEG signals in an embodiment of the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the drawings, not the entire structure.

[0045] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0046] This invention provides an auditory spatial attention decoding method and apparatus based on complex network conversion of multi-channel electroencephalogram (EEG) signals, and offers the following embodiments:

[0047] Based on Embodiment 1 of the present invention

[0048] This embodiment illustrates an auditory spatial attention decoding method based on complex network transformation of multi-channel EEG signals. See [link to previous document]. Figure 1 The method includes the following steps:

[0049] S1. Using the visible graph complex network construction method, the EEG time series of each channel is converted into a visible graph;

[0050] S2. Using a similarity calculation method based on adjacency matrix, obtain the network topology similarity of the visible graphs corresponding to the EEG time series of each channel;

[0051] S3. Based on the network topology similarity calculation results of different visible graphs, generate network graph feature representations of multi-channel EEG signals;

[0052] S4. Using the network graph feature representation of multi-channel EEG time series as input to the machine learning model, auditory spatial attention decoding based on multi-channel EEG signal network graph is realized.

[0053] In S1, methods including but not limited to using the nearest neighbor method or ordinal-to-network method to replace the visible graph complex network construction method are used to convert multi-channel EEG time series into visible graphs. Visible graphs are a simple and effective method for transforming time series data into complex networks, providing a new perspective for studying and extracting information from time series data using graph theory and network analysis methods. The algorithm flow for constructing visible graphs of EEG time series is as follows: Figure 2 As shown, this visible graph construction method can transform a single-channel EEG time series into a graph form, where the data points of the EEG time series correspond to the nodes in the visible graph, and the visibility relationships between the data points correspond to the edges in the visible graph.

[0054] See Figure 2 Specifically, in the EEG time series {s i In the corresponding visible graph, adjacent data points satisfy the visibility relationship, and there is always an edge connecting the nodes in the visible graph. For two non-adjacent data points X and Y, their time indices and magnitudes are i, j, and s, respectively. i s j It can be seen that when constructing the graph, it is necessary to determine whether there are other data points K (time index and magnitude are represented as k and s). k ), such that (k, s k ) in (i, s i ) and (j, s j Above the connecting line, if there are no such data points, it means they satisfy a visibility relationship, and there is an edge connecting nodes i and j in the corresponding visible graph. More specifically:

[0055] Input a single-channel EEG time series s of length n iS = {s1, s2, s3…s} n}, s i This represents the data at the i-th point in the EEG time series;

[0056] Based on the time index i of the EEG data, the set of nodes V of the visible graph G is directly defined, and V is denoted as {i|i=1,2,...n};

[0057] Based on the EEG time series data values ​​corresponding to arbitrary time indices i and j (j > i), determine whether the nodes in the visible graph G satisfy the visibility relationship to determine the edges of the visible graph. Specifically:

[0058] If time indices i and j are adjacent, then nodes i and j in graph G are connected by an edge.

[0059] For any time index k, the EEG time series data values ​​satisfy the following relationship: If i < k < j (i ≤ 1, j ≤ n) and j > i + 1, then it can be seen that there is an edge connecting nodes i and j in graph G.

[0060] Based on the edge calculation results of the visible graph G, the visible graph transformation of the single-channel EEG time series is realized, G = (V, E).

[0061] Using visible graph construction methods, EEG signals from each channel can be converted into visible graphs. To measure the similarity of the network topology of any two visible graphs, S2 employs a similarity calculation method based on adjacency matrices. For any two visible graphs, adj_M_1 and adj_M_2 represent their adjacency matrices. Assuming the adjacency matrix size is N×N, both adj_M_1 and adj_M_2 are two-dimensional matrices composed of elements 0 or 1, with N×N elements. When an element in the adjacency matrix takes a value of 1, it indicates that there is an edge between the corresponding node pairs. In adjacency matrices adj_M_1 and adj_M_2, for any position (c, d) where c and d range from 1 to N, if the corresponding element at that position is 1, it means that the visible graphs corresponding to the two adjacency matrices have edges at the same node positions. Therefore, these two visible graphs (or adjacency matrices) can be said to have one similar edge. By calculating the total number of similar connecting edges at all the same node positions in two visible graphs and denoting it as SD, the SD value can directly reflect the similarity of the network topology of the two visible graphs.

[0062] Specifically, the main process of the visible graph network topology similarity calculation method in S2 is shown in Table 1: Input any two visible graphs, represented by adj_M_1 and adj_M_2 adjacency matrices (size N×N); adjacency matrices adj_M_1 and adj_M_2 are two-dimensional matrices composed of elements 0 or 1. When a matrix element is 1, it indicates that there is an edge between the corresponding node pairs; for adjacency matrices adj_M_1 and adj_M_2, the matrix row index a is traversed from 1 to N and the matrix column index b is traversed from 1 to N: it is determined whether each element adj_M_1(a, b) and adj_M_2(a, b) of the adjacency matrix is ​​1; after traversing all matrix rows a and matrix columns b, the number of elements in the adjacency matrices adj_M_1 and adj_M_2 that are all 1 in the same position is counted, denoted as SD. The SD parameter directly measures the similarity of the network topology of the two visible graphs.

[0063] Table 1 shows the method for calculating the similarity of network topology in the graph.

[0064]

[0065]

[0066] The network graph feature representation of multi-channel EEG signals generated based on the network topology similarity calculation results of different visible graphs, as described in S3, specifically includes:

[0067] For C-channel EEG signals, the visible graph complex network construction method is used to convert them into C visible graphs;

[0068] Using the network topology similarity calculation method of visible graphs, the similarity SD(e,v) of the network topology of any two visible graphs is obtained. The channel indices e and v range from 1 to C. SD(e,v) represents the similarity of the network topology of the visible graphs of EEG channels e and v.

[0069] By forming a similarity matrix from all SD(e,v), the network graph SD feature representation of the C-channel EEG signal is realized.

[0070] The specific implementation process is as follows:

[0071] (1) Input multi-channel EEG signals, where C represents the number of EEG channels and N represents the length of EEG data in each channel; then, use the visible graph method to directly convert each channel EEG signal into a visible graph, and calculate the visible graph of C EEG signals.

[0072] (2) Obtain the corresponding adjacency matrix adj_M[h] from the visible graph of the C channel EEG signal, where h represents the visible graph after the h-th channel EEG signal, and there are C such graphs, each with a size of N×N;

[0073] (3) For channel index h, iterate from 1 to C: obtain the visible graph adjacency matrix adj_M[h] of the current EEG channel h;

[0074] (4) For channel index m, iterate from 1 to C: obtain the visible graph adjacency matrix adj_M[m] of the current EEG channel m;

[0075] (5) For the adjacency matrices adj_M[h] and adj_M[m], perform similarity calculation of the visible graph network topology. The similarity value is denoted as SD(h,m). SD(h,m) represents the similarity of the visible graph network topology of the EEG signals corresponding to EEG channels h and m.

[0076] (6) Repeat steps 3) and 5) to traverse all EEG channels h and m, and calculate the similarity value of the visible graph network topology of all any two channels of EEG signals; and for the channel index h = m, directly define SD(h,m) = 0;

[0077] (7) Based on the similarity calculation results of the visible graph network topology of any two channels, a similarity matrix is ​​constructed to realize the network graph feature representation of multi-channel EEG signals.

[0078] The machine learning model mentioned in S4 can be any one of the following: Naive Bayes model, neural network model, support vector machine, and random forest model.

[0079] Specifically, in S3, multi-channel EEG signals are represented as a network graph based on the similarity of visible graph network topology. This multi-channel EEG signal network graph feature is used as the input feature vector for the EEG pattern classifier, further constructing a novel auditory spatial attention decoding scheme with excellent decoding performance. The EEG pattern classifier can employ various common machine learning models, such as neural network models, support vector machines, and random forest models.

[0080] Based on Embodiment 2 of the present invention

[0081] Based on Embodiment 1, this invention designs and implements an auditory spatial attention decoding method using an auditory selective spatial attention EEG dataset as an example, specifically including:

[0082] 1) Auditory selective spatial attention EEG data collection

[0083] When listening to continuous audio, the brain's auditory system exhibits certain EEG patterns in the information processing corresponding to left-ear and right-ear hearing. This experiment recruited 16 participants to participate in a binaural hearing experiment. During the experiment, different auditory audio was played to both ears simultaneously. Participants were instructed to selectively focus on the information heard by one ear (following ear) while ignoring the information heard by the other ear (non-following ear). EEG data on auditory attention was collected from the participants' scalp using an EEG device. The experiment involved two different auditory activities: left-ear hearing (L) and right-ear hearing (R). Each participant performed 20 trials. The EEG signal duration was 6 minutes for the first 8 trials and 2 minutes for the subsequent 12 trials. The sampling rate was 128 Hz, and the EEG data was collected from 64 channels.

[0084] 2) Creation of auditory attention EEG dataset

[0085] Based on the collected EEG data, each participant had 72 minutes of auditory attention EEG data collected. After filtering and preprocessing, the collected EEG data was segmented into 1-second units to generate EEG signal samples. Each participant's 72 minutes of data generated 4320 samples, with left-ear and right-ear auditory spatial attention EEG samples each accounting for 50%. Each EEG sample consisted of 64 channels, with each channel containing 128 data points (i.e., 1 second, sampling rate 128). The auditory attention EEG dataset was labeled using L and R to represent left-ear and right-ear auditory spatial attention, respectively.

[0086] 3) Feature extraction from multichannel EEG signal network graph

[0087] To create a high-quality EEG auditory attention dataset and perform auditory spatial attention decoding based on multi-channel EEG signal network graphs, feature extraction of 64-channel EEG samples is required. First, using the visible graph method, visible graphs are constructed from the 64-channel EEG signal samples (128 pixels in length), resulting in 64 visible graphs of size 128×128. Then, a similarity algorithm for the visible graph network topology is used to calculate the similarity of the visible graph network topology of any two EEG channels. A 64×64 similarity matrix is ​​formed based on all calculated similarities, realizing the network graph feature representation of the 64-channel EEG signals. To demonstrate the EEG pattern features of left-ear and right-ear auditory spatial attention, a network graph feature of an EEG signal is randomly selected for display, such as... Figure 3 As shown.

[0088] The calculated 64×64 similarity matrix was used as the input feature of the pattern classifier. Random forest, support vector machine, and Naive Bayes model were used as auditory spatial attention decoding models to conduct auditory selective spatial attention EEG pattern recognition. For each participant, the auditory attention EEG dataset contained 4230 samples. Simple cross-validation was used to divide the entire dataset into a training set (80%) and a test set (20%). Then, the extracted multi-channel EEG signal network graph features were used as the input feature vectors for machine learning to perform EEG decoding. Table 2 shows the experimental results of auditory spatial attention EEG decoding using the three machine learning models: random forest, support vector machine, and Naive Bayes model.

[0089] Table 2. Results of auditory spatial attention decoding based on multichannel EEG signal network maps

[0090]

[0091] Based on Embodiment 3 of the present invention

[0092] Example 3 illustrates an auditory spatial attention decoding device 400 based on complex multi-channel EEG signal conversion, such as... Figure 4 As shown, the device includes:

[0093] The visible graph acquisition module 401 is used to convert the EEG time series of each channel into a visible graph using the visible graph complex network construction method;

[0094] The similarity acquisition module 402 is used to obtain the network topology similarity of the visible graphs corresponding to the EEG time series of each channel using a similarity calculation method based on the adjacency matrix.

[0095] The feature representation acquisition module 403 is used to generate network graph feature representations of multi-channel EEG signals based on the network topology similarity calculation results of the visible graph;

[0096] The auditory spatial attention decoding module 404 is used to take the network graph feature representation of the multi-channel EEG time series as the input of the machine learning model to realize auditory spatial attention decoding based on the multi-channel EEG signal network graph.

[0097] In addition to the above module, the auditory spatial attention decoding device 400 based on the complex network conversion of multi-channel EEG signals may also include other components. However, since these components are not related to the content of the embodiments of this disclosure, their illustrations and descriptions are omitted here.

[0098] The other specific working processes of the auditory spatial attention decoding device 400 based on the transformation of complex multi-channel EEG signals refer to the descriptions of Embodiments 1-2 of the auditory spatial attention decoding method based on the transformation of complex multi-channel EEG signals, and will not be repeated here.

[0099] Based on Embodiment 4 of the present invention

[0100] This embodiment is used to illustrate that the device of the present invention can also be used with the aid of... Figure 5 The device architecture shown is used to implement this. Figure 5 An auditory spatial attention decoding device 500 based on complex network transformation of multi-channel EEG signals is illustrated. The device includes: an EEG signal acquisition module 501, an EEG signal set creation module 502, and a processor 503. The EEG signal acquisition module 501 is used for EEG data acquisition. The EEG signal set creation module 502 filters and preprocesses the EEG data acquired by the EEG signal acquisition module 501, then segments it to generate EEG signal samples, which are then transmitted to the processor 503. The processor 503 calls stored program instructions to cause the device to execute the steps of the auditory spatial attention decoding method based on complex network transformation of multi-channel EEG signals. The steps of the method include:

[0101] Using a visible graph complex network construction method, the EEG time series of each channel is converted into a visible graph;

[0102] By using a similarity calculation method based on adjacency matrix, the network topology similarity of the visible graphs corresponding to the EEG time series of each channel is obtained;

[0103] Based on the network topology similarity calculation results of the visible graph, a network graph feature representation of the multi-channel EEG signal is generated.

[0104] By using the network graph feature representation of multichannel EEG time series as input to a machine learning model, auditory spatial attention decoding based on multichannel EEG signal network graphs is achieved.

[0105] The other specific working processes of the auditory spatial attention decoding device 500 based on the transformation of complex networks of multi-channel EEG signals are described in the above-described embodiments 1-3 of the auditory spatial attention decoding method based on the transformation of complex networks of multi-channel EEG signals, and will not be repeated here.

[0106] Based on Embodiment 5 of the present invention

[0107] This embodiment is used to illustrate that the device of the present invention can also be used with the aid of... Figure 6 The architecture of the computing device shown is used to implement this. Figure 6 The architecture of the computing device is shown. For example... Figure 6As shown, the computer system 610 includes a system bus 630, one or more CPUs 640, input / output 620, and memory 650. The memory 650 can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU, including the relay forwarding method based on the sensor-integrated UAV in the embodiment. Figure 6 The architecture shown is merely exemplary and should be adjusted according to actual needs when implementing different devices. Figure 6 One or more components in the system. The memory 650, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the auditory spatial attention decoding method based on multi-channel EEG signal complex network conversion in this embodiment of the invention (e.g., the visible image acquisition module 401, similarity acquisition module 402, feature representation acquisition module 403, and auditory spatial attention decoding module 404 in the auditory spatial attention decoding device 400 based on multi-channel EEG signal complex network conversion). One or more CPUs 640 execute various functional applications and data processing of the system of the present invention by running the software programs, instructions, and modules stored in the memory 650, that is, implementing the above-mentioned auditory spatial attention decoding method based on multi-channel EEG signal complex network conversion, which includes:

[0108] Using a visible graph complex network construction method, the EEG time series of each channel is converted into a visible graph;

[0109] By using a similarity calculation method based on adjacency matrix, the network topology similarity of the visible graphs corresponding to the EEG time series of each channel is obtained;

[0110] Based on the network topology similarity calculation results of the visible graph, a network graph feature representation of the multi-channel EEG signal is generated.

[0111] By using the network graph feature representation of multichannel EEG time series as input to a machine learning model, auditory spatial attention decoding based on multichannel EEG signal network graphs is achieved.

[0112] Of course, the processor of the server provided in the embodiments of the present invention is not limited to performing the method operations described above, but can also perform related operations in the auditory spatial attention decoding method based on multi-channel EEG signal complex network conversion provided in any embodiment of the present invention.

[0113] The memory 650 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 650 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 650 may further include memory remotely configured relative to one or more CPUs 640, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0114] Input / output 620 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Input / output 620 may also include a display device such as a display screen.

[0115] As can be seen from the embodiments, the auditory spatial attention decoding method and device based on complex network conversion of multi-channel EEG signals provided by the present invention, from the perspective of complex network analysis of multi-channel EEG signals, uses a time series complex network construction method to convert the time series of EEG signals into a visible graph, and uses the network topology similarity calculation of the visible graph to convert the multi-channel EEG signals into a feature representation of a network graph, thereby realizing a more accurate auditory attention decoding method in traditional machine learning models.

[0116] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0117] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An auditory spatial attention decoding method based on multi-channel electroencephalogram signal complex network conversion, characterized in that, The method includes the following steps: Auditory attention EEG data were collected using an EEG instrument, and multi-channel EEG time series were extracted from the data; Using a visible graph complex network construction method, the EEG time series of each channel is converted into a visible graph; By using a similarity calculation method based on adjacency matrix, the network topology similarity of the visible graphs corresponding to the EEG time series of each channel is obtained; Based on the network topology similarity calculation results of the visible graph, a network graph feature representation of the multi-channel EEG signal is generated. By using the network graph feature representation of multichannel EEG time series as input to a machine learning model, auditory spatial attention decoding based on multichannel EEG signal network graphs is achieved. The method for constructing a visible graph complex network specifically includes: An input single-channel electroencephalogram time series s of length n i , S = {s1, s2, s3...sn} n}, s i represents the i-th point data in the electroencephalogram time series Based on the time index i of the EEG data, the set of nodes V of the visible graph G is directly defined, and V is denoted as {i|i=1,2,...n}; Based on the EEG time series data values ​​corresponding to arbitrary time indices i and j (j>i), determine whether the nodes in the visible graph G satisfy the visibility relationship to determine the edges of the visible graph, specifically: If time indices i and j are adjacent, then nodes i and j in graph G are connected by an edge. For any time index k, the electroencephalogram time series data values satisfy the relationship i < k < j (i ≤ 1, j ≤ n), j > i + 1, then it can be seen that there is an edge between nodes i and j of the graph G; Based on the edge calculation results of the visible graph G, the visible graph transformation of a single-channel EEG time series is realized, G = (V, E); the similarity calculation method based on the adjacency matrix is ​​used to obtain the network topology similarity of different visible graphs, specifically including: Let adj_M_1 and adj_M_2 represent the adjacency matrices of any two visible graphs. adj_M_1 and adj_M_2 are N*N two-dimensional matrices composed of elements 0 or 1. When a matrix element is 1, it means that there is an edge between the corresponding node pairs. For the adjacency matrices adj_M_1 and adj_M_2, traverse the matrix row index a from 1 to N and the matrix column index b from 1 to N, and determine whether each element adj_M_1(a,b) and adj_M_2(a,b) of the adjacency matrix is ​​1. After traversing all matrix rows a and matrix columns b, count the number of elements in the same position of adjacency matrices adj_M_1 and adj_M_2 that are all 1, denoted as SD. SD represents the similarity of the topological structures of two visible graph networks.

2. The method of claim 1, wherein, The multichannel EEG time series can be converted into a visible graph by using the nearest neighbor method or ordinal transfer network method instead of the aforementioned visible graph complex network construction method.

3. The method of claim 1, wherein, The generation of network graph feature representations of multi-channel EEG signals based on the network topology similarity calculation results of the visible graph specifically includes: For C-channel EEG signals, the visible graph complex network construction method is used to convert them into C visible graphs; Using the network topology similarity calculation method of visible graphs, the similarity SD(e,v) of the network topology of any two visible graphs is obtained. The channel indices e and v range from 1 to C. SD(e,v) represents the similarity of the network topology of the visible graphs of EEG channels e and v. By forming a similarity matrix from all SD(e,v), the network graph SD feature representation of the C-channel EEG signal is realized.

4. The method of claim 1, wherein, The machine learning model is any one of a linear classifier, K-nearest neighbor, naive Bayes model, support vector machine, decision tree, ensemble model, random forest model, neural network model, and convolutional neural network model.

5. An auditory spatial attention decoding device based on multi-channel electroencephalogram signal complex network conversion, characterized in that, The device comprises: A visible graph acquisition module is configured to collect auditory attention electroencephalogram data by using an electroencephalograph, extract multi-channel electroencephalogram time series from the data, and convert the electroencephalogram time series of each channel into a visible graph by using a visible graph complex network construction method. A similarity acquisition module is configured to acquire network topological structure similarity of the visible graph corresponding to the electroencephalogram time series of each channel by using an adjacency matrix-based similarity calculation method. A feature representation acquisition module is configured to generate network graph feature representation of the multi-channel electroencephalogram signal according to the network topological structure similarity calculation result of the visible graph. An auditory spatial attention decoding module is configured to take the network graph feature representation of the multi-channel electroencephalogram time series as input of a machine learning model, and realize auditory spatial attention decoding based on the network graph of the multi-channel electroencephalogram signal. The visible graph complex network construction method comprises the following steps: An input single-channel electroencephalogram time series s of length n i , S = {s1, s2, s3...sn} n}, s i represents the i-th point data in the electroencephalogram time series A node set V of the visible graph G is directly defined according to a time index i of the electroencephalogram data, and the node set V is denoted as {i|i=1, 2,...n}; Whether the nodes of the visible graph satisfy a visible relationship is determined according to the electroencephalogram time series data values corresponding to any time indexes i and j (j>i), so as to determine the edges of the visible graph, and the specific process is as follows: If the time indexes i and j are adjacent, the nodes i and j of the visible graph G have an edge. For any time index k, the electroencephalogram time series data values satisfy the relationship i < k < j (i ≤ 1, j ≤ n), j > i + 1, then it can be seen that there is an edge between nodes i and j of the graph G; The network topological structure similarity of different visible graphs is acquired by using the adjacency matrix-based similarity calculation method, and the specific process is as follows: adj_M_1 and adj_M_2 represent the adjacency matrices of any two visible graphs, and adj_M_1 and adj_M_2 are N*N two-dimensional matrices composed of elements 0 or 1, and when the matrix element is 1, it indicates that there is an edge between the corresponding node pair. The elements adj_M_1(a,b) and adj_M_2(a,b) of the adjacency matrices adj_M_1 and adj_M_2 are judged whether they are both 1, with the matrix row index a from 1 to N and the matrix column index b from 1 to N. After all the matrix rows a and the matrix columns b are traversed, the number of elements of the adjacency matrices adj_M_1 and adj_M_2 that are both 1 at the same position is counted, and the number is denoted as SD, which represents the similarity of the network topological structures of the two visible graphs.

6. An auditory spatial attention decoding device based on multi-channel electroencephalogram signal complex network conversion, characterized in that, The device comprises an electroencephalogram signal acquisition module, an electroencephalogram signal set manufacturing module, and a processor. The electroencephalogram signal acquisition module is configured to collect electroencephalogram data. The electroencephalogram signal set manufacturing module is configured to perform filtering preprocessing on the electroencephalogram data collected by the electroencephalogram signal acquisition module, perform segmentation and slicing, generate electroencephalogram signal samples, and transmit the electroencephalogram signal samples to the processor. The processor is configured to call stored program instructions to enable the device to perform the method steps of any one of claims 1 to 4. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor, when executing the computer program, implements the method steps of any one of claims 1 to 4.

Images