A visual imaginary character recognition method and system based on EEG signal analysis

Abstract

The present application discloses a method and system for visual imagination character recognition based on EEG signal analysis, which relates to the technical field of EEG signal analysis. The method includes: first obtaining an EEG signal to be recognized; using a channel selection method based on maximum mutual information to screen the channels of the EEG signal to be recognized, and obtaining the EEG signal to be recognized after channel screening; inputting the EEG signal to be recognized after channel screening into a visual imagination character recognition model to obtain a visual imagination character recognition result. The present application makes up for the current problem of fuzzy channel positioning of brain-computer interfaces for visual imagination tasks by using a channel selection algorithm based on maximum mutual information discrimination. In addition, by using a visual imagination character recognition model based on a parallel space-time network framework, the temporal variation law of EEG signals in visual imagination tasks, the key frequency band characteristics, and the rich spatial correlation information contained in multi-channel EEG signals are fully utilized, thereby improving the accuracy of character reconstruction based on EEG signal recognition.

Description

Technical Field

[0001] The present application relates to the technical field of electroencephalogram (EEG) signal analysis, and in particular to a method and system for visually imagining character recognition based on EEG signal analysis. Background Art

[0002] Vision is one of the various important senses of the human body. According to research, 87% of people's external information depends on their eyes, and 75%-90% of human activities are dominated by vision. It can be seen that vision plays an important role in human activities. Visual imagery (VI) refers to the ability of people to construct certain graphics or abstract concepts and sensory appearances in their brains according to their needs when thinking. The human brain is like having eyes. These visual imaginations can move, rotate, change and be analyzed. With visual imagination, people's brain waves will change differently. Therefore, in the prior art, the decoding of visual imagination tasks is mostly based on the collected electroencephalogram (EEG) signals.

[0003] For example, Koji et al. collected EEG signals of visual imagination of 6 Japanese words, extracted power spectral density (PSD) as features and classified them with support vector machine (SVM), with an average classification accuracy of 84.6%. Rami Alarai et al. proposed a VI task decoding method based on EEG. This method uses Choi-Williams time-frequency distribution to analyze EEG signals and constructs a multi-class SVM classifier to decode various VI tasks. Sunghan Lee et al. collected EEG signals of 7 subjects performing VI on three image categories (objects, numbers, and shapes), each category including 3 tasks, and used event-related spectral perturbation (ERSP) and Choi-Williams time-frequency distribution to extract EEG features, and used the features in a variety of neural network classifiers, among which the MultiRocket network achieved a classification accuracy of 70.38% in 3 categories containing 9 tasks.

[0004] The above-mentioned prior art only analyzes and processes the EEG signal based on VI, and does not clearly indicate the channels related to the VI task, resulting in the problem of fuzzy channel positioning; and when processing the EEG signal based on VI, the prior art does not make sufficient use of the timing characteristics, frequency characteristics and spatial correlation information in the multi-channel EEG signal, which has a certain impact on the accurate recognition of the EEG signal. Therefore, how to overcome the problems of the prior art to improve the accuracy of character recognition and reconstruction based on EEG signals is crucial to the development of this field. Summary of the invention

[0005] The purpose of this application is to provide a method and system for visual imagination character recognition based on EEG signal analysis, which can improve the accuracy of character recognition and reconstruction based on EEG signal analysis.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for visual imaginary character recognition based on EEG signal analysis, comprising the following steps:

[0008] Obtain an electroencephalogram signal to be identified; the electroencephalogram signal to be identified is an electroencephalogram signal having several channels.

[0009] The channel selection method based on maximum mutual information is used to screen the channels of the EEG signal to be identified, and the EEG signal to be identified after channel screening is obtained.

[0010] The EEG signal to be recognized after channel screening is input into the visual imagination character recognition model to obtain the visual imagination character recognition result; the visual imagination character recognition model is a model based on the parallel space-time neural network framework, and the parallel space-time neural network framework includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the temporal features of the characters from the input signal; the multi-band nested residual dense network is used to learn the spatial features of the characters from the input signal; the parallel feature fusion network is used to fuse the temporal features of the characters and the spatial features of the characters to obtain the character fusion features; the character reconstruction network is used to reconstruct the visual imagination character recognition result according to the character fusion features.

[0011] Optionally, the visual imaginary character recognition result is specifically a category of the visual imaginary character; using a channel selection method based on maximum mutual information to screen the channels of the electroencephalogram signal to be recognized, and obtaining the electroencephalogram signal to be recognized after channel screening, specifically includes the following steps:

[0012] For any category of visual imaginary characters, a number of experimental EEG signals are extracted; the experimental EEG signals are EEG signals with a number of channels.

[0013] The z-score standardization processing algorithm is used to standardize the test EEG signals to obtain standardized test EEG signals.

[0014] Based on the standardized test EEG signals, the intra-class similarity and inter-class similarity are calculated; the intra-class similarity is the maximum mutual information mean between the standardized test EEG signals of the same category, and the inter-class similarity is the maximum mutual information mean between the standardized test EEG signals of different categories.

[0015] According to the intra-class similarity and inter-class similarity, the discriminant score of each channel is calculated.

[0016] According to the discrimination scores of each channel, the channels of the EEG signals to be identified are screened to obtain the EEG signals to be identified after channel screening.

[0017] Optionally, the test EEG signal is normalized according to the following formula:

[0018]

[0019] in, is the normalized EEG signal of the hth trial, is the mean value of the h-th trial EEG signal of each category, x h is the hth trial EEG signal, σ xh is the standard deviation of the hth trial EEG signal of each category, and H is the number of trial EEG signals of each category.

[0020] Optionally, the intra-class similarity is calculated according to the following formula:

[0021]

[0022] Among them, D w is the intra-class similarity, mean() is the mean function, MIC() is the maximum mutual information function, is the standardized EEG signal of the tth trial, c h and c t They represent the categories of the h-th trial EEG signal and the t-th trial EEG signal respectively.

[0023] The inter-class similarity is calculated according to the following formula:

[0024]

[0025] The discriminant score of the channel is calculated according to the following formula:

[0026] D=μD w -(1-μ)D i .

[0027] Among them, D is the discriminant score of the channel and μ is the weighted hyperparameter.

[0028] Optionally, the maximum mutual information is calculated according to the following formula:

[0029]

[0030] Where S is a two-dimensional finite set, x and y are the coordinates of the two-dimensional finite set S in the X and Y directions, respectively, B(|S|) is the upper limit of the grid partition x×y, p(x,y) is the joint probability density, and p(x) and p(y) are the edge probability densities.

[0031] Optionally, the recurrent neural network is a bidirectional long short-term memory network, including a forward long short-term memory network and a backward long short-term memory network; the forward long short-term memory network is used to extract a forward hidden state from an input signal; the backward long short-term memory network is used to extract a backward hidden state from an input signal; and the character timing feature is obtained by concatenating the forward hidden state and the backward hidden state.

[0032] Optionally, the multi-band nested residual dense network includes a filter group, a nested residual dense network and a channel space attention mechanism; the filter group is used to perform multi-band filtering on the input signal to obtain multi-band data; the nested residual dense network is used to learn character space features from the multi-band data; the channel space attention mechanism is used to enhance the feature representation of the nested residual dense network.

[0033] Optionally, the nested residual dense network specifically includes a first residual dense block, a second residual dense block, a third residual dense block, a normalization layer, a ReLU activation function layer and an average pooling layer connected in sequence; the output of the first residual dense block is fused with the output of the second residual dense block and used as the input of the third residual dense block; the output of the first residual dense block, the output of the second residual dense block and the output of the third residual dense block are fused as the input of the normalization layer.

[0034] The channel space attention mechanism, the first convolutional attention block and the second convolutional attention block include: the first convolutional attention block is connected between the second residual dense block and the third residual dense block, and the second convolutional attention block is connected between the third residual dense block and the normalization layer.

[0035] Optionally, a parallel feature fusion network is used to assign different weights to different channels according to the learned inter-channel correlations to automatically adjust the importance of each channel feature map, so that the model can more effectively capture key features, reduce redundant information, and improve generalization capabilities.

[0036] In the second aspect, the present application provides a visual imaginary character recognition system based on EEG signal analysis, comprising the following modules:

[0037] The EEG signal acquisition module to be identified is used to acquire the EEG signal to be identified; the EEG signal to be identified is an EEG signal with several channels.

[0038] The EEG signal channel screening module is used to screen the channels of the EEG signals to be identified by using the channel selection method based on maximum mutual information, and obtain the EEG signals to be identified after channel screening.

[0039] The visual imagination character recognition module is used to input the EEG signal to be recognized after channel screening into the visual imagination character recognition model to obtain the visual imagination character recognition result. The visual imagination character recognition model is a model based on the parallel spatiotemporal neural network framework, which includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the temporal features of the characters from the input signal; the multi-band nested residual dense network is used to learn the spatial features of the characters from the input signal; the parallel feature fusion network is used to fuse the temporal features of the characters and the spatial features of the characters to obtain the fusion features of the characters; the character reconstruction network is used to reconstruct the visual imagination character recognition result based on the fusion features of the characters.

[0040] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0041] The present application provides a method and system for visual imagination character recognition based on EEG signal analysis, the method comprising: first obtaining the EEG signal to be recognized; using a channel selection method based on maximum mutual information to screen the channels of the EEG signal to be recognized, and obtaining the EEG signal to be recognized after channel screening; inputting the EEG signal to be recognized after channel screening into a visual imagination character recognition model to obtain a visual imagination character recognition result. The present application makes up for the current problem of fuzzy channel positioning of brain-computer interfaces for visual imagination tasks through a channel selection algorithm based on maximum mutual information discrimination, and clarifies the importance of each EEG signal channel to the reconstruction of visual imagination characters; in addition, by using a visual imagination character recognition model based on a parallel space-time network framework, full use is made of the temporal variation law of EEG signals in visual imagination tasks, the key frequency band characteristics, and the rich spatial correlation information contained in multi-channel EEG signals, thereby improving the accuracy of character recognition and reconstruction based on EEG signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flowchart of a method for visual imaginative character recognition based on EEG signal analysis is provided for one embodiment of the present application.

[0044] Figure 2A flowchart of step S2 in a method for visual imaginary character recognition based on EEG signal analysis provided in one embodiment of the present application.

[0045] Figure 3 A schematic diagram of the structure of a recurrent neural network in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0046] Figure 4 A schematic diagram of the structure of a nested residual dense network in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0047] Figure 5 A schematic diagram of the structure of a residual dense block in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0048] Figure 6 A schematic diagram of the structure of a convolutional attention block in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0049] Figure 7 A schematic diagram of the working principle of a channel attention mechanism in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0050] Figure 8 A schematic diagram of the working principle of the spatial attention mechanism in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0051] Fig. 9 A schematic diagram of the structure of a parallel feature fusion network in a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0052] Fig.10 A schematic diagram of the technical concept of a visual imagination character recognition method based on EEG signal analysis provided in one embodiment of the present application.

[0053] Fig.11 A schematic diagram of the functional modules of a visual imagination character recognition system based on EEG signal analysis provided in one embodiment of the present application.

[0054] Fig.12 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0057] In an exemplary embodiment, Figure 1 As shown, a visual imagination character recognition method based on EEG signal analysis is provided, comprising the following steps:

[0058] S1. Obtain an EEG signal to be identified; the EEG signal to be identified is an EEG signal having several channels.

[0059] S2. Filter the channels of the EEG signals to be identified by using a channel selection method based on maximum mutual information to obtain the EEG signals to be identified after channel screening.

[0060] Maximum Information Coefficient (MIC) is a useful information measure in information theory. It is mainly calculated by mutual information and grid partitioning method. It is a method to detect nonlinear correlation between variables. Its value range is 0-1. The higher the MIC value, the stronger the correlation. For a two-dimensional finite set S(X,Y) divided into x and y segments in the X and Y directions respectively, its maximum mutual information MIC(S) is defined as:

[0061]

[0062] Where S is a two-dimensional finite set, x and y are the coordinates of the two-dimensional finite set S in the X and Y directions, respectively, B(|S|) is the upper limit of the grid partition x×y, p(x,y) is the joint probability density, and p(x) and p(y) are the edge probability densities.

[0063] The visual imaginary character recognition result is specifically the category of the visual imaginary character; in this embodiment, Figure 2 In the flowchart shown, step S2 specifically includes the following steps:

[0064] S21. For any category of visual imaginary characters, extract a number of test EEG signals; the test EEG signals are EEG signals having a number of channels.

[0065] S22, using the z-score standardization processing algorithm to standardize the test EEG signal to obtain a standardized test EEG signal. The test EEG signal is standardized according to the following formula:

[0066]

[0067] in, is the normalized EEG signal of the hth trial, is the mean value of the h-th trial EEG signal of each category, x h is the hth trial EEG signal, σ xh is the standard deviation of the hth trial EEG signal of each category, and H is the number of trial EEG signals of each category.

[0068] S23. Based on the standardized test EEG signals, calculate the intra-class similarity and inter-class similarity; the intra-class similarity is the maximum mutual information mean between the standardized test EEG signals of the same category, and the inter-class similarity is the maximum mutual information mean between the standardized test EEG signals of different categories. In this embodiment, the intra-class similarity is calculated according to the following formula:

[0069]

[0070] Among them, D w is the intra-class similarity, mean() is the mean function, MIC() is the maximum mutual information function, is the standardized EEG signal of the tth trial, c h and c t They represent the categories of the h-th trial EEG signal and the t-th trial EEG signal respectively.

[0071] The inter-class similarity is calculated according to the following formula:

[0072]

[0073] S24. Calculate the discrimination score of each channel based on the intra-class similarity and the inter-class similarity. Calculate the discrimination score of the channel according to the following formula:

[0074] D=μD w -(1-μ)D i .

[0075] Among them, D is the discriminant score of the channel and μ is the weighted hyperparameter.

[0076] S25. According to the discrimination scores of each channel, the channels of the EEG signals to be identified are screened to obtain the EEG signals to be identified after channel screening. The discrimination score D of each channel is calculated using the above formula, and the selected channels are sorted in descending order according to the discrimination score D. At the same time, the channel threshold L (L = 1, ..., N C ) to select the first L channels for subsequent EEG signal processing.

[0077] S3. Input the EEG signal to be recognized after channel screening into the visual imagination character recognition model to obtain the visual imagination character recognition result. The visual imagination character recognition model is a model based on the parallel spatiotemporal neural network framework, which includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the temporal features of the characters from the input signal; the multi-band nested residual dense network is used to learn the spatial features of the characters from the input signal; the parallel feature fusion network is used to fuse the temporal features of the characters and the spatial features of the characters to obtain the fusion features of the characters; the character reconstruction network is used to reconstruct the visual imagination character recognition result based on the fusion features of the characters.

[0078] In an example solution, a bidirectional long short-term memory network (BiLSTM) combining forward LSTM and backward LSTM is used to construct a recurrent neural network when learning the character timing features of EEG signals. The network effectively integrates the past and future information in the EEG data by introducing a bidirectional structure, so that the network can more comprehensively capture the timing features in the EEG signal and provide more accurate character recognition results. The recurrent neural network structure proposed in this embodiment is as follows: Figure 3 As shown in FIG, the model inputs the EEG signal X of each character into a single-layer BiLSTM network, and uses the BiLSTM network to learn the information in the EEG signal to obtain the character timing features, which are concatenated from the forward and backward hidden states.

[0079] In this embodiment, the recurrent neural network is a bidirectional long short-term memory network, including a forward long short-term memory network and a backward long short-term memory network; the forward long short-term memory network is used to extract a forward hidden state from an input signal; the backward long short-term memory network is used to extract a backward hidden state from an input signal; and the character timing feature is obtained by concatenating the forward hidden state and the backward hidden state.

[0080] The filter bank performs multi-band filtering on the EEG signal, and encapsulates the EEG signal of each character from two-dimensional data C×T into three-dimensional data F×C×T, aiming to better extract feature information in different frequency ranges according to the frequency band characteristics of the EEG signal, where C represents the number of channels, T represents the sampling points, and F represents the number of frequency bands. The output of the filter bank is used as the input of the nested residual dense network. Specifically, the multi-band nested residual dense network includes a filter bank, a nested residual dense network, and a channel space attention mechanism; the filter bank is used to perform multi-band filtering on the input signal to obtain multi-band data; the nested residual dense network is used to learn character space features from multi-band data; and the channel space attention mechanism is used to enhance the feature representation of the nested residual dense network.

[0081] To obtain data for each frequency band of the EEG signal, a filter bank is constructed using a Chebyshev II filter to perform spectral filtering on the X of each character to generate multi-band data. The filter bank is constructed from 8 non-overlapping frequency bands ranging from 2 to 32 Hz, namely 2-4 Hz, 4-8 Hz, 8-12 Hz, 12-16 Hz, 16-20 Hz, 20-24 Hz, 24-28 Hz and 28-32 Hz.

[0082] Furthermore, the nested residual dense network is used to learn spatial information from multi-band data to further explore and utilize the valuable information hidden in the multi-band data, and embed the channel space attention mechanism in it to enhance the feature representation of the residual dense network. By nesting the residual dense network to form a multi-level network structure to more fully learn the multi-band data, the multi-scale and multi-level feature representation of the characters is extracted to further improve the performance and generalization ability of the network.

[0083] In this embodiment, the nested residual dense network specifically includes a first residual dense block, a second residual dense block, a third residual dense block, a normalization layer, a ReLU activation function layer and an average pooling layer connected in sequence; the output of the first residual dense block is fused with the output of the second residual dense block and used as the input of the third residual dense block; the output of the first residual dense block, the output of the second residual dense block and the output of the third residual dense block are fused as the input of the normalization layer.

[0084] like Figure 4 The network consists of three residual dense blocks RDB with the same structure (labeled as RDB1, RDB2 and RDB3), forming a multi-level structure to fully learn multi-band data and extract multi-scale and multi-level feature representations of characters. A normalization layer (BN), a ReLU layer and an average pooling layer (AvgPool) are added after RDB3 to improve network performance and generalization ability. The structure of the residual dense block RDB is shown in Figure 5As shown in the figure, it consists of a convolutional layer, a normalization layer, and a ReLU. The convolutional layer captures character features of different scales through different convolution kernels and retains the spatial connection of the original input data. The normalization layer normalizes the convolutional data to improve data stability. As a nonlinear activation function, ReLU enhances the network's expressive power.

[0085] Since the nested residual dense network layer retains the feature information of each level, however, too much feature information is often accompanied by the generation of redundant information, which may not only affect the network's ability to represent complex data, but also increase the computational complexity. Therefore, this paper introduces the Convolutional Block Attention Module (CBAM) in the nested residual dense network layer to help the network adaptively focus on important character features and reduce the transmission of redundant information. A CBAM is added after RDB2 and RDB3 respectively. In this embodiment, the channel space attention mechanism, the first convolutional attention block and the second convolutional attention block include; the first convolutional attention block is connected between the second residual dense block and the third residual dense block, and the second convolutional attention block is connected between the third residual dense block and the normalization layer.

[0086] Convolutional attention block CBAM is an attention mechanism block that combines the channel attention mechanism and the spatial attention mechanism. Compared with the traditional attention mechanism, CBAM connects the channel attention mechanism and the spatial attention mechanism in series for feature learning. The structure of CBAM is as follows: Figure 6 As shown, the working principles of the channel attention mechanism and the spatial attention mechanism are as follows: Figure 7-Figure 8 As shown. This embodiment adopts a parallel connection method to apply the character features obtained through RDB to the channel attention mechanism and the spatial attention mechanism at the same time to obtain refined character features. The maxout and avgout in the channel attention mechanism are the results obtained by respectively applying the global maximum pooling and global average pooling to the original input features and connecting to the same fully connected layer. The maxout and avgout in the spatial attention mechanism are the results obtained by taking the maximum value and average value of each feature point in the original input features on the channel.

[0087] Furthermore, the parallel feature fusion network adopts the "Squeeze-and-Excitation" (SE) attention mechanism, which is mainly used to fuse the features of the recurrent neural network and the multi-band nested residual dense network, and remove redundant features through information complementarity between features to obtain character fusion features with strong discriminativeness. In this embodiment, the parallel feature fusion network is used to assign different weights to different channels according to the learned inter-channel correlation to automatically adjust the importance of each channel feature map, so that the model can more effectively capture key features, reduce redundant information, and improve generalization ability. The parallel feature fusion network structure is as follows: Fig. 9 As shown, C, H, and W represent the number of channels, width, and height of the input features, respectively.

[0088] The character reconstruction module mainly maps the character features obtained in the parallel feature fusion module into labels corresponding to the characters.

[0089] 1) Character feature transformation. With high latitude characteristics, this paper calculates The logarithmic variance of the character variance feature is obtained This ensures that Characterization capability based on the reduction The dimension of is increased and the Gaussianity of the variance feature is enhanced by logarithmic transformation.

[0090] 2) Feature flattening. The feature map in is transformed into a one-dimensional vector And use it as the input of the fully connected layer.

[0091] 3) Weight normalization. The weights of the fully connected layer are regularized using the maximum norm constraint of 0.5, and the LogSoftmax activation function is added after the fully connected layer to convert the output of the network into a logarithmic probability distribution, thereby obtaining the label corresponding to the character. The LogSoftmax activation function is a variant of the Softmax activation function, which has better stability in data processing. For the input vector z, the logarithmic probability LogSoftmax(z i )The calculation formula of the activation function is as follows:

[0092]

[0093] Among them, z i is the i-th element in the input vector, e is the base of the natural constant, N c Represents all EEG signal categories.

[0094] The visual imaginary character recognition method based on EEG signal analysis provided in this embodiment has the following overall technical concept: Fig.10 As shown in the figure, a channel selection algorithm based on maximum mutual information discrimination is proposed to make up for the current problem of fuzzy channel positioning for VI brain-computer interface. Based on the maximum mutual information, the algorithm calculates the discrimination score of each channel by considering the intra-class similarity and inter-class similarity to quantify the degree of association between the channel and the VI task. At the same time, the threshold is combined to obtain the channel, and the importance of each channel to the reconstruction of the VI task is further clarified by characterizing the evaluation index curve. In addition, a character reconstruction network based on a parallel spatiotemporal network framework is proposed to make full use of the temporal variation law of visual imagination EEG signals, the key frequency band characteristics and the rich spatial correlation information contained in multi-channel EEG signals. The character reconstruction network uses the preprocessed EEG signal as input data, and extracts the temporal features and multi-level spatial features of the EEG signal in specific frequency bands through a recurrent neural network and a multi-band nested residual dense network, respectively, and uses the SE attention mechanism for feature fusion to further realize character reconstruction.

[0095] Based on the same inventive concept, the embodiment of the present application also provides a visual imagination character recognition system based on EEG signal analysis for implementing the visual imagination character recognition method based on EEG signal analysis involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the visual imagination character recognition system based on EEG signal analysis provided below can refer to the limitations of the visual imagination character recognition method based on EEG signal analysis above, and will not be repeated here.

[0096] In an exemplary embodiment, Fig.11 As shown, a visual imagination character recognition system based on EEG signal analysis is provided, comprising the following modules:

[0097] The EEG signal acquisition module to be identified is used to acquire the EEG signal to be identified; the EEG signal to be identified is an EEG signal with several channels.

[0098] The EEG signal channel screening module is used to screen the channels of the EEG signals to be identified by using the channel selection method based on maximum mutual information, and obtain the EEG signals to be identified after channel screening.

[0099] The visual imagination character recognition module is used to input the EEG signal to be recognized after channel screening into the visual imagination character recognition model to obtain the visual imagination character recognition result. The visual imagination character recognition model is a model based on the parallel spatiotemporal neural network framework, which includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the temporal features of the characters from the input signal; the multi-band nested residual dense network is used to learn the spatial features of the characters from the input signal; the parallel feature fusion network is used to fuse the temporal features of the characters and the spatial features of the characters to obtain the fusion features of the characters; the character reconstruction network is used to reconstruct the visual imagination character recognition result based on the fusion features of the characters.

[0100] Some modules may also have subunits to implement their functions. Fig.11 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different functions. Fig.11 One or at least two components of the system shown.

[0101] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a visual imagination character recognition method based on EEG signal analysis is implemented.

[0102] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0104] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0106] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0107] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0108] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A visual imaginary character recognition method based on EEG signal analysis, characterized in that: include: Obtaining the EEG signal to be identified; The electroencephalogram signal to be identified is an electroencephalogram signal having several channels; Using a channel selection method based on maximum mutual information, the channels of the to-be-identified electroencephalogram signal are screened to obtain the to-be-identified electroencephalogram signal after channel screening; The EEG signal to be recognized after the channel screening is input into the visual imagination character recognition model to obtain the visual imagination character recognition result; the visual imagination character recognition model is a model based on a parallel spatiotemporal neural network framework, and the parallel spatiotemporal neural network framework includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the character temporal features from the input signal; the multi-band nested residual dense network is used to learn the character spatial features from the input signal; the parallel feature fusion network is used to fuse the character temporal features and the character spatial features to obtain the character fusion features; the character reconstruction network is used to reconstruct the visual imagination character recognition result according to the character fusion features; The multi-band nested residual dense network includes a filter bank, a nested residual dense network and a channel space attention mechanism; the filter bank is used to perform multi-band filtering on the input signal to obtain multi-band data; the nested residual dense network is used to learn character space features from the multi-band data; The channel space attention mechanism is used to enhance the feature representation of the nested residual dense network; The nested residual dense network specifically includes a first residual dense block, a second residual dense block, a third residual dense block, a normalization layer, a ReLU activation function layer and an average pooling layer connected in sequence; the output of the first residual dense block is fused with the output of the second residual dense block and used as the input of the third residual dense block; the output of the first residual dense block, the output of the second residual dense block and the output of the third residual dense block are fused as the input of the normalization layer; The channel space attention mechanism includes a first convolutional attention block and a second convolutional attention block; the first convolutional attention block is connected between the second residual dense block and the third residual dense block, and the second convolutional attention block is connected between the third residual dense block and the normalization layer.

2. The visual imaginary character recognition method based on EEG signal analysis according to claim 1, characterized in that: The visual imaginary character recognition result is specifically the category of the visual imaginary character; using the channel selection method based on maximum mutual information, the channel of the to-be-recognized electroencephalogram signal is screened to obtain the to-be-recognized electroencephalogram signal after channel screening, specifically including: For any category of visual imaginary characters, a number of test EEG signals are extracted; the test EEG signals are EEG signals having a number of channels; Using a z-score standardization processing algorithm to perform standardization processing on the test EEG signal to obtain a standardized test EEG signal; According to the standardized test EEG signals, the intra-class similarity and the inter-class similarity are calculated; the intra-class similarity is the maximum mutual information mean between the standardized test EEG signals of the same category, and the inter-class similarity is the maximum mutual information mean between the standardized test EEG signals of different categories; Calculate the discrimination score of each channel according to the intra-class similarity and the inter-class similarity; According to the discrimination scores of each channel, the channels of the to-be-identified electroencephalogram signal are screened to obtain the to-be-identified electroencephalogram signal after channel screening.

3. The visual imaginative character recognition method based on EEG signal analysis according to claim 2 is characterized in that: The experimental EEG signal was standardized according to the following formula: in, is the normalized EEG signal of the hth trial, is the mean value of the h-th trial EEG signal of each category, x h is the hth trial EEG signal, σ xh is the standard deviation of the hth trial EEG signal of each category, and H is the number of trial EEG signals of each category.

4. The visual imaginative character recognition method based on EEG signal analysis according to claim 3 is characterized in that: The intra-class similarity is calculated according to the following formula: Among them, D w is the intra-class similarity, mean() is the mean function, MIC() is the maximum mutual information function, is the standardized EEG signal of the tth trial, c h and c t Represent the categories of the hth trial EEG signal and the tth trial EEG signal respectively; The inter-class similarity is calculated according to the following formula: The discriminant score of the channel is calculated according to the following formula: D=μD w -(1-μ)D i ; Among them, D is the discriminant score of the channel and μ is the weighted hyperparameter.

5. The visual imaginary character recognition method based on EEG signal analysis according to claim 4 is characterized in that: The maximum mutual information is calculated according to the following formula: Where S is a two-dimensional finite set, x and y are the coordinates of the two-dimensional finite set S in the X and Y directions, respectively, B(|S|) is the upper limit of the grid partition x×y, p(x,y) is the joint probability density, and p(x) and p(y) are the edge probability densities.

6. The visual imaginary character recognition method based on EEG signal analysis according to claim 1, characterized in that: The recurrent neural network is a bidirectional long short-term memory network, including a forward long short-term memory network and a backward long short-term memory network; the forward long short-term memory network is used to extract a forward hidden state from an input signal; the backward long short-term memory network is used to extract a backward hidden state from an input signal; the character timing feature is obtained by splicing the forward hidden state and the backward hidden state.

7. The visual imaginative character recognition method based on EEG signal analysis according to claim 1, characterized in that: The parallel feature fusion network is used to assign different weights to different channels according to the learned inter-channel correlation to automatically adjust the importance of each channel feature map, so that the model can more effectively capture key features, reduce redundant information, and improve generalization ability.

8. A visual imaginary character recognition system based on EEG signal analysis, characterized in that: Used to implement the visual imaginary character recognition method based on EEG signal analysis as described in any one of claims 1 to 7, the visual imaginary character recognition system based on EEG signal analysis comprises: The EEG signal acquisition module to be identified is used to acquire the EEG signal to be identified; the EEG signal to be identified is an EEG signal with several channels; An EEG signal channel screening module is used to screen the channels of the EEG signal to be identified by using a channel selection method based on maximum mutual information to obtain the EEG signal to be identified after channel screening; A visual imagination character recognition module is used to input the EEG signal to be recognized after the channel screening into the visual imagination character recognition model to obtain a visual imagination character recognition result; the visual imagination character recognition model is a model based on a parallel space-time neural network framework, and the parallel space-time neural network framework includes a recurrent neural network, a multi-band nested residual dense network, a parallel feature fusion network and a character reconstruction network; the recurrent neural network is used to learn the temporal features of the characters from the input signal; the multi-band nested residual dense network is used to learn the spatial features of the characters from the input signal; the parallel feature fusion network is used to fuse the temporal features of the characters and the spatial features of the characters to obtain character fusion features; the character reconstruction network is used to reconstruct the visual imagination character recognition result based on the character fusion features.

Images