Three-dimensional convolution method for decoding imaginary language electroencephalogram topographic map

Abstract

The invention discloses a three-dimensional convolution method for decoding an imaginary language electroencephalogram topographic map. The method solves the problem that for a multi-rhythm electroencephalogram topographic map, an existing convolutional network method is insufficient in modeling capability, so that the decoding precision of the multi-rhythm electroencephalogram topographic map in an imaginary language is remarkably reduced. In the aspect of rectangular BEAM reconstruction, a frequency band specificity self-adaptive variation Kriging interpolation model is built, an optimal variation function is automatically selected according to spatial variation characteristics of electroencephalogram power of each frequency band, a high-resolution BEAM graph sequence under multiple frequency bands and multiple time steps is built, and spatial continuity and CNN structure adaptability are considered. A double-branch three-dimensional convolutional neural network is designed, high-dimensional feature extraction is performed on space-time and space-frequency band tensors, a multi-scale structure and dynamic expression ability of neural activity are mined, discrimination of two feature vector branches is dynamically weighted, and the recognition ability of a classifier to a language imagination electroencephalogram mode is improved.

Description

Technical Field

[0001] This invention belongs to the field of decoding technology of imaginary language in brain topography, specifically involving a three-dimensional convolution method for decoding imaginary language brain topography. Background Technology

[0002] In the field of Brain-Computer Interface (BCI) research, decoding imaginary language, as a natural and intuitive interaction method, has received widespread attention in recent years. Compared to motor imagery, imaginary language is closer to the actual intentions of human expression, possessing higher information capacity and practical potential. Precisely because of this characteristic of being closer to natural expression, decoding imaginary language is considered a key direction for breaking through the bottleneck of traditional BCI interaction efficiency, but it also faces more complex technical challenges.

[0003] The challenges of decoding imaginary language based on EEG signals mainly lie in the signal level and feature representation. EEG signals generated during imaginary language processing typically exhibit low signal-to-noise ratios, high nonlinearity, and non-stationarity, making it exceptionally difficult to extract stable language-related features from EEG data mixed with physiological noise. Furthermore, significant differences exist among individuals in language strategies and brain activation regions. When these inter-individual differences in neural response patterns exceed the model's fit, a unified decoding framework becomes ineffective, further complicating the construction of a universal model.

[0004] To address these challenges, researchers initially focused on the one-dimensional temporal dynamics of EEG signals, employing models such as recurrent neural networks, variant long short-term memory networks, and gated recurrent units to learn the temporal dependencies in EEG signals and classify the time series of EEG signals. However, these models, designed primarily for the temporal dimension, have limitations in capturing the spatial topological information of EEG signals, making it difficult to fully exploit the potential spatial structural information within EEG signals.

[0005] To fully extract the spatial features of EEG signals, researchers have begun to focus on modeling two-dimensional spatial features. Among these, Brain Electrical Activity Mapping (BEAM) has become a core tool for spatial information analysis because it can visually represent the spatial distribution of brain region activation. Researchers typically use the topoplot function of EEGLAB to generate event-related potential (ERP) topographic maps using nearest neighbor, linear, or spline interpolation methods, or convert EEG signals into two-dimensional image input models using EEG2Image technology. Some studies process EEG signals from multiple perspectives, such as generating task-specific topographic maps based on energy differences, generating mean, standard deviation, and peak distribution maps based on the power spectral density of EEG signals and ERP time-series statistics, extracting frequency band power using wavelet packet transform and mapping it to head shape space to construct frequency domain BEAM maps.

[0006] Although existing methods have made up for the lack of spatial information capture to some extent, the inherent defects of traditional BEAM still restrict the decoding performance: (1) Traditional BEAM is mostly circular structure, which is geometrically mismatched with the rectangular input format of convolutional neural network, which makes the edge information easy to be distorted in convolution operation and reduces the utilization efficiency of spatial features; (2) Research on the decoding of imaginary language mainly uses one-dimensional or two-dimensional convolutional networks, focusing on the single-dimensional extraction of one-dimensional temporal signals or two-dimensional EEG spatial features (circular EEG topography), lacking in-depth exploration of the multi-band EEG time-space-frequency feature subspace representation. Summary of the Invention

[0007] To address the shortcomings of existing convolutional network methods in modeling multi-rhythm EEG topography, which leads to a significant decrease in decoding accuracy in imaginary language, this invention provides a three-dimensional convolutional method for decoding EEG topography in imaginary language. The method includes the following steps: S1. Collect a dataset of real-life imagined language EEG signals using electrodes, including Types of imagination; S2. Frequency band specific Kriging interpolation method is used to construct frequency band specific rectangular BEAM sequence for EEG signal, and the original electrode spatial layout is mapped to a fixed-size two-dimensional rectangular grid to obtain the BEAM map of EEG signal. S3. Employ a three-dimensional continuous space-frequency band representation method to stack BEAM maps of multiple frequency bands of EEG signals at the same time point; S4. Using a three-dimensional continuous space-time representation method, stack BEAM maps of multiple consecutive time points in the same frequency band of EEG signals; S5. A dual-branch 3D imaginary language encoding method is adopted, integrating the discriminative features from the perspectives of steps S3 and S4. An attention-weighted feature fusion decoding method is used to achieve adaptive weighted fusion of the importance of features from different modalities. Then, a 2-layer MLP network is used to map the fused features to... Given a classification result, obtain the probability corresponding to each classification result.

[0008] Furthermore, in step S2, when constructing a frequency-specific rectangular BEAM sequence using the frequency-specific Kriging interpolation method on the EEG signal, the spatial structure characteristics of different frequency bands are set as linear, Gaussian, exponential, or spherical variation models. A leave-one-out cross-validation strategy is adopted, and the mean square error is selected as the index. The variation model with the smallest mean square error is selected for full-image interpolation.

[0009] Furthermore, in step S2, when mapping the original electrode spatial layout to a fixed-size two-dimensional rectangular mesh, an edge compensation strategy is proposed to extract the radius of the furthest point from the electrode. edge area , Represents a point in a two-dimensional rectangular grid. The width of the edge region, for all Calculate the minimum distance from the point to the edge point. Find the nearest edge value Applying the exponential decay interpolation formula To achieve a natural transition from a circular beam diagram to a rectangular beam diagram, the formula is as follows: The power value is the power value at the nearest boundary point. The normalized distance from the external point to the nearest point on the boundary. This represents the power estimate for the region outside the circular mask in the rectangular BEAM diagram.

[0010] Furthermore, step S3 specifically involves: processing the EEG signals corresponding to each electrode. Power spectral density was extracted from six typical frequency bands using a bandpass filter, yielding the characteristic representation of each channel in different frequency bands. , Number of frequency bands; The six typical frequency bands are, in order: delta: 1–4 Hz, theta: 4–8 Hz, alpha: 8–13 Hz, beta: 13–30 Hz, low-gamma: 30–55 Hz, and high-gamma: 55–100 Hz, respectively. , , , , and correspond; Then for each typical frequency band Frequency-specific rectangular EEG topographic maps are constructed using the interpolation method in step S2. , It is the size of the spatially interpolated image. After stacking all the frequency band images, a spatial-frequency joint tensor is formed. : .

[0011] Furthermore, step S4 specifically involves: continuously The electrode amplitude at each time point within each time step is interpolated in step S2 to generate a rectangular EEG topography map of consecutive frames. This indicates the electric field distribution at this point in time. Indicates the number of electrodes, including all The frames are stacked in chronological order to form a space-time joint tensor. , .

[0012] Furthermore, in step S5, the dual-branch 3D imaginative language encoding method is performed through a spatial-temporal domain branch network and a spatial-frequency domain branch network, and the attention-weighted feature fusion decoding method is performed through an attention-weighted feature fusion classification network.

[0013] Furthermore, in the space-time domain branching network, the input is a five-dimensional tensor. Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions. For batch size, For channel dimensions; First, a layer-by-layer 3D convolution operation is performed: ; Get output ,in, Number of output channels Input the number of channels. The kernel size is the convolution kernel size. Indicates input, This represents layer-by-layer 3D convolution. Represents the convolution kernel. Indicates offset, subscript Indicates the first 3D convolution; Then Batch normalization and activation to obtain , ,in For activation function, This is a 3D batch normalization operation; Will The input layers are sequentially a pooling layer, an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. . Furthermore, in the space-frequency domain branch network, the input is a five-dimensional tensor. Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions; First, a layer-by-layer 3D convolution operation is performed: ; Get output ,in, Number of output channels Input the number of channels. The kernel size is the convolution kernel size. Indicates input, This represents layer-by-layer 3D convolution. Represents the convolution kernel. Indicates offset, subscript Indicates the first 3D convolution; Then Batch normalization and activation to obtain , ; Will The input layers are sequentially a pooling layer, an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. . Furthermore, in the attention-weighted feature fusion classification network, two feature vectors are connected. , express Dimensions express The dimension is input into a standard 2-layer MLP network, and the fusion weights of the two branches are calculated. ,in These are the weighting coefficient parameters. This indicates the number of neurons in the hidden layer of an MLP network. express Dimensions Weights are assigned to attention, and the final fused features are Finally, a two-layer MLP network is used to fuse the features. Mapping to imagined category classification results This yields the probability corresponding to each category of imagination.

[0014] The beneficial effects of the method described in this invention are as follows: (1) In terms of rectangular BEAM reconstruction, a frequency band-specific adaptive variogram Kriging interpolation model was built. The optimal variogram function was automatically selected based on the spatial variation characteristics of EEG power in each frequency band, and a high-resolution BEAM sequence under multiple frequency bands and multiple time steps was constructed, taking into account both spatial continuity and CNN structural adaptability. (2) In terms of continuous EEG time-frequency-space structure optimization, based on bandpass filter and power spectrum estimation, the spatial distribution features of multiple typical frequency bands are extracted. By stacking the frequency band maps, a three-dimensional space-frequency band tensor is formed to model the spectral band information and its spatial projection rules in the language imagination process. A space-time BEAM map sequence is constructed, and the BEAM maps corresponding to the continuous time frames in the language imagination trials are stacked into a three-dimensional tensor to capture the spatiotemporal dynamic evolution mode of the electric field. (3) A dual-branch three-dimensional convolutional neural network was designed to extract high-dimensional features from the space-time and space-frequency tensors respectively, explore the multi-scale structure and dynamic expression ability of neural activity, dynamically weight the discriminativeness of the two feature vector branches, and improve the classifier's ability to recognize language imagination EEG patterns. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the three-dimensional continuous space-frequency band characterization method and the three-dimensional continuous space-time characterization method in the embodiments of the present invention; Figure 3 This is a schematic diagram of the feature fusion encoding and decoding method for attention weights in an embodiment of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0017] Example 1 This embodiment provides a three-dimensional convolution method for decoding EEG topography of imaginary language. The flow of the method is as follows: Figure 1 As shown, it includes the following steps: S1. Collect a dataset of real-world imaginary language EEG signals, including four types of imaginary up, down, left, and right; S2. Frequency band specific Kriging interpolation method is used to construct frequency band specific rectangular BEAM sequence for EEG signal, and the original electrode spatial layout is mapped to a fixed-size two-dimensional rectangular grid to obtain the BEAM map of EEG signal. S3. Employ a three-dimensional continuous space-frequency band representation method to stack BEAM maps of multiple frequency bands of EEG signals at the same time point; S4. Using a three-dimensional continuous space-time representation method, stack BEAM maps of multiple consecutive time points in the same frequency band of EEG signals; S5. A two-branch three-dimensional imaginary language encoding method is adopted to integrate the discriminative features from the perspectives of steps S3 and S4. An attention-weighted feature fusion decoding method is used to achieve adaptive weighted fusion of the importance of features of different modalities. Then, a two-layer MLP network is used to map the fused features to the 4-classification results.

[0018] Example 2 This embodiment further defines embodiment 1 and provides further explanation of step S2.

[0019] This embodiment constructs a frequency band-specific adaptive Kriging interpolation method to map the original electrode spatial layout onto a fixed-size two-dimensional rectangular grid to adapt the input structure of EEG signal data in subsequent space-time and space-frequency band modeling.

[0020] The frequency band-specific Kriging interpolation method sets linear, Gaussian, exponential, and spherical variation models according to the spatial structure characteristics of different frequency bands. A leave-one-out cross-validation (LOOCV) strategy is employed, using the mean squared error (MSE) as the metric, and selecting the variation model with the smallest MSE for full-map interpolation.

[0021] First, the raw EEG signal is segmented and processed, with frequency bands set. Divided into logarithmic intervals Each frequency point is segmented into Each frequency band, and the Morlet wavelet period in each frequency band. It is used to balance frequency band and time resolution.

[0022] The six typical frequency bands are, in order: delta: 1–4 Hz, theta: 4–8 Hz, alpha: 8–13 Hz, beta: 13–30 Hz, low-gamma: 30–55 Hz, and high-gamma: 55–100 Hz.

[0023] For the Signal of each channel Extraction time step Time band Power spectrum under , The center frequency band is Morlet wavelet, This is a convolution operation. Channel refers to the physical lead channels (number of electrodes) during EEG signal acquisition.

[0024] Then, the two-dimensional coordinates of each electrode in the standard scalp electrode distribution are extracted. Construct within the minimum outer bound matrix of the electrodes (the minimum outer bound matrix of all electrodes as a whole). Two-dimensional grid With the maximum radius of the electrode Construct a circular mask to exclude the outer area of ​​the scalp.

[0025] For each time step With frequency band Within this process, the average power of all channels is calculated to obtain stable spatial distribution characteristics. : ; Considering the frequency band dependence of spatial patterns of EEG activity across different frequency bands, this embodiment employs an optimal selection mechanism based on the variogram function to model each frequency band separately and estimate all channels within each band. interpolation results : ; This represents the number of EEG channels involved in the interpolation calculation, for each of the n channels. Multiply by the corresponding weight Then summate to obtain the interpolation result. .

[0026] To be based on the variation function model The calculated weights are set according to the spatial structure characteristics of different frequency bands. Linear, Gaussian, exponential, and spherical variation models were used. A leave-one-out cross-validation strategy was employed, and the mean squared error (MSE) was selected as the metric. The variation model with the smallest MSE was chosen for full-map interpolation.

[0027] Set the rectangular grid size to 500×500, and for each point in the grid... By applying the frequency band-specific Kriging interpolation method described above, a spatially continuous power estimation map, i.e., a circular BEAM map, is obtained.

[0028] Since the grid is rectangular while the electrode distribution is typically approximately circular, there are missing regions outside the grid. To improve image continuity and network adaptability, an edge compensation strategy is proposed. For spatially continuous power estimation maps, the edge compensation strategy extracts the region with the radius of the furthest point from the electrode. edge area , This represents the width of the edge region. For all... Calculate the minimum distance from the point to the edge point. Find the nearest edge value Applying the exponential decay interpolation formula To achieve a natural transition from a circular beam diagram to a rectangular beam diagram, the formula is as follows: The power value is the power value at the nearest boundary point. The normalized distance from the external point to the nearest point on the boundary. This represents the power estimate for the region outside the circular mask in the rectangular BEAM diagram. It does not directly represent the entire rectangular beam image, but rather the power estimate of the region outside the circular mask in the rectangular beam image (i.e., the rectangular region not covered by the original circular beam image). It is a supplement to the circular beam image, and together with the power values ​​of the circular region, it forms the complete rectangular beam image.

[0029] Example 3 This embodiment further defines embodiment 1 and provides further explanation of steps S3-S4.

[0030] like Figure 2 As shown, for each EEG channel signal Power spectral density was extracted for six typical frequency bands (delta: 1–4 Hz, theta: 4–8 Hz, alpha: 8–13 Hz, beta: 13–30 Hz, low-gamma: 30–55 Hz, and high-gamma: 55–100 Hz) using bandpass filters, yielding the characteristic representation of each channel in different frequency bands. , This refers to the number of frequency bands.

[0031] Then for each frequency band Frequency-specific rectangular EEG topographic maps are constructed using the interpolation method in step S2. , It is the size of the spatially interpolated image. After stacking all the frequency band images, a spatial-frequency joint tensor is formed. : .

[0032] Figure 2 In this model, each channel of the tensor corresponds to the spatial distribution of a specific frequency band, which can be viewed as a spatial projection of brain neural activity under multiple frequencies during language imagination. This representation method effectively preserves the physiological meaning of different frequency bands and enhances the spatial specificity of local brain region activity through spatial interpolation.

[0033] like Figure 2 As shown, the continuous The electrode amplitude at each time point within each time step is interpolated in step S2 to generate a rectangular EEG topography map of consecutive frames. This represents the electric field distribution at this point in time. Stacking all t frames in chronological order forms a three-dimensional continuous space-time joint tensor. , .like Figure 2 As shown, each channel of this tensor corresponds to a spatial distribution at a point in time, representing the dynamic changes of EEG signals over a continuous period of time.

[0034] Example 4 This embodiment further defines embodiment 1 and provides further explanation of step S5.

[0035] like Figure 3 As shown, this embodiment designs a bi-branch 3D convolutional model based on three-dimensional continuous space-time-frequency band features. High-dimensional dynamic representations are extracted from the space-time domain and the space-frequency band domain respectively. Combined with the temporal evolution pattern and frequency band characteristics of EEG signals, the model's ability to perceive differences in brain activity during language conception is enhanced.

[0036] The model includes a space-time domain branch network, a space-frequency domain branch network, and a feature fusion classification network (decoding network) with attention weights.

[0037] (1) Spatial-temporal branch network: This network is used to extract the three-dimensional features of spatial-temporal EEG images (spatial structure + time series dimension) and map them into 128-dimensional vectors.

[0038] Input five-dimensional tensor Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions, The batch size is 1, where 1 represents the number of channels and the C-dimensional dimension. Each pixel contains only one amplitude value. For time frames, Given a rectangular BEAM graph space, the following 3D convolution operations are performed layer by layer: ; get ,in, Number of output channels Input the number of channels. The kernel size is 5×3×3 (capturing 3×3 local spatial features of dynamics over 5 consecutive time steps).

[0039] Will Batch normalization and activation to obtain , ,in For activation function, This is for 3D batch normalization.

[0040] Will The input layers are sequentially a pooling layer (spatial downsampling only), an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. The operations performed in the pooling layer are as follows: ; in, Indicates input features, and These represent the features after the first and second pooling operations, respectively. , and These represent the first layer of 3D max pooling, the second layer of 3D max pooling, and the third layer of 3D max pooling, respectively.

[0041] ; in, This indicates an adaptive average pooling operation. Indicates the flattening operation. This indicates a fully connected operation.

[0042] (2) Spatial-Frequency Domain Branch Network: This network is used to extract the three-dimensional features (spatial structure + frequency sequence dimension) of spatial-frequency EEG images, and finally map them into a 128-dimensional vector. The input shape is a five-dimensional tensor. Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions; After 3D convolution operation: ; in, Number of output channels Input the number of channels. kernel size ×3×3 (Capture adjacent) (3×3 local spatial features of interaction of each frequency band).

[0043] Will Batch normalization and activation to obtain : .

[0044] Will The input layers are sequentially a pooling layer (spatial downsampling only), an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. The operations performed in the pooling layer are as follows: ; in, Indicates input features, and These represent the features after the first and second pooling operations, respectively. , and These represent the first layer of 3D max pooling, the second layer of 3D max pooling, and the third layer of 3D max pooling, respectively.

[0045] .

[0046] (3) Feature fusion classification network based on attention weight: In order to make full use of the complementarity of the two modalities and avoid the redundancy or information interference caused by simple splicing, a feature fusion module based on attention weight is designed to realize adaptive weighting of the importance of features of different modalities, thereby enhancing the expressive power of the final feature representation.

[0047] , express Dimensions express The dimension is input into a standard 2-layer MLP network, and the fusion weights of the two branches are calculated. ,in These are the weighting coefficient parameters. This indicates the number of neurons in the hidden layer of an MLP network. express Dimensions Weights are assigned to attention, and the final fused features are Finally, a two-layer MLP network is used to fuse the features. Mapping to imagined category classification results This yields the probability corresponding to each category of imagination.

[0048] Example 5 This embodiment further defines embodiments 1-4, and illustrates the beneficial effects of the method described in this invention through comparative experiments.

[0049] Using the method of this invention, 1200 samples (each sample consisting of 50 consecutive frames) of four directional imagination tasks (up, down, left, and right) were classified and predicted under three task conditions (actual vocalization Pron, language imagination Inner, and visual imagination Vis) involving 10 healthy subjects. The comparison results are shown in Table 1 (This paper represents the method of this invention). Compared with recent EEG decoding methods such as KNN, SVM, XGBoost, LSTM, BiLSTM, and EEGNet, the 3D CNN method proposed in this invention achieved superior classification accuracy in both the Pron and Inner task modalities, reaching 62.67% and 61.79% respectively. This is significantly higher than traditional KNN and SVM models (maximum 59.55%), and also significantly better than widely used temporal modeling methods such as LSTM and BiLSTM (maximum 31.30%). Even in the most challenging visual imagination tasks, our model still achieves an accuracy of 53.25%, far exceeding EEGNet's 29.67% and the performance of other deep learning methods.

[0050] Where Work represents the source of the method, Classifier represents the classification method corresponding to the method, and Accuracy represents the classification accuracy.

[0051] [1] The specific reference is: Lopez-Bernal D, Balderas D, Ponce P, Molina A. Exploring inter-trial coherence for inner speech classification in EEG-based brain-computer interface. J Neural Eng 26, 21 (2024). [2] The specific literature mentioned is: Van den Berg B, Van Donkelaar S and Alimardani M2021 Inner speech classification using EEG signals: A deep learning approach2021 IEEE 2nd Int. Conf. on Human-Machine Systems (ICHMS) (IEEE) 1–4 (2021). [3] The specific reference is: Gasparini F, Cazzaniga E and Saibene A. Innerspeech recognition through electroencephalographic signals, (2022). [4] The specific reference is: Merola, NR, Venkataswamy, NG, Imtiaz, MHCan Machine Learning Algorithms Classify Inner Speech from EEG Brain Signals? 2023 IEEE World AI IoT Congress (AIIoT), Seattle, WA, USA, 7–10 June 2023, 466–470 (2023). [5] The specific reference is: Ng, HW, & Guan, C. Efficient representation learning for inner speech domain generalization. In International conference on computer analysis of images and patterns, 131–141 (2023). [6] The specific reference is: Ng HW, Guan C. Subject-independent meta-learning framework towards optimal training of EEG-based classifiers. NeuralNetworks 172, (2024). Table 1. Comparison of four-class classification accuracy of different methods on the same dataset:

Claims

1. A three-dimensional convolution method for decoding brainwave topography of imaginary language, characterized in that, The method includes the following steps: S1. Collect a dataset of real-life imagined language EEG signals using electrodes, including Types of imagination; S2. Frequency band specific Kriging interpolation method is used to construct frequency band specific rectangular BEAM sequence for EEG signal, and the original electrode spatial layout is mapped to a fixed-size two-dimensional rectangular grid to obtain the BEAM map of EEG signal; S3. Employ a three-dimensional continuous space-frequency band representation method to stack BEAM maps of multiple frequency bands of EEG signals at the same time point; S4. Using a three-dimensional continuous space-time representation method, stack BEAM maps of multiple consecutive time points in the same frequency band of EEG signals; S5. A dual-branch 3D imaginary language encoding method is adopted, integrating the discriminative features from the perspectives of steps S3 and S4. An attention-weighted feature fusion decoding method is used to achieve adaptive weighted fusion of the importance of features from different modalities. Then, a 2-layer MLP network is used to map the fused features to... Given a classification result, obtain the probability corresponding to each classification result.

2. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 1, characterized in that, In step S2, when constructing a frequency-specific rectangular BEAM sequence using the frequency band-specific Kriging interpolation method for the EEG signal, the spatial structure characteristics of different frequency bands are set as linear, Gaussian, exponential, or spherical variation models. The leave-one-out cross-validation strategy is adopted, and the mean square error is selected as the index. The variation model with the smallest mean square error is selected for full-image interpolation.

3. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 2, characterized in that, In step S2, when mapping the original electrode spatial layout to a fixed-size two-dimensional rectangular mesh, an edge compensation strategy is proposed to extract the radius of the furthest point from the electrode. edge area , Represents a point in a two-dimensional rectangular grid. The width of the edge region, for all Calculate the minimum distance from the point to the edge point. Find the nearest edge value Applying the exponential decay interpolation formula To achieve a natural transition from a circular beam diagram to a rectangular beam diagram, the formula is as follows: The power value is the power value at the nearest boundary point. The normalized distance from the external point to the nearest point on the boundary. This represents the power estimate for the region outside the circular mask in a rectangular BEAM diagram.

4. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 3, characterized in that, Step S3 specifically involves: processing the EEG signals corresponding to each electrode. Power spectral density was extracted from six typical frequency bands using a bandpass filter, yielding the characteristic representation of each channel in different frequency bands. , Number of frequency bands; The six typical frequency bands are, in order: delta: 1–4 Hz, theta: 4–8 Hz, alpha: 8–13 Hz, beta: 13–30 Hz, low-gamma: 30–55 Hz, and high-gamma: 55–100 Hz, respectively. , , , , and correspond; Then for each typical frequency band Frequency-specific rectangular EEG topographic maps are constructed using the interpolation method in step S2. , It is the size of the spatially interpolated image. After stacking all the frequency band images, a spatial-frequency joint tensor is formed. : .

5. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 4, characterized in that, Step S4 specifically involves: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The electrode amplitude at each time point within each time step is interpolated in step S2 to generate a rectangular EEG topography map of consecutive frames. This indicates the electric field distribution at this point in time. Indicates the number of electrodes, including all The frames are stacked in chronological order to form a space-time joint tensor. , .

6. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 5, characterized in that, In step S5, the dual-branch 3D imaginative language encoding method is performed through a spatial-temporal domain branch network and a spatial-frequency domain branch network, while the attention-weighted feature fusion decoding method is performed through an attention-weighted feature fusion classification network.

7. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 6, characterized in that, In the aforementioned space-time domain branching network, the input is a five-dimensional tensor. Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions. For batch size, For channel dimensions; First, a layer-by-layer 3D convolution operation is performed: ； Get output ,in, Number of output channels Input the number of channels. The kernel size is the convolution kernel size. Indicates input, This represents layer-by-layer 3D convolution. Represents the convolution kernel. Indicates offset, subscript Indicates the first 3D convolution; Then Batch normalization and activation to obtain , ,in For activation function, This is a 3D batch normalization operation; Will The input layers are sequentially a pooling layer, an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. .

8. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 7, characterized in that, In the space-frequency domain branch network, the input is a five-dimensional tensor. Five-dimensional tensor Yes The tensor obtained after expanding the batch and channel dimensions; First, a layer-by-layer 3D convolution operation is performed: ； Get output ,in, Number of output channels Input the number of channels. The kernel size is the convolution kernel size. Indicates input, This represents layer-by-layer 3D convolution. Represents the convolution kernel. Indicates offset, subscript Indicates the first 3D convolution; Then Batch normalization and activation to obtain , ; Will The input layers are sequentially a pooling layer, an adaptive average pooling layer, and a flattening and fully connected layer, and the output is a feature vector. .

9. The three-dimensional convolution method for decoding EEG topography of imaginary language according to claim 8, characterized in that, In the attention-weighted feature fusion classification network, two feature vectors are connected. , express Dimensions express The dimension is input into a standard 2-layer MLP network, and the fusion weights of the two branches are calculated. ,in These are the weighting coefficient parameters. This indicates the number of neurons in the hidden layer of an MLP network. express Dimensions Weights are assigned to attention, and the final fused features are Finally, a two-layer MLP network is used to fuse the features. Mapping to imagined category classification results This yields the probability corresponding to each category of imagination.

Images