Method and system for decoding free association semantics by utilizing stereotactic electroencephalogram signals

Abstract

The invention discloses a method and system for decoding free association semantics through stereotactic electroencephalogram signals. The method comprises the steps that the free association stereotactic electroencephalogram signals generated when a user carries out free association on a target image are obtained; the method comprises the following steps: preprocessing free association stereotactic electroencephalogram signals, inputting the preprocessed free association stereotactic electroencephalogram signals into a feature extraction module and a classifier in a pre-trained free association electroencephalogram signal decoding network, and extracting electroencephalogram signal semantic features through the feature extraction module; the semantic features of the electroencephalogram signals are classified through a classifier to obtain the image category of the target image, and the free association electroencephalogram signal decoding network further comprises an auxiliary feature extraction module, a domain discriminator, a pre-training image feature extractor and a global adaptation module. The method aims at solving the problems that in existing research, free association brain signals are small in data size and high in prediction difficulty, and an existing deep learning model cannot be well suitable for semantic decoding of all-channel stereotactic electroencephalogram signals.

Description

Technical Field

[0001] The present invention relates to the field of vision and memory decoding of brain signals, and in particular to a method and system for decoding free association semantics using stereoscopic electroencephalographic signals. Background Art

[0002] Decoding the semantic information of brain signals during free association has always been one of the core goals of neuroscience research. By decoding brain signals during free association and predicting the semantic information of association, it is expected to promote the understanding of the association and memory mechanisms of the human brain, which is of great significance for the development of a universal brain-computer interface that is not constrained by experimental paradigms. Experiments to collect brain signals during free association take a lot of time and have a small amount of data. Therefore, it is difficult to predict the content of association using brain signals during free association. In recent years, significant progress has been made in the study of the physiological mechanisms of free association and recollection in the brain. However, research on the semantic decoding of free association using these physiological mechanisms is still very lacking. In addition, stereotactic EEG signals have problems such as different signal channels and implanted brain areas between subjects. Traditional deep learning models for cortical electroencephalogram (EEG) are not well suited for semantic decoding of full-channel stereotactic EEG signals. Summary of the invention

[0003] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method and system for decoding free association semantics using stereotactic EEG signals are provided. The present invention aims to solve the problems in existing research such as the small amount of free association brain signal data, high prediction difficulty and the inability of existing deep learning models to be well applicable to semantic decoding of full-channel stereotactic EEG signals.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for decoding free association semantics using stereotactic EEG signals, comprising the following steps: obtaining free association stereotactic EEG signals when a user freely associates with a target image; preprocessing the free association stereotactic EEG signals; inputting the preprocessed free association stereotactic EEG signals into a feature extraction module and a classifier in a pre-trained free association EEG signal decoding network, extracting EEG signal semantic features through the feature extraction module, and classifying the EEG signal semantic features through the classifier to obtain the image category of the target image, wherein the free association EEG signal decoding network includes, in addition to the feature extraction module and the classifier, an auxiliary feature extraction module. The auxiliary feature extraction module comprises a feature extraction module, a domain discriminator, a pre-trained image feature extractor and a global adaptation module, wherein the auxiliary feature extraction module and the feature extraction module have the same structure and share parameters, the auxiliary feature extraction module is used to extract EEG signal semantic features from image viewing stereoscopic EEG signals when a user views an image, the domain discriminator is used to distinguish whether the EEG signal semantic features are obtained by free association or image viewing, and the pre-trained image feature extractor is used to extract image features from a target image; the global adaptation module is used to map the EEG signal semantic features of the free association stereoscopic EEG signals and the image viewing stereoscopic EEG signals to the same semantic space.

[0005] Optionally, the training of the free association EEG signal decoding network includes: S101, for different types of target images, respectively obtaining two types of stereoscopic EEG signal samples, namely, free association stereoscopic EEG signal samples when the user freely associates with the target image and image viewing stereoscopic EEG signal samples when the user views the image; S102, preprocessing the two types of stereoscopic EEG signal samples in the data set, and dividing the target image and the two types of stereoscopic EEG signal samples after preprocessing into the data set; S103, extracting EEG signal semantic features from the preprocessed free association stereoscopic EEG signal samples in the data set using a feature extraction module, extracting EEG signal semantic features from the preprocessed image viewing stereoscopic EEG signal samples using an auxiliary feature extraction module, inputting the two types of EEG signal semantic features into the global adaptation module to map them to the same semantic space, and then distinguishing whether the EEG signal semantic features are obtained by free association or image viewing through a domain discriminator, and classifying them through a classifier, and continuously updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator by adversarial training until the specified round of training is completed or the classification accuracy of the classifier reaches a preset threshold, and when updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator, the loss function used for the feature extraction module, the global adaptation module, and the classifier is a weighted sum of the spatial aggregation loss and the feature alignment loss, the spatial aggregation loss is the class center distance of the two tasks in the feature space, the feature alignment loss is the distance between the features extracted from the stereoscopic EEG signal and the image features extracted using the pre-training model, and the loss function used for the domain discriminator is the cross entropy loss.

[0006] Optionally, the calculation function expression of the spatial aggregation loss is: , in, is the spatial aggregation loss, and Respectively indicate categories c Semantic features of EEG signals in source and target domains , where the source domain is the image viewing stereoscopic EEG signal samples, and the target domain is the free association stereoscopic EEG signal samples, Indicates the target domain category c The sample feature center of Represents the sample set about the source domain Semantic features of arbitrary EEG signals The expected value of Represents a sample set about the target domain Semantic features of arbitrary EEG signals The expected value of is the sample set of the source domain, is a sample set of the target domain; the calculation function expression of the feature alignment loss is: , in, is the feature alignment loss, represents the number of samples, Indicates i The semantic features of EEG signals of samples, Indicatesi The image corresponding to the sample is the image feature output by the pre-trained image feature extractor.

[0007] Optionally, the global adaptation module includes a self-attention mechanism within the domain and a cross-attention mechanism across domains, and mapping the EEG signal semantic features of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to the same semantic space includes: first, applying the self-attention mechanism to extract features for the EEG signal semantic features of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to achieve alignment within the domain, and then using the cross-attention mechanism to extract features for the aligned EEG signal semantic features within the domain to achieve mapping to the same semantic space.

[0008] Optionally, the preprocessing includes: using the label information of the brain area where each electrode in the input stereotactic EEG signal is located to eliminate the data of the electrodes located in the white matter area; for the electrodes on the same probe in the input stereotactic EEG signal, subtracting the data of the electrodes in the shallow brain area from the data of the adjacent electrodes in the deep brain area to achieve re-referencing of the stereotactic EEG signal; and performing a 50Hz notch filter on the re-referenced stereotactic EEG signal to remove power frequency interference.

[0009] Optionally, the feature extraction module includes a channel sampling module connected in sequence and N2 cascade-connected feature extraction units, the channel sampling module is used to randomly select channel slices to generate synthetic data for the input original data, and then expand it to a specified number of channels by splicing it to the original data; the feature extraction unit is composed of a time domain convolution of size 1×T and N1 cascade-connected information fusion modules, the information fusion module includes a layer of depth-separable convolution connected in sequence, a layer of convolution module with a convolution kernel size of 1×n for fusing the features of each channel, and a layer of convolution kernel size of The convolution module for fusing all channels and the residual connection, the residual connection combines the input features of the information fusion module with the convolution kernel size of The output features of the convolution module used to fuse all channels are fused and used as the output features of the information fusion module, where N1, N2 and T are natural numbers, n is the number of features, is the number of channels of stereotactic EEG signals.

[0010] Optionally, the image categories of the target image include buildings, natural scenes and faces.

[0011] In addition, the present invention also provides a system for decoding free association semantics using stereotactic EEG signals, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for decoding free association semantics using stereotactic EEG signals.

[0012] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the method of decoding free association semantics using stereotactic EEG signals through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method of decoding free association semantics using stereotactic EEG signals through a processor.

[0014] Compared with the prior art, the present invention mainly has the following advantages: the method of the present invention comprises obtaining a free association stereoscopic EEG signal when a user freely associates with a target image; preprocessing the free association stereoscopic EEG signal and inputting it into a feature extraction module and a classifier in a pre-trained free association EEG signal decoding network, extracting the semantic features of the EEG signal through the feature extraction module, and classifying the semantic features of the EEG signal through the classifier to obtain the image category of the target image, the free association EEG signal decoding network also comprises an auxiliary feature extraction module, a domain discriminator, a pre-trained image feature extractor and a global adaptation module, the present invention can solve the problems in existing research of small amount of free association brain signal data, high prediction difficulty and the problem that existing deep learning models are not well applicable to the semantic decoding of full-channel stereoscopic EEG signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0016] Figure 2 Schematic diagram of the network structure of the free association EEG signal decoding network in an embodiment of the present invention.

[0017] Figure 3 Schematic diagram of the training process of the free association EEG signal decoding network in an embodiment of the present invention.

[0018] Figure 4 Schematic diagram of the network structure of the feature extraction module in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0020] like Figure 1As shown, the method for decoding free association semantics using stereotactic EEG signals in this embodiment includes the following steps: obtaining a free association stereotactic EEG signal when a user freely associates with a target image; preprocessing the free association stereotactic EEG signal; inputting the preprocessed free association stereotactic EEG signal into a feature extraction module and a classifier in a pre-trained free association EEG signal decoding network, extracting EEG signal semantic features through the feature extraction module, and classifying the EEG signal semantic features through the classifier to obtain the image category of the target image. In this embodiment, the image categories of the target image include buildings, natural scenes, and faces, respectively. The free association stereotactic EEG signal when the user freely associates with the target image is a free association stereotactic EEG signal when the user freely associates with a target image of one of the three categories of buildings, natural scenes, and faces. Free association is free association with images that have been viewed, and the semantic features of the EEG signal are classified by the classifier to obtain the image category of the target image as buildings, natural scenes, or faces.

[0021] like Figure 2 As shown, in addition to the feature extraction module and the classifier, the free association EEG signal decoding network in this embodiment also includes an auxiliary feature extraction module, a domain discriminator, a pre-trained image feature extractor and a global adaptation module. The auxiliary feature extraction module and the feature extraction module have the same structure and share parameters. The auxiliary feature extraction module is used to extract EEG signal semantic features from the image viewing stereoscopic EEG signals when the user views the image, the domain discriminator is used to distinguish whether the EEG signal semantic features are obtained by free association or image viewing, and the pre-trained image feature extractor is used to extract image features from the target image; the global adaptation module is used to map the EEG signal semantic features of the free association stereoscopic EEG signals and the image viewing stereoscopic EEG signals to the same semantic space.

[0022] like Figure 3 As shown, the training of the free association EEG signal decoding network in this embodiment includes: S101, for different types of target images, respectively obtaining two types of stereoscopic EEG signal samples, namely, free association stereoscopic EEG signal samples when the user freely associates with the target image and image viewing stereoscopic EEG signal samples when the user views the image; S102, preprocessing the two types of stereoscopic EEG signal samples in the data set, and dividing the target image and the two types of stereoscopic EEG signal samples after preprocessing into the data set; S103, extracting EEG signal semantic features from the preprocessed free association stereoscopic EEG signal samples in the data set using a feature extraction module, extracting EEG signal semantic features from the preprocessed image viewing stereoscopic EEG signal samples using an auxiliary feature extraction module, inputting the two types of EEG signal semantic features into the global adaptation module to map them to the same semantic space, and then distinguishing whether the EEG signal semantic features are obtained by free association or image viewing through a domain discriminator, and classifying them through a classifier, and continuously updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator using an adversarial training method until the specified round of training is completed or the classification accuracy of the classifier reaches a preset threshold, and when updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator, the loss function used for the feature extraction module, the global adaptation module, and the classifier is a weighted sum of the spatial aggregation loss and the feature alignment loss, the spatial aggregation loss is the class center distance of the two tasks in the feature space, the feature alignment loss is the distance between the features extracted from the stereoscopic EEG signal and the image features extracted using the pre-training model, and the loss function used for the domain discriminator is the cross entropy loss. It should be noted that the method of combining domain discriminators for adversarial training is an existing method, and its implementation details are not described in detail here. The training method of the free association EEG signal decoding network in this embodiment (cross-task domain adaptation method) uses the brain signals of the subjects when watching related content to assist the cross-task domain adaptation training of free association semantic decoding. By aligning in the common semantic space, the accuracy of free association semantic decoding by the free association EEG signal decoding network is significantly improved.

[0023] Among them, the calculation function expression of spatial aggregation loss is: , in, is the spatial aggregation loss, and Respectively indicate categories c Semantic features of EEG signals in source and target domains , where the source domain is the image viewing stereoscopic EEG signal samples, and the target domain is the free association stereoscopic EEG signal samples, Indicates the target domain category c The sample feature center of Represents the sample set about the source domain Semantic features of arbitrary EEG signals The expected value of Represents a sample set about the target domain Semantic features of arbitrary EEG signals The expected value of is the sample set of the source domain, is the sample set of the target domain.

[0024] Among them, the calculation function expression of feature alignment loss is: , in, is the feature alignment loss, represents the number of samples, Indicates i The semantic features of EEG signals of samples, Indicates i The image corresponding to the sample is the image feature output by the pre-trained image feature extractor.

[0025] In this embodiment, the global adaptation module includes a self-attention mechanism within the domain and a cross-attention mechanism across domains, and mapping the semantic features of the EEG signals of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to the same semantic space includes: first, applying the self-attention mechanism to extract features for the semantic features of the EEG signals of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to achieve alignment within the domain, and then using the cross-attention mechanism to extract features for the semantic features of the EEG signals after alignment within the domain to achieve mapping to the same semantic space.

[0026] In this embodiment, preprocessing includes: using the label information of the brain area where each electrode in the input stereotactic EEG signal is located to eliminate the data of the electrode located in the white matter area; for the electrodes on the same probe in the input stereotactic EEG signal, subtracting the data of the electrode in the shallow brain area from the data of the adjacent electrode in the deep brain area to achieve re-reference of the stereotactic EEG signal; and performing a 50Hz notch filter on the stereotactic EEG signal after re-reference to remove power frequency interference.

[0027] In this embodiment, in step S102, the target image and its two types of preprocessed stereoscopic EEG signal samples are divided into data sets: all stereoscopic EEG signals when viewing the image and part of the stereoscopic EEG signals during free association are used for training, and the remaining stereoscopic EEG signals during free association are used for testing, and multi-fold cross-validation is performed.

[0028] like Figure 4 As shown, in this embodiment, the feature extraction module includes a channel sampling module connected in sequence and N2 cascade-connected feature extraction units. The channel sampling module is used to randomly select channel slices to generate synthetic data for the input original data and then expand it to a specified number of channels by splicing it to the original data; the feature extraction unit is composed of a time domain convolution of size 1×T and N1 cascade-connected information fusion modules. The information fusion module includes a layer of depth-separable convolution connected in sequence, a layer of convolution module with a convolution kernel size of 1×n for fusing the features of each channel, and a layer of convolution kernel size of The convolution module and residual connection used to fuse all channels. The residual connection combines the input features of the information fusion module with the convolution kernel size of The output features of the convolution module used to fuse all channels are fused and used as the output features of the information fusion module, where N1, N2 and T are natural numbers, n is the number of features, is the number of channels of stereotactic EEG signals.

[0029] The feature extraction module in this embodiment has the following advantages: first, by randomly selecting channel slices and splicing to expand to the specified number of channels, it can effectively adapt to data input with different numbers of channels; second, the feature extraction unit is composed of a time domain convolution of size 1×T and N1 cascaded information fusion modules, and the information fusion module includes a layer of depthwise separable convolution, a layer of convolution module with a convolution kernel size of 1×n for fusing the features of each channel, and a layer of convolution kernel size of The convolutional modules used to fuse all channels and the residual connection network structure can effectively capture cross-time and cross-channel dependencies at multiple levels.

[0030] In order to verify the effectiveness of the training method (cross-task domain adaptation method) of the free association EEG signal decoding network in the method of this embodiment in improving the accuracy of free association semantic decoding, the accuracy and variance obtained by using the stereoscopic EEG signal data of three experimental subjects collected by the present embodiment and the five-fold cross-validation method are shown in Table 1. In Table 1, the accuracy and variance of semantic decoding using only free association EEG signals (the deep learning model used is the free association EEG signal decoding network in this embodiment) are also compared.

[0031] Table 1 Comparison of classification accuracy using the cross-task domain adaptation method of this embodiment and using only free association stereotactic EEG data for training (classification of free association content categories, i.e., buildings, natural scenes, and faces)

[0032] As can be seen from Table 1, compared with training and testing using only stereoscopic EEG signals during free association, the training method of the free association EEG signal decoding network in the method of this embodiment (cross-task domain adaptation method) uses stereoscopic EEG signals during image viewing and free association for joint training, which can achieve higher accuracy and more stable prediction of free association semantic types.

[0033] In order to verify the effect of the feature extraction module in the method of this embodiment on decoding semantic features, this embodiment uses the stereoscopic EEG signal data of three experimental subjects collected by itself, and adopts the five-fold cross-validation method to obtain the accuracy and variance as shown in Table 2. At the same time, Table 2 also compares the accuracy and variance of semantic feature extraction using convolutional neural networks.

[0034] Table 2 Comparison of classification accuracy using the feature extraction module in the method of this embodiment and using a convolutional neural network (classification of free association content categories, i.e., buildings, natural scenes, and faces)

[0035] It can be seen from Table 2 that compared with using a convolutional neural network to extract features from stereotactic EEG signals, using the feature extraction module in the method of this embodiment can better realize the free association semantic extraction of stereotactic EEG signal encoding and achieve higher accuracy and more stable prediction. In summary, in order to solve the problems of small amount of free association data and high prediction difficulty in existing research, this embodiment uses the "psychological time retracing" theory in the episodic memory theory, that is, when recalling, the brain will reactivate a series of representations when the event occurs, and the semantic features of the stereoscopic EEG signals when the subjects watch the image and the stereoscopic EEG signals when they freely associate with the image are mapped to the same semantic space, and aligned with the image semantics. At the same time, in view of the large difference in the number of channels of stereoscopic EEG signals and the insufficient modeling of the dependency relationship between stereoscopic EEG signal channels and time in the existing model, a multi-level feature extraction module that can effectively capture cross-time and cross-channel dependencies is designed, which effectively improves the free association semantic decoding and improves the prediction accuracy and robustness.

[0036] In addition, this embodiment also provides a system for decoding free association semantics using stereotactic EEG signals, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for decoding free association semantics using stereotactic EEG signals.

[0037] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the method of decoding free association semantics using stereotactic EEG signals through a processor.

[0038] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method of decoding free association semantics using stereotactic EEG signals through a processor.

[0039] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present invention may be in the form of methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that instructions executed by the processor of a computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0040] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for decoding free association semantics using stereotactic EEG signals, characterized in that: The method comprises the following steps: obtaining a free association stereoscopic EEG signal when a user freely associates with a target image; preprocessing the free association stereoscopic EEG signal; The preprocessed free association stereotactic EEG signal is input into the feature extraction module and classifier in the pre-trained free association EEG signal decoding network, the semantic features of the EEG signal are extracted by the feature extraction module, and the semantic features of the EEG signal are classified by the classifier to obtain the image category of the target image. In addition to the feature extraction module and the classifier, the free association EEG signal decoding network also includes an auxiliary feature extraction module, a domain discriminator, a pre-trained image feature extractor and a global adaptation module. The auxiliary feature extraction module and the feature extraction module have the same structure and share parameters. The auxiliary feature extraction module is used to extract EEG signal semantic features from the image viewing stereotactic EEG signal when the user views the image, the domain discriminator is used to distinguish whether the EEG signal semantic features are obtained by free association or image viewing, and the pre-trained image feature extractor is used to extract image features from the target image; the global adaptation module is used to map the EEG signal semantic features of the free association stereotactic EEG signal and the image viewing stereotactic EEG signal to the same semantic space.

2. The method for decoding free association semantics using stereotactic EEG signals according to claim 1, characterized in that: The training of the free association EEG signal decoding network includes: S101, for different types of target images, respectively obtaining two types of stereoscopic EEG signal samples, namely, free association stereoscopic EEG signal samples when the user freely associates with the target image and image viewing stereoscopic EEG signal samples when the user views the image; S102, preprocessing the two types of stereoscopic EEG signal samples in the data set, and dividing the target image and the two types of stereoscopic EEG signal samples after preprocessing into the data set; S103, extracting EEG signal semantic features from the preprocessed free association stereoscopic EEG signal samples in the data set using a feature extraction module, extracting EEG signal semantic features from the preprocessed image viewing stereoscopic EEG signal samples using an auxiliary feature extraction module, inputting the two types of EEG signal semantic features into the global adaptation module to map them to the same semantic space, and then distinguishing whether the EEG signal semantic features are obtained by free association or image viewing through a domain discriminator, and classifying them through a classifier, and continuously updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator by adversarial training until the specified round of training is completed or the classification accuracy of the classifier reaches a preset threshold, and when updating the parameters of the feature extraction module, the global adaptation module, the classifier, and the domain discriminator, the loss function used for the feature extraction module, the global adaptation module, and the classifier is a weighted sum of the spatial aggregation loss and the feature alignment loss, the spatial aggregation loss is the class center distance of the two tasks in the feature space, the feature alignment loss is the distance between the features extracted from the stereoscopic EEG signal and the image features extracted using the pre-training model, and the loss function used for the domain discriminator is the cross entropy loss.

3. The method for decoding free association semantics using stereotactic EEG signals according to claim 2, characterized in that: The calculation function expression of the spatial aggregation loss is: ， in, is the spatial aggregation loss, and Respectively indicate categories c Semantic features of EEG signals in source and target domains , where the source domain is the image viewing stereoscopic EEG signal samples, and the target domain is the free association stereoscopic EEG signal samples, Indicates the target domain category c The sample feature center of Represents the sample set about the source domain Semantic features of arbitrary EEG signals The expected value of Represents a sample set about the target domain Semantic features of arbitrary EEG signals The expected value of is the sample set of the source domain, is a sample set of the target domain; the calculation function expression of the feature alignment loss is: ， in, is the feature alignment loss, represents the number of samples, Indicates i The semantic features of EEG signals of samples, Indicates i The image corresponding to the sample is the image feature output by the pre-trained image feature extractor.

4. The method for decoding free association semantics using stereotactic EEG signals according to claim 1, characterized in that: The global adaptation module includes a self-attention mechanism within the domain and a cross-attention mechanism across domains. The mapping of the EEG signal semantic features of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to the same semantic space includes: first, applying the self-attention mechanism to extract features for the EEG signal semantic features of the free association stereotactic EEG signals and the image viewing stereotactic EEG signals to achieve alignment within the domain, and then using the cross-attention mechanism to extract features for the aligned EEG signal semantic features within the domain to achieve mapping to the same semantic space.

5. The method for decoding free association semantics using stereotactic EEG signals according to claim 1, characterized in that: The preprocessing includes: using the label information of the brain area where each electrode in the input stereotactic EEG signal is located to eliminate the data of the electrode located in the white matter area; for the electrodes on the same probe in the input stereotactic EEG signal, subtracting the data of the adjacent electrode in the deep brain area from the data of the electrode in the shallow brain area to achieve re-reference of the stereotactic EEG signal; and performing 50Hz notch filtering on the stereotactic EEG signal after re-reference to remove power frequency interference.

6. The method for decoding free association semantics using stereotactic EEG signals according to claim 1, characterized in that: The feature extraction module includes a channel sampling module connected in sequence and N2 cascade-connected feature extraction units. The channel sampling module is used to randomly select channel slices to generate synthetic data for the input original data and expand it to a specified number of channels by splicing it to the original data; the feature extraction unit is composed of a time domain convolution of size 1×T and N1 cascade-connected information fusion modules. The information fusion module includes a layer of depth-separable convolution connected in sequence, a layer of convolution module with a convolution kernel size of 1×n for fusing the features of each channel, and a layer of convolution kernel size of The convolution module for fusing all channels and the residual connection, the residual connection combines the input features of the information fusion module with the convolution kernel size of The output features of the convolution module used to fuse all channels are fused and used as the output features of the information fusion module, where N1, N2 and T are natural numbers, n is the number of features, is the number of channels of stereotactic EEG signals.

7. The method for decoding free association semantics using stereotactic EEG signals according to claim 1, characterized in that: The image categories of the target images include buildings, natural scenes and faces.

8. A system for decoding free association semantics using stereotactic EEG signals, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the method for decoding free association semantics using stereotactic EEG signals as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the method for decoding free association semantics using stereotactic EEG signals as described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the method for decoding free association semantics using stereotactic EEG signals as described in any one of claims 1 to 7 through a processor.

Images