Electroencephalogram signal processing method and device and computer device

Abstract

The application relates to an electroencephalogram signal processing method, device and computer equipment. The method comprises the following steps: introducing a frequency domain modeling branch on the basis of an original electroencephalogram base model structure by respectively performing time domain processing and frequency domain processing on electroencephalogram signals, realizing time domain and frequency domain dual-mode collaborative coding, and constructing a target neural codebook containing a time domain special codebook, a frequency domain special codebook and a time-frequency shared codebook. Then, the target neural codebook is used as the basis for mask pre-training, the precise construction of the target electroencephalogram model is realized, and key support is provided for efficient decoding of electroencephalogram signals and upgrading of brain-computer interface technology.

Description

Technical Field

[0001] This application relates to the field of data intelligence processing technology, and in particular to a method, apparatus and computer device for processing electroencephalogram (EEG) signals. Background Technology

[0002] Electroencephalography (EEG) signals, due to their ability to directly reflect brain neural activity, have advantages over traditional signal recognition methods such as facial expressions and speech recognition in terms of resistance to spoofing and real-time performance. They have become a research hotspot in recent years, with applications in fields such as emotion recognition, disease diagnosis and physical rehabilitation, and consciousness decoding.

[0003] Existing EEG base models, such as the LaBraM model, only employ time-domain encoding and are built based on a single neural codebook. In emotion recognition applications, they suffer from the following drawbacks: Firstly, the time-domain single-dimensional encoding representation is incomplete; for emotion signals exhibiting both temporal abrupt changes and frequency-domain rhythmic features, it ignores the energy distribution patterns of EEG in the frequency domain. Secondly, the granularity of a single neural codebook is limited, but neural activity patterns have significant time-frequency decoupling characteristics; different neural frequency components may be contained simultaneously within the same time slice, making it difficult for a single codebook to accurately represent the differences in distribution and semantics between time-domain and frequency-domain features. Thirdly, they lack cross-modal complementary information, resulting in poor anti-interference ability and robustness; that is, they rely solely on self-supervised learning within a single modality and lack multi-view or cross-modal information interaction. Summary of the Invention

[0004] Based on this, the purpose of this application is to provide a brainwave signal processing method, apparatus, and computer device that can improve the accuracy of emotion recognition, so as to solve the technical problems mentioned in the background art.

[0005] Firstly, this application provides a method for processing electroencephalogram (EEG) signals. This includes:

[0006] The EEG signal is processed in time and frequency to obtain the target neural codebook; the target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook;

[0007] Based on the target neural codebook, mask pre-training is performed to determine the target EEG model.

[0008] In one embodiment, time-frequency processing of an EEG signal to obtain a target neural codebook includes: extracting initial features of the EEG signal and embedding the initial features at locations to obtain candidate features; the candidate features include candidate time-domain features and candidate frequency-domain features; processing the candidate features through a first encoder to obtain target time-domain features and target frequency-domain features; and constructing a target neural codebook based on the target time-domain features and the target frequency-domain features.

[0009] In one embodiment, the initial features include initial time-domain features and initial frequency-domain features; the extraction of the initial features of the EEG signal includes: slicing the EEG signal into multiple segments, extracting the frequency-domain features of the segments through real-valued fast Fourier transform to obtain initial frequency-domain features; and extracting features from the segments through a time-domain convolution module to obtain initial time-domain features; the time-domain convolution module includes one-dimensional convolution, group normalization, and a target activation function.

[0010] In one embodiment, constructing a target neural codebook based on the target time-domain features and the target frequency-domain features includes: integrating the target time-domain features and the target frequency-domain features to obtain a first fusion feature; determining the embedding vectors contained in the target neural codebook to be generated; quantizing the target time-domain features, the target frequency-domain features, and the first fusion feature according to the embedding vectors to obtain a quantization vector corresponding to the target neural codebook; the quantization vector includes a time-domain quantization vector, a frequency-domain quantization vector, and a shared quantization vector; and decoding and reconstructing the various quantization vectors to obtain a reconstructed signal corresponding to the target neural codebook.

[0011] In one embodiment, decoding and reconstructing multiple quantization vectors to obtain a reconstructed signal corresponding to the target neural codebook includes: additively fusing the time-domain quantization vector and the shared quantization vector, and processing the fused quantization vector through a time-domain decoder to obtain a reconstructed time-domain signal; additively fusing the frequency-domain quantization vector and the shared quantization vector, and processing the fused quantization vector through a frequency-domain decoder to obtain the reconstructed signal amplitude spectrum and phase spectrum.

[0012] In one embodiment, determining a target EEG model through mask pre-training based on the target neural codebook includes: performing a masking operation on the initial features of the EEG signal according to a pre-determined mask matrix to obtain mask features; the mask matrix includes a primary mask and a corresponding inverse mask; obtaining a second fusion feature based on a second encoder and the mask features; processing the second fusion feature through a third encoder to obtain training features; the third encoder has half the number of layers of the second encoder; mapping the training features to the corresponding codebook space through a linear classification head to obtain predicted features; updating parameters based on a loss function and according to the difference between the predicted features and the time-frequency shared codebook, until the target EEG model is determined.

[0013] In one embodiment, the target neural codebook is determined by a vector quantization time-frequency prediction model, which includes quantization loss, reconstruction loss, contrastive loss, and consistency loss. The quantization loss includes commitment loss and codebook learning loss, and the consistency loss is determined based on the structure of positive and negative samples constructed by information noise contrastive estimation. The loss function of the target EEG model includes a cross-entropy loss function, and the model training loss includes the prediction loss corresponding to the master mask and the prediction loss corresponding to the inverse mask.

[0014] Secondly, this application also provides a brainwave signal processing device. It includes:

[0015] A neural codebook determination module is used to perform time-frequency processing on EEG signals to obtain a target neural codebook; the target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook.

[0016] The mask pre-training module is used to perform mask pre-training based on the target neural codebook to determine the target EEG model.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described electroencephalogram (EEG) signal processing method.

[0018] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-described electroencephalogram (EEG) signal processing method.

[0019] The aforementioned EEG signal processing method, device, computer equipment, and readable storage medium, by performing time-domain and frequency-domain processing on EEG signals respectively, introduce a frequency-domain modeling branch based on the original EEG base model structure, realizing dual-modal collaborative coding in the time and frequency domains, and constructing a target neural codebook containing time-domain dedicated, frequency-domain dedicated, and time-frequency shared codebooks. Then, based on this target neural codebook, mask pre-training is carried out, achieving accurate construction of the target EEG model, providing key support for efficient EEG signal decoding and the upgrading of brain-computer interface technology. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating the application environment of an EEG signal processing method in one embodiment.

[0021] Figure 2 This is a flowchart illustrating an EEG signal processing method in one embodiment;

[0022] Figure 3 This is a schematic diagram of the architecture for determining the target neural codebook in one embodiment;

[0023] Figure 4 This is a schematic diagram of the mask pre-training architecture in one embodiment;

[0024] Figure 5 This is a schematic diagram of the architecture for fine-tuning downstream tasks in one embodiment;

[0025] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] The EEG signal processing method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is implemented through interaction between terminal 102 and server 104. Terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 is used to acquire the user's EEG signals. Server 104 is used to perform time-frequency processing on the EEG signals to obtain the target neural codebook, and performs mask pre-training based on the target neural codebook to determine the target EEG model. Terminal 102 can be, but is not limited to, various EEG patch sensors or portable wearable devices. Server 104 is implemented using a standalone server or a server cluster composed of multiple servers. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0028] In one embodiment, such as Figure 2 As shown, a brainwave signal processing method is provided, and its application to a server is illustrated using this method as an example. The method includes the following steps:

[0029] Step 202: Perform time-frequency processing on the EEG signal to obtain the target neural codebook.

[0030] The target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook. The neural codebook represents a set of discrete vectors that are trained end-to-end with the neural network and can be dynamically optimized according to the task. It is used to map "continuous" inputs (such as features, embeddings, channel states, glyph fine-tuning, etc.) to "discrete" codeword indices, and then use the corresponding codewords to represent or reconstruct the original signal.

[0031] Specifically, after acquiring the electroencephalogram (EEG) signal, the server performs time-domain and frequency-domain processing on the EEG signal to obtain corresponding time-domain and frequency-domain features. Then, it determines the first fusion feature, which combines the time-domain and frequency-domain features. The server identifies the embedding vectors contained in the target neural codebook to be generated and performs quantization processing based on the embedding vectors and time-domain features to obtain a time-domain quantized vector, as well as quantization processing based on the embedding vectors and frequency-domain features to obtain a frequency-domain quantized vector, and quantization processing based on the embedding vectors and the first fusion feature to obtain a shared quantized vector. Next, the server performs quantization decoding and reconstruction on the time-domain quantized vectors, frequency-domain quantized vectors, and shared quantized vectors to construct three independent discrete neural codebooks: a dedicated time-domain codebook, a dedicated frequency-domain codebook, and a cross-modal time-frequency shared codebook. This achieves effective decoupling of the time-frequency features of the EEG signal and cross-modal semantic alignment.

[0032] Because dedicated time-domain and frequency-domain codebooks, along with a cross-modal time-frequency shared codebook, form a decoupled discrete representation system, the model can learn modality-specific structures and modality-shared semantics separately. During the reconstruction phase, the dedicated time-domain and frequency-domain codebooks are responsible for recovering modal details, while the time-frequency codebook is responsible for recovering cross-modal semantics. This theoretically avoids the problem of modal contamination found in traditional single-codebook structures, resulting in more stable, purer discrete representations and higher reconstruction fidelity. This structure provides higher-quality, more structured discrete supervision signals for subsequent mask prediction.

[0033] Step 204: Perform mask pre-training based on the target neural codebook to determine the target EEG model.

[0034] In this process, masking pre-training representations randomly mask or replace a portion of the input sequence, allowing the model to predict the masked original content based solely on the remaining context. Then, through large-scale unsupervised training, the system learns deep semantic, syntactic, or spatial relationships. Finally, this set of pre-trained weights is fine-tuned or directly used for downstream tasks.

[0035] Specifically, the server performs a masking operation on the initial features of the EEG signal to obtain masked features, which are then processed by a target encoder to obtain training features. The target encoder includes two encoders with different numbers of layers. The server maps the training features to the corresponding codebook space using a linear classification head to obtain predicted features. The pre-training supervision signal comes from a cross-modal time-frequency shared codebook. The model is trained by analyzing the differences between the predicted features and the time-frequency shared codebook until the target EEG model is determined. Therefore, the target EEG model can be used for emotion recognition, disease diagnosis and physical rehabilitation, and consciousness decoding.

[0036] In the aforementioned EEG signal processing method, by performing time-domain and frequency-domain processing on the EEG signals respectively, a frequency-domain modeling branch is introduced based on the original EEG base model structure, realizing dual-modal collaborative coding in the time and frequency domains, and constructing a target neural codebook containing time-domain dedicated, frequency-domain dedicated, and time-frequency shared codebooks. Then, based on this target neural codebook, mask pre-training is carried out, achieving accurate construction of the target EEG model, providing key support for efficient EEG signal decoding and the upgrading of brain-computer interface technology.

[0037] In one embodiment, time-frequency processing of the EEG signal to obtain a target neural codebook includes: extracting initial features of the EEG signal and embedding the initial features at positions to obtain candidate features; the candidate features include candidate time-domain features and candidate frequency-domain features; processing the candidate features through a first encoder to obtain target time-domain features and target frequency-domain features; and constructing a target neural codebook based on the target time-domain features and target frequency-domain features.

[0038] Specifically, such as Figure 3 As shown, Figure 3 This is a schematic diagram of an architecture for determining a target neural codebook. The input EEG signal is... Where B is the training batch size, C is the number of electrode channels, T is the time in seconds, F is the sampling frequency, and CT is the segment dimension. After the server extracts the initial features of the EEG signal, in order for the model to identify the location of each segment in the time series and its corresponding electrode channel, a learnable temporal location embedding list and a spatial channel embedding list are introduced to perform location embedding on the initial features to obtain candidate features. The first encoder is... Figure 3 The multi-layer Transformer Encoder Blocks are used to process candidate features to obtain target time-domain features and target frequency-domain features. Finally, the target neural codebook can be constructed based on the target time-domain features and target frequency-domain features.

[0039] In this embodiment, by simultaneously constructing two independent deep coding paths in the time and frequency domains, the model can capture two complementary features in the EEG signal in parallel: rapid transient discharges and frequency band energy abnormalities. After spatiotemporal location coding enhancement, the model can identify the topological distribution of discharges at different electrode locations and time periods, achieving structured modeling of typical waveforms of emotions or diseases.

[0040] This dual-stream architecture theoretically significantly enhances the model's ability to cover pathological patterns, enabling the subsequent reconstruction stage to simultaneously restore temporal morphology and spectral structure, reducing feature omissions, and naturally achieving higher classification sensitivity and clinical interpretability during the fine-tuning stage.

[0041] In one embodiment, extracting initial features of the EEG signal includes: slicing the EEG signal into multiple segments, extracting the frequency domain features of the segments through real-valued fast Fourier transform to obtain initial frequency domain features; and extracting features from the segments through a time-domain convolution module to obtain initial time-domain features.

[0042] The initial features include initial temporal features and initial frequency domain features; the temporal convolution module includes one-dimensional convolution, group normalization, and a target activation function, such as the GELU activation function.

[0043] Specifically, refer to Figure 3 As shown, the server segments the raw EEG signal of any number of channels and any length into multiple segments (patches) of fixed duration. Then, it extracts frequency domain features from each segment using real-valued fast Fourier transform to obtain the initial frequency domain features. ,in This represents the frequency domain dimension. For the temporal portion of the segment, a temporal convolution module is used to extract features, obtaining initial temporal features. , where D is the feature embedding dimension.

[0044] In this embodiment, the EEG signal is first processed by segmentation, and then the frequency domain information of the segments is extracted by real-valued fast Fourier transform to generate initial frequency domain features. At the same time, the temporal domain convolution module is used to mine the temporal correlation of the segments to obtain initial temporal features, which lays the feature foundation for the subsequent construction of a multi-dimensional neural codebook.

[0045] In one embodiment, for the initial time-domain features, a learnable list of time location embeddings is introduced as follows:

[0046] , ,

[0047] The list of embedded spatial channels is as follows:

[0048] , ,

[0049] in, It is the maximum number of time segments, ensuring For any temporal segment embedding Add the corresponding temporal and spatial embeddings to the fragment embedding: The embedded candidate temporal features are as follows Similarly, initial frequency domain features Candidate frequency domain features are obtained after feature mapping and location embedding. .

[0050] In one embodiment, constructing a target neural codebook based on target time-domain features and target frequency-domain features includes: integrating target time-domain features and target frequency-domain features to obtain a first fusion feature; determining the embedding vector contained in the target neural codebook to be generated; quantizing the target time-domain features, target frequency-domain features, and the first fusion feature according to the embedding vector to obtain a quantization vector corresponding to the target neural codebook; and decoding and reconstructing multiple quantization vectors to obtain a reconstructed signal corresponding to the target neural codebook.

[0051] The quantization vector includes time-domain quantization vector, frequency-domain quantization vector, and shared quantization vector.

[0052] Specifically, refer to Figure 3 As shown, the server fuses the target's time-domain features and target frequency-domain features to obtain the first fused feature. Each target neural codebook to be generated contains K learnable embedding vectors. Where D represents the embedding dimension. For the target time-domain feature, target frequency-domain feature, and first fused feature, each segment is mapped to the embedding vector with the closest Euclidean distance in the corresponding codebook using a quantizer to achieve the segment quantization process. Similarly, the frequency-domain codebook and the shared codebook use the exact same quantization method to obtain the frequency-domain quantized vector and the shared quantized vector, respectively. Finally, in the decoding and reconstruction stage, the server uses an additive fusion strategy to process multiple quantized vectors to obtain the reconstructed signal corresponding to the target neural codebook, thereby driving self-supervised learning.

[0053] In this embodiment, the target time-domain features and target frequency-domain features are first fused, and then the embedding vector of the target neural codebook is determined. The time-domain features, frequency-domain features, and fused features are then quantized to generate corresponding quantized vectors. Finally, by decoding and reconstructing multiple quantized vectors, the reconstructed signal of the target neural codebook is obtained, providing a solid foundation for the feature encoding and efficient parsing of EEG signals.

[0054] In one embodiment, taking the target temporal features as an example, for any segment representation The quantification process is as follows:

[0055]

[0056] in L2 normalization is equivalent to using cosine similarity to find the nearest neighbor, which can effectively improve codebook utilization and suppress the risk of codeword collapse.

[0057] In one embodiment, the candidate time-domain features and candidate frequency-domain features are extracted and fused by the first encoder, respectively: The first fusion feature is obtained as .

[0058] In one embodiment, decoding and reconstructing multiple quantization vectors to obtain a reconstructed signal corresponding to the target neural codebook includes: additively fusing time-domain quantization vectors and shared quantization vectors, and processing the fused quantization vectors through a time-domain decoder to obtain a reconstructed time-domain signal; additively fusing frequency-domain quantization vectors and shared quantization vectors, and processing the fused quantization vectors through a frequency-domain decoder to obtain the reconstructed signal amplitude spectrum and phase spectrum.

[0059] Specifically, refer to Figure 3 As shown, for the time-domain reconstruction branch, the time-domain quantization vector and the shared quantization vector are added element-wise, and then input into the time-domain decoder to output the reconstructed time-domain signal. For the frequency-domain reconstruction branch, the frequency-domain quantization vector and the shared quantization vector are added element-wise, and then input into the frequency-domain decoder to output the reconstructed signal amplitude spectrum and phase spectrum. Both the time-domain decoder and the frequency-domain decoder can be multi-layer Transformer Encoder Blocks.

[0060] In this embodiment, by additively fusing the time-domain quantization vector, the frequency-domain quantization vector, and the shared quantization vector, and then selectively introducing a time-domain decoder and a frequency-domain decoder for processing, accurate decoding and reconstruction of time-frequency features are achieved, providing a key signal restoration scheme for the efficient construction of the target neural codebook.

[0061] In one embodiment, determining the target EEG model through mask pre-training based on the target neural codebook includes: performing a masking operation on the initial features of the EEG signal according to a pre-determined mask matrix to obtain masked features; obtaining second fusion features based on a second encoder and the masked features; processing the second fusion features through a third encoder to obtain training features; mapping the training features to the corresponding codebook space through a linear classification head to obtain predicted features; and updating parameters based on a loss function and the difference between the predicted features and the time-frequency shared codebook until the target EEG model is determined.

[0062] The mask matrix includes a primary mask and a corresponding inverse mask; the third encoder has half the number of layers as the second encoder. The second encoder is also... Figure 4 Multi-layered Transformer Encoder Blocks.

[0063] Specifically, such as Figure 4 As shown, Figure 4This is a schematic diagram of the mask pre-training architecture. The mask pre-training process based on a time-frequency shared codebook employs a time-frequency dual-stream architecture. After processing the input EEG signal through real-valued Fast Fourier Transform to obtain initial frequency domain features, and through a time-domain convolution module to obtain initial time domain features, both are mapped to a sequence of slice vectors of the same dimension. A random binary mask matrix M is then generated, and masking operations are performed simultaneously on the time-domain and frequency-domain slice vector sequences. Specifically, for positions marked as 1 in the mask matrix, a learnable mask token replaces the original slice vector, resulting in mask features. Next, the mask features are embedded positionally using a temporal position embedding list and a spatial channel embedding list to preserve the temporal and channel geometric information of the sequence. The processed time-domain and frequency-domain sequences are then input into a second encoder for feature extraction, yielding temporal and frequency-domain context features, respectively.

[0064] To achieve deep interaction of cross-modal information, the model performs additive fusion of the extracted dual-stream features. Specifically, the temporal and frequency-domain context features of the two modalities are added element-wise, and the result is normalized through layers to obtain the second fused feature. The second fusion feature is then fed into the third encoder, that is... Figure 4 The shallow Transformer Encoder Blocks in the model aim to further integrate time-frequency semantics with lower computational cost. Then, the training features output by the third encoder are mapped to the corresponding codebook space via a linear classification head, calculating the probability distribution of the masked positions of the masked words belonging to each discrete codeword in the time-frequency shared codebook. Finally, the target EEG model is determined based on the predicted features and with the time-frequency shared codebook as the supervision signal.

[0065] In this embodiment, the cross-modal time-frequency dual-stream masking pre-training mechanism based on a time-frequency shared codebook forces the model to rely on both the temporal and frequency domain contextual structures during the passive prediction of masked segments, thereby compelling the learning of cross-modal global semantic associations. Additive fusion and a shallow encoder further integrate time-frequency semantics, giving the model stronger structural completion capabilities. This training method enables the model to stably infer emotion or disease-related structures even when faced with noise loss, frequency distortion, or sampling anomalies, significantly enhancing the generalization ability for downstream tasks.

[0066] In one embodiment, the target neural codebook is determined by a vector quantization time-frequency prediction model, which includes quantization loss, reconstruction loss, contrastive loss, and consistency loss. The quantization loss includes commitment loss and codebook learning loss, and the consistency loss is determined based on the structure of positive and negative samples constructed by information noise contrastive estimation. The loss function of the target EEG model includes the cross-entropy loss function, and the model training loss includes the prediction loss corresponding to the master mask and the prediction loss corresponding to the inverse mask.

[0067] The overall loss for training the time-frequency prediction model is defined as follows:

[0068]

[0069] Where D represents all EEG signal data and N represents the number of segments; , , Reconstruct the time domain, amplitude spectrum, and phase spectrum of the i-th segment, respectively; , , This corresponds to the original objective; where the quantization loss is... The bidirectional stop-gradient structure is represented as:

[0070]

[0071] In addition, it also includes consistency loss. This loss function is used to constrain the consistency between temporal and frequency domain features in the semantic space. It improves the alignment between the two modalities by normalizing and projecting the temporal and frequency domain features and constructing positive and negative sample pairs based on the Information Noise Contrast Estimation (InfoNCE) mechanism. This enhances the model's ability to capture shared structures of specific emotional or disease discharge patterns across different modalities.

[0072] Therefore, in addition to quantization and reconstruction, cross-modal consistency loss is further introduced, enabling the model to learn the semantic rules of expressing the same emotional structure in both modalities, thereby enhancing the robustness of the model under device noise, changes in acquisition conditions, and individual user differences.

[0073] Furthermore, for mask position i in the mask pre-training, the model's goal is to predict its corresponding shared codebook index. The cross-entropy loss function of the target EEG model is:

[0074]

[0075] in This is the actual index of that position in the shared codebook. Furthermore, to improve data utilization, the server is trained using a symmetric masking strategy. That is, in one iteration, both the primary mask M and its inverse mask are constructed simultaneously. The aforementioned forward propagation process is performed separately, and the loss is calculated. The final total pre-training loss is the sum of the two mask prediction losses, thus ensuring that the model can cover all time-frequency segments of the EEG signal in a single update, achieving efficient and robust feature learning.

[0076] In one embodiment, after mask pre-training based on the target neural codebook, downstream fine-tuning is performed. Downstream task adaptation is based on the pre-trained weights to achieve emotion or disease classification and prediction, emotion classification recognition, etc., such as epilepsy diagnosis, including determining whether it is epilepsy or which subtype of epilepsy it is; in consciousness decoding, it enables reverse inference of the brain's conscious content, cognitive state, or subjective intention, etc. The downstream task fine-tuning framework is as follows: Figure 5 As shown, the overall model architecture is consistent with the mask pre-training stage, except that the classification head changes according to the task, and the pre-trained model weights are loaded during training. Therefore, the overall training of the model in this implementation includes three stages: target neural codebook learning, cross-modal mask pre-training, and task-level fine-tuning, thereby achieving fine modeling and generalization enhancement of different emotional or disease features.

[0077] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0078] Based on the same inventive concept, this application also provides an electroencephalogram (EEG) signal processing device for implementing the aforementioned EEG signal processing method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more EEG signal processing device embodiments provided below can be found in the limitations of the EEG signal processing method described above, and will not be repeated here.

[0079] In one embodiment, an electroencephalogram (EEG) signal processing device is provided, comprising: a neural codebook determination module and a mask pre-training module, wherein:

[0080] The neural codebook determination module is used to perform time-frequency processing on the EEG signal to obtain the target neural codebook; the target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook.

[0081] The mask pre-training module is used to perform mask pre-training based on the target neural codebook to determine the target EEG model.

[0082] The modules in the above-mentioned EEG signal processing can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0083] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores group test results. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a brainwave signal processing method.

[0084] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0086] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0087] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0088] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing electroencephalogram (EEG) signals, characterized in that, The method includes: The EEG signal is processed in time and frequency to obtain the target neural codebook; the target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook; Based on the target neural codebook, mask pre-training is performed to determine the target EEG model; including: According to a predetermined masking matrix, the initial features of the EEG signal are masked to obtain masked features; the masking matrix includes a primary mask and a corresponding inverse mask. The second fusion feature is obtained based on the second encoder and the mask features; The second fused feature is processed by a third encoder to obtain training features; the third encoder has half the number of layers as the second encoder. The training features are mapped to the corresponding codebook space using a linear classification head to obtain the predicted features; Based on the loss function, and according to the difference between the predicted features and the time-frequency shared codebook, the parameters are updated until the target EEG model is determined.

2. The method according to claim 1, characterized in that, The process of performing time-frequency processing on the electroencephalogram (EEG) signal to obtain the target neural codebook includes: Initial features of the EEG signal are extracted, and the initial features are embedded in a position to obtain candidate features; the candidate features include candidate time-domain features and candidate frequency-domain features. The candidate features are processed by the first encoder to obtain the target time-domain features and the target frequency-domain features; Based on the target time-domain features and the target frequency-domain features, a target neural codebook is constructed.

3. The method according to claim 2, characterized in that, The initial features include initial time-domain features and initial frequency-domain features; The initial features extracted from the EEG signal include: The EEG signal is sliced ​​into multiple segments, and the frequency domain features of the segments are extracted by real-valued fast Fourier transform to obtain the initial frequency domain features. The segment is subjected to feature extraction by a temporal convolution module to obtain initial temporal features; the temporal convolution module includes one-dimensional convolution, group normalization and target activation function.

4. The method according to claim 2, characterized in that, The step of constructing the target neural codebook based on the target time-domain features and the target frequency-domain features includes: By combining the target time-domain features and the target frequency-domain features, a first fused feature is obtained; Determine the embedding vectors contained in the target neural codebook to be generated; Based on the embedding vector, the target time-domain features, target frequency-domain features, and first fusion features are quantized respectively to obtain the quantization vector corresponding to the target neural codebook; the quantization vector includes a time-domain quantization vector, a frequency-domain quantization vector, and a shared quantization vector. The quantization vectors are decoded and reconstructed to obtain the reconstructed signal corresponding to the target neural codebook.

5. The method according to claim 4, characterized in that, The process of decoding and reconstructing multiple quantization vectors to obtain the reconstructed signal corresponding to the target neural codebook includes: The time-domain quantization vector and the shared quantization vector are additively fused, and the fused quantization vector is processed by a time-domain decoder to obtain the reconstructed time-domain signal. The frequency domain quantization vector and the shared quantization vector are additively fused, and the fused quantization vector is processed by a frequency domain decoder to obtain the reconstructed signal amplitude spectrum and phase spectrum.

6. The method according to any one of claims 1 to 5, characterized in that, The target neural codebook is determined by a vector quantization time-frequency prediction model, which includes quantization loss, reconstruction loss, contrast loss, and consistency loss. The quantization loss includes commitment loss and codebook learning loss, and the consistency loss is determined based on the structure of positive and negative samples constructed by information noise contrast estimation. The loss function of the target EEG model includes the cross-entropy loss function, and the model training loss includes the prediction loss corresponding to the master mask and the prediction loss corresponding to the inverse mask.

7. A brainwave signal processing device, characterized in that, The apparatus for implementing the method according to any one of claims 1 to 6 comprises: A neural codebook determination module is used to perform time-frequency processing on EEG signals to obtain a target neural codebook; the target neural codebook includes a time-domain codebook, a frequency-domain codebook, and a time-frequency shared codebook. The mask pre-training module is used to perform mask pre-training based on the target neural codebook to determine the target EEG model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images