An electroencephalogram completion method and system based on a class trait and class state double-flow framework

Abstract

The application discloses an electroencephalogram (EEG) completion method and system based on a class trait and class state double-flow framework, which comprises the following steps: obtaining an incomplete EEG sample, a missing mask and an observed EEG matrix; in the training stage, a complete reference sample is also obtained; the sample is normalized along the time dimension sliding window and the statistics are saved; a time, space and space-time trait codebook is constructed to obtain three class trait embeddings; in the class trait flow, the three class embeddings are spliced and fused to generate a fused class trait prior through compression excitation; in the class state flow, a feature extractor is used to obtain complete and incomplete features, a compensation vector is determined based on the difference, and a state completion specific feature is obtained through feature space interpolation; the results of the above two flows are fused to generate a double-flow fusion representation; an input de-stationary attention reconstruction module is used to recover the non-stationary dynamics, and the missing positions are replaced according to the mask to output the completed sequence. The application significantly improves the accuracy and robustness of EEG data completion under various missing modes.

Description

Technical Field

[0001] This invention belongs to the field of EEG signal processing and artificial intelligence technology, and relates to an EEG completion method and system based on a dual-stream framework of class traits and class states. Background Technology

[0002] Electroencephalography (EEG), as a non-invasive, low-cost method for measuring neural activity with high temporal resolution, has wide applications in brain-computer interfaces, neuroscience research, and clinical diagnosis of neurological diseases. By analyzing the potential changes recorded by multiple electrodes distributed on the scalp, specific EEG patterns closely related to cognitive, motor, or pathological states can be identified. However, in actual EEG signal acquisition, due to problems such as poor electrode contact, subject motion artifacts, electromagnetic interference, and data packet loss during wireless transmission, varying degrees of signal loss often occur in the acquired EEG data. This loss disrupts the continuity of EEG signals in both time and space, severely impacting subsequent source analysis, functional connectivity calculations, and deep learning-based pattern recognition tasks.

[0003] Currently, common methods for handling missing EEG signals include interpolation-based methods, matrix factorization-based methods, and deep learning-based methods. Interpolation-based methods, such as linear interpolation or spline interpolation, mainly use values ​​from adjacent time points within the same channel to fill in the missing data, but they do not consider the spatial correlation between different channels, often resulting in poor performance when dealing with continuous block-like missing data. Matrix factorization-based methods treat EEG data as a low-rank matrix, using observed partial values ​​to estimate the missing parts; however, these methods have limited capabilities when dealing with complex nonlinear dynamics. In recent years, some generative models or end-to-end completion networks based on deep learning have been proposed, which achieve missing data completion by learning the distribution of large amounts of complete EEG data. However, these methods often focus on extracting features from the missing samples themselves for reconstruction, ignoring the class-specific information in EEG signals that should remain stable over a long period of time, such as typical rhythmic patterns of specific brain regions over a certain period of time or certain spatial activity patterns shared by different individuals. They also fail to fully consider the non-stationary characteristics of mean and variance in the time dimension that are common in time series data, which limits the generalization ability of the model when faced with missing patterns that have not appeared in the training phase. Summary of the Invention

[0004] The purpose of this invention is to provide an EEG completion method and system based on a dual-stream framework of class traits and class states, in order to solve the above-mentioned problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an EEG completion method based on a dual-stream framework of class traits and class states, comprising the following steps: The process involves acquiring EEG samples with missing values, missing masks, and observed EEG matrices. During the training phase, a complete EEG matrix without missing values ​​is also acquired as a complete reference sample. The EEG samples with missing values ​​are normalized along the time dimension using a sliding window to obtain normalized samples with missing values, and the mean and standard deviation statistics are saved. The complete reference samples are normalized in the same way to obtain normalized complete reference samples. Construct time-feature codebooks, spatial-feature codebooks, and spatiotemporal-feature codebooks to obtain time-feature embeddings, spatial-feature embeddings, and spatiotemporal-feature embeddings; In the trait-based flow, temporal trait embeddings, spatial trait embeddings, and spatiotemporal trait embeddings are concatenated and compressed and fused to generate fused trait-based priors. In the state-based flow, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, feature compensation vectors are determined and feature space interpolation is performed to obtain state-completed partial features. The state completion partial features are fused with the fusion class feature prior to generate a two-stream fusion representation. The two-stream fusion representation is then input into the destationary attention reconstruction module, which generates a scaling factor and an offset vector from the mean statistic, standard deviation statistic, and input sequence, and performs destationary attention reconstruction to output a reconstruction sequence under normalized scale. The reconstructed sequence is denormalized using mean and standard deviation statistics to obtain the model reconstruction sequence. The observed EEG matrix is ​​preserved according to the missing mask, and the corresponding values ​​of the model reconstruction sequence are filled in at the missing positions to output the final completed EEG sequence.

[0006] Furthermore, the acquisition includes missing EEG samples, a missing mask, and an observed EEG matrix; during the training phase, a complete EEG matrix without missing data is also acquired as a complete reference sample, including: The EEG sample is represented as ,in Represents the real number field. Indicates the number of EEG electrodes. The number of time sampling points is indicated, and the observed EEG matrix is ​​determined based on the missing mask M. and EEG input with missing values During the training phase, the complete EEG matrix without missing components is recorded as the complete reference sample. ; This indicates that the EEG value of the i-th EEG electrode at the t-th time sampling point has been observed. This indicates that the corresponding EEG value is missing; the observed EEG matrix is ​​denoted as... Contains missing EEG samples Depend on It consists of preset padding values ​​at the missing locations; during the training phase, it uses EEG samples without introduced missing values. As a complete reference sample, M and M are generated based on preset point-like missing, continuous block-like missing, or channel-like missing patterns. .

[0007] Furthermore, the missing EEG samples are normalized along the time dimension using a sliding window to obtain normalized missing samples, and the mean statistic and standard deviation statistic are saved; the complete reference samples are normalized in the same way to obtain normalized complete reference samples, including: The time normalization is performed according to the following formula: , , ;in, , ⊙ represents element-wise multiplication. This indicates element-wise division.

[0008] Furthermore, the construction of the temporal feature codebook, spatial feature codebook, and spatiotemporal feature codebook to obtain the temporal feature embedding, spatial feature embedding, and spatiotemporal feature embedding includes: The time-related trait codebook The spatial class trait codebook is used to represent persistent time patterns that remain stable across time periods. The spatiotemporal trait codebook is used to represent stable and persistent spatial patterns between different brain regions. Used to represent persistent spatiotemporal patterns determined by both time location and EEG electrode location.

[0009] The general mapping of a learnable codebook is as follows The time-related trait codebook is represented as follows: The spatial class trait codebook is represented as follows: Each spatiotemporal identifier in the spatiotemporal characteristic codebook Corresponding learnable value .

[0010] Furthermore, in the trait-like flow, the temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding are concatenated and compressed, and then fused to generate a fused trait-like prior, including: In the trait flow, , and splicing into multi-view feature embedding Furthermore, prior fusion-type traits are obtained through compression-incentive fusion. ; , , ;in, and For learnable weights, This represents the Sigmoid function.

[0011] Furthermore, in the state-like flow, normalized complete reference samples and normalized samples with missing values ​​are input into the EEG signal feature extractor to obtain complete reference features and features with missing values. Based on the difference between the two, a feature compensation vector is determined and feature space interpolation is performed to obtain state-complete partial features, including: Establish a class-state flow parallel to the class-trait flow, and normalize the complete reference sample. and normalized samples with missing data Inputting the EEG signal feature extractor Φ yields complete reference features. and features containing missing characteristics ,according to and The feature differences between them determine the feature compensation vector. Then, the feature compensation vector is used to perform feature space interpolation on the actual EEG state features to obtain state-completed partial features. ; The EEG feature extractor Φ is an encoder based on a dynamic graph convolutional neural network. ; and They are respectively and Mean feature embedding, .

[0012] Furthermore, the state completion partial features are fused with the fusion class feature prior to generate a two-stream fusion representation; the two-stream fusion representation is input into the destationary attention reconstruction module, which generates a scaling factor and offset vector from the mean statistic, standard deviation statistic, and input sequence, and performs destationary attention reconstruction to output a reconstructed sequence at a normalized scale, including: Interpolation features of state-like flow output Fusion of trait priors with trait stream output The two streams are fused to obtain the fused representation. The fusion represents according to

[0013] Obtain, among which Represents the interpolation features in the EEG feature space. This indicates a priori traits of the fusion class.

[0014] The positive scaling scalar and offset vector according to Obtain; to calm attention according to Calculate; where 1 represents a completely one vector. This represents the characteristic dimension of the key matrix.

[0015] Furthermore, the reconstructed sequence is denormalized using mean and standard deviation statistics to obtain the model reconstructed sequence. The observed EEG matrix is ​​preserved based on the missing mask, and the corresponding values ​​from the model reconstructed sequence are filled into the missing positions. The final completed EEG sequence is then output, including: Anti-normalization according to To proceed; among them, The final completed matrix is ​​according to Obtain, and Corresponding model output ; Mean squared error loss is used during the training phase: ;in, and These represent the actual EEG values ​​and the reconstructed EEG values, respectively. Save the time-based trait codebook after training. Spatial trait codebook and spacetime-related trait codebook During the completion phase, it is obtained by searching the codebook. , and .

[0016] Secondly, the present invention provides an EEG completion system based on a dual-stream framework of class traits and class states, comprising: The sample acquisition module is used to acquire EEG samples with missing values, missing masks, and observed EEG matrices. During the training phase, the complete EEG matrix without missing values ​​is also acquired as a complete reference sample. The EEG samples with missing values ​​are normalized along the time dimension using a sliding window to obtain normalized samples with missing values, and the mean statistic and standard deviation statistic are saved. The complete reference sample is normalized in the same way to obtain a normalized complete reference sample. The embedding module is used to construct the temporal trait codebook, spatial trait codebook, and spatiotemporal trait codebook, and obtain the temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding; The processing module is used to concatenate and compress the temporal feature embedding, spatial feature embedding, and spatiotemporal feature embedding in the trait-like stream, and fuse them to generate fused trait-like priors; in the state-like stream, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, the feature compensation vector is determined and feature space interpolation is performed to obtain state-complete partial features. The fusion module is used to fuse the state completion partial features with the fusion class prior to generate a two-stream fusion representation. The two-stream fusion representation is input into the destationary attention reconstruction module, which generates a scaling factor and an offset vector from the mean statistic, standard deviation statistic and input sequence and performs destationary attention reconstruction to output a reconstruction sequence under normalized scale. The output module is used to denormalize the reconstructed sequence using mean and standard deviation statistics to obtain the model reconstructed sequence. It retains the observed EEG matrix according to the missing mask and fills the missing positions with the corresponding values ​​of the model reconstructed sequence, and outputs the final completed EEG sequence.

[0017] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the EEG completion method based on a dual-stream framework of class traits and class states.

[0018] Compared with the prior art, the present invention has the following technical effects: This invention employs sliding window normalization to process EEG samples along the time dimension and preserves mean and standard deviation statistics. This effectively reduces the interference of the non-stationary characteristics of EEG signals in the time dimension on model training, allowing subsequent modules to learn the inherent dynamic patterns of the signals more stably at a stationary scale. By constructing temporal, spatial, and spatiotemporal characteristic codebooks, it can explicitly capture and store stable class characteristic information widely existing across samples in EEG signals from three perspectives: time, space, and spatiotemporal joint, providing strong prior knowledge for missing data completion. By fusing the above three types of embedded compression excitations in the class trait stream, class trait information from different sources can be dynamically weighted, highlighting the priors most useful for the current reconstruction task and avoiding redundancy caused by simple splicing; by using an EEG signal feature extractor to extract complete reference features and missing features in the class state stream and calculating feature compensation vectors for interpolation, the distribution information of complete data can be used to perform guided correction of the feature space of missing samples, making up for the feature distribution offset caused by the missing data. By fusing state completion features with prior fusion class traits to generate a dual-stream fusion representation, a deep integration of the dynamic change information of the current instance and the stable pattern knowledge of the category is achieved. This allows the reconstruction process to respect the actual state of the individual while following general pattern rules. By inputting the dual-stream fusion representation into the destationary attention reconstruction module and generating scaling factors and offset vectors from the saved mean and standard deviation statistics to regulate the attention mechanism, the weakened non-stationary information after normalization can be consciously restored, making the reconstructed sequence more closely resemble the non-stationary fluctuation characteristics of real EEG signals. Finally, through inverse normalization and local replacement based on missing masks, the true values ​​of the observed regions are not modified, and only the missing regions are filled with high precision. The overall scheme significantly improves the completion accuracy and model robustness under various complex missing modes such as point-like missing, continuous block-like missing, and even channel missing, and is suitable for applications such as brain-computer interfaces, neurofeedback training, and clinical auxiliary diagnosis that require high-quality EEG signals. Attached Figure Description

[0019] Figure 1 This is a flowchart of an EEG completion method based on a dual-stream framework of class traits and class states, according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] The present invention will now be described in further detail with reference to the accompanying drawings: Example 1, see Figure 1In one embodiment of the present invention, an EEG completion method based on a dual-stream framework of class traits and class states is provided to solve the problem of signal loss caused by poor electrode contact, motion artifacts or wireless transmission packet loss during EEG signal acquisition, and to achieve high-precision completion of missing EEG signals.

[0023] Specifically, the EEG completion method based on the dual-stream framework of class traits and class states of this invention includes the following steps: S1: Obtain missing EEG samples, missing masks, and observed EEG matrices; during the training phase, a complete EEG matrix without missing data is also obtained as a complete reference sample; the missing EEG samples are normalized along the time dimension using a sliding window to obtain normalized missing samples, and the mean and standard deviation statistics are saved; the complete reference samples are normalized in the same way to obtain normalized complete reference samples.

[0024] Electroencephalogram (EEG) signals are a typical non-stationary time series, with their statistical properties constantly changing over time. If this change is directly input into a deep learning model without processing, the model will struggle to learn stable intrinsic patterns from drastically fluctuating numerical ranges, especially when data is missing. Therefore, this step employs a sliding window normalization strategy along the time dimension. Specifically, firstly, missing EEG samples need to be acquired. These samples are two-dimensional matrices composed of electrophysiological data recorded at continuous time sampling points by multiple EEG electrode channels. The missing mask is a binary matrix of the same size as the EEG sample, used to clearly identify whether each element is a true measurement or a missing location. The observed EEG matrix is ​​obtained by filling the missing locations in the missing EEG samples with zero values ​​or other preset padding values ​​to facilitate numerical calculations by the model. During the model training phase, a complete EEG matrix of the same size but without any missing data processing is also acquired as a complete reference sample. This complete reference sample will serve as both part of the network input and a supervision label when calculating the training loss function. The sliding window normalization process involves setting a fixed-length window and a step size, sliding it from the starting point to the ending point on the time axis. At each window position, the mean and standard deviation of the data at all time points within that window are calculated. Then, the calculated mean and standard deviation are used to standardize the data within that window, making the data distribution within that local time window approximately equal to a mean of zero and a standard deviation of one. The mean calculated for each window constitutes the mean statistic for the entire sequence, and the standard deviation constitutes the standard deviation statistic for the entire sequence. These statistics are not only used to complete the current normalization operation but are also carefully stored, as they need to be reused in the subsequent destationary attention reconstruction stage and the final denormalization operation to restore the non-stationary dynamic characteristics of the original EEG data. For example, for an EEG sample containing 64 EEG electrodes and 500 time sampling points, the window length can be set to the number of sampling points corresponding to one second, and the step size to half the window length, and the above normalization calculations can be performed sequentially.

[0025] S2: Construct time trait codebook, spatial trait codebook, and spatiotemporal trait codebook to obtain time trait embedding, spatial trait embedding, and spatiotemporal trait embedding.

[0026] Although the EEG signals of this invention vary greatly depending on the individual, state, and time, certain stable trait patterns exist across samples and time. These patterns can be categorized into temporal traits, spatial traits, and spatiotemporal joint traits. Temporal traits refer to persistent fluctuation patterns that recur in the time-series evolution of certain EEG rhythms, such as the typical decay curve of alpha waves after the main peak or the waveform morphology of specific components of event-related potentials that are time-locked with cognitive processing. Spatial traits refer to stable potential distribution patterns that remain stable between different brain regions on the scalp surface, such as the inherent activation or inhibition relationships between certain electrodes during specific tasks. Spatiotemporal traits are more refined persistent patterns determined by specific time points and specific electrode locations. All three codebooks are constructed using discrete vector quantization, with each codebook containing a finite number of embedding vectors. In the initial stage of the model, these embedding vectors are randomly initialized. During training, the model searches for the best-matching embedding vector from each codebook based on the features of the input EEG samples containing missing data, using it as the trait representation from that perspective. For example, for a specific EEG segment, the temporal feature codebook might find an embedding vector representing the typical temporal pattern of "slow rise followed by rapid fall," while the spatial feature codebook might find an embedding vector representing the spatial topological pattern of "frontal lobe activity and occipital lobe inhibition."

[0027] S3: In the trait-based flow, temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding are concatenated and compressed and fused to generate fused trait-based priors; in the state-based flow, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, feature compensation vectors are determined and feature space interpolation is performed to obtain state-completed partial features.

[0028] In this step, the trait-based flow and the state-based flow operate in parallel, processing two different types of information. The trait-based flow processes stable trait information from three perspectives: temporal trait embeddings, spatial trait embeddings, and spatiotemporal trait embeddings, retrieved from the codebook. These three embedding vectors are obtained from their respective independent codebook spaces, and their emphasis and dimensions may differ. This step concatenates them along a specific dimension to form a larger-dimensional multi-perspective trait embedding that integrates temporal, spatial, and spatiotemporal joint prior knowledge. This step further performs a compression-stimulus fusion operation on the concatenated multi-perspective embedding. The compression operation uses global average pooling to compress spatial or temporal dimension information into a global description, while the stimulation operation uses two fully connected layers and activation functions to learn the weights for each channel dimension. This weight vector represents the model's adaptive filtering and emphasis on trait information from different perspectives. Multiplying the weights by the original multi-perspective embedding generates a weighted fused trait prior. Meanwhile, the state-based flow processes the instantiation state information of the current specific sample. Normalized complete reference samples and normalized samples with missing information are simultaneously fed into a parameter-shared EEG signal feature extractor. This feature extractor is typically a deep network capable of capturing the complex spatiotemporal relationships of EEG data, such as an encoder based on a dynamic graph convolutional neural network. The feature extractor outputs complete reference features and features with missing information, respectively. Due to the missing information in the samples, the distribution of the extracted features with missing information in the feature space often deviates from the distribution of the complete reference features. This step determines a feature compensation vector by calculating the difference between these two features. This vector essentially indicates the direction and distance that needs to be moved in the feature space from the missing features to the complete features. Then, this compensation vector is used to interpolate the features with missing information in the feature space, bringing them closer to the distribution of the complete features, thus obtaining state-complete partial features.

[0029] S4: Fuse the state completion partial features with the fusion class feature prior to generate a two-stream fusion representation; input the two-stream fusion representation into the destationary attention reconstruction module, generate scaling factor and offset vector from the mean statistic, standard deviation statistic and input sequence and perform destationary attention reconstruction, outputting the reconstruction sequence under normalized scale.

[0030] Following the previous step, the state-like flow has performed preliminary interpolation and correction on the dynamic features of the current sample, forming a state-completed partial feature in the feature space; the trait-like flow has also generated a fusion trait prior that has general guiding significance for the current sample category. This step fuses these two pieces of information from different sources and with different properties, simply splicing them together to form a two-stream fusion representation. Such a representation is richer and more reliable than inference based solely on missing samples or solely on codebook generation. Subsequently, this two-stream fusion representation is input into the destationary attention reconstruction module for final sequence generation. In step S1, the data is manually destationary to facilitate stable learning. However, during reconstruction, if only an overly stationary sequence at the normalized scale is generated, when denormalized back to the original scale, it will exhibit unnatural smoothness, losing the rich non-stationary dynamics of real EEG signals, such as the time-varying or sudden fluctuations in energy of certain bands. Therefore, the destationary attention reconstruction module is designed to consciously restore the previously weakened non-stationarity. The module receives the mean and standard deviation statistics saved in step S1, along with the current input sequence itself, and learns to generate scaling factors and offset vectors through a multilayer perceptron. During the calculation of the attention score, these scaling factors and offset vectors are applied in a specific way. Simply put, the scaling factor can enhance the attention response at certain time steps or channels, and the offset vector can change the center of the attention distribution. This mechanism means that attention calculation is no longer performed under the assumption of complete stationarity, but rather adjusted according to the statistical fluctuations of the original data. This allows the reconstruction module to output a reconstructed sequence that, although at a normalized scale, already implicitly contains the original non-stationary dynamic information.

[0031] S5: Use mean and standard deviation statistics to denormalize the reconstructed sequence to obtain the model reconstructed sequence. Based on the missing mask, retain the observed EEG matrix and fill in the corresponding values ​​of the model reconstructed sequence at the missing positions, and output the final completed EEG sequence.

[0032] The normalization of the input data in step S1 is for the convenience of model learning. Now that the model has learned and outputs a reconstructed sequence at the normalized scale, it must be restored to the true physical value range of the original EEG signal through an inverse normalization operation. Specifically, the inverse normalization operation uses the mean and standard deviation statistics, also saved from step S1, to perform scale restoration on each element of the reconstructed sequence. After inverse normalization, a model reconstructed sequence with actual microvolt dimensions and non-stationary dynamic characteristics is obtained. This step utilizes the missing mask from step S1 to perform a selective replacement operation: for positions marked as observed in the missing mask, the final output value is the original true value in the observed EEG matrix; for positions marked as missing in the missing mask, the final output value is the generated value for the corresponding position in the model reconstructed sequence. Through this step, the model's output is integrated into the original observation data framework, repairing missing points without altering the true record, forming a final complete EEG sequence. During the training phase, the entire model aims to minimize the mean squared error between the model's reconstructed sequence and the complete real EEG sequence. It updates all learnable parameters through backpropagation, including all codebook terms, feature extractor weights, and attention module weights. After training, three learnable codebooks from the trait stream are saved and used directly in subsequent inference application phases.

[0033] This invention presents an EEG completion method based on a dual-stream framework of class traits and class states. First, it introduces sliding window normalization to extract and preserve non-stationary statistics, constructing a stable and recoverable learning environment for the model. Then, through a parallel temporal, spatial, and spatiotemporal three-codebook mechanism and compressed stimulus fusion, it systematically captures and dynamically filters class-trait priors of EEG signals. Simultaneously, a dedicated class-state stream corrects the features of missing samples through a feature compensation mechanism. Deep fusion of the two streams enables the reconstruction process to possess both global pattern guidance and local instance adaptation capabilities. The destationary attention design specifically addresses the dynamic loss problem caused by normalization. Finally, a selective replacement strategy based on a missing mask ensures the absolute authenticity of the observed data. The entire method forms a complete closed loop from low-level preprocessing, mid-level dual-stream representation learning to high-level non-stationary reconstruction. Each step works closely together, significantly improving completion accuracy and model robustness under various extreme missing conditions, such as point-like missing data, continuous block-like missing data, and channel-like missing data.

[0034] Example 2: This invention provides an EEG completion method based on a dual-stream framework of class traits and class states, comprising the following steps: S1: Obtain EEG samples and corresponding missing masks, wherein the EEG samples are represented as follows: ,in Represents the real number field. Indicates the number of EEG electrodes. The number of time sampling points is indicated, and the observed EEG matrix is ​​determined based on the missing mask M. and EEG input with missing values During the training phase, the complete EEG matrix without missing components is recorded as the complete reference sample. ; S2: Input to EEG with missing values Sliding window normalization along the time dimension yields normalized samples with missing values. And save the mean statistic. Sum and standard deviation statistics During the training phase, the complete reference sample is used. Normalize using the same method to obtain a normalized complete reference sample. ; S3: Constructing a temporal trait codebook through a multi-perspective trait modeling module Spatial Characteristics Codebook and spacetime characteristic codebook Time-related embeddings were obtained respectively. Spatial characteristics embedded Embedded with spatiotemporal characteristics ; S4: In the trait flow, , and splicing into multi-view feature embedding Furthermore, prior fusion-type traits are obtained through compression-incentive fusion. ; S5: Establish a class-state flow that runs parallel to the class-trait flow, and normalize the complete reference sample. and normalized samples with missing data Inputting the EEG signal feature extractor Φ yields complete reference features. and features containing missing characteristics ,according to and The feature differences between them determine the feature compensation vector. Then, the feature compensation vector is used to perform feature space interpolation on the actual EEG state features to obtain state-completed partial features. ; S6: Interpolation features of the state-like stream output Fusion of trait priors with trait stream output The two streams are fused to obtain the fused representation. ; S7: Represent the fusion of two streams Input to the de-stationary attention reconstruction module, based on mean statistics Standard deviation statistic Generate scaling factor τ and offset vector from the input sequence. It restores the weakened non-stationary dynamics during the normalization process and outputs the reconstructed sequence at the normalized scale. ; S8: Using the mean statistic Sum and standard deviation statistics For reconstructed sequences Perform inverse normalization to obtain the model reconstruction sequence. Preserve the observed EEG matrix based on the missing mask M. The missing positions were then replaced to obtain the final complete EEG sequence. During the training phase, mean squared error loss (MSE) is used to optimize the model parameters, and the learnable codebook in the class trait stream is saved after training is completed.

[0035] In step S1, This indicates that the EEG value of the i-th EEG electrode at the t-th time sampling point has been observed. This indicates that the corresponding EEG value is missing; the observed EEG matrix is ​​denoted as... Contains missing EEG samples Depend on It consists of preset padding values ​​at the missing locations; during the training phase, it uses EEG samples without introduced missing values. As a complete reference sample, M and M are generated based on preset point-like missing, continuous block-like missing, or channel-like missing patterns. .

[0036] In step S2, the time normalization is performed according to the following formula: , , ;in, , ⊙ represents element-wise multiplication. This indicates element-wise division.

[0037] In step S3, the time-type feature codebook The spatial class trait codebook is used to represent persistent time patterns that remain stable across time periods. The spatiotemporal trait codebook is used to represent stable and persistent spatial patterns between different brain regions. Used to represent persistent spatiotemporal patterns determined by both time location and EEG electrode location.

[0038] The general mapping of a learnable codebook is as follows The time-related trait codebook is represented as follows: The spatial class trait codebook is represented as follows: Each spatiotemporal identifier in the spatiotemporal characteristic codebook Corresponding learnable value .

[0039] In step S4, , , ;in, and For learnable weights, This represents the Sigmoid function.

[0040] In step S5, the EEG feature extractor Φ is an encoder based on a dynamic graph convolutional neural network. ; and They are respectively and Mean feature embedding, .

[0041] In step S6, the fusion representation is according to

[0042] Obtain, among which Represents the interpolation features in the EEG feature space. This indicates a priori traits of the fusion class.

[0043] In step S7, the positive scaling scalar and offset vector according to Obtain; to calm attention according to Calculate; where 1 represents a completely one vector. This represents the characteristic dimension of the key matrix.

[0044] In step S8, the inverse normalization is performed according to... To proceed; among them, The final completed matrix is ​​according to Obtain, and Corresponding model output .

[0045] Mean squared error loss is used during the training phase: ;in, and These represent the actual EEG value and the reconstructed EEG value, respectively.

[0046] Save the time-based trait codebook after training. Spatial trait codebook and spacetime-related trait codebook During the completion phase, it is obtained by searching the codebook. , and .

[0047] Example 3: Construction of EEG Samples and Deletion Masks Obtaining EEG Matrix Where R represents the real number field, N represents the number of EEG electrodes, and T represents the time step. A missing mask is generated based on the acquisition device status, channel quality score, artifact detection results, or artificially constructed missing patterns. The observation EEG matrix is ​​determined by M. and EEG input with missing values During the training phase, the complete EEG matrix without missing components is recorded as the complete reference sample. When the data of the i-th electrode is observable at the t-th time step, 1; When data at this location is missing, 0.

[0048]

[0049] in Represents the model reconstruction sequence. This indicates the final completion of the EEG sequence. This method only replaces the missing locations with the model output, while retaining the original acquired values ​​at the observation locations.

[0050] Time normalization To reduce amplitude shift and distribution drift between different EEG segments, a sliding window was set along the time axis and the mean statistic was calculated. Sum and standard deviation statistics The normalization process can be represented as:

[0051] in Indicates missing EEG input The EEG vector at time step t, This represents the normalized sample containing missing data; the training phase also includes the complete reference sample. Normalized in the same way to obtain Normalized input is beneficial for class-specific codebooks and reconstructing stable features learned by the network, but in subsequent steps, this invention restores the weakened class-state changes by de-stabilizing attention and denormalizing.

[0052] Multi-perspective trait modeling A multi-view learnable codebook is established based on normalized EEG signals. The general mapping of the learnable codebook is CB:{k→v|k∈K,v∈V}, where each key Corresponding value set A learnable tensor The codebook is randomly initialized and trained jointly with the remaining parameters of the model. After training, it is saved.

[0053] ; C

[0054] Time-related trait codebook Encode the trial periodicity PoT and the trial characteristics ToS of the same subject to obtain ,in Spatial Trait Codebook C Encoding hemispherical and whole-brain persistent spatial patterns yields ,in Spacetime-type trait codebook C Assign a spatiotemporal identifier to each data point and corresponding values ,get .

[0055] Compression excitation fusion The three persistence modes are concatenated along the feature embedding dimension. and first of all Perform global average pooling:

[0056]

[0057] in, and For learnable weights, for The function is S, where S is the importance score. After linear transformation and compression, the output is fused with the output of the noise robust feature interpolation module.

[0058] Noise Robust Feature Interpolation An EEG feature extractor Φ based on a dynamic graph convolutional neural network is used to normalize complete EEG segments. and normalized incomplete EEG fragments Mapping to the EEG feature space:

[0059] Calculate the mean feature embedding of the two sets of features and The feature space attribute vector is defined by the difference between the two: ,

[0060] Where d represents the average semantic offset required to shift a portion of the observed EEG to the complete EEG distribution, and α controls the interpolation intensity. This represents the interpolation features in the EEG feature space.

[0061] Class traits and class states fusion interpolation features With persistent multi-view codebook features By splicing them together, we get:

[0062] in, It also includes state-related features of the current incomplete EEG fragments and persistent class trait priors provided by a multi-view learnable codebook, which serve as inputs to the non-stationary perception reconstruction module.

[0063] Non-stationary perception reconstruction To mitigate the effects of excessive stationarization caused by sequence stationarization, based on the non-stationarized input sequence x and the mean statistic... Sum and standard deviation statistics Learning positive scaling scalars through multilayer perceptron and offset vector :

[0064] To stabilize attention, calculate using the following formula: Attn(Q,K,V,τ,Δ)=Softmax((τQK^T+1Δ^T) / √d_k)V Where Q, K, and V are the query matrix, key matrix, and value matrix obtained from the stationary representation, and 1 is a one-dimensional vector. The feature dimension of the key matrix. Attention output is processed by a linear output reconstruction module to obtain a reconstructed EEG fragment at a normalized scale. .

[0065] Inverse normalization and model training set up Utilizing the data saved during normalization and Restore it to the original EEG signal scale:

[0066] Therefore, we obtain During the training phase, mean squared error is used as the loss function.

[0067] in, and These represent the true value and the reconstructed value, respectively. After training, the time-based trait codebook... Spatial trait codebook and spacetime-related trait codebook It is saved and retrieved during the completion phase via a lookup method; the state-aware reconstruction module processes each EEG segment according to the neural network feedforward process.

[0068] Application scenarios This invention can be applied to scenarios such as online preprocessing of brain-computer interfaces, wearable EEG data restoration, emotion recognition data cleaning, attention monitoring, motor imagery EEG completion, clinical EEG-assisted analysis, and multimodal neural signal modeling. For point-like missing data, continuous segment missing data, and channel-level missing data, this invention can perform unified completion based on the missing data mask.

[0069] In another embodiment of the present invention, an EEG completion system based on a dual-stream framework of class traits and class states is provided, which can be used to implement the above-mentioned EEG completion method based on a dual-stream framework of class traits and class states. Specifically, the EEG completion system based on a dual-stream framework of class traits and class states includes: The sample acquisition module is used to acquire EEG samples with missing values, missing masks, and observed EEG matrices. During the training phase, a complete EEG matrix without missing values ​​is also acquired as a complete reference sample. The EEG samples with missing values ​​are normalized along the time dimension using a sliding window to obtain normalized samples with missing values, and the mean and standard deviation statistics are saved. The complete reference samples are normalized in the same way to obtain normalized complete reference samples.

[0070] The embedding module is used to construct the temporal trait codebook, spatial trait codebook, and spatiotemporal trait codebook, and obtain the temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding.

[0071] The processing module is used to concatenate and compress excitations and fuse temporal feature embeddings, spatial feature embeddings, and spatiotemporal feature embeddings in the trait-like stream to generate fused trait-like priors; in the state-like stream, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, the feature compensation vector is determined and feature space interpolation is performed to obtain state-complete partial features.

[0072] The fusion module is used to fuse the state completion partial features with the fusion class prior to generate a two-stream fusion representation. The two-stream fusion representation is input into the destationary attention reconstruction module, which generates a scaling factor and an offset vector from the mean statistic, standard deviation statistic and input sequence and performs destationary attention reconstruction, outputting a reconstructed sequence under normalized scale.

[0073] The output module is used to denormalize the reconstructed sequence using mean and standard deviation statistics to obtain the model reconstructed sequence. It retains the observed EEG matrix according to the missing mask and fills the missing positions with the corresponding values ​​of the model reconstructed sequence, and outputs the final completed EEG sequence.

[0074] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0075] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of an EEG completion method based on a dual-stream framework of class traits and class states.

[0076] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the EEG completion method based on a class-trait and class-state dual-stream framework in the above embodiments.

[0077] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0079] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A brainwave completion method based on a dual-stream framework of class traits and class states, characterized in that, Includes the following steps: Obtain missing EEG samples, missing masks, and observed EEG matrices; During the training phase, a complete EEG matrix without missing components was also obtained as a complete reference sample. The missing EEG samples were normalized along the time dimension using a sliding window to obtain normalized missing samples, and the mean statistic and standard deviation statistic were saved. Normalized complete reference samples are obtained by normalizing the complete reference samples in the same way; Construct time-feature codebooks, spatial-feature codebooks, and spatiotemporal-feature codebooks to obtain time-feature embeddings, spatial-feature embeddings, and spatiotemporal-feature embeddings; In the trait-based flow, temporal trait embeddings, spatial trait embeddings, and spatiotemporal trait embeddings are concatenated and compressed and fused to generate fused trait-based priors. In the state-based flow, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, feature compensation vectors are determined and feature space interpolation is performed to obtain state-completed partial features. The state completion partial features are fused with the fusion class feature prior to generate a two-stream fusion representation. The two-stream fusion representation is then input into the destationary attention reconstruction module, which generates a scaling factor and an offset vector from the mean statistic, standard deviation statistic, and input sequence, and performs destationary attention reconstruction to output a reconstruction sequence under normalized scale. The reconstructed sequence is denormalized using mean and standard deviation statistics to obtain the model reconstruction sequence. The observed EEG matrix is ​​preserved according to the missing mask, and the corresponding values ​​of the model reconstruction sequence are filled in at the missing positions to output the final completed EEG sequence.

2. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, The process involves acquiring missing EEG samples, a missing mask, and an observed EEG matrix. During the training phase, a complete EEG matrix without missing components was also acquired as a complete reference sample, including: The EEG sample is represented as ,in Represents the real number field. Indicates the number of EEG electrodes. The number of time sampling points is indicated, and the observed EEG matrix is ​​determined based on the missing mask M. and EEG input with missing values During the training phase, the complete EEG matrix without missing components is recorded as the complete reference sample. ; This indicates that the EEG value of the i-th EEG electrode at the t-th time sampling point has been observed. This indicates that the corresponding EEG value is missing; the observed EEG matrix is ​​denoted as... Contains missing EEG samples Depend on It consists of preset padding values ​​at the missing locations; during the training phase, it uses EEG samples without introduced missing values. As a complete reference sample, M and M are generated based on preset point-like missing, continuous block-like missing, or channel-like missing patterns. .

3. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, The missing EEG samples are normalized along the time dimension using a sliding window to obtain normalized missing samples, and the mean statistic and standard deviation statistic are saved. Normalized complete reference samples are obtained by normalizing the complete reference samples in the same way, including: The time normalization is performed according to the following formula: , , ;in, , ⊙ represents element-wise multiplication. This indicates element-wise division.

4. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, The construction of the temporal feature codebook, spatial feature codebook, and spatiotemporal feature codebook yields temporal feature embeddings, spatial feature embeddings, and spatiotemporal feature embeddings, including: The time-related trait codebook The spatial class trait codebook is used to represent persistent time patterns that remain stable across time periods. The spatiotemporal trait codebook is used to represent stable and persistent spatial patterns between different brain regions. Used to represent persistent spatiotemporal patterns determined by both temporal location and EEG electrode location; The general mapping of a learnable codebook is as follows The time-related trait codebook is represented as follows: The spatial class trait codebook is represented as follows: Each spatiotemporal identifier in the spatiotemporal characteristic codebook Corresponding learnable value .

5. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, In the trait-like flow, the temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding are concatenated and compressed, and then fused to generate a fused trait-like prior, including: In the trait flow, , and splicing into multi-view feature embedding Furthermore, prior fusion-type traits are obtained through compression-incentive fusion. ; , , ;in, and For learnable weights, This represents the Sigmoid function.

6. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, In the state-like flow, normalized complete reference samples and normalized samples with missing values ​​are input into the EEG signal feature extractor to obtain complete reference features and features with missing values. Based on the difference between the two, a feature compensation vector is determined and feature space interpolation is performed to obtain state-completed partial features, including: Establish a class-state flow parallel to the class-trait flow, and normalize the complete reference sample. and normalized samples with missing data Inputting the EEG signal feature extractor Φ yields complete reference features. and features containing missing characteristics ,according to and The feature differences between them determine the feature compensation vector. Then, the feature compensation vector is used to perform feature space interpolation on the actual EEG state features to obtain state-completed partial features. ; The EEG feature extractor Φ is an encoder based on a dynamic graph convolutional neural network. ; and They are respectively and Mean feature embedding, .

7. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, The process involves fusing state completion partial features with fusion class priors to generate a two-stream fusion representation. This two-stream fusion representation is then input into a destationary attention reconstruction module, where the mean statistic, standard deviation statistic, and input sequence are used to generate a scaling factor and offset vector. Destationary attention reconstruction is then performed, outputting a reconstructed sequence at a normalized scale, including: Interpolation features of state-like flow output Fusion of trait priors with trait stream output The two streams are fused to obtain the fused representation. The fusion represents according to Obtain, among which Represents the interpolation features in the EEG feature space. Indicates the prior characteristics of fusion-type traits; The positive scaling scalar and offset vector according to Obtain; to calm attention according to Calculate; where 1 represents a completely one vector. This represents the characteristic dimension of the key matrix.

8. The EEG completion method based on a dual-stream framework of class traits and class states according to claim 1, characterized in that, The reconstructed sequence is denormalized using mean and standard deviation statistics to obtain the model reconstructed sequence. The observed EEG matrix is ​​preserved based on the missing data mask, and the corresponding values ​​from the model reconstructed sequence are filled into the missing positions. The final completed EEG sequence is then output, including: Anti-normalization according to To proceed; among them, The final completed matrix is ​​according to Obtain, and Corresponding model output ; Mean squared error loss is used during the training phase: ;in, and These represent the actual EEG values ​​and the reconstructed EEG values, respectively. Save the time-based trait codebook after training. Spatial trait codebook and spacetime-related trait codebook During the completion phase, it is obtained by searching the codebook. , and .

9. An EEG completion system based on a dual-stream framework of class traits and class states, characterized in that, include: The sample acquisition module is used to acquire EEG samples containing missing values, missing masks, and observed EEG matrices. During the training phase, a complete EEG matrix without missing components was also obtained as a complete reference sample. The missing EEG samples were normalized along the time dimension using a sliding window to obtain normalized missing samples, and the mean statistic and standard deviation statistic were saved. Normalized complete reference samples are obtained by normalizing the complete reference samples in the same way; The embedding module is used to construct the temporal trait codebook, spatial trait codebook, and spatiotemporal trait codebook, and obtain the temporal trait embedding, spatial trait embedding, and spatiotemporal trait embedding; The processing module is used to concatenate and compress the temporal feature embedding, spatial feature embedding, and spatiotemporal feature embedding in the trait-like stream, and fuse them to generate fused trait-like priors; in the state-like stream, normalized complete reference samples and normalized samples with missing features are input into the EEG signal feature extractor to obtain complete reference features and features with missing features. Based on the difference between the two, the feature compensation vector is determined and feature space interpolation is performed to obtain state-complete partial features. The fusion module is used to fuse the state completion partial features with the fusion class prior to generate a two-stream fusion representation. The two-stream fusion representation is input into the destationary attention reconstruction module, which generates a scaling factor and an offset vector from the mean statistic, standard deviation statistic and input sequence and performs destationary attention reconstruction to output a reconstruction sequence under normalized scale. The output module is used to denormalize the reconstructed sequence using mean and standard deviation statistics to obtain the model reconstructed sequence. It retains the observed EEG matrix according to the missing mask and fills the missing positions with the corresponding values ​​of the model reconstructed sequence, and outputs the final completed EEG sequence.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the EEG completion method based on a dual-stream framework of class traits and class states as described in any one of claims 1 to 8.

Images