Multi-level electroencephalogram signal analysis method, device and equipment based on space-time coding

Abstract

The invention discloses a multi-level electroencephalogram signal analysis method, device and equipment based on space-time coding. The method comprises the steps that multiple pieces of electroencephalogram channel data to be analyzed are acquired; dividing the electroencephalogram channel data into a parallel group or a series group according to channel information corresponding to the electroencephalogram channel data; extracting multi-scale time domain and frequency domain features from each electroencephalogram channel data of the parallel group, and constructing a first feature set according to the multi-scale time domain and frequency domain features; extracting a time sequence dependence feature from each electroencephalogram channel data of the series group, and constructing a second feature set according to the time sequence dependence feature; determining a primary feature set and a secondary feature set according to the first feature set and the second feature set; dividing the main feature set and the secondary feature set into row and column coding branches, and obtaining electroencephalogram signal feature representation according to the row and column coding branches so as to complete cognitive analysis according to the electroencephalogram signal feature representation.

Description

Technical Field

[0001] The present application relates to the field of brain-computer interface technology, and in particular to a multi-level EEG signal analysis method, device and equipment based on spatiotemporal coding. Background Art

[0002] Traditional EEG signal processing methods have many shortcomings. Some methods only focus on the time domain waveform characteristics of the signal, such as studying cognitive processing by detecting the waveform characteristics of event-related potentials (ERP); other methods focus on extracting the frequency characteristics of the signal and using the power spectrum or phase synchronization of different frequency bands to analyze the activation patterns of various brain regions. Since these methods only focus on a single time domain or frequency domain perspective, they cannot fully describe the temporal and spatial coding rules inherent in EEG signals.

[0003] Some methods attempt to combine temporal and spatial information, such as using blind source separation (BSS) to mine the spatiotemporal distribution of signal sources, or using dynamic causal models (DCM) to analyze effective connections between brain regions. However, due to the use of linear models or overly simplified frameworks, these methods have obvious deficiencies in explaining complex non-stationary neural dynamic systems, and it is difficult to accurately characterize the temporal evolution and causal flow laws of brain regions. In addition, most existing methods lack analysis strategies for different levels and modalities, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in their ability to explain generalization.

[0004] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the invention

[0005] The embodiment of the present application provides a multi-level EEG signal analysis method, device and equipment based on spatiotemporal coding, which aims to solve the problem of using linear models or overly simplified frameworks. Such methods have obvious deficiencies in explaining complex non-stationary neurodynamic systems and it is difficult to accurately describe the temporal evolution and causal flow laws between brain regions. In addition, most existing methods lack analysis strategies for different levels and modes, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in the ability to explain generalization.

[0006] In a first aspect, the present application provides a multi-level EEG signal analysis method based on spatiotemporal coding, comprising:

[0007] Acquire multiple EEG channel data to be analyzed;

[0008] Dividing the EEG channel data into parallel groups or serial groups according to channel information corresponding to the EEG channel data;

[0009] Extracting multi-scale time domain and frequency domain features from each of the EEG channel data of the parallel group, and constructing a first feature set according to the multi-scale time domain and frequency domain features;

[0010] Extracting a time-dependent feature from each of the EEG channel data of the series group, and constructing a second feature set according to the time-dependent feature;

[0011] Determine a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0012] The main feature set and the secondary feature set are divided into row and column coding branches, and the feature representation of the electroencephalogram signal is obtained according to the row and column coding branches, so as to complete the cognitive analysis according to the feature representation of the electroencephalogram signal.

[0013] In a second aspect, the present application also provides a multi-level EEG signal analysis device, comprising:

[0014] A data acquisition module, used to acquire multiple EEG channel data to be analyzed;

[0015] A data grouping module, used for dividing the EEG channel data into parallel groups or series groups according to the channel information corresponding to the EEG channel data;

[0016] A first construction module is used to preprocess each EEG channel data in the parallel group and the serial group, extract multi-scale time domain and frequency domain features from each EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time domain and frequency domain features;

[0017] A second construction module is used to extract a time-dependent feature from each of the EEG channel data in the series group, and construct a second feature set according to the time-dependent feature;

[0018] A primary and secondary determination module, used to determine a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0019] The fusion completion module is used to divide the main feature set and the secondary feature set into row and column coding branches, obtain the EEG signal feature representation according to the row and column coding branches, and complete the cognitive analysis according to the EEG signal feature representation.

[0020] In a third aspect, the present application also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the multi-level EEG signal analysis method based on spatiotemporal coding as described in the first aspect is implemented.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-level EEG signal analysis method based on spatiotemporal coding as described in the first aspect.

[0022] Compared with the prior art, this application has at least the following beneficial effects:

[0023] 1. Hierarchical spatiotemporal coding reflects the hierarchical relationship between primary and secondary features:

[0024] This method adopts the innovative idea of ​​hierarchical coding, maps the primary and secondary feature sets to row and column coding branches respectively, and realizes seamless switching and hierarchical fusion between the primary and secondary features through alternating high-level activation in time. Specifically, during the encoding time period of the primary feature set, the row branch always occupies the high-energy output of the encoding, while the column branch is in a silent state; when entering the encoding stage of the secondary feature set, the situation is reversed. Through the tensor product construction, the outputs of the two branches are naturally merged into a unified fourth-order tensor representation, which not only retains the prominent position of the primary feature, but also fully expresses the detailed information of the secondary feature, effectively solving the information loss problem caused by hard-coded splitting in existing methods.

[0025] 2. Adaptive time allocation to achieve coding fairness:

[0026] The encoding period of the primary and secondary feature sets is not hard-coded, but dynamically determined through an adaptive time allocation strategy, thereby achieving fairness in encoding time between the two. This strategy automatically allocates reasonable encoding time resources according to the actual dimensions of the primary and secondary feature sets, avoiding the encoding imbalance problem caused by fixed time allocation. This design not only ensures the dominance of the primary feature, but also gives the secondary feature sufficient time resources, making the expression of spatiotemporal coding more balanced and comprehensive.

[0027] 3. Unified tensor encoding facilitates parallel computing:

[0028] The final spatiotemporal coding output is a fourth-order tensor, in which two modes correspond to the main feature coding and the other two modes correspond to the secondary feature coding. The coding structure is compact and orderly distributed. This tensor representation is not only conducive to preserving the hierarchical relationship between the main and secondary features, but also conducive to subsequent parallel computing and pattern recognition tasks. In fact, tensor decomposition and tensor networks have become current research hotspots, which can greatly improve computing efficiency while preserving data structure and correlation.

[0029] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1A schematic diagram of the process of a multi-level EEG signal analysis method based on spatiotemporal coding according to an embodiment of the present application;

[0031] Figure 2 This is a schematic diagram of the structure of a multi-level EEG signal analysis device shown in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of a computer device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0034] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0035] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0037] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0038] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0039] The technical solution of the embodiment of the present application is introduced below.

[0040] As a bioelectric signal that directly reflects the activity of brain neuron groups, EEG signals have important research value in the fields of neuroscience, cognitive psychology, and artificial intelligence. Compared with other brain imaging technologies (such as functional magnetic resonance imaging (fMRI), EEG signals have extremely high temporal resolution of milliseconds, which can capture rapid dynamic changes in the brain in real time, such as cognitive processing, decision-making, and emotional fluctuations. At the same time, EEG signals also have a certain spatial resolution and can distinguish the activity patterns of different brain regions in the cerebral cortex. Therefore, by analyzing and decoding EEG signals, we can deeply understand the information encoding mechanism inside the human nervous system, reveal the neural basis of the brain's advanced cognitive functions, and provide theoretical guidance and technical support for cognitive computing research in the field of artificial intelligence.

[0041] Traditional EEG signal processing methods have many shortcomings. Some methods only focus on the time domain waveform characteristics of the signal, such as studying cognitive processing by detecting the waveform characteristics of event-related potentials (ERP); other methods focus on extracting the frequency characteristics of the signal and using the power spectrum or phase synchronization of different frequency bands to analyze the activation patterns of various brain regions. Since these methods only focus on a single time domain or frequency domain perspective, they cannot fully describe the temporal and spatial coding rules inherent in EEG signals.

[0042] Some methods attempt to combine temporal and spatial information, such as using blind source separation (BSS) to mine the spatiotemporal distribution of signal sources, or using dynamic causal models (DCM) to analyze effective connections between brain regions. However, due to the use of linear models or overly simplified frameworks, these methods have obvious deficiencies in explaining complex non-stationary neural dynamic systems, and it is difficult to accurately characterize the temporal evolution and causal flow laws of brain regions. In addition, most existing methods lack analysis strategies for different levels and modalities, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in their ability to explain generalization.

[0043] Please refer to Figure 1, Figure 1 A flowchart of a multi-level EEG signal analysis method based on spatiotemporal coding provided in an embodiment of the present application. The multi-level EEG signal analysis method based on spatiotemporal coding in an embodiment of the present application can be applied to computer devices, including but not limited to smart phones, laptops, tablet computers, desktop computers, physical servers, cloud servers and other devices. Figure 1 As shown, the multi-level EEG signal analysis method based on spatiotemporal coding of this embodiment includes steps S101 to S106, which are described in detail as follows:

[0044] Step S101, obtaining multiple EEG channel data to be analyzed.

[0045] Specifically, first prepare the EEG acquisition system, including a headband or mesh electrode array that integrates multiple electrodes. This headband or array uses a dry electrode or gel electrode design, which can form a good electrical connection with the scalp without the use of additional conductive paste. The dry electrode is made of metal alloy or conductive polymer material, and the gel electrode is wrapped around a layer of conductive gel so that it can fit the curved surface of the scalp.

[0046] Most modern headbands are designed with adjustable size and position, which can adapt to different head shapes and correctly align the electrodes to the standard positions specified by the international 10-20 system, such as Fp1, Fp2, F3, Fz, F4, C3, Cz, C4, P3, Pz, P4, O1, O2, etc., covering the main brain areas of the frontal lobe, parietal lobe, temporal lobe and occipital lobe. Some headbands also include additional electrode positions to provide higher spatial resolution. In addition, reference electrodes and ground electrodes are required, which are usually located at specific positions of the headband or separate connection points to provide potential reference and noise reduction.

[0047] When everything is ready, put the headband with integrated electrodes on your head correctly. For dry electrode headbands, just make sure that the electrodes are in good contact with the scalp. For gel electrode headbands, squeeze out an appropriate amount of conductive gel at each electrode to enhance coupling. However, modern gel electrodes are reusable or disposable and do not require much operation. Next, correctly connect the reference electrode and the ground electrode.

[0048] Then turn on the power of the EEG acquisition device according to the instructions, wait for self-test and establish a wireless data transmission connection. Depending on the different devices, it may be necessary to start the acquisition through buttons, gestures or voice. During the formal acquisition, the applicant should try to keep the head from moving drastically to reduce artifacts caused by blinking or muscle activity, and avoid the influence of external electromagnetic interference sources, although the environmental requirements for ordinary use scenarios are relatively loose.

[0049] The acquisition device will transmit the EEG data wirelessly to the dedicated software or application on the mobile device or personal computer in real time for display, storage and preliminary analysis. Some devices will also provide real-time feedback such as brain wave visualization and attention score to help this application understand the brain state. The acquisition time can be determined according to the needs until enough data is collected. In addition, according to different usage scenarios and purposes, this application needs to adjust the acquisition strategy and parameter settings to optimize the data quality and information volume. For example, if this application wants to evaluate the attention or working memory ability of an individual, a reasonable strategy is: set the corresponding behavioral paradigm, keep the sampling rate at around 500Hz to obtain sufficient time resolution; the number of electrodes is 32-64, with a focus on the frontal and parietal regions; add an additional nose tip reference electrode to improve the detection sensitivity of frontal lobe activity; and set an appropriate bandpass filter to filter out interference components with too low and too high frequencies.

[0050] Step S102: dividing the EEG channel data into parallel groups or serial groups according to the channel information corresponding to the EEG channel data.

[0051] Specifically, assuming that the present application collects data from N EEG channels, it is usually necessary to first divide these N channels into different subsets according to the functional areas of the brain, such as the prefrontal region, parietal region, temporal region, etc. Each subset contains all the channels of the corresponding brain region. This preliminary grouping based on spatial location helps the present application to better capture and explain the activity pattern of each brain region. Next, for each subset, the present application proposes an optimized grouping method based on spectral clustering and spatiotemporal constraints to divide the channels into parallel groups and serial groups.

[0052] In some embodiments, the dividing the EEG channel data into parallel groups or serial groups according to the channel information corresponding to the EEG channel data includes: determining the brain functional area corresponding to each of the EEG channel data according to the channel information; dividing the EEG channel data with the same brain functional area into a subset; and dividing the EEG channel data corresponding to each of the subsets into parallel groups or serial groups based on an optimized grouping method of spectral clustering and spatiotemporal constraints.

[0053] First, for all channel pairs in each subset, this application calculates the functional connection strength between them and constructs an NxN weighted undirected graph, where N is the number of channels in the subset, and the weight of the edge reflects the degree of functional correlation between the two channels. The functional connection strength can be quantified in a variety of ways, such as phase locking value, coherence, mutual information, etc. The specific indicator to be used needs to be selected based on the actual data and application scenarios. Next, this application performs spectral clustering on this weighted undirected graph to divide highly correlated channels into the same functional subset. Spectral clustering is an effective graph segmentation algorithm that can divide nodes (channels) into different clusters based on the Laplace matrix eigenvectors of the graph, so that the similarity between nodes within the cluster is high and the similarity between nodes between clusters is low. Through spectral clustering, this application will obtain two functional subsets, each containing a channel group with a close functional connection.

[0054] Exemplarily, the dividing of the EEG channel data corresponding to each of the subsets into parallel groups or serial groups includes: calculating the functional connection strength corresponding to each of the subsets; constructing a weighted undirected graph based on the functional connection strength; performing spectral clustering on the weighted undirected graph to divide the EEG channel data corresponding to each of the subsets into a first functional subset and a second functional subset; calculating the EEG channel data of the first functional subset and the second functional subset according to a preset objective function to divide the EEG channel data of the first functional subset and the second functional subset into parallel groups or serial groups; the objective function includes functional constraints and spatial constraints.

[0055] After obtaining two functional subsets, the goal of this application is to further divide the channels into parallel groups and series groups within each functional subset, while fully considering the proximity of the channels in spatial position. To this end, this application proposes a spatiotemporal constraint optimization algorithm that fuses functional connectivity and spatial position information to automatically determine the optimal grouping scheme. Specifically, this application constructs an objective function that contains two parts: a functional constraint term and a spatial constraint term. The functional constraint term is used to maximize the functional correlation between channels within the group, while the spatial constraint term is used to minimize the degree of discreteness of the channels within the group in spatial position. By optimizing this objective function, this application can find a grouping scheme that maintains high functional connectivity and has good spatial aggregation.

[0056] This application first defines the functional constraint term Ffunc, which includes the sum of the functional connection strengths between channel pairs in the same group and the sum of the functional connection strengths between channel pairs in different groups, corresponding to intra-group connections and inter-group connections, respectively. The goal is to maximize intra-group connections and minimize inter-group connections. Its expression is: Ffunc = Σi,j∈Gk Wij- λΣi∈Gk,j∉GkWij where Gk (k=1 or 2) represents the kth group, Wij is the functional connection strength between channel i and channel j, and λ is a balance parameter.

[0057] Next, the spatial constraint term Fspatial is defined, which contains the sum of the Euclidean distances between channel pairs in the same group. The goal is to minimize the spatial discreteness between channels in the group: Fspatial=Σi,j∈Gk dij where dij is the Euclidean distance between channel i and channel j in three-dimensional space.

[0058] Combining the functional constraint term with the spatial constraint term, this application obtains the objective function: F = Ffunc - μFspatial, where μ is the weight coefficient of the spatial constraint term, which is used to balance the trade-off between functional connectivity and spatial aggregation.

[0059] Now the goal of this application is to find a grouping scheme G1 and G2 in each functional subset so that the objective function F reaches the maximum value. In this problem, this application needs to find an optimal solution to divide the channels into parallel groups G1 and series groups G2 in each functional subset so that the objective function F=Ffunc-μFspatial reaches the maximum value. Among them, Ffunc is a functional constraint term, which is used to maximize the sum of the functional connection strengths of the channels within the group, minus the sum of the functional connection strengths of the channels between groups; Fspatial is a spatial constraint term, which is used to minimize the degree of discreteness of the channels within the group in spatial position. This application will use a genetic algorithm to solve this combinatorial optimization problem. A genetic algorithm is an optimization algorithm that simulates the biological evolution process. Through operations such as selection, crossover, and mutation of individuals in a population, it continuously generates new solutions and evaluates their fitness, gradually converging to the optimal solution.

[0060] First, the application needs to encode the problem and represent the candidate solution (i.e., channel grouping scheme) in a form suitable for genetic operations. A common encoding method is binary encoding. For example, for a functional subset containing 8 channels, the application can use a binary string of length 8 to represent a grouping scheme, where 0 indicates that the channel belongs to a parallel group and 1 indicates that it belongs to a serial group. For example, the encoding 01011010 corresponds to assigning channels 1, 4, 5, and 8 to the parallel group and the remaining channels to the serial group.

[0061] Next, the application initializes a population of N individuals (candidate solutions), which are usually randomly generated. Then, the objective function value F is calculated for each individual as the basis for evaluating its fitness. The higher the fitness, the closer the individual is to the optimal solution.

[0062] In each generation of inheritance, the application first performs a selection operation based on the fitness value of the individual, and selects several individuals as the parent generation. The individuals with higher fitness have a greater probability of being selected. Common selection methods include roulette selection, sorting selection, tournament selection, etc.

[0063] After selecting the parent individuals, the present application performs a crossover operation on them to generate new offspring individuals. Crossover is the most critical operation in the genetic algorithm, which simulates the chromosome exchange process in biological evolution. For binary-coded individuals, the present application can adopt a single-point crossover or a multi-point crossover. Taking a single-point crossover as an example, the present application randomly selects two parent individuals and a crossover point, and swaps the coding segments of the two parents before the crossover point, thereby generating two new offspring individuals. For example, the parents 01011010 and 10100101 cross at the 4th position, generating offspring 0110101 and 1001010.

[0064] After the crossover is completed, the application also needs to perform mutation operations on the offspring individuals to maintain the diversity of the population and avoid premature convergence. The mutation operation is achieved by changing certain genes (bits) in the individual code. Common mutation methods include basic bit mutation (randomly inverting certain bits) and uniform mutation (changing certain bits to random values). The probability of mutation operation is low, usually between 0.01 and 0.1.

[0065] After operations such as selection, crossover, and mutation, the present application obtains a new generation of offspring populations. The present application then calculates the fitness of each new individual and selects to replace some of the parent individuals with lower fitness according to the fitness value, thereby generating a new generation of mixed populations.

[0066] Repeat the above process and perform multiple generations of genetic operations until the termination conditions are met (such as reaching the maximum number of evolutionary generations, fitness convergence, etc.). Finally, the individual with the highest fitness in the population corresponds to the (approximate) optimal solution to the combinatorial optimization problem, that is, the best EEG channel grouping scheme.

[0067] This grouping method based on spectral clustering and spatiotemporal constraint optimization can automatically balance the two key factors of functional connectivity and spatial proximity to obtain the best parallel group and serial group division. It overcomes the limitations of manually designed grouping rules and can adaptively determine the optimal grouping strategy based on data characteristics. At the same time, it also avoids the grouping imbalance problem caused by considering only a single factor (such as functional connectivity or spatial position), thus hopefully obtaining a more efficient and comprehensive representation of EEG signals.

[0068] In some embodiments, before extracting multi-scale time domain and frequency domain features from each of the EEG channel data in the parallel group, it also includes: preprocessing each of the EEG channel data in the parallel group and the series group; the preprocessing includes at least one or more of filtering denoising, standardization, and outlier correction.

[0069] For the original EEG data of each channel group (including parallel group and series group), first use a notch filter to remove narrow-band noise of a specific frequency, such as setting the notch center frequency to 50 or 60 Hz to eliminate power line noise, and set the stopband width according to the specific situation. Next, use a bandpass filter to retain only the frequency range of interest. For example, design an FIR or IIR bandpass filter with a passband lower boundary of 0.5 Hz and an upper boundary of 40 Hz. The transition band width is combined with specific needs. In this way, the EEG frequency band signal of 0.5-40 Hz can be retained and the low-frequency and high-frequency interference components can be filtered out. In addition, this application can also use blind source separation methods such as independent component analysis to extract and remove the influence of physiological artifacts such as electrooculography and electromyography.

[0070] After filtering and denoising, the EEG data needs to be baseline corrected. Due to factors such as DC drift and instrument offset, the baseline of the original data is usually not 0. This application can use piecewise linear fitting or wavelet estimation methods within a certain time window to estimate the baseline signal, and then subtract it from the original data to normalize the baseline of the denoised EEG signal to near 0. Next, the data needs to be re-referenced. During acquisition, this application usually sets a reference electrode, and the potentials of all other electrodes are measured relative to this reference electrode, but a single reference may introduce additional artifacts. Therefore, this application recalculates all electrode potentials relative to the average reference. It can also use reference electrode estimation technology to estimate an ideal reference, or directly perform Laplace or surface Laplace transform on the original data to achieve no reference.

[0071] Then standardization is required. Since the EEG amplitudes between different channels and at different time points vary greatly, in order to eliminate this scale effect, this application can perform Z-Score standardization on each channel, that is, first calculate the channel mean and standard deviation, then subtract the mean and divide by the standard deviation; or perform Min-Max standardization to scale the data to the range of 0-1. Some severely damaged channels may be eliminated during the preprocessing process. In order to ensure spatial resolution, spherical interpolation or other interpolation algorithms are needed to reconstruct these missing channels based on the surrounding channel data. For example, the formula for spherical interpolation is x(r, theta, phi) = sum(gi*Vi(r, theta, phi)), where gi is the interpolation coefficient and Vi is the spherical chi-square basis function.

[0072] In addition, even after filtering and other processing, there may still be outliers in the EEG data due to instrument failure and other reasons. This application can detect and correct or delete these outliers based on the absolute median deviation. The specific method is to first calculate the median medi of each channel data xi, and then calculate the median absolute deviation med_abs_dev=medi(|xi-medi|). Any data point whose deviation from medi is greater than k times med_abs_dev (k is usually 6) is considered an outlier. Finally, if EEG analysis is performed for a specific cognitive task, it is also necessary to divide the continuous EEG data into multiple time periods according to the experimental paradigm, and average all the time period data corresponding to each cognitive process condition to extract the evoked potential or rhythm feature of interest. The time window and baseline correction period need to be set according to the specific task.

[0073] Step S103, extracting multi-scale time domain and frequency domain features from each of the EEG channel data in the parallel group, and constructing a first feature set according to the multi-scale time domain and frequency domain features.

[0074] Specifically, the goal of parallel feature extraction is to use the EEG channel data in the parallel group to extract multi-scale time domain and frequency domain features to form a first feature set. The present application first separates the relevant EEG channels of each parallel group from the pre-processed EEG data, and then applies wavelet packet analysis to these parallel channels for multi-scale time-frequency decomposition, thereby extracting rich time domain and frequency domain features.

[0075] Wavelet packet analysis is an extension of wavelet analysis. It not only further decomposes low-frequency components, but also decomposes high-frequency components at a deeper level, thereby obtaining time-frequency information of all frequency bands. Specifically, this application selects a suitable wavelet packet basis, such as Daubechies wavelet db4, to perform wavelet packet transform on the EEG data of each parallel channel. Assuming that the number of wavelet packet decomposition layers is J=6, 2^(J+1)-1=127 wavelet packet coefficient sequences will be obtained, and each sequence corresponds to a specific time-frequency subdivision band.

[0076] In the process of wavelet packet decomposition, the EEG signal is first decomposed into an approximate component A1 and a detail component D1, corresponding to the low-frequency and high-frequency parts. Then, both A1 and D1 are further binary decomposed to generate four new sequences A2, D2, AD2, and DD2, and these four sequences are decomposed again, and so on, until the pre-set maximum decomposition level J=6 is reached. Finally, this application obtains 63 approximate and detail coefficient sequences (Ajk and Djk, j=1...6, k=1...2^j), each of which covers the behavior of the signal in a specific time-frequency interval.

[0077] For each wavelet packet coefficient sequence, this application extracts the following time domain and frequency domain features:

[0078] 1. Energy feature Ej: Calculate the energy of the sequence coefficient, that is, E(jk) = sum(|C(jk)|^2). Where C(jk) is the coefficient sequence of the sequence, and |·| takes the absolute value. The energy feature reflects the energy distribution of the signal in the corresponding time-frequency band.

[0079] 2. Entropy feature Hj: Calculate the energy entropy of the sequence coefficients as a feature to measure the complexity of the signal in this time-frequency band. The higher the entropy value, the more uniform the distribution of the signal in the band and the greater the uncertainty.

[0080] 3. Standard deviation Sj and peak feature Pj: Calculate the standard deviation std(C(jk)) and peak value (maximum / minimum value) of the sequence coefficients respectively, which reflects the distribution of the time-frequency coefficients in terms of value.

[0081] 4. Dip coefficient Rj: Calculate the dip coefficient of the power spectrum of the sequence coefficients, which is used to characterize the attenuation characteristics of the spectrum outside the time-frequency band, that is, Rj=sum(|C(f>fh)|^2) / sum(|C(f)|^2), where fh is the upper boundary frequency of the time-frequency band.

[0082] In addition, this application also extracts the correlation characteristics between adjacent time-frequency bands:

[0083] 1. Adjacent correlation coefficient rjk: For each coefficient sequence C(jk), calculate its correlation coefficient with the four adjacent sequences C(j-1,k), C(j-1,k+1), C(j,k-1), and C(j,k+1) as a feature to characterize the time-frequency correlation. The correlation coefficient can be defined in many ways, such as Pearson correlation and mutual information.

[0084] Through the above methods, this application obtains a large number of features describing the signal's time domain and frequency domain behavior from wavelet packet decomposition. These features not only cover all time-frequency bands, but also contain information in multiple dimensions such as energy, entropy, statistics, and correlation, which can comprehensively characterize the dynamic characteristics of EEG signals at various time-frequency scales.

[0085] The above features are arranged in the order of time-frequency bands to form the first feature set. It should be noted that the dimension of the first feature set is equal to the number of extracted features multiplied by the number of time-frequency bands after wavelet packet decomposition (127*5=635 dimensions in this case). This feature set contains the behavior patterns of each EEG channel in the parallel group at different time scales and frequency scales.

[0086] Step S104, extracting time-dependent features from each of the EEG channel data in the series group, and constructing a second feature set according to the time-dependent features.

[0087] Specifically, the serial group is used to extract the temporal dependency features and obtain the second feature set. Different from the parallel group capturing the synchronization pattern in space, the goal of the serial group is to analyze the temporal interaction and causal dependency between parallel groups. The channels in the serial group are arranged in a certain functional topological order, and each channel represents a potential brain region or network node. Therefore, the characteristics of the serial group actually reflect the dynamics of the temporal information flow between brain regions.

[0088] This application will use the Hidden Markov Model (HMM) to model the feature sequences of each parallel group in the series group, so as to extract the time-dependent features. HMM is a statistical Markov model that can well describe random sequences containing hidden unknown states and has been widely used in speech recognition, gene sequencing and other fields. In neural signal processing, this application can regard the series group as a non-stationary random process generated by multiple hidden states, and use HMM to automatically mine these hidden states and their evolution laws to reveal the time-series interaction patterns between different brain regions.

[0089] Specifically, assuming that the series group contains M parallel groups, each parallel group is represented by a P-dimensional feature vector in the first feature set, then the entire series group can be described by a P*M-dimensional feature matrix sequence {O(t)}, where O(t)=[O1(t), O2(t), ..., OM(t)] is the observation matrix at the tth moment. The goal of this application is to establish an N-state HMM λ=(A, B, π), where A is the NxN state transition probability matrix, B is the observation probability distribution, and π is the initial state probability vector.

[0090] In HMM, the hidden state {X(t)} of the system follows a Markov chain, that is, the current state only depends on the previous state, and the state transition probability is:

[0091] P(X(t+1)=j|X(t)=i,...,X(0)=x0) = P(X(t+1)=j|X(t)=i) = aij

[0092] Where aij is an element of the state transfer matrix A. The probability distribution of the observation matrix O(t) is determined by the output probability of the hidden state X(t), assuming that O(t) obeys the Gaussian Mixture Model (GMM):

[0093] B = {bj(O(t)) = Σk=1^M cjk * N(Oj(t); μjk, Σjk)}, j=1...N

[0094] where bj(O(t)) is the conditional probability density function of the observation O(t) in hidden state j, which is composed of the weighted sum of M Gaussian components, cjk is the coefficient of the kth Gaussian mixture model in state j, satisfying Σk cjk=1, and N(·;μ,Σ) is the Gaussian distribution density function with mean μ and covariance Σ.

[0095] Using the observation sequence {O(t)} and the known HMM parameters λ=(A, B, π), the present application can use the classic forward-backward algorithm or the Viterbi algorithm to calculate the probability P(O|λ) of the observation sequence under the model, and further estimate the optimal parameter value through the Baum-Welch expectation maximization algorithm, that is, the HMM training process.

[0096] After the HMM training converges, this application first extracts the following features from the obtained state transition matrix A and the initial state π:

[0097] 1. Stationary distribution πs: This distribution reflects the proportion of each state of the HMM, and thus reflects the average contribution of activation of different brain regions. Let I be the unit matrix, πs = (I-A+1)^(-1) * π.

[0098] 2. Spectral radius ρ(A): The maximum absolute eigenvalue of the state transfer matrix A, which reflects the time scale of the HMM system. The larger ρ(A) is, the stronger the system's ability to maintain its initial state.

[0099] 3. Entropy characteristic Hs: Calculate the entropy Hs = -Σi πs(i)logπs(i) of the state sequence {X(t)} to measure the uncertainty and complexity of the system.

[0100] 4. Intersection time τ: It is the average time τ required for the system to reach a stable distribution from any state, reflecting the stable nature of the system.

[0101] 5. Absorption state count Na: Count the number of absorbing states in the transfer matrix (i.e. states that will not be transferred once entered). The more Na there are, the more unstable the system tends to be.

[0102] Then, this application analyzes the parameters of the observed Gaussian mixture model GMM under each hidden state j:

[0103] 6. Mean vector {μjk} and covariance matrix {Σjk}: The mean vector reflects the average activation pattern of the series group in hidden state j; the covariance matrix describes the variability of activation and the covariance structure between channels.

[0104] 7. Mixing coefficient {cjk}: reflects the contribution weights of different Gaussian components in the hidden state j, which can be used to distinguish multiple potential activation sub-patterns.

[0105] 8. Mahalanobis distance dM(j,j'): For any two hidden states j and j', calculate the Mahalanobis distance between their GMM parameters to quantify the degree of separability of different states.

[0106] 9. BIC value: The Bayesian Information Criterion (BIC) is calculated for each hidden state of the GMM to weigh the model complexity and data fit. The smaller the BIC, the more reliable the model.

[0107] The above features extracted from the state transition structure and observation model of HMM capture the temporal evolution of the feature sequence of the parallel group in the series group, reflecting the interaction pattern and causal dependency between different brain regions. Combining these temporal dependency features in sequence forms the second feature set.

[0108] Step S105: determining a primary feature set and a secondary feature set according to the first feature set and the second feature set.

[0109] Specifically, the number of the first and second feature sets is compared to determine the primary and secondary feature sets. The primary feature set will serve as the main line of spatiotemporal coding, dominating the rhythm and pace of the entire coding process; while the secondary feature set provides a powerful supplement to the primary feature set, enriching the representation form and enhancing the detailed description of spatiotemporal features. Therefore, this application needs to comprehensively judge which one should be used as the primary feature set based on the characteristics and importance of the first and second feature sets.

[0110] In some embodiments, before determining the primary feature set and the secondary feature set based on the first feature set and the second feature set, it also includes: generating a time coding period based on the first feature set and the second feature set; and positioning encoding the first feature set and the second feature set respectively according to the time coding period.

[0111] The first feature set is the multi-scale time domain and frequency domain features extracted by the parallel group, which can fully characterize the dynamic behavior of the signal at different time scales and frequency scales, and has a high dimension. Specifically, if wavelet packet analysis is used, the dimension of the first feature set is equal to the number of extracted features multiplied by the number of time-frequency bands after wavelet packet decomposition. Assuming that 5 features are extracted and the number of wavelet packet decomposition layers is 6, the dimension of the first feature set is 635. The second feature set uses the hidden Markov model (HMM) to model the series group and extract time-dependent features. It has a low dimension but contains extremely rich time dynamics information, showing the evolution logic of the parallel group features in time. Taking 9 features and 10 hidden states as an example, the dimension of the second feature set is 90 dimensions.

[0112] Given the large differences in the dimensions of the two feature sets, if they are directly used as the input of time coding, the time period of the coding will be biased towards the first feature set, thus failing to fully reflect the temporal dependency relationship implied by the second feature set. In order to balance the weights of the two feature sets in time coding, this application proposes an adaptive time period setting method:

[0113] Assume that the dimension of the first feature set is D1, the dimension of the second feature set is D2, the total period of time coding is T, where the coding of the first feature set lasts for T1 time steps, the coding of the second feature set lasts for T2 time steps, and T1+T2=T. This application needs to determine appropriate T1 and T2 so that the contributions of the two feature sets in time coding are equivalent.

[0114] Considering that the second feature set has a lower dimension but contains extremely valuable temporal dynamics information, this application hopes that it will be encoded at least once in the entire time encoding period, that is, T2>=1. At the same time, the encoding step size T1 of the first feature set should not be too small, so as not to be unable to fully express the time domain and frequency domain details. Therefore, this application can set a threshold α so that T2 takes the maximum value in the interval [1, αT], and the corresponding T1=T-T2.

[0115] Specifically, let T2 = argmax(D2,α*T), then T1=T-T2. Where argmax(x,y) means taking the larger value of x and y. If α is set to 0.5, it means that the second feature set occupies at most half of the total encoding period, and the remaining time steps will be allocated to the first feature set. In this way, the present application can adaptively determine T1 and T2, thereby achieving a weight balance between the two feature sets in time encoding.

[0116] Suppose the total coding period T=100, the first feature set dimension D1=635, the second feature set dimension D2=90, α=0.5, then: T2 = argmax(90, 0.5*100) = 50 T1 = 100 - 50 = 50.

[0117] That is to say, the second feature set will be encoded in the first 50 time steps, and the first feature set will be encoded in the last 50 time steps, and the weights of the two in time encoding are equal.

[0118] Based on the set time period, the first and second feature sets are positioned and encoded.

[0119] The purpose of location encoding is to map the feature vector to a high-dimensional position vector, giving each feature a position and time encoding. By projecting the internal feature state to the external position vector, this application can explicitly model the distribution of features in the spatiotemporal domain, thereby capturing the underlying spatiotemporal structure.

[0120] For the first feature set, the present application adopts a positioning coding scheme based on radial basis function (RBF). RBF coding can construct any response function through translation, scale transformation and combination operation, thereby effectively expressing the activation pattern of input features at various positions and times.

[0121] Specifically, assume that the dimension of the first feature set is M, the dimension of the encoded position vector is N (N>>M), and the time encoding period is T1. This application first initializes N RBF bases, each of which is represented by an N-dimensional position vector μi (i=1...N), and these vectors are randomly and uniformly distributed in the N-dimensional space. Then, for the M-dimensional feature vector x(t)=[x1(t),...,xM(t)]^T at the t-th time (t=1...T1), this application calculates its distance to each RBF base:

[0122] ri(t) = ||x(t) - μi|| , i=1...N (1).

[0123] Where ||·|| represents the Euclidean distance. Then, each distance is nonlinearly encoded through a Gaussian radial basis function g(ri) to obtain an N-dimensional position vector:

[0124] y(t) = [g(r1(t)), g(r2(t)), ..., g(rN(t))]^T (2).

[0125] Where g(r) = exp(-r^2 / 2σ^2), σ is the standard deviation parameter of the RBF kernel, which is used to control the smoothness of the response.

[0126] Through the above encoding process, each M-dimensional feature vector x(t) is mapped to an N-dimensional position vector y(t), where each dimension represents the activation response of the feature at the corresponding position. Since N is much larger than M, the position vector y(t) has a higher dimension and richer expression ability than the original feature vector x(t), and can capture the distribution pattern of the feature in the spatiotemporal domain.

[0127] For the second feature set, this application adopts a positioning encoding scheme based on phase encoding. Unlike the first feature set that reflects the spatial distribution of the parallel group, the second feature set reflects the temporal evolution logic of the series group. Therefore, this application requires an encoding method that can naturally capture the temporal dynamics. Phase encoding is based on this idea, mapping the feature to a circular position vector, and the change in phase corresponds to the passage of time, thereby implicitly modeling the passage of time.

[0128] Assume that the dimension of the second feature set is K, the time encoding period is T2, and the dimension of the positioning vector is N (N>>K). This application initializes N phase encoding vectors:

[0129] mj = [cos(2πj / N), sin(2πj / N)], j=1...N (3).

[0130] These vectors are evenly distributed on a unit circle in N-dimensional Euclidean space. For the K-dimensional feature vector z(t)=[z1(t),...,zK(t)]^T at the t-th time (t=1...T2), this application first calculates the length of its projection on each phase encoding vector mj:

[0131] lj(t) = mj^T * z(t) , j=1...N (4).

[0132] These projection lengths are then Gaussian kernel encoded:

[0133] w(t) = [g(l1(t)), g(l2(t)), ..., g(lN(t))]^T (5).

[0134] Where g(l)=exp(-l^2 / 2σ^2) is the Gaussian kernel function and σ is the smoothing parameter. Through this phase encoding method, each K-dimensional feature vector z(t) is mapped to an N-dimensional position vector w(t), which shows the distribution of the timing characteristics at different phases on the unit circle.

[0135] It is worth noting that since the phase encoding vectors are uniformly distributed on the unit ring in sequence, when time t passes, the activation position of the position vector w(t) on the ring will also move accordingly, thereby implicitly capturing the passage of time. This makes phase encoding particularly suitable for position encoding of temporal dependent features.

[0136] Through RBF encoding and phase encoding, the present application maps the first and second feature sets to position vectors with rich structural and temporal information respectively.

[0137] In some embodiments, determining the primary feature set and the secondary feature set based on the first feature set and the second feature set includes: calculating a first importance score value for the first feature; calculating a second importance score value corresponding to the second feature; the importance score value is used to determine the importance of the first feature set and the second feature set to spatiotemporal coding; determining the primary feature set and the secondary feature set based on the first importance score value and the second importance score value.

[0138] For the first feature set, let its dimension be D1, and define an importance score function f1(D1) to indicate the importance of the first feature set to spatiotemporal coding; for the second feature set, let its dimension be D2, and the importance score be f2(D2). Then, this application compares the size of f1(D1) and f2(D2), and determines the feature set with higher importance as the main feature set.

[0139] The importance scoring function can be designed based on specific tasks and background knowledge, and the following factors can be considered:

[0140] 1. Dimension size: A feature set with a higher dimension often contains more information and is more important for encoding. This application can design a function f(D)=D^α(α>0) to directly weight the dimension D.

[0141] 2. Information content: Based on information theory, the information content of a feature set can be defined and used as an importance score. For example, for the first feature set, the information entropy of each feature can be calculated, and then the entropy of all features can be summed as f1(D1). The higher the information entropy, the more information the feature set contains.

[0142] 3. Discriminative ability: According to the specific task, a discriminative ability score can be considered to be defined to measure the discrimination ability of the feature set for the task. For example, in the cognitive analysis task of EEG, based on a small amount of labeled data, the discriminative scores of each feature under different cognitive conditions can be calculated, and the weighted sum of all feature discriminative scores can be used as f1(D1) or f2(D2). The discriminative scores can be calculated based on information gain, chi-square statistic, or other discriminative criteria.

[0143] 4. Subjective weight: In some applications, the present application can assign subjective importance weights to different feature sets according to domain knowledge or expert experience. For example, if the present application pre-assumes that the time-series dependent features are more important than the frequency-domain features, then f2(D2)>f1(D1) can be set.

[0144] After designing a suitable scoring function, the present application applies it to the first and second feature sets to obtain f1(D1) and f2(D2). Without loss of generality, assuming f1(D1)>f2(D2), the first feature set is determined as the main feature set, and the second feature set is determined as the secondary feature set. Of course, if f1(D1)<f2(D2), the opposite determination is made.

[0145] For specific illustration, assume that the dimension of the first feature set D1 = 635 and the dimension of the second feature set D2 = 90. The present application uses dimension weighting as the importance score: f(D)=D. Then: f1(635) = 635 f2(90) = 90 Obviously, f1(635)>f2(90), so the first feature set is determined as the main feature set, and the second feature set is determined as the secondary feature set.

[0146] In addition, the present application can also introduce an importance threshold τ. Only when the importance difference between the main feature set and the secondary feature set exceeds τ, it is confirmed. Otherwise, the importance of the two is regarded as equivalent, and equal-weight parallel coding is performed. For example, let τ = 100. In the above example: f1(635) - f2(90) = 545 > τ Therefore, it is reasonable to confirm that the first feature set is the main feature set and the second feature set is the secondary feature set. If the difference is less than τ, further analysis or adjustment of the scoring function may be required. Through the above method, the present application can adaptively determine the primary and secondary feature sets.

[0147] In some embodiments, the division into row and column coding branches according to the main feature set and the secondary feature set includes: allocating each EEG coding data corresponding to the main feature set to the row branch; allocating each EEG coding data corresponding to the secondary feature set to the column branch; setting an encoding period according to the primary and secondary relationships corresponding to the main feature set and the secondary feature set; encoding the row branch and the column branch respectively according to the encoding period to obtain the row coding branch corresponding to the row branch and the column coding branch corresponding to the column branch, so as to generate the row and column coding branches.

[0148] The goal of the coding control stage is to map the primary and secondary feature sets to the row and column coding branches respectively, and set a reasonable coding cycle according to their primary and secondary relationship, so as to achieve alternating high-level outputs of the two branches in time. Through this alternating timing modulation mechanism, the present application can simultaneously reflect the information of the primary feature set and the secondary feature set in the spatiotemporal coding, ensuring the balanced integration of features at different levels.

[0149] Exemplarily, the method of obtaining the characteristic representation of the EEG signal according to the row and column coding branches includes: obtaining a row coding high-level activation interval corresponding to the row coding branch; obtaining a column coding high-level activation interval corresponding to the column coding branch; alternately activating the row coding branch and the column coding branch according to the row coding high-level activation interval and the column coding high-level activation interval, and fusing to generate the characteristic representation of the EEG signal.

[0150] Assuming that the first feature set is determined as the main feature set and the second feature set is the secondary feature set, the present application assigns the main feature set code to the row branch and the secondary feature set code to the column branch. Specifically, let the dimension of the main feature set be D1, and after positioning coding, an N1-dimensional position vector sequence {y(t)} is generated, and the time coding period is T1; let the dimension of the secondary feature set be D2, and an N2-dimensional position vector sequence {w(t)} is generated, and the time coding period is T2, where T1+T2=T, and T is the total period of the entire space-time coding. The goal of the present application is to adjust T1 and T2 so that the main row branch and the secondary column branch alternately occupy the high level of coding in time, thereby fully reflecting the primary and secondary hierarchical relationship between the two.

[0151] First, define the row encoding and column encoding activation, represented by two real number sequences {arow(t)} and {acol(t)} in the range of 0-1, respectively, where:

[0152] arow(t)=1 if t∈[Trow_start, Trow_end] else 0 (1) acol(t)=1 if t∈[Tcol_start, Tcol_end] else 0 (2).

[0153] Where [Trow_start, Trow_end] is the time interval for row coding activation, and [Tcol_start, Tcol_end] is the time interval for column coding activation. This application needs to set these intervals so that row and column coding are activated alternately, that is:

[0154] Trow_start = 1, Trow_end = T1 Tcol_start = T1+1, Tcol_end = T1+T2 =T.

[0155] Therefore, when t∈[1,T1], arow(t)=1 and acol(t)=0, indicating that only the row coding is active; and when t∈[T1+1,T], acol(t)=1 and arow(t)=0, indicating that only the column coding is active. In this way, the present application successfully implements the time alternation mechanism of row and column coding.

[0156] Next, this application defines the high-level output of row coding and column coding. For the tth moment, the high-level output of row coding is a K1xN1 matrix:

[0157] Orow(t) = arow(t) * y(t) * ones(1,N1) (3).

[0158] Where y(t) is the N1-dimensional position vector of the main feature set at the tth moment, ones(1,N1) is a 1×N1 matrix of all ones, then each column of Orow(t) is arow(t)*y(t), when arow(t)=1, the value of Orow(t) is equal to the value of y(t); when arow(t)=0, Orow(t) is a matrix of all zeros, indicating that the row code is in a silent state.

[0159] Similarly, for time t, the column-encoded high-level output is a K2xN2 matrix:

[0160] Ocol(t) = acol(t) * w(t)^T * ones(N2,1) (4).

[0161] Where w(t) is the N2-dimensional position vector of the sub-feature set at the tth time, ones(N2,1) is an N2×1 all-one column vector, then each row of Ocol(t) is acol(t)*w(t)^T, when acol(t)=1, the value of Ocol(t) is equal to the value of w(t)^T; when acol(t)=0, Ocol(t) is an all-0 matrix, indicating that the column encoding is in a silent state.

[0162] Substituting (3) and (4) into (1) and (2), the present application obtains the complete expression of the row-column coded high-level output:

[0163] Orow(t) = y(t) * ones(1,N1) if t∈[1,T1] (row encoding is active) 0 else (row encoding is silent).

[0164] Ocol(t) = w(t)^T * ones(N2,1) if t∈[T1+1,T] (column encoding is active) 0 else (column encoding is silent).

[0165] It can be seen that when t∈[1,T1], the row encoding activates the output y(t); and when t∈[T1+1,T], the column encoding activates the output w(t)^T. This temporal segmented activation mechanism enables the main feature set (row encoding) and the secondary feature set (column encoding) to switch smoothly during the encoding process, avoiding direct hard cutting, thereby achieving continuous and smooth feature fusion.

[0166] It should be noted that for time step t, since N1 and N2 are often not equal, the dimensions of Orow(t) and Ocol(t) are inconsistent and cannot be directly spliced. In order to obtain a coded output of uniform dimension, this application can expand the dimension through tensor product construction (Tensor Outer Product). Specifically, the high-level output of spatiotemporal coding is defined as a fourth-order tensor O(t) of K1xN1xK2xN2:

[0167] O(t) = Orow(t) ⨂ Ocol(t) (5).

[0168] Where ⨂ is the tensor product operator, which expands Orow(t) and Ocol(t) in the row and column directions. If Orow(t) is a 1xN1 matrix and Ocol(t) is a K2xN2 matrix, then O(t) is a fourth-order tensor of K1xN1xK2xN2, with the first and second modes corresponding to row encoding and the third and fourth modes corresponding to column encoding. This tensor product encoding can well preserve the structural information of row and column encoding and reflect the hierarchical relationship between primary and secondary features.

[0169] Finally, it should be noted that in order to more smoothly transition the primary and secondary feature sets, the present application can introduce a transition interval at their junction, so that the row and column codes are activated simultaneously within a certain period of time, rather than directly switching hard. Assuming the length of the transition interval is Δt, the high-level activation interval of the row code is corrected to [1, T1+Δt], and the high-level activation interval of the column code is corrected to [T1-Δt+1, T], and they overlap and activate in the [T1-Δt+1, T1+Δt] time period. Through this "soft switching" mechanism, spatiotemporal coding will be smoother and more continuous.

[0170] Step S106, dividing into row and column coding branches according to the main feature set and the secondary feature set, acquiring EEG signal feature representation according to the row and column coding branches, so as to complete cognitive analysis according to the EEG signal feature representation.

[0171] Specifically, first, the present application needs to reconstruct the low-level original signal representation corresponding to the high-level spatiotemporal coding tensor O(t), that is, the main feature set position vector y(t) and the secondary feature set position vector w(t). Since the row and column coding branches are activated alternately in time, the reconstruction process is reversible. When arow(t)=1, the present application directly extracts y(t) from the first and second modes of O(t); when acol(t)=1, the present application extracts w(t)^T from the third and fourth modes, and then transposes it to restore w(t). Specifically, let the four modes of O(t) be indexed from 1, and we have:

[0172] y(t) = O(t)(:,:,1,1) if arow(t)=1.

[0173] w(t) = O(t)(1,1,:,:)^T if acol(t)=1.

[0174] Among them, O(t)(:,:,1,1) represents the first and second modes of the tensor O(t), which is a matrix of the same dimension as y(t); O(t)(1,1,:,:) is a K2xN2 matrix composed of the third and fourth modes, and its transposition gives the N2xK2 w(t). In this way, the application successfully separated the encoding representation of the primary and secondary feature sets from the high-level spatiotemporal coding.

[0175] Next, y(t) and w(t) need to be decoded to reconstruct the original first feature set {x(t)} and the second feature set {z(t)}. For y(t), this application uses a radial basis function decoder (Radial Basis Function Decoder) for decoding. Specifically, first calculate the response distance from y(t) to each RBF basis μi:

[0176] ri(t) = ||y(t) - μi||, i=1...N1 (1).

[0177] A decoding function f is then used to map these distances back to the original M-dimensional feature space:

[0178] x(t) = f([r1(t),r2(t),...,rN1(t)]) (2).

[0179] Where f can be a feedforward neural network, kernel method or any other function approximator that can perform nonlinear mapping on N1-dimensional distance vector. Through training, f can learn the inverse mapping from RBF codebook to original feature space, thus completing the decoding process. The decoded x(t) is the original representation of the first feature set, with dimension M.

[0180] Similarly, for w(t), the present application uses a phase decoder to reconstruct the second feature set {z(t)}. The specific method is: first calculate the projection of w(t) on each phase encoding vector mj:

[0181] pj(t) = mj^T * w(t) , j=1...N2 (3).

[0182] A decoding function g is then used to map these projections back to the K-dimensional original feature space:

[0183] z(t) = g([p1(t),p2(t),...,pN2(t)]) (4).

[0184] Where g can be any function approximator similar to f, and the inverse mapping from the phase codebook to the original feature space is learned through training. The decoded z(t) is the original representation of the second feature set with dimension K.

[0185] Through the above method, the present application successfully decoded two levels of original feature sequences {x(t)} and {z(t)} from the unified spatiotemporal coding representation O(t). Next, it is necessary to conduct cognitive analysis on these two feature sequences to explain the intrinsic structure and functional properties of EEG signals. Considering that {x(t)} reflects the spatial distribution characteristics of the parallel group, the present application can perform brain region segmentation and pattern recognition tasks for this sequence, and associate EEG activity with functional areas on the cerebral cortex; while {z(t)} reflects the temporal evolution law of the series group, so the present application can perform brain region connectivity and dynamic causal modeling analysis on it, and clarify the information flow path between different brain regions.

[0186] Specifically, for {x(t)}, this application first extracts a suitable time window as the analysis unit. For example, for tasks evaluating working memory processing, [0,1000]ms can be set as the time period of interest. Then, x(t) within this time period is averaged to obtain an average feature vector x:

[0187] x = (1 / L) * Σt=1:L x(t) (5).

[0188] Where L is the length of the time window. Next, the present application trains a classifier, such as a support vector machine (SVM), using x as input, marking different functional areas on the cerebral cortex (such as the prefrontal lobe, parietal lobe, etc.) as output categories y, thereby learning a mapping f:x→y. For new test data, the present application first calculates its average feature vector, and then uses the trained classifier f to predict the functional area, thereby completing the brain area segmentation task. In addition, the present application can also perform similar supervised or unsupervised pattern recognition analysis, such as using a clustering algorithm to divide x into multiple clusters, each cluster corresponding to a potential brain activity pattern. This analysis method based on the first feature set helps the present application understand the spatial encoding principle of EEG signals.

[0189] For the {z(t)} sequence, this application focuses on the temporal interaction pattern between parallel groups in the series group. A feasible method is to use a dynamic Bayesian network to model {z(t)}, and learn the conditional independence and causal relationship between brain regions. Specifically, let z(t)=[z1(t),z2(t),...,zK(t)]^T. This application first needs to determine the directed acyclic graph (DAG) topology of the network, that is, which brain regions are parent nodes and which are child nodes. This can be automatically learned from the data through causal discovery algorithms (such as PC, IC*, etc.), or manually specified based on existing neuroscience knowledge. After determining the DAG, this application uses the observed data in the {z(t)} sequence to estimate the conditional probability distribution parameters of each node through structural learning algorithms such as expectation maximization, so as to finally obtain a dynamic Bayesian network model.

[0190] In this model, each random variable zi(t) corresponds to the time-varying state of a brain region, and the variables in its parent node set Pa(i) reflect the factors that affect the evolution of the state of the brain region, which may include the state of other brain regions at the previous moment, or the state of the node itself at the previous moment. The dynamic Bayesian network uses a state transition probability P(zi(t)|Pa(i),t) to model the state of each node, indicating the probability that the node zi itself will transition to a new state when the parent node takes a certain state configuration.

[0191] Through the dynamic Bayesian network model obtained through learning, the present application can not only make out-of-sample predictions for the sequence {z(t)}, but more importantly, it can analyze the causal semantics in the network and reveal the interaction patterns between different brain regions. For example, if there is a directed edge from node zj to zi, it means that brain region j is activated before brain region i in time and can affect the state evolution of i, reflecting a "j→i" information flow path. The weight of the edge reflects the intensity of this influence. By examining the directed edges and their weights in the network, the present application can reconstruct the entire brain region connectivity topology and information flow mechanism.

[0192] In general, the branch feature fusion stage first separates the encoding of the primary and secondary feature sets from the unified spatiotemporal encoding representation, and uses the corresponding decoder to reconstruct them into the original feature sequence. Then, cognitive analysis tasks such as brain region segmentation, pattern recognition, temporal modeling, and causal discovery are performed on the two sequences respectively, integrating the explanatory knowledge at these two levels, and finally achieving a comprehensive understanding of the EEG signal encoding principle and neural computing mechanism.

[0193] In order to implement the multi-level EEG signal analysis method based on spatiotemporal coding corresponding to the above method embodiment, so as to achieve the corresponding functions and technical effects. Figure 2 , Figure 2 The structure block diagram of a multi-level EEG signal analysis device 200 provided in an embodiment of the present application is shown. For the convenience of description, only the parts related to the present embodiment are shown. The multi-level EEG signal analysis device 200 provided in an embodiment of the present application includes:

[0194] The data acquisition module 201 is used to acquire multiple EEG channel data to be analyzed;

[0195] A data grouping module 202, for dividing the EEG channel data into parallel groups or serial groups according to the channel information corresponding to the EEG channel data;

[0196] A first construction module 203 is used to preprocess each EEG channel data in the parallel group and the serial group, extract multi-scale time domain and frequency domain features from each EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time domain and frequency domain features;

[0197] A second construction module 204 is used to extract a time-dependent feature from each of the EEG channel data in the series group, and construct a second feature set according to the time-dependent feature;

[0198] A primary and secondary determination module 205, configured to determine a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0199] The fusion completion module 206 is used to divide the main feature set and the secondary feature set into row and column coding branches, obtain the EEG signal feature representation according to the row and column coding branches, and complete the cognitive analysis according to the EEG signal feature representation.

[0200] The above-mentioned multi-level EEG signal analysis device 200 can implement the multi-level EEG signal analysis method based on spatiotemporal coding of the above-mentioned method embodiment. The options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of the present application can refer to the contents of the above-mentioned method embodiment, and will not be repeated in this embodiment.

[0201] Figure 3 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. Figure 3 As shown, the computer device 3 of this embodiment includes: at least one processor 30 ( Figure 3 Only one is shown in the figure), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 implements the steps of any of the above method embodiments when executing the computer program 32.

[0202] The computer device 3 may be a computing device such as a smart phone, a tablet computer, a desktop computer, a cloud server, etc. The computer device may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will understand that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0203] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0204] In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 31 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0205] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0206] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in the above-mentioned method embodiments when executing the computer device.

[0207] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.

[0208] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0209] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A multi-level EEG signal analysis method based on spatiotemporal coding, characterized in that: include: Acquire multiple EEG channel data to be analyzed; Dividing the EEG channel data into parallel groups or serial groups according to channel information corresponding to the EEG channel data; Extracting multi-scale time domain and frequency domain features from each of the EEG channel data of the parallel group, and constructing a first feature set according to the multi-scale time domain and frequency domain features; Extracting a time-dependent feature from each of the EEG channel data of the series group, and constructing a second feature set according to the time-dependent feature; Determine a primary feature set and a secondary feature set according to the first feature set and the second feature set; The main feature set and the secondary feature set are divided into row and column coding branches, and the feature representation of the electroencephalogram signal is obtained according to the row and column coding branches, so as to complete the cognitive analysis according to the feature representation of the electroencephalogram signal.

2. The method according to claim 1, characterized in that: Before determining the primary feature set and the secondary feature set according to the first feature set and the second feature set, the method further includes: generating a time coding period according to the first feature set and the second feature set; The first feature set and the second feature set are respectively position-coded according to the time coding period.

3. The method according to claim 1, characterized in that: The step of dividing the EEG channel data into parallel groups or series groups according to the channel information corresponding to the EEG channel data comprises: Determine the brain functional area corresponding to each of the EEG channel data according to the channel information; Dividing the EEG channel data having the same brain functional area into a subset; Based on the optimized grouping method of spectral clustering and spatiotemporal constraints, the EEG channel data corresponding to each subset is divided into parallel groups or serial groups.

4. The method according to claim 3, characterized in that The step of dividing the EEG channel data corresponding to each subset into parallel groups or series groups comprises: Calculating the functional connection strength corresponding to each of the subsets; constructing a weighted undirected graph according to the functional connection strength; Performing spectral clustering on the weighted undirected graph, dividing the EEG channel data corresponding to each subset into a first functional subset and a second functional subset; The EEG channel data of the first functional subset and the second functional subset are calculated according to a preset objective function to divide the EEG channel data of the first functional subset and the second functional subset into parallel groups or series groups; the objective function includes functional constraints and spatial constraints.

5. The method according to claim 1, characterized in that The determining of the primary feature set and the secondary feature set according to the first feature set and the second feature set comprises: Calculating a first importance score of the first feature; Calculating a second importance score corresponding to the second feature; the importance score is used to determine the importance of the first feature set and the second feature set to spatiotemporal coding; A primary feature set and a secondary feature set are determined according to the first importance score value and the second importance score value.

6. The method according to claim 1, characterized in that The dividing into row and column coding branches according to the main feature set and the secondary feature set includes: Allocating each EEG coding data corresponding to the main feature set to a row branch; Allocating each EEG encoding data corresponding to the sub-feature set to a column branch; The encoding period is set according to the primary and secondary relationship between the primary feature set and the secondary feature set; The row branches and column branches are respectively encoded according to the encoding period, and row encoding branches corresponding to the row branches and column encoding branches corresponding to the column branches are obtained to generate the row and column encoding branches.

7. The method according to claim 6, characterized in that The step of obtaining the characteristic representation of the EEG signal according to the row and column coding branch comprises: Obtaining a row coding high level activation interval corresponding to the row coding branch; Obtaining a column code high level activation interval corresponding to the column code branch; The row coding branch and the column coding branch are alternately activated according to the row coding high level activation interval and the column coding high level activation interval, and the feature representation of the electroencephalogram signal is generated by fusion.

8. The method according to claim 1, characterized in that: Before extracting multi-scale time domain and frequency domain features from each of the EEG channel data of the parallel group, the method further includes: Preprocessing is performed on each EEG channel data in the parallel group and the serial group; the preprocessing includes at least one or more of filtering and denoising, standardization, and outlier correction.

9. A multi-level EEG signal analysis device, characterized in that: include: A data acquisition module, used to acquire multiple EEG channel data to be analyzed; A data grouping module, used for dividing the EEG channel data into parallel groups or series groups according to the channel information corresponding to the EEG channel data; A first construction module is used to preprocess each EEG channel data in the parallel group and the serial group, extract multi-scale time domain and frequency domain features from each EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time domain and frequency domain features; A second construction module is used to extract a time-dependent feature from each of the EEG channel data in the series group, and construct a second feature set according to the time-dependent feature; A primary and secondary determination module, used to determine a primary feature set and a secondary feature set according to the first feature set and the second feature set; The fusion completion module is used to divide the main feature set and the secondary feature set into row and column coding branches, obtain the EEG signal feature representation according to the row and column coding branches, and complete the cognitive analysis according to the EEG signal feature representation.

10. A computer device, characterized in that: The method comprises a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the method according to any one of claims 1 to 8 when executing the computer program.

Images