Multilevel electroencephalogram signal analysis method, device and equipment based on spatio-temporal coding

Abstract

The present application discloses a multi-level electroencephalogram (EEG) signal analysis method, device, and equipment based on spatio-temporal coding. The method includes: obtaining multiple EEG channel data to be analyzed; dividing the EEG channel data into a parallel group or a serial group according to the channel information corresponding to the EEG channel data; extracting multi-scale time-domain and frequency-domain features from each EEG channel data in the parallel group, and constructing a first feature set according to the multi-scale time-domain and frequency-domain features; extracting temporal dependence features from each EEG channel data in the serial group, and constructing a second feature set according to the temporal dependence features; determining a main feature set and a secondary feature set according to the first feature set and the second feature set; dividing them into row-column coding branches according to the main feature set and the secondary feature set, and obtaining the EEG signal feature representation according to the row-column coding branches, so as to complete cognitive analysis according to the EEG signal feature representation.

Description

Technical Field

[0001] The present application relates to the field of brain-computer interface technology, and in particular to a multi-level EEG signal analysis method, device and equipment based on spatiotemporal coding. Background Art

[0002] Traditional EEG signal processing methods have many shortcomings. Some methods only focus on the time domain waveform characteristics of the signal, such as studying cognitive processing by detecting the waveform characteristics of event-related potentials (ERP); other methods focus on extracting the frequency characteristics of the signal and using the power spectrum or phase synchronization of different frequency bands to analyze the activation patterns of various brain regions. Since these methods only focus on a single time domain or frequency domain perspective, they cannot fully describe the temporal and spatial coding rules inherent in EEG signals.

[0003] Some methods attempt to combine temporal and spatial information, such as using blind source separation (BSS) to mine the spatiotemporal distribution of signal sources, or using dynamic causal models (DCM) to analyze effective connections between brain regions. However, due to the use of linear models or overly simplified frameworks, these methods have obvious deficiencies in explaining complex non-stationary neural dynamic systems, and it is difficult to accurately characterize the temporal evolution and causal flow laws of brain regions. In addition, most existing methods lack analysis strategies for different levels and modalities, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in their ability to explain generalization.

[0004] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the invention

[0005] The embodiment of the present application provides a multi-level EEG signal analysis method, device and equipment based on spatiotemporal coding, which aims to solve the problem of using linear models or overly simplified frameworks. Such methods have obvious deficiencies in explaining complex non-stationary neurodynamic systems and it is difficult to accurately describe the temporal evolution and causal flow laws between brain regions. In addition, most existing methods lack analysis strategies for different levels and modes, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in the ability to explain generalization.

[0006] In a first aspect, the present application provides a multi-level EEG signal analysis method based on spatiotemporal coding, comprising:

[0007] Acquire multiple EEG channel data to be analyzed;

[0008] Dividing the EEG channel data into parallel groups or serial groups according to channel information corresponding to the EEG channel data;

[0009] Extract multi-scale time-domain and frequency-domain features from each of the EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time-domain and frequency-domain features;

[0010] Extract time-sequence dependence features from each of the EEG channel data in the series group, and construct a second feature set according to the time-sequence dependence features;

[0011] Determine a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0012] Divide into row-column coding branches according to the primary feature set and the secondary feature set, and obtain a feature representation of the EEG signal according to the row-column coding branches, so as to complete cognitive analysis according to the feature representation of the EEG signal.

[0013] In a second aspect, the present application further provides a multi-level EEG signal analysis device, including:

[0014] A data acquisition module, configured to acquire multiple EEG channel data to be analyzed;

[0015] A data grouping module, configured to divide the EEG channel data into a parallel group or a series group according to the channel information corresponding to the EEG channel data;

[0016] A first construction module, configured to preprocess each of the EEG channel data in the parallel group and the series group, extract multi-scale time-domain and frequency-domain features from each of the EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time-domain and frequency-domain features;

[0017] A second construction module, configured to extract time-sequence dependence features from each of the EEG channel data in the series group, and construct a second feature set according to the time-sequence dependence features;

[0018] A primary-secondary determination module, configured to determine a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0019] A fusion completion module, configured to divide into row-column coding branches according to the primary feature set and the secondary feature set, obtain a feature representation of the EEG signal according to the row-column coding branches, so as to complete cognitive analysis according to the feature representation of the EEG signal.

[0020] In a third aspect, the present application further provides a computer device, including a processor and a memory, where the memory is used to store a computer program, and when the computer program is executed by the processor, the multi-level EEG signal analysis method based on spatio-temporal coding as described in the first aspect is implemented.

[0021] Fourthly, the present application also provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the method for analyzing multi-level electroencephalogram signals based on spatio-temporal coding as described in the first aspect.

[0022] Compared with the prior art, the present application has at least the following beneficial effects:

[0023] 1. Hierarchical spatio-temporal coding reflects the hierarchical relationship between primary and secondary features:

[0024] This method adopts an innovative idea of hierarchical coding, maps the primary and secondary feature sets to the row and column coding branches respectively, and realizes seamless switching and hierarchical fusion between the primary and secondary features through alternating high-level activation in time. Specifically, during the coding period of the primary feature set, the row branch always occupies the high-energy output of coding, while the column branch is in a silent state; when entering the coding stage of the secondary feature set, the situation is reversed. Through tensor product construction, the outputs of the two branches are naturally fused into a unified fourth-order tensor representation, which not only retains the prominent position of the primary feature but also completely expresses the detailed information of the secondary feature, effectively solving the problem of information loss caused by hard coding fragmentation in the existing methods.

[0025] 2. Adaptive time allocation realizes coding fairness:

[0026] The coding periods of the primary and secondary feature sets are not specified by hard coding, but are dynamically determined through an adaptive time allocation strategy, thus realizing fairness in coding time between the two. This strategy automatically allocates reasonable coding time resources according to the actual dimensions of the primary and secondary feature sets, avoiding the coding imbalance problem caused by fixed time allocation. This design not only ensures the dominance of the primary feature but also gives sufficient time resources to the secondary feature, making the expression form of spatio-temporal coding more balanced and comprehensive.

[0027] 3. Unified tensor coding is conducive to parallel computing:

[0028] The final spatio-temporal coding output is a fourth-order tensor, where two modes correspond to the primary feature coding and the other two modes correspond to the secondary feature coding, with a compact and orderly coding structure. This tensor representation form is not only conducive to retaining the hierarchical relationship between the primary and secondary features, but also more conducive to subsequent parallel computing and pattern recognition tasks. In fact, tensor decomposition and tensor networks have become current research hotspots, which can greatly improve the computing efficiency while retaining the data structure and correlation.

[0029] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Description of the Drawings

[0030] Figure 1Schematic flowchart of the multi-level EEG signal analysis method based on spatio-temporal coding shown in the embodiments of the present application;

[0031] Figure 2 Schematic structural diagram of the multi-level EEG signal analysis device shown in the embodiments of the present application;

[0032] Figure 3 Schematic structural diagram of the computer device shown in the embodiments of the present application. Detailed implementation manners

[0033] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0034] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0035] It should also be understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0037] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0038] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0039] The technical solution of the embodiment of the present application is introduced below.

[0040] As a bioelectric signal that directly reflects the activity of brain neuron groups, EEG signals have important research value in the fields of neuroscience, cognitive psychology, and artificial intelligence. Compared with other brain imaging technologies (such as functional magnetic resonance imaging (fMRI), EEG signals have extremely high temporal resolution of milliseconds, which can capture rapid dynamic changes in the brain in real time, such as cognitive processing, decision-making, and emotional fluctuations. At the same time, EEG signals also have a certain spatial resolution and can distinguish the activity patterns of different brain regions in the cerebral cortex. Therefore, by analyzing and decoding EEG signals, we can deeply understand the information encoding mechanism inside the human nervous system, reveal the neural basis of the brain's advanced cognitive functions, and provide theoretical guidance and technical support for cognitive computing research in the field of artificial intelligence.

[0041] Traditional EEG signal processing methods have many shortcomings. Some methods only focus on the time domain waveform characteristics of the signal, such as studying cognitive processing by detecting the waveform characteristics of event-related potentials (ERP); other methods focus on extracting the frequency characteristics of the signal and using the power spectrum or phase synchronization of different frequency bands to analyze the activation patterns of various brain regions. Since these methods only focus on a single time domain or frequency domain perspective, they cannot fully describe the temporal and spatial coding rules inherent in EEG signals.

[0042] Some methods attempt to combine temporal and spatial information, such as using blind source separation (BSS) to mine the spatiotemporal distribution of signal sources, or using dynamic causal models (DCM) to analyze effective connections between brain regions. However, due to the use of linear models or overly simplified frameworks, these methods have obvious deficiencies in explaining complex non-stationary neural dynamic systems, and it is difficult to accurately characterize the temporal evolution and causal flow laws of brain regions. In addition, most existing methods lack analysis strategies for different levels and modalities, and cannot fully integrate multi-source heterogeneous information, so there are certain limitations in their ability to explain generalization.

[0043] Please refer to Figure 1, Figure 1 It is a schematic flowchart of a multi-level electroencephalogram (EEG) signal analysis method based on spatio-temporal coding provided by an embodiment of the present application. The multi-level EEG signal analysis method based on spatio-temporal coding in the embodiment of the present application can be applied to computer devices, including but not limited to devices such as smart phones, laptop computers, tablet computers, desktop computers, physical servers, and cloud servers. As Figure 1 shown, the multi-level EEG signal analysis method based on spatio-temporal coding in this embodiment includes steps S101 to S106, which are described in detail as follows:

[0044] Step S101, obtain multiple EEG channel data to be analyzed.

[0045] Specifically, first prepare an EEG acquisition system, including a headband or a mesh electrode array integrating multiple electrodes. This headband or array uses dry electrodes or gel electrode designs, and can form a good electrical connection with the scalp without using additional conductive paste. Dry electrodes are made of metal alloys or conductive polymer materials, while gel electrodes are wrapped with a layer of conductive gel around the electrodes to enable them to fit the scalp curve.

[0046] Modern headbands mostly adopt designs with adjustable size and position, which can adapt to different head shapes, and at the same time correctly align the electrodes with the standard positions specified by the international 10-20 system, such as Fp1, Fp2, F3, Fz, F4, C3, Cz, C4, P3, Pz, P4, O1, O2, etc., covering the main brain regions of the prefrontal lobe, parietal lobe, temporal lobe, and occipital lobe. Some headbands also include additional electrode positions to provide higher spatial resolution. In addition, reference electrodes and ground electrodes are required, usually located at specific positions on the headband or separate connection points, for providing potential reference and noise reduction.

[0047] After preparation, correctly wear the headband with integrated electrodes on the head. For a dry electrode headband, only ensure good contact between the electrodes and the scalp. For a gel electrode headband, squeeze an appropriate amount of conductive gel at each electrode to enhance the coupling degree. However, modern gel electrodes adopt reusable or disposable designs, and do not require too much operation. Next, correctly connect the reference electrode and the ground electrode.

[0048] Then turn on the power of the EEG acquisition device according to the instructions, wait for self-check and establish a wireless data transmission connection. Depending on different devices, it may be necessary to start the acquisition by means of buttons, gestures, or voices, etc. During the formal acquisition, the present application should try to keep the head from making large movements as much as possible, reduce artifacts caused by blinking or muscle activities, and at the same time avoid the influence of external electromagnetic interference sources, although the environmental requirements for ordinary usage scenarios are relatively loose.

[0049] The acquisition device will transmit the EEG data to dedicated software or applications on mobile devices or personal computers in real time for display, storage, and preliminary analysis. Some devices also provide real-time feedback such as brain wave visualization and attention scoring to help this application understand the brain state. The acquisition time can be determined according to requirements until sufficient data is collected. In addition, according to different usage scenarios and purposes, this application needs to adjust the acquisition strategy and parameter settings to optimize the data quality and information content. For example, if this application wants to evaluate an individual's attention or working memory ability, a reasonable strategy is: set the corresponding behavioral paradigm, keep the sampling rate at about 500 Hz to obtain sufficient time resolution; the number of electrodes is between 32 and 64, with a focus on the prefrontal and parietal regions; add an additional reference electrode on the tip of the nose to improve the detection sensitivity of frontal lobe activity; and set an appropriate band-pass filter to filter out interference components with too low or too high frequencies.

[0050] Step S102, according to the channel information corresponding to the EEG channel data, divide the EEG channel data into parallel groups or serial groups.

[0051] Specifically, assuming that this application has collected data from N EEG channels, usually first, these N channels need to be divided into different subsets according to the brain functional regions, such as the prefrontal region, parietal region, temporal region, etc. Each subset contains all the channels corresponding to the corresponding brain region. This preliminary grouping based on spatial location helps this application better capture and interpret the activity patterns of each brain region. Next, for each subset, this application proposes an optimized grouping method based on spectral clustering and spatio-temporal constraints to divide the channels into parallel groups and serial groups.

[0052] In some embodiments, the dividing the EEG channel data into parallel groups or serial groups according to the channel information corresponding to the EEG channel data includes: determining the brain functional region corresponding to each EEG channel data according to the channel information; dividing the EEG channel data with the same brain functional region into a subset; and based on an optimized grouping method of spectral clustering and spatio-temporal constraints, dividing the EEG channel data corresponding to each subset into parallel groups or serial groups.

[0053] First, for all channel pairs within each subset, the present application calculates the functional connection strength between them and constructs a weighted undirected graph of NxN, where N is the number of channels within the subset, and the weight of the edge reflects the degree of functional correlation between two channels. The functional connection strength can be quantified in various ways, such as phase locking value, coherence, mutual information, etc. Which specific metric to adopt needs to be selected according to the actual data and application scenarios. Next, the present application performs spectral clustering on this weighted undirected graph to divide highly correlated channels into the same functional subset. Spectral clustering is an effective graph partitioning algorithm that can divide nodes (channels) into different clusters based on the eigenvectors of the Laplacian matrix of the graph, such that the similarity between nodes within a cluster is relatively high, and the similarity between nodes in different clusters is relatively low. Through spectral clustering, the present application will obtain two functional subsets, each containing a group of channels with strong functional connections.

[0054] Exemplarily, the step of dividing the EEG channel data corresponding to each of the subsets into a parallel group or a serial group includes: calculating the functional connection strength corresponding to each of the subsets; constructing a weighted undirected graph according to the functional connection strength; performing spectral clustering on the weighted undirected graph to divide the EEG channel data corresponding to each of the subsets into a first functional subset and a second functional subset; calculating the EEG channel data of the first functional subset and the second functional subset according to a preset objective function to divide the EEG channel data of the first functional subset and the second functional subset into a parallel group or a serial group; the objective function includes a functional constraint and a spatial constraint.

[0055] After obtaining the two functional subsets, the goal of the present application is to further divide the channels into parallel groups and serial groups within each functional subset, while fully considering the proximity relationship of the channels in terms of spatial position. To this end, the present application proposes a spatio-temporal constraint optimization algorithm to fuse the functional connection and spatial position information and automatically determine the optimal grouping scheme. Specifically, the present application constructs an objective function, which includes two parts: a functional constraint term and a spatial constraint term. The functional constraint term is used to maximize the functional correlation between channels within a group, while the spatial constraint term is used to minimize the dispersion of channels within a group in terms of spatial position. By optimizing this objective function, the present application can find a grouping scheme that not only maintains a high functional connectivity but also has a good spatial aggregation.

[0056] This application first defines the functional constraint term Ffunc, which includes the sum of the functional connection strengths between pairs of channels in the same group and the sum of the functional connection strengths between pairs of channels in different groups, corresponding to the within-group connection and the between-group connection respectively. The goal is to maximize the within-group connection and minimize the between-group connection, and its expression is: Ffunc = Σi,j∈Gk Wij - λΣi∈Gk,j∉GkWij, where Gk (k = 1 or 2) represents the k-th group, Wij is the functional connection strength between channel i and channel j, and λ is the balance parameter.

[0057] Next, it defines the spatial constraint term Fspatial, which includes the sum of the Euclidean distances between pairs of channels in the same group. The goal is to minimize the spatial dispersion between channels within the group: Fspatial = Σi,j∈Gk dij, where dij is the Euclidean distance between channel i and channel j in three-dimensional space.

[0058] By combining the functional constraint term and the spatial constraint term, this application obtains the objective function: F = Ffunc - μFspatial, where μ is the weight coefficient of the spatial constraint term, used to balance the trade-off between functional connectivity and spatial aggregation.

[0059] Now the goal of this application is to find a grouping scheme G1 and G2 within each functional subset such that the objective function F reaches the maximum value. In this problem, this application needs to find an optimal scheme for dividing channels into parallel group G1 and series group G2 within each functional subset, so that the objective function F = Ffunc - μFspatial reaches the maximum value. Among them, Ffunc is the functional constraint term, used to maximize the sum of the functional connection strengths of channels within the group and subtract the sum of the functional connection strengths of channels between groups; Fspatial is the spatial constraint term, used to minimize the dispersion of channels within the group in terms of spatial positions. This application will use the Genetic Algorithm to solve this combinatorial optimization problem. The Genetic Algorithm is an optimization algorithm that simulates the process of biological evolution. Through operations such as selection, crossover, and mutation of individuals in the population, new solutions are continuously generated and their fitness is evaluated, gradually converging to the optimal solution.

[0060] First of all, this application needs to encode the problem and represent the candidate solution (i.e., the channel grouping scheme) in a form suitable for genetic operations. A common encoding method is binary encoding. For example, for a functional subset containing 8 channels, this application can use a binary string of length 8 to represent a grouping scheme, where 0 means the channel belongs to the parallel group and 1 means it belongs to the series group. For example, the encoding 01011010 corresponds to dividing channels 1, 4, 5, and 8 into the parallel group and the remaining channels into the series group.

[0061] Next, this application initializes a population consisting of N individuals (candidate solutions), which are usually randomly generated. Then, the objective function value F of each individual is calculated as the basis for evaluating its fitness. The higher the fitness, the closer the individual is to the optimal solution.

[0062] In each generation of the genetic process, this application first performs a selection operation based on the fitness values of the individuals, selecting several individuals as parents. The higher the fitness of an individual, the greater the probability of being selected. Common selection methods include roulette wheel selection, ranking selection, tournament selection, etc.

[0063] After selecting the parent individuals, this application performs a crossover operation on them to generate new offspring individuals. Crossover is the most crucial operation in the genetic algorithm, which simulates the chromosome exchange process in biological evolution. For individuals with binary encoding, this application can adopt single-point crossover or multi-point crossover. Taking single-point crossover as an example, this application randomly selects two parent individuals and a crossover point, and exchanges the encoding segments of the two parents before this crossover point, thus generating two new offspring individuals. For example, if the parents are 01011010 and 10100101 and they crossover at the 4th position, the offspring are 0110101 and 1001010.

[0064] After the crossover is completed, this application also needs to perform a mutation operation on the offspring individuals to maintain the diversity of the population and avoid premature convergence. The mutation operation is achieved by changing some genes (bit positions) in the individual encoding. Common mutation methods include basic bit mutation (randomly reversing some bits), uniform mutation (changing some bits to random values), etc. The probability of the mutation operation is relatively low, usually between 0.01 and 0.1.

[0065] After operations such as selection, crossover, and mutation, this application obtains a new generation of offspring population. Then this application calculates the fitness of each new individual and selects and replaces some of the parent individuals with lower fitness according to the fitness values, thus generating a new generation of mixed population.

[0066] Repeat the above process, continuously performing multiple generations of genetic operations until the termination conditions are met (such as reaching the maximum number of generations of evolution, fitness convergence, etc.). Finally, the individual with the highest fitness in the population corresponds to the (approximate) optimal solution of the combinatorial optimization problem, that is, the best EEG channel grouping scheme.

[0067] This grouping method based on spectral clustering and spatio-temporal constraint optimization can automatically balance the two key factors of functional connectivity and spatial proximity, so as to obtain the best parallel group and series group divisions. It overcomes the limitations of manually designed grouping rules and can adaptively determine the optimal grouping strategy according to the data characteristics. At the same time, it also avoids the problem of grouping imbalance caused by only considering a single factor (such as functional connectivity or spatial position), thus promising to obtain a more efficient and comprehensive representation of EEG signals.

[0068] In some embodiments, before extracting multi-scale time-domain and frequency-domain features from each of the EEG channel data in the parallel group, it further includes: preprocessing each of the EEG channel data in the parallel group and the series group; the preprocessing includes at least one or more of filtering and denoising, normalization, and outlier correction.

[0069] For the original EEG data of each channel group (including the parallel group and the series group), first use a notch filter to remove narrowband noise at a specific frequency. For example, set the notch center frequency to 50 or 60 Hz to eliminate power line noise, and set the stopband width according to specific circumstances. Next, use a band-pass filter to retain only the frequency range of interest. For example, design a FIR or IIR band-pass filter with a lower passband boundary of 0.5 Hz and an upper passband boundary of 40 Hz, and set the transition band width according to specific requirements. In this way, the EEG band signals in the range of 0.5 - 40 Hz can be retained, and the interference components in the low-frequency and high-frequency bands can be filtered out. In addition, this application can also use blind source separation methods such as independent component analysis to extract and remove the influence of physiological artifacts such as electrooculogram and electromyogram.

[0070] After filtering and denoising, it is necessary to perform baseline correction on the EEG data. Due to factors such as DC drift and instrument offset, the baseline of the original data is usually not 0. This application can estimate the baseline signal using methods such as piecewise linear fitting or wavelet estimation within a certain time window, and then subtract it from the original data to normalize the baseline of the denoised EEG signal to near 0. Next, it is necessary to perform re-referencing processing on the data. During acquisition, this application usually sets a reference electrode, and the potentials of all other electrodes are measured relative to this reference electrode. However, a single reference may introduce additional artifacts. Therefore, this application recalculates the potentials of all electrodes relative to the average reference, or can also use reference electrode estimation techniques to estimate an ideal reference, or directly perform Laplacian or surface Laplacian transformation on the original data to achieve a reference-free state.

[0071] Then, standardization processing is required. Since there are significant differences in EEG amplitudes between different channels and at different time points, in order to eliminate this scale effect, this application can perform Z-Score standardization on each channel separately, that is, first calculate the channel mean and standard deviation, and then subtract the mean and divide by the standard deviation; or perform Min-Max standardization to scale the data to the 0-1 interval. During the preprocessing process, some severely damaged channels may be removed. To ensure spatial resolution, spherical interpolation or other interpolation algorithms need to be used to reconstruct these missing channels based on the data of surrounding channels. For example, the formula for spherical interpolation is x(r, theta, phi)=sum(gi*Vi(r, theta, phi)), where gi is the interpolation coefficient and Vi is the spherical chi-square basis function.

[0072] In addition, even after processing such as filtering, there may still be outliers in the EEG data due to reasons such as instrument failures. This application can detect, correct, or delete these outliers based on the absolute median deviation. The specific method is for each channel data xi, first calculate its median medi, and then calculate the median absolute deviation med_abs_dev = medi(|xi - medi|). Any data point with a deviation from medi greater than k times med_abs_dev (k is usually taken as 6) is regarded as an outlier. Finally, if EEG analysis is performed for a specific cognitive task, the continuous EEG data also needs to be segmented into multiple time periods according to the experimental paradigm, and the data of all time periods corresponding to each cognitive process condition are averaged to extract the evoked potential or rhythm features of interest. The time window and baseline correction period need to be set according to the specific task.

[0073] Step S103, extract multi-scale time-domain and frequency-domain features from the EEG channel data of each of the parallel groups, and construct a first feature set according to the multi-scale time-domain and frequency-domain features.

[0074] Specifically, the goal of parallel feature extraction is to use the EEG channel data in the parallel groups to extract multi-scale time-domain and frequency-domain features and form a first feature set. This application first separates the relevant EEG channels of each parallel group from the preprocessed EEG data, and then applies wavelet packet analysis to these parallel channels for multi-scale time-frequency decomposition, so as to extract rich time-domain and frequency-domain features.

[0075] Wavelet packet analysis is an extension of wavelet analysis. It not only further decomposes the low-frequency components but also can perform deeper decomposition on the high-frequency components, thereby obtaining the time-frequency information of the entire frequency band. Specifically, in this application, a suitable wavelet packet basis, such as the Daubechies wavelet db4, is selected to perform wavelet packet transform on the EEG data of each parallel channel. Assuming the wavelet packet decomposition level is J = 6, then 2^(J + 1) - 1 = 127 wavelet packet coefficient sequences will be obtained, and each sequence corresponds to a specific time-frequency sub-band.

[0076] During the wavelet packet decomposition process, first, the EEG signal is decomposed into an approximation component A1 and a detail component D1, corresponding to the low-frequency and high-frequency parts. Then, both A1 and D1 are further decomposed into two parts, generating four new sequences A2, D2, AD2, and DD2. These four sequences are then decomposed again, and so on, until the pre-set maximum decomposition level J = 6 is reached. Finally, this application obtains 63 approximation and detail coefficient sequences (Aj,k and Dj,k, j = 1...6, k = 1...2^j), and each sequence covers the behavior of the signal in a specific time-frequency interval.

[0077] For each wavelet packet coefficient sequence, this application extracts the following time-domain and frequency-domain features:

[0078] 1. Energy feature Ej: Calculate the energy of the coefficients of this sequence, that is, E(jk) = sum(|C(jk)|^2). Where C(jk) is the coefficient sequence of this sequence, and |·| takes the absolute value. The energy feature reflects the energy distribution of the signal in the corresponding time-frequency band.

[0079] 2. Entropy feature Hj: Calculate the energy entropy of the coefficients of this sequence as a feature to measure the complexity of the signal in this time-frequency band. The higher the entropy value, the more uniform the distribution of the signal in this band and the greater the uncertainty.

[0080] 3. Standard deviation Sj and peak feature Pj: Calculate the standard deviation std(C(jk)) and peak value (maximum value / minimum value) of the coefficients of this sequence respectively, which reflect the distribution of the time-frequency coefficients in terms of magnitude.

[0081] 4. Drop coefficient Rj: Calculate the drop coefficient of the power spectrum of the coefficients of this sequence, which is used to characterize the attenuation characteristics of the spectrum outside this time-frequency band, that is, Rj = sum(|C(f>fh)|^2) / sum(|C(f)|^2), where fh is the upper boundary frequency of this time-frequency band.

[0082] In addition, this application also extracts the correlation features between adjacent time-frequency bands:

[0083] 1. Adjacent correlation coefficient rjk: For each coefficient sequence C(jk), calculate its correlation coefficients with the four adjacent sequences C(j-1,k), C(j-1,k+1), C(j,k-1), and C(j,k+1) as the features characterizing the time-frequency correlation. Multiple definitions such as Pearson correlation and mutual information can be used for the correlation coefficient.

[0084] In the above way, this application obtains a large number of features describing the time-domain and frequency-domain behaviors of the signal from wavelet packet decomposition. These features not only cover the entire time-frequency band but also contain information in multiple dimensions such as energy, entropy, statistics, and correlation, and can comprehensively characterize the dynamic characteristics of EEG signals at various time-frequency scales.

[0085] The above features are arranged in the order of time-frequency bands to form the first feature set. It should be noted that the dimension of the first feature set is equal to the number of extracted features multiplied by the number of time-frequency bands after wavelet packet decomposition (635 dimensions in this example, which is 127*5). This feature set contains the behavioral patterns of each EEG channel in the parallel group at different time scales and frequency scales.

[0086] Step S104, extract the time-sequence dependence features from the EEG channel data of each of the series groups, and construct a second feature set according to the time-sequence dependence features.

[0087] Specifically, use the series group to extract the time-sequence dependence features to obtain the second feature set. Different from the parallel group that captures the synchronous patterns in space, the goal of the series group is to analyze the time-sequence interaction and causal dependence relationships between the parallel groups. The channels in the series group are arranged in a certain functional topology order, and each channel represents a potential brain region or network node. Therefore, the features of the series group actually reflect the dynamic state of the time-sequence information flow between brain regions.

[0088] This application will use the Hidden Markov Model (HMM) to model the feature sequences of each parallel group in the series group, so as to extract the time-sequence dependence features. HMM is a statistical Markov model that can well describe random sequences containing hidden unknown states and has been widely used in fields such as speech recognition and gene sequencing. In neural signal processing, this application can regard the series group as a non-stationary random process generated by multiple hidden states, and use HMM to automatically mine these hidden states and their evolution laws to reveal the time-sequence interaction patterns between different brain regions.

[0089] Specifically, assume that the tandem group contains M parallel groups, and each parallel group is represented by a P-dimensional feature vector in the first feature set. Then the entire tandem group can be described by a sequence of P*M-dimensional feature matrices {O(t)}, where O(t)=[O1(t),O2(t),...,OM(t)] is the observation matrix at the t-th moment. The goal of this application is to establish an N-state HMM λ=(A,B,π), where A is an NxN state transition probability matrix, B is the observation probability distribution, and π is the initial state probability vector.

[0090] In the HMM, the hidden state {X(t)} of the system follows a Markov chain, that is, the current state only depends on the previous state, and the state transition probability is:

[0091] P(X(t+1)=j|X(t)=i,...,X(0)=x0) = P(X(t+1)=j|X(t)=i) = aij

[0092] where aij is an element of the state transition matrix A. The probability distribution of the observation matrix O(t) is determined by the output probability of the hidden state X(t). Assume that O(t) follows a Gaussian Mixture Model (GMM):

[0093] B = {bj(O(t)) = Σk=1^M cjk * N(Oj(t); μjk, Σjk)}, j=1...N

[0094] where bj(O(t)) is the conditional probability density function of the observation O(t) under the hidden state j, which is composed of the weighted sum of M Gaussian components, cjk is the coefficient of the k-th Gaussian mixture model under the state j, satisfying Σk cjk=1, and N(·;μ,Σ) is the Gaussian distribution density function with mean μ and covariance Σ.

[0095] Using the observation sequence {O(t)} and the known HMM parameters λ=(A,B,π), this application can calculate the probability P(O|λ) of the observation sequence under this model using the classical forward-backward algorithm or Viterbi algorithm, and further estimate the optimal parameter values through the Baum-Welch expectation maximization algorithm, that is, the training process of the HMM.

[0096] After the HMM training converges, this application first extracts the following features from the obtained state transition matrix A and initial state π:

[0097] 1. Stationary distribution πs: This distribution reflects the proportion of each state of the HMM, and thus reflects the average contribution of the activation of different brain regions. Let I be the identity matrix, and πs = (I-A+1)^(-1) * π.

[0098] 2. Spectral radius ρ(A): The largest absolute eigenvalue of the state transition matrix A, which reflects the time scale of the HMM system. The larger ρ(A) is, the stronger the system's ability to maintain the initial state.

[0099] 3. Entropy feature Hs: Calculate the entropy Hs = -Σi πs(i)logπs(i) of the state sequence {X(t)}, which measures the uncertainty and complexity of the system.

[0100] 4. Intersection time τ: Calculate the average time τ required for the system to reach the stationary distribution from any state, which reflects the stationary property of the system.

[0101] 5. Absorbing state count Na: Count the number of absorbing states (i.e., states that no longer transfer once entered) in the transition matrix. The more Na there are, the more unstable the system tends to be.

[0102] Then, this application analyzes the parameters of the observed Gaussian mixture model GMM under each hidden state j:

[0103] 6. Mean vector {μjk} and covariance matrix {Σjk}: The mean vector reflects the average activation pattern of the tandem group under the hidden state j; the covariance matrix describes the variability of activation and the covariance structure between channels.

[0104] 7. Mixture coefficient {cjk}: Reflects the contribution weights of different Gaussian components under the hidden state j, and can be used to distinguish multiple potential activation sub-patterns.

[0105] 8. Mahalanobis distance dM(j,j'): For any two hidden states j and j', calculate the Mahalanobis distance between their GMM parameters, which is used to quantify the separability of different states.

[0106] 9. BIC value: Calculate the Bayesian information criterion BIC for the GMM under each hidden state, which is used to balance the model complexity and the degree of data fitting. The smaller the BIC, the more reliable the pattern.

[0107] The features extracted from the state transition structure and the observation model of the HMM capture the temporal evolution law of the feature sequences of the parallel groups in the tandem group, and reflect the interaction patterns and causal dependencies between different brain regions. Combining these time series dependent features in order forms the second feature set.

[0108] Step S105, determine the main feature set and the secondary feature set according to the first feature set and the second feature set.

[0109] Specifically, compare the quantities of the first and second feature sets to determine the primary and secondary feature sets. The primary feature set will serve as the main line for spatio-temporal coding, dictating the rhythm and pace of the entire coding process; while the secondary feature set provides a strong complement to the primary feature set, enriching the representation form and enhancing the detailed description of spatio-temporal features. Therefore, this application needs to comprehensively judge which one should be used as the primary feature set based on the respective characteristics and importance levels of the first and second feature sets.

[0110] In some embodiments, before determining the primary and secondary feature sets according to the first and second feature sets, it further includes: generating a time coding period according to the first and second feature sets; performing positioning coding on the first and second feature sets respectively according to the time coding period.

[0111] The first feature set is multi-scale time-domain and frequency-domain features extracted using parallel groups, which can comprehensively depict the dynamic behavior of signals at different time scales and frequency scales, and has a higher dimension. Specifically, if wavelet packet analysis is used, the dimension of the first feature set is equal to the number of extracted features multiplied by the number of time-frequency bands after wavelet packet decomposition. Assuming 5 features are extracted and the wavelet packet decomposition layer is 6, then the dimension of the first feature set is 635 dimensions. The second feature set is to model the serial group using a hidden Markov model (HMM) to extract time-sequence dependence features, with a lower dimension but containing extremely rich time dynamics information, showing the evolution logic of the parallel group features in time. Taking 9 features and 10 hidden states as an example, the dimension of the second feature set is 90 dimensions.

[0112] Given that the dimensions of the two feature sets differ greatly, if they are directly juxtaposed as the input of time coding, it will cause the coding time period to bias towards the first feature set, thus unable to fully reflect the time-sequence dependence relationship contained in the second feature set. To balance the weights of the two feature sets in time coding, this application proposes an adaptive time period setting method:

[0113] Let the dimension of the first feature set be D1, the dimension of the second feature set be D2, and the total period of time coding be T, where the coding of the first feature set lasts for T1 time steps, the coding of the second feature set lasts for T2 time steps, and T1 + T2 = T. This application needs to determine appropriate T1 and T2 such that the contribution degrees of the two feature sets in time coding are equivalent.

[0114] Considering that although the second feature set has a lower dimension, it contains extremely valuable time dynamics information. This application hopes that it is encoded at least once within the entire time coding period, that is, T2 >= 1. At the same time, the coding step T1 of the first feature set should not be too small to avoid being unable to fully express the time-domain and frequency-domain details therein. Therefore, this application can set a threshold α such that T2 takes the maximum value within the interval [1, αT], and the corresponding T1 = T - T2.

[0115] Specifically, let T2 = argmax(D2, α*T), then T1 = T - T2. Here, argmax(x, y) represents taking the larger value between x and y. If α is set to 0.5, it means that the second feature set occupies at most half of the total coding period, and the remaining time steps will be allocated to the first feature set. Through this method, the present application can adaptively determine T1 and T2, thereby achieving a weight balance between the two feature sets in time coding.

[0116] Suppose the total coding period T = 100, the dimension of the first feature set D1 = 635, the dimension of the second feature set D2 = 90, and α = 0.5. Then: T2 = argmax(90, 0.5 * 100) = 50, T1 = 100 - 50 = 50.

[0117] That is to say, the second feature set will be encoded in the first 50 time steps, and the first feature set will be encoded in the last 50 time steps, and their weights in time coding are equivalent.

[0118] Based on the set time period, position coding is performed on the first and second feature sets.

[0119] The purpose of position coding is to map the feature vector to a high-dimensional position vector, endowing each feature with coding in terms of position and time. By projecting the internal feature state onto the external position vector, the present application can explicitly model the distribution of features in the spatio-temporal domain, thereby capturing the potential spatio-temporal structure.

[0120] For the first feature set, the present application adopts a position coding scheme based on the Radial Basis Function (RBF). RBF coding can construct arbitrary response functions through translation, scale transformation, and combination operations, thereby effectively expressing the activation patterns of input features at various positions and times.

[0121] Specifically, assume that the dimension of the first feature set is M, the dimension of the encoded position vector is N (N >> M), and the time coding period is T1. The present application first initializes N RBF bases, each represented by an N-dimensional position vector μi (i = 1...N), and these vectors are randomly and uniformly distributed in the N-dimensional space. Then, for the M-dimensional feature vector x(t) = [x1(t),..., xM(t)]^T at the t-th moment (t = 1...T1), the present application calculates its distance to each RBF base:

[0122] ri(t) = ||x(t) - μi||, i = 1...N (1).

[0123] where ||·|| represents the Euclidean distance. Then, each distance is non-linearly encoded through a Gaussian radial basis function g(ri) to obtain an N-dimensional position vector:

[0124] y(t) = [g(r1(t)), g(r2(t)), ..., g(rN(t))]^T (2).

[0125] where g(r) = exp(-r^2 / 2σ^2), and σ is the standard deviation parameter of the RBF kernel, which is used to control the smoothness of the response.

[0126] Through the above encoding process, each M-dimensional feature vector x(t) is mapped to an N-dimensional position vector y(t), where each dimension represents the activation response of the feature at the corresponding position. Since N is much larger than M, the position vector y(t) has a higher dimension and richer expressive power than the original feature vector x(t), and can capture the distribution pattern of the feature in the spatio-temporal domain.

[0127] For the second feature set, this application adopts a positioning encoding scheme based on phase encoding. Different from the spatial distribution of the parallel groups reflected by the first feature set, the second feature set reflects the evolution logic of the series groups in time. Therefore, this application requires an encoding method that can naturally capture the temporal dynamics. Phase encoding is exactly based on this idea, mapping the feature to a circular position vector, where the change of the phase corresponds to the passage of time, thus implicitly modeling the passage of time.

[0128] Assume that the dimension of the second feature set is K, the time encoding period is T2, and the dimension of the positioning vector is N (N>>K). This application initializes N phase encoding vectors:

[0129] mj = [cos(2πj / N), sin(2πj / N)], j = 1...N (3).

[0130] These vectors are evenly distributed on a unit circle in the N-dimensional Euclidean space. For the K-dimensional feature vector z(t)=[z1(t),...,zK(t)]^T at the t-th moment (t = 1...T2), this application first calculates the length of its projection on each phase encoding vector mj:

[0131] lj(t) = mj^T * z(t), j = 1...N (4).

[0132] Then, Gaussian kernel encoding is performed on these projection lengths:

[0133] w(t) = [g(l1(t)), g(l2(t)), ..., g(lN(t))]^T (5).

[0134] where \(g(l)=\exp(-l^{2} / 2\sigma^{2})\) is the Gaussian kernel function and \(\sigma\) is the smoothing parameter. Through this phase encoding method, each \(K\)-dimensional feature vector \(z(t)\) is mapped to an \(N\)-dimensional position vector \(w(t)\), and the position vector shows the distribution of the temporal features at different phases on the unit circle.

[0135] It should be noted that since the phase encoding vectors are evenly distributed on the unit circle in sequence, when time \(t\) progresses, the activation position of the position vector \(w(t)\) on the circle will also move accordingly, thereby implicitly capturing the passage of time. This makes phase encoding particularly suitable for position encoding of temporal dependence features.

[0136] Through RBF encoding and phase encoding, this application maps the first and second feature sets to position vectors with rich structural and temporal information respectively.

[0137] In some embodiments, determining the primary feature set and the secondary feature set according to the first feature set and the second feature set includes: calculating a first importance scoring value of the first feature; calculating a second importance scoring value corresponding to the second feature; the importance scoring value is used to determine the importance degree of the first feature set and the second feature set for spatio-temporal encoding; determining the primary feature set and the secondary feature set according to the first importance scoring value and the second importance scoring value.

[0138] For the first feature set, assuming its dimension is \(D1\), an importance scoring function \(f1(D1)\) is defined to represent the importance degree of the first feature set for spatio-temporal encoding; for the second feature set, assuming its dimension is \(D2\), the importance score is \(f2(D2)\). Then, this application compares the magnitudes of \(f1(D1)\) and \(f2(D2)\), and determines the feature set with higher importance as the primary feature set.

[0139] The importance scoring function can be designed according to specific tasks and background knowledge, and the following factors can be considered:

[0140] 1. Dimension size: A feature set with a higher dimension often contains more information and is more important for encoding. This application can design the function \(f(D)=D^{\alpha}(\alpha\gt0)\) to directly weight the dimension \(D\).

[0141] 2. Information content: Based on information theory, the information content of the feature set can be defined and used as the importance score. For example, for the first feature set, the information entropy of each feature can be calculated, and then the sum of the entropies of all features is used as \(f1(D1)\). The higher the information entropy, the more information the feature set contains.

[0142] 3. Discriminative ability: According to the specific task, a discriminative ability score can be considered to be defined to measure the discrimination ability of the feature set for the task. For example, in the cognitive analysis task of EEG, based on a small amount of labeled data, the discriminative scores of each feature under different cognitive conditions can be calculated, and the weighted sum of all feature discriminative scores can be used as f1(D1) or f2(D2). The discriminative scores can be calculated based on information gain, chi-square statistic, or other discriminative criteria.

[0143] 4. Subjective weight: In some applications, the present application can assign subjective importance weights to different feature sets according to domain knowledge or expert experience. For example, if the present application pre-assumes that time-series dependent features are more important than frequency-domain features, then f2(D2)>f1(D1) can be set.

[0144] After designing a suitable scoring function, the present application applies it to the first and second feature sets to obtain f1(D1) and f2(D2). Without loss of generality, assume f1(D1)>f2(D2), then the first feature set is determined as the main feature set and the second feature set is determined as the secondary feature set. Of course, if f1(D1)<f2(D2), the opposite determination is made.

[0145] For specific illustration, assume that the dimension of the first feature set D1 = 635 and the dimension of the second feature set D2 = 90. The present application uses dimension weighting as the importance score: f(D)=D. Then: f1(635) = 635, f2(90) = 90. Obviously, f1(635)>f2(90), so the first feature set is determined as the main feature set and the second feature set is determined as the secondary feature set.

[0146] In addition, the present application can also introduce an importance threshold τ. Only when the importance difference between the main feature set and the secondary feature set exceeds τ, it is confirmed. Otherwise, the importance of the two is regarded as equivalent and equal-weight parallel coding is performed. For example, let τ = 100. In the above example: f1(635) - f2(90) = 545 > τ. Therefore, it is reasonable to confirm that the first feature set is the main feature set and the second feature set is the secondary feature set. If the difference is less than τ, further analysis or adjustment of the scoring function may be required. Through the above method, the present application can adaptively determine the main and secondary feature sets.

[0147] In some embodiments, the dividing into row and column coding branches according to the main feature set and the secondary feature set includes: allocating each EEG coding data corresponding to the main feature set to the row branch; allocating each EEG coding data corresponding to the secondary feature set to the column branch; setting an encoding period according to the primary and secondary relationships corresponding to the main feature set and the secondary feature set; encoding the row branch and the column branch respectively according to the encoding period to obtain the row coding branch corresponding to the row branch and the column coding branch corresponding to the column branch, so as to generate the row and column coding branches.

[0148] The goal of the encoding control stage is to map the primary and secondary feature sets to the row and column encoding branches respectively, set reasonable encoding periods according to their primary and secondary relationships, so as to achieve the alternating high-level output of the two branches in time. Through this alternating timing modulation mechanism, the present application can reflect the information of both the primary and secondary feature sets in spatio-temporal encoding, ensuring the balanced integration of different-level features.

[0149] Exemplarily, obtaining the electroencephalogram signal feature representation according to the row-column encoding branches includes: obtaining the row encoding high-level activation interval corresponding to the row encoding branch; obtaining the column encoding high-level activation interval corresponding to the column encoding branch; alternately activating the row encoding branch and the column encoding branch according to the row encoding high-level activation interval and the column encoding high-level activation interval, and fusing to generate the electroencephalogram signal feature representation.

[0150] Assume that the first feature set is determined as the primary feature set and the second feature set is the secondary feature set. The present application allocates the primary feature set encoding to the row branch and the secondary feature set encoding to the column branch. Specifically, let the dimension of the primary feature set be D1, and after position encoding, an N1-dimensional position vector sequence {y(t)} is generated, and the time encoding period is T1; the dimension of the secondary feature set is D2, and an N2-dimensional position vector sequence {w(t)} is generated, and the time encoding period is T2, where T1 + T2 = T, and T is the total period of the entire spatio-temporal encoding. The goal of the present application is to adjust T1 and T2 so that the primary row branch and the secondary column branch alternately occupy the encoding high level in time, thereby fully reflecting the primary-secondary hierarchical relationship between the two.

[0151] First, define the activation degrees of row encoding and column encoding, which are respectively represented by two real number sequences {arow(t)} and {acol(t)} in the 0-1 interval, where:

[0152] arow(t) = 1 if t ∈ [Trow_start, Trow_end] else 0 (1) acol(t) = 1 if t ∈ [Tcol_start, Tcol_end] else 0 (2).

[0153] Among them, [Trow_start, Trow_end] is the time interval of row encoding activation, and [Tcol_start, Tcol_end] is the time interval of column encoding activation. The present application needs to set these intervals so that the row and column encodings are alternately activated, that is:

[0154] Trow_start = 1, Trow_end = T1 Tcol_start = T1 + 1, Tcol_end = T1 + T2 = T.

[0155] Therefore, when t ∈ [1, T1], arow(t) = 1 and acol(t) = 0, indicating that only the row encoding is active; while when t ∈ [T1 + 1, T], acol(t) = 1 and arow(t) = 0, indicating that only the column encoding is active. In this way, the present application successfully implements the time-alternating mechanism of row and column encodings.

[0156] Next, the present application defines the high-level outputs of row and column encodings. For the t-th moment, the high-level output of row encoding is a K1xN1 matrix:

[0157] Orow(t) = arow(t) * y(t) * ones(1, N1) (3).

[0158] Where y(t) is the N1-dimensional position vector of the main feature set at the t-th moment, and ones(1, N1) is a 1×N1 all-ones matrix. Then each column of Orow(t) is arow(t) * y(t). When arow(t) = 1, the value of Orow(t) is equal to the value of y(t); when arow(t) = 0, Orow(t) is a zero matrix, indicating that the row encoding is in a silent state.

[0159] Similarly, for the t-th moment, the high-level output of column encoding is a K2xN2 matrix:

[0160] Ocol(t) = acol(t) * w(t)^T * ones(N2, 1) (4).

[0161] Where w(t) is the N2-dimensional position vector of the secondary feature set at the t-th moment, and ones(N2, 1) is an N2×1 all-ones column vector. Then each row of Ocol(t) is acol(t) * w(t)^T. When acol(t) = 1, the value of Ocol(t) is equal to the value of w(t)^T; when acol(t) = 0, Ocol(t) is a zero matrix, indicating that the column encoding is in a silent state.

[0162] Substituting (3) and (4) into (1) and (2), the present application obtains the complete expressions for the high-level outputs of row and column encodings:

[0163] Orow(t) = y(t) * ones(1, N1) if t ∈ [1, T1] (row encoding active) 0 else (row encoding silent).

[0164] Ocol(t) = w(t)^T * ones(N2, 1) if t ∈ [T1 + 1, T] (column encoding active) 0 else (column encoding silent).

[0165] It can be seen that when \(t\in[1, T1]\), the row encoding activates and outputs \(y(t)\); while when \(t\in[T1 + 1, T]\), the column encoding activates and outputs \(w(t)^T\). This time-segmented activation mechanism enables the main feature set (row encoding) and the secondary feature set (column encoding) to smoothly switch during the encoding process, avoiding direct hard cutting, thereby achieving continuous and smooth feature fusion.

[0166] It should be noted that for the time step \(t\), since \(N1\) and \(N2\) are often not equal, the dimensions of \(O_{row}(t)\) and \(O_{col}(t)\) are inconsistent and cannot be directly concatenated. To obtain an encoded output with a unified dimension, this application can expand the dimension through the Tensor Outer Product. Specifically, define the spatio-temporal encoding high-level output as a fourth-order tensor \(O(t)\) of \(K1\times N1\times K2\times N2\):

[0167] \(O(t)=O_{row}(t)\otimes O_{col}(t)\ (5)\).

[0168] Where \(\otimes\) is the tensor product operator, which expands \(O_{row}(t)\) and \(O_{col}(t)\) in the row and column directions. If \(O_{row}(t)\) is a \(1\times N1\) matrix and \(O_{col}(t)\) is a \(K2\times N2\) matrix, then \(O(t)\) is a fourth-order tensor of \(K1\times N1\times K2\times N2\), and the first and second modes correspond to row encoding, and the third and fourth modes correspond to column encoding. This tensor product encoding can well retain the structural information of row and column encoding and reflect the hierarchical relationship between primary and secondary features.

[0169] Finally, it should be noted that in order to make a smoother transition between the primary and secondary feature sets, this application can introduce a transition interval at their junction, making the row and column encodings activate simultaneously for a certain period of time instead of a direct hard switch. Let the length of the transition interval be \(\Delta t\), then the high-level activation interval of the row encoding is corrected to \([1, T1+\Delta t]\), and the high-level activation interval of the column encoding is corrected to \([T1-\Delta t + 1, T]\), and they overlap and activate during the time period of \([T1-\Delta t + 1, T1+\Delta t]\). Through this "soft switching" mechanism, the spatio-temporal encoding will be smoother and more continuous.

[0170] Step S106: Divide the main feature set and the secondary feature set into row and column encoding branches, and obtain the electroencephalogram signal feature representation according to the row and column encoding branches, so as to complete the cognitive analysis according to the electroencephalogram signal feature representation.

[0171] Specifically, first, the present application needs to reconstruct the low-level original signal representations corresponding to the high-level spatio-temporal coding tensor O(t), namely the main feature set position vector y(t) and the secondary feature set position vector w(t). Since the row and column coding branches are alternately activated in time, the reconstruction process is reversible. When arow(t) = 1, the present application directly extracts y(t) from the first and second modes of O(t); when acol(t) = 1, the present application extracts w(t)^T from the third and fourth modes and then transposes it to recover w(t). Specifically, assuming that the four modes of O(t) are indexed starting from 1, we have:

[0172] y(t) = O(t)(:,:,1,1) if arow(t)=1.

[0173] w(t) = O(t)(1,1,:,:)^T if acol(t)=1.

[0174] Where O(t)(:,:,1,1) represents the first and second modes of the tensor O(t), which is a matrix of the same dimension as y(t); O(t)(1,1,:,:) is the K2xN2 matrix formed by the third and fourth modes, and its transpose gives the N2xK2 w(t). In this way, the present application successfully separates the coding representations of the primary and secondary feature sets from the high-level spatio-temporal coding.

[0175] Next, it is necessary to decode y(t) and w(t) to reconstruct the original first feature set {x(t)} and second feature set {z(t)}. For y(t), the present application uses a Radial Basis Function Decoder for decoding. Specifically, first, calculate the response distance from y(t) to each RBF basis μi:

[0176] ri(t) = ||y(t) - μi||, i = 1...N1 (1).

[0177] Then use a decoding function f to map these distances back to the original M-dimensional feature space:

[0178] x(t) = f([r1(t),r2(t),...,rN1(t)]) (2).

[0179] Where f can be a feed-forward neural network, a kernel method, or any other function approximator that can perform a non-linear mapping on the N1-dimensional distance vector. Through training, f can learn the inverse mapping from the RBF codebook to the original feature space, thus completing the decoding process. The decoded x(t) is the original representation of the first feature set, with a dimension of M.

[0180] Similarly, for w(t), this application uses a Phase Decoder to reconstruct the second feature set {z(t)}. The specific method is as follows: First, calculate the projection of w(t) on each phase encoding vector mj:

[0181] pj(t) = mj^T * w(t), j = 1...N2 (3).

[0182] Then use a decoding function g to map these projections back to the K-dimensional original feature space:

[0183] z(t) = g([p1(t), p2(t),..., pN2(t)]) (4).

[0184] Where g can be any function approximator similar to f, and it learns the inverse mapping from the phase codebook to the original feature space through training. The decoded z(t) is the original representation of the second feature set with dimension K.

[0185] Through the above method, this application has successfully decoded two levels of original feature sequences {x(t)} and {z(t)} from the unified spatio-temporal coding representation O(t). Next, cognitive analysis needs to be performed on these two feature sequences to explain the internal structure and functional properties of the EEG signals. Considering that {x(t)} reflects the spatial distribution characteristics of the parallel group, this application can perform brain region partitioning and pattern recognition tasks on this sequence to associate EEG activities with functional regions on the cerebral cortex; while {z(t)} reflects the temporal evolution law of the serial group, so this application can perform brain region connectivity and dynamic causal modeling analysis on it to clarify the information flow path between different brain regions.

[0186] Specifically, for {x(t)}, this application first extracts a suitable time window as the analysis unit. For example, for the task of evaluating the working memory processing process, [0, 1000]ms can be set as the period of interest. Then average x(t) within this period to obtain an average feature vector x:

[0187] x = (1 / L) * Σt = 1:L x(t) (5).

[0188] Among them, L is the length of the time window. Next, this application trains a classifier, such as a Support Vector Machine (SVM), using x as the input and marking different functional regions on the cerebral cortex (such as the prefrontal lobe, parietal lobe, etc.) as the output category y, so as to learn a mapping f: x → y. For new test data, this application first calculates its average feature vector, and then uses the trained classifier f to predict the functional region, thus completing the brain region partitioning task. In addition, this application can also perform similar supervised or unsupervised pattern recognition analysis, such as using a clustering algorithm to divide x into multiple clusters, and each cluster corresponds to a potential brain activity pattern. This analysis method based on the first feature set helps this application understand the encoding principle of EEG signals in space.

[0189] For the sequence {z(t)}, this application focuses on the temporal interaction pattern between parallel groups in the tandem group. A feasible method is to use a Dynamic Bayesian Network to model {z(t)} and learn the conditional independence and causal relationship between brain regions from it. Specifically, let z(t)=[z1(t), z2(t),..., zK(t)]^T. This application first needs to determine the directed acyclic graph (DAG) topological structure of the network, that is, which brain regions are parent nodes and which are child nodes. This can be automatically learned from the data through a causal discovery algorithm (such as PC, IC*, etc.), or manually specified based on existing neuroscience knowledge. After determining the DAG, this application uses the observed data in the sequence {z(t)} and estimates the conditional probability distribution parameters of each node through a structure learning algorithm such as expectation maximization, so as to finally obtain a dynamic Bayesian network model.

[0190] In this model, each random variable zi(t) corresponds to the time-varying state of a brain region, and the variables in its parent node set Pa(i) reflect the factors that affect the state evolution of this brain region, which may include the states of other brain regions at the previous moment, or its own state at the previous moment. The dynamic Bayesian network uses a state transition probability P(zi(t)|Pa(i), t) to model the state of each node, indicating the probability that the node zi itself transfers to a new state when the parent node takes a certain state configuration.

[0191] With the learned dynamic Bayesian network model, this application can not only perform out-of-sample prediction on the sequence {z(t)}, but more importantly, it can analyze the causal semantics in the network and reveal the interaction patterns between different brain regions. For example, if there is a directed edge from node zj to zi, it means that brain region j is activated earlier in time than brain region i and can affect the state evolution of i, reflecting an "j→i" information flow path. The weight of the edge reflects the strength of this influence. By examining the directed edges and their weights in the network, this application can reconstruct the entire brain region connectivity topology and information flow mechanism.

[0192] Generally speaking, in the branch feature fusion stage, the encodings of the primary and secondary feature sets are first separated from the unified spatio-temporal encoding representation, and the corresponding decoders are used to reconstruct them into the original feature sequences. Then, cognitive analysis tasks such as brain region partitioning, pattern recognition, temporal modeling, and causal discovery are respectively performed on the two sequences, and the explanatory knowledge at these two levels is fused, ultimately achieving a comprehensive understanding of the encoding principle of EEG signals and the neural computing mechanism.

[0193] To execute the multi-level EEG signal analysis method corresponding to the above method embodiments to achieve the corresponding functions and technical effects. Refer to Figure 2 , Figure 2 The block diagram of a multi-level EEG signal analysis device 200 provided by an embodiment of the present application is shown. For the sake of convenience of description, only the parts related to this embodiment are shown. The multi-level EEG signal analysis device 200 provided by the embodiment of the present application includes:

[0194] A data acquisition module 201 for acquiring multiple EEG channel data to be analyzed;

[0195] A data grouping module 202 for dividing the EEG channel data into parallel groups or serial groups according to the channel information corresponding to the EEG channel data;

[0196] A first construction module 203 for preprocessing each EEG channel data in the parallel group and the serial group, extracting multi-scale time-domain and frequency-domain features from each EEG channel data in the parallel group, and constructing a first feature set according to the multi-scale time-domain and frequency-domain features;

[0197] A second construction module 204 for extracting time-sequence dependence features from each EEG channel data in the serial group and constructing a second feature set according to the time-sequence dependence features;

[0198] A primary and secondary determination module 205 for determining a primary feature set and a secondary feature set according to the first feature set and the second feature set;

[0199] The fusion completion module 206 is configured to divide into row-column coding branches according to the primary feature set and the secondary feature set, obtain a brain-computer signal feature representation according to the row-column coding branches, so as to complete cognitive analysis according to the brain-computer signal feature representation.

[0200] The above multi-level brain-computer signal analysis device 200 can implement the multi-level brain-computer signal analysis method based on spatio-temporal coding in the above method embodiment. The optional items in the above method embodiment are also applicable to this embodiment, which will not be elaborated here. The remaining content of the embodiment of the present application can refer to the content of the above method embodiment, and will not be repeated in this embodiment.

[0201] Figure 3 It is a schematic structural diagram of a computer device provided in an embodiment of the present application. As Figure 3 shown, the computer device 3 in this embodiment includes: at least one processor 30 ( Figure 3 only one is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and operable on the at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above method embodiments are implemented.

[0202] The computer device 3 may be a computing device such as a smart phone, a tablet computer, a desktop computer, and a cloud server. The computer device may include, but is not limited to, the processor 30 and the memory 31. Those skilled in the art can understand that Figure 3 merely examples of the computer device 3, which do not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0203] The so-called processor 30 may be a central processing unit (CPU). The processor 30 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0204] In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In some other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the computer device 3. Further, the memory 31 may also include both the internal storage unit and the external storage device of the computer device 3. The memory 31 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0205] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0206] An embodiment of the present application provides a computer program product, and when the computer program product runs on a computer device, the computer device is caused to implement the steps in each of the above method embodiments when executed.

[0207] In several embodiments provided by the present application, it can be understood that each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, the program segment, or the part of code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.

[0208] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0209] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only the specific embodiments of this application and is not used to limit the protection scope of this application. It is particularly pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. A multi-level electroencephalogram signal analysis method based on spatio-temporal coding, characterized in that, Including: Obtain multiple electroencephalogram (EEG) channel data to be analyzed; According to the channel information corresponding to the EEG channel data, divide the EEG channel data into a parallel group or a serial group; Extract multi-scale time-domain and frequency-domain features from each of the EEG channel data in the parallel group, and construct a first feature set according to the multi-scale time-domain and frequency-domain features; Extract time-series dependence features from each of the EEG channel data in the serial group, and construct a second feature set according to the time-series dependence features; Determine a main feature set and a secondary feature set according to the first feature set and the second feature set; Divide into row and column coding branches according to the main feature set and the secondary feature set, and obtain a feature representation of the EEG signal according to the row and column coding branches, so as to complete cognitive analysis according to the feature representation of the EEG signal.

2. The method according to claim 1, characterized in that, Before determining the main feature set and the secondary feature set according to the first feature set and the second feature set, it further includes: Generate a time coding period according to the first feature set and the second feature set; Perform location coding on the first feature set and the second feature set respectively according to the time coding period.

3. The method according to claim 1, characterized in that, The step of dividing the EEG channel data into a parallel group or a serial group according to the channel information corresponding to the EEG channel data includes: Determine the brain functional region corresponding to each of the EEG channel data according to the channel information; Divide the EEG channel data with the same brain functional region into a subset; Based on an optimized grouping method of spectral clustering and spatio-temporal constraints, divide the EEG channel data corresponding to each subset into a parallel group or a serial group.

4. The method according to claim 3, wherein The step of dividing the EEG channel data corresponding to each subset into a parallel group or a serial group includes: Calculate the functional connection strength corresponding to each subset; Construct a weighted undirected graph according to the functional connection strength; Perform spectral clustering on the weighted undirected graph, and divide the EEG channel data corresponding to each subset into a first functional subset and a second functional subset; Calculate the EEG channel data of the first functional subset and the second functional subset according to a preset objective function, so as to divide the EEG channel data of the first functional subset and the second functional subset into a parallel group or a serial group; the objective function includes functional constraints and spatial constraints.

5. The method according to claim 1, characterized in that The step of determining the main feature set and the secondary feature set according to the first feature set and the second feature set includes: Calculate a first importance scoring value of the first feature; Calculate a second importance scoring value corresponding to the second feature; the importance scoring value is used to determine the importance degree of the first feature set and the second feature set for spatio-temporal coding; Determine the main feature set and the secondary feature set according to the first importance scoring value and the second importance scoring value.

6. The method according to claim 1, wherein The step of dividing into row and column coding branches according to the main feature set and the secondary feature set includes: Allocate each EEG coding data corresponding to the main feature set to a row branch; Allocate each EEG coding data corresponding to the secondary feature set to a column branch; Set a coding period according to the primary-secondary relationship corresponding to the main feature set and the secondary feature set; Perform coding on the row branch and the column branch respectively according to the coding period, and obtain a row coding branch corresponding to the row branch and a column coding branch corresponding to the column branch, so as to generate the row and column coding branches.

7. The method according to claim 6, wherein Obtaining the electroencephalogram (EEG) signal feature representation according to the row-column coding branch includes: Obtaining the high-level activation interval of the row coding corresponding to the row coding branch; Obtaining the high-level activation interval of the column coding corresponding to the column coding branch; Alternately activating the row coding branch and the column coding branch according to the high-level activation interval of the row coding and the high-level activation interval of the column coding, and fusing to generate the EEG signal feature representation.

8. The method according to claim 1, characterized in that Before extracting multi-scale time-domain and frequency-domain features from each EEG channel data in the parallel group, it further includes: Preprocessing each EEG channel data in the parallel group and the serial group; the preprocessing includes at least one or more of filtering and denoising, normalization, and outlier correction.

9. A multi-level electroencephalogram signal analysis device, characterized in that, It includes: A data acquisition module for acquiring multiple EEG channel data to be analyzed; A data grouping module for dividing the EEG channel data into a parallel group or a serial group according to the channel information corresponding to the EEG channel data; A first construction module for preprocessing each EEG channel data in the parallel group and the serial group, extracting multi-scale time-domain and frequency-domain features from each EEG channel data in the parallel group, and constructing a first feature set according to the multi-scale time-domain and frequency-domain features; A second construction module for extracting time-sequence dependence features from each EEG channel data in the serial group and constructing a second feature set according to the time-sequence dependence features; A primary-secondary determination module for determining a primary feature set and a secondary feature set according to the first feature set and the second feature set; A fusion completion module for dividing into row-column coding branches according to the primary feature set and the secondary feature set, obtaining the EEG signal feature representation according to the row-column coding branch, and completing cognitive analysis according to the EEG signal feature representation.

10. A computer device, characterized in that, It includes a memory and a processor; the memory is used for storing a computer program; the processor is used for executing the computer program and implementing the method according to any one of claims 1 to 8 when executing the computer program.

Images