Dynamic time sequence decoding method for electroencephalogram signals under sensory stimulation

Abstract

The invention belongs to the field of electroencephalogram signal processing, nerve decoding, neuroinformatics and sensory stimulation recognition, and provides a dynamic time sequence decoding method for electroencephalogram signals under sensory stimulation, which comprises the following steps of: preprocessing the electroencephalogram signals acquired in different sensory stimulation processes; then slicing the preprocessed electroencephalogram data by time points, and constructing an instantaneous feature matrix; training a classifier on each time slice by adopting a support vector machine and calculating the area under the working characteristic curve of the subject; and recognizing a time window of which the classification performance is remarkably higher than a random level by combining replacement test based on clusters, so as to obtain characteristic nerve response modes induced by different sensory stimuli. According to the method, an electroencephalogram time sequence decoding method is introduced into sensory stimulation recognition for the first time, the nerve response process can be analyzed on the millisecond scale, a stimulation specific significant time window can be extracted, a universal stimulation recognition template is constructed, and automatic, objective and quantifiable sensory stimulation classification is achieved.

Description

Technical Field

[0001] This invention belongs to the fields of electroencephalogram (EEG) signal processing, neural decoding, neuroinformatics and sensory stimulation recognition, and specifically relates to a dynamic temporal decoding method for EEG signals under sensory stimulation. Background Technology

[0002] Electroencephalography (EEG), as a high-temporal-resolution measurement of neural activity, is widely used to study neural responses evoked by external stimuli such as vision, hearing, smell, and taste. However, traditional EEG analysis mainly focuses on methods such as static power spectral analysis (FFT); frequency band energy analysis (alpha, beta, etc.); ERP average waveform analysis; and time-frequency transform analysis (Wavelet, STFT). These methods all have significant limitations: they cannot capture millisecond-level rapid neural processing; they are difficult to distinguish between early and late processing stages of different stimuli; they lack direct indicators of "classification ability" (such as whether three types of odors can be reliably distinguished); and they are difficult to form the classification templates required for "automatic recognition systems."

[0003] With the development of neurodata science and deep machine learning, "temporal decoding" is gradually becoming a new trend in the field of cognitive neuroscience. This technology trains a classifier separately for each time point, enabling the analysis of neural processing trajectories in milliseconds and obtaining the ability to recognize changes over time.

[0004] However, there is still a lack of general EEG temporal decoding methods that can be directly used for sensory stimulus recognition, neural recognition temporal window extraction methods with statistical significance verification, and sensory neural recognition template construction processes that can be industrialized. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a dynamic temporal decoding method for electroencephalogram (EEG) signals under sensory stimulation, thereby resolving the issues in the prior art. The technical solution adopted by this invention is as follows: A dynamic temporal decoding method for electroencephalogram (EEG) signals under sensory stimulation includes the following steps: Step 1: Collect EEG signals during different sensory stimulation processes, perform bandpass filtering, independent component analysis, artifact removal, and bad channel interpolation on the signals, and align them with the stimulation start time to obtain EEG segments from 1 second before stimulation to 8 seconds after stimulation. ;in For time points, For the number of channels, Number of trials Step 2: Slice the EEG segments time-by-time according to the original sampling rate to construct an instantaneous EEG feature matrix including the number of trials and the number of channels; Step 3: Train the SVM classifier at each time point slice, and use five-fold cross-validation to calculate the receiver operating characteristic (AUC) of the classification performance to form a dynamic classification performance sequence. Step 4: Perform a one-sided t-test on the dynamic classification performance sequence and combine it with the cluster permutation test to obtain a continuous time window in which the AUC is significantly higher than the random level; Step 5: Based on the classification performance characteristics of different sensory stimuli in a continuous time window, construct a sensory stimulus-specific EEG temporal recognition template. Step 6: Match the dynamic decoding features of the unknown stimulus with the EEG timing recognition template to achieve automatic stimulus category identification.

[0006] Furthermore, step 2 includes: Step 2.1, Time axis discretization: If the sampling rate is Then the point in time is defined as: ; Step 2.2, Slice operator definition: That is, the first The EEG matrix at each time point is: ; Step 2.3, output the time series feature set: .

[0007] Furthermore, step 3 includes: Step 3.1, Classifier Construction: ,in Stimulus category labels; Step 3.2, Classification performance calculation: ; Step 3.3, Dynamic Performance Sequence: .

[0008] Furthermore, step 4 includes: Step 4.1, one-sided t-test: ; Step 4.2, Preliminary set of significant time points: ; Step 4.3, cluster consecutive salient points into clusters: the cluster set is represented as ; Step 4.4, Cluster Statistics: ; Step 4.5, Permutation test to determine significance: If: Then cluster This represents a significant time window.

[0009] Furthermore, in step 5, the EEG timing recognition template includes: the start and end times of the continuous time window, the AUC peak position, the window integral value, the dynamic slope, and the corresponding feature vector; For each significant cluster Peak position: Peak value: Window integration: Window slope: ; Finally, construct the template vector: .

[0010] Furthermore, step 6 includes: Step 6.1, given a template for the unknown sample With reference template collection: ; Step 6.2, calculate the Euclidean distance: ; Step 6.3, determine the category: .

[0011] The present invention has the following beneficial effects: (1) This invention reveals neural dynamic features that cannot be detected by traditional methods. For example, the neural differences between light-aroma, strong-aroma, and soy sauce-aroma types are not manifested at the same point in time, but correspond to different stages of sensory processing. This temporal difference cannot be observed by traditional power spectra or topographic maps.

[0012] (2) The dynamic decoding performance of the present invention is significantly higher than that of random levels, and has statistical significance and repeatability. The cluster permutation test is used to ensure that the results are not affected by accidental noise, making the significant time window highly reliable.

[0013] (3) The identification template of the present invention can classify any sensory stimulation that can induce an EEG response.

[0014] (4) The identification template of the present invention has scalability and can be used in the fields of food flavor, smell, human-computer interaction, etc.

[0015] (5) The identification template of this invention can provide the industry with objective and automated sensory evaluation technology. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the dynamic decoding of EEG data in this invention.

[0017] Figure 2 This is a schematic diagram of the machine learning dynamic decoding results of EEG signals induced by different types of baijiu (Chinese liquor) according to the present invention.

[0018] Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] The following will be based on embodiments of the present invention. Figures 1-3 The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0020] A dynamic temporal decoding method for electroencephalogram (EEG) signals under sensory stimulation includes the following steps: Step 1: Collect EEG signals during different sensory stimulation processes, perform bandpass filtering, independent component analysis (ICA) artifact removal, and bad channel interpolation on the signals, and align them with the stimulation start time to obtain standardized EEG segments from 1 second before stimulation to 8 seconds after stimulation. ,in For time points, For the number of channels, Number of trials; Step 2 involves slicing the EEG segments time-by-time according to the original sampling rate to construct an instantaneous EEG feature matrix composed of the number of trials and the number of channels. Step 2 specifically includes: Step 2.1, Time axis discretization: If the sampling rate is Then the point in time is defined as: ; Step 2.2, Slice operator definition: That is, the first The EEG matrix at each time point is: .

[0021] Step 2.3, output the time series feature set: ; Step 3 involves training an SVM classifier at each time slice and calculating the receiver operating characteristic (AUC) of the classification performance using five-fold cross-validation to form a dynamic classification performance sequence. Step 3 specifically includes: Step 3.1, Classifier Construction: ,in For stimulus category labels.

[0022] Step 3.2, Classification performance calculation: .

[0023] Step 3.3, Dynamic Performance Sequence: ; Step 4: Perform a one-sided t-test on the dynamic classification performance sequence and combine it with the cluster permutation test to obtain a continuous time window in which the AUC is significantly higher than the random level.

[0024] Step 4.1, one-sided t-test: .

[0025] Step 4.2, Preliminary set of significant time points: .

[0026] Step 4.3, cluster consecutive salient points into clusters: the cluster set is represented as .

[0027] Step 4.4, Cluster Statistics: .

[0028] Step 4.5, Permutation test to determine significance: If: Then cluster A significant time window; Step 5: Based on the classification performance characteristics of different sensory stimuli within the significant time window, construct a sensory stimulus-specific EEG timing recognition template.

[0029] In step 5, the AUC time history curves of different sensory stimuli within the salient time window exhibit non-overlapping or differential distributions, which are used to construct a stimulus recognition template. The recognition template includes, but is not limited to: the start and end times of the salient window, the peak position of the AUC, the window integral value, the dynamic slope, and a feature vector constructed based on the above data. Specifically, for each salient cluster... Peak position: Peak value: Window integration: Window slope: Finally, construct the template vector: .

[0030] Step 6 involves matching the dynamic decoding features of the unknown stimulus with the recognition template to achieve automatic stimulus category identification. Step 6 specifically includes: Step 6.1, given a template for the unknown sample With reference template collection: .

[0031] Step 6.2, calculate the Euclidean distance: .

[0032] Step 6.3, determine the category: .

[0033] Furthermore, in step 2, the resolution of the time slice is 1 ms, which is consistent with the EEG sampling rate, in order to capture the rapid neural dynamic features induced by the stimulus.

[0034] Furthermore, the SVM classifier in step 3 employs a linear kernel function: This ensures the interpretability of the model, thereby enabling precise analysis of neural dynamic trajectories.

[0035] Furthermore, the significant time window in step 4 is corrected using multiple comparisons based on clustering permutation tests to improve the statistical reliability of time-series decoding.

[0036] This invention is applicable to flavor, odor, taste, tactile stimulation and other sensory stimuli that can induce brain electrical responses.

[0037] The present invention provides the following specific experimental examples: This invention selects soy sauce aroma type, strong aroma type, and light aroma type baijiu as three types of sensory stimuli, and uses a 64-channel EEG acquisition system to collect the electroencephalogram (EEG) signals of subjects during the baijiu tasting process.

[0038] The EEG acquisition and preprocessing steps include: using a 64-channel EEG system with a sampling rate of 1000Hz, and processing according to the following procedure: 1) 0.1–30Hz filtering; 2) ICA blind source separation and removal of eye movement / EMG components; 3) alignment by stimulation time and truncating from -1s to 8s; 4) removal of abnormal epochs and standardization of data.

[0039] The preprocessed EEG data was sliced ​​at a resolution of 1ms to construct a temporal decoding matrix, which is the instantaneous feature matrix of trial number × channel.

[0040] A linear kernel SVM from the Python scikit-learn library is used as the base classifier to ensure good interpretability of the results and reduce computational burden. A diagram illustrating dynamic data decoding is shown below. Figure 1 As shown. To ensure optimal performance of the SVM classifier, the data was standardized. Then, labels were assigned to all trials based on the samples consumed by the evaluators. Each sample was labeled 1, while the other two samples were labeled 0. During decoding, all trial data were divided into five equal parts, which were used sequentially as the test set, and the remaining four parts as the training set for five-fold cross-validation. The AUC value was used as the evaluation metric for classifier performance. At each time point, the five AUC values ​​generated by the five-fold cross-validation were averaged to obtain the AUC time-history curve for each evaluator. Based on this, the AUC time-history curves of the 16 evaluators were averaged to generate the final dynamic curve of the classification performance.

[0041] Temporal feature window identification uses bilateral shading to represent the 95% confidence interval for each time point, and performs one-sided t-tests and cluster-based permutation tests with a significance threshold of 0.01 to obtain time windows with significant classification effects. For example... Figure 2As shown, the dark curve represents the mean level of dynamic decoding performance for each fragrance type, and the light shaded area marks the 95% confidence interval (one-sided t-test). After multiple test correction using cluster-based permutation tests, the time periods when the performance of all classifiers was significantly higher than the random level (0.5) are marked by the thick line at the bottom rounded endpoint. For example: • Light fragrance type: Several significant ranges appear around 5.1–7.6s.

[0042] • Strong aroma type: Significant early differentiation appears at approximately 5.0–5.3s.

[0043] • Soy sauce aroma type: The delay window appears at approximately 6.9–7.2 seconds.

[0044] In this embodiment, for unknown samples, the above steps are repeated to extract their AUC time-series patterns and match them with three fragrance templates to obtain fragrance classification results.

[0045] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, alterations, alterations, or substitutions made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method of dynamic temporal decoding of electroencephalographic signals under sensory stimulation, characterized in that, The method comprises the following steps: Step 1, collect the electroencephalogram signals in the process of different sensory stimulation, band-pass filter the signals, independent component analysis, artifact removal, bad channel interpolation, and align with the starting time of the stimulation as the reference to obtain the electroencephalogram segment from 1s before stimulation to 8s after stimulation ; wherein is the number of time points, is the number of channels, is the number of tests Step 2, slice the EEG segment at each time point according to the original sampling rate, and construct an instantaneous EEG feature matrix comprising the number of trials and the number of channels; Step 3, train the SVM classifier on each time point slice, calculate the AUC of the subject work characteristic by five-fold cross-validation, and form a dynamic classification performance sequence; Step 4, perform one-tailed t-test on the dynamic classification performance sequence, and obtain a continuous time window with AUC significantly higher than the random level by combining cluster permutation test; Step 5, construct an EEG time sequence recognition template specific to the sensory stimulus according to the classification performance characteristics of different sensory stimuli in the continuous time window; Step 6, match the dynamic decoding characteristics of unknown stimuli with the EEG time sequence recognition template to automatically identify the stimulus category.

2. The method of claim 1, wherein, Step 2 comprises: Step 2.1, time axis discretization: if the sampling rate is then the time points are defined as: ； Step 2.2, Slice operator Slice definition: EEG matrix at the th time point is: ; Step 2.3, output timing feature set: .

3. The method of claim 1, wherein the method is a dynamic temporal decoding method of brain electrical signals under sensory stimulation. Step 3 comprises: Step 3.1, Classifier Construction: wherein is the stimulus class label; Step 3.2, Classification performance calculation: ; Step 3.3, Dynamic Performance Sequence: .

4. The method of claim 1, wherein, Step 4 comprises: Step 4.1, one-sided t-test: ; Step 4.2, preliminary set of significant time points: ; Step 4.

3. Clustering consecutive salient points into clusters: The set of clusters is denoted as ; Step 4.4, cluster statistics: ; Step 4.5, replacement test determines significance: if: then cluster is a significant time window.

5. The method of claim 1, wherein, In step 5, the EEG time sequence recognition template comprises: the start and end time of the continuous time window, the AUC peak position, the window integral value, the dynamic slope and the corresponding feature vector; For each salient cluster , peak position: ; peak value: ; window integral: ; window slope: ; Finally, construct the template vector: .

6. The method of claim 1, wherein, Step 6 comprises: Step 6.1, Template for given unknown sample With reference to the set of templates: ; Step 6.2, calculating the Euclidean distance: ; Step 6.3, judging the category: .

Images