Brain electrical based on name evoked consciousness response evaluation model, training and evaluation method

Abstract

The application relates to the field of electroencephalogram processing, in particular to a consciousness response evaluation model based on name-induced electroencephalogram, a training method and an evaluation method. The model comprises a space-time feature extraction module, which is used for receiving multi-channel electroencephalogram signals generated by a tester under the action of a name voice stimulus, and performing space-time feature extraction on the multi-channel electroencephalogram signals to obtain electroencephalogram space-time features; a name voice process coding module, which is used for receiving a name voice process sequence corresponding to the name voice, and coding the name voice process sequence to obtain name condition coding; a name condition time sequence reorganization module, which is used for reorganizing the electroencephalogram space-time features based on the name condition coding to obtain brain state feature information; and a classification module, which is used for classifying the name voice based on the brain state feature information to obtain a classification result. The application has the advantages of effectively capturing the electroencephalogram response difference in the name voice process and improving the accuracy and reliability of the consciousness state evaluation.

Description

Technical Field

[0001] This application relates to the field of electroencephalogram (EEG) signal processing, and in particular to a consciousness response assessment model, training, and assessment method based on name-evoked EEG. Background Technology

[0002] One's own name is an auditory stimulus with high self-reference significance, which can induce changes in event-related potential components such as P200 and P300 and functional connectivity of brain regions. It is an effective neuroelectrophysiological indicator for assessing an individual's ability to process auditory stimuli in a self-related manner. In the assessment of consciousness response, traditional assessment methods are limited for test subjects who cannot stably perform behavioral responses or actively cooperate with tasks. However, the stimulus of one's own name can be presented in a passive state. By comparing the differences in EEG induced by the target name and non-target name, an objective indicator of consciousness response can be obtained.

[0003] However, name-evoked EEG differs significantly from traditional standard sound-evoked EEG. A name is not a momentary stimulus, but a continuous speech process. This is especially true in Chinese names, where two-character, three-character, and multi-character names differ in pronunciation duration, character-level boundaries, syllable structure, tone changes, and semantic integration time.

[0004] Traditional event-related potential (ERP) analysis typically uses the sound initiation point as a uniform lock point and extracts components such as P200 and P300 within a fixed time window. However, in auditory stimuli related to names, the time when the test subject truly recognizes that it is their own name may occur during the surname stage, given name stage, or after the full name is presented, and it is not necessarily fixed to the sound initiation point. If a uniform sound initiation point lock point and a fixed response window are still used, it may lead to temporal misalignment, dilution, or masking of self-related EEG components, thereby affecting the accuracy of extracting differences between target and non-target names and assessing conscious responses. Summary of the Invention

[0005] This application provides a consciousness response assessment model, training, and assessment method based on name-evoked EEG, which can effectively capture differences in EEG responses during the name speech process and improve the accuracy and reliability of consciousness state assessment.

[0006] On one hand, embodiments of this application provide a consciousness response assessment model based on name-evoked EEG, including: The spatiotemporal feature extraction module is used to receive multi-channel EEG signals generated by the test subject under the stimulation of the name voice, and to extract spatiotemporal features from the multi-channel EEG signals to obtain spatiotemporal features of EEG. The name speech process encoding module is used to receive the name speech process sequence corresponding to the name speech, and encode the name speech process sequence to obtain the name condition encoding. The name speech includes target name and non-target name, the name speech process sequence includes acoustic feature information and speech playback process information, and the name condition encoding is used to describe the temporal structure and speech process of the name speech. The name conditional temporal reconstruction module is connected to the spatiotemporal feature extraction module and the name speech process encoding module. It is used to perform name conditional temporal reconstruction on the EEG spatiotemporal features based on the name conditional encoding to obtain brain state feature information. The classification module, connected to the name conditional temporal reconstruction module, is used to classify the name speech based on the brain state feature information to obtain a classification result, wherein the classification result includes target name or non-target name.

[0007] Furthermore, the name voice process encoding module is specifically used for: The received name speech sequence is mapped to name conditional encoding based on the speech process encoder, wherein the speech process encoder adopts at least one of recurrent neural network, long short-term memory network, gated recurrent unit or Transformer.

[0008] Furthermore, the spatiotemporal feature extraction module is specifically used for: The spatiotemporal features of the multi-channel EEG signals are extracted based on multi-scale temporal convolutional layers and spatial convolutional layers, and the spatiotemporal features of EEG are output. The multi-scale temporal convolutional layers use multiple convolutional kernels of different lengths to extract EEG response patterns within different latency ranges, and the spatial convolutional layers are used to learn the spatial combination relationships between different electrodes.

[0009] Furthermore, the name conditional time sequence reorganization module is specifically used for: Multi-scale temporal unfolding of the aforementioned spatiotemporal features of EEG yields EEG temporal features at multiple time scales. Based on the name conditional encoding, the EEG temporal features at each time scale are conditionally modulated to obtain the modulated EEG temporal features at each time scale. The brain state feature information is obtained by fusing the modulated EEG temporal features at multiple time scales.

[0010] Furthermore, based on the name conditional encoding, conditional modulation is performed on the EEG temporal features at each time scale, including: Generate conditional gain coefficients and conditional bias terms based on name conditional coding; The EEG timing features are scaled and adjusted based on the conditional gain coefficient to obtain the processed EEG timing features. The processed EEG timing features are offset and adjusted based on the conditional bias term to obtain the modulated EEG timing features.

[0011] Furthermore, the fusion of modulated EEG temporal features across multiple time scales includes: For each time scale, a query vector is generated based on the name condition code; Generate key vectors and value vectors based on the modulated EEG temporal features; The degree of matching between different modulated EEG time segments and the name speech process at each time scale is calculated based on the attention mechanism; Based on the matching degree, the modulated EEG temporal features are weighted and recombined to obtain recombined information; The brain state feature information is obtained by cross-scale fusion of recombined information from multiple time scales.

[0012] Furthermore, the classification module is specifically used for: The brain state feature information is normalized to obtain normalized features; The normalized features are subjected to a nonlinear mapping layer transformation to obtain the hidden layer features; The feature units in the hidden layer are randomly masked using a Dropout layer and a preset dropout rate. The hidden features processed by the Dropout layer are input into the fully connected classification layer, which outputs the classification result of the target name or non-target name.

[0013] On the other hand, embodiments of this application provide a training method for a consciousness response assessment model based on name-evoked EEG, including: Obtain a training set, wherein each sample in the training set includes a multi-channel EEG signal generated by the test subject under name speech stimulation, a name speech process sequence corresponding to the name speech, and a real label, wherein the real label includes a target name or a non-target name; The training set is input into a consciousness response assessment model based on name-induced EEG as described in any of the first aspects to obtain the predicted classification results; The error value between the predicted classification result and the true label is calculated based on the loss function; Based on the error value, the parameters of the spatiotemporal feature extraction module, the name speech process encoding module, the name conditional temporal reorganization module, and the classification module are updated simultaneously using the backpropagation algorithm. Repeat the steps of inputting the training set, calculating the error value, and updating the parameters until the error value converges or falls within a preset range, thus completing the training of the consciousness response assessment model based on name-evoked EEG and obtaining the trained consciousness response assessment model based on name-evoked EEG.

[0014] Furthermore, it also includes: Add weight decay to the loss function, whereby the weight decay is used to constrain the magnitude of the model parameters.

[0015] Furthermore, embodiments of this application provide a method for assessing conscious response based on name-evoked EEG, including: Acquire the multi-channel EEG signal to be evaluated and the name speech process sequence corresponding to the multi-channel EEG signal to be evaluated; The multi-channel EEG signal to be evaluated and the name speech process sequence are input into a consciousness response evaluation model based on name-evoked EEG, which is trained by the training method described in any of the second aspects, to obtain the classification result.

[0016] This application discloses a consciousness response assessment model, training, and assessment method based on name-evoked EEG. This model uses a name-speech process encoding module to encode the acoustic features and speech playback process information of a target name or non-target name, obtaining a name conditional encoding describing the temporal structure and speech process of the name's speech. Through name conditional encoding, the name conditional temporal reconstruction module no longer uses the traditional fixed-time locking method. Instead, it reconstructs the name conditional temporal sequence based on the actual playback process of the name's speech, using the EEG spatiotemporal features output by the spatiotemporal feature extraction module. This yields brain state feature information that matches the name's speech process. Therefore, multi-channel EEG signals and corresponding name speech process information can be aligned in the time dimension, ensuring that the neural responses corresponding to the key time points for truly recognizing one's own name are preserved and mapped into the brain state features, avoiding component misalignment problems caused by fixed time windows. Furthermore, a classification module classifies the brain state feature information to obtain classification results for target and non-target names. Since the neural response patterns of patients with impaired consciousness to their own names may differ from those of healthy individuals or patients in the recovery phase of consciousness, a consciousness response assessment model based on name-evoked EEG can capture and decode these differences, and then assess consciousness responses based on the classification results, thereby improving the accuracy of consciousness response assessment to some extent. Attached Figure Description

[0017] Figure 1 This is a structural block diagram illustrating a consciousness response assessment model based on name-evoked EEG in the embodiments of this application; Figure 2 This is a flowchart illustrating a training method for a consciousness response assessment model based on name-induced EEG in this embodiment. Figure 3 This is a flowchart illustrating a method for assessing consciousness response based on name-induced EEG in this embodiment. Detailed Implementation

[0018] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0019] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0020] Traditional methods for assessing conscious response face limitations when test subjects cannot actively cooperate. As a continuous phonological stimulus, the name has a complex internal temporal structure. The traditional method of locking the time at the start of the sound and extracting EEG features by fixing the window may cause self-related EEG components to be diluted or masked due to temporal misalignment, thus affecting the accurate extraction of differences between the target name and non-target names and reducing the reliability of conscious response assessment.

[0021] To address the problems of existing technologies, embodiments of this application provide a consciousness response assessment model, training, and assessment method based on name-evoked EEG. The consciousness response assessment model based on name-evoked EEG provided in this application embodiment will be described below first.

[0022] Figure 1 This diagram illustrates a structural block diagram of a consciousness response assessment model based on name-evoked EEG, according to an embodiment of this application. Figure 1 As shown, the consciousness response assessment model 100 based on name-evoked EEG includes the following modules: The spatiotemporal feature extraction module 101 is used to receive the multi-channel EEG signals generated by the test subject under the stimulation of the name voice, and to extract the spatiotemporal features of the multi-channel EEG signals to obtain the spatiotemporal features of the EEG. The spatiotemporal feature extraction module is specifically used to: extract spatiotemporal features from multi-channel EEG signals based on multi-scale temporal convolutional layers and spatial convolutional layers, and output spatiotemporal features of EEG; wherein, the multi-scale temporal convolutional layer uses multiple convolutional kernels of different lengths to extract EEG response patterns within different latency ranges, and the spatial convolutional layer is used to learn the spatial combination relationship between different electrodes.

[0023] In this embodiment, the name speech is recorded in an acoustically controlled environment, pronounced by a trained Mandarin speaker, and captured using professional audio equipment to ensure consistency in speech clarity, loudness, speech rate, and timbre. The name speech includes target names and non-target names. Non-target names include three-character names and two-character names. The duration of a single name stimulus is approximately 600ms. The interval between adjacent stimuli is the time interval between the end of the previous name stimulus and the start of the next name stimulus, which is randomly set to 500–800ms.

[0024] In this embodiment, a multi-channel EEG system is used to collect EEG signals within a preset time range before and after the name-based voice stimulation. For example, EEG segments are collected from 200ms before the start of the name-based voice stimulation to 1000ms or 1200ms after the start of the name-based voice stimulation. The preset time range can be determined according to the sampling rate, the duration of the name stimulation, and the analysis requirements. After acquiring the EEG segments, basic preprocessing is performed on the EEG segments. The basic preprocessing includes filtering, power frequency interference suppression, rereference, baseline correction, abnormal channel processing, and obvious artifact removal, thereby obtaining the multi-channel EEG signals input to the spatiotemporal feature extraction module 101.

[0025] In this embodiment, the multi-channel EEG signal can be represented as: Where C is the number of EEG channels and T is the number of EEG sampling points. This refers to multi-channel EEG signals. Spatiotemporal feature extraction can be represented as: ,in, For EEG spatiotemporal feature extraction, These are spatiotemporal characteristics of brainwaves.

[0026] In this embodiment, the EEG spatiotemporal feature extractor includes a multi-scale temporal convolutional layer and a spatial convolutional layer that are fused in series or parallel. The multi-scale temporal convolutional layer is a convolutional neural network layer that can process information at different time scales simultaneously. It captures features of multi-channel EEG signals within different time windows by using convolutional kernels of different sizes in parallel or in series. In this embodiment, multiple 1D convolutional kernels of different lengths can be used in parallel to act on the time dimension, and the outputs of multiple convolutional kernels can be spliced ​​or fused. Alternatively, multiple 1D convolutional kernels of different lengths can be connected in series to form a multi-layer structure, with each layer processing information at different time scales.

[0027] Among them, convolutional kernels of different lengths focus on the patterns of EEG signals over different durations. For example, shorter convolutional kernels, such as those with a length of 5 or 10, can be used to capture early auditory responses, such as the P200 component; medium-length convolutional kernels, such as those with a length of 20 or 30, can be used to capture latent name recognition responses, such as the P300 component; and longer convolutional kernels, such as those with a length of 50 or 100, can be used to capture late self-related responses.

[0028] Spatial convolutional layers are convolutional layers specifically designed to handle the spatial correlation of multi-channel signals. They learn the spatial combination relationships between different electrodes by performing convolution operations on the electrode channel dimension. For example, electrode channels can be treated as sequences, and 1D convolutional kernels can be used to slide along the channel dimension to capture the relationships between adjacent or specific arrangements of electrodes. When the electrode arrangement can be mapped to a 2D mesh, 2D convolutional kernels can be used to perform convolution on the electrode topology graph to capture local spatial features. In addition, graph convolutional networks can be used to treat electrodes as nodes in a graph and the connections between electrodes as edges in the graph, learning complex spatial dependencies between electrodes through graph convolution operations.

[0029] In this embodiment, the multi-scale temporal convolutional layer captures EEG response patterns within different latencies using convolutional kernels of different lengths, avoiding the masking of multi-channel EEG signals due to temporal misalignment. The spatial convolutional layer learns and integrates the spatial combination relationship between multi-channel electrodes, extracting spatial information of EEG, which to a certain extent improves the representation ability of spatiotemporal features, thereby more accurately distinguishing the differences in EEG induced by target and non-target names, and providing reliable spatiotemporal features of EEG for consciousness response assessment.

[0030] The name speech process encoding module 102 is used to receive the name speech process sequence corresponding to the name speech, and encode the name speech process sequence to obtain the name condition encoding. The name speech includes target name and non-target name, the name speech process sequence includes acoustic feature information and speech playback process information, and the name condition encoding is used to describe the temporal structure and speech process of the name speech. The name speech process encoding module is specifically used to map the received name speech process sequence into name conditional encoding based on the speech process encoder, wherein the speech process encoder adopts at least one of recurrent neural network, long short-term memory network, gated recurrent unit or Transformer.

[0031] In this embodiment, the name speech process includes acoustic feature information and speech playback process information. The acoustic features include Mel spectrum, acoustic envelope, fundamental frequency, and energy. The acoustic feature information can be extracted on a preset time frame using a standard speech processing algorithm. The speech playback process information includes character-level boundaries, syllable boundaries, name progress, and the end point of the complete name. The speech playback process information can be obtained through speech forced alignment technology or manual annotation. The character-level boundaries are the start and end times of the pronunciation of each Chinese character in the name. The syllable boundaries are the start and end times of each syllable or its initial and final parts. The name progress is the relative position of the current speech frame in the entire name pronunciation process, which can be represented by time ratio, character sequence number, or syllable sequence number. The end point of the complete name is the moment when the entire name speech is completely played out.

[0032] For a single name pronunciation, the name pronunciation process sequence can be represented as: Where TS is the length of the name speech sequence. Let be the feature vector of the i-th speech time frame, where the feature vector includes acoustic feature information and speech playback process information.

[0033] In this embodiment, the name condition code can be represented as: ,in, For name voice process encoder, For conditional encoding, S represents the name speech process.

[0034] Among them, the speech process encoder is a computational model or algorithm that converts the sequence of name speech processes into a structured encoded representation for subsequent processing, in order to capture the temporal dynamic information and event points of name speech. For example, a machine learning-based method can be used to train a model to learn the mapping from the original speech features to the name process state.

[0035] In this embodiment, the speech process encoder can employ at least one of recurrent neural networks, long short-term memory networks, gated recurrent units, or Transformers. The recurrent neural network structure processes sequential data through its internal recurrent connections to capture temporal dependencies. The long short-term memory network structure, as a variant of RNN, can effectively solve the gradient vanishing or exploding problem in long sequence training and capture long-distance dependencies. The gated recurrent unit is a simplified version of the long short-term memory network, reducing the number of parameters and improving computational efficiency while maintaining similar performance. The Transformer structure processes all elements in the sequence in parallel through a self-attention mechanism, capturing global dependencies.

[0036] In this embodiment, the name conditional encoding can be a multi-dimensional vector, where different dimensions represent different speech process features. For example, one dimension indicates whether the current stage is the surname stage (Boolean value), another dimension indicates whether the current stage is the given name stage (Boolean value), and yet another dimension indicates the percentage progress of the current name playback (floating-point number). The name conditional encoding can also be a discrete label sequence, with each time step corresponding to a label indicating the name stage at that time step, such as the beginning of the surname, the middle of the surname, the beginning of the given name, the middle of the given name, the end of the complete name, etc., supplemented by time point markers, such as the end of the surname and the end of the given name. In this embodiment, the name conditional encoding includes indication information of whether the current stage is the surname stage, the given name stage, or the complete name presentation stage. For example, the above stages can be represented by one-hot encoding or embedding vectors. The name conditional encoding can also include indication information of whether key time points in the name process have been reached. For example, binary flags can be used to indicate whether the end of the surname, the end of the given name, or the end of the entire name has been reached.

[0037] In this embodiment, the original speech process sequence is converted into a structured name conditional code by the name speech process encoding module, which solves the problem of not being able to effectively capture speech temporal changes and process information. Compared with the traditional fixed time window method, it avoids the misalignment, dilution or masking of self-related EEG components, and improves the accuracy of extracting the difference between the target name and the non-target name and the accuracy of consciousness response assessment to a certain extent. The above-mentioned name conditional code is not used directly as the classification result, but is input as conditional information into the name conditional temporal reconstruction module to guide the subsequent reconstruction, modulation and fusion of EEG spatiotemporal features.

[0038] The name conditional temporal reconstruction module 103 is connected to the spatiotemporal feature extraction module and the name speech process encoding module. It is used to perform name conditional temporal reconstruction on the spatiotemporal features of EEG based on name conditional encoding to obtain brain state feature information. The name-conditional temporal reconstruction module is specifically used for: performing multi-scale temporal unfolding of EEG spatiotemporal features to obtain EEG temporal features at multiple time scales; conditionally modulating the EEG temporal features at each time scale based on name conditional encoding to obtain modulated EEG temporal features at each time scale; and fusing the modulated EEG temporal features at multiple time scales to obtain brain state feature information.

[0039] In this embodiment, the conditional temporal reconstruction module is used to capture the response patterns of name-evoked EEG at different time scales. The response patterns include early auditory responses at shorter time scales, name recognition-related responses at intermediate time scales, and late self-related processing responses at longer time scales. For example, the EEG temporal features obtained at the m-th time scale are as follows: ,in, Let be the temporal unrolling operator at the m-th time scale. The temporal unrolling operator can be implemented by temporal convolutions of different lengths, sliding window unrolling, piecewise rearrangement, or multi-scale temporal reassembly structures. For the temporal features of the EEG at the m-th scale, These are spatiotemporal characteristics of brainwaves.

[0040] The process of conditionally modulating the EEG temporal features at each time scale based on name conditional coding includes: generating conditional gain coefficients and conditional bias terms based on name conditional coding; scaling the EEG temporal features based on the conditional gain coefficients to obtain the processed EEG temporal features; and shifting the processed EEG temporal features based on the conditional bias terms to obtain the modulated EEG temporal features.

[0041] In this embodiment, the complete name speech includes different phonemes, syllables or semantic units in the time process. The amount of self-related information varies at different stages. Therefore, the focus of EEG feature extraction is adjusted according to the progress of the name speech. That is, the EEG feature expression of the corresponding time window is enhanced in the stage with dense self-related information, and suppressed in the stage with sparse information. For example, when the name is first presented, more attention is paid to the early auditory processing related features. When the name speech reaches the stage of complete name or includes a stage with high self-related information, the EEG response features related to target recognition and self-related processing are enhanced.

[0042] For example, the EEG temporal features at the m-th time scale are: The scale feature modulated by name is The conditional modulation process can be represented as: ,in, The conditional gain coefficient, Both are conditional bias terms, and are encoded by name conditions. Dynamically generated, in this embodiment, the conditional gain coefficients and conditional bias terms can be generated by a linear mapping layer or a small neural network, for example, using a multilayer perceptron for name conditional encoding. As input, the conditional gain coefficient and conditional bias term at the corresponding time scale are output respectively. The conditional gain coefficient is used to scale and adjust the EEG time sequence features, and the conditional bias term is used to offset and adjust the EEG time sequence features, so that the modulated EEG features can better match the current name speech process.

[0043] in, This refers to scaling adjustments, which dynamically enhance or suppress parts of the EEG features that are related to or irrelevant to the current name speech information based on the current progress of the speech. For example, during the critical recognition stage of name speech, the gain coefficient of EEG features related to self-recognition is increased, giving them a more important role in subsequent processing; while in non-critical stages, the gain coefficient is decreased to reduce the influence of noise or irrelevant information. In this embodiment, this can be element-wise multiplication or matrix multiplication, depending on the dimensions of the features and coefficients. The offset adjustment calibrates the overall activation level or response pattern of the EEG features based on the current progress of the name speech, so that it is better aligned with the current name speech state. For example, at a certain stage of name speech, the brain exhibits overall excitation or inhibition. The bias term can simulate or compensate for this overall change. In this embodiment, it can be element-wise addition or vector addition to ensure that the modulated EEG features can more accurately reflect the true state of the brain under a specific name speech process.

[0044] The process involves fusing modulated EEG temporal features across multiple time scales, including: generating a query vector based on name conditional encoding for each time scale; generating key and value vectors based on modulated EEG temporal features; calculating the matching degree between different modulated EEG temporal segments and the name speech process at each time scale based on an attention mechanism; weighting and recombining the modulated EEG temporal features based on the matching degree to obtain recombined information; and fusing the recombined information across multiple time scales across scales to obtain brain state feature information.

[0045] Among them, different modulated EEG time sequence segments are multiple local segments obtained by dividing the modulated EEG time sequence features according to time windows, and each modulated EEG time sequence segment corresponds to the EEG response within a time interval.

[0046] In this embodiment, the query vector can be obtained by mapping the name condition code through a fully connected layer, or by performing a nonlinear transformation on the name condition code through a multilayer perceptron. The key vector and value vector can be obtained by mapping the modulated EEG temporal features through two independent fully connected layers, or by first mapping to an intermediate representation through a fully connected layer, and then extracting the key vector and value vector from it.

[0047] In this embodiment, the matching degree can be obtained by calculating the dot product of the query vector and the key vector and then performing Softmax normalization. Alternatively, the matching degree can be calculated using additive attention. The query vector and key vector are concatenated and transformed through a feedforward neural network, and then the matching score is obtained through a scalar output layer and performed Softmax normalization. In this embodiment, the matching degree after Softmax normalization is used as the weight. The weight is multiplied element-wise with the corresponding value vector, and all weighted value vectors are summed to obtain the recombined information.

[0048] In this embodiment, cross-scale fusion can be achieved in several ways. For example, the recombined information from all time scales can be concatenated along the feature dimension, and then a linear transformation can be performed through a fully connected layer after concatenation. Alternatively, a learnable weight can be assigned to the recombined information from each time scale, and the weighted recombined information can be summed. Another method is to use a global query vector, calculate the matching degree between the query vector and the recombined information from all time scales through an attention mechanism, and then perform a weighted summation of the recombined information based on the matching degree, i.e., attention pooling.

[0049] In this embodiment, the fusion of conditionally modulated EEG features from multiple time scales can be represented as follows: ; in, Let M represent the fusion function, and M represent the number of time scales. This represents the final brain state characteristic information obtained. To modulate the temporal characteristics of EEG.

[0050] For example, for the m-th time scale, a query vector is generated using the name speech process encoding, and key and value vectors are generated using conditionally modulated EEG multi-scale features. The matching degree between different EEG time segments and the name speech process at this scale is calculated using attention weights, and the EEG features are weighted and reorganized accordingly. The attention reorganization process can be represented as follows: ; in, For query vector, They are the key vector and the value vector, respectively. This refers to the reorganization information at this time scale.

[0051] Brain state characteristic information can be represented as: ; in, This represents a cross-scale fusion function, which can be implemented using methods such as concatenated linear mapping, weighted summation, attention pooling, or convolutional fusion.

[0052] In this embodiment, by performing multi-scale temporal unfolding of EEG spatiotemporal features, EEG response patterns within different latencies and time windows are comprehensively captured, avoiding the omission of autocorrelated EEG components. Based on name conditional coding, EEG temporal features at each time scale are conditionally modulated, so that the EEG features of each time period can be integrated with the current name speech process information. This makes the target EEG response that matches the current process prominent, suppresses irrelevant interference signals, and achieves alignment between EEG features and the current name speech process. The modulated EEG temporal features of multiple time scales are fused to integrate effective EEG features at different time scales, taking into account the matching results of name speech process and EEG response at different scales, and thus obtaining brain state feature information including the EEG response information corresponding to the complete name process.

[0053] The classification module 104 is connected to the name conditional temporal reconstruction module and is used to classify the speech of a name based on brain state feature information to obtain classification results, which include target names or non-target names.

[0054] The classification module is specifically used for: normalizing brain state feature information to obtain normalized features; performing nonlinear mapping layer transformation on the normalized features to obtain hidden layer features; randomly masking the feature units in the hidden layer features using a Dropout layer and a preset dropout rate; and inputting the hidden layer features processed by the Dropout layer into a fully connected classification layer to output the classification result of the target name or non-target name.

[0055] In this embodiment, the process of the classification module can be represented as follows: ,in, For normalization processing, and These are the hidden layer mapping parameters. It is a non-linear activation function. Used to randomly mask some feature units during training to reduce the risk of overfitting, h d This represents the hidden layer features used for classification.

[0056] In this embodiment, the classification layer outputs the probability distribution of the current trial belonging to a non-target name or the target name based on the hidden layer features: ,in, and Here, P represents the weight matrix and bias term of the classification layer, respectively, and P=[P0,P1] represents the predicted probabilities of the two categories: non-target name and target name.

[0057] The final classification result is determined by the category with the higher probability: ,in, =1 indicates that the current trial is for the target name stimulus. =0 indicates that the current trial is a non-target name stimulus. In this embodiment, the name speech stimulus sequence may include one or more name stimuli. When multiple name stimuli are included, each name stimulus corresponds to an independent event tag and is processed as an independent trial. For example, when "Zhang San" and "Li Si" are presented in sequence in a continuously played name speech stimulus sequence, "Zhang San" corresponds to one trial and "Li Si" corresponds to another trial. The corresponding EEG segments are extracted for each and input into the model for classification.

[0058] In this embodiment, normalization can be implemented in various ways. For example, Min-Max normalization can linearly scale the feature values ​​to a preset interval, such as [0,1] or [-1,1], or Z-score normalization can convert the feature values ​​into a distribution with a mean of 0 and a standard deviation of 1. Decimal Scaling normalization can also be used to adjust the value by moving the decimal point, or Sigmoid normalization can be used to map the feature values ​​to... 0,1 The nonlinear mapping layer can be composed of one or more fully connected layers and combined with nonlinear activation functions, such as ReLU, Sigmoid, and Tanh functions. The nonlinear mapping layer can also be implemented through the hidden layers in a multilayer perceptron.

[0059] A Dropout layer and a preset dropout rate are used to randomly mask feature units in the hidden layer. The preset dropout rate is set as needed, such as 0.3 or 0.5. In practice, Dropout is usually enabled only during the training phase and disabled during the testing or inference phase to ensure the determinism of the results. The hidden layer features processed by the Dropout layer are input into a fully connected classification layer, thereby mapping the features after multiple processing steps to the final explicit classification output. The fully connected classification layer can be one or more fully connected layers, and its last layer selects an appropriate activation function according to the nature of the classification task. For example, the Sigmoid activation function can be used for binary classification tasks, and the Softmax activation function can be used for multi-class classification tasks.

[0060] In this embodiment, scale differences are eliminated by normalizing brain state features, deep features that distinguish target names from non-target names are extracted through nonlinear mapping, some feature units are randomly masked by Dropout to prevent overfitting and improve generalization ability, and the classification result of target name or non-target name is output through a fully connected classification layer, providing a stable basis for consciousness response assessment.

[0061] In this embodiment, the test subject does not need to press buttons, make verbal reports, or cooperate with complex actions. They only need to receive name-evoked speech stimulation to collect EEG data. The target name or non-target name is directly classified through an end-to-end classification model, reducing the process of manually selecting feature windows, manually extracting P300 amplitude, and manually setting classification features. This helps to improve the automation level of name-evoked EEG assessment and is suitable for test subjects with limited behavioral response ability or difficulty in actively cooperating.

[0062] In this embodiment, by using a name speech process encoding module and a name conditional temporal reconstruction module, information such as character-level boundaries, syllable boundaries, name progress, and complete name stages are added to the EEG feature learning process. This enables the consciousness response assessment model based on name-evoked EEG to adaptively learn EEG response patterns according to the name speech process, reducing reliance on manually set P200 / P300 analysis windows. Furthermore, the name conditional temporal reconstruction module uses the name speech process as conditional information to guide the extraction of spatiotemporal features and temporal modeling of EEG, forming brain state feature information for classification. Compared with temporal models that rely solely on EEG features, this approach utilizes the character-level structure and complete name information in the name speech to enhance the expression of EEG features related to target name recognition and self-related processing, thereby improving the separability between target name and non-target name-evoked EEG.

[0063] This application also provides a training method for a consciousness response assessment model based on name-evoked EEG, such as... Figure 2 As shown in the embodiments of this application, a training method for a consciousness response assessment model based on name-evoked EEG includes the following steps: S201, Obtain the training set, wherein each sample in the training set includes the multi-channel EEG signal generated by the test subject under the stimulation of the name voice, the name voice process sequence corresponding to the name voice, and the real label, which includes the target name or non-target name. In this embodiment, EEG data is collected based on the name-based auditory evoked event-related potential paradigm. The name-based auditory evoked event-related potential paradigm is based on the classic Oddball experiment structure. The test subject's own name is set as the target stimulus, and other people's names are set as non-target stimuli, which is used to induce EEG signals related to name recognition, self-related processing, and conscious response.

[0064] To improve the controllability and comparability of name-based stimulus conditions, in this embodiment, with the target stimulus fixed as the test subject's own name, the non-target stimuli are further divided into three-character names and two-character names. By controlling the key variable of the length of the non-target name, the influence of different name phonological structures on the EEG response induced by the target name is determined.

[0065] In this embodiment, four experimental tasks are formed based on the task state and non-target name type: passive 3CC task, where the non-target stimulus is a three-character name and the tester only listens to the stimulus; active 3CC task, where the non-target stimulus is a three-character name and the tester silently counts the target name stimulus; passive 2CC task, where the non-target stimulus is a two-character name and the tester only listens to the stimulus; and active 2CC task, where the non-target stimulus is a two-character name and the tester silently counts the target name stimulus. The above tasks are organized and implemented according to the hierarchical structure of trial-stimulus block-conversation-task. A single trial corresponds to one name speech stimulus and simultaneous EEG recording. Multiple trials form a stimulus block, multiple stimulus blocks form a conversation, and multiple conversations together constitute a complete task. By combining active / passive tasks with two-character / three-character name non-target stimuli, EEG response data under different attentional states and different name speech lengths are obtained simultaneously.

[0066] In this embodiment, target name stimuli and non-target name stimuli are randomly presented within each stimulus block. The interval between adjacent stimuli is the time interval between the end of the previous name stimulus and the start of the next name stimulus. This interval is randomly set to 500-800ms to reduce the test subject's expectation effect on the timing of stimulus appearance. Stimulus event markers and multi-channel EEG signals are recorded synchronously throughout the experiment, so that each EEG trial can be matched with a specific stimulus type, task type, and non-target name character count type. For active tasks, the test subject's counting or behavioral response to the target name stimulus can also be recorded to help verify the effectiveness of the stimulus paradigm. For passive tasks, the test subject's neural processing state of their own name stimulus is mainly assessed by relying on the EEG response itself.

[0067] The name-based speech stimulation materials were all recorded in an acoustically controlled environment, with pronunciation performed by trained Mandarin speakers and captured using professional audio equipment to ensure consistency in speech clarity, loudness, speed, and timbre. After recording, the name-based speech stimulation materials underwent standardized post-processing, including volume normalization, duration calibration, noise control, and acoustic parameter checks, thereby forming a name-based speech stimulation library with relatively uniform parameters and reusable operation. In this embodiment, not only is the starting point of each name-based speech stimulation recorded, but the continuous process of the name speech is also marked, such as the start and end times of each character in the name, syllable boundaries, acoustic envelope, name progress, and the end point of the complete name.

[0068] In terms of EEG acquisition, a multi-channel EEG system is used to record EEG signals within a preset time range before and after name stimulation, such as EEG segments from 200ms before the stimulation start point to 1000ms or 1200ms after the stimulation start point. The preset time range is adjusted according to the sampling rate, name stimulation duration, and analysis requirements. Data such as channel information, sampling rate, event markers, stimulation type, and trial labels are saved. After acquiring the EEG segments, basic preprocessing is performed on the EEG segments, including filtering, power line interference suppression, rereference, baseline correction, abnormal channel processing, and obvious artifact removal, thereby obtaining EEG trial data corresponding to target name and non-target name stimuli.

[0069] In this embodiment, each trial includes a one-to-one correspondence of multi-channel EEG signals, a name speech process sequence, and a real label. A training set, a validation set, and a test set are constructed based on multiple trials, wherein each trial is a sample.

[0070] S202, Input the training set into the consciousness response assessment model based on name-induced EEG to obtain the predicted classification results; In this embodiment, multiple samples are extracted from the training set each time. The multi-channel EEG signals of the multiple samples and the corresponding name speech process sequences are simultaneously input into the consciousness response assessment model 100 based on name-evoked EEG. The consciousness response assessment model based on name-evoked EEG performs forward propagation calculation. The spatiotemporal feature extraction module 101 extracts spatiotemporal features from the multi-channel EEG signals and outputs the spatiotemporal features of the EEG. The name speech process encoding module 102 encodes the name speech process sequence and outputs the name conditional code. The name conditional temporal reconstruction module 103 performs multi-scale temporal reconstruction and conditional modulation on the name conditional code and the spatiotemporal features of the EEG and outputs brain state feature information. The classification module 104 classifies the brain state feature information and outputs the predicted probability distribution P=[P0,P10,P2 ... 1] Where P0 represents the probability that the current trial belongs to a non-target name, and P1 represents the probability that the current trial belongs to the target name; the predicted classification result is determined based on the category with the higher probability.

[0071] S203, calculate the error value between the predicted classification result and the true label based on the loss function; In this embodiment, the loss function used is the cross-entropy loss function: in, CE stands for Cross Entropy, representing the cross-entropy loss value. It measures the difference between the model's predicted probability distribution and the true label distribution. N is the number of training samples, and r is the class index. Let i be the true label of the i-th sample in the r-th class. This is the predicted probability of this category based on a name-evoked EEG consciousness response assessment model.

[0072] In this embodiment, weight decay can also be added to the loss function to constrain the magnitude of the model parameters. The loss function with added weight decay is as follows: Where θ is the parameter set of the consciousness response assessment model based on name-evoked EEG. Here, λ represents the weight decay regularization term, and λ is the regularization weight. In this embodiment, the value range of the regularization weight is 1×10. -5 Up to 1×10 -3 The regularization weights can be selected from a range of values ​​based on the specific dataset size and model complexity.

[0073] S204, based on the error value, simultaneously updates the parameters of the spatiotemporal feature extraction module, the name speech process encoding module, the name conditional temporal reorganization module, and the classification module through the backpropagation algorithm; In this embodiment, based on the loss value, the backpropagation algorithm is used to calculate the gradient of the loss value with respect to each trainable parameter in the consciousness response evaluation model based on name-induced EEG. According to the calculated gradient, the optimizer is used to simultaneously update all parameters in the spatiotemporal feature extraction module 101, the name speech process encoding module 102, the name conditional temporal reconstruction module 103, and the classification module 104. The optimizer is an adaptive moment estimation optimizer.

[0074] For example, the hyperparameters of the adaptive moment estimation optimizer are: learning rate 0.001, exponential decay rate of the first-order moment estimate 0.9, exponential decay rate of the second-order moment estimate 0.999, and numerical stability term 1 × 10⁻⁶. -8 The above update process is performed once during each training session to minimize the loss function value.

[0075] S205, Repeat the steps of inputting the training set, calculating the error value, and updating the parameters until the error value converges or falls within a preset range, thus completing the training of the consciousness response assessment model based on name-evoked EEG and obtaining the trained consciousness response assessment model based on name-evoked EEG.

[0076] In this embodiment, steps S202 to S204 are repeated to perform multiple rounds of iterative training on the training set. After each round of training, the loss value and classification accuracy of the current name-evoked EEG consciousness response assessment model are calculated using the validation set.

[0077] When the loss value on the validation set no longer decreases for multiple consecutive rounds, such as 10 rounds, that is, when the absolute value of the loss change is less than a preset threshold, such as 1×10. -4If the accuracy of the validation set no longer improves, or if the preset maximum number of training rounds is reached, such as 200 rounds, training is stopped. At this point, all parameters of the consciousness response assessment model based on name-evoked EEG are saved, and the trained consciousness response assessment model based on name-evoked EEG is obtained.

[0078] This application also provides a method for assessing conscious responses based on name-evoked EEG, such as... Figure 3 As shown in the figure, the method for assessing consciousness response based on name-evoked EEG provided in this application includes the following steps: S301, acquire the multi-channel EEG signal to be evaluated and the name speech process sequence corresponding to the multi-channel EEG signal to be evaluated; S302, input the multi-channel EEG signal to be evaluated and the name speech process sequence into the trained consciousness response evaluation model based on name-evoked EEG, and obtain the classification result.

[0079] In this embodiment, the multi-channel EEG signal to be evaluated and the corresponding name speech process sequence are input into a trained consciousness response evaluation model based on name-evoked EEG to obtain the classification result. When the classification result is 1, the current trial is determined to be the target name, indicating that an EEG response related to self-name recognition was generated in the current trial. When the classification result is 0, the current trial is determined to be a non-target name, indicating that no EEG response related to self-name recognition was generated in the current trial.

[0080] In this embodiment, a series of name-based verbal stimuli can be played continuously to the test subject, including multiple target name stimuli and non-target name stimuli. For example, a mixed sequence containing 20 target name stimuli and 80 non-target name stimuli. Based on the name-evoked EEG consciousness response assessment model, each trial is independently classified to obtain the following evaluation indicators: Hit rate = Number of attempts where the target name was correctly classified as the target / Total number of attempts for the target name; False positive rate = Number of attempts where non-target names were incorrectly classified as target names / Total number of attempts with non-target names; Classification accuracy = Number of all correctly classified stimuli / Total number of stimuli.

[0081] When the hit rate is greater than the first preset threshold and the false alarm rate is less than the second preset threshold, it is determined that the tester has a stable and decodeable self-name awareness response. For example, the hit rate is >70% and the false alarm rate is <30%. The first and second preset thresholds can be set as needed, but no specific limitation is made.

[0082] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A consciousness response assessment model based on name-evoked EEG, characterized in that, include: The spatiotemporal feature extraction module is used to receive multi-channel EEG signals generated by the test subject under the stimulation of the name voice, and to extract spatiotemporal features from the multi-channel EEG signals to obtain spatiotemporal features of EEG. The name speech process encoding module is used to receive the name speech process sequence corresponding to the name speech, and encode the name speech process sequence to obtain the name condition encoding. The name speech includes target name and non-target name, the name speech process sequence includes acoustic feature information and speech playback process information, and the name condition encoding is used to describe the temporal structure and speech process of the name speech. The name conditional temporal reconstruction module is connected to the spatiotemporal feature extraction module and the name speech process encoding module. It is used to perform name conditional temporal reconstruction on the EEG spatiotemporal features based on the name conditional encoding to obtain brain state feature information. The classification module, connected to the name conditional temporal reconstruction module, is used to classify the name speech based on the brain state feature information to obtain a classification result, wherein the classification result includes target name or non-target name.

2. The consciousness response assessment model based on name-evoked EEG as described in claim 1, characterized in that, The name voice process encoding module is specifically used for: The received name speech sequence is mapped to name conditional encoding based on the speech process encoder, wherein the speech process encoder adopts at least one of recurrent neural network, long short-term memory network, gated recurrent unit or Transformer.

3. The consciousness response assessment model based on name-evoked EEG as described in claim 1, characterized in that, The spatiotemporal feature extraction module is specifically used for: The spatiotemporal features of the multi-channel EEG signals are extracted based on multi-scale temporal convolutional layers and spatial convolutional layers, and the spatiotemporal features of EEG are output. The multi-scale temporal convolutional layers use multiple convolutional kernels of different lengths to extract EEG response patterns within different latency ranges, and the spatial convolutional layers are used to learn the spatial combination relationships between different electrodes.

4. The consciousness response assessment model based on name-evoked EEG as described in claim 1, characterized in that, The name condition time sequence reorganization module is specifically used for: Multi-scale temporal unfolding of the aforementioned spatiotemporal features of EEG yields EEG temporal features at multiple time scales. Based on the name conditional encoding, the EEG temporal features at each time scale are conditionally modulated to obtain the modulated EEG temporal features at each time scale. The brain state feature information is obtained by fusing the modulated EEG temporal features at multiple time scales.

5. The consciousness response assessment model based on name-evoked EEG as described in claim 4, characterized in that, Based on the aforementioned name conditional encoding, conditional modulation is performed on the EEG temporal features at each time scale, including: Generate conditional gain coefficients and conditional bias terms based on name conditional coding; The EEG timing features are scaled and adjusted based on the conditional gain coefficient to obtain the processed EEG timing features. The processed EEG timing features are offset and adjusted based on the conditional bias term to obtain the modulated EEG timing features.

6. The consciousness response assessment model based on name-evoked EEG according to claim 4, characterized in that, The process of fusing modulated EEG temporal features across multiple time scales includes: For each time scale, a query vector is generated based on the name condition code; Generate key vectors and value vectors based on the modulated EEG temporal features; The degree of matching between different modulated EEG time segments and the name speech process at each time scale is calculated based on the attention mechanism; Based on the matching degree, the modulated EEG temporal features are weighted and recombined to obtain recombined information; The brain state feature information is obtained by cross-scale fusion of recombined information from multiple time scales.

7. The consciousness response assessment model based on name-evoked EEG according to claim 1, characterized in that, The classification module is specifically used for: The brain state feature information is normalized to obtain normalized features; The normalized features are subjected to a nonlinear mapping layer transformation to obtain the hidden layer features; The feature units in the hidden layer are randomly masked using a Dropout layer and a preset dropout rate. The hidden features processed by the Dropout layer are input into the fully connected classification layer, which outputs the classification result of the target name or non-target name.

8. A training method for a consciousness response assessment model based on name-evoked EEG, characterized in that, include: Obtain a training set, wherein each sample in the training set includes a multi-channel EEG signal generated by the test subject under name speech stimulation, a name speech process sequence corresponding to the name speech, and a real label, wherein the real label includes a target name or a non-target name; The training set is input into a consciousness response assessment model based on name-induced EEG as described in any one of claims 1 to 7 to obtain the predicted classification result; The error value between the predicted classification result and the true label is calculated based on the loss function; Based on the error value, the parameters of the spatiotemporal feature extraction module, the name speech process encoding module, the name conditional temporal reorganization module, and the classification module are updated simultaneously using the backpropagation algorithm. Repeat the steps of inputting the training set, calculating the error value, and updating the parameters until the error value converges or falls within a preset range, thus completing the training of the consciousness response assessment model based on name-evoked EEG and obtaining the trained consciousness response assessment model based on name-evoked EEG.

9. The training method for a consciousness response assessment model based on name-evoked EEG according to claim 8, characterized in that, Also includes: Add weight decay to the loss function, whereby the weight decay is used to constrain the magnitude of the model parameters.

10. A method for assessing conscious response based on name-evoked EEG, characterized in that, include: Acquire the multi-channel EEG signal to be evaluated and the name speech process sequence corresponding to the multi-channel EEG signal to be evaluated; The multi-channel EEG signal to be evaluated and the name speech process sequence are input into the consciousness response evaluation model based on name-evoked EEG trained by the training method of claim 8 or 9 to obtain the classification result.

Images