Autism domain self-adaptive spatio-temporal eeg modeling emotion recognition method and system

Abstract

The application discloses a domain self-adaptive spatio-temporal electroencephalogram modeling emotion recognition method and system for autism. The method comprises the following steps: taking the labeled typical development child electroencephalogram signal segment as source domain data, taking the unlabeled autism child electroencephalogram signal segment as target domain data, and taking the source domain data and the target domain data as training data; inputting the training data into an emotion recognition model for training and optimization, dynamically constructing a graph based on the physical coordinates of the channel, dynamically updating the graph structure based on the graph, and then learning the spatial topological structure features of the data through graph convolution; meanwhile, the time sequence features are obtained through a time sequence feature extraction module, and then the space-time features are fused through a multi-head attention mechanism; after obtaining the fused features, the domain invariant characteristics of the electroencephalogram segment are fully learned in the feature space through a transfer learning method. The application can complete the emotion recognition task of the autism children only by using the label of the typical development children.

Description

Technical Field

[0001] This invention belongs to the field of emotion recognition technology, and more specifically, relates to a domain-adaptive spatiotemporal EEG modeling emotion recognition method and system for autism. Background Technology

[0002] One of the core characteristics of autism spectrum disorders is the high heterogeneity in cognitive and emotional processing. This difference is reflected not only in higher-order psychological functions such as social cognition, attention allocation, and emotional understanding, but also in individual variations in neural activity patterns and physiological responses. Therefore, accurately identifying and characterizing the emotional states of individuals with autism has become a highly challenging yet crucial task in the clinical and neuroscience fields. Only by accurately identifying their emotional response patterns can individualized and targeted intervention programs be developed to improve emotion regulation, social interaction, and overall psychological adaptability. Electroencephalography (EEG), as a non-invasive neuroimaging technique, provides a macroscopic perspective on brain activity by placing electrodes on the scalp and recording changes in the electrical potential of the cerebral cortex. Compared to other brain imaging techniques, EEG signals have millisecond-level temporal resolution, enabling them to capture minute and rapid changes in the brain during emotional processing, especially the EEG responses triggered by emotional stimuli. Therefore, EEG can not only track the timeliness of emotional responses in real time but also reveal brain functional patterns under different emotional states, providing a valuable tool for the field of emotion recognition.

[0003] EEG-based emotion recognition for autism spectrum disorder (ASD) still faces multiple challenges. First, emotional EEG features are inherently complex and variable, exhibiting significant inter- and intra-individual variability, making feature extraction difficult. Second, unpredictable behavior during EEG signal acquisition by ASD children often leads to motion artifacts and recording instability, limiting the acquisition of sufficiently high-quality and reliably labeled emotional data. Third, fundamental differences in emotion perception between typical developmental children and ASD children result in cross-domain distributional differences, hindering direct model transfer. These factors combined make it difficult to obtain robust feature representations and achieve high-precision emotion recognition in ASD children using EEG signals. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a domain-adaptive spatiotemporal EEG modeling emotion recognition method and system for autism, which can effectively extract cross-domain common emotion features, thereby significantly improving the accuracy and generalization ability of emotion recognition.

[0005] To achieve the above objectives, according to one aspect of the present invention, a domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism is provided, comprising the steps of: EEG signals were collected from children with typical development and children with autism, and preprocessed and segmented to obtain EEG signal fragments for the two types of children. The EEG signal fragments of children with typical development were labeled. The labeled EEG signal fragments of children with typical development were used as source domain data, and the unlabeled EEG signal fragments of children with autism were used as target domain data. The source domain data and target domain data were used together as training data. Training data is input into the emotion recognition model for training and optimization. The emotion recognition model includes a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments. The temporal feature extraction module is used to extract temporal features of EEG signal segments. The fusion module is used to fuse the spatial and temporal features of EEG signal segments to generate fused features. The domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fusion features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fusion features of the source domain data and the target domain data into the generator, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fusion features of the source domain is aligned with the data distribution of the fusion features of the target domain.

[0006] Preferably, the extraction of spatial features from the EEG signal fragment includes the following steps: The three-dimensional physical coordinates of multiple channel electrodes used to acquire EEG signals are obtained. An initial adjacency matrix is ​​constructed based on the three-dimensional physical coordinates of the multiple channel electrodes, denoted as... And extract the input features of the EEG signal segments for each channel; Set a symmetric residual matrix As trainable parameters, Receive the gradient from the classification loss; A graph learner is used, with channels as nodes and the concatenation result of features of channel pairs as input. The output is a similarity score in the range [0,1]. A matrix is ​​constructed using the similarity scores of all channel pairs. ; For matrix , , Perform linear fusion to obtain a normalized adjacency matrix. The calculation formula is: , in Indicates the relative intensity of the control dynamic graph items; Add a self-loop and perform symmetric normalization to obtain the adjacency matrix. The calculation formula is: ; in, Represents the identity matrix. This represents the matrix obtained by adding the identity matrix I to the adjacency matrix W. Let D denote the extended degree matrix. Representation matrix Elements in; Obtain the adjacency matrix Afterward, the graph branch performs K rounds of parameterless propagation, using simplified graph convolution for K rounds of parameterless propagation, and then performs a single linear mapping to obtain spatial features.

[0007] Preferably, the calculation formula for constructing the initial adjacency matrix based on the three-dimensional physical coordinates of multiple channel electrodes is as follows: ; in, Represents the initial adjacency matrix The elements in This represents the physical distance between channels i and j. This represents the scaling hyperparameter. Represents the numerical stability term. () indicates taking the smaller value; The step of extracting the input features of the EEG signal segments of each channel includes the following steps: extracting the differential entropy features of the five frequency bands δ, θ, α, β, and γ from the EEG signal segments of each channel as the input features of the EEG signal segments of each channel.

[0008] Preferably, for the first Individual samples and channels The formula for calculating the similarity score is: ; in , where is the input feature of node i, These are the input features of node j. , Represents the weight matrix. , Indicates the deviation variable. ()express function, express function, This represents the similarity score between node i and node j. This indicates feature splicing.

[0009] Preferably, the time-series feature extraction module includes: EEGNet is used to extract features from EEG signal segments using a depthwise separable convolutional structure. A Transformer encoder is used to receive the features extracted by the EEGNet and output temporal features.

[0010] Preferably, the process of generating fused features by fusing spatial and temporal features of EEG signal fragments includes the following steps: The fused features are obtained using the following cross-attention mechanism. : ; in, , and This represents the mapping vector obtained through linear transformation of each attention head in a multi-head attention mechanism; , and This is the weight matrix. Indicates spatial characteristics, Indicates time characteristics, Indicates matrix transpose. express function; Will The projected residuals are summed and a linear transformation is performed to obtain the fused representation. The calculation formula is: ; in, This represents the output projection matrix of the cross-attention module. This indicates that the signal has been processed by the ReLU function; The fused features are obtained through a fully connected layer and ReLU. : ; in, This indicates a fully connected layer.

[0011] Preferably, the step of using the generator and the discriminator to perform adversarial training, so that the data distribution of the fused feature features of the source domain is aligned with the data distribution of the fused feature features of the target domain data, includes the following steps: Calculate the loss of the generator, the loss of the discriminator, and the classification loss of the emotion recognition model during adversarial training; The parameters of the generator and the discriminator are optimized until the sum of the generator loss and the classification loss of the emotion recognition model during adversarial training is minimized, while the discriminator loss is maintained within a preset range.

[0012] Preferably, the formula for calculating the loss of the generator is: ; in, This represents the loss of the generator. This represents the output of the discriminator. This represents an ideal uniform distribution. Indicates the Kullback-Leibler divergence; The formula for calculating the classification loss of the emotion recognition model during adversarial training is as follows: ; in, This represents the classification loss of the emotion recognition model. Representing the source domain sentiment label and CE, which represents the cross-entropy loss function. This represents the source domain sentiment prediction value. The weight matrix of z is represented by z. The deviation variable representing z; The formula for calculating the loss of the discriminator is: ; in, This represents the loss of the discriminator. For the sample size, For domain tags, This represents the predicted probability of belonging to the target domain.

[0013] Preferably, the domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism further includes the following steps: The EEG signals of the autistic children to be identified are preprocessed and segmented to obtain EEG signal fragments. These fragments are then input into a trained and optimized emotion recognition model. Through the processing of the dynamic graph construction module, the temporal feature extraction module, and the fusion module, fused features of the EEG signal fragments of the autistic children are obtained. These fused features are then input into the generator for domain alignment mapping, and finally, the aligned and mapped features are input into the emotion classifier for emotion recognition.

[0014] According to another aspect of the present invention, a domain-adaptive spatiotemporal EEG modeling emotion recognition system for autism is provided, comprising: The training data construction module is used to collect EEG signals from children with typical development and children with autism, and to preprocess and segment them to obtain EEG signal segments of the two types of children. The EEG signal segments of children with typical development are labeled. The labeled EEG signal segments of children with typical development are used as source domain data, and the unlabeled EEG signal segments of children with autism are used as target domain data. The source domain data and target domain data are used together as training data. An emotion recognition model is provided, comprising a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments, the temporal feature extraction module is used to extract temporal features of EEG signal segments, the fusion module is used to fuse spatial and temporal features of EEG signal segments to generate fused features, and the domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fusion features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fusion features of the source domain data and the target domain data into the generator, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fusion features of the source domain is aligned with the data distribution of the fusion features of the target domain.

[0015] Overall, this invention has beneficial effects compared with the prior art: (1) This invention proposes a novel EEG analysis framework for emotion recognition in autism, aiming to achieve unified modeling and accurate identification of emotional characteristics among different subject groups through the capture and optimization of cross-domain feature representations. This framework can mine and integrate representative emotional features from multi-source EEG data, thereby improving the model's recognition performance and robustness under complex individual differences.

[0016] (2) This invention overcomes the problem of scarce labeled data in autism emotion recognition to a certain extent. By introducing an adversarial transfer learning strategy, the model can classify the emotional state of autistic children by relying only on labeled data of typical developmental children, without using the label information of the target domain during training.

[0017] (3) This invention proposes a joint encoder model that integrates graph convolutional networks and the Transformer architecture to achieve efficient feature modeling of EEG signals. This joint encoder can simultaneously capture the spatial topological structure and temporal dynamic changes of EEG signals, thereby more comprehensively depicting the intrinsic laws of brain activity. By introducing a feature fusion technique based on attention mechanism, the model can adaptively assign importance weights to different features, highlight key information related to emotion recognition, and enhance the discriminative and expressive capabilities of the features. Attached Figure Description

[0018] Figure 1 A flowchart of the domain-adaptive spatiotemporal EEG modeling emotion recognition method provided by the present invention; Figure 2 This is an overall structural diagram of the emotion recognition model provided by the present invention; Figure 3 This is a structural diagram of the dynamic graph construction module provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0020] In the description of the embodiments of this application, the term "multiple" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0021] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This invention provides a domain-adaptive spatiotemporal EEG modeling emotion recognition method and system for autism, which will be described below.

[0024] An embodiment of the present invention provides a domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism, comprising steps 1 and 2.

[0025] Step 1: Collect EEG signals from children with typical development and children with autism, respectively, and preprocess and segment them to obtain EEG signal segments of the two types of children. Label the EEG signal segments of children with typical development. Use the labeled EEG signal segments of children with typical development as source domain data and the unlabeled EEG signal segments of children with autism as target domain data. Use the source domain data and target domain data together as training data.

[0026] The specific processing can be divided into the following parts: (1) Reduce the interference of noise such as electrooculography and electromyography on subsequent analysis by using bandpass filtering and artifact removal algorithm; (2) Divide the data into segments of two seconds each; (3) Use labeled typical developmental children's data as the source domain and unlabeled data of autistic children as the target domain; (4) Divide the training set, validation set and test set by randomly selecting different subjects.

[0027] In one embodiment, step 1 includes: 1.1 The raw EEG signal is preprocessed using a bandpass filtering algorithm to filter out irrelevant components outside the EEG frequency band, thereby retaining the effective signals related to neural activity. Simultaneously, an artifact removal algorithm is used to effectively remove artifact interference caused by eye movements, blinking, and muscle activity from the EEG data.

[0028] 1.2 The raw data was divided into two-second intervals to form a data format of (x, 14, 256), where x represents the number of time intervals into which the subject's data was divided, 14 represents the number of channels in the EEG recording, and 256 is the number of sampling points in each time interval.

[0029] 1.3 Data from labeled children with typical developmental milestones was used as the source domain, and data from unlabeled children with autism was used as the target domain; these two datasets together constituted the training set. During training, no labeling information for children with autism was used. Finally, data from labeled children with autism was used as the test set for evaluation.

[0030] Step 2: Input the training data into the emotion recognition model for training and optimization. The emotion recognition model includes a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments. The temporal feature extraction module is used to extract temporal features of EEG signal segments. The fusion module is used to fuse the spatial and temporal features of EEG signal segments to generate fused features. The domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fused features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fused features of the source domain data and the target domain data into the generator respectively, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fused features of the source domain is aligned with the data distribution of the fused features of the target domain.

[0031] The principles of each module are explained in detail below.

[0032] (2.1) Dynamic graph construction module The principle behind the dynamic graph construction module is as follows: 2.1.1 First, obtain the three-dimensional physical coordinates of the EEG cap electrodes, and use them as the basis for constructing the adjacency matrix. The initial structure is determined. These physical coordinates provide spatial information about the location of each electrode, thus helping to determine the connections between electrodes. Next, differential entropy features of five frequency bands (δ (1–4 Hz), θ (4–8 Hz), α (8–13 Hz), β (13–30 Hz), γ (30–50 Hz)) are extracted from the raw EEG data. These differential entropy features are used as input features for the EEG signal segments of each channel. The initial adjacency matrix is ​​constructed as follows: ; in, N is the number of channels; This represents the physical distance between channels i and j. This represents the scaling hyperparameter, which can be set to 5 in this embodiment of the invention. This represents the numerical stability term.

[0033] 2.1.2 The connection relationships between electrodes cannot be fully characterized solely by physical coordinates. To simulate stable cross-domain structural deviations, prior mapping is required. Based on this, a symmetric residual matrix is ​​set. As a trainable parameter, It also receives gradients from the classification loss, thus enabling the learning of globally shared channel relationships across samples during end-to-end training.

[0034] 2.1.3 To capture sample-specific functional connectivity, a graph learner is employed. Using channels as nodes, it takes the concatenated feature pairs of nodes as input and outputs a similarity score in the range [0,1]. Taking the first channel as an example... Individual samples and channels As an example, the calculation formula is: ; in , is the input feature of node i (differential entropy feature of five frequency bands), where , is the input feature of node j (differential entropy feature of five frequency bands). , Represents the weight matrix. , Indicates the deviation variable. ()express function, express function, This represents the similarity score between node i and node j. This represents feature concatenation. Other samples and channel pairs are calculated using the same method, and the similarity scores of all channel pairs for each sample constitute a matrix P.

[0035] ; 2.1.4 After linearly fusing the three components, an unnormalized adjacency matrix is ​​obtained. .

[0036] ; in This represents the relative intensity of the control dynamic graph item, which is set to 0.5 in this embodiment of the invention.

[0037] 2.1.5 Then, a self-loop is added, and symmetric normalization is performed to obtain the adjacency matrix. .

[0038] ; in, Represents the identity matrix. Let W be the adjacency matrix W plus the identity matrix I. Represents the extended degree matrix. Representation matrix The elements in.

[0039] 2.1.6 Obtaining the Adjacency Matrix Afterward, the graph branch performs K rounds of parameterless propagation, using Simplified Graph Convolution (SGC) for K rounds of parameterless propagation, followed by a single linear mapping to derive the spatial features. In one embodiment, K=2.

[0040] 2.2 Temporal feature extraction module The temporal feature extraction module includes: EEGNet (a lightweight convolutional neural network designed specifically for EEG signal classification tasks), which is used to extract features from EEG signal segments using a depthwise separable convolutional structure; and a Transformer encoder, which is used to receive the features extracted by EEGNet and output temporal features.

[0041] Specific implementation principle

[0042] 2.2.1 EEGNet Part: The input data has dimensions (B, 14, 256), where B represents the batch size, 14 is the number of channels in the EEG signal, and 256 is the number of time points per sample. The model first uses a depthwise separable convolutional structure to extract features from the EEG signal. This process includes one-dimensional temporal convolution, depthwise spatial convolution, and pointwise convolution compression, thereby efficiently capturing local temporal patterns and cross-channel spatial dependencies in the EEG signal while maintaining a small number of parameters.

[0043] 2.2.2 Transformer Part: The extracted features are then fed into the Transformer encoder for further modeling. In this process, the original C×T feature map is flattened and transformed into a sequence of tokens, enabling the model to process temporal information in a sequential manner. The Transformer establishes global dependencies among all tokens through a multi-head self-attention mechanism, dynamically aggregating feature information from different time segments, thereby effectively capturing long-range dependencies across time.

[0044] (2.3) Fusion Module Fusion module

[0045] 2.3 1. After obtaining the spatial features G and the temporal features T, the following cross-attention mechanism is used to obtain the fused features. : ; in, , and This represents the mapping vector obtained through the linear transformation of each attention head in a multi-head attention mechanism. , and These are the corresponding weight matrices.

[0046] 2.3 2 The projected residuals are summed and a linear transformation is performed to obtain the fused representation. : ; in This represents the output projection matrix of the cross-attention module.

[0047] 2.3 3. Finally, the fused features are obtained through a fully connected layer and ReLU (Rectified Linear Unit). : ; in This indicates a fully connected layer.

[0048] (2.4) Domain Migration Module The domain transfer module includes a sentiment classifier, a generator, and a discriminator. The sentiment classifier outputs a sentiment prediction value based on the input features. The generator and discriminator extract domain-invariant features, aligning the fused feature data distribution of the source domain with the fused feature data distribution of the target domain. Specifically, the generator outputs features to the discriminator, which determines which domain the generated features belong to.

[0049] The training and optimization of emotion recognition models involves two stages: The first stage involves inputting the fused features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier. The second stage involves inputting the fused features of the source and target domain data into the generator, and then performing adversarial training between the generator and the discriminator. The generator and discriminator are optimized until the distribution of the fused feature data from the source and target domains is aligned. Specifically, the losses of the generator, the discriminator, and the classification loss of the emotion recognition model during adversarial training are calculated. The parameters of the generator and discriminator are optimized until the sum of the generator's loss and the classification loss of the emotion recognition model during adversarial training is minimized, while the discriminator's loss remains within a preset range. Inside.

[0050] Phase 1: Train an emotion classifier (softmax) using labeled data from typical developmental children, and use validation set accuracy as the model selection criterion. At the same time, implement an early stop strategy and save the best checkpoint.

[0051] ; in, CE and CE represent source domain sentiment labels and cross-entropy loss functions, respectively. This represents the source domain sentiment prediction value. The weight matrix of z is represented by z. This represents the deviation variable of z.

[0052] The second stage involves introducing domain adversarial loss to this feature, gradually bringing the unlabeled features of the target domain closer to the manifold space of the source domain features. To achieve this adversarial interaction, the generator optimizes the symmetric KL divergence, forcing the two distributions to tend towards uniformity. Unlike the unidirectional nature of standard KL divergence, symmetric KL divergence eliminates directional bias by summing the two-way KL divergences and taking their average. ; Among them, This represents the output of the discriminator. This represents an ideal uniform distribution. This represents the Kullback-Leibler divergence. This design ensures that the features learned by the generator can progressively achieve spatial alignment.

[0053] The discriminator predicts whether the input features originate from the source domain or the target domain, forming an adversarial pair with the generator to promote the learning of domain-invariant feature representations. Its binary cross-entropy (BCE) loss function is: ; in For the sample size, For domain tags, This represents the predicted probability of belonging to the target domain.

[0054] Ultimately, the loss function in the adversarial training process combines the classification loss and the generator loss: ; Classification loss in adversarial training The calculation formula is also the same as formula (9).

[0055] After training, the final performance of the model can be evaluated using a test dataset, including test accuracy and F1 score.

[0056] After the test is completed, the EEG signals of the autistic children to be identified are preprocessed and segmented to obtain EEG signal fragments of the autistic children. The EEG signal fragments of the autistic children are input into the trained and optimized emotion recognition model. Through the processing of the dynamic graph construction module, the temporal feature extraction module, and the fusion module, the fusion features of the EEG signal fragments of the autistic children are obtained. The fusion features of the EEG signal fragments of the autistic children are input into the generator for domain alignment mapping. Then, the aligned and mapped features are input into the emotion classifier for emotion recognition to obtain the emotion prediction value.

[0057] This invention provides a domain-adaptive spatiotemporal EEG modeling emotion recognition system for autism, comprising: The training data construction module is used to collect EEG signals from children with typical development and children with autism, and to preprocess and segment them to obtain EEG signal segments of the two types of children. The EEG signal segments of children with typical development are labeled. The labeled EEG signal segments of children with typical development are used as source domain data, and the unlabeled EEG signal segments of children with autism are used as target domain data. The source domain data and target domain data are used together as training data. An emotion recognition model is provided, comprising a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments, the temporal feature extraction module is used to extract temporal features of EEG signal segments, the fusion module is used to fuse spatial and temporal features of EEG signal segments to generate fused features, and the domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fusion features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fusion features of the source domain data and the target domain data into the generator, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fusion features of the source domain is aligned with the data distribution of the fusion features of the target domain.

[0058] The working principle and technical effects of the domain adaptive spatiotemporal EEG modeling emotion recognition system are the same, and will not be repeated here.

[0059] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism, characterized in that, EEG signals were collected from children with typical development and children with autism, and preprocessed and segmented to obtain EEG signal fragments for the two types of children. The EEG signal fragments of children with typical development were labeled. The labeled EEG signal fragments of children with typical development were used as source domain data, and the unlabeled EEG signal fragments of children with autism were used as target domain data. The source domain data and target domain data were used together as training data. Training data is input into the emotion recognition model for training and optimization. The emotion recognition model includes a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments. The temporal feature extraction module is used to extract temporal features of EEG signal segments. The fusion module is used to fuse the spatial and temporal features of EEG signal segments to generate fused features. The domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fusion features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fusion features of the source domain data and the target domain data into the generator, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fusion features of the source domain is aligned with the data distribution of the fusion features of the target domain. The extraction of spatial features from EEG signal fragments includes the following steps: The three-dimensional physical coordinates of multiple channel electrodes used to acquire EEG signals are obtained. An initial adjacency matrix is ​​constructed based on the three-dimensional physical coordinates of the multiple channel electrodes, denoted as... And extract the input features of the EEG signal segments for each channel; Set a symmetric residual matrix As trainable parameters, Receive the gradient from the classification loss; A graph learner is used, with channels as nodes and the concatenation result of features of channel pairs as input. The output is a similarity score in the range [0,1]. A matrix is ​​constructed using the similarity scores of all channel pairs. ; For matrix , , Perform linear fusion to obtain a normalized adjacency matrix. The calculation formula is: ， in, Indicates the relative intensity of the control dynamic graph items; Add a self-loop and perform symmetric normalization to obtain the adjacency matrix. The calculation formula is: ； in, Represents the identity matrix. This represents the matrix obtained by adding the identity matrix I to the adjacency matrix W. Represents the extended degree matrix. Representation matrix Elements in; Obtain the adjacency matrix Afterward, the graph branch performs K rounds of parameterless propagation, using simplified graph convolution for K rounds of parameterless propagation, and then performs a single linear mapping to obtain spatial features.

2. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, The formula for calculating the initial adjacency matrix based on the three-dimensional physical coordinates of multiple channel electrodes is as follows: ； in, Represents the initial adjacency matrix The elements in This represents the physical distance between channels i and j. This represents the scaling hyperparameter. Represents the numerical stability term. () indicates taking the smaller value; The step of extracting the input features of the EEG signal segments of each channel includes the following steps: using the differential entropy features of the five frequency bands δ, θ, α, β, and γ from the EEG signal segments of each channel as the input features of the EEG signal segments of each channel.

3. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, For the Individual samples and channels The formula for calculating the similarity score is: ； in , where is the input feature of node i, These are the input features of node j. , Represents the weight matrix. , Indicates the deviation variable. ()express function, express function, This represents the similarity score between node i and node j. This indicates feature splicing.

4. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, The temporal feature extraction module package: EEGNet is used to extract features from EEG signal segments using a depthwise separable convolutional structure. A Transformer encoder is used to receive the features extracted by the EEGNet and output temporal features.

5. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, The process of generating fused features by fusing spatial and temporal features of EEG signal fragments includes the following steps: The fused features are obtained using the following cross-attention mechanism. : ； in, , and This represents the mapping vector obtained through linear transformation of each attention head in a multi-head attention mechanism; , and This is the weight matrix. Indicates spatial characteristics, Indicates time characteristics, Indicates matrix transpose. express function; Will The projected residuals are summed and a linear transformation is performed to obtain the fused representation. The calculation formula is: ； in, This represents the output projection matrix of the cross-attention module. This indicates that the signal has been processed by the ReLU function; The fused features are obtained through a fully connected layer and ReLU. : ； in, This indicates a fully connected layer.

6. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, The step of using the generator and the discriminator to perform adversarial training, so that the fusion feature data distribution of the source domain is aligned with the fusion feature data distribution of the target domain, includes the following steps: Calculate the loss of the generator, the loss of the discriminator, and the classification loss of the emotion recognition model during adversarial training; The parameters of the generator and the discriminator are optimized until the sum of the generator loss and the classification loss of the emotion recognition model during adversarial training is minimized, while the discriminator loss is maintained within a preset range.

7. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 6, characterized in that, The formula for calculating the loss of the generator is: ； in, This represents the loss of the generator. This represents the output of the discriminator. This represents an ideal uniform distribution. Indicates the Kullback-Leibler divergence; The formula for calculating the classification loss of the emotion recognition model during adversarial training is as follows: ； in, This represents the classification loss of the emotion recognition model. Representing the source domain sentiment label and CE, which represents the cross-entropy loss function. This represents the source domain sentiment prediction value. The weight matrix of z is represented by z. The deviation variable representing z; The formula for calculating the loss of the discriminator is: ； in, This represents the loss of the discriminator. For the sample size, For domain tags, This represents the predicted probability of belonging to the target domain.

8. The domain-adaptive spatiotemporal EEG modeling emotion recognition method for autism as described in claim 1, characterized in that, It also includes the following steps: The EEG signals of the autistic children to be identified are preprocessed and segmented to obtain EEG signal fragments. These fragments are then input into a trained and optimized emotion recognition model. Through the processing of the dynamic graph construction module, the temporal feature extraction module, and the fusion module, fused features of the EEG signal fragments of the autistic children are obtained. These fused features are then input into the generator for domain alignment mapping, and finally, the aligned and mapped features are input into the emotion classifier for emotion recognition.

9. A domain-adaptive spatiotemporal EEG modeling emotion recognition system for autism, characterized in that, include: The training data construction module is used to collect EEG signals from children with typical development and children with autism, and to preprocess and segment them to obtain EEG signal segments of the two types of children. The EEG signal segments of children with typical development are labeled. The labeled EEG signal segments of children with typical development are used as source domain data, and the unlabeled EEG signal segments of children with autism are used as target domain data. The source domain data and target domain data are used together as training data. An emotion recognition model is provided, comprising a dynamic graph construction module, a temporal feature extraction module, a fusion module, and a domain transfer module. The dynamic graph construction module is used to extract spatial features of EEG signal segments, the temporal feature extraction module is used to extract temporal features of EEG signal segments, the fusion module is used to fuse spatial and temporal features of EEG signal segments to generate fused features, and the domain transfer module includes an emotion classifier, a generator, and a discriminator. The training and optimization of the emotion recognition model includes the following steps: inputting the fusion features of the source domain data into the emotion classifier, calculating the emotion classification loss, and using the emotion classification loss to optimize the dynamic graph construction module, the temporal feature extraction module, the fusion module, and the emotion classifier; then inputting the fusion features of the source domain data and the target domain data into the generator, using the generator and the discriminator for adversarial training, and optimizing the generator and the discriminator until the data distribution of the fusion features of the source domain is aligned with the data distribution of the fusion features of the target domain. The extraction of spatial features from EEG signal fragments includes the following steps: The three-dimensional physical coordinates of multiple channel electrodes used to acquire EEG signals are obtained. An initial adjacency matrix is ​​constructed based on the three-dimensional physical coordinates of the multiple channel electrodes, denoted as... And extract the input features of the EEG signal segments for each channel; Set a symmetric residual matrix As trainable parameters, Receive the gradient from the classification loss; A graph learner is used, with channels as nodes and the concatenation result of features of channel pairs as input. The output is a similarity score in the range [0,1]. A matrix is ​​constructed using the similarity scores of all channel pairs. ; For matrix , , Perform linear fusion to obtain a normalized adjacency matrix. The calculation formula is: ， in, Indicates the relative intensity of the control dynamic graph items; Add a self-loop and perform symmetric normalization to obtain the adjacency matrix. The calculation formula is: ； in, Represents the identity matrix. This represents the matrix obtained by adding the identity matrix I to the adjacency matrix W. Represents the extended degree matrix. Representation matrix Elements in; Obtain the adjacency matrix Afterward, the graph branch performs K rounds of parameterless propagation, using simplified graph convolution for K rounds of parameterless propagation, and then performs a single linear mapping to obtain spatial features.

Images