Depressed emotional state LFP-EEG joint decoding method and device based on CBAM

Abstract

The invention discloses a depressed emotional state LFP-EEG joint decoding method and device based on CBAM, and the method comprises the steps: employing discrete short-time Fourier transform to extract corresponding time-frequency features for electroencephalogram data of LFP and EEG modes; time-frequency features corresponding to the LFP and the EEG are used as input of the CBAM model, modal complementarity of the LFP and the EEG is utilized, and a channel attention module and a time-frequency attention module in the CBAM model are combined to realize frequency-time-channel three-dimensional feature fusion enhancement, LFP and EEG combined decoding is realized, emotion-related electroencephalogram activities of the depression patient are accurately captured, and the emotion-related electroencephalogram activity of the depression patient is accurately captured. The method is used for patient positive emotional state and negative emotional state classification tasks. According to the method, a lightweight model structure is realized, depression emotion state decoding can be better realized, and the method is used for an emotion state classification task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural signal processing and decoding, and particularly relates to a depression state LFP-EEG joint decoding method and device based on CBAM. BACKGROUND

[0002] Depression disorder is a high-incidence mental disease in the world, and its core features are persistent low mood, impaired cognitive function and social function degradation, which seriously affect the quality of life of patients. At present, the evaluation of depression state in the clinic mainly depends on scale scores such as Hamilton Depression Scale (HAMD) and Self-Rating Depression Scale (SDS). Such evaluation methods are easily affected by factors such as patient subjective expression bias and evaluator judgment difference, lack of objective and quantitative neurophysiological indicators, and are difficult to accurately capture the subtle fluctuations of depression and the abnormal neural mechanisms in the brain, resulting in low early identification rate and delayed treatment effect evaluation.

[0003] With the development of neural electrophysiological technology, local field potential (LFP) and electroencephalogram (EEG) have become important means to reveal depression-related brain function activities. LFP is a neural electrical signal recorded by locally implanting microelectrodes (such as implanting microelectrodes into key brain regions of depression such as prefrontal cortex, amygdala and hippocampus through stereotactic surgery) in deep brain nuclei or cortex, reflecting the synchronous activity of local neuron groups, having high spatial resolution advantage, and being able to accurately capture the electrophysiological changes of depression-related brain regions (such as local neural circuit abnormalities of prefrontal cortex regulating emotion and excessive activation of amygdala in emotion processing). Its signal frequency band covers delta (0.5-4Hz, related to low mood), theta (4-8Hz, involved in emotion regulation), alpha (8-13Hz, associated with cognitive inhibition), beta (13-30Hz, related to emotional arousal) and gamma (above 30Hz, involved in neural information integration), and the activity patterns of different frequency bands are closely related to the severity of depression. EEG is a non-invasive recording of overall brain electrical activity signals by pasting electrodes (such as international 10-20 system leads) on the scalp, which has the characteristics of high time resolution, convenient operation and non-invasiveness, and can reflect the coordinated activity of neural networks in a wide range of brain cortex, and can capture the overall brain function abnormalities such as reduced prefrontal-temporal cortex functional connectivity and excessive activation of default network in depression patients. The signal characteristics (such as reduced alpha wave power and increased theta wave power) of specific leads (such as prefrontal cortex Fp1 / Fp2 leads and temporal cortex T3 / T4 leads) have been proven to be significantly related to depression state.

[0004] The local precision of LFP and the overall coverage of EEG are complementary, and the joint decoding of the two can construct a neural electrophysiological feature map of depressive mood from the "local neural circuit-global brain network" two-dimensional, which can more comprehensively and accurately reflect the abnormal brain mechanism of depressive mood than single signal decoding, and provide more abundant information support for the objective evaluation of depressive mood. However, the current LFP-EEG joint decoding method still has significant limitations: first, the feature extraction efficiency is low, and traditional methods such as wavelet transform, short-time Fourier transform combined with support vector machine (SVM), random forest and other machine learning algorithms are difficult to effectively mine the key features in high-dimensional neural signals that are strongly related to depressive mood, and are easily disturbed by signal noise and redundant information; second, the signal fusion mechanism is imperfect, and the different contribution weights of the two in the encoding of depressive mood are not considered, so the efficient cooperation of "local-global" features cannot be realized; third, there is a lack of targeted attention guidance, and existing deep learning methods such as basic convolutional neural network (CNN) and long short-term memory network (LSTM) do not focus on key brain areas and key signal frequency bands of depression, so that useful features are covered by irrelevant information, resulting in insufficient decoding accuracy and generalization.

[0005] The convolutional attention module (CBAM) can optimize feature extraction from the perspectives of channel dimension and spatial dimension as an efficient attention mechanism: the channel attention can adaptively strengthen the feature weights of key leads in LFP and EEG, and suppress noise channel interference; the spatial attention can focus on time-frequency features, highlighting the feature time domain and feature frequency domain. By introducing CBAM into LFP-EEG joint decoding, the problems of inefficient feature extraction and imperfect fusion of existing methods can be solved, and the accuracy and robustness of depressive mood state decoding can be improved. SUMMARY

[0006] The present application aims to solve the problems of the prior art, and provides a CBAM-based LFP-EEG joint decoding method and device for depressive mood state.

[0007] The present application aims to solve the problems of the prior art, and provides a CBAM-based LFP-EEG joint decoding method and device for depressive mood state. The present application aims to solve the problems of the prior art, and provides a CBAM-based LFP-EEG joint decoding method and device for depressive mood state. (1) For the brain electrical data of the two modalities of LFP and EEG, the corresponding time-frequency features are extracted by using discrete short-time Fourier transform;

[0008] Further, before the step (1), the following steps are further included: The patient performs a positive and negative emotion picture viewing task under the guidance of a doctor, and physiological data of EEG and LFP of the patient are synchronously acquired by an optoelectronic system as acquired brain electrical data, wherein the brain electrical data of 3 seconds before the start of the picture viewing emotion task is taken as resting state data.

[0009] Further, the step (1) specifically comprises: For the brain electrical data of the two modalities of EEG and LFP, for each channel, the brain electrical data from 3 seconds before the start of the picture viewing emotion task to the end of the picture viewing emotion task is intercepted, the time-frequency matrix from 3 seconds before the start of the picture viewing emotion task to the end of the picture viewing emotion task is calculated by using discrete short-time Fourier transform respectively, and the calculated time-frequency matrix is normalized with the brain electrical data of 3 seconds before the start of the picture viewing emotion task as a baseline.

[0010] Further, the CBAM model comprises a channel attention module, a time-frequency attention module and an emotion prediction module, wherein the channel attention module comprises a global average pooling, a global maximum pooling and a multi-layer perception; the time-frequency attention module comprises a channel average pooling, a channel maximum pooling and a convolution layer; and the emotion prediction module comprises a fully connected network and a Softmax activation function. The time-frequency features of LFP and EEG are taken as the input of the CBAM model, first enter the channel attention module, and the corresponding channel feature vectors are obtained through global average pooling and global maximum pooling respectively, the channel feature vectors output by the global average pooling and the global maximum pooling are activated through the multi-layer perception with parameter sharing and then through the Sigmoid activation function to obtain the channel attention weight, and the channel enhanced features are obtained by weighting the time-frequency features through the channel attention weight; the channel enhanced features enter the time-frequency attention module, and the average pooling single-channel time-frequency graph and the maximum pooling single-channel time-frequency graph are obtained through the channel average pooling and the channel maximum pooling respectively, the two are spliced to generate the time-frequency attention weight through the 1x1 convolution and the Sigmoid activation function, and the joint enhanced features are obtained by weighting the channel enhanced features through the time-frequency attention weight; the joint enhanced features are flattened and processed through the fully connected network and the Softmax activation function to output the emotion category prediction probability, and the emotion decoding is completed.

[0011] The second aspect of the embodiment of the present application provides a CBAM-based depression emotion state LFP-EEG joint decoding device, comprising one or more processors and a memory, the memory is coupled with the processor; wherein the memory is used for storing program data, and the processor is used for executing the program data to realize the CBAM-based depression emotion state LFP-EEG joint decoding method.

[0012] The third aspect of the embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the CBAM-based depression emotional state LFP-EEG joint decoding method.

[0013] The CBAM-based depression emotional state LFP-EEG joint decoding method has the advantages that the CBAM-based depression emotional state LFP-EEG joint decoding method utilizes the modality complementarity of LFP and EEG, realizes feature enhancement in the three dimensions of frequency, time and channel by combining CBAM, is used for LFP and EEG joint decoding, and accurately captures the emotional related brain electrical activity of a depression patient; the LFP and EEG joint decoding is superior to decoding by using EEG or LFP data alone; and the method realizes a light-weight model structure, and can better realize depression emotional state decoding compared with other decoding methods. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A flowchart of the CBAM-based depression emotional state LFP-EEG joint decoding method of the present application is shown in FIG. 1. Figure 2 A structure flowchart of the CBAM model of the present application is shown in FIG. 2. Figure 3 An example diagram of the LFP-EEG joint decoding of the present application is shown in FIG. 3. Figure 4 A structure diagram of the CBAM-based depression emotional state LFP-EEG joint decoding device of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0015] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only exemplary and is not intended to limit the present application. The following exemplary embodiments described in the detailed description are examples of apparatus and methods consistent with the present application as detailed in the following claims.

[0016] The terms used in the present application are merely for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0017] It should be understood that, although the terms first, second, third, etc. can be employed in this application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of the application, first information can also be referred to as second information, and similarly, second information can also be referred to as first information. Depending on the context, the word "if' as used herein can be interpreted as "when" or "upon determination" or "in response to a determination".

[0018] The CBAM-based depression emotional state LFP-EEG joint decoding method of the application can capture the emotional related brain electrical activity of the depressed patients, and can better realize the decoding of the depression emotional state. Figure 1 As shown, the method specifically comprises the following steps: (1) For the brain electrical data of the two modalities of LFP and EEG, the corresponding time-frequency features are extracted by using discrete short-time Fourier transform. The brain electrical data includes the data of the two modalities of LFP and EEG.

[0019] Further, before the brain electrical data of the two modalities of LFP and EEG are extracted by using discrete short-time Fourier transform, i.e. before step (1), it further comprises: the patient performs a positive and negative emotional picture viewing task under the guidance of the doctor, and the EEG and LFP physiological data of the patient are synchronously acquired by using an optoelectronic system as the acquired brain electrical data, wherein the brain electrical data of 3 seconds before the picture viewing emotional task starts is taken as the resting state data.

[0020] Further, for the brain electrical data of the two modalities of LFP and EEG, the corresponding time-frequency features are extracted by using discrete short-time Fourier transform, specifically comprising: for the given brain electrical data of the two modalities of EEG and LFP, for each channel, the brain electrical data (i.e. EEG and LFP data) from 3 seconds before the picture viewing emotional task starts to the end of the picture viewing emotional task is intercepted, the time-frequency matrix from 3 seconds before the picture viewing emotional task starts to the end of the picture viewing emotional task is calculated by using discrete short-time Fourier transform, and the calculated time-frequency matrix is normalized by taking the brain electrical data of 3 seconds before the picture viewing emotional task starts as the baseline, and the normalized time-frequency matrix is taken as the input feature of the subsequent CBAM model.

[0021] Specifically, in the processing process of the single-channel intracranial neural signal (such as EEG or LFP), on the one hand, the time-frequency features are extracted by using discrete short-time Fourier transform, and the single-channel intracranial neural signal is a discrete time sequence, and the expression is:

[0022] In the formula, n is the discrete sampling point index, t is the continuous time, is the sampling frequency, is the signal amplitude at time t, is the signal amplitude at the nth sampling point. The signal segment from 3 seconds before the start of the picture emotion task to the end of the picture emotion task is taken as the signal in the window , and then the segmented Fourier transform is performed on it by using a sliding window function to obtain time-frequency matrix elements:

[0023] In the formula, is the complex element in the mth row (time frame) and the kth column (frequency point) of the time-frequency matrix, and the amplitude or power thereof reflects the signal energy strength at the corresponding time and frequency point; is the local sampling point index in the window, and is only valid within the range covered by the window function; m is a time shift parameter (time frame index) that determines the center position of the window function on the signal, and its value range is , and M is the total number of time frames; is the window function length (window length), which is a key parameter for balancing time and frequency resolution; represents the amplitude of the signal (signal segment from 3 seconds before the start of the picture emotion task to the end of the picture emotion task) in the intercepted window at the local sampling point ; is the window function (such as the Hanning window, the Hamming window), which is used for localized weighting of the signal to reduce spectral leakage, is the relative position index of the window function with m as the center; j is the imaginary unit, which satisfies ; k is the frequency index, which corresponds to the frequency axis of the time-frequency matrix, and its value range is ; is the complex exponential base function of the discrete Fourier transform, which is used to extract the information of the kth frequency component in the signal. The (power spectrum) is used as the time-frequency feature to eliminate the influence of the complex phase.

[0024] (2) The time-frequency features corresponding to the LFP and the EEG are taken as the input of the CBAM model, and the channel attention (CA) module and the time-frequency attention (TFA) module are used for feature fusion and enhancement to realize emotion decoding.

[0025] In this embodiment, the CBAM model includes a channel attention module, a time-frequency attention module, and an emotion prediction module, as shown in Figure 2 and Figure 3The CBAM model is shown. Among them, the channel attention module includes global average pooling, global maximum pooling and multi-layer perceptron, the time-frequency attention module includes channel average pooling, channel maximum pooling and convolution layer, and the emotion prediction module includes a fully connected network and a Softmax activation function. Taking the time-frequency features of LFP and EEG as the input of the CBAM model, first enter the channel attention module, and obtain the corresponding channel feature vectors through global average pooling and global maximum pooling respectively. The channel feature vectors output by the global average pooling and the global maximum pooling are activated through the parameter-shared multi-layer perceptron and then the Sigmoid activation function to obtain the channel attention weight. The channel enhanced features are obtained by weighting the time-frequency features through the channel attention weight; the channel enhanced features enter the time-frequency attention module, and the average pooling single-channel time-frequency graph and the maximum pooling single-channel time-frequency graph are obtained through channel average pooling and channel maximum pooling respectively. The two are spliced, and the time-frequency attention weight is generated after 1x1 convolution and Sigmoid activation function. The joint enhanced features are obtained by weighting the channel enhanced features through the time-frequency attention weight; the joint enhanced features are flattened and processed through the fully connected network and the Softmax activation function to output the emotion category prediction probability, and the emotion decoding is completed.

[0026] Specifically, the channel attention module and the time-frequency attention module in the CBAM model are used to realize emotion decoding, which specifically includes: taking the three-dimensional time-frequency feature tensor extracted by discrete short-time Fourier transform as the input of the CBAM model , wherein C is the feature channel number, T is the time frame dimension, and H is the frequency dimension, , wherein C is the feature channel number, T is the time frame dimension, and H is the frequency dimension,

[0027]

[0028] , wherein, , and are the channel feature vectors after global average pooling and global maximum pooling respectively. Then, the two are processed through the parameter-shared multi-layer perceptron (MLP), and the expression is:

[0029]

[0030]

[0031] , wherein, is the attention weight of the cth channel, denotes a Sigmoid activation function, denotes a ReLU activation function, and denote the weight matrix of ReLU activation function and MLP respectively. Then the channel enhanced features are weighted by the channel attention weights to obtain the channel enhanced features, denoted as:

[0032] wherein, denotes the channel enhanced feature of the c-th channel, the t-th time frame and the f-th frequency point.

[0033] Then, the key time-frequency area is focused by the time-frequency attention module: the channel enhanced features are compressed in the channel, and the average pooling in the channel and the maximum pooling in the channel are performed respectively to obtain the average pooling single-channel time-frequency map and the maximum pooling single-channel time-frequency map. After the feature concatenation, the time-frequency attention weights are generated by the 1x1 convolution and the Sigmoid activation function, and the expression is:

[0034]

[0035]

[0036] wherein, denotes the average pooling single-channel time-frequency map, denotes the maximum pooling single-channel time-frequency map, denotes the feature concatenation, denotes the 1x1 convolution, X denotes the input of the 1x1 convolution, and b are the convolution weight and the bias of the 1x1 convolution respectively, denotes the time-frequency attention weight of the t-th time frame and the f-th frequency point. The joint enhanced features are weighted by the time-frequency attention weights to obtain the joint enhanced features, and the expression is:

[0037] wherein, denotes the joint enhanced feature of the c-th channel, the t-th time frame and the f-th frequency point.

[0038] Finally, the joint enhanced features are flattened and processed by the full connection network and the Softmax activation function to output the probability of positive and negative emotion category binary classification, and the expression is:

[0039] wherein, denotes the flattening operation, denotes a fully connected network, denotes a Softmax activation function, is a predicted emotion category probability vector, and the maximum probability is taken as the final prediction label to complete emotion decoding.

[0040] It should be understood that the time-frequency features of LFP and EEG are taken as the input of the CBAM model, the channel attention weight and the time-frequency attention weight are extracted by using the CBAM model, different channels and time-frequency points are given different weights, and the model weight that makes the emotion decoding task classification accuracy highest is obtained by dynamically updating in the model training process.

[0041] In some other embodiments, the effect of the CBAM-based depression emotion state LFP-EEG joint decoding method described in the application is also verified, and its advantages and disadvantages are evaluated. The prediction decoding accuracy of the method described in the application is compared with that of other classical machine learning methods. Among them, linear discriminant analysis (LDA, Linear Discriminant Analysis), support vector machine (SVM, Support Vector Machine), Transformer model, and convolutional attention module (CBAM, Convolutional Block Attention Module) in the application. LDA is a classical supervised learning method, mainly used for feature dimension reduction and classification, and the feature space is optimized by maximizing the difference between classes and minimizing the difference within classes; SVM is also a classical supervised learning algorithm, the core of which is to find the optimal classification hyperplane to maximize its interval with two classes of samples, and it can also process nonlinear classification problems through kernel functions; the Transformer model is a deep learning architecture based on self-attention mechanism, which can capture long-distance dependencies without relying on recurrent or convolutional structures, and is widely used in natural language processing, computer vision and time-frequency signal analysis fields; CBAM is a lightweight attention mechanism module designed for convolutional neural networks, which focuses on key feature channels through channel attention and focuses on key local areas through spatial or time-frequency attention, so as to enhance effective features and suppress redundant information, and is commonly used in image classification, target detection and time-frequency feature (such as emotion decoding) enhancement tasks.

[0042] In order to evaluate the advantages and disadvantages of different models, in the classification task such as emotion decoding, accuracy (ACC), F1 score and area under ROC curve (AUC) are the core performance evaluation indexes, ACC measures the proportion of the predicted results consistent with the true labels in all samples, and is suitable for the scenario with balanced class distribution, and its calculation formula is:

[0043] In the formula, TP is the number of true positive samples, TN is the number of true negative samples, FP is the number of false positive samples, and FN is the number of false negative samples. The F1 score is the harmonic mean of precision and recall, which can solve the class imbalance problem, and the calculation formula is:

[0044]

[0045]

[0046] In the formula, P is the precision, and R is the recall. The AUC quantifies the discrimination ability of the binary classification model through the area under the ROC curve, and the ROC curve takes the false positive rate as the horizontal axis and the true positive rate as the vertical axis. The trapezoidal method is commonly used to calculate AUC, and the calculation formula is:

[0047]

[0048]

[0049] In the formula, M is the total number of coordinate points of the ROC curve, is the false positive rate, is the true positive rate, and are the false positive rate and the true positive rate corresponding to the mth coordinate point, respectively. In practical applications, the combination of the three can comprehensively evaluate the model: ACC reflects the overall correctness, F1 score focuses on the class-level prediction ability, and AUC measures the discrimination.

[0050] The LDA, SVM, Transformer model and the CBAM described in the application are used for emotion decoding, and the results are shown in Table 1.

[0051] Table 1: Prediction accuracy percentage of different emotion decoding methods

[0052] As can be seen from Table 1, the effect of CBAM described in the application is optimal.

[0053] In summary, the CBAM-based depression emotional state LFP-EEG joint decoding method of the application utilizes the modal complementarity of LFP and EEG, combines CBAM to realize feature enhancement in the three dimensions of frequency, time and channel, and is used for LFP and EEG joint decoding to accurately capture the emotion-related brain electrical activity of depression patients. The application realizes a lightweight model structure, and compared with other decoding methods, can better realize depression emotional state decoding.

[0054] Corresponding to the foregoing embodiment of the CBAM-based depression emotional state LFP-EEG joint decoding method, the present application also provides an embodiment of a CBAM-based depression emotional state LFP-EEG joint decoding device.

[0055] Referring to Figure 4 The CBAM-based depression emotional state LFP-EEG joint decoding device provided by the embodiment of the present application comprises one or more processors and a memory, and the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the CBAM-based depression emotional state LFP-EEG joint decoding method in the foregoing embodiment.

[0056] The CBAM-based depression emotional state LFP-EEG joint decoding device of the embodiment of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability where the device is located. From the hardware level, as shown in Figure 4 As shown in the figure, it is a hardware structure diagram of the device with data processing capability where the CBAM-based depression emotional state LFP-EEG joint decoding device of the present application is located. In addition to the processor, the memory, the network interface, and the non-volatile memory shown in the figure, the device with data processing capability where the device of the embodiment is located usually also includes other hardware according to the actual functions of the device with data processing capability, and details are not described here. Figure 4

[0057] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, and is not described here.

[0058] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to the actual needs, part or all of the modules can be selected to realize the purpose of the present application scheme. Those skilled in the art can understand and implement it without creative labor.

[0059] ​The embodiment of the present application further provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the CBAM-based depression state LFP-EEG joint decoding method in the above embodiment.

[0060] The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory, according to any of the preceding embodiments. The computer readable storage medium can also be any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both an internal storage unit of any data processing capable device and an external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0061] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for joint decoding of depressive mood states using LFP-EEG based on CBAM, characterized in that, Includes the following steps: (1) For the two modalities of LFP and EEG, the discrete short-time Fourier transform is used to extract the corresponding time-frequency features; (2) The time-frequency features corresponding to LFP and EEG are used as input to the CBAM model. The channel attention module and time-frequency attention module are used to perform feature fusion enhancement to achieve emotion decoding.

2. The LFP-EEG joint decoding method for depressive mood states based on CBAM according to claim 1, characterized in that, Before step (1), the following is also included: Under the guidance of doctors, patients performed a picture-viewing task that evoked positive and negative emotions. The photoelectric system simultaneously acquired the patients' EEG and LFP physiological data as the collected EEG data, and the EEG data of the first 3 seconds before the start of the picture-viewing emotional task was used as the resting state data.

3. The LFP-EEG joint decoding method for depressive mood states based on CBAM according to claim 1, characterized in that, Step (1) specifically includes: For a given EEG and LFP modalities, for each channel, the EEG data from 3 seconds before the start of the picture-viewing emotion task to the end of the picture-viewing emotion task is extracted. The time-frequency matrix from 3 seconds before the start of the picture-viewing emotion task to the end of the picture-viewing emotion task is calculated using discrete short-time Fourier transform. The calculated time-frequency matrix is ​​then normalized using the EEG data from 3 seconds before the start of the picture-viewing emotion task as the baseline.

4. The LFP-EEG joint decoding method for depressive mood states based on CBAM according to claim 1, characterized in that, The CBAM model includes a channel attention module, a time-frequency attention module, and a sentiment prediction module. The channel attention module includes global average pooling, global max pooling, and a multilayer perceptron. The time-frequency attention module includes channel average pooling, channel max pooling, and convolutional layers. The sentiment prediction module includes a fully connected network and a Softmax activation function. Using the time-frequency features of LFP and EEG as input to the CBAM model, the features first enter the channel attention module, where they undergo global average pooling and global max pooling to obtain the corresponding channel feature vectors. The channel feature vectors output by global average pooling and global max pooling are then passed through a parameter-shared multilayer perceptron and activated by a sigmoid activation function to obtain channel attention weights. These channel attention weights are then used to weight the time-frequency features to obtain channel enhancement features. The channel enhancement features then enter the time-frequency attention module, where they undergo channel average pooling and channel max pooling to obtain average pooling single-channel time-frequency maps and max pooling single-channel time-frequency maps. These two maps are concatenated and then processed by a 1×1 convolution and a sigmoid activation function to generate time-frequency attention weights. These weights are then used to weight the channel enhancement features to obtain joint enhancement features. The joint enhancement features are flattened and then processed by a fully connected network and a softmax activation function to output the predicted probability of the sentiment category, thus completing the sentiment decoding.

5. A CBAM-based LFP-EEG joint decoding device for depressive mood states, comprising one or more processors and a memory, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the LFP-EEG joint decoding method for depressive mood states based on CBAM as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, is used to implement the LFP-EEG joint decoding method for depressive mood states based on CBAM as described in any one of claims 1-4.

Images