Mental state recognition system and mental state regulation and control system

Abstract

The invention relates to a mental state recognition system and a regulation and control system, and the mental state recognition system is used for collecting a current electroencephalogram signal of a user, obtaining self-collected mental state electroencephalogram data, constructing a mental state recognition model based on the self-collected mental state electroencephalogram data, and adjusting and controlling the mental state recognition model. Inputting the current electroencephalogram signal into the mental state recognition model, and outputting an evaluation result of the current mental state of the user; and the regulation and control system is used for acquiring an evaluation result of the mental state of the user, regulating and controlling the mental state of the user based on a regulation and control strategy, monitoring feedback information of the user in a regulation and control process, and adjusting the regulation and control strategy according to the feedback information. Therefore, accurate recognition of mental states such as insomnia and anxiety is realized, a personalized regulation and control scheme can be provided for the user according to the real-time electroencephalogram data to regulate and control cognitive competence, and the cognitive state of the user is optimized.

Description

Technical Field

[0001] This application mainly relates to the field of electroencephalogram signals, and particularly to a mental state recognition system and a regulation system. Background Art

[0002] With the increase of modern social work pressure, mental health problems, especially emotional disorders such as insomnia, anxiety and distractibility, have become public health problems widely concerned globally. These mental states not only affect the work efficiency and quality of life of individuals, but also have a negative impact on physical health. Most of the existing methods for mental state regulation are self-regulation. However, for sub-healthy people with serious mental states such as severe anxiety and insomnia, self-regulation often cannot play a good role, and external forces are needed to adjust the mental state, such as doing psychotherapy, drug relief, etc. But it usually relies on artificial diagnosis or drug treatment, and the treatment effect and side effects vary due to individual differences, lacking personalization and accuracy.

[0003] In recent years, with the development of electroencephalogram (EEG) technology, the mental state detection and regulation technology based on electroencephalogram signals has gradually become a research hotspot; electroencephalogram signals can reflect the state of brain activity in real time, especially providing a convenient and non-invasive method for the monitoring of mental states. However, the traditional electroencephalogram signal analysis methods have the disadvantages of complex signal processing, low recognition accuracy and difficulty in real-time regulation.

[0004] Therefore, how to use electroencephalogram signals for efficient and accurate mental state assessment, and optimize cognitive ability and improve emotional state through effective regulation means has become an important direction of current technology research. Although the existing neurofeedback technology has been applied to some mental health regulations, there is still a lack of personalized regulation schemes for specific mental states. For example, the regulation schemes for anxiety or insomnia are not personalized. At the same time, there is also a lack of the ability to adjust the regulation strategy in real time. Summary of the Invention

[0005] An object of this application is to provide a mental state recognition system and a regulation system, which solve the problems in the prior art that the electroencephalogram signal analysis method has the disadvantages of complex signal processing, low recognition accuracy, difficulty in real-time regulation, and the problem of inability to perform personalized regulation of specific mental states.

[0006] According to one aspect of this application, a mental state recognition system based on electroencephalogram data is provided. The mental state recognition system includes: a signal acquisition module and a mental state classification module, wherein,

[0007] The signal acquisition module is used to acquire the current electroencephalogram signal of the user;

[0008] The mental state classification module includes an identification model construction unit and an identification unit;

[0009] The identification model construction unit is used to obtain EEG data samples with different state labels and construct a mental state identification model based on the EEG data samples. Among them, the EEG data samples include a dataset with a sleep state label, EEG data with an anxiety state label, and resting-state EEG data with a healthy state label;

[0010] The identification unit is used to input the current EEG signal into the mental state identification model to obtain an evaluation result of the user's current mental state.

[0011] Optionally, the mental state classification module includes an insomnia detection model construction unit and an anxiety detection model construction unit, where

[0012] The insomnia detection model construction unit is used to construct an insomnia state detection model, and the anxiety detection model construction unit is used to construct an anxiety state detection model;

[0013] The identification unit is used to input the current EEG signal into the insomnia state detection model, the anxiety state detection model, and the mental state identification model respectively to obtain a first detection result, a second detection result, and a third detection result, and perform maximum probability discrimination by integrating all the detection results to obtain a final evaluation result of the current mental state.

[0014] Optionally, the insomnia detection model construction unit is used to obtain a dataset with a sleep state label from the database and construct an insomnia state detection model for EEG signals according to the dataset; the identification unit is used to identify the current EEG signal using the insomnia state detection model to obtain a first detection result, where the first detection result includes an insomnia state category and a corresponding intensity value.

[0015] Optionally, the insomnia detection model construction unit is used to implement the following steps:

[0016] Downsample the dataset and then filter it through a band-pass filter, decompose it into multiple frequency bands, and determine the range of each frequency band;

[0017] Extract P-norm features and cross-frequency coupling features based on each frequency band range respectively;

[0018] Determine time-domain features based on the P-norm features and frequency-domain features based on the cross-frequency coupling features, where the frequency-domain features include phase-amplitude coupling features of phase synchronization and amplitude correlation features.

[0019] Optionally, the phase-amplitude coupling features of phase synchronization are extracted using the following steps:

[0020] Determine the low-frequency signal and high-frequency signal of the electroencephalogram signal in each frequency band;

[0021] Perform Hilbert transform on the low-frequency signal and high-frequency signal respectively to obtain the instantaneous amplitude of the high-frequency signal and the instantaneous phase value of the low-frequency signal;

[0022] Perform band-pass filtering on the instantaneous amplitude of the high-frequency signal within the low-frequency band frequency range, and perform Hilbert transform on the filtered signal again to obtain the high-frequency synchronization phase value;

[0023] Extract the phase-amplitude coupling feature of phase synchronization based on the instantaneous phase value of the low-frequency signal and the high-frequency synchronization phase value.

[0024] Optionally, the anxiety detection model construction unit is used to obtain electroencephalogram data carrying anxiety state labels and resting-state electroencephalogram data carrying healthy state labels, and construct an anxiety state detection model of electroencephalogram signals according to the obtained labeled electroencephalogram data; the recognition unit is used to use the anxiety state detection model to recognize the current electroencephalogram signal to obtain a second detection result, where the second detection result includes the anxiety state category and the corresponding intensity value.

[0025] Optionally, the anxiety detection model construction unit is used to process the obtained labeled electroencephalogram data to obtain data in multiple frequency bands, extract features of the electroencephalogram signal in each frequency band using filter bank common spatial patterns, and calculate non-linear features, and splice the features of the filter bank common spatial patterns and the non-linear features as the input of the classifier to obtain an anxiety state detection model of the electroencephalogram signal.

[0026] Optionally, the step of extracting features using the filter bank common spatial pattern includes:

[0027] Determine the average spatial covariance matrix in the first category and the average spatial covariance matrix in the second category of each frequency band;

[0028] Calculate the mixed spatial covariance in the first category and the second category, and calculate the spatial projection matrix according to the mixed spatial covariance, where the spatial projection matrix minimizes the within-class variance and maximizes the between-class variance;

[0029] Use the spatial projection matrix to extract the log variance feature of the electroencephalogram signal within the frequency band.

[0030] Optionally, the non-linear features include Renyi entropy feature and correlation dimension,

[0031] The Renyi entropy feature is determined using the following steps: divide the amplitude of the electroencephalogram signal into multiple sub-intervals and determine the probability of each sub-interval, and determine the Renyi entropy according to the sub-interval and the corresponding probability;

[0032] The associated dimension is determined using the following steps: The electroencephalogram signal is set as a time series, and the time series is calculated according to the selected time delay parameter and embedding dimension to obtain the associated dimension.

[0033] Optionally, the recognition model construction unit is configured to input the electroencephalogram data sample into a multi-size temporal convolutional neural network for multi-layer convolutional operations to extract features, classify the extracted features, and train to obtain a mental state recognition model; the recognition unit is configured to use the mental state recognition model to recognize the current electroencephalogram signal to obtain a third detection result.

[0034] Optionally, the multi-size temporal convolutional neural network includes a first module, a second module, and a classification model;

[0035] The first module includes a first convolutional layer, a second convolutional layer, a first batch normalization layer, a second batch normalization layer, an activation layer, an average pooling layer, and a dropout layer, and a batch normalization layer follows each convolutional layer;

[0036] The second module includes a third convolutional layer, a fourth convolutional layer, a third batch normalization layer, an activation layer, an average pooling layer, and a dropout layer;

[0037] The classification model includes a fully connected layer.

[0038] According to another aspect of the present application, there is also provided a regulation system using the foregoing mental state recognition system, the regulation system including: a neurofeedback regulation module and a user feedback module, wherein,

[0039] The neurofeedback regulation module is configured to monitor the electroencephalogram signal of the user in real time, obtain an evaluation result of the mental state of the user recognized by the mental state recognition system based on the electroencephalogram signal, and regulate the mental state of the user based on a regulation strategy;

[0040] The user feedback module is configured to monitor the feedback information of the user during the regulation process and adjust the regulation strategy according to the feedback information.

[0041] Optionally, the neurofeedback regulation module includes a transcranial magnetic stimulation unit, a neuroelectrical stimulation unit, and a virtual reality interaction unit;

[0042] The transcranial magnetic stimulation unit is configured to transmit a magnetic field to the cerebral cortex of the user and adjust the frequency and intensity of the magnetic stimulation according to the electroencephalogram signal and feedback information of the user;

[0043] The neuroelectrical stimulation unit is configured to stimulate the cerebral cortex or a specified neural pathway of the user through an electric current and adjust the intensity and stimulation pattern of the electric current according to the electroencephalogram signal and feedback information of the user;

[0044] The virtual reality interaction unit is used to provide a virtual environment for the user, so that the user can perform cognitive task training in the virtual environment, and adjust the stimulation content and intensity of the virtual environment according to the feedback information.

[0045] Optionally, the regulation strategy includes transcranial magnetic stimulation, neuroelectrical stimulation, and regulation information of virtual reality. Among them, the regulation information includes regulation intensity, duration, frequency, and regulation content.

[0046] Optionally, the user feedback module includes a physiological feedback monitoring unit and a psychological feedback evaluation unit. Among them, the physiological feedback monitoring unit is used to monitor the physiological parameters of the user to evaluate the regulation effect;

[0047] The psychological feedback evaluation unit is used to evaluate the changes in the user's psychological state and cognitive ability in real time according to the relevant data of the user. Among them, the relevant data includes the user's self-report and emotion scale.

[0048] Compared with the prior art, the mental state recognition system and regulation system provided by the present application, the mental state recognition system is used to collect the user's current electroencephalogram signal, obtain the self-collected mental state electroencephalogram data, construct a mental state recognition model based on the self-collected mental state electroencephalogram data, preprocess the current electroencephalogram signal and input it into the mental state recognition model, and output the user's current mental state; the regulation system is used to obtain the mental state of the user recognized by the mental state recognition system, regulate the mental state of the user based on the regulation strategy, monitor the feedback information of the user during the regulation process, and adjust the regulation strategy according to the feedback information. Therefore, it can identify the user's mental state in real time, optimize their cognitive ability through neurofeedback regulation technology, and help the user improve their mood and mental state. The mental recognition system can efficiently classify mental states through the time domain, frequency domain, and non-linear characteristics of electroencephalogram signals, and combine multiple pattern recognition methods to improve the detection accuracy and efficiency. Compared with the traditional mental state evaluation method, the system described in the present application not only realizes the accurate recognition of mental states such as insomnia and anxiety, but also can provide a personalized regulation plan for the user according to the real-time electroencephalogram data, and regulate the cognitive ability by using technologies such as transcranial magnetic stimulation, neuroelectrical stimulation, and virtual reality to optimize the user's cognitive state.

[0049] In addition, the system described in the present application improves the diagnostic accuracy through multi-modal data fusion, and can be widely applied to fields such as mental health management, cognitive training, and emotion regulation, especially suitable for scenarios such as smart homes, education, and medical assistance. The system described in the present application has high efficiency, accuracy, and personalized regulation ability, can significantly improve the management effect of mental states, and has broad social value and application prospects. Description of the Drawings

[0050] To make the above objects, features, and advantages of the present application more obvious and understandable, the following provides a detailed description of the specific implementation manners of the present application with reference to the accompanying drawings, where:

[0051] Figure 1 Shows a schematic structural diagram of a mental state recognition system provided according to one aspect of the present application;

[0052] Figure 2 Shows a schematic diagram of the PSQI scale division standard in an embodiment of the present application;

[0053] Figure 3 Shows a schematic diagram of the description of multi-channel EEG features in an embodiment of the present application;

[0054] Figure 4 Shows a schematic diagram of the network structure of a multi-size temporal convolutional neural network in an embodiment of the present application;

[0055] Figure 5 Shows a schematic structural diagram of a cognitive ability regulation system provided according to another aspect of the present application;

[0056] Figure 6 Shows a schematic diagram of the system framework composed of the mental state recognition system and the regulation system in an embodiment of the present application.

[0057] The same or similar reference numerals in the drawings represent the same or similar components. Specific implementation manners

[0058] To make the above objects, features, and advantages of the present application more obvious and understandable, the following provides a detailed description of the specific implementation manners of the present application with reference to the accompanying drawings.

[0059] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application may be implemented in other ways different from those described herein, and thus the present application is not limited by the specific embodiments disclosed below.

[0060] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "including" and "comprising" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0061] This application combines EEG signal acquisition, mental state classification, and neural feedback regulation to provide a new and efficient mental state recognition system and regulation system that can accurately identify the user's mental state and adjust their cognitive ability in real time. It has broad application prospects, especially in the fields of mental health management, cognitive training, and emotion regulation. The specific implementation process is as follows:

[0062] Figure 1 The structure diagram of a mental state recognition system based on EEG data according to one aspect of the present application is shown. The mental state recognition system 1 comprises: a signal acquisition module 11 and a mental state classification module 12, wherein:

[0063] The signal acquisition module 11 is used to collect the current EEG signal of the user; here, the signal acquisition module can use an EEG device, which includes a 10 / 20 system standard 64-lead EEG cap, a sampling rate of 1000Hz, and an electrode impedance of less than 50Ω; and then collect the current EEG signal of the user currently wearing the EEG cap, and the collected EEG signal is a multi-channel signal, and the EEG device has a built-in 50Hz notch filter to remove power frequency noise interference. By collecting the user's EEG signal, the user's mental state is monitored in real time, providing basic data for subsequent classification and regulation.

[0064] The mental state classification module 12 includes a recognition model construction unit 121 and an identification unit 122; wherein the recognition model construction unit is used to obtain EEG data samples with different state labels, and construct a mental state recognition model based on the EEG data samples, wherein the EEG data samples include a data set carrying a sleep state label, EEG data carrying an anxiety state label, and resting state EEG data carrying a health state label; the identification unit is used to input the current EEG signal into the mental state recognition model to obtain an evaluation result of the user's current mental state.

[0065] Construct a recognition model for evaluating the user's mental state. The recognition model is constructed after preprocessing the sample data. The sample data is the control group data formed by the self-collected EEG data of the mental state. The data group of the patient's insomnia state label, the data group of the anxiety state label and the data group of the healthy state label of the healthy person can be used as the control group. Based on the sample data, a mental state recognition model is constructed, and then the EEG signal of the user currently wearing the EEG cap is recognized to evaluate and analyze the current mental state of the user. This provides a data basis for optimizing its cognitive ability through the subsequent neurofeedback control system, helping users improve their emotions and mental states.

[0066] When constructing a mental state recognition model, various pattern recognition models can be first constructed based on sample data, and then the mental state recognition model can be obtained by combining the various pattern recognition models with a classification model.

[0067] In some embodiments of the present application, the mental state classification module 12 includes an insomnia detection model construction unit 123 and an anxiety detection model construction unit 124. Among them, the insomnia detection model construction unit 123 is used to construct an insomnia state detection model, and the anxiety detection model construction unit 124 is used to construct an anxiety state detection model; the recognition unit 122 is used to input the current electroencephalogram signal into the insomnia state detection model, the anxiety state detection model, and the mental state recognition model respectively, to obtain a first detection result, a second detection result, and a third detection result, and perform maximum probability discrimination by integrating all the detection results to obtain an evaluation result of the current mental state.

[0068] In the mental state classification module, multiple models are constructed, including an insomnia detection model, an anxiety detection model, and a mental state recognition model. The current electroencephalogram signal of the user is respectively identified and detected to obtain a first detection result, a second detection result, and a third detection result corresponding to the multiple models. By integrating the three detection results, discrimination is performed according to the maximum probability, the probabilities of the three detection results are summarized, and decision-level fusion is performed to obtain a comprehensive decision final result; examples of misjudgment by individual models can be corrected, and the overall discrimination credibility can be improved by using multiple models.

[0069] Specifically, the insomnia detection model construction unit 123 is used to obtain a data set carrying sleep state labels from the database, and construct an insomnia state detection model for electroencephalogram signals according to the data set; the recognition unit 122 is used to use the insomnia state detection model to identify the electroencephalogram signal of the collected user to obtain a first detection result, where the first detection result includes an insomnia state category and a corresponding intensity value.

[0070] In some embodiments of the present application, the insomnia detection model construction unit 123 is used to obtain a cyclic alternating pattern sleep data set and a self - collected mental illness data set, extract time - domain and frequency - domain features from the data sets, and construct an insomnia state detection model for electroencephalogram signals based on the extracted features. Here, in the data selection stage, a cyclic alternating pattern sleep data set and a self - collected mental illness data set are adopted. In the data pre - processing stage, the data sets are pre - processed and then feature extraction is carried out. This feature extraction includes the extraction of time - domain features and frequency - domain features. Furthermore, the time - domain and frequency - domain features are combined with non - linear features for splicing and fusion to construct a feature vector. In the model training stage, support vector machine (SVM) is used as the classification method, and the processed binary classification data set is divided into a training set and a test set according to a ratio of 4:1. To avoid overfitting, the classification performance of a 5 - fold cross - validation model can be used for verification, and finally an insomnia state detection model is constructed. In the recognition unit 122, the constructed insomnia state detection model is used to recognize the electroencephalogram signals of the collected users, and the insomnia state category and the corresponding intensity value are obtained. For example, the categories are divided into normal, mild insomnia, severe insomnia, etc. If the intensity value is a score, a score between 5 and 7 points indicates the mild insomnia category, and a score less than 5 points indicates the severe insomnia category. The intensity value can also be represented as a probability value. For example, an insomnia probability of 80% corresponds to the severe insomnia category.

[0071] Among them, the cyclic alternating pattern (CAP) is a kind of periodic electroencephalogram activity that appears during non - rapid eye movement sleep. Abnormal CAP is related to various sleep - related diseases. The CAP sleep database is an open - source database. The data sets in this database are recorded by a 108 - channel polysomnography (PSG) machine, which contains 16 healthy subjects, 9 insomnia subjects and other patients with sleep disorders. The signals recorded by PSG are multi - modal signals, including electroencephalograms of at least 3 channels, electro - oculograms of 2 channels, electrocardiograms, electromyograms and other respiratory signals. This database aims to provide a large number of carefully annotated representative CAP examples under various pathophysiological conditions for the development and evaluation of automatic CAP analyzers, and to support basic research on CAP dynamics.

[0072] For the insomnia patients in the self - collected mental illness data set, the score division can be determined by using the Pittsburgh sleep quality index (PSQI). As Figure 2 shown in the schematic diagram of the PSQI scale division standard, the criterion for defining insomnia in the data set is that the PSQI score is greater than 5 points.

[0073] Continuing with the above - mentioned embodiments, the insomnia detection model construction unit 123 is used to implement the following steps:

[0074] After downsampling the dataset, filter it through a band-pass filter, decompose it into multiple frequency bands, and determine the range of each frequency band; extract the P-norm feature and the cross-frequency coupling feature respectively based on each frequency band range; determine the time-domain feature based on the P-norm feature and the frequency-domain feature based on the cross-frequency coupling feature, where the frequency-domain feature includes the phase-amplitude coupling feature of phase synchronization and the amplitude correlation feature.

[0075] In the data preprocessing stage, downsample the data in the dataset to 250 Hz, filter it using a band-pass filter, and decompose it into multiple frequency bands. The divided ranges are: δ (0.5 - 4 Hz), θ (4 - 8 Hz), α1 (8 - 10 Hz), α2 (10 - 13 Hz), β1 (14 - 20 Hz), β2 (21 - 30 Hz), and γ1 (31–45 Hz). Extract the P-norm feature and the cross-frequency coupling feature (Cross-frequency coupling, CFC) respectively based on the divided sub-band intervals. The P-norm feature is a type of time norm feature used to measure the distance of vectors in a vector space. Take the three norm forms of the reconstructed electroencephalogram signal x t (n) as the time-domain statistical features of the signal. Assume the signal sample length is N, and the P-norm is defined as follows:

[0076]

[0077] Then the p1-norm feature is The p2-norm feature can be regarded as the root mean square statistical feature of the time series signal, expressed as: p ∞ -norm feature represents the maximum peak feature of the sequence, and its expression is as follows: ||x|| ∞ = max|x(n)|; since a total of 7 sub-bands are divided, the total number of norm features extracted from one sample is 21.

[0078] It should be noted that the above division of the 7 frequency bands avoids excessive complexity caused by overly detailed division and also avoids insufficient resolution due to overly broad division. Each frequency band is sensitive to different neural activity states. For example: δ waves are usually associated with deep sleep and certain pathological states (such as epilepsy); θ waves are more active in states such as attention, memory, and sleep; α waves are mainly related to states such as relaxation and eyes-closed rest; β waves are usually associated with states such as attention concentration and motor control; γ waves are closely related to higher cognitive functions (such as thinking, consciousness, perception, etc.).

[0079] In addition, the insomnia detection model construction unit 123 also extracts the phase - amplitude coupling feature (PAC) of phase synchronization and the amplitude correlation feature (AAC) to describe the interaction between different frequency bands in the electroencephalogram signal. Among them, the phase synchronization PAC feature is extracted using the following steps:

[0080] Determine the low - frequency signal and high - frequency signal of the electroencephalogram signal in each frequency band; perform Hilbert transforms on the low - frequency signal and high - frequency signal respectively to obtain the instantaneous amplitude of the high - frequency signal and the instantaneous phase value of the low - frequency signal; perform band - pass filtering on the instantaneous amplitude of the high - frequency signal within the low - frequency band frequency range, and perform Hilbert transform on the filtered signal again to obtain the high - frequency synchronization phase value; extract the phase - amplitude coupling feature of phase synchronization based on the instantaneous phase value of the low - frequency signal and the high - frequency synchronization phase value.

[0081] The electroencephalogram signal is collected through multiple channels. Assume that the time - series electroencephalogram signal on a certain channel (such as C4 - A1) is x t (n), where n = 1, 2, 3, ……, N. The phase synchronization PAC feature is extracted as follows: Calculate the phase synchronization PAC feature between low and high frequencies. Given the low - frequency section signal as x LF (t) and the high - frequency section signal as x HF (t); perform Hilbert transforms on the two signals respectively, and the following formula can be obtained:

[0082]

[0083] where, Z LF (t) and Z HF (t) are the analytic forms of the low - frequency signal and high - frequency signal respectively, A HF (t) is the instantaneous amplitude of the high - frequency signal, is the instantaneous phase value of the low - frequency signal.

[0084] Next, perform band - pass filtering on the A HF (t) value within the low - frequency band frequency range, and perform Hilbert transform on the filtered envelope spectrum again to obtain the high - frequency synchronization phase value:

[0085] Based on the definition of the phase - locking value (PLV), obtain the PLV feature and the improved phase - locking value (iPLV) feature as follows:

[0086]

[0087] iPLV is mainly used for individual analysis, that is, it considers phase locking at each moment during the calculation, rather than processing the overall signal. Therefore, iPLV can reflect the phase coupling strength between different frequency bands at each time point. Therefore, the iPLV feature can be selected as the PAC feature of the cross-frequency coupling feature, with a value range of 0 to 1. The higher the value, the stronger the interaction between PACs. Through the iPLV feature, the energy fluctuations between different frequency bands of insomnia disorder patients can be fully indicated. Since a total of 7 frequency band ranges are divided during the above frequency band division, the number of iPLV features that can be extracted for each sample is 21.

[0088] AAC is a concept used to describe the amplitude correlation between different frequency oscillations in electroencephalogram signals, mainly used to study the amplitude correlation between faster frequency oscillations (such as the γ frequency band) in electroencephalogram signals. AAC is a non-linear feature.

[0089] Finally, the above-extracted time-domain and frequency-domain features and non-linear features are concatenated and fused to construct a feature vector, and then model training is carried out to construct an insomnia detection model.

[0090] In some embodiments of the present application, the anxiety detection model construction unit 124 is used to obtain electroencephalogram data carrying an anxiety state label and resting-state electroencephalogram data carrying a healthy state label, and construct an anxiety state detection model of the electroencephalogram signal according to the obtained labeled electroencephalogram data; the recognition unit 122 is used to use the anxiety state detection model to recognize the electroencephalogram signal of the collected user, and obtain a second detection result, where the second detection result includes the anxiety state category and the corresponding intensity value.

[0091] The anxiety state detection model is constructed based on multi-channel electroencephalogram signals, specifically using electroencephalogram data of anxiety patients and resting-state electroencephalogram data of healthy people; among them, the electroencephalogram data of anxiety patients is provided by the Rest dataset, and the Rest dataset is obtained by collecting electroencephalogram signals from anxiety patients in the following way: the electroencephalogram signal acquisition device has 64 channels, the acquisition duration is 30 minutes, and the sampling frequency of each lead is 500 Hz. The resting-state electroencephalogram data of healthy people is collected in the following way: collect 64-channel electroencephalogram data of multiple (such as 10) healthy subjects at a specified long time (such as 105 minutes) in the eyes-open resting state and the eyes-closed resting state respectively, as a control group for the electroencephalogram data of anxiety patients. By using multi-channel electroencephalogram signals, the electroencephalogram activities of different brain regions can be captured, thereby improving the accuracy of anxiety state detection.

[0092] Continuing with the above embodiments, the anxiety detection model construction unit 124 is configured to process the acquired labeled EEG data to obtain data in multiple frequency bands, extract features from the EEG signals in each frequency band using filter bank common spatial patterns, calculate non-linear features, concatenate the features of the filter bank common spatial patterns and the non-linear features as the input of the classifier, and obtain an anxiety state detection model for EEG signals.

[0093] In the data preprocessing stage, first, the acquired EEG signals (EEG data of anxiety patients and resting-state EEG data of healthy subjects) are downsampled to 250 Hz, and a type-I Chebyshev band-pass filter is used for band-pass filtering in the frequency band of 1 - 30 Hz to remove low-frequency and high-frequency noise. Then, the signal is further decomposed into multiple frequency bands: α (8 - 13 Hz), β (13 - 30 Hz), θ (4 - 8 Hz), and δ (1 - 4 Hz). In the feature extraction stage, for each frequency band decomposed in the data preprocessing stage, filter bank common spatial patterns (CSP) are used to extract features from the signals to capture the spectral differences between anxiety patients and healthy control groups. In addition, non-linear features such as Renyi entropy are also calculated as auxiliary information to improve the accuracy of the anxiety detection model. The extracted spectral features and non-linear features are concatenated as the input of the classifier. In the model training stage, support vector machine (SVM) is used as the classification algorithm. The processed data set is divided into a training set and a test set, and the leave-one-out cross-validation (LOOCV) method is used to verify the accuracy of the anxiety state detection model to ensure its stability and reliability in practical applications.

[0094] Continuing with the above embodiments, the steps of using filter bank common spatial patterns for feature extraction include: determining the average spatial covariance matrix in the first category and the average spatial covariance matrix in the second category for each frequency band; calculating the mixed spatial covariance in the first category and the second category, and calculating the spatial projection matrix based on the mixed spatial covariance, where the spatial projection matrix minimizes the within-class variance and maximizes the between-class variance; using the spatial projection matrix to extract the logarithmic variance features of the EEG signals within the frequency band.

[0095] Since there are significant differences in the energy characteristics of the resting EEG between anxiety patients and healthy subjects in different brain regions within the 1 - 30 Hz frequency band; in order to make full use of the spatial spectral energy difference features between different frequency bands, an embodiment of the present application proposes a feature extraction method based on filter bank common spatial patterns, and the specific steps are as follows:

[0096] Given that the i-th EEG data sample is represented as X i , where X i is a two-dimensional matrix with the number of channels N c ×N s and the number of sampling points. Assume that Xi The number of decomposed sub-bands is N b , and the EEG data of the k-th sub-band can be expressed as

[0097] In the process of solving the spatial filter of CSP, assume that the total number of training samples in the first category (category I) is m1, and the total number of training samples in the second category (category II) is m2. The basic idea of the common spatial pattern algorithm based on the filter bank is: find a spatial projection matrix W csp to minimize the within-class variance and maximize the between-class variance.

[0098] For the EEG signal of the k-th sub-band, the average spatial covariance matrix in category I is as follows:

[0099]

[0100] The average spatial covariance matrix in category II is calculated as follows:

[0101]

[0102] where T represents the transpose operation of the matrix, and trace() represents the sum of the diagonal elements of the matrix. represents the EEG signal of the k-th sub-band in category I. represents the EEG signal of the k-th sub-band in category II.

[0103] Then the mixed spatial covariance in category I and category II can be diagonalized as:

[0104]

[0105] where λ is a diagonal matrix of N c ×N c , the diagonal elements are the eigenvalues of the matrix and are arranged in descending order, and U is the eigenvector matrix of.

[0106] Next, the characteristic whitening matrix is calculated as: S1 = PR1P T , S2 = PR2P T ;

[0107] It is further diagonalized as:

[0108] where S1 and S2 share the same eigenvalues. Therefore, we get B1 = B2 = B and λ1 + λ2 = I, that is, the larger the eigenvalue of S1, the smaller the eigenvalue of S2.

[0109] Finally, the spatial projection matrix Wcsp The solution is as follows: W csp = B T P, the required spatial projection matrix W csp is a matrix with the number of channels N c × N c In this task, the first m eigenvectors and the last m eigenvectors with the largest eigenvalues are selected to construct the spatial projection matrix for the subsequent spatial transformation and feature extraction; as shown in the following formula, the original EEG signal of the k-th sub-band after spatial projection transformation is:

[0110]

[0111] where, is the spatial projection matrix after selection.

[0112] Next, the logarithmic variance features are extracted from the time series of the c-th channel of the transformed electroencephalogram signal, and its specific expression is as follows:

[0113]

[0114] Therefore, the number of logarithmic variance statistical features that can be extracted for each sample is: N b × 2m.

[0115] In some embodiments of the present application, the non-linear features include Rényi entropy features and correlation dimension, and the Rényi entropy features are determined using the following steps: dividing the amplitude of the electroencephalogram signal into multiple sub-intervals and determining the probability of each sub-interval, and determining the Rényi entropy according to the sub-intervals and the corresponding probabilities.

[0116] Rényi entropy features are often used to extract dynamic information of EEG activities. It is an entropy measurement method in information theory and a way to calculate the entropy of the probability distribution of a random variable, so as to measure the uncertainty or chaos degree of information. Using Rényi entropy features can represent the activity complexity of the brain. In information theory, entropy is a measurement of uncertainty in random events. For a random variable X, its entropy H(X) can be expressed as:

[0117] H(X) = -∑P(x)*log(P(x));

[0118] where, Σ represents summation, P(x) is the probability that the random variable X takes the value x, and log is the logarithm with base 2. Of course, other bases can also be used.

[0119] Research shows that the signal complexity of the brain activities of patients with anxiety disorder is very low and is significantly different from that of the control group. Compared with other entropy algorithms, Rényi entropy does not require the assumption that the signal must satisfy a Gaussian distribution or a stationary signal.

[0120] Suppose the original EEG signal can be expressed as follows: X = {x1, x2, ……, x n}, according to the amplitude of the EEG signal, it can be divided into N sub-intervals. According to the difference in the amplitude of the EEG signal, the EEG signal X belongs to different sub-intervals. Suppose p i is the probability corresponding to each sub-interval, and the Renyi entropy is defined as follows:

[0121]

[0122] Among them, when α = 1, it represents the Shannon entropy feature, that is, the common information entropy. In the embodiments of the present application, α = 2 is selected.

[0123] Continuing with the above embodiments, the correlation dimension is determined using the following steps: The EEG signal is set as a time series, and the time series is calculated according to the selected time delay parameter and embedding dimension to obtain the correlation dimension.

[0124] The dimension is one of the most basic quantities in geometric objects, used to describe the number of independent parameters or coordinates contained in a space or set. Usually, the dimension needs to be a non-negative integer because the dimension is used to represent the number of independent parameters in a space or set, and the number of independent parameters cannot be negative or fractional. However, there are many limitations in describing and analyzing complex non-linear systems, so the correlation dimension is introduced, and this correlation dimension is the correlation dimension of the model. The correlation dimension can measure the complexity of non-linear signals. There are significant differences in the correlation dimensions of EEG signals in different mental task awareness states. The corresponding feature calculation method is as follows:

[0125] First, the EEG signal is set as a time series (a t |t = 1, 2, …, T). After selecting the time delay τ and embedding dimension m, the phase space vector can be expressed as:

[0126]

[0127] Among them, is the reconstructed phase space vector, a(i) is the scalar time series, N is the length of the data sequence, and the correlation integral C(r) can be expressed as:

[0128]

[0129] In the formula, r represents the distance between neighbors, N represents the number of vectors, and H represents the heightening function proposed by Taylor, and its expression is as follows:

[0130]

[0131] Theoretically, when m, n → ∞, r → 0, the following equation holds: logC mIf \(C(r)=\log C + D_2\log r\), the correlation dimension can be defined as:

[0132]

[0133] Complex signals can be recognized as a mixture of regular components and random components. To obtain the proportion of irregular components in the original signal, the corresponding C0 complexity features are extracted in the embodiments of the present application, and the method is as follows:

[0134] The C0 complexity represents the proportion of irregular components in the original signal. Assuming that the electroencephalogram signal consists of N time series, it is expressed as: \(\{x(n), n = 0, 1, \ldots, N - 1\}\). First, the signal is subjected to a Fast Fourier Transform (FFT):

[0135]

[0136] Next, the average amplitude of the power spectrum is calculated as:

[0137]

[0138] A new frequency spectrum sequence \(Y(k)\) is generated as follows:

[0139]

[0140] Next, the new frequency spectrum sequence is subjected to FFT to obtain a new time series \(y(n)\). Then, the C0 complexity feature is defined as follows:

[0141]

[0142] In summary, by focusing on the spectral features and non-linear features of the diffuse electroencephalogram signals of patients with anxiety disorder, a total of 960-dimensional features are extracted, as Figure 3 shown in the schematic diagram of the multi-channel EEG feature description. These features are concatenated and used as the input to the classifier.

[0143] In some embodiments of the present application, a mental state recognition model is constructed to simultaneously detect insomnia and anxiety states, and is used in combination with an insomnia detection model and an anxiety detection model. Among them, when constructing the mental state recognition model, the recognition model construction unit 121 is used to input the electroencephalogram data sample into a multi-size temporal convolutional neural network for multi-layer convolutional operations to extract features, classify the extracted features, and train to obtain a mental state recognition model; the recognition unit is used to use the mental state recognition model to recognize the current electroencephalogram signal to obtain a third detection result. Here, the self-collected electroencephalogram data of the mental state is used as a sample, including the electroencephalogram signals of insomnia, anxiety, and healthy control groups. Based on the multi-size temporal convolutional neural network (CNN), the collected data is extracted and classified to train a mental state recognition model; this model can effectively extract multi-level features from the electroencephalogram signal, and can simultaneously identify and detect the electroencephalogram signal with the insomnia detection model and the anxiety detection model, and then obtain the corresponding detection results, and comprehensively judge all the detection results to improve the recognition accuracy of different mental states.

[0144] The construction of the recognition model is trained by a temporal network composed of a convolutional layer, a pooling layer, a fully connected layer, etc. The temporal network is divided into two main blocks. The first module extracts the shallow features of the signal, and the shallow features refer to the primary spatio-temporal features obtained by the multi-scale convolutional layer; the second module extracts the deep features of the signal, and the deep features are further processed by the convolutional layer on the primary spatio-temporal features to be transformed into high-level spatio-temporal features. Finally, the shallow features output by the first module and the deep features output by the second module are input into the classification module for classification. Specifically, the multi-size temporal convolutional neural network includes a first module, a second module, and a classification model; the first module includes a first convolutional layer, a second convolutional layer, a first batch normalization layer, a second batch normalization layer, an activation layer, an average pooling layer, and a dropout layer, and a batch normalization layer follows each convolutional layer; the second module includes a third convolutional layer, a fourth convolutional layer, a third batch normalization layer, an activation layer, an average pooling layer, and a dropout layer; the classification model includes a fully connected layer.

[0145] Such as Figure 4Schematic diagram of the network structure of the multi-size temporal convolutional neural network shown. In the first module, there are 2 convolutional layers. The kernel sizes of convolutional layer 1 and convolutional layer 2 are both (1, 128). The input channel of convolutional layer 1 is 1 and the output channel is 16. The input channel of convolutional layer 2 is 16 and the output channel is 32. The output contains the F1 feature map of different band-pass frequency electroencephalogram signals. After each convolutional layer, there is 1 batch normalization layer to accelerate the training speed and maintain the stability of model training. At the end of the first module, an activation layer ELU, an average pooling layer, and a dropout layer are successively adopted. To help with regularization or modeling, the dropout technique is used, and the dropout probability for within-subject classification is set to 0.5 to help prevent overfitting during training with small samples, and an average pooling layer with a size of (1, 4) is applied for dimensionality reduction.

[0146] Depth convolution is performed in the second module. For example, separable convolution is used. The separable convolution is a convolution with a size of (1, 16), followed by a point convolution with a size of (1, 1). By using separable convolution, the number of fitting parameters can be reduced, and it can learn to summarize each feature map separately and then optimally merge the outputs, decoupling the relationships within and across feature maps. When used in EEG-specific applications, this operation separates how to summarize individual feature maps in a timely manner (depth convolution) from how to optimally combine feature maps (point convolution). This operation is also particularly useful for electroencephalogram signals because different feature maps may represent information data at different time scales, and an average pooling layer with a size of (1, 8) is applied for dimensionality reduction.

[0147] A fully connected layer is used in the classification module to directly pass the features output by module 1 and module 2 to a softmax classification with N units, where N is the number of classes in the data. Before the Softmax classification layer, the use of a dense layer for feature aggregation is omitted to reduce the number of free parameters in the model.

[0148] During the model training stage, the Focal loss function is used to optimize the model to handle the class imbalance problem. The model is trained using the Stochastic Gradient Descent (SGD) optimizer, and the cross-validation is used to verify the model performance. Through multiple experimental verifications, the model can accurately identify mental states such as insomnia and anxiety and has good generalization ability.

[0149] Figure 5The structural schematic diagram of a regulation system for cognitive ability provided according to another aspect of the present application is shown. The regulation system 2 includes: a neurofeedback regulation module 21 and a user feedback module 22. Among them, the neurofeedback regulation module 21 is used to monitor the electroencephalogram signal of the user in real time, obtain the evaluation result of the mental state of the user recognized by the mental state recognition system based on the electroencephalogram signal, and regulate the mental state of the user based on the regulation strategy; the user feedback module 22 is used to monitor the feedback information of the user during the regulation process and adjust the regulation strategy according to the feedback information.

[0150] The neurofeedback regulation module 21 is used to regulate the cognitive ability of the user by monitoring the electroencephalogram signal of the user in real time and adopting different regulation strategies according to the evaluation result of the mental state of the user by the mental state recognition system in the above embodiment. Among them, the regulation strategy may include regulation means provided by technologies such as transcranial magnetic stimulation, neuroelectrical stimulation, and virtual reality; through this module, the system can accurately adjust the brain function state of the user and optimize its cognitive ability and emotional state. The user feedback module 22 evaluates the effect of neurofeedback regulation by collecting the physiological and psychological parameters of the user in real time and dynamically adjusts the regulation strategy according to this feedback information; this module ensures that the regulation plan is not only effective in the short term, but also can maintain cognitive improvement for a long time.

[0151] In some embodiments of the present application, the neurofeedback regulation module 21 includes a transcranial magnetic stimulation unit 211, a neuroelectrical stimulation unit 212, and a virtual reality interaction unit 213; the transcranial magnetic stimulation unit 211 is used to transmit a magnetic field to the cerebral cortex of the user and adjust the frequency and intensity of magnetic stimulation according to the electroencephalogram signal and feedback information of the user; the neuroelectrical stimulation unit 212 is used to stimulate the cerebral cortex or a specified neural pathway of the user through an electric current and adjust the intensity and stimulation mode of the electric current according to the electroencephalogram signal and feedback information of the user; the virtual reality interaction unit 213 is used to provide a virtual environment for the user so that the user can perform cognitive task training in the virtual environment and adjust the stimulation content and intensity of the virtual environment according to the feedback information.

[0152] Transcranial magnetic stimulation is a non-invasive brain stimulation technology that affects the electrical activity of neurons by transmitting a magnetic field to the cerebral cortex, thereby regulating brain function. In the embodiment of the present application, the transcranial magnetic stimulation unit 211 adjusts the frequency and intensity of magnetic stimulation through the electroencephalogram signal data collected in real time, and accurately regulates the activity of specific brain regions. The changes in stimulation intensity and frequency will be dynamically adjusted according to the electroencephalogram data of the user to ensure the improvement of cognitive ability. Especially in the intervention of mental problems such as anxiety and insomnia, it can effectively help users regulate their emotions, improve attention and memory.

[0153] The nerve electrical stimulation unit 212 stimulates the cerebral cortex or specific nerve pathways through microcurrents to regulate the nerve activities of the brain, achieving the purpose of enhancing cognitive ability and emotion regulation; according to the real-time electroencephalogram data and user feedback, the system automatically adjusts the intensity and stimulation mode of the current. Different current intensities, frequencies and durations can affect the excitability of neurons, thus helping to improve the user's attention, memory and cognitive control ability, and is particularly applicable to cognitive intervention in situations such as anxiety and insomnia.

[0154] The virtual reality interaction unit 213 stimulates the user's senses (vision, hearing, touch, etc.) through an immersive environment, providing the user with an experience of cognitive training and emotion regulation. In the embodiment of the present application, the virtual reality interaction unit 213 helps the user to perform cognitive task training in a simulated scenario by creating a dynamic and interactive virtual environment. For example, the virtual environment can be designed to regulate the user's attention, deal with anxiety emotions, improve memory, etc. The system adjusts the stimulation content and intensity of the virtual environment in real time according to the user's feedback, ensuring the most suitable cognitive training effect, and optimizing the emotion regulation and cognitive training strategies through system feedback.

[0155] In some embodiments of the present application, the regulation strategy includes regulation information of transcranial magnetic stimulation, nerve electrical stimulation and virtual reality, wherein the regulation information includes regulation intensity, duration, frequency and regulation content. Here, when the mental state of the user is determined, the nerve feedback regulation module 21 provides corresponding cognitive ability regulation according to the evaluation result; for example, by means of transcranial magnetic stimulation (TMS), nerve electrical stimulation or virtual reality (VR) interaction, etc., to optimize the user's cognitive state and emotional response, and the system adjusts the stimulation intensity, frequency, duration and regulation content (such as feedback mode) in real time to help the user improve their attention, emotion and cognitive ability.

[0156] In some embodiments of the present application, the user feedback module 22 includes a physiological feedback monitoring unit 221 and a psychological feedback evaluation unit 222. Among them, the physiological feedback monitoring unit 221 is used to monitor the physiological parameters of the user to evaluate the regulation effect; the psychological feedback evaluation unit 222 is used to evaluate the changes in the user's psychological state and cognitive ability in real time according to the relevant data of the user, where the relevant data includes the user's self-report and mood scale. Here, the physiological feedback monitoring unit 221 is responsible for monitoring and recording the physiological parameters of the user in real time. The physiological parameters mainly include data such as heart rate, skin conductance response, and blood pressure, and the user's emotional state and psychological burden are reflected through the physiological parameters. Through the analysis of physiological signals, the system can judge the user's emotional fluctuations, stress levels, etc., and then adjust the cognitive ability regulation. For example, when it is detected that the user is anxious or stressed, the system may increase the adjustment intensity of the virtual reality interaction unit or adjust the TMS stimulation mode to better relieve the user's emotional stress. The psychological feedback evaluation unit 222 evaluates the user's psychological state in real time through the user's self-report (such as mood scale, cognitive self-evaluation, etc.) and professional evaluation tools (such as HAM-D, PSQI, etc.). Based on the mood scale or psychological evaluation results feedback by the user, the system can accurately understand the user's current emotional and cognitive state, and accordingly adjust the intervention measures of the neurofeedback regulation module. For example, when the user reports a low mood or difficulty in concentrating, the system will timely adjust the neuroelectrical stimulation and the stimulation content in the VR environment to help the user restore a good mood and cognitive ability.

[0157] After the user feedback information is analyzed in real time, it is fed back to the neurofeedback regulation module 21, and the system will make personalized adjustments to the regulation plan according to the user's emotional response and physiological indicators. For example, if the user reports a high level of anxiety, the system will increase the intensity of neuroelectrical stimulation, or provide a more immersive emotional regulation experience through the virtual reality environment. If the user reports an improvement in cognitive ability, the stimulation intensity can be appropriately reduced to avoid over-regulation. Through continuous feedback and adjustment, the system can continuously optimize the regulation plan to ensure the persistence and accuracy of cognitive ability improvement.

[0158] Figure 6The schematic diagram of the system framework composed of the mental state recognition system and the control system in one embodiment of the present application is shown, wherein the mental state recognition system 1 is used to collect the current EEG signal of the user, and obtain the self-collected mental state EEG data, build a mental state recognition model based on the self-collected mental state EEG data, pre-process the current EEG signal and input it into the mental state recognition model, and output the user's current mental state; the control system 2 is used to obtain the mental state of the user recognized by the mental state recognition system, control the mental state of the user based on the control strategy, monitor the user's feedback information during the control process, and adjust the control strategy according to the feedback information. In this way, not only accurate recognition of mental states such as insomnia and anxiety is achieved, but also personalized control schemes can be provided for users to control cognitive abilities based on real-time EEG data, thereby optimizing the user's cognitive state.

[0159] In some embodiments of the present application, the following steps can be implemented using the mental state recognition system and control system described above:

[0160] Step S11, collect the user's current EEG signal; here, the user wears a 64-channel EEG cap, the sampling rate is 1000Hz, and the electrode impedance is less than 50Ω, ensuring that multi-channel EEG signals can be collected; and the 50Hz notch filter built into the EEG cap can be used to remove power frequency noise interference. By collecting the user's EEG signal, the user's mental state is monitored in real time, providing basic data for subsequent classification and regulation.

[0161] The type of paradigm for the experiment can be determined, and the scale assessment and attention test can be performed according to the paradigm type to determine the relevant data of the user, wherein the paradigm type includes an open-eye resting task and a closed-eye resting task; and the user's psychological state and cognitive ability changes are evaluated in real time according to the relevant data of the user. Here, the system first performs a scale assessment and an attention test according to the selected paradigm; wherein the scale assessment is used to record the user's condition and attention level, so as to design a personalized regulation plan for the user later; the attention test includes the attention network test (ANT) and the sustained attention response task (SART) test. ANT is used to evaluate the user's reaction speed and accuracy and analyze its performance under different attention networks; SART is used to evaluate the user's sustained attention maintenance ability and ability to inhibit response in long-term tasks. Next, the system conducts experiments based on resting state EEG signals. The resting state experimental paradigm includes an open-eye resting task and a closed-eye resting task; each task lasts 4 minutes, with a 2-minute rest adjustment time between tasks, and the experiment is repeated 2 to 3 times. Through the data collection of these tasks, the system analyzes the user's EEG signals in real time to provide a basis for the subsequent mental state classification.

[0162] Specifically, corresponding stimulus types and label signals are activated according to the paradigm type, and the user's current electroencephalogram (EEG) signal is collected according to the acquisition type and label signal. Here, in the data acquisition stage, the system manages and controls the experimental paradigm through BCI software. The software activates the corresponding stimulus types and label signals according to the selected paradigm, and evaluates the attention and emotional state by real-time recording of the EEG signal.

[0163] Step S12: Obtain EEG data samples of different state categories and corresponding intensity values, and construct a mental state recognition model based on the EEG data samples. Here, the different state categories include the insomnia state category, the anxiety state category, and the healthy state category. Here, when the user is sitting in front of the display, the system starts to operate according to a preset method, collects the user's EEG signal in real time through the EEG signal acquisition module, and analyzes the user's mental state through the mental state classification module. A recognition model for evaluating the user's mental state is constructed. The recognition model is constructed after preprocessing the sample data. The sample data is the control group data formed by the self-collected mental state EEG data. The data groups with the insomnia state category labels, the anxiety state category labels of the obtained patients, and the data group with the healthy state category labels of healthy individuals can form a control group, and a mental state recognition model is constructed based on the sample data.

[0164] Step S13: Input the current EEG signal into the mental state recognition model to obtain an evaluation result of the user's current mental state. Here, the EEG signal of the user currently wearing the EEG cap is recognized to evaluate and analyze the current mental state of the user. Here, after the user completes the task, the system performs data analysis through the mental state classification module. This module extracts time domain, frequency domain, and non-linear features to classify and identify the user's mental state, and identifies whether there are mental state problems such as insomnia and anxiety. Thus, it provides a data basis for optimizing the user's cognitive ability through the neurofeedback regulation system in the follow-up, and helps the user improve their mood and mental state.

[0165] Step S14: Regulate the user's current mental state based on the regulation strategy. Here, after determining the user's mental state, the neurofeedback regulation module provides corresponding cognitive ability regulation according to the evaluation result. For example, through transcranial magnetic stimulation (TMS), neuroelectrical stimulation, or virtual reality (VR) interaction, etc., to optimize the user's cognitive state and emotional response. The system adjusts the stimulation intensity, duration, and feedback mode in real time to help the user improve their attention, mood, and cognitive ability.

[0166] Step S15: Monitor the feedback information of the user during the regulation process, and adjust the regulation strategy according to the feedback information. During the regulation process, the user feedback module continuously monitors and records the feedback information of the user, and the feedback information includes physiological parameters (such as heart rate, skin conductance response) and psychological feedback; the system dynamically adjusts the regulation strategy according to the feedback information to ensure the persistence and accuracy of the regulation effect. Throughout the process, the system also implements an error detection and status feedback mechanism to ensure the precise execution of each link; if an abnormality occurs, the system will inform the user of the current operation status and the problem through the interface feedback, and provide corresponding solutions.

[0167] Finally, through the combination of EEG data and cognitive regulation programs, an efficient and personalized mental state management service is provided for users, which is widely applied in the fields of mental health management, cognitive training, emotion regulation, etc.

[0168] It should be understood that the embodiments described above are illustrative only. The embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or a combination thereof.

[0169] Some aspects of the present application can be executed entirely by hardware, can be executed entirely by software (including firmware, resident software, microcode, etc.), or can be executed by a combination of hardware and software. The above hardware or software can be referred to as "data blocks", "modules", "engines", "units", "components", or "systems". The processor can be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or a combination thereof. In addition, aspects of the present application may be embodied as a computer product located on one or more computer readable media, which includes computer readable program codes. For example, the computer readable media may include, but is not limited to, magnetic storage devices (such as hard disks, floppy disks, magnetic tapes...), optical disks (such as compact disks CD, digital versatile disks DVD...), smart cards, and flash memory devices (such as cards, sticks, key drives...).

[0170] A computer-readable medium may include a propagated data signal having computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take any of a variety of forms, including electromagnetic, optical, or the like, or any suitable combination thereof. A computer-readable medium may be any computer-readable medium other than a computer-readable storage medium that can communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device. Program code located on a computer-readable medium may be propagated by any suitable medium, including radio, cable, fiber optic cable, RF signals, or similar media, or any combination of the foregoing media.

[0171] The basic concepts have been described above. Obviously, for those skilled in the art, the above invention disclosure is only an example and does not constitute a limitation to this application. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are proposed in this application, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0172] Meanwhile, this application uses specific terms to describe the embodiments of this application. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0173] In some embodiments, numbers are used to describe components and attribute quantities. It should be understood that such numbers used in the description of embodiments are modified by the modifiers "about", "approximate", or "substantially" in some examples. Unless otherwise specified, "about", "approximate", or "substantially" indicate that the said numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used to confirm the breadth of the scope in some embodiments of this application are approximate values, in specific embodiments, such numerical settings are made as precise as possible within the feasible range.

Claims

1. A mental state recognition system based on electroencephalogram data, characterized in that, The mental state recognition system includes: a signal acquisition module and a mental state classification module, where the signal acquisition module is used to acquire the current electroencephalogram (EEG) signal of the user; the mental state classification module includes an identification model construction unit and an identification unit; the identification model construction unit is used to obtain EEG data samples with different state labels, and construct a mental state recognition model based on the EEG data samples. Among them, the EEG data samples include a dataset with a sleep state label, EEG data with an anxiety state label, and resting-state EEG data with a healthy state label; the identification unit is used to input the current EEG signal into the mental state recognition model to obtain an evaluation result of the user's current mental state.

2. The mental state recognition system according to claim 1, characterized in that the mental state classification module includes an insomnia detection model construction unit and an anxiety detection model construction unit, where the insomnia detection model construction unit is used to construct an insomnia state detection model, and the anxiety detection model construction unit is used to construct an anxiety state detection model; the identification unit is used to input the current EEG signal into the insomnia state detection model, the anxiety state detection model, and the mental state recognition model respectively to obtain a first detection result, a second detection result, and a third detection result, and perform maximum probability discrimination by integrating all detection results to obtain a final evaluation result of the current mental state.

3. The mental state recognition system according to claim 2, characterized in that, The insomnia detection model construction unit is used to obtain a dataset with a sleep state label from the database, and construct an insomnia state detection model for the EEG signal according to the dataset; the identification unit is used to use the insomnia state detection model to identify the current EEG signal to obtain a first detection result, where the first detection result includes an insomnia state category and a corresponding intensity value.

4. The mental state recognition system according to claim 3, characterized in that The insomnia detection model construction unit is used to implement the following steps: Downsample the dataset and filter it through a band-pass filter, decompose it into multiple frequency bands, and determine the range of each frequency band; Extract P-norm features and cross-frequency coupling features based on each frequency band range respectively; Determine time-domain features based on the P-norm features, and determine frequency-domain features based on the cross-frequency coupling features. Among them, the frequency-domain features include phase-amplitude coupling features of phase synchronization and amplitude correlation features; Construct an insomnia detection model for the EEG signal based on the extracted time-domain features and frequency-domain features.

5. The mental state recognition system according to claim 4, characterized in that, The phase-amplitude coupling feature of phase synchronization is extracted using the following steps: Determine the low-frequency signal and high-frequency signal of the EEG signal in each frequency band; Perform Hilbert transforms on the low-frequency signal and high-frequency signal respectively to obtain the instantaneous amplitude of the high-frequency signal and the instantaneous phase value of the low-frequency signal; Perform band-pass filtering on the instantaneous amplitude of the high-frequency signal within the low-frequency band frequency range, and perform Hilbert transform on the filtered signal again to obtain the high-frequency synchronization phase value; Extract the phase-amplitude coupling feature of phase synchronization based on the instantaneous phase value of the low-frequency signal and the high-frequency synchronization phase value.

6. The mental state recognition system according to claim 2, characterized in that The anxiety detection model construction unit is used to obtain EEG data with anxiety state labels and resting-state EEG data with healthy state labels, and construct an anxiety state detection model for EEG signals based on the obtained labeled EEG data; the recognition unit is used to use the anxiety state detection model to recognize the current EEG signal, and a second detection result, where the second detection result includes the anxiety state category and the corresponding intensity value.

7. The mental state recognition system according to claim 6, characterized in that, The anxiety detection model construction unit is used to process the obtained labeled EEG data to obtain data in multiple frequency bands, extract features from the EEG signals in each frequency band using a filter bank common spatial pattern, and calculate non-linear features. The features of the filter bank common spatial pattern and the non-linear features are concatenated as the input of the classifier to obtain an anxiety state detection model for EEG signals.

8. The mental state recognition system according to claim 7, characterized in that The steps of extracting features using the filter bank common spatial pattern include: Determine the average spatial covariance matrix in the first category and the average spatial covariance matrix in the second category for each frequency band; Calculate the mixed spatial covariance in the first category and the second category, and calculate the spatial projection matrix based on the mixed spatial covariance, where the spatial projection matrix minimizes the within-class variance and maximizes the between-class variance; Use the spatial projection matrix to extract the logarithmic variance features of the EEG signals within the frequency band.

9. The mental state recognition system according to claim 7, characterized in that, The non-linear features include Renyi entropy features and correlation dimensions, The Renyi entropy features are determined using the following steps: divide the amplitude of the EEG signal into multiple sub-intervals and determine the probability of each sub-interval, and determine the Renyi entropy based on the sub-intervals and the corresponding probabilities; The correlation dimension is determined using the following steps: set the EEG signal as a time series, and calculate the time series based on the selected time delay parameter and embedding dimension to obtain the correlation dimension.

10. The mental state recognition system according to claim 2, wherein The recognition model construction unit is used to input EEG data samples into a multi-size temporal convolutional neural network for multi-layer convolutional operations to extract features, perform classification operations on the extracted features, and train to obtain a mental state recognition model; the recognition unit is used to use the mental state recognition model to recognize the current EEG signal to obtain a third detection result.

11. The mental state recognition system according to claim 10, characterized in that, The multi-size temporal convolutional neural network includes a first module, a second module, and a classification model; The first module includes a first convolutional layer, a second convolutional layer, a first batch normalization layer, a second batch normalization layer, an activation layer, an average pooling layer, and a dropout layer, and a batch normalization layer follows each convolutional layer; The second module includes a third convolutional layer, a fourth convolutional layer, a third batch normalization layer, an activation layer, an average pooling layer, and a dropout layer; The classification model includes a fully connected layer.

12. A regulation system using the mental state recognition system according to any one of claims 1 to 11, characterized in that, The regulation system includes: a neurofeedback regulation module and a user feedback module, where The neurofeedback regulation module is used to monitor the EEG signals of the user in real time, obtain the evaluation result of the mental state of the user recognized by the mental state recognition system based on the EEG signals, and regulate the mental state of the user based on the regulation strategy; The user feedback module is used to monitor the feedback information of the user during the regulation process, and adjust the regulation strategy according to the feedback information.

13. The regulation system according to claim 12, wherein The neurofeedback regulation module includes a transcranial magnetic stimulation unit, a neural electrical stimulation unit, and a virtual reality interaction unit; The transcranial magnetic stimulation unit is used to transmit a magnetic field to the cerebral cortex of the user and adjust the frequency and intensity of magnetic stimulation according to the electroencephalogram signal and feedback information of the user; The neural electrical stimulation unit is used to stimulate the cerebral cortex or a specified neural pathway of the user through an electric current and adjust the intensity and stimulation mode of the current according to the electroencephalogram signal and feedback information of the user; The virtual reality interaction unit is used to provide a virtual environment for the user so that the user can perform cognitive task training in the virtual environment and adjust the stimulation content and intensity of the virtual environment according to the feedback information.

14. The regulation system according to claim 13, wherein, The regulation strategy includes the regulation information of transcranial magnetic stimulation, neural electrical stimulation, and virtual reality. Among them, the regulation information includes regulation intensity, duration, frequency, and regulation content.

15. The regulation system according to claim 12, characterized in that, The user feedback module includes a physiological feedback monitoring unit and a psychological feedback evaluation unit. Among them, the physiological feedback monitoring unit is used to monitor the physiological parameters of the user to evaluate the regulation effect; The psychological feedback evaluation unit is used to evaluate the changes in the psychological state and cognitive ability of the user in real time according to the relevant data of the user. Among them, the relevant data includes the user's self-report and emotion scale.

Images