Audio Intelligent Intervention System and Method for Mental State Monitoring and Intervention

Abstract

An audio intelligent intervention system and method for mental state monitoring and intervention. The information acquisition module is used to collect the information and audio type preferences of the subject, and the electroencephalogram (EEG) signal acquisition and processing module collects the EEG signals at three positions of Fp1, FpZ, and Fp2 of the subject and performs preprocessing operations; the mental state evaluation module using the transfer learning algorithm and regression model extracts features from the preprocessed EEG signals and predicts the negative emotion probability to achieve the evaluation of the mental state; the audio playback module plays the corresponding audio according to the evaluation result and the selected audio type preference, continues to monitor the mental state, and uses the audio element adjustment module containing the reinforcement learning algorithm to adjust the music elements to achieve the playback of new audio and thus eliminate the negative emotions of the subject; the storage and output module stores the evaluation result information, outputs the subject information, the mental state evaluation result before audio playback, and the mental state evaluation result after audio element adjustment.

Description

Technical Field

[0001] The present invention relates to the technical field of mental state monitoring and intervention, and particularly to an audio intelligent intervention system and method for mental state monitoring and intervention. Background Art

[0002] Existing research has shown that negative emotions are likely to have negative impacts on quality of life, interpersonal communication, etc., such as in references [1][2]. Therefore, timely detection and intervention of negative emotions are of great significance for realizing mental state monitoring and intervention, and promoting people's psychological and physiological health.

[0003] However, the current judgment of negative emotions mainly relies on scales, such as in references [3][4], which is likely to obtain inaccurate information, thus leading to wrong judgments of the results. In addition, this method of relying on scales for judgment requires professional evaluation, resulting in a certain degree of subjectivity.

[0004] Secondly, the types of audio used for negative emotion intervention are single, mainly using music as the single audio type, such as in reference [5], making it difficult to achieve the purpose of effective intervention; moreover, the music used is pre-set, making it difficult to achieve personalized and precise intervention. Patents [6][7] adopt the paradigm of pre-set audio stimuli, however, the pre-set audio cannot achieve personalized and precise intervention; secondly, patents [6][7][8] classify emotions, and this method can only obtain a certain emotion, but cannot obtain the specific probability value or degree of a certain emotion, thus making it difficult to provide an accurate and highly targeted intervention plan. At the same time, in patent [8], it emphasizes a music generation method, that is, mapping electroencephalogram signals into music parameters, and then intervening on the subjects. The invention point in patent [9] is a set of hardware systems, emphasizing the connection relationship between each hardware, but lacking an algorithm for mental state assessment.

[0005] The references are as follows:

[0006] [1] Ekici A, Ekici M, Kara T, et al. Negative Mood and Quality of Life in Patients with Asthma[J]. Quality of Life Research, 2006, 15(1): 49-56;

[0007] [2]Winton E C, Clark D M, Edelmann R J. Social Anxiety, Fear of Negative Evaluation and the Detection of Negative Emotion in Others[J]. Behaviour research and therapy, 1995, 33(2): 193-196.

[0008] [3]Spada M M, Nikčević A V, Moneta G B, et al. Metacognition, Perceived Stress, and Negative Emotion[J]. Personality and Individual Differences, 2008, 44(5):1172-1181;

[0009] [4]Kim G, Walden T, Harris V, et al. Positive Emotion, Negative Emotion, and Emotion Control in the Externalizing Problems of School-aged Children[J]. Child psychiatry and human development, 2007, 37(3): 221-239;

[0010] [5]Feneberg A C, Mewes R, Doerr J M, et al. The Effects of Music Listening on Somatic Symptoms and Stress Markers in the Everyday Life of Women with Somatic Complaints and Depression[J]. Scientific reports, 2021, 11(1): 1-12;

[0011] [6]Zhang Tong, Jia Xue, Xu Xiangmin, Chen Junlong, Hu Bin. Mental State Assessment System Based on Holographic Projection Technology and Its Working Method[P]. Guangdong Province: CN109585021A, 2019-04-05;

[0012] [7] Zhang Tong, Jia Xue, Hu Bin, Xu Xiangmin, Chen Junlong. Audio Intelligent Intervention System and Its Working Method for Mental State Monitoring and Intervention [P]. Guangdong Province: CN109620257B, December 22, 2020;

[0013] [8] Zhang Tong, Jia Xue. Personalized Mental State Regulation System and Regulation Method Based on Brain Wave Music [P]. Guangdong Province: CN110947075A, April 3, 2020;

[0014] [9] Hu Bin, Zhu Lixian, Dong Qunxi. A System and Method for Mental State Monitoring and Treatment Based on Closed-Loop Feedback [P]. Beijing City: CN114081490A, February 25, 2022. Summary of the Invention

[0015] To solve the above problems existing in the prior art, the present invention provides an audio intelligent intervention system and its method for mental state monitoring and intervention.

[0016] To achieve the above object, the present invention provides the following solutions:

[0017] An audio intelligent intervention system for mental state monitoring and intervention, comprising:

[0018] An information collection module for collecting subject information;

[0019] An electroencephalogram (EEG) signal collection and processing module for collecting and processing EEG signals;

[0020] A mental state evaluation module for objectively quantifying and evaluating the mental state of the subject;

[0021] An audio playback module for playing audio;

[0022] An audio element adjustment module for adjusting audio elements;

[0023] A storage and output module for storing and outputting data.

[0024] An audio intelligent intervention method for the above-mentioned mental state monitoring and intervention, characterized by comprising the following steps:

[0025] Step S1, the subject inputs personal name or alias, gender, age, and audio type preference information into the system through the information collection module;

[0026] Step S2, the EEG signal collection and processing module collects the EEG signals of the subject and preprocesses the EEG signals;

[0027] Among them, the EEG acquisition and processing module includes an acquisition unit, an analog-to-digital conversion unit, a signal processing unit, and a communication unit;

[0028] Preferably, in step S2, the EEG signal of the subject is acquired by the acquisition unit to acquire the three-channel EEG signal of the prefrontal lobe of the subject; an impedance matching circuit is used to adapt to the impedance of different subjects, and acquisition is performed when the impedance value is less than 5 KΩ; the acquisition unit uses an EEG electrode sensor, and the three-channel EEG positions are Fp1, FpZ, and Fp2;

[0029] Preferably, the preprocessing step in step S2 includes filtering and artifact removal; the filtering is implemented by a zero-phase filter to reduce the phase delay time of the EEG signal; the artifact removal includes removing the 50 Hz power frequency interference;

[0030] In step S3, the mental state evaluation module extracts features from the preprocessed EEG signal in step S2, and uses transfer learning and establishes a regression model between the EEG signal features and the negative emotion probability to objectively and quantitatively evaluate the mental state, and obtains an evaluation result;

[0031] Among them, the mental state evaluation module includes a feature extraction unit, a prediction unit, and a communication unit; preferably, in step S3 of the mental state evaluation module, the feature extraction unit extracts features from the preprocessed EEG signal to obtain a feature set; uses a transfer learning algorithm and a regression model to predict the mental state of the feature set, and realizes the quantitative evaluation of the mental state;

[0032] Preferably, the transfer learning is first trained on a 128-channel EEG dataset containing negative emotions, with the EEG signal map as the input and whether it is negative emotion as the output, to establish a pre-trained model; secondly, by applying the pre-trained model to new data, the parameters of the transfer learning are fine-tuned: adding L2 regularization to the pre-trained model, and the specific formula is:

[0033]

[0034] Among them, is the new loss function, is the original loss function, the sample size in the training set, is the regularization term coefficient, is the neural network weight.

[0035] Establish a regression model to monitor the mental state, including: first, in the 128-channel EEG dataset, select the EEG signals of Fp1, FpZ, and Fp2, extract features, and establish a regression model between the EEG signal features and the negative emotion probability; the specific formula is as follows:

[0036]

[0037] wherein represents the number of features, represents the coefficient, represents the average value of the features, represents the negative emotion probability;

[0038] Preferably, the extracted features are power spectral density and sample entropy, and the specific formulas are as follows:

[0039] For the power spectral density, using the Welch algorithm, according to the formula to determine the power spectral density values of the electroencephalogram (EEG) signal data of the three leads Fp1, FpZ, and Fp2 in the 128 leads ; wherein, j is the imaginary unit, W represents the frequency, n represents the number of sampling points of the EEG signal data in one channel, m represents the serial number of the EEG signal sampling data of the three leads taken, ε* is the EEG signal data in one channel, e is the natural logarithm; since the alpha frequency band (8 - 13 Hz) of the EEG signal is related to emotions, the average value of the power spectral density of the alpha frequency band is calculated, that is where represents the average value, represents the number of frequencies in the frequency band.

[0040] For the sample entropy, according to the formula to calculate the sample entropy of the EEG signal data of the three leads Fp1, FpZ, and Fp2 in the 128 leads, where is the EEG signal data with a length of , is the EEG signal data with a length of ; then use the formula to calculate the average value of the sample entropy of one channel, where represents the number of sample entropies extracted in one channel; finally use the formula to calculate the average value of the sample entropies of the three channels, where is the number of channels.

[0041] Use to fit the regression equation of the EEG signal feature - negative emotion probability, where after obtaining the coefficient , apply it to the three - lead EEG to realize the prediction of the negative emotion probability;

[0042] The assessment of mental state is to predict the probability of negative emotions. First, the mental state is assessed based on the transfer learning result (whether it is negative emotion or not). If the result is yes, then directly use this result as the assessment result, and the probability of negative emotion is 100%; if the result is no, then the power spectral density and sample entropy are automatically extracted to predict the probability of negative emotion, and the probability is displayed as the result.

[0043] Step S4, according to the audio type preference information in Step S1 and the assessment result in Step S3, play the audio belonging to the audio type;

[0044] The audio playback module includes an audio storage unit, an audio type selection unit, and a communication unit; the audio types include light music, natural sounds, and human voices;

[0045] Preferably, in Step S4, the audio playback module plays the corresponding audio according to the mental state assessment result and the audio type preference of the subject in Step S1;

[0046] Step S5, the electroencephalogram signal acquisition and processing module and the mental state assessment module continue to collect and process the electroencephalogram signals of the subject in real time and monitor the mental state;

[0047] Step S6, the audio element adjustment module adjusts the audio elements using the reinforcement learning algorithm according to the mental state assessment result in Step S5, so as to achieve the purpose of restoring the mental state of the subject to a stable level;

[0048] The audio element adjustment module includes a communication unit and a music element unit; the music element unit includes pitch, rhythm, and melody; in order to be able to play the audio with modified audio elements, the communication unit of the audio playback module is connected to the communication unit of the audio element adjustment module;

[0049] Preferably, in Step S6, the audio elements in the audio element unit are adjusted using the audio element adjustment module, including pitch adjustment, rhythm adjustment, and melody adjustment;

[0050] Step S7, the storage and output module outputs the subject information, the mental state assessment result before audio playback, and the mental state assessment result after audio element adjustment for the subject to refer to; save the subject information, all assessment results, and the corresponding electroencephalogram signals.

[0051] The storage and output module includes a communication unit, a storage unit, and an output unit; among them, the communication unit of the storage and output module is respectively connected to the communication unit of the information acquisition module, the communication unit of the electroencephalogram acquisition and processing module, the communication unit of the mental state assessment module, and the communication unit of the audio playback module;

[0052] Preferably, in step S7, the communication unit in the storage and output module is connected to the communication units of each module. The storage unit stores information such as the personal name or alias, gender, and age in the information collection module, the preprocessed EEG signals in the EEG collection and processing module, the evaluation results in the mental state evaluation module, and the audio types in the audio playback module. The output unit outputs the personal name or code number, gender, and age in the information collection module, the mental state evaluation results before audio playback, and the mental state evaluation results after audio element adjustment.

[0053] Compared with the prior art, the present invention application has the following advantages:

[0054] An audio intelligent intervention system for mental state monitoring and intervention involved in the present invention application can objectively quantify the mental state of the subject by using computer technology and artificial intelligence technology;

[0055] Compared with the traditional method, an audio intelligent intervention system for mental state monitoring and intervention involved in the present invention integrates mental state monitoring and intervention, achieving the purpose of closed-loop negative feedback of monitoring - evaluation - intervention;

[0056] The audio intelligent intervention system for mental state monitoring and intervention involved in the present invention application uses a rich variety of audio types, which can meet the different audio type preferences of the subject. Secondly, by adjusting the audio elements through reinforcement learning, new audio is produced, and then the subject is intervened. Through the evaluation of the mental state before and after audio intervention, personalized and precise intervention of the negative emotions of the subject can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 It is a block diagram of the audio intelligent intervention system for mental state monitoring and intervention provided by the present invention;

[0059] Figure 2 It is a block diagram of the information collection module in the system of the present invention;

[0060] Figure 3 It is a block diagram of the EEG signal collection and processing module in the system of the present invention;

[0061] Figure 4 It is a block diagram of the mental state evaluation module in the system of the present invention;

[0062] Figure 5 This is the block diagram of the audio playback module in the system of the present invention;

[0063] Figure 6 This is the block diagram of the audio element adjustment module in the system of the present invention;

[0064] Figure 7 This is the block diagram of the storage and output module in the system of the present invention. Specific embodiments

[0065] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0066] The purpose of the present invention is to provide an audio intelligent intervention system for mental state monitoring and intervention, so as to provide a mental state intervention method based on artificial intelligence technology and realize a closed-loop negative feedback process from mental state monitoring to mental state intervention.

[0067] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0068] The present invention provides a mental state intervention method, that is, an audio intelligent intervention system for mental state monitoring and intervention.

[0069] As Figure 1 shown, an audio intelligent intervention system for mental state monitoring and intervention includes:

[0070] An information collection module for collecting information of the subject;

[0071] An electroencephalogram signal collection and processing module for collecting and processing electroencephalogram signals;

[0072] A mental state evaluation module for objectively quantifying and evaluating the mental state of the subject;

[0073] An audio playback module for playing audio;

[0074] An audio element adjustment module for adjusting audio elements;

[0075] A storage and output module for storing and outputting data.

[0076] An above-mentioned audio intelligent intervention method for mental state monitoring and intervention is characterized in that it includes the following steps:

[0077] Step S1: The subject inputs personal name or alias, gender, age, and audio type preference information into the system through the information collection module;

[0078] Step S2: The electroencephalogram (EEG) signal acquisition and processing module acquires the EEG signal of the subject and preprocesses the EEG signal;

[0079] Among them, the EEG acquisition and processing module includes an acquisition unit, an analog-to-digital conversion unit, a signal processing unit, and a communication unit;

[0080] Preferably, in Step S2, the EEG signal of the subject is acquired by the acquisition unit to acquire the three-channel EEG signal of the prefrontal lobe of the subject. Due to individual differences among subjects, an impedance matching circuit is used to adapt to the impedance of different subjects; to ensure the quality of the EEG signal, the EEG signal is only acquired when all impedances are less than 5 kΩ; the analog-to-digital conversion unit records the EEG signal of the subject; the acquisition unit uses an EEG electrode sensor. Since the positions of Fp1, FpZ, and Fp2 in the prefrontal lobe have been proven to be related to emotions, the three-channel EEG positions are Fp1, FpZ, and Fp2;

[0081] Preferably, the preprocessing steps in Step S2 include filtering and artifact removal; the filtering is implemented using a zero-phase filter to reduce the phase delay time of the EEG signal; the artifact removal includes removing the 50 Hz power frequency interference;

[0082] Step S3: The mental state assessment module extracts features from the preprocessed EEG signal in Step S2, and objectively quantifies and assesses the mental state by using transfer learning and establishing a regression model between the EEG signal features and the probability of negative emotions to obtain an assessment result;

[0083] Among them, the mental state assessment module includes a feature extraction unit, a prediction unit, and a communication unit; preferably, in Step S3 of the mental state assessment module, the feature extraction unit extracts features from the preprocessed EEG signal to obtain a feature set; the transfer learning algorithm and the regression model are used to predict the mental state of the feature set to achieve the quantitative assessment of the mental state;

[0084] Preferably, the transfer learning is first trained on a 128-channel EEG data set containing negative emotions, with the EEG signal map as the input and whether it is negative emotion as the output to establish a pre-trained model; secondly, the pre-trained model is used for new data to fine-tune the parameters of the transfer learning: the L2 regularization is added to the pre-trained model, and the specific formula is:

[0085]

[0086] Among them, is the new loss function, is the original loss function, the sample size in the training set, is the regularization term coefficient, are the neural network weights.

[0087] A regression model is established to monitor the mental state, including: first, in the 128-channel EEG dataset, select the EEG signals of Fp1, FpZ, and Fp2, extract features, and establish a regression model between the EEG signal features and the negative emotion probability; the specific formula is as follows:

[0088]

[0089] where represents the number of features, represents the coefficient, represents the average value of the features, represents the negative emotion probability;

[0090] Preferably, the extracted features are power spectral density and sample entropy, and the specific formula is as follows:

[0091] For the power spectral density, using the Welch algorithm, according to the formula to determine the power spectral density values of the EEG signal data of the three leads of Fp1, FpZ, and Fp2 in the 128 leads ; where j is the imaginary unit, W represents the frequency, n represents the number of sampling points of the EEG signal data in one channel, m represents the serial number of the sampled data of the EEG signals of the three leads taken, ε* is the EEG signal data in one channel, e is the natural logarithm; since the alpha band (8 - 13 Hz) of the EEG signal is related to emotions, so calculate the average value of the power spectral density in the alpha band, that is where represents the average value, represents the number of frequencies in the band.

[0092] For the sample entropy, according to the formula calculate the sample entropy of the EEG signal data of the three leads of Fp1, FpZ, and Fp2 in the 128 leads, where is the EEG signal data with a length of is the EEG signal data with a length of ; then use the formula to calculate the average value of the sample entropy of one channel, where represents the number of sample entropies extracted in one channel; finally use the formula ​Calculate the average sample entropy of three channels, where is the number of channels.

[0093] Use to fit the regression equation of EEG signal features - negative emotion probability, where after obtaining the coefficient apply it to the three-channel EEG to realize the prediction of the negative emotion probability;

[0094] The assessment of mental state is to assess the negative emotion probability. First, conduct a mental state assessment based on the transfer learning result (whether it is a negative emotion). If the result is yes, directly use this result as the assessment result, and the negative emotion probability is 100%. If the result is no, automatically extract the power spectral density and sample entropy to predict the negative emotion probability, and display the probability as the result.

[0095] Step S4, according to the audio type preference information in Step S1 and the assessment result in Step S3, play the audio belonging to the audio type;

[0096] The audio playback module includes an audio storage unit, an audio type selection unit, and a communication unit; the audio types include light music, natural sounds, and human voices;

[0097] Preferably, in Step S4, the audio playback module plays the corresponding audio according to the mental state assessment result and the audio type preference of the subject in Step S1;

[0098] Furthermore, the audio types include light music, natural sounds, and human voices; among them, the music playback module uses a speaker or in-ear headphones to achieve audio playback; the audio volume is the most comfortable volume that the subject can accept;

[0099] Step S5, the EEG signal acquisition and processing module and the mental state assessment module continue to collect and process the EEG signals of the subject in real time and monitor the mental state;

[0100] Step S6, the audio element adjustment module adjusts the audio elements using a reinforcement learning algorithm according to the mental state assessment result in Step S5 to eliminate the negative emotions of the subject;

[0101] The audio element adjustment module includes a communication unit and a music element unit; the music element unit includes pitch, rhythm, and melody; in order to be able to play the audio with modified audio elements, the communication unit of the audio playback module is connected to the communication unit of the audio element adjustment module;

[0102] Preferably, in Step S6, the audio element adjustment module adjusts the audio elements in the audio element unit, including pitch adjustment, rhythm adjustment, and melody adjustment;

[0103] Furthermore, reinforcement learning is used to adjust the pitch, rhythm, and melody of the audio. First, the pitch is adjusted. The reinforcement learning algorithm evaluates the pitch adjustment that has been made based on the mental state assessment result. If the negative emotion is improved, the pitch adjustment is retained; otherwise, the adjustment continues. Using similar steps, the melody and rhythm are adjusted respectively to obtain the audio with the best intervention effect on the current subject.

[0104] Furthermore, in reinforcement learning, the mental state assessment result is used as the score in reinforcement learning, and the pitch adjustment is used as the action, enabling the computer to continuously learn. When the mental state assessment result (negative emotion probability) decreases and no longer changes, the pitch corresponding to the current score is saved. Similarly, the melody and rhythm with the lowest mental state assessment result are saved.

[0105] Step S7: The saving and output module outputs the subject information, the mental state assessment result before audio playback, and the mental state assessment result after audio element adjustment for the subject to refer to; the subject information, all assessment results, and the corresponding EEG signals are saved.

[0106] The saving and output module includes a communication unit, a storage unit, and an output unit. Among them, the communication unit of the saving and output module is respectively connected to the communication units of the information acquisition module, the EEG acquisition and processing module, the mental state assessment module, and the audio playback module.

[0107] Preferably, in step S7, the communication unit in the saving and output module is connected to the communication units of each module. The storage unit saves the personal name or alias, gender, and age in the information acquisition module, the preprocessed EEG signals in the EEG acquisition and processing module, the assessment results in the mental assessment module, the audio type in the audio playback module, etc. The output unit outputs the personal name or code number, gender, and age in the information acquisition module, the mental state assessment result before audio playback, and the mental state assessment result after audio element adjustment.

[0108] The audio element adjustment module includes a communication unit and a music element unit. The music element unit includes pitch, rhythm, and melody.

[0109] The communication unit of the saving and output module is respectively connected to the communication units of the information acquisition module, the EEG acquisition and processing module, the mental state assessment module, and the audio playback module. The communication unit connection is implemented using the serial port module RS232 to enable communication between different units.

[0110] The communication unit of the audio playback module is connected to the communication unit of the audio element adjustment module. The communication unit connection is implemented using the serial port module RS232 to enable communication between different units.

[0111] In this specific embodiment, one subject is selected. First, the subject inputs personal information and selects music type preferences. Then, in a natural scenario, an electroencephalogram (EEG) acquisition device is used to collect the subject's EEG signals. After passing through a preprocessing unit to remove artifacts such as electrooculogram (EOG), electrocardiogram (ECG), electromyogram (EMG), and power frequency from the EEG signals, the preprocessed EEG signals are obtained. The preprocessed EEG signals pass through the transfer learning and regression model of the mental state evaluation unit to obtain evaluation result 1. Based on the music type preferences selected by the subject, corresponding audio is played to obtain corresponding mental state evaluation result 2. According to mental state evaluation result 2 and audio type preferences, the system uses reinforcement learning to automatically adjust audio elements, including pitch, melody, rhythm, etc., to obtain adjusted audio, which is played to obtain corresponding mental state evaluation result 3 for the subject. At the end of the experiment, the system automatically saves all mental state evaluation results, personal information, music type preferences, and the original EEG signals used for mental state evaluation, and displays the subject's personal information and mental state evaluation results before and after audio intervention, including mental state evaluation result 1 and mental state evaluation result 3, through a display screen.

[0112] The present invention has the following advantages:

[0113] In the process of mental state monitoring, compared with the traditional method of using a scale to judge the negative emotions of a subject, since the audio intelligent intervention system of the present invention uses computer methods such as transfer learning and regression models to judge the negative emotions of the subject, it can objectively monitor the mental state of the subject with high efficiency.

[0114] The audio intelligent intervention system for mental state monitoring and intervention involved in the present invention integrates mental state monitoring and intervention, achieving the purpose of a closed-loop negative feedback of monitoring - evaluation - intervention;

[0115] The audio intelligent intervention system for mental state monitoring and intervention involved in the present invention uses a rich variety of audio types, which can meet the different audio type preferences of the subject. Secondly, through the adjustment of audio elements by reinforcement learning, new audio can be generated, and then the subject can be intervened. By evaluating the mental state before and after audio intervention, personalized and precise intervention of the subject's negative emotions can be achieved;

[0116] In this article, specific examples are used to elaborate on the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be construed as a limitation on the present invention.

[0117] An audio intelligent intervention system and method for mental state monitoring and intervention. The information acquisition module is used to collect the information of the subject and the audio type preference. The electroencephalogram (EEG) signal acquisition and processing module collects the EEG signals at three positions of Fp1, FpZ, and Fp2 of the subject and performs preprocessing operations. The mental state evaluation module using the transfer learning algorithm and the regression model extracts features from the preprocessed EEG signals and predicts the negative emotion probability to achieve the evaluation of the mental state. The audio playback module plays the corresponding audio according to the evaluation result and the selected audio type preference, continues to monitor the mental state, and uses the audio element adjustment module containing the reinforcement learning algorithm to adjust the music elements to achieve the playback of new audio and then eliminate the negative emotions of the subject. The storage and output module stores the evaluation result information, outputs the subject information, the mental state evaluation result before audio playback, and the mental state evaluation result after audio element adjustment.

Claims

1. An audio intelligent intervention system for mental state monitoring and intervention, characterized in that: An information collection module for collecting subject information; an electroencephalogram (EEG) signal collection and processing module for collecting and processing EEG signals; a mental state evaluation module for objectively quantifying and evaluating the mental state of the subject: The mental state evaluation module includes a feature extraction unit, a prediction unit, and a communication unit; the feature extraction unit extracts features from the preprocessed EEG signals to obtain a feature set; Using a transfer learning algorithm and a regression model to predict the mental state of the feature set, realizing the quantitative evaluation of the mental state; Transfer learning is trained on a 128-channel EEG dataset containing negative emotions, with EEG signal maps as input and whether it is negative emotion as output to establish a pre-trained model; the pre-trained model is used for new data, and the transfer learning parameters are fine-tuned: L2 regularization is added to the pre-trained model; a regression model is established to monitor the mental state: in the 128-channel EEG dataset, select the EEG signals of Fp1, FpZ, and Fp2, extract the power spectral density and sample entropy features, and establish a regression model between the EEG signal features and the negative emotion probability; for the power spectral density, use the Welch algorithm, according to the formula to determine the power spectral density values of the EEG signal data of the three leads of Fp1, FpZ, and Fp2 in the 128 leads , j is the imaginary unit, W represents the frequency, n represents the number of sampling points of the EEG signal data in one channel, m represents the serial number of the sampled data of the EEG signals of the three leads taken, ε* is the EEG signal data in one channel, and e is the natural logarithm; calculate the average value of the power spectral density in the alpha frequency band of the EEG signal; for the sample entropy, according to the formula calculate the sample entropy of the EEG signal data of the three leads of Fp1, FpZ, and Fp2 in the 128 leads, is the length of EEG signal data, is the length of EEG signal data; Use the formula to calculate the average sample entropy of one channel, where is the number of sample entropies extracted within one channel; Using the formula calculate the average sample entropy of the three channels, where is the number of channels; Use to fit the regression equation of the EEG signal feature - negative emotion probability, and obtain the coefficient where Xi is the average value of the i-th feature; is the negative emotion probability; an audio playback module for playing audio and adjusting audio elements, an audio element adjustment module; a saving and output module for saving and outputting data.

2. An audio intelligent intervention system for mental state monitoring and intervention according to claim 1, characterized in that: The mental state evaluation module includes a feature extraction unit, a prediction unit, and a communication unit; the storage and output module includes a communication unit, a storage unit, and an output unit; the audio playback module includes an audio storage unit, an audio type selection unit, and a communication unit; the audio types include light music, natural sounds, and human voices; the audio element adjustment module includes a communication unit and a music element unit; the music element unit includes pitch, rhythm, and melody; the communication unit of the storage and output module is respectively connected to the communication unit of the information collection module, the communication unit of the EEG collection and processing module, the communication unit of the mental state evaluation module, and the communication unit of the audio playback module; the communication unit of the audio playback module is connected to the communication unit of the audio element adjustment module.

3. A method using the audio intelligent intervention system for mental state monitoring and intervention according to claim 2, characterized in that: It includes the following steps: Step S1, the subject inputs personal name or alias, gender, age, and audio type preference information into the system through the information collection module; Step S2, the EEG signal collection and processing module collects the EEG signals of the subject and preprocesses the EEG signals; Among them, the EEG collection and processing module includes a collection unit, an analog-to-digital conversion unit, a signal processing unit, and a communication unit; In step S2, the EEG signals of the subject are collected by the collection unit by collecting the three-channel EEG signals of the prefrontal lobe of the subject; an impedance matching circuit is used to adapt to the impedance of different subjects, and collection is performed when the impedance value is less than 5 K ohms; the collection unit uses an EEG electrode sensor, and the three-channel EEG positions are Fp1, FpZ, and Fp2; The preprocessing steps in step S2 include filtering and artifact removal; the filtering is implemented by a zero-phase filter to reduce the phase delay time of the EEG signals; the artifact removal includes removing 50 Hz power frequency interference; Step S3, the mental state evaluation module extracts features from the preprocessed EEG signals in step S2, and uses transfer learning and establishes a regression model between EEG signal features and negative emotion probability to objectively and quantitatively evaluate the mental state, obtaining an evaluation result; In step S3, the feature extraction unit of the mental state evaluation module extracts features from the preprocessed EEG signals to obtain a feature set; uses a transfer learning algorithm and a regression model to predict the negative emotion probability of the feature set, realizing the objective quantitative evaluation of the mental state; The transfer learning is first trained on a 128-channel EEG dataset containing negative emotions, with the EEG signal map as the input and whether it is negative emotion as the output, to establish a pre-trained model; secondly, by applying the pre-trained model to new data, the parameters of the transfer learning are fine-tuned: the pre-trained model is added with L2 regularization, and the specific formula is: ； Among them, is the new loss function, is the original loss function, is the sample size in the training set, is the regularization term coefficient, is the neural network weight; A regression model is established to monitor the mental state, including: first, in the 128-channel EEG dataset, the EEG signals of Fp1, FpZ, and Fp2 are selected, features are extracted, and a regression model of the EEG signal features and the negative emotion probability is established; the specific formula is as follows: ； The coefficient representing the i-th feature; Xi represents the average value of the i-th feature; Represents the probability of negative emotion; The extracted features are power spectral density and sample entropy, and the specific formula is as follows: For the power spectral density, using the Welch algorithm, according to the formula Determine the power spectral density values of the EEG signal data of the three leads Fp1, FpZ, and Fp2 in the 128 leads ; where j is the imaginary unit, W represents the frequency, n represents the number of sampling points of the EEG signal data in one channel, m represents the serial number of the sampled EEG signal data of the three leads taken, ε* is the EEG signal data in one channel, and e is the natural logarithm; since the alpha band of the EEG signal, 8 - 13 Hz, is related to emotions, the average value of the power spectral density in the alpha band is calculated; For sample entropy, according to the formula calculate the sample entropy of the EEG signal data of the three leads Fp1, FpZ, and Fp2 in the 128 leads, where is the EEG signal data with a length of ; is the EEG signal data with a length of ; then use the formula to calculate the average value of the sample entropy of one channel, where represents the number of sample entropies extracted within one channel; finally use the formula to calculate the average value of the sample entropy of the three channels, where is the number of channels; Utilize to fit the regression equation of EEG signal features - negative emotion probability, and after obtaining the coefficient apply it to three-channel EEG to achieve the prediction of negative emotion probability; The evaluation of the mental state is to evaluate the negative emotion probability: first, the mental state is evaluated according to the transfer learning result. If the result is yes, then directly use this result as the evaluation result, and the negative emotion probability is 100%; if the result is no, then the power spectral density and sample entropy are automatically extracted to predict the negative emotion probability, and the probability is used as the result for display; In step S4, according to the audio type preference information in step S1 and the evaluation result in step S3, play the audio belonging to the audio type; The audio playback module includes an audio storage unit, an audio type selection unit, and a communication unit; the audio types include light music, natural sounds, and human voices; In step S4, the audio playback module in step S4 plays the corresponding audio according to the mental state evaluation result and the audio type preference of the subject in step S1; In step S5, the EEG signal acquisition and processing module and the mental state evaluation module continue to collect and process the EEG signals of the subject in real time and monitor the mental state; In step S6, the audio element adjustment module adjusts the audio elements by using the reinforcement learning algorithm according to the mental state evaluation result in step S5, so as to achieve the purpose of restoring the mental state of the subject to a stable level; The audio element adjustment module includes a communication unit and a music element unit; the music element unit includes pitch, rhythm, and melody; in order to be able to play the audio with the modified audio elements, the communication unit of the audio playback module is connected to the communication unit of the audio element adjustment module; In step S6, the audio elements in the audio element unit are adjusted by using the audio element adjustment module, including pitch adjustment, rhythm adjustment, and melody adjustment; In step S7, the storage and output module outputs the subject information, the mental state evaluation result before audio playback, and the mental state evaluation result after audio element adjustment for the subject to refer to; save the subject information, all evaluation results and the corresponding EEG signals; The storage and output module includes a communication unit, a storage unit, and an output unit; among them, the communication unit of the storage and output module is respectively connected to the communication unit of the information acquisition module, the communication unit of the EEG acquisition and processing module, the communication unit of the mental state evaluation module, and the communication unit of the audio playback module; In step S7, the communication unit in the storage and output module is connected to the communication units of each module. The storage unit stores the personal name or alias, gender, age, preprocessed EEG signals in the EEG acquisition and processing module, evaluation results in the mental state evaluation module, and audio type information in the audio playback module. The output unit outputs the personal name or code number, gender, age, mental state evaluation result before audio playback, and mental state evaluation result after audio element adjustment in the information acquisition module.

4. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: In step S2, the EEG signals of the subject are collected by the acquisition unit by collecting the three-channel EEG signals of the prefrontal lobe of the subject; an impedance matching circuit is used to adapt to the impedance of different subjects, and the acquisition is performed when the impedance value is less than 5 KΩ; the EEG signals of the subject are recorded by the analog-to-digital conversion unit; the acquisition unit uses an EEG electrode sensor, and the three-channel EEG positions are Fp1, FpZ, and Fp2.

5. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: The preprocessing steps in step S2 include filtering and artifact removal; the filtering is implemented by a zero-phase filter to reduce the phase delay time of the EEG signals; the artifact removal includes removing 50 Hz power frequency interference.

6. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: In step S3, the feature extraction unit in the mental state evaluation module extracts features from the preprocessed EEG signals to obtain a feature set; The negative emotion probability is predicted for the feature set by using a transfer learning algorithm and a regression model to realize the evaluation of the mental state.

7. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: In step S4, the audio playback module plays the corresponding audio according to the mental state evaluation result and the audio type preference of the subject in step S1.

8. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: In step S6, according to the mental state evaluation result, the audio elements are continuously adjusted by using reinforcement learning to generate a new audio, which can realize personalized and precise intervention.

9. The method of an audio intelligent intervention system for mental state monitoring and intervention according to claim 3, wherein: The system involved in the method is a closed-loop system that can realize mental state monitoring-evaluation-intervention, and can monitor-evaluate and intervene in the mental state of the subject under natural conditions without giving stimuli in advance.

Images