Sleep cabin for emotion memory regulation and control

Abstract

The invention belongs to the technical field of medical equipment, and particularly relates to a sleep cabin for emotion memory regulation and control. The sleep cabin of the present invention comprises: a sleep bed for a subject to sleep; the sleep emotion multi-module monitoring equipment is used for monitoring sleep sign data of a subject in real time; the data judgment and decision judgment equipment is integrated with a decision operation unit and is used for obtaining a stimulation mode and parameters for regulating and controlling the sleep of the subject according to the sleep sign data; and the multi-sensory stimulation intervention strengthening equipment is used for stimulating the subject according to the stimulation mode and the parameters so as to regulate and control sleep. According to the invention, emotion memory regulation and control in sleep can be carried out on depression patients (common people and people with sleep disorders), negative memory can be eliminated, and positive memory can be enhanced; the sleep is improved; adjusting the emotion; the depressive symptom is relieved, and the life quality is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical equipment, and in particular relates to a sleeping cabin for regulating emotion and memory. Background Art

[0002] Depression is a common mental illness and the most common mental health problem worldwide. Patients usually experience low mood, decreased interest and lack of energy.

[0003] The severity of depression has prompted many researchers to conduct in-depth theoretical and practical research on its pathogenesis, revealing the pathological characteristics of patients with depression, including: ① persistent and long-term low mood or imbalance in emotional memory regulation; ② enhancement of negative memories and weakening of positive memories; ③ sleep disorders.

[0004] Treatments for depression include medication, psychotherapy, and physical therapy, but the efficacy of these methods varies greatly among individuals. This difference has led to an increasingly prominent contradiction between the increasing number of patients and the improvement of treatment plans. There is an urgent need to develop specific, targeted, widely applicable, and highly operable equipment and methods to achieve individualized and precise treatment of patients with depression.

[0005] Clinically, about 70% of patients with depression have sleep problems, and people with sleep problems are at a significantly higher risk of developing depression or anxiety than those with normal sleep. Therefore, treating depression by intervening in the sleep of patients with depression is a new treatment method (in addition, intervening in the sleep of normal people may also have a preventive effect on depression). In the prior art, although studies have explored the relationship between depression and sleep, how to determine the specific means and parameters for sleep intervention in patients with depression is still a problem that needs to be solved. Summary of the invention

[0006] In view of the problems of the prior art, the present invention provides a sleeping cabin for regulating emotion and memory.

[0007] A sleeping cabin for emotion memory regulation, characterized by comprising:

[0008] A sleeping bed, for the subjects to sleep;

[0009] Sleep and mood multi-module monitoring equipment, used to monitor the subjects' sleep sign data in real time;

[0010] A data discrimination and decision-making device, integrated with a decision-making operation unit, is used to obtain a stimulation mode and parameters for regulating the sleep of a subject according to the sleep sign data;

[0011] Multi-sensory stimulation intervention and enhancement equipment, used to stimulate the subject according to the stimulation mode and parameters to achieve sleep regulation;

[0012] Wherein, the decision operation unit includes:

[0013] The information receiving port is configured to: receive the subject's sleep sign data in real time;

[0014] The central computer processing system is configured to: input the sleep sign data into a machine learning model, and obtain the judgment result of the subject's sleep mood and the stimulation mode and parameters for regulating the subject's sleep through the machine learning model; the machine learning model is a hybrid deep neural network model;

[0015] The information sending port is configured to output the judgment result of the subject's sleep mood, the stimulation mode and parameters.

[0016] Preferably, the sleep emotion multi-module monitoring device includes at least one of the following devices: a contact polysomnogram, a fiber optic sensing sleep monitoring device, and a smart glove integrated with a sensor.

[0017] Preferably, the multi-sensory stimulation intervention enhancement equipment includes at least one of the following devices: VR visual stimulation equipment, auditory stimulation equipment, tactile stimulation equipment, olfactory stimulation equipment, temperature control system, air humidity control equipment.

[0018] Preferably, the multi-sensory stimulation intervention enhancement device is configured to stimulate the subject when the subject enters the rapid eye movement sleep stage.

[0019] Preferably, the sleep sign data include at least one of the following indicators: electroencephalogram, electrocardiogram, heart rate, blood pressure, FVC, FEV1, FEV1 / FVC, electromyogram, and body temperature; wherein the meanings of each FVC, FEV1, and FEV1 / FVC are: forced vital capacity (FVC), forced respiratory volume in the first second (FEV1), and one-second rate (FEV1 / FVC).

[0020] And / or, in the central computer processing system, basic information of the patient is also input into the machine learning model, wherein the basic information includes at least one of the following indicators: age, gender, height, weight, waist circumference, smoking history, drinking history, medical history / hospitalization history / accident history, surgical history, blood transfusion history, allergy history, long-term medication, and family genetic diseases;

[0021] And / or, the stimulation mode includes at least one of the following stimulation modes: tactile stimulation, temperature stimulation, auditory stimulation, olfactory stimulation, air humidity, and transcranial direct current stimulation.

[0022] Preferably, the hybrid deep neural network model includes:

[0023] The raw data extraction layer is used to decode, decompile and disassemble the input raw data into a more readable and easier to understand data type;

[0024] The feature extraction layer is used to extract features from the processing results of the original data extraction layer;

[0025] The feature vector generation layer is used to generate several feature vectors using different algorithms for existence features and discrete features, among which some features are spliced ​​by fuzzy algorithm to generate feature vectors together;

[0026] A multimodal sleep emotion detection layer, used to obtain the judgment result of the subject's sleep emotion according to the feature vector, as well as the stimulation mode and parameters for regulating the subject's sleep;

[0027] The multimodal sleep emotion detection layer includes an initial network layer and a final network layer. The initial network layer includes a number of unconnected DNN networks, and the feature vectors are independently sent to the initial network layer. The final network layer includes a DNN network structure. The multimodal sleep emotion detection layer uses a pre-fusion method to fuse deep features. The DNN network structure of the final network layer outputs the judgment result of the subject's sleep emotion, as well as the stimulation mode and parameters for regulating the subject's sleep.

[0028] Preferably, in the initial network layer, each independent DNN network consists of an input layer and several hidden layers. In the DNN network, neurons in each layer are not connected to each other, and neurons in each layer are only connected to adjacent previous and next layers, and the layers are fully connected.

[0029] In the final network layer, the DNN network structure consists of a fusion layer, several hidden layers and an output layer. The output layer has only one neuron, which is responsible for outputting the analysis results.

[0030] Preferably, in the feature vector generation layer, the method for generating the existential feature vector is:

[0031] All the existence features extracted by the feature extraction layer form an existence feature database; query each sample's corresponding feature in turn to see if it appears in the feature database. If it does, the feature is assigned a value of 1, otherwise it is 0;

[0032] The method for generating discrete feature vectors is:

[0033] Normalize the feature data; perform K-means clustering on the normalized data to obtain a set of centroids for each type of feature data; the feature data extracted from the sample will be calculated separately for their distances to different centroids of the corresponding type; select the centroid with the shortest distance, and the length of the projection of the vector pointing from the centroid to the current feature point in different dimensions represents the similarity of the features of the data and the centroid in different dimensions; use the projection length of 0.4 as the threshold, and set the elements in the feature vector corresponding to the features in the dimension where the projection length exceeds 0.4 to 0, otherwise set to 1.

[0034] Preferably, the feature vector generation layer extracts a feature vector for each discrete feature, and concatenates all the existence features to generate a feature vector.

[0035] Preferably, in the feature vector generation layer, 0 vectors are used to complete the extracted incomplete features.

[0036] The present invention provides a sleeping cabin and a control system thereof. The system utilizes deep learning to realize real-time sleep regulation of "monitoring-processing-feedback-re-monitoring-processing-better feedback", and can regulate the sleep of the subject in an individualized, adjustable, and precise stimulating effect.

[0037] According to this purpose, the present invention provides an optimized hybrid deep neural network model. Compared with existing similar models, the hybrid deep neural network model of the present invention is optimized for uncertainty, accuracy, and personalization. It is a multimodal fusion emotion recognition model based on Choquet fuzzy integral, which is the most robust, accurate, and complete data learning and prediction method reported to date.

[0038] In the prior art, a hybrid deep neural network model has been proposed for the prediction of drug-drug interactions (DDI). The present invention applies it to emotional memory regulation for the first time, and optimizes the model for this purpose, further improving its accuracy and personalization.

[0039] The above advantages are specifically reflected in:

[0040] 1. Handling uncertainty and enhancing the generalization ability of the model: The hybrid deep neural network model of the present invention can provide more comprehensive data analysis by integrating information from different data sources (such as respiration, ECG, body temperature, etc.). Fuzzy integrals can help handle uncertainty and ambiguity in clinical data, allowing the model to learn and make predictions from these data more robustly.

[0041] 2. Improve diagnostic accuracy: By combining fuzzy integrals, the hybrid deep neural network model of the present invention can more accurately diagnose diseases or predict treatment outcomes, especially when the data is incomplete or there is variability.

[0042] 3. Personalized treatment plan: Fuzzy integral can help the hybrid deep neural network model of the present invention better understand and predict the response of individual patients to treatment, thereby helping the sleeping cabin to develop a more personalized treatment plan.

[0043] 4. Optimize decision support: In clinical decision support systems, fuzzy integrals can help weigh the importance of different factors and develop more comprehensive and balanced treatment plans for sleeping cabins.

[0044] 5. Support Explainable Artificial Intelligence (XAI): Fuzzy integrals provide a natural way to explain how the model makes decisions by integrating different factors, which can enhance the interpretability of the model. This is particularly important for applications in the medical field because it can provide a more transparent decision-making process, which is crucial to improving doctors' trust and acceptance of AI systems. The pathological characteristics of patients with depression are continuous and long-term low mood, enhanced negative memory and weakened positive memory, and sleep disorders. Regulation of these is the core of curing depression. Through the sleeping cabin system of the present invention, emotional memory sleep regulation can be performed on patients with depression (it can also be used for the general population and people with sleep disorders), eliminating negative memories and enhancing positive memories; improving sleep; adjusting emotions; alleviating depressive symptoms and improving the quality of life. Therefore, the present invention has a good application prospect.

[0045] Obviously, according to the above contents of the present invention, in accordance with common technical knowledge and customary means in the art, without departing from the above basic technical ideas of the present invention, other various forms of modification, replacement or change may be made.

[0046] The above contents of the present invention are further described in detail below through specific implementation methods in the form of embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples. All technologies realized based on the above contents of the present invention belong to the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the composition of the sleeping cabin system of the present invention;

[0048] Figure 2 This is a schematic diagram of a preferred structure of a sleeping cabin of the present invention;

[0049] Figure 3 It is a schematic diagram of the process of the hybrid deep neural network model in the present invention;

[0050] Figure 4Schematic diagram of the network structure of the multimodal sleep emotion detection layer in the present invention;

[0051] Figure 5 Schematic diagram of fuzzy integral fusion of multiple classifiers. DETAILED DESCRIPTION

[0052] It should be noted that the algorithms of data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structure, circuit connection, etc. not specifically described can all be implemented through the contents disclosed in the prior art.

[0053] Example 1 Sleeping cabin for emotion memory regulation

[0054] The sleeping cabin provided in this embodiment can detect the sleep sign data of the subject in real time, judge the sleep mood of the subject based on the data, and provide appropriate stimulation in real time to regulate the sleep of the subject.

[0055] Specifically, Figure 1-Figure 2 As shown, the system of this embodiment includes:

[0056] A sleeping bed, for the subjects to sleep;

[0057] Sleep and mood multi-module monitoring equipment, used to monitor the subjects' sleep sign data in real time;

[0058] A data discrimination and decision-making device, integrated with a decision-making operation unit, is used to obtain a stimulation mode and parameters for regulating the sleep of a subject according to the sleep sign data;

[0059] The multi-sensory stimulation intervention and enhancement device is used to stimulate the subject according to the stimulation mode and parameters to achieve sleep regulation.

[0060] The specific structure of each part is as follows:

[0061] 1. Sleep and mood multi-module monitoring equipment

[0062] The sleep emotion multi-module monitoring device can be realized by existing technology. As a preferred method, the sleep emotion multi-module monitoring device of this embodiment includes the following components:

[0063] 1. Devices connected to the bedside

[0064] Specifically, a contact polysomnography is used to record EEG, ECG and EMG information of the patient during sleep;

[0065] The principle is: the electrodes attached to the skin transmit bioelectric information to the sensor, and different parts transmit information together. The sensor cable writes the signal to the head box device, and then the analog signal is converted into a digital signal after amplification, filtering and digitization through the head box and sent to the fixed station. The signal can also be sent to the fixed station through the auxiliary input device, and the signal can be directly amplified and adjusted in the auxiliary device.

[0066] 2. Connection on the right side of the sleeping bed: sleep monitoring equipment based on low-cost fiber optic sensing

[0067] A low-cost fiber optic sensing device embedded in a mattress can be used to measure respiratory rate and heart rate for a long time, non-invasively and with immunity to electromagnetic interference. It can distinguish four behavioral states (not in bed, lying, moving and leaving the bed), and measure respiration and heart rate in different positions and postures. It is suitable for long-term sleep monitoring, and the data obtained can be used for the analysis of diseases such as sleep disorders.

[0068] Principle: This is a temperature-pressure integrated fiber Bragg grating sensor and human sleep multi-sign monitoring system. The system includes a mattress, several temperature-pressure integrated fiber Bragg grating sensors, a demodulator, and a host computer. Several vital sign fiber Bragg grating sensors are fixed in the gap between the mattress and the bed net, and the depth is kept at the same level. Several vital sign fiber Bragg grating sensors are connected to the demodulator to transmit data to the host computer. The system identifies the pressure on the sensor by detecting the wavelength of the fiber Bragg grating spectrum signal peak splitting point, and identifies the ambient temperature of the sensor by detecting the center wavelength drift of the fiber Bragg grating spectrum signal. It monitors the multi-sign information of the user in bed, including body temperature, respiratory rate, heart rate, and state of falling out of bed. It has the advantages of simple structure, small size, anti-electromagnetic interference, and low cost.

[0069] 3. Connection on the left side of the sleeping bed: the front end of the Dormio system

[0070] Monitor the time and status of sleep onset. Use three physiological indicators of sleep onset.

[0071] ① Subjects lie down to sleep and gently close their hands, allowing a flex sensor to monitor the gradual loss of muscle tone by opening the hand. Sensors around the index finger can track muscle tone (Kelly JM, Strecker RE, Bianchi MT. Recent developments in home sleep-monitoring devices. ISRN Neurol. 2012; 2012: 768794. doi: 10.5402 / 2012 / 768794. Epub 2012 Oct 14. PMID: 23097718; PMCID: PMC3477711.) because the loss of muscle tone is related to the appearance of sleep imagery in time.

[0072] ② Decreased heart rate: consistent with the loss of muscle tone (Herlan A, Ottenbacher J, Schneider J, Riemann D, Feige B. Electrodermal activity patterns in sleep stages and their utility for sleep versus wake classification. J Sleep Res. 2019Apr;28(2):e12694.doi:10.1111 / jsr.12694.Epub 2018May 2.PMID:29722079.), able to monitor heart rate on the subject’s middle finger

[0073] ③ Electrodermal activity (EDA): consistent with the loss of muscle tone (Herlan A, Ottenbacher J, Schneider J, Riemann D, Feige B. Electrodermal activity patterns in sleep stages and their utility for sleep versus wake classification. J Sleep Res. 2019

[0074] Apr;28(2):e12694.doi:10.1111 / jsr.12694.Epub 2018May 2.PMID:29722079.), two electrodes at the base of the wrist can measure EDA.

[0075] 4. Integrated devices for sleep monitoring

[0076] The integrated device is a smart glove, including a blood oxygen detection module, a humidity detection module, a temperature detection module, a blood pressure detection module and / or a pulse detection module. Among them, the blood oxygen detection module is used to detect the blood oxygen parameters and respiratory parameters of the subject, and then record the changes in blood oxygen and respiratory function when the emotions change, the humidity detection module is used to detect the sweat secretion on the palm surface of the subject, the temperature detection module is used to detect the body temperature data and changes of the subject, the blood pressure detection module is used to detect the blood pressure parameters of the subject, and the pulse detection module is used to detect the pulse parameters of the subject. In order to improve the accuracy of the physiological characteristic parameters detected by each detection module, the blood oxygen detection module is arranged at the fingertips inside the smart glove, and the blood oxygen detection module can be multiple, respectively arranged at each fingertip, the humidity detection module is arranged at the palm part inside the smart glove, the temperature detection module is arranged at the outer wrist part inside the smart glove, and the blood pressure detection module and / or pulse detection module are arranged at the inner wrist part inside the smart glove. As an optimization, the smart glove can be a smart glove made of elastic breathable material. The elastic breathable material can improve the comfort of the subject, and the breathable material can improve the accuracy of the detection of physiological characteristic parameters. The various detection modules mentioned above can be implemented according to the existing technology.

[0077] A preferred specific form of the integrated sleep monitoring device can be configured according to the Chinese patent application “2020212854437 Mental Health Monitoring System”.

[0078] 5. Record storage device

[0079] It includes a memory chip to record the specific changes in various vital signs such as blood oxygen and respiratory function when emotions change.

[0080] 6. Communication equipment

[0081] The sleep bed is connected to a Bluetooth data processing component, which is the core component of the sleep emotion multi-module monitoring device, and is used to aggregate and transmit data. The modules use wireless methods such as Bluetooth and WiFi or wired methods such as USB and HDMI to transmit and exchange data and signals.

[0082] 2. Data identification and decision-making equipment

[0083] The decision operation unit is used to select the stimulation mode and parameters suitable for the subject according to the subject's sleep sign data. The system includes:

[0084] The information receiving port is configured to: receive the subject's sleep sign data in real time;

[0085] The central computer processing system is configured to: input the basic information of the subject and the sleep sign data into a machine learning model, and obtain a stimulation mode and parameters for regulating the subject's sleep through the machine learning model;

[0086] The information sending port is configured to output the judgment result, stimulation mode and parameters of the subject's sleep mood.

[0087] in,

[0088] The sleep sign data include the following indicators: electroencephalogram, cardiac function (electrocardiogram, heart rate, blood pressure), respiratory function (FVC, FEV1, FEV1 / FVC), electromyogram, and body temperature.

[0089] The basic information includes the following indicators: age, gender, height, weight, waist circumference, smoking history, drinking history, medical history, surgical history, blood transfusion history, allergy history, long-term medication, and family genetic diseases.

[0090] The machine learning model is a hybrid deep neural network model. Figure 3 As shown, the specific structure is:

[0091] The raw data extraction layer is used to decode, decompile and disassemble the input raw data into a more readable and easier to understand data type;

[0092] The feature extraction layer is used to extract features from the processing results of the original data extraction layer;

[0093] The feature vector generation layer is used to generate several feature vectors using different algorithms for existence features and discrete features, and some features are concatenated to generate feature vectors together;

[0094] A multimodal sleep emotion detection layer, used to obtain the judgment result of the subject's sleep emotion according to the feature vector, as well as the stimulation mode and parameters for regulating the subject's sleep;

[0095] Among them, Figure 4 As shown, the multimodal sleep emotion detection layer includes an initial network layer and a final network layer, the initial network layer includes a plurality of unconnected DNN networks, and the feature vectors are independently sent to the initial network layer; the final network layer includes a DNN network structure; the last layer of each DNN network in the initial network layer is fully connected with the first layer of the DNN network in the final network layer; the multimodal sleep emotion detection layer adopts a pre-fusion method to fuse deep features, and the DNN network structure of the final network layer outputs the judgment result of the subject's sleep emotion, as well as the stimulation mode and parameters for regulating the subject's sleep.

[0096] In the initial network layer, each independent DNN network consists of an input layer and several hidden layers. In the DNN network, the neurons in each layer are not connected to each other, and each layer of neurons is only connected to the adjacent previous and next layers, and the layers are fully connected. In the final network layer, the DNN network structure consists of a fusion layer, several hidden layers and an output layer. The output layer has only one neuron, which is responsible for outputting the analysis results.

[0097] A plurality of feature vectors are independently fed into the DNN network of the initial network layer, and the DNN network of the final network layer outputs the judgment result of the subject's sleep mood, as well as the stimulation mode and parameters for regulating the subject's sleep.

[0098] The workflow of the above hybrid deep neural network model is as follows:

[0099] (1) Executed in the original data extraction layer:

[0100] For all basic information and sleep sign data, the first step is to classify them into existence-type features (categorical variables such as presence / absence, normal / abnormal, too high / normal / too low, etc.) and discrete features (specific values). Then, the original data is decoded, decompiled, and disassembled to convert it into a data type that is more readable and easier to understand.

[0101] (2) Execution in the feature extraction layer:

[0102] Feature extraction is performed from the converted data. As a preferred method, the specific method of feature extraction can use a CNN model or an RNN model. As a preferred method, examples of extracting various sleep sign data and basic information are shown in the following table:

[0103] Table 1 Example of feature extraction of sleep sign data and basic information

[0104]

[0105]

[0106] (3) Execute in the feature vector generation layer:

[0107] The feature information extracted in this embodiment is divided into multiple types, and according to different forms of expression, it can be divided into two categories, namely, existence features and discrete features.

[0108] Specifically, the method for generating corresponding feature vectors from existence features and discrete features is:

[0109] (1) Existential feature vector generation algorithm

[0110] The process of generating the existence feature vector is intuitive and simple. First, all the existence features extracted by the feature extraction layer are combined into a large existence feature database. Then, the corresponding feature of each sample is queried in turn to see if it appears in the feature database. If it does, the feature is assigned a value of 1, otherwise it is 0. This system generates a total of several existence feature vectors (for example, there are 2, namely the respiratory rate feature vector and the combined feature vector composed of the heart rate feature, the electrocardiogram feature and the body temperature feature).

[0111] (2) Discrete feature vector generation algorithm

[0112] The performance of discrete data features is a combination of frequencies with great differences. Therefore, before generating feature vectors, the data needs to be normalized first. This system uses the linear function normalization (MinMax Scaling) method to normalize the feature frequency data to the range of [0,1] in turn. The normalization equation is shown in the following formula, where x and y represent the feature data before and after normalization, respectively, and Max and Min represent the maximum and minimum values ​​of the frequency, respectively:

[0113]

[0114] Then, the feature data are clustered by K-means (reference: IBMohamad, D. Usman. Standardization and its effects on k-means clustering algorithm [J]. Research Journal of Applied Sciences, Engineering and Technology, 2013, 6 (17): 3299-3303) to obtain a set of centroids for each feature data. The feature data extracted from the sample will be calculated to calculate the distance between the different centroids of the corresponding type. The distance calculation uses the Euclidean distance calculation formula (as shown below), where t represents the coordinate dimension of points M and N.

[0115]

[0116] Select the centroid with the shortest distance, and the length of the projection of the vector from the centroid to the current feature point in different dimensions indicates the similarity of the features of the data and the centroid in different dimensions. The longer the projection length on a dimension, the lower the similarity of the features in that dimension. On the contrary, the shorter the projection length, the higher the similarity. In this system, the projection length of 0.4 is used as the threshold. The elements in the feature vector corresponding to the features in the dimension with a projection length exceeding 0.4 are set to 0, otherwise they are set to 1, thereby avoiding some weakly correlated features from having a greater impact on subsequent prediction tasks, and also simplifying the amount of calculation in the deep learning process.

[0117] (4) Execution in the multimodal sleep emotion detection layer:

[0118] The feature vectors finally generated by the feature vector generation layer will be independently sent to the initial network layer composed of multiple DNN networks. The multimodal sleep emotion detection layer uses a pre-fusion method to fuse deep features and output the judgment results of the subject's sleep emotion, as well as the stimulation mode and parameters for regulating the subject's sleep.

[0119] Through the above-mentioned hybrid deep neural network model, the sleep mood of the subject can be judged in real time, and the stimulation mode and parameters for regulating the patient's sleep can be obtained.

[0120] The above hybrid deep neural network model is used to judge the sleep mood of the subjects, which has the following characteristics:

[0121] Based on the existing multimodal DNN (MDNN) model, this embodiment uses a fuzzy algorithm in the feature vector generation layer to concatenate some features to jointly generate a feature vector (i.e., fuzzy integral is added).

[0122] There are two special cases when performing feature concatenation in the feature vector generation layer:

[0123] (1) When there is an interaction between modes: the combined effect of the two modes is not simply equivalent to the sum of the effects of the individual modes.

[0124] (2) When the classification result is unclear: Unclear classification results mean that a sample cannot be clearly determined as positive or negative.

[0125] At this time we can use the fuzzy integral fuser (such as Figure 5 As shown in Figure 3, the multimodal results are effectively fused.

[0126] Compared with the existing fusion operators, a prominent advantage of fuzzy integral is that it can better reflect the importance of the classifiers corresponding to each mode and can better represent the interaction between the modes. Generally, there are three forms of interaction between classifiers Di and Dj:

[0127] (1) Negative synergy: If the overall importance of sub-classifiers Di and Dj is less than the sum of individual sub-classifiers. In other words, compared with single classifiers Di and Dj, fusion classifiers Ci and Cj cannot improve the classification accuracy of a certain category.

[0128] (2) Positive synergy: If the overall significance of sub-classifiers Di, c fj for classification is greater than the sum of the significance of individual classifiers. In other words, compared with single classifiers Di, Dj, fusion classifiers Ci, Cj can improve the classification accuracy of a certain category.

[0129] (3) Independence: It is to add the influence of individual classifiers Di and Dj on the classification accuracy.

[0130] When there is interaction between different modes, this embodiment uses fuzzy measurement to express it. It can be seen from the following Choquet fuzzy integral formula that f is the membership of the mode, that is, the performance ability of different modes on the classifier, which is multiplied by its measurement and finally summed up to obtain the score of the input in this category.

[0131] Let X be any set, that is, μ is a fuzzy measure defined on the set X, f:X→[0,1] is a non-negative real-valued measurable function defined on X, and the value is converted into Sort, and the minimum value of a can be 0, A i Indicates a i The Choquet fuzzy integral formula is as follows:

[0132]

[0133] It can be seen from the following fuzzy measure table that if the input is three modes, the measure not only includes the measure values ​​of all single modes, but also includes the measure of the combination of two modes and three modes. While indicating the importance of the single mode, the interaction between the multiple modes is also shown in the measure. The fuzzy integral mainly characterizes the interaction between different modes through the calculation of the measure. Therefore, this embodiment uses the fuzzy system to model multimodal fusion, and uses the measure to simultaneously characterize the importance and interactivity of the multiple modes, and find the optimal measure calculation method for multimodal fusion.

[0134] Table 2 Three-modal fuzzy measurement table

[0135]

[0136] In summary, the fuzzy integral added in this embodiment allows the interaction between different data to be considered on the basis of the two data analyses of "mutually independent unit system" and "pre-fusion". Because the data are related, the weighted sum of the single-modal measure (Choquet fuzzy integral) is performed through the lifting degree to form a multi-modal measure, and then based on the calculation result of this fuzzy measure, a decision fusion framework of multi-modal multi-classifiers is constructed. The framework trains and optimizes multiple different types of emotional state basic classifiers for each modality, thereby obtaining decision information of multiple basic classifiers on different modal data. Then, the decision information of each modality is integrated and calculated using a newly designed multi-basic classifier decision information integration method to provide input information for the subsequent fuzzy measure calculation, which is far superior to the existing MDNN model.

[0137] Among them, "mutually independent unit systems" means that the indicators such as "EEG", "EMG" and "ECG" are all independent, non-interfering unit systems (because an ECG contains a lot of content, such as frequency and waveform, atrial flutter is F wave, atrial fibrillation is f wave). "Pre-fusion" means that in the feature extraction and classification stage of the data set, different feature extraction algorithms and classification algorithms are first fused to obtain better data analysis results; and post-fusion algorithm means that after the feature extraction and classification stage, multiple classifiers are fused to obtain better classification results.

[0138] In the actual model training and testing process, it is not necessarily possible to ensure that all basic information and sleep sign data can be fully collected. The hybrid deep neural network model in this embodiment also has an appropriate response plan for this situation. The details are as follows:

[0139] ① Training scheme when feature extraction is incomplete

[0140] The hybrid deep neural network model of this embodiment adopts a multimodal feature input structure. Each DNN network in the initial network layer is independent of each other and is only fused in the final network layer. This makes it possible for each independent DNN network in the initial network layer to be trained separately in sequence. During the model training process, zero vector completion is not performed for feature information that fails to be extracted, and the successfully extracted feature vectors of various types are input into the corresponding DNN network of the initial network layer in sequence for training. Finally, the final network layer is trained on the basis of the completion of the training of the entire initial network layer. Such a training strategy allows the system to maintain feature independence to the greatest extent while reducing the reliance on all features to be successfully extracted at the same time, and maximizes the use of all extracted feature information.

[0141] ②Testing plan when feature extraction is incomplete

[0142] In the detection phase, if the basic information or sleep sign data of the tested subject is incomplete, the 0 vector is first used to complete it, and then all feature vectors are input into the hybrid deep neural network model for detection. Because the final output of the initial network corresponding to the 0 vector input only calculates the bias term, the input of the 0 vector has little effect on the final classification result, ensuring the accuracy of the overall detection result.

[0143] 3. Multi-sensory stimulation intervention and enhancement equipment

[0144] The multi-sensory stimulation intervention and strengthening device can be implemented by existing technologies. As a preferred method, the multi-sensory stimulation intervention and strengthening device of this embodiment includes the following components:

[0145] 1. VR visual stimulation equipment: Use VR to play visual stimulation that has been seen while awake, such as teaching films, emotional videos, etc., to the sleeper without affecting sleep, so as to enhance related memory.

[0146] 2. Auditory stimulation device (can be an external speaker or in-ear player): used to provide soothing, pleasant or motivating music, sounds or language to the user. The device selects audio content of different types, styles, rhythms and volumes.

[0147] 3. Tactile stimulation equipment:

[0148] The specific form can be:

[0149] (1) Wearable glove type, with multiple stimulation modules in the glove;

[0150] (2) Foot massager;

[0151] (3) Mattress-integrated massager;

[0152] It is used to provide users with gentle, comfortable or stimulating tactile sensations. The device can adjust tactile stimulation in different parts, intensities, frequencies and patterns.

[0153] 4. Olfactory stimulation equipment (can be an indoor aromatherapy machine): used to provide users with fresh, fragrant or exciting smells. The equipment can select olfactory stimulation of different types, concentrations and durations.

[0154] 5. Treatment room equipment: including temperature control system (central air conditioning) and air humidity control equipment (indoor humidifier).

[0155] When the sleeping cabin system of this embodiment is in operation, it may further include the following features:

[0156] 1. Timing (sleep stage):

[0157] Human sleep can be divided into two parts: rapid eye movement (REM) sleep and non-rapid eye movement (NREM) sleep.

[0158] Combined with the above theory, regulating the patient's sleep and memory during rapid eye movement (REM) sleep is a better choice, which may achieve more significant therapeutic effects. In this embodiment, the sleep emotion multi-module monitoring device can determine when the patient enters the REM sleep stage, so that VR multi-sensory stimulation can be performed at this stage through the multi-sensory stimulation intervention enhancement device.

[0159] 2. Selection of parameters for multi-sensory stimulation intervention reinforcement module:

[0160] The specific stimulation parameters will be determined based on the results of data discrimination and decision-making equipment, striving to bring better treatment effects to patients with the optimal stimulation mode and parameters.

[0161] The parameter selection ranges for various stimulation modes are as follows:

[0162] Tactile stimulation: hand massage: 0.5-2 times / s, strength: gentle;

[0163] Foot massage: 0.5 to 2 times / s, strength: gentle to moderately strong;

[0164] Auditory stimulation: sound intensity: 20-40 decibels, sound nature: beautiful and soft melody;

[0165] Olfactory stimulation: comfortable and pleasant smells, such as: elegant floral fragrance, sweet fruit fragrance, fresh grass fragrance;

[0166] Indoor ambient temperature: 18-25 degrees Celsius;

[0167] Indoor environmental humidity: Generally, the suitable humidity is 45-65%, the relative humidity is 40-80% during cooling in summer, and 30-60% during heating in winter. The suitable humidity for the elderly and children is 45-50%, and the suitable humidity for patients with respiratory diseases such as asthma is 40-50%.

[0168] Through the above embodiments, it can be seen that the present invention provides a sleeping cabin device, which can monitor the sleep sign data of the subject in real time during the sleep process, and judge the subject's emotions during sleep, the stimulation model and stimulation parameters that need to be accepted in real time, and then provide appropriate stimulation during the subject's sleep. The present invention can regulate the emotional memory during sleep for patients with depression (it can also be used for the general population and people with sleep disorders), eliminate negative memories, enhance positive memories; improve sleep; adjust emotions; relieve depression symptoms, and improve the quality of life, and has a good application prospect.

Claims

1. A sleeping cabin for emotional memory regulation, characterized in that: include: A sleeping bed, for the subjects to sleep; Sleep and mood multi-module monitoring equipment, used to monitor the subjects' sleep sign data in real time; A data discrimination and decision-making device, integrated with a decision-making operation unit, is used to obtain a stimulation mode and parameters for regulating the sleep of a subject according to the sleep sign data; Multi-sensory stimulation intervention and enhancement equipment, used to stimulate the subject according to the stimulation mode and parameters to achieve sleep regulation; Wherein, the decision operation unit includes: The information receiving port is configured to: receive the subject's sleep sign data in real time; The central computer processing system is configured to: input the sleep sign data into a machine learning model, and obtain the judgment result of the subject's sleep mood and the stimulation mode and parameters for regulating the subject's sleep through the machine learning model; the machine learning model is a hybrid deep neural network model; The information sending port is configured to output the judgment result of the subject's sleep mood, the stimulation mode and parameters.

2. The sleeping cabin according to claim 1, characterized in that: The sleep emotion multi-module monitoring device includes at least one of the following devices: a contact polysomnography device, a fiber optic sensing sleep monitoring device, and a smart glove integrated with a sensor.

3. The sleeping cabin according to claim 1, characterized in that: The multi-sensory stimulation intervention and enhancement equipment includes at least one of the following devices: VR visual stimulation equipment, auditory stimulation equipment, tactile stimulation equipment, olfactory stimulation equipment, temperature control system, and air humidity control equipment.

4. The sleeping cabin according to claim 3, characterized in that: The multi-sensory stimulation intervention enhancement device is configured to stimulate the subject when the subject enters the rapid eye movement sleep stage.

5. The sleeping cabin according to claim 1, characterized in that: The sleep sign data includes at least one of the following indicators: electroencephalogram, electrocardiogram, heart rate, blood pressure, FVC, FEV1, FEV1 / FVC, electromyogram, and body temperature; And / or, in the central computer processing system, basic information of the patient is also input into the machine learning model, wherein the basic information includes at least one of the following indicators: age, gender, height, weight, waist circumference, smoking history, drinking history, medical history / hospitalization history / accident history, surgical history, blood transfusion history, allergy history, long-term medication, and family genetic diseases; And / or, the stimulation mode includes at least one of the following stimulation modes: tactile stimulation, temperature stimulation, auditory stimulation, olfactory stimulation, air humidity, and transcranial direct current stimulation.

6. The sleeping cabin according to claim 1, characterized in that: The hybrid deep neural network model includes: The raw data extraction layer is used to decode, decompile and disassemble the input raw data into a more readable and easier to understand data type; The feature extraction layer is used to extract features from the processing results of the original data extraction layer; The feature vector generation layer is used to generate several feature vectors using different algorithms for existence features and discrete features, among which some features are spliced ​​by fuzzy algorithm to generate feature vectors together; A multimodal sleep emotion detection layer, used to obtain the judgment result of the subject's sleep emotion according to the feature vector, as well as the stimulation mode and parameters for regulating the subject's sleep; Among them, the multimodal sleep emotion detection layer includes an initial network layer and a final network layer, the initial network layer includes a number of unconnected DNN networks, and the feature vectors are independently sent to the initial network layer; the final network layer includes a DNN network structure; the last layer of each DNN network in the initial network layer is fully connected with the first layer of the DNN network in the final network layer; the multimodal sleep emotion detection layer adopts a pre-fusion method to fuse deep features, and the DNN network structure of the final network layer outputs the judgment result of the subject's sleep emotion, as well as the stimulation mode and parameters for regulating the subject's sleep.

7. The sleeping cabin according to claim 6, characterized in that: In the initial network layer, each independent DNN network consists of an input layer and several hidden layers. In the DNN network, neurons in each layer are not connected to each other, and neurons in each layer are only connected to adjacent previous and next layers, and the layers are fully connected. In the final network layer, the DNN network structure consists of a fusion layer, several hidden layers and an output layer. The output layer has only one neuron, which is responsible for outputting the analysis results.

8. The sleeping cabin according to claim 6, characterized in that: In the feature vector generation layer, the method for generating the existential feature vector is: All the existence features extracted by the feature extraction layer form an existence feature database; query the corresponding feature of each sample in turn to see whether it appears in the feature database. If it exists, the feature is assigned a value of 1, otherwise it is 0; The method for generating discrete feature vectors is: Normalize the feature data; perform K-means clustering on the normalized data to obtain a set of centroids for each type of feature data; the feature data extracted from the sample will be calculated separately for their distances to different centroids of the corresponding type; select the centroid with the shortest distance, and the length of the projection of the vector pointing from the centroid to the current feature point in different dimensions represents the similarity of the features of the data and the centroid in different dimensions; use the projection length of 0.4 as the threshold, and set the elements in the feature vector corresponding to the features in the dimension where the projection length exceeds 0.4 to 0, otherwise set to 1.

9. The sleeping cabin according to claim 6, characterized in that: The feature vector generation layer extracts a feature vector for each discrete feature and concatenates all the existence features to generate a feature vector.

10. The sleeping cabin according to claim 6, characterized in that: In the feature vector generation layer, 0 vectors are used to complete the incomplete features.

Images