A sleep pod for emotional memory modulation

Abstract

The present application belongs to the technical field of medical equipment, and particularly relates to a sleep cabin for emotion memory regulation. The sleep cabin comprises: a sleep bed for a subject to sleep; a sleep emotion multi-module monitoring device for monitoring sleep physical data of the subject in real time; a data discrimination and decision determination device integrated with a decision operation unit for obtaining a stimulation mode and parameters for regulating the sleep of the subject according to the sleep physical data; and a multi-sensory stimulation intervention reinforcement device for stimulating the subject according to the stimulation mode and parameters to regulate the sleep. The present application can regulate the emotion memory of a depression patient (also applicable to ordinary people and people with sleep disorders) in sleep, fade negative memory, enhance positive memory, improve sleep, adjust emotion, relieve depression symptoms, and improve the quality of life.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical equipment, and particularly relates to a sleep cabin for emotion memory regulation. BACKGROUND

[0002] Depression disorder is a common mental illness and is the most common mental health problem in the world. Patients usually show low mood, decreased interest and lack of energy.

[0003] The severity of depression has prompted many researchers to conduct in-depth theoretical and practical research on its pathogenesis, revealing the pathological characteristics of patients with depression, including: ① persistent and long-term low mood or imbalance of emotion memory regulation; ② enhancement of negative memory and weakening of positive memory; ③ sleep disorders.

[0004] Methods for treating depression include drug therapy, psychological therapy and physical therapy, but these methods have large differences in efficacy for different individuals. This difference makes the contradiction between the increasing number of patients and the perfect treatment plan increasingly prominent, and it is urgent to develop a device and method with specificity, pertinence, wide applicability and strong operability to realize individualized precision treatment for patients with depression.

[0005] In clinical practice, about 70% of patients with depression have sleep problems, and the risk of people with sleep problems suffering from depression or anxiety is also significantly higher than that of people with normal sleep. Therefore, treating depression by intervening in the sleep of patients with depression is a new treatment method (in addition, intervening in the sleep of normal people may also have a preventive effect on depression). In the prior art, although the relationship between depression and sleep has been studied, how to determine the specific means and parameters of sleep intervention for patients with depression is still a problem to be solved. SUMMARY

[0006] In view of the problems in the prior art, the present application provides a sleep cabin for emotion memory regulation.

[0007] A sleep cabin for emotion memory regulation, characterized in that it comprises:

[0008] a sleep bed for a subject to sleep;

[0009] a sleep emotion multi-module monitoring device for monitoring sleep vital sign data of the subject in real time;

[0010] a data discrimination and decision determination device integrated with a decision operation unit for obtaining a stimulation mode and parameters for regulating the sleep of the subject according to the sleep vital sign data;

[0011] The multi-sensory stimulation intervention reinforcement device is used for stimulating the subject according to the stimulation mode and parameters, and realizing the regulation of sleep.

[0012] The decision operation unit comprises:

[0013] The information receiving port is configured to receive the sleep sign data of the subject in real time.

[0014] The central computer processing system is configured to input the sleep sign data into a machine learning model, obtain a judgment result of the sleep emotion of the subject and a stimulation mode and parameters for regulating the sleep of the subject through the machine learning model, and the machine learning model is a hybrid deep neural network model.

[0015] The information sending port is configured to output the judgment result of the sleep emotion of the subject and the stimulation mode and parameters.

[0016] Preferably, the sleep emotion multi-module monitoring device comprises at least one of the following devices: a contact multi-lead sleep instrument, a sleep monitoring device with optical fiber sensing, and a smart glove integrated with a sensor.

[0017] Preferably, the multi-sensory stimulation intervention reinforcement device comprises at least one of the following devices: a VR visual stimulation device, an auditory stimulation device, a tactile stimulation device, an olfactory stimulation device, a temperature regulation system, and an air humidity control device.

[0018] Preferably, the multi-sensory stimulation intervention reinforcement device is configured to stimulate the subject when the subject enters a rapid eye movement sleep stage.

[0019] Preferably, the sleep sign data comprises at least one of the following indexes: electroencephalogram, electrocardiogram, heart rate, blood pressure, FVC, FEV1, FEV1 / FVC, electromyogram, and body temperature; wherein, the FVC, FEV1, and FEV1 / FVC represent forced vital capacity, forced expiratory volume in one second, and one-second rate, respectively.

[0020] And / or, the central computer processing system further inputs basic information of the patient into the machine learning model, and the basic information comprises at least one of the following indexes: age, gender, height, weight, waist circumference, smoking history, drinking history, disease history / hospitalization history / accident, operation history, blood transfusion history, allergy history, long-term medication, and family genetic disease.

[0021] And / or, the stimulation mode comprises at least one of the following stimulation methods: tactile stimulation, temperature sensation stimulation, auditory stimulation, olfactory stimulation, air humidity, and transcranial direct current stimulation.

[0022] Preferably, the mixed deep neural network model comprises:

[0023] a raw data extraction layer for decoding, decompiling and disassembling the input raw data to convert them into a more readable and understandable data type;

[0024] a feature extraction layer for extracting features from the processing results of the raw data extraction layer;

[0025] a feature vector generation layer for generating a plurality of feature vectors using different algorithms for the presence type features and the discrete type features respectively, wherein part of the features are spliced together by a fuzzy algorithm to generate a feature vector;

[0026] a multi-modal sleep emotion detection layer for obtaining the judgment result of the subject's sleep emotion and the stimulation mode and parameters for regulating the subject's sleep according to the feature vectors;

[0027] Preferably, the multi-modal sleep emotion detection layer comprises an initial network layer and a final network layer, the initial network layer comprises a plurality of DNN networks that are not connected to each other, and the feature vectors are independently fed into the initial network layer; the final network layer comprises a DNN network structure; the multi-modal sleep emotion detection layer adopts a front fusion manner for fusion between deep features, and the DNN network structure of the final network layer outputs the judgment result of the subject's sleep emotion and the stimulation mode and parameters for regulating the subject's sleep.

[0028] Preferably, in the initial network layer, each independent DNN network is composed of an input layer and a plurality of hidden layers, in the DNN network, the neurons in each layer are not connected to each other, each layer of neurons is only connected to the adjacent front and rear layers, and the layers are fully connected structures.

[0029] In the final network layer, the DNN network structure is composed of a fusion layer, a plurality of hidden layers and an output layer, and the output layer has only one neuron responsible for outputting the analysis result.

[0030] Preferably, in the feature vector generation layer, the presence type feature vector generation method is:

[0031] all presence type features extracted by the feature extraction layer are composed into a presence type feature database; whether the corresponding features of each sample exist in the feature database is queried in sequence, if they exist, the feature is assigned a value of 1, otherwise, a value of 0;

[0032] the discrete type feature vector generation method is:

[0033] The normalization of the feature data is performed, and the normalized data is subjected to K-means clustering to obtain a group of centroids of each type of feature data; the feature data extracted from the sample is calculated to have a distance to different centroids of the corresponding type; the shortest distance centroid is selected, and the length of the vector from the centroid to the current feature point in different dimensions represents the similarity of the data and the centroid in different dimensions; the projection length 0.4 is taken as a threshold, and the elements in the feature vector corresponding to the features in the dimensions with the projection length greater than 0.4 are set to 0, otherwise, set to 1.

[0034] Preferably, the feature vector generation layer extracts a feature vector for each discrete feature, and splices all the existing features to generate a feature vector.

[0035] Preferably, in the feature vector generation layer, a 0 vector is used to complete the extraction of incomplete features.

[0036] The application provides a sleep cabin and a control system thereof, which can realize real-time sleep regulation of“monitoring-treatment-feedback-remonitoring-treatment-better feedback” through deep learning, and can regulate the sleep of a subject in an individualized, adjustable and precise stimulation effect.

[0037] To achieve the above object, the application provides an optimized hybrid deep neural network model, which is optimized for uncertainty, accuracy and individualization compared with the existing similar model, is a multi-modal fusion emotion recognition model based on Choquet fuzzy integral, and is the most robust, accurate and complete data learning and prediction method reported so far.

[0038] In the prior art, a hybrid deep neural network model has been proposed for predicting drug-drug interactions (DDI). The application first applies it to emotion memory regulation and optimizes the model for this purpose, further improving accuracy and individualization.

[0039] The above advantages are embodied in the following aspects:

[0040] 1. Processing uncertainty and enhancing the generalization ability of the model: the hybrid deep neural network model of the application can provide more comprehensive data analysis by integrating information from different data sources (such as respiration, electrocardiogram, body temperature, etc.). Fuzzy integral can help to process the uncertainty and fuzziness in clinical data, so that the model can learn and make predictions from these data more robustly.

[0041] 2. Improved diagnostic accuracy: By incorporating fuzzy integration, the hybrid deep neural network model of the present invention can more accurately diagnose diseases or predict treatment outcomes, especially in cases where data is incomplete or variability exists.

[0042] 3. Personalized treatment planning: Fuzzy integration can help the hybrid deep neural network model of the present invention better understand and predict individual patient responses to treatment, thereby helping the sleep cabin develop more personalized treatment plans.

[0043] 4. Optimized decision support: In a clinical decision support system, fuzzy integration can help weigh the importance of different factors, allowing the sleep cabin to develop more comprehensive and balanced treatment plans.

[0044] 5. Support for explainable artificial intelligence (XAI): Fuzzy integration provides a natural way to explain how the model integrates different factors to make decisions, enhancing the model's explainability, which is particularly important in the medical field as it provides a more transparent decision-making process, which is crucial for improving doctors' trust and acceptance of AI systems. The pathological features of patients with depression are continuous and long-term low mood, enhanced negative memory and weakened positive memory, and sleep disorders, and the regulation of these features is the core of curing depression. Through the sleep cabin system of the present invention, the sleep of patients with depression (which can also be used for ordinary people and people with sleep disorders) can be regulated, negative memories can be reduced, positive memories can be enhanced, sleep can be improved, emotions can be adjusted, and depression symptoms can be alleviated, thereby improving the quality of life. Therefore, the present invention has good application prospects.

[0045] Obviously, according to the above content of the present invention, according to the ordinary technical knowledge and common practice in the art, other modifications, replacements or changes can be made without departing from the above technical ideas of the present invention.

[0046] The above content of the present invention will be further described in detail through the specific embodiments below. However, it should not be understood that the above subject matter of the present invention is limited to the following examples. Any technology achieved based on the above content of the present invention belongs to the scope of the present invention. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a schematic diagram of the composition of the sleep cabin system of the present invention;

[0048] Figure 2 is a schematic diagram of the preferred structure of the sleep cabin of the present invention;

[0049] Figure 3 is a schematic diagram of the flow of the hybrid deep neural network model in the present invention;

[0050] Figure 4This is a schematic diagram of the network structure of the multimodal sleep emotion detection layer in this invention;

[0051] Figure 5 A schematic diagram of fuzzy integral fusion for multi-classifiers. Detailed Implementation

[0052] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.

[0053] Example 1: Sleep pod for emotional memory regulation

[0054] The sleep chamber provided in this embodiment can detect the sleep vital signs data of the subject in real time, judge the subject's sleep mood based on this, and provide appropriate stimulation in real time to regulate the subject's sleep.

[0055] Specifically, such as Figures 1-2 As shown, the system in this embodiment includes:

[0056] Sleeping beds for test subjects to sleep on;

[0057] A multi-module sleep-emotion monitoring device is used to monitor the sleep vital signs data of subjects in real time;

[0058] The data discrimination and decision-making device integrates a decision-making unit, which is used to obtain stimulation patterns and parameters for regulating the sleep of the subject based on the sleep signs data;

[0059] A multi-sensory stimulation intervention enhancement device is used to stimulate subjects according to the stimulation patterns and parameters to regulate sleep.

[0060] The specific structure of each part is as follows:

[0061] I. Multi-module sleep and mood monitoring equipment

[0062] A multi-module sleep mood monitoring device can be implemented using existing technology. As a preferred embodiment, the multi-module sleep mood monitoring device of this embodiment includes the following components:

[0063] 1. Devices connected to the head of the sleeping bed

[0064] Specifically, a contact-type polysomnography device is used to record electroencephalogram (EEG), electrocardiogram (ECG), and electromyography (EMG), recording the patient's EEG, ECG, and EMG information during sleep.

[0065] The principle is that the electrode attached to the skin transmits bioelectric information to the sensor, and different parts jointly transmit information. The sensor cable writes the signal to the headbox device, and then the analog signal is converted into a digital signal after the amplification, filtering and digitization process through the headbox, and is sent to the fixed station. The signal can also be sent to the fixed station through the auxiliary input device, and the signal can be directly amplified and adjusted in the auxiliary device.

[0066] 2. Right side connection of sleep bed: sleep monitoring device based on low-cost fiber sensing

[0067] A low-cost fiber sensing device embedded in a mattress that can be used to measure respiratory rate and heart rate for long periods of time, non-invasively and resistant to electromagnetic interference. Four behavioral states can be distinguished (not in bed, lying, moving and out of bed), as well as measuring respiratory and heart rate in different positions and postures, suitable for long-term sleep monitoring, and the data obtained can be used for the analysis of diseases such as sleep disorders.

[0068] Principle: This is a temperature-pressure integrated fiber Bragg grating sensor and human sleep multi-physiological monitoring system. The system includes a mattress, a plurality of temperature-pressure integrated fiber Bragg grating sensors, a demodulator, an upper computer, a plurality of physiological fiber Bragg grating sensors fixedly arranged in the gap between the mattress and the bed net, and kept at the same level. The depth of the plurality of physiological fiber Bragg grating sensors is connected with the demodulator, and the data is transmitted to the upper computer. The system identifies the pressure on the sensor by detecting the wavelength of the wavelength splitting point of the fiber Bragg grating spectrum signal, and identifies the temperature of the environment where the sensor is located by detecting the center wavelength drift of the fiber Bragg grating spectrum signal. The multi-physiological information of the user when in bed is monitored, including body temperature, respiratory rate, heart rate, out-of-bed state, etc., and has the advantages of simple structure, small size, resistance to electromagnetic interference, low cost, etc.

[0069] 3. Left side connection of sleep bed: front end of the Dormio system

[0070] Monitor the time and state of sleep onset. Three physiological indicators of sleep onset are used.

[0071] ① Subjects are asked to lie down to sleep with their hands gently closed, allowing a curved sensor to pass through the hand to monitor the gradual loss of muscle tone, and sensors around the index finger can track muscle tone (Kelly JM, Strecker RE, Bianchi MT. Recent developments in home sleep-monitoring devices. ISRN Neurol. 2012; 2012: 768794. doi: 10.5402 / 2012 / 768794. Epub 2012 Oct 14. PMID: 23097718; PMCID: PMC3477711.). Because the loss of muscle tone is temporally related to the presence of sleep imagery.

[0072] ② Heart rate decrease: in line with the loss of muscle tone (Herlan A, Ottenbacher J, Schneider J, Riemann D, Feige B. Electrodermal activity patterns in sleep stages and their utility for sleep versus wake classification. J Sleep Res. 2019 Apr; 28(2): e12694. doi: 10.1111 / jsr.12694. Epub 2018 May 2. PMID: 29722079.), heart rate can be monitored on the middle finger of the subject

[0073] ③ Electrodermal activity (EDA): in line with the loss of muscle tone (Herlan A, Ottenbacher J, Schneider J, Riemann D, Feige B. Electrodermal activity patterns in sleep stages and their utility for sleep versus wake classification. J Sleep Res. 2019

[0074] Apr; 28(2): e12694. doi: 10.1111 / jsr.12694. Epub 2018 May 2. PMID: 29722079.), two electrodes at the bottom of the wrist can measure EDA.

[0075] 4. Integrated device for sleep monitoring

[0076] The integrated device is a smart glove, which comprises a blood oxygen detection module, a humidity detection module, a temperature detection module, a blood pressure detection module and / or a pulse detection module. The blood oxygen detection module is used to detect the blood oxygen parameters and respiratory parameters of the subject, and then record the changes of blood oxygen and respiratory function when the emotion changes. The humidity detection module is used to detect the palm sweat secretion of the subject. The temperature detection module is used to detect the body temperature data and changes of the subject. The blood pressure detection module is used to detect the blood pressure parameters of the subject. The pulse detection module is used to detect the pulse parameters of the subject. In order to improve the accuracy of the physiological characteristic parameters detected by each detection module, the blood oxygen detection module is arranged at the fingertip position inside the smart glove. The blood oxygen detection module can be multiple, and is arranged at each fingertip respectively. The humidity detection module is arranged at the palm position inside the smart glove. The temperature detection module is arranged at the outside wrist position inside the smart glove. The blood pressure detection module and / or the pulse detection module are arranged at the inside wrist position inside the smart glove. As an optimization, the smart glove can be made of elastic breathable material. The elastic breathable material can improve the comfort of the subject, and the breathable material can improve the accuracy of the physiological characteristic parameter detection. The above various detection modules can be realized according to the existing technology.

[0077] The integrated device for sleep monitoring can be arranged according to the Chinese patent application "2020212854437 psychological health detection system" in a preferred specific form.

[0078] 5、Record storage device

[0079] The memory chip is used to record the specific change of each vital sign data such as blood oxygen and respiratory function when the emotion changes,

[0080] 6、Communication device

[0081] The sleep bed is connected with a Bluetooth data processing element, which is the core element of the sleep emotion multi-module monitoring device, and is used to collect and transmit data. Bluetooth, WiFi and other wireless methods or USB, HDMI and other wired methods are used for data and signal transmission and exchange between modules.

[0082] II、Data discrimination and decision-making device

[0083] The decision-making operation unit is used to select the stimulation mode and parameters suitable for the subject according to the sleep vital sign data of the subject. The system comprises:

[0084] The information receiving port is configured to receive the sleep vital sign data of the subject in real time.

[0085] A central computer processing system is configured to: input basic information of a subject and the sleep sign data into a machine learning model, and obtain a stimulation mode and parameters for regulating sleep of the subject through the machine learning model.

[0086] An information sending port is configured to output the judgment result of the sleep emotion of the subject, the stimulation mode and the parameters.

[0087] Among them,

[0088] The sleep sign data includes the following indexes: electroencephalogram, heart function (electrocardiogram, heart rate, blood pressure), respiratory function (FVC, FEV1, FEV1 / FVC), electromyogram, body temperature.

[0089] The basic information includes the following indexes: age, gender, height, weight, waist circumference, smoking history, drinking history, disease history, operation history, blood transfusion history, allergy history, long-term medication, family genetic disease.

[0090] The machine learning model is a hybrid deep neural network model, and the hybrid deep neural network model is as shown in Figure 3 The specific structure is as follows:

[0091] An original data extraction layer is used to decode, decompile and disassemble the input original data, and convert them into a data type with stronger readability and easier understanding;

[0092] A feature extraction layer is used to extract features from the processing results of the original data extraction layer;

[0093] A feature vector generation layer is used to generate a plurality of feature vectors by using different algorithms for the existing feature and the discrete feature, respectively, and part of the features are spliced to generate a feature vector together;

[0094] A multi-modal sleep emotion detection layer is used to obtain a judgment result of a sleep emotion of a subject, and a stimulation mode and parameters for regulating sleep of the subject according to the feature vector;

[0095] Among them, as shown in Figure 4 The multi-modal sleep emotion detection layer includes an initial network layer and a final network layer, the initial network layer includes a plurality of DNN networks that are not connected to each other, the feature vectors are independently sent into the initial network layer, the final network layer includes a DNN network structure, the last layer of each DNN network in the initial network layer is fully connected with the first layer of the DNN network in the final network layer, the multi-modal sleep emotion detection layer adopts a front fusion manner to fuse the deep features, and the DNN network structure of the final network layer outputs the judgment result of the sleep emotion of the subject, and the stimulation mode and the parameters for regulating the sleep of the subject.

[0096] Each independent DNN network in the initial network layer is composed of an input layer and several hidden layers. In the DNN network, the neurons in each layer are not connected to each other, and each layer of neurons is only connected to the adjacent front and rear layers, and the layers are fully connected structures. In the final network layer, the DNN network structure is composed of a fusion layer, several hidden layers and an output layer. The output layer has only one neuron, which is responsible for outputting the analysis result.

[0097] The several feature vectors are independently sent to the DNN network of the initial network layer, and the DNN network of the final network layer outputs the judgment result of the sleep emotion of the subject and the stimulation mode and parameters for regulating the sleep of the subject.

[0098] The working process of the above-mentioned hybrid deep neural network model is specifically as follows:

[0099] (1) Executed in the original data extraction layer:

[0100] For all basic information and sleep sign data, the first step is to classify into existing type features (yes / no, normal / abnormal, too high / normal / low, etc. classification variables) and discrete type features (specific numerical values), and then perform decoding, decompilation and disassembly operations on the original data to convert them into data types that are more readable and easier to understand.

[0101] (2) Executed in the feature extraction layer:

[0102] Feature extraction is performed from the converted data. As a preferred way, the specific method of feature extraction can use a CNN model or an RNN model. As a preferred way, examples of sleep sign data and basic information extraction are shown in the following table:

[0103] Table 1: Examples of sleep sign data and basic information feature extraction

[0104]

[0105]

[0106] (3) Executed in the feature vector generation layer:

[0107] The feature information extracted in this embodiment is divided into multiple types, and according to different forms, it can be divided into two categories, namely existing type features and discrete type features.

[0108] Specifically, the method for generating corresponding feature vectors for existing type features and discrete type features is:

[0109] (1) Existing type feature vector generation algorithm

[0110] The generation process of the presence type feature vector is intuitive and simple. First, all the presence type features extracted by the feature extraction layer are combined into a large presence type feature database. Then, it is queried whether the corresponding features of each sample exist in the feature database. If they exist, the feature is assigned a value of 1, otherwise 0. The system generates several presence type feature vectors (for example, 2, including a respiratory rate feature vector and a combined feature vector composed of heart rate features, electrocardiogram features, and body temperature features).

[0111] (2) Discrete feature vector generation algorithm

[0112] The discrete data feature is a combination of a group of very different frequencies, so before generating the feature vector, the data needs to be normalized first. The system uses the linear function normalization (MinMax Scaling) method to normalize the feature frequency data to the range of [0, 1] one by one. The normalization equation is shown in the following formula, where x and y represent the feature data before and after normalization, respectively, and Max and Min represent the maximum and minimum values of the frequency, respectively:

[0113]

[0114] Next, the feature data is respectively clustered by K-means clustering (Reference: I. B. Mohammad, D. Usman. Standardization and its effects on k-means clustering algorithm [J]. Research Journal of Applied Sciences, Engineering and Technology, 2013, 6(17): 3299-3303), to obtain a set of centroids for each type of feature data. The feature data extracted from the sample will be calculated for its distance to different centroids of the corresponding type, and the Euclidean distance calculation formula (as follows) is used for distance calculation, t represents the coordinate dimension of points M and N.

[0115]

[0116] The centroid with the shortest distance is selected, and the length of the vector from the centroid to the current feature point in different dimensions represents the similarity of the data and the centroid in different dimensions. The longer the projection length in one dimension, the lower the similarity of the feature in that dimension, and vice versa. In the system, the projection length 0.4 is taken as the threshold, and the elements in the feature vector corresponding to the features in the dimension with a projection length greater than 0.4 are set to 0, otherwise set to 1, so as to avoid the influence of part of the weakly related features on the subsequent prediction task, and also simplify the calculation amount in the deep learning process.

[0117] (4) The multi-modal sleep emotion detection layer performs:

[0118] The feature vectors generated by the feature vector generation layer are independently sent to the initial network layer composed of multiple DNN networks, and the multi-modal sleep emotion detection layer adopts a front fusion method to fuse the deep features, and outputs the judgment result of the subject's sleep emotion, and the stimulation mode and parameters for regulating the subject's sleep.

[0119] Through the above hybrid deep neural network model, the sleep emotion of the subject can be judged in real time, and the stimulation mode and parameters for regulating the patient's sleep are obtained.

[0120] The judgment of the sleep emotion of the subject through the above hybrid deep neural network model has the following characteristics:

[0121] The embodiment is based on the existing multi-modal DNN (MDNN) model, and part of the features are spliced in the feature vector generation layer through a fuzzy algorithm to generate a feature vector (i.e. fuzzy integral is added),

[0122] There are two special cases when splicing features in the feature vector generation layer:

[0123] (1) When there is an interactive effect between modalities: the combined effect between two modalities is not simply equal to the sum of the effects of single modalities

[0124] (2) When the classification result is not clear: the classification result is not clear, which means that a sample cannot be clearly determined as a positive class or a negative class.

[0125] At this time, we can use a fuzzy integral fusion device (such as Figure 5 ) to effectively fuse the multi-modal results.

[0126] Compared with existing fusion operators, a prominent advantage of fuzzy integration is that it can better reflect the importance of the classifiers corresponding to each modality, and can better represent the interaction between each modality. There are three forms of interaction between general classifiers Di and Dj:

[0127] (1) Negative synergy: If the overall importance of the sub-classifiers Di, Dj is less than the sum of the individual sub-classifiers. In other words, compared with the single classifier Di, Dj, the fusion classifier Ci, Cj cannot improve the classification accuracy of a certain class.

[0128] (2) Positive synergy: If the overall significance of the sub-classifiers Di, Dj to classification is greater than the sum of the individual classifiers. In other words, compared with the single classifier Di, Dj, the fusion classifier Ci, Cj can improve the classification accuracy of a certain class.

[0129] (3) Independency: That is, the influence of the individual classifiers Di, Dj on the classification accuracy is added.

[0130] When there is an interaction between different modalities, the embodiment utilizes the fuzzy measure to represent it. As can be seen from the following Choquet fuzzy integral formula, f is the membership degree of the modality, that is, the performance ability of different modalities on the classifier, multiplied by its measure, and finally summed to obtain the score of the input in this class.

[0131] Let X be an arbitrary set, that is, μ is a fuzzy measure defined on the set X, and f: X→[0, 1] is a non-negative real-valued measurable function defined on X, which sorts the values of the function f according to , and the minimum value of a can be 0, A i represents the set of a i . The Choquet fuzzy integral formula is as follows:

[0132]

[0133] As can be seen from the following fuzzy measure table, if the input is three modalities, the measure not only contains the measure value of all single modalities, but also contains the double modality combination and three modality combination measure. The interaction between the modalities is represented in the measure while representing the importance of the single modality. The fuzzy integral mainly represents the interaction between different modalities through the calculation of the measure. Therefore, the embodiment models the multi-modality fusion by means of the fuzzy system, and utilizes the measure to represent the importance and interaction between the multi-modalities at the same time, and finds the optimal measure calculation method for multi-modality fusion.

[0134] Table 2 Three-modality fuzzy measure table

[0135]

[0136] In summary, the fuzzy integral added in this embodiment makes it possible to consider the interaction between different data on the basis of two data analysis methods, i.e., the “independent unit system” and “pre-fusion”, because the data are related. The multi-modal measure is formed by weighting and summing the single-modal measure through the lifting degree (Choquet fuzzy integral), and then the multi-modal multi-classifier decision fusion framework is constructed according to the fuzzy measure calculation result. This framework trains and optimizes multiple different types of emotion state basic classifiers for each modality, so as to obtain the decision information of multiple basic classifiers on different modal data. Then, the newly designed multi-basic classifier decision information integration method is used to integrate and calculate the decision information of each modality, providing input information for the subsequent fuzzy measure calculation. This is much better than the existing MDNN model.

[0137] Among them, the “independent unit system” means that “electroencephalogram”, “electromyogram” and “electrocardiogram” are independent, non-interfering and unit systems (because an electrocardiogram contains many contents, such as frequency and waveform, and a premature atrial contraction is F wave and atrial fibrillation is f wave). The “pre-fusion” means that different feature extraction algorithms and classification algorithms are fused in the feature extraction and classification stage of the data set to obtain better data analysis results. The post-fusion algorithm means that multiple classifiers are fused after the feature extraction and classification stage to obtain better classification results.

[0138] In the actual model training and detection process, it is not necessarily possible to ensure that all basic information and sleep sign data can be completely collected. The hybrid deep neural network model in this embodiment also has a proper solution for this situation. Specifically as follows:

[0139] ①Training scheme when feature extraction is incomplete

[0140] The hybrid deep neural network model of this embodiment adopts a multi-modal feature input structure. Each DNN network in the initial network layer is independent of each other and is fused only in the final network layer. This makes it possible to train each independent DNN network in the initial network layer one by one. In the model training process, the 0 vector is not completed for the feature information that fails to be extracted, and the successfully extracted feature vectors of multiple types are input into the corresponding DNN network of the initial network layer for training one by one. Finally, the final network layer is trained on the basis of the training of the entire initial network layer. Such a training strategy makes the system maintain the feature independence to the greatest extent while also reducing the dependence on the simultaneous extraction of all features, and maximizes the use of all extracted feature information.

[0141] ②Test scheme when feature extraction is incomplete

[0142] In the case that the basic information or sleep physical data of the detected subject is not complete, the 0 vector is used for completion, and then all feature vectors are input into the mixed deep neural network model for detection. Because the final output of the initial network corresponding to the 0 vector input only calculates the bias term, the input of the 0 vector has little effect on the final classification result, thereby ensuring the accuracy of the overall detection result.

[0143] III. Multi-sensory stimulation intervention reinforcement device

[0144] The multi-sensory stimulation intervention reinforcement device can be realized by the prior art. As a preferred mode, the multi-sensory stimulation intervention reinforcement device of the embodiment comprises the following components:

[0145] 1. VR visual stimulation device: using VR, the visual stimulation, such as teaching film, emotional video, etc. that has been seen in the waking time, is played to the sleeper under the premise of not affecting sleep, so as to achieve the effect of enhancing related memory.

[0146] 2. Auditory stimulation device (which can be an external speaker or an earphone player): used to provide soothing, pleasant or stimulating music, sound or language to the user. The device selects audio content of different types, styles, rhythms and volumes.

[0147] 3. Tactile stimulation device:

[0148] The specific form can be:

[0149] (1) wearable glove type, with multi-point stimulation modules in the glove;

[0150] (2) foot massage instrument;

[0151] (3) mattress integrated massage instrument;

[0152] used to provide soft, comfortable or stimulating tactile sensations to the user. The device can adjust the tactile stimulation of different parts, intensity, frequency and mode.

[0153] 4. Olfactory stimulation device (which can be a room aromatherapy machine): used to provide fresh, fragrant or exciting smells to the user. The device can select olfactory stimulation of different types, concentrations and durations.

[0154] 5. Treatment room configuration device: including temperature adjustment system (central air conditioner) and air humidity control device (room humidifier).

[0155] When the sleep cabin system of the embodiment works, the following features can be further included:

[0156] 1. Selection of opportunity (sleep stage):

[0157] Human sleep can be divided into two parts: rapid eye movement (REM) sleep and non-rapid eye movement (NREM) sleep.

[0158] In combination with the above theory, it is a better choice to regulate the sleep and memory of the patient during the rapid eye movement (REM) sleep, or a more significant treatment effect will be achieved. In this embodiment, the sleep emotion multi-module monitoring device can determine when the patient enters the rapid eye movement sleep stage, so that the VR multi-sensory stimulation is performed by the multi-sensory stimulation intervention reinforcement device in this stage.

[0159] 2. Selection of parameters of the multi-sensory stimulation intervention reinforcement module:

[0160] The specific stimulation parameters will be determined according to the determination result of the data discrimination and decision-making device, and the best stimulation mode and parameters are sought to bring better treatment effect to the patient.

[0161] The parameter selection range of various stimulation modes is as follows:

[0162] Tactile stimulation: hand massage: 0.5-2 times / S, force: gentle;

[0163] Foot massage: 0.5-2 times / S, force: gentle to moderately strong;

[0164] Auditory stimulation: sound intensity: 20-40 decibels, sound quality: beautiful and soft melody;

[0165] Olfactory stimulation: comfortable and pleasant smell, such as light and elegant flower fragrance, sweet fruit fragrance, and fresh grass fragrance;

[0166] Indoor environment temperature: 18-25 degrees Celsius;

[0167] Indoor environment humidity: generally suitable humidity 45-65%, relative humidity 40-80% during summer cooling, 30-60% during winter heating, suitable humidity 45-50% for the elderly and children, and suitable humidity 40-50% for patients with respiratory system diseases such as asthma.

[0168] As can be seen from the above embodiments, the present application provides a sleep cabin device which can monitor the sleep sign data of the subject in the sleep process in real time, and determine the emotion of the subject in the sleep process, the stimulation model and the stimulation parameters needed to be received, and then provide appropriate stimulation in the sleep process of the subject. The present application can regulate the emotion memory in sleep of the depression patients (which can also be used for ordinary people and people with sleep disorders), eliminate negative memory, enhance positive memory, improve sleep, adjust emotion, relieve depression symptoms, and improve life quality, which has good application prospect.

Claims

1. A sleep pod for mood memory modulation, characterized in that, The method comprises the following steps: a sleep bed for a subject to sleep; a sleep emotion multi-module monitoring device for monitoring sleep physical data of the subject in real time; a data discrimination and decision-making device integrated with a decision-making operation unit for obtaining a stimulation mode and parameters for regulating the sleep of the subject according to the sleep physical data; a multi-sensory stimulation intervention reinforcement device for stimulating the subject according to the stimulation mode and parameters to regulate the sleep; wherein the decision-making operation unit comprises: an information receiving port configured to receive the sleep physical data of the subject in real time; a central computer processing system configured to input the sleep physical data into a machine learning model, obtain a judgment result of the sleep emotion of the subject and the stimulation mode and parameters for regulating the sleep of the subject through the machine learning model; and the machine learning model is a hybrid deep neural network model; an information sending port configured to output the judgment result of the sleep emotion of the subject, the stimulation mode and the parameters; the hybrid deep neural network model comprises: an original data extraction layer for decoding, decompiling and disassembling the input original data and converting them into a data type that is more readable and easier to understand; a feature extraction layer for extracting features from the processing results of the original data extraction layer; a feature vector generation layer for generating a plurality of feature vectors by using different algorithms for the existing feature and the discrete feature, respectively, wherein part of the features are spliced by a fuzzy algorithm to generate the feature vector together; a multi-modal sleep emotion detection layer for obtaining the judgment result of the sleep emotion of the subject and the stimulation mode and parameters for regulating the sleep of the subject according to the feature vector; wherein the multi-modal sleep emotion detection layer comprises an initial network layer and a final network layer, the initial network layer comprises a plurality of DNN networks that are not connected to each other, and the feature vectors are independently sent into the initial network layer; the final network layer comprises a DNN network structure; the last layer of each DNN network in the initial network layer is fully connected with the first layer of the DNN network in the final network layer; the multi-modal sleep emotion detection layer adopts a front fusion manner to fuse the deep features, and the DNN network structure of the final network layer outputs the judgment result of the sleep emotion of the subject and the stimulation mode and parameters for regulating the sleep of the subject.

2. The sleep pod of claim 1, wherein: The sleep emotion multi-module monitoring device comprises at least one of the following devices: a contact multi-channel sleep monitor, an optical fiber sensing sleep monitoring device, and a smart glove integrated with a sensor.

3. The sleep pod of claim 1, wherein: The multi-sensory stimulation intervention reinforcement device comprises at least one of the following devices: a VR visual stimulation device, an auditory stimulation device, a tactile stimulation device, an olfactory stimulation device, a temperature regulation system, and an air humidity control device.

4. The sleep pod of claim 3, wherein: The multi-sensory stimulation intervention reinforcement device is configured to stimulate the subject when the subject enters the rapid eye movement sleep stage.

5. The sleep pod of claim 1, wherein: The sleep physical data comprises at least one of the following indicators: electroencephalogram, electrocardiogram, heart rate, blood pressure, FVC, FEV1, FEV1 / FVC, electromyogram, and body temperature. And / or, the central computer processing system will also input the basic information of the patient into the machine learning model, and the basic information includes at least one of the following indicators: age, gender, height, weight, waist circumference, smoking history, drinking history, disease history / hospitalization history / accident, operation history, blood transfusion history, allergy history, long-term medication, family genetic disease; And / or, the stimulation mode includes at least one of the following stimulation methods: tactile stimulation, temperature sensation stimulation, auditory stimulation, olfactory stimulation, air humidity, transcranial direct current stimulation.

6. The sleep pod of claim 1, wherein: In the initial network layer, each independent DNN network is composed of an input layer and several hidden layers, in the DNN network, the neurons in each layer are not connected to each other, each layer of neurons is only connected to the adjacent front and rear layers, and the layers are fully connected structures; In the final network layer, the DNN network structure is composed of a fusion layer, several hidden layers and an output layer, and the output layer has only one neuron responsible for outputting the analysis result.

7. The sleep pod of claim 1, wherein: In the feature vector generation layer, the existing feature vector generation method is: All existing features extracted by the feature extraction layer form an existing feature database; whether the corresponding features of each sample exist in the feature database is queried in turn, if they exist, the feature is assigned a value of 1, otherwise, a value of 0; The discrete feature vector generation method is: The feature data is normalized; the normalized data is clustered by K-means to obtain a group of centroids of each type of feature data; the feature data extracted from the sample is calculated to the distance between different centroids of the corresponding type; the shortest centroid is selected, and the length of the vector from the centroid to the current feature point in different dimensions indicates the similarity of the data and the centroid in different dimensions; the projection length 0.4 is taken as the threshold, and the elements in the feature vector corresponding to the features in the dimensions with projection length greater than 0.4 are set to 0, otherwise, they are set to 1.

8. The sleep pod of claim 1, wherein: The feature vector generation layer extracts a feature vector for each discrete feature, and splices all existing features to generate a feature vector.

9. The sleep pod of claim 1, wherein: In the feature vector generation layer, for the incomplete features, a 0 vector is used for completion.

Images