Intelligent accompanying method and emotion monitoring device

Abstract

The invention discloses an intelligent accompanying method and an emotion monitoring device, which are applied to an emotion monitoring intelligent bed, and the method comprises the steps: obtaining emotion data of a user, the emotion data comprising voice features, expression features and posture change features; judging different emotion data to obtain different emotional states, and obtaining a target emotional state of the user according to the different emotional states and preset weight information of the different emotional states; when the target emotional state is an abnormal state, sending out a first feedback signal; and according to the first feedback signal, guiding the user to perform voice interaction through a semantic analysis model, and outputting an audio reply corresponding to the user voice information. According to the method, the emotion state of the user can be detected and fed back in real time, the emotion of the user is adjusted in a voice dialogue communication mode, and the nursing quality of the user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart beds, and in particular to a smart accompanying method and a mood monitoring device. Background Art

[0002] In modern nursing, bedridden patients not only face physical discomfort due to being confined to bed for a long time, but also often face psychological problems such as loneliness, anxiety, and depression. These problems may aggravate the condition and affect the recovery process. Therefore, designing a smart bed to alleviate the user's discomfort is an important direction of the current development of smart bed technology.

[0003] Smart beds in the prior art usually focus on the patient's physical care, such as mattress pressure adjustment, body position rotation, vital sign monitoring, etc. Although these functions can effectively relieve the patient's physical discomfort, they do not pay enough attention to the patient's mental health, cannot monitor and feedback the patient's psychological changes in real time, and do not provide an effective emotional counseling mechanism, ignoring the psychological problems that patients may have during long-term bed rest. Summary of the invention

[0004] The embodiments of the present application disclose an intelligent nursing method and an emotion monitoring device, which detect and feedback the user's emotional state in real time, so as to regulate the user's emotions through voice interaction, improve the quality of nursing care for the user, and thus enhance the user's life experience.

[0005] A first aspect of an embodiment of the present application discloses an intelligent accompanying method, which is applied to an emotion monitoring intelligent bed, wherein the intelligent bed is provided with an emotion monitoring device, and the method comprises:

[0006] Acquire the user's emotional data, which includes voice features, facial expression features, and body shape change features; judge different emotional data respectively to obtain different emotional states, and obtain the user's target emotional state according to different emotional states and preset weight information of different emotional states, which includes normal state and abnormal state;

[0007] When the target emotional state is in an abnormal state, sending a first feedback signal;

[0008] According to the first feedback signal, the user is guided to perform voice interaction through the semantic analysis model, and an audio response corresponding to the user's voice information is output.

[0009] As an optional implementation manner, in the first aspect of this embodiment, different emotion data are judged respectively to obtain different emotion states, including:

[0010] Collect the user's voice information and obtain voice features based on the voice information. The voice features include Mel frequency cepstral coefficients, pitch, energy, zero crossing rate, and speech speed.

[0011] The speech features are input into the emotion classification model to obtain the emotion category, and the first emotion state is obtained according to the emotion category.

[0012] As an optional implementation manner, in the first aspect of this embodiment, collecting voice information of the user and obtaining voice features according to the voice information includes:

[0013] Converting voice information into digital voice information;

[0014] According to the digital voice information, obtaining the digital voice information after noise removal processing;

[0015] Acquire first digital information having a frequency greater than a threshold frequency in the digital voice information after noise removal, and increase a preset frequency on the basis of the frequency of the first digital information to acquire second digital information;

[0016] The speech feature is obtained according to the second digital information and the third digital information whose frequency is less than the threshold frequency in the digital speech information after noise removal.

[0017] As an optional implementation, in the first aspect of this embodiment, the smart bed is further provided with a camera device to judge different emotional data respectively to obtain different emotional states, and further includes:

[0018] Collecting a user's facial video through a camera device;

[0019] Obtaining facial key areas based on facial videos;

[0020] Obtain facial key points in facial key areas, and obtain expression features based on the facial key points, wherein the expression features include motion information of the facial key points;

[0021] The motion information is input into the emotion classification model to obtain the emotion category, and the second emotion state is obtained according to the emotion category.

[0022] As an optional implementation manner, in the first aspect of this embodiment, different emotion data are judged respectively to obtain different emotion states, and further includes:

[0023] Collecting pressure data on the smart bed, and obtaining first pressure data after noise removal according to the pressure data;

[0024] Compressing the value of the first pressure data into a preset range to obtain second pressure data;

[0025] converting the second pressure data into time series data;

[0026] Obtaining posture change characteristics according to the time series data, the posture change characteristics including pressure distribution information, pressure change information and pressure change frequency information;

[0027] The posture change features are input into the emotion classification model to obtain the emotion category, and the third emotion state is obtained according to the emotion category.

[0028] As an optional implementation, in the first aspect of this embodiment, the semantic model includes a first semantic model, and according to the first feedback signal, the user is guided to perform voice interaction through the semantic analysis model, and an audio response corresponding to the user voice information is output, including:

[0029] In response to the first feedback signal, outputting first voice information according to the first semantic model, where the first voice information is used to inquire about the user's mood at the moment;

[0030] When receiving the second voice information fed back by the user, analyzing the second voice information according to the first semantic model to obtain the semantic content of the second voice information;

[0031] The corresponding third voice information is generated according to the semantic content and outputted. The third voice information is used to feed back the second voice information of the user.

[0032] As an optional implementation manner, in the first aspect of this embodiment, after sending the first voice information according to the first semantic model in response to the first feedback signal, the method further includes:

[0033] In the case where the second voice message fed back by the user is not received within the first preset time period, a fourth voice message is sent, and the fourth voice message is used to guide the user to communicate.

[0034] As an optional implementation manner, in the first aspect of this embodiment, the semantic model includes a second semantic model, and in response to the first feedback signal, after the first voice information is sent according to the first semantic model, the method further includes:

[0035] Obtaining speech information density within a second preset time length according to the speech feature, determining whether the speech information density is greater than a set threshold, and the second preset time length is greater than the first preset time length;

[0036] When the voice information density is greater than a set threshold, sending a second feedback signal;

[0037] In response to the second feedback signal, voice interaction is performed with the user through a second semantic model, and an audio reply corresponding to the user's voice information is output, and the second semantic model is different from the first semantic model.

[0038] As an optional implementation manner, in the first aspect of this embodiment, the method further includes:

[0039] When the voice information density is less than a set threshold, sending a third feedback signal;

[0040] In response to the third feedback signal, a fifth voice message is sent; the fifth voice message is used to remind the user to select a target working mode of the emotion monitoring device, the target working mode being meditation music broadcasting, short story broadcasting or soothing music broadcasting.

[0041] A second aspect of an embodiment of the present application discloses an emotion monitoring device, including an emotion perception module, an emotion judgment module, and an intelligent voice interaction system, wherein the emotion judgment module is connected to the emotion perception module and is also connected to the intelligent voice interaction system, wherein;

[0042] The emotion perception module is used to obtain the user's emotion data and send the emotion data to the emotion judgment module. The emotion data includes voice features, expression features, and body shape change features;

[0043] An emotion judgment module is used to judge different emotion data respectively to obtain different emotion states; and

[0044] for obtaining a target emotional state of a user according to different emotional states and preset weight information of different emotional states, where the emotional state includes a normal state and an abnormal state; and,

[0045] Used to send a first feedback signal when the target emotional state is in an abnormal state;

[0046] The intelligent voice interaction system is used to guide the user to perform voice interaction through a semantic analysis model according to a first feedback signal, and output an audio response corresponding to the user's voice information.

[0047] Compared with the related art, the embodiments of the present application include at least the following beneficial effects:

[0048] The intelligent accompanying method disclosed in the embodiment of the present application is applied to an emotion monitoring intelligent bed, which is provided with an emotion monitoring device, and the method includes: obtaining the user's emotion data, the emotion data including voice features, expression features and body change features; judging different emotion data respectively, obtaining different emotion states, and obtaining the user's target emotion state according to different emotion states and preset weight information of different emotion states; when the target emotion state is an abnormal state, issuing a first feedback signal; according to the first feedback signal, guiding the user to perform voice interaction through a semantic analysis model, and outputting an audio response corresponding to the user's voice information. The method can detect and feedback the patient's emotional state in real time, so as to adjust the patient's emotion through voice dialogue communication, adjust the user's emotion through voice interaction, improve the quality of care for the user, and thus improve the user's life experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of an intelligent accompanying method provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of an intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application;

[0053] Figure 4 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application;

[0054] Figure 5 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application;

[0055] Figure 6 A flowchart of another intelligent accompanying method provided in an embodiment of the present application;

[0056] Figure 7 A flowchart of another intelligent accompanying method provided in an embodiment of the present application;

[0057] Figure 8 A flowchart of another intelligent accompanying method provided in an embodiment of the present application;

[0058] Fig. 9 A schematic diagram of an emotion detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are used to distinguish similar or different objects, and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0061] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0062] In modern nursing, long-term bedridden patients not only have to face physical discomfort caused by lack of exercise and position changes, such as bedsores and muscle atrophy, but also often face psychological problems such as loneliness, anxiety and depression caused by long-term confinement in bed. Studies have shown that during bed rest, patients lack interaction and emotional support with the outside world, which can easily lead to negative emotions. If these emotions are not channeled and intervened in time, they may lead to emotional deterioration and affect physical health.

[0063] Existing smart bed technologies mainly focus on the physical needs of patients, such as automatically adjusting mattress pressure, helping patients change their body positions, monitoring vital signs, etc., to reduce physical discomfort of patients in bed and prevent some common complications. These technologies provide patients with a more comfortable nursing experience to some extent, but most smart beds currently fail to effectively pay attention to the patient's psychological state. This results in nursing staff often being unable to detect problems such as depression or anxiety in patients in a timely manner, missing the best time for psychological intervention.

[0064] The embodiments of the present application disclose an intelligent nursing method and an emotion monitoring device, which detect and feedback the user's emotional state in real time, so as to regulate the user's emotions through voice interaction, improve the quality of care for the user, and thus enhance the user's life experience. The following are detailed descriptions of each.

[0065] The intelligent accompanying method disclosed in the embodiment of the present application can detect and feedback the user's emotional state in real time, and can be applied to a variety of scenarios, such as hospitals, clinics, nursing homes, home care, schools, and enterprises.

[0066] Among them, in hospital and clinic scenarios, the intelligent accompanying method and emotion monitoring device disclosed in this application can be applied to intensive care units, psychiatric wards, postoperative recovery and other places. In nursing homes and home care scenarios, the elderly are prone to loneliness, anxiety, depression and other psychological problems due to the decline of physiological functions, especially those who are bedridden or living alone for a long time. The intelligent accompanying method and emotion monitoring device disclosed in this application can pay attention to the emotional state of the elderly in real time, and provide real-time feedback to soothe the elderly's emotions and alleviate the psychological problems of the elderly. In the school environment, especially for middle school and college students, the academic pressure, social problems and self-cognition confusion faced by students often lead to emotional problems, such as anxiety, depression and excessive stress. The intelligent accompanying method and emotion monitoring device disclosed in this application can timely detect the emotional state of students, and timely feedback to soothe students' emotions. Similarly, in the corporate environment, especially in a high-pressure, high-intensity work environment, emotional fluctuations may lead to a decrease in production efficiency or employee turnover. The intelligent accompanying method and emotion monitoring device disclosed in this application can provide timely psychological counseling for employees, help employees maintain mental health, and thus improve overall work efficiency.

[0067] See also Figure 1 , Figure 1 A flowchart of an intelligent accompanying method provided in an embodiment of the present application is applied to an emotion monitoring smart bed, where the smart bed is provided with an emotion monitoring device. The flowchart includes at least the following steps S101-S102:

[0068] Step S101: Acquire the user's emotion data, which includes voice features, expression features, and body shape change features;

[0069] Optionally, the emotion data may also include physiological data and sleep data, which is not specifically limited here.

[0070] The physiological data may include heart rate data, skin galvanic response data, respiratory rate data, blood pressure data, body temperature data, and brain wave data, which are not specifically limited here. The sleep data may include sleep time, sleep cycle, sleep interruption, sleep quality, and other data, which are not specifically limited here.

[0071] In this step, by setting up an emotion monitoring device on the smart bed and obtaining different types of emotion data of the user in real time, the user's emotional state can be comprehensively evaluated from multiple dimensions, which can greatly improve the accuracy and effectiveness of emotion monitoring.

[0072] Step S102: judging different emotion data respectively to obtain different emotion states;

[0073] In this step, different types of data are judged independently to identify different emotional states. This process can distinguish the change patterns of different emotional data, help to fully capture the subtle changes in user emotions, and effectively avoid emotion recognition errors caused by inaccurate judgment of a single dimension of data.

[0074] Step S103: obtaining the user's target emotional state according to different emotional states and preset weight information of different emotional states;

[0075] Among them, emotional states include normal states and abnormal states;

[0076] The preset weight information is obtained by technicians through multiple experiments. It is mainly aimed at the distribution ratio of different emotional data to the accuracy of user emotion judgment. When the preset weight information is distributed on different types of emotional data, more accurate emotional state monitoring results can be obtained. For example, in some cases, voice data may be more sensitive to judging anxious emotions, while body data may be more effective in judging angry emotions. In this way, the system can adjust the weights of different emotional data in a targeted manner, thereby improving the overall emotion monitoring accuracy and making the assessment of emotional state more in line with the user's actual emotional expression.

[0077] Step S104: when the target emotional state is abnormal, sending a first feedback signal;

[0078] Optionally, the first feedback signal may include an emotion regulation instruction issued by the system to the emotion monitoring device module, and the emotion monitoring device proceeds to the next step according to the emotion regulation instruction. The first feedback signal may also include a simple reminder, prompt or emotion regulation suggestion voice signal, the purpose of which is to make the user aware of the existence of emotional problems and stimulate them to adjust or interact further.

[0079] In this step, when the emotion monitoring device detects that the target emotional state is abnormal, it can send out a first feedback signal in time. This feedback signal serves as a warning, reminding the user that there may be problems with their emotional state, thereby providing conditions for subsequent intervention and support. For example, the monitoring device identifies that the user's emotional fluctuations are abnormal, and promptly provides feedback to the user, helping him or her to realize the current emotional problems and provide early warning for adjusting emotions.

[0080] Step S105: According to the first feedback signal, the user is guided to perform voice interaction through the semantic analysis model, and an audio response corresponding to the user's voice information is output.

[0081] In this step, when an abnormality is detected, the emotion monitoring device can guide the user to perform voice interaction based on the first feedback signal through the semantic analysis model and output the corresponding audio response. This feedback method can provide a more personalized and emotional response based on the user's actual emotional needs. For example, when the user is monitored to be in an anxious state, the system can provide suggestions for soothing emotions through voice interaction, or help the user relieve emotional stress through a relaxed and pleasant tone. This voice interaction method can not only enhance the affinity of the system, but also effectively enhance the user's emotional relief experience.

[0082] The emotion data that can be obtained by the emotion monitoring device include voice features; it can be understood that the smart bed is provided with a microphone sensor, and the emotion monitoring device collects the user's voice information through the built-in microphone sensor, and then obtains the voice features through the voice information.

[0083] The emotion data acquired by the emotion monitoring device also includes expression features. It can be understood that the smart bed has a built-in camera, and the emotion monitoring device collects the user's facial video through the camera, and then acquires facial expression features based on the facial video.

[0084] The emotion data acquired by the emotion monitoring device also includes body posture change characteristics. It can be understood that a motion sensor is provided on the smart mattress, and the emotion monitoring device acquires the user's body posture change characteristics by acquiring pressure data on the smart mattress.

[0085] Optionally, the emotion monitoring device can also obtain physiological characteristics. It is understandable that the smart bed can be provided with a variety of sensors, and the emotion monitoring device obtains physiological characteristics through a variety of sensors. For example, the heart rate data is obtained by embedding a photoplethysmography sensor into the mattress of the smart bed, the skin electrical response data is obtained by equipping the smart bed or the wearable device with a skin electrical sensor, the breathing rate data is obtained by building a pressure sensor or an airflow sensor into the mattress or pillow of the smart bed, the pressure data is obtained by setting a vibration sensor or an airbag sensor on the mattress, the body temperature data is obtained by setting an infrared sensor or a contact sensor on the mattress or pillow of the smart bed, and the brain wave data is obtained by integrating an electroencephalogram sensor on the mattress or pillow of the smart bed.

[0086] Optionally, the emotion monitoring device can also obtain sleep characteristics, which can be obtained by monitoring the user's sleep behavior and physiological characteristics. It can be understood that the smart bed can be equipped with multiple sensors, and the emotion monitoring device obtains sleep characteristics through the multiple sensors and sleep monitoring algorithms. For example, by setting an acceleration sensor and a heart rate sensor on the smart bed and combining a sleep monitoring algorithm to obtain sleep time, sleep cycle and sleep interruption, by setting multiple sensors on the smart bed to obtain physiological characteristics, and obtaining sleep quality based on the physiological characteristics and the sleep monitoring algorithm.

[0087] For example, first of all, emotional data collection is the basic link of the whole process. The system collects the user's emotional data in real time through various sensors, cameras, microphones and other devices. Emotional data can include three dimensions: voice features, expression features and body change features. Voice features include intonation, speaking speed, volume, tone changes, etc., which can reflect the user's emotional fluctuations; expression features reflect subtle changes in emotions through changes in facial expressions, such as anger, joy, anxiety, etc.; body change features reflect the user's emotional state through the user's body posture, movements, body language, etc. For example, when nervous, it may manifest as tight shoulders, restless hands and feet, etc.

[0088] Secondly, after collecting these multi-dimensional emotional data, the emotional state is judged. The emotion monitoring device will analyze and integrate the voice features, expression features and body change features through the preset emotion recognition model, and judge the emotional state represented by these different features. For example, if the user's speech speed is faster and the tone is higher, it may indicate that the user is in a state of tension or anxiety; if the user's facial expression shows a frown or the corners of the mouth droop, it may indicate that the user is depressed; if the user's body posture is curled up or avoiding, it may indicate some kind of uneasiness or fear. Through the analysis of these features, the emotion monitoring device will comprehensively judge the user's emotional state, which is divided into two categories: "normal state" and "abnormal state". By combining the preset weight information corresponding to each emotional state, the user's target emotional state is obtained. When the system determines that the user's target emotional state is an abnormal state, the system will send a first feedback signal. The function of this signal is to remind the user that his current emotion is in an unstable or unhealthy state and needs guidance or intervention.

[0089] Finally, based on the first feedback signal, the system will further conduct voice interaction guidance. Through the semantic analysis model, the system will have an in-depth understanding of the user's voice input, generate corresponding emotion guidance content, and send the content in the form of an audio response to conduct personalized voice communication with the user. This audio response usually converts the feedback content into a natural, easy-to-understand voice form through speech synthesis technology. The audio response may include warm comfort, suggestions for emotion regulation techniques, and even some motivational positive language. Therefore, in the process of this accompanying method, through multi-dimensional emotion data collection, accurate emotion judgment, intelligent feedback mechanism and voice interaction guidance, a closed-loop emotion regulation system is formed, which can provide users with accurate emotional support in real-time and dynamic emotional changes, help them regulate negative emotions, improve emotional stability, and promote emotional health.

[0090] Optionally, the semantic analysis model can help the machine understand the contextual relationships, implicit meanings and various semantic levels in the language. There are many types of semantic analysis models, which can be selected according to actual needs. For example, rule-based semantic analysis models, vector space-based semantic analysis models, context-based pre-trained semantic analysis models, graph-based semantic analysis models, deep learning-based semantic analysis models, and multimodal semantic analysis models, etc., are not specifically limited here.

[0091] Among them, the rule-based semantic analysis model usually uses manually defined rules and dictionaries to perform semantic analysis, such as word sense disambiguation and grammatical semantic combination, which are not specifically limited here.

[0092] The meanings of words and sentences in the vector space-based semantic analysis model are represented by vectors, which can usually be modeled using a bag-of-words model or a TF-IDF method, and no specific limitations are made here.

[0093] The context-based pre-trained language model is pre-trained on a large-scale corpus and can better capture the contextual information of words, sentences and paragraphs. For example, the bidirectional encoder representation transformer (Bidirectional Encoder Representations from Transformers, BERT), the generative pretrained transformer (Generative Pretrained Transformer, GPT) and the text-to-text transfer transformer (Text-to-Text Transfer Transformer, T5), etc., without specific restrictions here.

[0094] Graph-based semantic analysis models usually use graph neural networks for analysis, such as knowledge graphs, etc., which are not specifically limited here.

[0095] There are also many semantic analysis models based on deep learning, which can be selected according to actual needs, such as recurrent neural networks, long short-term memory, and gated recurrent units, etc., and no specific restrictions are made here.

[0096] The multimodal semantic analysis model integrates information from different modalities for joint analysis, such as the Contrastive Language-Image Pretraining (CLIP) model and DALL·E. There is no specific restriction here and you can choose according to actual needs.

[0097] In one embodiment, when the emotion monitoring device detects that the user's target state is abnormal, the emotion monitoring device may also send information about the user's abnormal emotions to the terminal device of the user's relatives or caregivers to remind the user's relatives or caregivers to pay attention to the user's emotional state in a timely manner and take corresponding measures to ease the user's emotions.

[0098] In one embodiment, the emotion monitoring device also uses password login, encryption protocol during data transmission, and confidential data storage to protect the user's privacy. For example, the user's relatives or caregivers need to use password login to obtain authentication of the emotion monitoring device before they can obtain the user's emotional state data. For another example, when the user's relatives or caregivers want to obtain the user's emotional data, the emotion monitoring device encrypts the data through TSL or SSL encryption protocol before transmitting it to ensure that the data transmission between the emotion monitoring device and the user's relatives or caregivers will not be obtained or tampered with by others. For another example, the data stored on the emotion monitoring device uses a storage encryption algorithm, such as AES, to ensure that even if a data leak occurs, the stored emotional data content cannot be read by a third party, thereby protecting the user's privacy security.

[0099] In some embodiments, for the method in step S102: judging different emotion data respectively to obtain different emotion states may be:

[0100] Collect the user's voice information and obtain voice features based on the voice information. The voice features include Mel frequency cepstral coefficients, pitch, energy, zero crossing rate, and speech speed.

[0101] The speech features are input into the emotion classification model to obtain the emotion category, and the first emotion state is obtained according to the emotion category.

[0102] To understand the above method more clearly, see Figure 2 , Figure 2 A flowchart of an intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application, the flowchart at least includes the following steps S201-S202:

[0103] Step S201: Collect the user's voice information and obtain voice features based on the voice information.

[0104] Among them, speech features include Mel-frequency cepstral coefficients, pitch, energy, zero-crossing rate, and speech rate;

[0105] Mel-frequency cepstral coefficients are one of the most commonly used features in speech signal processing and are used to extract the audio characteristics of speech. It simulates the perceptual characteristics of human hearing and converts the spectrum into a Mel scale, which is more consistent with the sensitivity of the human ear to different frequencies. It can capture the speaker's voice characteristics (such as timbre, intonation, sound quality, etc.). In emotion recognition, since emotions affect the pitch and timbre of the voice, such as anger, happiness or tension, it can be used to distinguish different emotional states.

[0106] Pitch refers to the height of a sound, and is usually determined by frequency. When a person speaks, the frequency of the vibration of the vocal cords determines the pitch. Changes in pitch can reflect fluctuations in emotions. For example, when angry or anxious, a person's pitch tends to rise, while when relaxed or contemplative, the pitch may be lower. In emotion recognition, changes in pitch can help the system determine whether the user's emotions are in an excited or nervous state.

[0107] Energy refers to the intensity or amplitude of a speech signal over a certain period of time. It reflects the volume of a speaker's voice and usually changes with different emotions. For example, when angry, people's voices are usually more intense and have higher energy, while when sad or depressed, the voice energy is lower. Energy changes can also help distinguish the intensity and type of emotions.

[0108] The zero crossing rate refers to the number of times the speech signal waveform passes through the zero point, that is, the number of times the signal switches from the positive half cycle to the negative half cycle or from the negative half cycle to the positive half cycle. It is often used to describe the roughness of the signal, that is, the clarity of the speech. A higher zero crossing rate may indicate that the sound has more high-frequency components, such as a tense, excited or angry tone; a lower zero crossing rate may be associated with a calm, relaxed state.

[0109] Speech rate refers to the number of syllables or words spoken per unit time. The speed of speech is directly related to the speaker's psychological and emotional state. Generally speaking, when people are nervous, anxious or excited, their speech speed will increase; when they are calm or sad, their speech speed may be slower. Changes in speech rate are important signals for emotional regulation and are particularly effective in identifying emotions such as anxiety, anger, and excitement.

[0110] Step S202: input the speech features into the emotion classification model to obtain the emotion category, and obtain the first emotional state according to the emotion category.

[0111] The sentiment classification model can be a traditional machine learning model, which can be selected according to actual needs, for example, Support Vector Machine (SVM), Decision Trees, Random Forest, K-Nearest Neighbor (KNN), Naive Bayes, etc., without specific limitation here.

[0112] The emotion categories that can be obtained include happiness, sadness, anger, fear, surprise, disgust, anxiety, confidence, depression, excitement, frustration, calmness, etc., without specific restrictions here.

[0113] According to the positive and negative emotions represented by these emotion categories, the first emotion state is obtained as normal or abnormal. For example, when the emotion category obtained is positive emotion joy, the first emotion state obtained according to the emotion category is a normal state. When the emotion category obtained is negative emotion anxiety, the first emotion state obtained according to the emotion category is an abnormal state.

[0114] In some embodiments, the collected original speech signal can also be input into a deep learning model to obtain the emotion category, and the deep model can be selected according to actual needs. For example, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Neural Networks (DNN) and Transformer models of self-attention mechanism can be selected according to actual needs.

[0115] In one embodiment, for the method in step S201: collecting the user's voice information and obtaining the voice features according to the voice information, it can be:

[0116] Converting voice information into digital voice information;

[0117] According to the digital voice information, obtaining the digital voice information after noise removal processing;

[0118] Acquire first digital information having a frequency greater than a threshold frequency in the digital voice information after noise removal, and increase a preset frequency on the basis of the frequency of the first digital information to acquire second digital information;

[0119] The speech feature is obtained according to the second digital information and the third digital information whose frequency is less than the threshold frequency in the digital speech information after noise removal.

[0120] To understand the above method more clearly, see Figure 3 , Figure 3 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application, the flowchart at least includes the following steps S301-S305:

[0121] Step S301: Collect the user's voice information and convert the voice information into digital voice information;

[0122] Step S302: obtaining digital voice information after noise removal according to the digital voice information;

[0123] Step S303: acquiring first digital information having a frequency greater than a threshold frequency in the digital voice information after noise removal, and adding a preset frequency to the frequency of the first voice signal to acquire second digital information;

[0124] Step S304: obtaining speech features according to the second digital information and the third digital information whose frequency is less than the threshold frequency in the digital speech information after noise removal.

[0125] Step S305: input the speech features into the emotion classification model to obtain the emotion category, and obtain the first emotional state according to the emotion category.

[0126] In this method, clearer digital voice information is obtained by removing noise from the digital voice information, and digital voice information with clearer human voice can be obtained by increasing the high-frequency features of the digital voice information, that is, first digital information with a frequency greater than a threshold frequency in the digital voice information is obtained, and a preset frequency is added to the frequency of the first voice signal to obtain second digital information. This is because the mid- and high-frequency components in the voice information usually contain more detailed information, such as changes in timbre and tone, thereby improving the intelligibility and clarity of the voice, which facilitates the subsequent extraction of voice features.

[0127] In some embodiments, the smart bed is further provided with a camera, through which the user's facial video information is collected. For the method in step S102: different emotional data are judged respectively to obtain different emotional states, which can be:

[0128] Collecting a user's facial video through a camera device;

[0129] Obtaining facial key areas based on facial videos;

[0130] Obtain facial key points in facial key areas, and obtain facial expression features based on the facial key points, wherein the facial expression features include motion information of the facial key points;

[0131] The motion information is input into the emotion classification model to obtain the emotion category, and the second emotion state is obtained according to the emotion category.

[0132] To understand the above method more clearly, see Figure 4 , Figure 4 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application, the flowchart at least includes the following steps S401-S404:

[0133] Step S401: Capturing a user's facial video through a camera device;

[0134] Step S402: Obtaining facial key areas according to the facial video;

[0135] Step S403: Obtain facial key points of the facial key area, and obtain facial expression features according to the facial key points;

[0136] Among them, facial expression features include motion information of facial key points;

[0137] Step S404: input the motion information into the emotion classification model to obtain the emotion category, and obtain the second emotional state according to the emotion category.

[0138] In this method, first, a camera device is needed to capture the user's facial video. This video is the basis of sentiment analysis and provides the dynamic changes of the user's facial expressions. The collection of facial videos usually includes multiple continuous frames, through which the evolution of the user's facial expressions on the timeline can be captured. Secondly, the key facial areas are obtained based on the facial video, including the eyes, mouth, eyebrows, nose and other main parts. In this process, computer vision algorithms can be used to locate and frame the key facial areas of the user, such as (Haar Cascade, MTCNN, Dlib, MediaPipe, etc. Next, the facial key points of the key facial areas are obtained, which can be the pupil center of the eye, the upper and lower lip positions of the mouth, the starting point of the eyebrows, etc. These key points are usually obtained through a facial calibration model, such as 68 facial calibration points of Dlib, 468 calibration points provided by MediaPipe Face Mesh, etc. Finally, by tracking the position changes of these key points, they are input into the emotion classification model to analyze the changes in facial expressions. For example, the upward movement of the corners of the mouth may indicate a smile, and the downward movement of the eyebrows may indicate anger or confusion.

[0139] In some embodiments, for the method in step S102: judging different emotion data respectively to obtain different emotion states may also be:

[0140] Collecting pressure data on the smart bed, and obtaining first pressure data after noise removal according to the pressure data;

[0141] Compressing the value of the first pressure data into a preset range to obtain second pressure data;

[0142] converting the second pressure data into time series data;

[0143] Obtaining posture change characteristics according to the time series data, the posture change characteristics including pressure distribution information, pressure change information and pressure change frequency information;

[0144] The posture change features are input into the emotion classification model to obtain the emotion category, and the third emotion state is obtained according to the emotion category.

[0145] To understand the above method more clearly, see Figure 5 , Figure 5 A flowchart of another intelligent accompanying method for obtaining emotional state provided in an embodiment of the present application, the flowchart at least includes the following steps S501-S505:

[0146] Step S501: collecting pressure data on the smart bed, and obtaining first pressure data after noise removal according to the pressure data;

[0147] Step S502: compressing the value of the first pressure data into a preset range to obtain second pressure data;

[0148] Step S503: converting the second pressure data into time series data;

[0149] Step S504: obtaining body posture change characteristics according to the time series data;

[0150] Among them, the body posture change characteristics include pressure distribution information, pressure change information and pressure change frequency information;

[0151] Step S505: input the posture change characteristics into the emotion classification model to obtain the emotion category, and obtain the third emotional state according to the emotion category.

[0152] In this method, the smart bed is equipped with multiple pressure sensors, which can be arranged in the mattress to detect the user's weight and body position. The pressure sensor can capture the user's movement on the bed, changes in body position, and pressure distribution in different parts in real time. The pressure data output by these pressure sensors is usually a pressure value within a certain time interval, indicating the pressure intensity at different positions. After obtaining the pressure data through the pressure sensor, in order to make the obtained pressure data more accurate and meaningful, it is usually necessary to remove the noise caused by environmental factors, equipment errors or technical limitations of the sensor itself through a filtering algorithm (such as low-pass filtering, Kalman filtering, etc.), so as to obtain the first pressure data after noise removal. Then, in order to facilitate subsequent processing and analysis, the value of the first pressure data needs to be compressed or normalized within a preset range to obtain the second pressure data. This preset range is usually a standard range (such as 0 to 1, or a specific pressure intensity interval), so that they can be compared within the same scale to avoid data inconsistency due to differences in sensitivity of different sensors. Then, the second pressure data is converted into time series data, which takes into account the time sequence and change law of the data, so that it is more meaningful to obtain the user status. Finally, pressure distribution information, pressure change information, and pressure frequency change information are obtained based on the time series data. These posture change data are input into the emotion classification model to obtain the emotion category, thereby obtaining the third emotional state. For example, based on the pressure frequency change information, it can be determined whether the user frequently adjusts his posture within a period of time. If it is detected that the user frequently adjusts his posture within a period of time, it indicates that the user feels uncomfortable or anxious.

[0153] In some embodiments, the above-mentioned process steps of obtaining the first emotional state, the second emotional state, and the third emotional state can be monitored and obtained simultaneously in the emotion monitoring device, and two emotional states of any three emotional states can be monitored, or one emotional state of any three emotional states can be monitored. The user can choose according to actual needs. It can be understood that when monitoring multiple emotional states at the same time, the emotion monitoring device consumes more energy, and when monitoring fewer types of emotional states, the emotion monitoring device is more energy-efficient.

[0154] In some embodiments, the fourth emotional state may also be acquired based on physiological data, and the fifth emotional state may be acquired based on sleep data, without specific limitation herein.

[0155] In some embodiments, the semantic model includes a first semantic model. For the method in step S105: according to the first feedback signal, guiding the user to perform voice interaction through the semantic analysis model, and outputting an audio response corresponding to the user's voice information can be:

[0156] In response to the first feedback signal, outputting first voice information according to the first semantic model, where the first voice information is used to inquire about the user's mood at the moment;

[0157] When receiving the second voice information fed back by the user, analyzing the second voice information according to the first semantic model to obtain the semantic content of the second voice information;

[0158] The corresponding third voice information is generated according to the semantic content and outputted. The third voice information is used to feed back the second voice information of the user.

[0159] For a clearer understanding of the above methods, please refer to Figure 6 , Figure 6 A flowchart of another intelligent accompanying method provided in an embodiment of the present application, the flowchart at least includes the following steps S601-S607:

[0160] Step S601: Acquire the user's emotion data, which includes voice features, expression features, and body shape change features;

[0161] Optionally, the emotion data may also include physiological data and sleep data, which is not specifically limited here.

[0162] The physiological data may include heart rate data, skin galvanic response data, respiratory rate data, blood pressure data, body temperature data, and brain wave data, which are not specifically limited here. The sleep data may include sleep time, sleep cycle, sleep interruption, sleep quality, and other data, which are not specifically limited here.

[0163] Step S602: judging different emotion data respectively to obtain different emotion states;

[0164] Step S603: obtaining the user's target emotional state according to different emotional states and preset weight information of different emotional states;

[0165] Among them, emotional states include normal states and abnormal states;

[0166] The preset weight information is obtained by technical personnel through multiple experiments, and is mainly the distribution ratio of different emotional data to the accuracy of user emotional judgment. When the preset weight information is distributed to different types of emotional data, more accurate emotional state monitoring results can be obtained.

[0167] Step S604: When the target emotional state is abnormal, a first feedback signal is issued;

[0168] Optionally, the first feedback signal may include an emotion regulation instruction issued by the system to the emotion monitoring device module, and the emotion monitoring device performs the next step according to the emotion regulation instruction;

[0169] Optionally, the first feedback signal may also include a simple reminder, prompt or voice signal of emotion regulation suggestion, the purpose of which is to make the user aware of the existence of emotional problems and motivate him to adjust or further interact.

[0170] Step S605: in response to the first feedback signal, outputting first voice information according to the first semantic model, where the first voice information is used to inquire about the user's mood at the moment;

[0171] Step S606: when receiving the second voice information fed back by the user, analyzing the second voice information according to the first semantic model to obtain the semantic content of the second voice information;

[0172] Step S607: Generate corresponding third voice information according to the semantic content, output the third voice information, and the third voice information is used to feed back the user's second voice information.

[0173] In some embodiments, after step S605, that is, after sending the first voice information according to the first semantic model in response to the first feedback signal, the method further includes:

[0174] When no second voice message of user feedback is received within the first preset waiting time, a fourth voice message is sent, and the fourth voice message is used to guide user communication.

[0175] To understand the above method more clearly, see Figure 7 , Figure 7 A flowchart of another intelligent accompanying method provided in an embodiment of the present application, the flowchart at least includes the following steps S701-S706:

[0176] Step S701: Acquire the user's emotion data, which includes voice features, expression features, and body shape change features;

[0177] Optionally, the emotion data may also include physiological data and sleep data, which is not specifically limited here.

[0178] The physiological data may include heart rate data, skin galvanic response data, respiratory rate data, blood pressure data, body temperature data, and brain wave data, which are not specifically limited here. The sleep data may include sleep time, sleep cycle, sleep interruption, sleep quality, and other data, which are not specifically limited here.

[0179] Step S702: judging different emotion data respectively to obtain different emotion states;

[0180] Step S703: obtaining the user's target emotional state according to different emotional states and preset weight information of different emotional states;

[0181] Among them, emotional states include normal states and abnormal states;

[0182] The preset weight information is obtained by technical personnel through multiple experiments, and is mainly the distribution ratio of different emotional data to the accuracy of user emotional judgment. When the preset weight information is distributed to different types of emotional data, more accurate emotional state monitoring results can be obtained.

[0183] Step S704: When the target emotional state is in an abnormal state, a first feedback signal is issued;

[0184] Optionally, the first feedback signal may include an emotion regulation instruction issued by the system to the emotion monitoring device module, and the emotion monitoring device performs the next step according to the emotion regulation instruction;

[0185] Optionally, the first feedback signal may also include a simple reminder, prompt or voice signal of emotion regulation suggestion, the purpose of which is to make the user aware of the existence of emotional problems and motivate him to adjust or further interact.

[0186] Step S705: in response to the first feedback signal, outputting first voice information according to the first semantic model, where the first voice information is used to inquire about the user's mood at the moment;

[0187] Step S706: When no second voice message of user feedback is received within the first preset waiting time, a fourth voice message is sent, where the fourth voice message is used to guide user communication.

[0188] Optionally, the fourth voice message may be of a gentle and caring type, for example: "Hello, I can feel that you seem a little unhappy. If you are willing, I am always available to listen. Do you want to chat with me now?"

[0189] Optionally, the fourth voice message may also be encouraging, for example: I understand that sometimes it is not easy to speak, but I am here ready to listen. If you have anything to say, or just want to be quiet, it is all fine. ”

[0190] Optionally, the fourth voice message may also be of an emotional soothing type, for example, "If you feel uncomfortable with your current mood, you can relax and take a break. If you need any help or just need someone to accompany you, I am here."

[0191] Exemplarily, the emotion detection device detects that the user is in an abnormal mood. In this case, the emotion detection device sends a first voice message based on the voice model to ask the user about his mood at this time, with the purpose of judging the user's own emotional perception and reaction to external information at this time. If the user does not respond to external information within the first preset time, or is in a state of not wanting to talk, it indicates that the user is in an abnormal mood. Therefore, it is necessary to send a fourth voice message again to guide the user to communicate. The fourth voice message conveys care and understanding to avoid putting pressure on the user, while giving them enough space to choose whether to communicate further and respect the user's emotional needs and boundaries.

[0192] In some embodiments, the semantic model includes a second semantic model. After step S605, that is, after sending the first speech information according to the first semantic model in response to the first feedback signal, the method further includes:

[0193] Obtaining speech information density within a second preset time length according to the speech feature, determining whether the speech information density is greater than a set threshold, and the second preset time length is greater than the first preset time length;

[0194] When the voice information density is greater than a set threshold, sending a second feedback signal;

[0195] In response to the second feedback information, voice interaction is performed with the user through a second semantic model, and an audio reply corresponding to the user's voice information is output, and the second semantic model is different from the first semantic model.

[0196] In some embodiments, after obtaining the voice information density within the second preset time length according to the voice feature and determining whether the voice information density is greater than a set threshold, if the voice information density is less than the set threshold, sending a third feedback signal;

[0197] In response to the third feedback signal, a fifth voice message is sent; the fifth voice message is used to remind the user to select a target working mode of the emotion monitoring device, the target working mode being meditation music broadcasting, short story broadcasting or soothing music broadcasting.

[0198] To understand the above method more clearly, see Figure 8 , Figure 8 A flowchart of another intelligent accompanying method provided in an embodiment of the present application, the flowchart at least includes the following steps S801-S808:

[0199] Step S801: Acquire the user's emotion data, which includes voice features, expression features, and body shape change features;

[0200] Optionally, the emotion data may also include physiological data and sleep data, which is not specifically limited here.

[0201] The physiological data may include heart rate data, skin galvanic response data, respiratory rate data, blood pressure data, body temperature data, and brain wave data, which are not specifically limited here. The sleep data may include sleep time, sleep cycle, sleep interruption, sleep quality, and other data, which are not specifically limited here.

[0202] Step S802: judging different emotion data respectively to obtain different emotion states;

[0203] Step S803: obtaining the user's target emotional state according to different emotional states and preset weight information of different emotional states;

[0204] Among them, emotional states include normal states and abnormal states;

[0205] The preset weight information is obtained by technical personnel through multiple experiments, and is mainly the distribution ratio of different emotional data to the accuracy of user emotional judgment. When the preset weight information is distributed to different types of emotional data, more accurate emotional state monitoring results can be obtained.

[0206] Step S804: When the target emotional state is in an abnormal state, a first feedback signal is issued;

[0207] Optionally, the first feedback signal may include an emotion regulation instruction issued by the system to the emotion monitoring device module, and the emotion monitoring device performs the next step according to the emotion regulation instruction;

[0208] Optionally, the first feedback signal may also include a simple reminder, prompt or voice signal of emotion regulation suggestion, the purpose of which is to make the user aware of the existence of emotional problems and motivate him to adjust or further interact.

[0209] Step S805: in response to the first feedback signal, outputting first voice information according to the first semantic model, where the first voice information is used to inquire about the user's mood at the moment;

[0210] Step S806: obtaining the speech information density within the second preset time period according to the speech feature, and determining whether the speech information density is greater than a set threshold;

[0211] Step S807: When the voice information density is greater than a set threshold, a second feedback signal is sent, and in response to the second feedback information, voice interaction is performed with the user through a second semantic model, and an audio response corresponding to the user's voice information is output.

[0212] Step S808: When the voice information density is less than the set threshold, a third feedback signal is sent, and in response to the third feedback signal, fifth voice information is sent;

[0213] Among them, the fifth voice information is used to remind the user to select the target working mode of the emotion monitoring device, and the target working mode is meditation music broadcasting, short story broadcasting or soothing music broadcasting.

[0214] Optionally, the target working mode may also be broadcasting of hilarious short clips, interesting questions and answers, etc., without specific limitation here.

[0215] Exemplarily, after the emotion detection device determines that the target emotional state of the user is abnormal, and the emotion detection device sends the first voice information according to the first semantic model, it waits until the second preset time, obtains the voice information density of the user received during the second preset time, and determines whether the obtained voice information density is greater than the set threshold, and the second preset time is greater than the first preset time. This is because it is possible to determine whether the user's desire to confide is strong by judging the voice information density received by the emotion detection device after a longer time interval.

[0216] When the voice information density is greater than the set threshold, it can be determined that the user has a strong desire to talk. According to the user's strong desire to talk, the emotion detection device can call the second semantic model to communicate with the user through the second semantic model. The content logic tendency generated by the second semantic model is different from that of the first semantic model. It can be understood that the first semantic model focuses on guiding users to communicate, while the second semantic model focuses on communicating with users in an interesting and more relaxed way. Therefore, the second semantic model can have the characteristics of humor and emotional perception, diversity, randomness, creativity and interactivity, and high language fluency.

[0217] When the voice information density is less than the set threshold, it can be determined that the user has no desire to talk at this time, that is, the state of not wanting to talk very much. Therefore, at this time, the emotion detection device sends a third feedback signal, and the emotion detection device sends a fifth information according to the third feedback signal. The fifth information user reminds the user to select the target working mode of the emotion detection device. After the user selects the target working mode, when the selected target working mode is meditation music broadcasting, the emotion detection device can broadcast meditation music according to the target working mode to help the user relax, improve concentration, improve mood, promote sleep, and enhance mental health. Or, when the selected target working mode is a short story broadcast, the emotion detection device can bring a pleasant emotional experience to the user according to the relaxed entertainment method of the short story broadcast, and promote mental health, language development, emotional resonance, creativity and cognitive ability in many aspects. In addition, a lot of mental growth and emotional comfort can be harvested unconsciously. Or, when the target working mode is selected to be soothing music broadcasting, the emotion detection device also broadcasts soothing music broadcasting to help users relieve stress and relieve emotions.

[0218] In this method, different emotion soothing modes are adopted by judging the density of voice information, which can provide users with more emotionally appropriate feedback, help them better regulate their emotions and psychological states, and thus achieve more effective emotion soothing effects.

[0219] In some embodiments, the mood monitoring device is also provided with an online search function, and the user can select the meditation music track, or the story name, or the soothing music name to be broadcasted according to his or her own preferences. For example, the user can tell the mood monitoring device the meditation music track he or she wants to hear through voice expression, and after the mood monitoring device receives the voice information, it searches for the meditation music track through the online search function, and after searching for the meditation music, it plays the meditation music.

[0220] In some embodiments, the emotion monitoring device can monitor the user's emotions in real time during the process of soothing the user's emotions, and provide real-time feedback of the monitoring results to correct the user's emotional state. If the user's emotional state is normal for a period of time, the emotion monitoring device can remind the user by voice whether to take a break. After receiving the user's feedback, the emotion monitoring device can continue to soothe the user's emotions according to the user's feedback content, for example, if the user chooses not to take a break. If the user chooses to take a break, the emotion monitoring device stops working.

[0221] The present application also discloses an emotion detection device. Fig. 9 , Fig. 9A schematic diagram of an emotion detection device disclosed in an embodiment of the present application, comprising an emotion perception module 91, an emotion judgment module 92 and an intelligent voice interaction system 93, wherein the emotion judgment module 92 is connected to the emotion perception module 91 and is also connected to the intelligent voice interaction system 93, wherein;

[0222] The emotion perception module 91 is used to obtain the user's emotion data and send the emotion data to the emotion judgment module. The emotion data includes voice features, expression features and body shape change features;

[0223] The emotion judgment module 92 is used to judge different emotion data respectively to obtain different emotion states; and

[0224] for obtaining a target emotional state of a user according to different emotional states and preset weight information of different emotional states, where the emotional state includes a normal state and an abnormal state; and,

[0225] Used to send a first feedback signal when the target emotional state is in an abnormal state;

[0226] The intelligent voice interaction system 93 is used to guide the user to perform voice interaction through a semantic analysis model according to the first feedback signal, and output an audio response corresponding to the user's voice information.

[0227] According to the above-mentioned intelligent accompanying method, an embodiment of the present application provides an electronic device, including: a memory and a processor, the memory stores a computer program that can be run on the processor, and the processor implements any of the above-mentioned visual system positioning detection algorithms when executing the computer program.

[0228] According to the above-mentioned intelligent accompanying method, an embodiment of the present application also provides a computer program product, including a computer program, which implements any of the above-mentioned visual system positioning detection algorithms when executed by a processor.

[0229] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a ROM, etc.

[0230] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Suitable nonvolatile memory may include ROM, programmable ROM (Programmable ROM, PROM), erasable PROM (Erasable PROM, EPROM), electrically erasable PROM (Electrically Erasable PROM, EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. By way of illustration and not limitation, RAM may be in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM) and direct memory bus dynamic RAM (DRDRAM).

[0231] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0232] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0233] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0236] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0237] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0238] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0239] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0240] The above is a detailed introduction to the intelligent accompanying method disclosed in the embodiment of the present application. The principles and implementation methods of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those of ordinary skill in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An intelligent accompanying method, characterized in that: Applied to an emotion monitoring smart bed, the smart bed is provided with an emotion monitoring device, and the method comprises: Acquire the user's emotional data, the emotional data including voice features, facial expression features, and body shape change features; judge different emotional data respectively to obtain different emotional states, and obtain the user's target emotional state according to the different emotional states and preset weight information of the different emotional states, the emotional state including a normal state and an abnormal state; When the target emotional state is in an abnormal state, sending a first feedback signal; According to the first feedback signal, the user is guided to perform voice interaction through a semantic analysis model, and an audio response corresponding to the user's voice information is output.

2. The method according to claim 1, characterized in that Different emotional data are judged respectively to obtain different emotional states, including: Collecting voice information of the user, and obtaining the voice features according to the voice information, wherein the voice features include Mel frequency cepstral coefficients, pitch, energy, zero crossing rate, and speech speed; The speech feature is input into a sentiment classification model to obtain a sentiment category, and a first emotional state is obtained according to the sentiment category.

3. The method according to claim 2, characterized in that Collecting the voice information of the user and obtaining voice features according to the voice information includes: Converting the voice information into digital voice information; According to the digital voice information, obtaining the digital voice information after noise removal processing; Acquire first digital information having a frequency greater than a threshold frequency from the digital voice information after noise removal, and increase a preset frequency on the basis of the frequency of the first digital information to acquire second digital information; The speech feature is obtained according to the second digital information and the third digital information whose frequency is less than the threshold frequency in the digital speech information after noise removal.

4. The method according to claim 2, characterized in that: The smart bed is also provided with a camera device, and the different emotional data are judged respectively to obtain different emotional states, and further includes: Collecting a facial video of the user by using the camera device; Obtaining facial key areas according to the facial video; Obtaining facial key points of the facial key area, and obtaining the expression features according to the facial key points, wherein the expression features include motion information of the facial key points; The motion information is input into the emotion classification model to obtain the emotion category, and a second emotion state is obtained according to the emotion category.

5. The method according to claim 2, characterized in that: The judging of different emotional data respectively to obtain different emotional states also includes: Collecting pressure data of the smart bed, and obtaining first pressure data after noise removal according to the pressure data; Compressing the value of the first pressure data into a preset range to obtain second pressure data; converting the second pressure data into time series data; Obtaining the body posture change characteristics according to the time series data, wherein the body posture change characteristics include pressure distribution information, pressure change information, and pressure change frequency information; The posture change feature is input into the emotion classification model to obtain an emotion category, and a third emotional state is obtained according to the emotion category.

6. The method according to claim 1, characterized in that The semantic model includes a first semantic model, and the step of guiding the user to perform voice interaction through a semantic analysis model according to the first feedback signal and outputting an audio response corresponding to the user voice information includes: In response to the first feedback signal, outputting first voice information according to the first semantic model, wherein the first voice information is used to inquire about the user's mood at this time; When receiving the second voice information fed back by the user, analyzing the second voice information according to the first semantic model to obtain the semantic content of the second voice information; Corresponding third voice information is generated according to the semantic content, and the third voice information is output, where the third voice information is used to feed back the second voice information to the user.

7. The method according to claim 6, characterized in that After sending the first voice information according to the first semantic model in response to the first feedback signal, the method further includes: In the case where the second voice message fed back by the user is not received within a first preset time period, a fourth voice message is sent, and the fourth voice message is used to guide the user to communicate.

8. The method according to claim 7, characterized in that The semantic model includes a second semantic model, and after the first speech information is sent out according to the first semantic model in response to the first feedback signal, the method further includes: Obtaining speech information density within a second preset time length according to the speech feature, and determining whether the speech information density is greater than a set threshold, and the second preset time length is greater than the first preset time length; When the voice information density is greater than the set threshold, sending a second feedback signal; In response to the second feedback signal, voice interaction is performed with the user through the second semantic model, and an audio reply corresponding to the user voice information is output, and the second semantic model is different from the first semantic model.

9. The method according to claim 8, characterized in that The method further comprises: When the voice information density is less than the set threshold, sending a third feedback signal; In response to the third feedback signal, a fifth voice message is issued; the fifth voice message is used to remind the user to select a target working mode of the emotion monitoring device, wherein the target working mode is meditation music broadcasting, short story broadcasting or soothing music broadcasting.

10. An emotion monitoring device, characterized in that: It includes an emotion perception module, an emotion judgment module and an intelligent voice interaction system, wherein the emotion judgment module is connected to the emotion perception module and is also connected to the intelligent voice interaction system, wherein; The emotion perception module is used to obtain the user's emotion data and send the emotion data to the emotion judgment module, wherein the emotion data includes voice features, expression features and body shape change features; The emotion judgment module is used to judge different emotion data respectively to obtain different emotion states; as well as, for obtaining a target emotional state of the user according to different emotional states and preset weight information of different emotional states, wherein the emotional state includes a normal state and an abnormal state; as well as, for sending a first feedback signal when the target emotional state is in an abnormal state; The intelligent voice interaction system is used to guide the user to perform voice interaction through a semantic analysis model according to the first feedback signal, and output an audio response corresponding to the user's voice information.

Images