Emotion management method and device, equipment and storage medium

Abstract

The invention discloses an emotion management method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The invention provides a new medical mode, that is, emotion management is carried out on the user online. Specifically, under the guidance of basic theories of traditional Chinese medicine, emotion evaluation is carried out on the user based on multi-modal data collected by wearable equipment worn by the user, and the online emotion evaluation mode enables the user to carry out self-service evaluation on emotion problems anytime and anywhere, so that the user experience is improved, and the user experience is improved. And the prevention efficiency of the emotion problem is effectively improved. Besides, the mode can output an emotion evaluation report and early warning information based on the emotion evaluation result, so that the user can timely and detailedly know own emotion problems, and the man-machine interaction efficiency is high. Besides, the mode can also recommend personalized emotion intervention suggestions to the user based on the emotion evaluation result and the personal physique of the user, so that the user can conveniently carry out self-service intervention on emotion problems in time.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an emotion management method, device, equipment and storage medium. Background Art

[0002] Emotional problems, also known as emotional illness, have gradually developed into a global public health issue. In the full-course management model of emotional problems, timely identification and intervention can achieve better prevention and treatment effects.

[0003] Currently, the identification and intervention of emotional problems mainly rely on users to go to the hospital for medical treatment offline. However, this traditional medical model not only has strict time and space limitations, but also has the problem of delayed treatment due to users not seeking medical treatment in time.

[0004] Based on the above description, it can be seen that a new medical model is urgently needed to manage users' emotions in a timely manner, thereby facilitating the identification and intervention of patients' emotional problems. Summary of the invention

[0005] The embodiment of the present application provides an emotion management method, device, equipment and storage medium. The technical solution is as follows:

[0006] In one aspect, a method for managing emotions is provided, the method comprising:

[0007] Acquire multimodal data collected by a wearable device worn by a target object; wherein the multimodal data includes physiological indicator data, audio data, and environmental data of the current environment of the target object;

[0008] Using basic theoretical knowledge of traditional Chinese medicine as a reference, emotion recognition is performed on the target object based on the multimodal data to obtain an emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object;

[0009] Based on the emotion recognition result, an emotion assessment report and warning information are output; wherein the warning information is used to indicate the severity of the emotional problem of the target object;

[0010] Based on the emotion recognition result and the physical characteristics of the target object, emotion intervention suggestions in the field of traditional Chinese medicine are output.

[0011] In a possible implementation, the method further includes:

[0012] Obtaining input data of the target object during the human-computer dialogue process;

[0013] Acquiring electronic health data of the target object;

[0014] Based on the emotion recognition result, the electronic health data and the knowledge base, response data matching the input data is output.

[0015] In another possible implementation, performing emotion recognition on the target object based on the multimodal data to obtain an emotion recognition result includes:

[0016] Based on the multimodal data and the input data, emotion recognition is performed on the target object to obtain the emotion recognition result.

[0017] In another possible implementation, the process of establishing the knowledge base includes:

[0018] Determine the knowledge scope of the field of traditional Chinese medicine, extract key concepts within the knowledge scope; analyze the semantic associations between the extracted key concepts, establish semantic association triples based on the analyzed semantic associations; and construct a traditional Chinese medicine field ontology within the knowledge scope;

[0019] Establish a literature database in the field of traditional Chinese medicine; wherein the literature database in the field of traditional Chinese medicine includes a plurality of documents, and the plurality of documents are used to treat emotional problems based on knowledge in the field of traditional Chinese medicine.

[0020] In another possible implementation, the acquiring of multimodal data collected by a wearable device worn by the target object includes at least one of the following:

[0021] Acquire the physiological sign data of the target object according to a first rule; wherein the first rule is used to instruct acquisition of physiological sign data related to TCM diagnosis;

[0022] Acquire the physiological sign data of the target object according to a second rule; wherein the second rule is used to instruct acquisition of physiological sign data affected by emotional changes.

[0023] In another possible implementation, the physiological indicator data of the target object includes at least one of the following:

[0024] the target subject's heart rate;

[0025] The heart rate variability of the target object; wherein the heart rate variability refers to the change in the difference between successive heartbeat cycles;

[0026] the sleep quality of the target subject;

[0027] the activity level of the target subject;

[0028] The physiological stress response of the target object; wherein the physiological stress response is used to measure the physical stress level of the target object;

[0029] the body temperature of the target subject;

[0030] the target object's posture and head movement;

[0031] The target subject's breathing rate.

[0032] In another possible implementation, outputting emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition result and the physical characteristics of the target object includes:

[0033] Acquire scene requirement information of the target object;

[0034] Based on the emotion recognition result and the physical characteristics of the target object, the emotion intervention suggestion is output in an output manner that matches the scenario requirement information.

[0035] In another possible implementation, the outputting the emotion intervention suggestion based on the emotion recognition result and the physical characteristics of the target object in an output manner matching the scenario requirement information includes:

[0036] Based on the emotion recognition result, an emotion intervention suggestion consistent with the TCM constitution type of the target object is output in an output manner that matches the scenario requirement information.

[0037] In another possible implementation, outputting an emotion assessment report based on the emotion recognition result includes:

[0038] Based on the emotion change trend included in the emotion recognition result, the emotion fluctuation record of the target object is output.

[0039] In another possible implementation, outputting the target object's emotion fluctuation record based on the target object's emotion change trend includes:

[0040] Obtaining the historical emotion change trend of the target object;

[0041] Based on the historical emotion change trend and the emotion change trend included in the emotion recognition result, the daily emotion fluctuation record of the target object in multiple time statistical dimensions is output in the form of a calendar.

[0042] In another possible implementation, when the severity of the emotional problem of the target object is above a preset threshold, the method further includes:

[0043] outputting follow-up information to the target object at intervals, and obtaining feedback data from the target object during each follow-up;

[0044] Based on the feedback data obtained during each follow-up, the emotional intervention suggestions for the target object are adjusted until the severity of the emotional problem of the target object is below the preset threshold.

[0045] In another possible implementation, adjusting the emotion intervention suggestion for the target object based on the feedback data obtained during each follow-up includes:

[0046] For any follow-up process, the feedback data obtained in the follow-up process and the multimodal data obtained in the first time range are input into the emotion recognition model, and emotion recognition is performed again on the target object based on the emotion recognition model;

[0047] Based on the emotion recognition results obtained during this follow-up and the physical characteristics of the target object, adjusting the emotion intervention suggestions for the target object;

[0048] The first time range refers to the period between the last follow-up process and the current follow-up process.

[0049] In another possible implementation, the emotion recognition of the target object is performed based on the multimodal data with reference to the basic theoretical knowledge of traditional Chinese medicine to obtain the emotion recognition result, including:

[0050] The multimodal data is input into an emotion recognition model, and emotion recognition is performed on the target object based on the emotion recognition model to obtain the emotion recognition result; wherein the emotion recognition model is an artificial intelligence model that integrates basic theoretical knowledge of traditional Chinese medicine; the training process of the emotion recognition model includes:

[0051] Constructing a sample database; wherein the sample database includes clinically collected TCM diagnosis and treatment cases, symptom information extracted from clinical trials and experimental studies of emotional problems, and multimodal data of sample subjects; the TCM diagnosis and treatment cases are related to emotional problems;

[0052] The artificial intelligence model is trained based on the sample database to obtain the emotion recognition model.

[0053] In another aspect, an emotion management device is provided, the device comprising:

[0054] An acquisition module is configured to acquire multimodal data collected by a wearable device worn by a target object; wherein the multimodal data includes physiological indicator data, audio data, and environmental data of the current environment of the target object;

[0055] A processing module is configured to use basic theoretical knowledge of traditional Chinese medicine as a reference, perform emotion recognition on the target object based on the multimodal data, and obtain an emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object;

[0056] An output module is configured to output an emotion assessment report and warning information based on the emotion recognition result; wherein the warning information is used to indicate the severity of the emotional problem of the target object;

[0057] The output module is further configured to output emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition results and the physical characteristics of the target object.

[0058] In a possible implementation, the acquisition module is further configured to acquire input data of the target object during the human-computer dialogue process;

[0059] The acquisition module is further configured to acquire the electronic health data of the target object;

[0060] The output module is further configured to output response data matching the input data based on the emotion recognition result, the electronic health data and the knowledge base.

[0061] In another possible implementation, the processing module is configured to perform emotion recognition on the target object based on the multimodal data and the input data to obtain the emotion recognition result.

[0062] In another possible implementation, the process of establishing the knowledge base includes:

[0063] Determine the knowledge scope of the field of traditional Chinese medicine, extract key concepts within the knowledge scope; analyze the semantic associations between the extracted key concepts, establish semantic association triples based on the analyzed semantic associations; and construct a traditional Chinese medicine field ontology within the knowledge scope;

[0064] Establish a literature database in the field of traditional Chinese medicine; wherein the literature database in the field of traditional Chinese medicine includes a plurality of documents, and the plurality of documents are used to treat emotional problems based on knowledge in the field of traditional Chinese medicine.

[0065] In another possible implementation, the acquisition module is configured to perform at least one of the following:

[0066] Acquire the physiological sign data of the target object according to a first rule; wherein the first rule is used to instruct acquisition of physiological sign data related to TCM diagnosis;

[0067] Acquire the physiological sign data of the target object according to a second rule; wherein the second rule is used to instruct acquisition of physiological sign data affected by emotional changes.

[0068] In another possible implementation, the physiological indicator data of the target object includes at least one of the following:

[0069] the target subject's heart rate;

[0070] The heart rate variability of the target object; wherein the heart rate variability refers to the change in the difference between successive heartbeat cycles;

[0071] the sleep quality of the target subject;

[0072] the activity level of the target subject;

[0073] The physiological stress response of the target object; wherein the physiological stress response is used to measure the physical stress level of the target object;

[0074] the body temperature of the target subject;

[0075] the target object's posture and head movement;

[0076] The target subject's breathing rate.

[0077] In another possible implementation, the output module is configured to obtain scenario requirement information of the target object; and output the emotion intervention suggestion in an output manner matching the scenario requirement information based on the emotion recognition result and the physical characteristics of the target object.

[0078] In another possible implementation, the output module is configured to output emotion intervention suggestions that are consistent with the TCM constitution type of the target object based on the emotion recognition result in an output manner that matches the scenario requirement information.

[0079] In another possible implementation, the output module is configured to output the target object's emotion fluctuation record based on the emotion change trend included in the emotion recognition result.

[0080] In another possible implementation, the output module is configured to obtain the historical emotion change trend of the target object; based on the historical emotion change trend and the emotion change trend included in the emotion recognition result, output the daily emotion fluctuation record of the target object in multiple time statistical dimensions in the form of a calendar.

[0081] In another possible implementation, the output module is further configured to output follow-up information to the target object at intervals when the severity of the emotional problem of the target object is above a preset threshold;

[0082] The acquisition module is further configured to acquire feedback data of the target subject during each follow-up;

[0083] The processing module is further configured to adjust the emotional intervention suggestions for the target object based on the feedback data obtained during each follow-up until the severity of the emotional problem of the target object is below the preset threshold.

[0084] In another possible implementation, the processing module is further configured to, for any follow-up process, input the feedback data obtained in this follow-up process and the multimodal data obtained within the first time range into the emotion recognition model, and perform emotion recognition on the target object again based on the emotion recognition model; based on the emotion recognition results obtained in this follow-up process and the physical characteristics of the target object, adjust the emotion intervention suggestions for the target object; wherein the first time range refers to the period between the last follow-up process and the current follow-up process.

[0085] In another possible implementation, the processing module is configured to input the multimodal data into an emotion recognition model, perform emotion recognition on the target object based on the emotion recognition model, and obtain the emotion recognition result; wherein the emotion recognition model is an artificial intelligence model that integrates basic theoretical knowledge of traditional Chinese medicine;

[0086] The training process of the emotion recognition model includes:

[0087] Constructing a sample database; wherein the sample database includes clinically collected TCM diagnosis and treatment cases, symptom information extracted from clinical trials and experimental studies of emotional problems, and multimodal data of sample subjects; the TCM diagnosis and treatment cases are related to emotional problems;

[0088] The artificial intelligence model is trained based on the sample database to obtain the emotion recognition model.

[0089] On the other hand, a computer device is provided, the device comprising a processor and a memory, the memory storing at least one program code, the at least one program code being loaded and executed by the processor to implement the above-mentioned emotion management method.

[0090] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned emotion management method.

[0091] On the other hand, a computer program product or a computer program is provided, which includes a computer program code, which is stored in a computer-readable storage medium, and a processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned emotion management method.

[0092] The embodiment of the present application proposes a new medical model, that is, online emotional management of users, such as emotional assessment, early warning and identification. In detail, this method implements emotional assessment of users based on multimodal data collected by wearable devices worn by users under the guidance of basic theories of traditional Chinese medicine. This online emotional assessment method not only makes it possible for users to conduct self-service assessments of emotional problems anytime and anywhere, but also effectively improves the efficiency of preventing emotional problems. In addition, this method can also output emotional assessment reports and early warning information based on the emotional assessment results, so that users can understand the emotional problems they face in a timely and detailed manner, and the human-computer interaction efficiency is high. In addition, this method can also recommend highly personalized emotional intervention suggestions to users based on the emotional assessment results and the user's personal physique, so as to facilitate users to conduct self-service interventions for emotional problems that match their own emotional and physical conditions in a timely manner, and the human-computer interaction effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0094] Figure 1 It is a schematic diagram of an implementation environment involved in an emotion management method provided in an embodiment of the present application;

[0095] Figure 2 is a flow chart of an emotion management method provided by an embodiment of the present application;

[0096] Figure 3 is a flow chart of another emotion management method provided by an embodiment of the present application;

[0097] Figure 4 is a schematic diagram of a human-computer interaction interface provided in an embodiment of the present application;

[0098] Figure 5 is a structural schematic diagram of an emotion management device provided in an embodiment of the present application;

[0099] Figure 6It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0101] In this application, the terms "first", "second", etc. are used to distinguish the same or similar items with substantially the same role and function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0102] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first element can be referred to as a second element, and similarly, a second element can also be referred to as a first element. Both the first element and the second element can be elements, and in some cases, can be separate and different elements.

[0103] Here, at least one means one or more than one, for example, at least one element can be one element, two elements, three elements, or any other integer greater than or equal to one. And multiple means two or more than two, for example, multiple elements can be two elements, three elements, or any other integer greater than or equal to two.

[0104] The "and / or" mentioned in this article indicates that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship.

[0105] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions. For example, the user input data and user physiological characteristics data involved in this application are all obtained with full authorization.

[0106] Figure 1 It is a schematic diagram of an implementation environment involved in an emotion management method provided in an embodiment of the present application.

[0107] See also Figure 1 , the implementation environment includes: a terminal 101, a server 102 and a wearable device 103.

[0108] Among them, the terminal 101 and the wearable device 103 are located on the user side, a wireless network connection or a wired network connection is established between the terminal 101 and the server 102, and a Bluetooth connection is established between the terminal 101 and the wearable device 103. The type of the wearable device 103 can be a smart watch, a smart bracelet or a smart headset, etc., which is not limited in this application. Figure 1 The description is only made by taking the wearable device 103 as a smart headset as an example.

[0109] In one possible implementation, the emotion management method is jointly executed by the terminal 101, the server 102 and the wearable device 103. Among them, the wearable device 103 is used to intelligently collect user data based on multimodal technology, and transmit the collected multimodal data to the terminal 101 via Bluetooth or directly upload it to the cloud. In other words, in order to achieve efficient and intelligent dynamic collection and continuous monitoring of multimodal data, the embodiment of the present application adopts wearable device technology. Among them, the wearable device is integrated with a multimodal sensor, which realizes dynamic data collection by real-time monitoring of the user's physiological indicators. It should be noted that, considering user privacy, the data transmission and storage process is encrypted throughout to ensure the security of personal information.

[0110] In an embodiment of the present application, a user can realize online medical consultation through terminal 101, for example, a client (also called an emotion assistant) that provides an online medical consultation server is installed on terminal 101. The emotion assistant can be either an independently running parent application or a sub-application running within a parent application, and this application does not limit this. Among them, a sub-application is an application that can be used without downloading and installing. Taking the emotion assistant as the parent application as an example, the user can download the application through a mobile phone or tablet, and the server 102 is used to provide background services for the application. Taking the emotion assistant as a sub-application as an example, the user can search for the sub-application in the parent application and use it directly.

[0111] During the online consultation process, the user can input text or audio through the emotion assistant, and the terminal 101 can transmit the user's emotion evaluation request to the server 102 by requesting the server interface. After receiving the user's emotion evaluation request, the server 102 will conduct a human-computer dialogue with the user through a digital virtual person, that is, enter the TCM question-and-answer process and simulate a real TCM consultation scene.

[0112] For example, during the medical consultation, the server 102 combines the multimodal data collected by the wearable device 103 worn by the user and the user's input data during the medical consultation to evaluate the user's emotions, and outputs an emotion assessment report and corresponding emotion intervention suggestions to the user through the terminal 102.

[0113] In addition, the embodiment of the present application also supports the server 102 to periodically and automatically perform emotional assessments on the user based on the multimodal data collected by the wearable device 103 worn by the user, and output an emotional assessment report and corresponding emotional intervention suggestions to the user through the terminal 102, so as to realize automatic emotional assessment and emotional intervention for the user.

[0114] In an embodiment of the present application, after the information output by the server 102 is returned to the terminal 101, the terminal 101 will render the page accordingly and display it through the emotion assistant to achieve interaction with the user.

[0115] In addition, in addition to simulating a doctor's treatment words and answering emotion-related questions, the emotion assistant can also allow users to obtain weather, news, music and other information through voice commands. Reminders can also be set to help users manage their daily lives. This application does not limit this.

[0116] In another possible implementation, through the emotion assistant, the server 102 can provide users with emotion assessment and warning services, thereby providing users with personalized assessment reports and warning information. Among them, the emotion assessment service integrates the basic theoretical knowledge of traditional Chinese medicine, that is, the emotion assessment is completed under the guidance of the basic theory of traditional Chinese medicine. Under the guidance of the holistic view of the unity of man and nature and the unity of mind and body, the theory of emotional disease has a long history in traditional Chinese medicine, forming a complete theoretical system of theory, method, prescription and medicine. Its theoretical doctrines are constantly developing, and it has accumulated rich experience in prevention, treatment, diagnosis and rehabilitation, which has obvious advantages in the prevention and treatment of emotional problems.

[0117] In detail, TCM and its characteristic treatment methods play an important role in preventing mental and psychological diseases before they occur, treating diseases after they occur, preventing changes after they occur, and preventing recurrence after they occur. Integrating TCM theory and Chinese cultural wisdom to guide the diagnosis and treatment of emotional problems, there are relatively advanced results in the diagnosis and evidence-based research of disease-symptom-syndrome differentiation, as well as non-drug treatment methods with localized psychotherapy as the core. There are also a large number of effective treatment plans in ancient books and modern medical records that may support the clinical treatment of emotional diseases.

[0118] Based on this, the embodiment of the present application proposes a set of psychological therapies and supporting non-drug treatment programs that integrate local Chinese wisdom and are applied to the clinical guidance of the diagnosis and treatment of emotional problems. Among them, non-drug therapies guided by the concept of traditional Chinese medicine, including psychological therapy, exercise therapy, diet adjustment, and sleep rhythm adjustment, are applied to the treatment of emotional problems by means of acupoint application, massage, aromatherapy, and qigong guidance. It can not only effectively improve the patient's mood, regulate the patient's body function as a whole, reduce toxic side effects, reduce the patient's personal risks, and improve the patient's compliance with treatment, but also reduce the medical expenses and economic losses caused by the treatment of emotional diseases.

[0119] In addition, the server 102 can also provide users with human-computer dialogue services based on digital virtual humans and personalized intervention suggestion recommendation services, as described below.

[0120] The terminal 101 is a computer device with a display screen, such as a smart phone or a tablet computer, etc. The server 102 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, which is not limited in this application.

[0121] In addition, the server involved in the embodiment of the present application may also include other servers to provide more comprehensive and diversified services. In addition, those skilled in the art may know that the number of terminals may be more or less than that shown in the figure. For example, the above-mentioned terminals may be only a few, or the above-mentioned terminals may be dozens or hundreds, or a larger number, which is not limited in the present application.

[0122] The emotion management solution provided in the embodiment of the present application is introduced through the following implementation method.

[0123] Figure 2 is a flow chart of an emotion management method provided by an embodiment of the present application. The execution subject of the method is a computer device. Taking the periodic automatic emotion assessment and intervention of the user as an example, see Figure 2 , the method flow provided in the embodiment of the present application includes:

[0124] 201. A computer device obtains multimodal data collected by a wearable device worn by a target object, where the multimodal data includes physiological indicator data, audio data, and environmental data of the current environment of the target object.

[0125] In the embodiments of the present application, multimodal data is a reliable self-service diagnosis and treatment resource to support clinical decision-making. Wearable devices can collect multimodal data conveniently, efficiently and continuously, and upload the collected multimodal data to cloud storage.

[0126] In another possible implementation, the embodiment of the present application applies the method of multi-source information fusion and deep learning to form a special storage platform for multimodal data, so as to modularly collect, preliminarily analyze and classify and store the multimodal data collected by wearable devices. Exemplarily, the embodiment of the present application preliminarily classifies the multimodal data collected by wearable devices through a multi-label classification learning algorithm and a multi-label feature selection algorithm that mark local correlations. Through this step, key information can be quickly and effectively extracted from huge amounts of data to provide a targeted data basis for subsequent emotion recognition and early warning. At the same time, these preliminarily classified data are securely stored in the cloud to ensure data reliability and accessibility.

[0127] In another possible implementation, the physiological indicator data of the target object includes at least one of the following:

[0128] 1. The target subject's heart rate (HR). HR refers to the number of heart beats per minute and is an important indicator for assessing cardiovascular health. For example, a wearable device monitors the user's heart rate in real time through a built-in PPG (photoplethysmography) sensor, recording changes in resting heart rate and heart rate during exercise.

[0129] 2. The target's heart rate variability (HRV). HRV refers to the variation in the difference between heartbeats, that is, HRV refers to the natural fluctuation of the heartbeat interval, which is usually used to assess the activity of the autonomic nervous system and the ability to cope with stress. A lower HRV may indicate a higher stress level or poor recovery ability.

[0130] 3. The sleep quality of the target object can be evaluated by monitoring the user's sleep pattern, such as sleep duration, deep sleep, rapid eye movement sleep, and number of awakenings at night, etc. This application does not limit this.

[0131] 4. The activity level of the target object. The accelerometer and gyroscope can be used to monitor the user's daily activity level, such as the number of steps, exercise distance, and calories consumed, etc. This application does not limit this.

[0132] 5. Physiological stress response of the target object: Physiological stress response is used to measure the physical stress level of the target object. For example, the user's physical stress level can be evaluated by combining indicators such as heart rate, HRV, and skin conductivity.

[0133] 6. The target object's body temperature can be measured by the built-in temperature sensor inside the ear, which can be used to detect fever symptoms early or monitor body temperature changes.

[0134] 7. Posture and Head Movement of the Target Object: By monitoring the movement and posture of the user’s head, it helps to analyze the user’s daily behavior patterns and potential movement disorders.

[0135] 8. The target person’s breathing rate.

[0136] The first point that needs to be explained is that, in addition to the items listed above, physiological indicator data can also include more types, which are not limited in this application.

[0137] In another possible implementation, the audio data of the target object is obtained because, in the process of emotional state recognition, the user's emotional state, such as calmness, anxiety, sadness or excitement, can be identified by analyzing the user's voice features, such as intonation, speaking speed and changes in sound intensity.

[0138] In another possible implementation, the environmental data of the target object's current environment is the environmental noise level, because excessive environmental noise may affect the user's mental health and sleep quality.

[0139] In another possible implementation, there may be a situation where the physiological indicator data collected by the wearable device worn by the target object contains data that is not suitable for emotional assessment, early warning and intervention. In this case, in order to reduce computing pressure, the computer device can also use the following method to obtain the target object's physiological sign data, which is not limited in this application.

[0140] Method 1: Acquire the physiological sign data of the target object according to the first rule, wherein the first rule is used to instruct acquisition of physiological sign data related to traditional Chinese medicine diagnosis.

[0141] Among them, TCM diagnosis is guided by TCM theory, and mainly uses the four methods of diagnosis (observation, auscultation, questioning, and palpation) to diagnose diseases, explore the cause, location, nature, and course of the disease, identify symptoms, diagnose the disease, and provide a basis for treatment. In this way, the computer device can screen the multimodal data collected by the wearable device according to the first rule, and then obtain the physiological sign data related to TCM diagnosis from the target object. Alternatively, the wearable device can directly collect the physiological indicator data of the target object according to the first rule, and upload the collected physiological indicator data to the cloud storage, and then the computer device can directly obtain the physiological sign data related to TCM diagnosis.

[0142] Method 2: Acquire the physiological sign data of the target object according to the second rule, wherein the second rule is used to instruct to acquire the physiological sign data affected by emotional changes.

[0143] In this manner, the computer device can filter the multimodal data collected by the wearable device according to the second rule, and then obtain the physiological sign data affected by the emotional changes from the target object. Alternatively, the wearable device can directly collect the physiological indicator data of the target object according to the second rule, that is, the wearable device automatically identifies and collects the physiological indicator data affected by the emotion, and uploads the collected physiological indicator data to the cloud for storage.

[0144] The emotion assessment service integrating basic theoretical knowledge of traditional Chinese medicine is introduced through the following step 202.

[0145] 202. The computer device uses the basic theoretical knowledge of traditional Chinese medicine as a reference, performs emotion recognition on the target object based on the multimodal number of the target object, and obtains the emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object.

[0146] In a possible implementation, the basic theoretical knowledge of traditional Chinese medicine is used as a reference, and emotion recognition is performed on the target object based on the multimodal data of the target object to obtain the emotion recognition result, including:

[0147] The multimodal data of the target object is input into the emotion recognition model, and the emotion recognition of the target object is performed based on the emotion recognition model to obtain the emotion recognition result. Among them, the emotion recognition model is an artificial intelligence model that integrates the basic theoretical knowledge of traditional Chinese medicine, and the multimodal data input into the emotion recognition model is the multimodal data that has been cleaned.

[0148] In another possible implementation, the computer device inputs the multimodal data of the target object within the second time range into the emotion recognition model for emotion recognition. The second time range may be one day, three days, one week, one month, or one quarter, etc., which is not limited in this application. In addition, the computer device may also periodically input the multimodal data within the second time range into the emotion recognition model for emotion recognition, which is also not limited in this application.

[0149] Exemplarily, taking emotion recognition performed once a day as an example, the second time range may be one day, for example, the computer device inputs the multimodal data of that day into the emotion recognition model for emotion recognition; taking emotion recognition performed once every three days as an example, the second time range may be three days, for example, the computer device inputs the multimodal data of the last three days into the emotion recognition model for emotion recognition; taking emotion recognition performed once a week as an example, the second time range may be one week, for example, the computer device inputs the multimodal data of the last week into the emotion recognition model for emotion recognition.

[0150] In the embodiment of the present application, the emotion recognition model is used to provide emotion assessment services. Among them, the emotion recognition model is an artificial intelligence model that integrates the basic theoretical knowledge of traditional Chinese medicine. The model can comprehensively analyze the relationship between emotional problems and multiple factors. Exemplarily, the emotion recognition model can be trained in the following way:

[0151] First, a sample database is constructed, which includes clinically collected TCM diagnosis and treatment cases, symptom information extracted from clinical trials and experimental studies of emotional problems, and multimodal data of sample objects; among which the collected TCM diagnosis and treatment cases are related to emotional problems; then, the artificial intelligence model is trained based on the sample database to obtain an emotion recognition model.

[0152] In another possible implementation, the emotion assessment service may perform disease diagnosis prediction and symptom diagnosis prediction on the target object.

[0153] For disease diagnosis prediction, the emotion assessment service will classify and grade the user's emotions, that is, the emotion recognition results include the target object's emotion type and emotion intensity level. In addition, in order to facilitate doctors and users to understand personal emotional fluctuations, the emotion recognition results also include the target object's emotional change trend. Exemplarily, taking the second time range as one day as an example, the emotion change trend is used to indicate the target object's emotional fluctuations in various periods of the day. Taking the second time range as three days as an example, the emotion change trend is used to indicate the target object's emotional fluctuations in various periods in the last three days.

[0154] For symptom diagnosis prediction, the emotion assessment service will classify the target object based on Chinese medicine syndrome differentiation, that is, the emotion recognition results also include the target object's Chinese medicine symptoms.

[0155] In another possible implementation, the computer device can also input the multimodal data and electronic health data of the target object into the emotion recognition model for emotion recognition, which is not limited in this application. The electronic health data includes the target object's health history, medical diagnosis results, medical prescriptions, and other information related to personal health. Exemplarily, the electronic health data input into the emotion recognition model includes the target object's neuroimaging data and clinical laboratory indicators. Neuroimaging data refers to medical imaging data of the target object's head, and clinical laboratory indicators include but are not limited to biochemical indicators, blood indicators, urine indicators or immune indicators, etc., which are not limited in this application.

[0156] 203. The computer device outputs an emotion assessment report and warning information based on the emotion recognition result, and the warning information is used to indicate the severity of the emotional problem of the target object.

[0157] Among them, after completing the emotional assessment of the target object, the computer device will provide the target object with a personalized emotional assessment report and output warning information.

[0158] In a possible implementation, outputting an emotion assessment report based on the emotion recognition result includes:

[0159] Based on the emotion change trend included in the emotion recognition result, the emotion fluctuation record of the target object is output.

[0160] Exemplarily, taking the second time range as one day as an example, the emotion recognition result includes the emotion change trend of the target object on that day. Accordingly, the output emotion fluctuation record is used to reflect the emotion fluctuation of the target object in each time period of the day (for example, accurate to each hour), forming the user's mood diary, based on which the target object can view his mood changes on that day. In addition, the mood diary can also include interpretation information of the emotion fluctuations in each time period, such as interpretation of the time period with large emotion fluctuations, which is not limited in this application.

[0161] In another possible implementation, based on the emotion change trend of the target object, outputting the emotion fluctuation record of the target object includes:

[0162] Obtain the historical emotion change trend of the target object; then, based on the historical emotion change trend of the target object and the emotion change trend included in the emotion recognition result, output the target object's daily emotion fluctuation record in multiple time statistical dimensions in the form of a calendar.

[0163] The above historical emotion change trend is the emotion change trend of the target object within the third time range. The third time range is earlier than the second time range in terms of time sequence. For example, if the second time range is the current day, the third time range may be the previous few days; if the second time range is the most recent week, the third time range may be the previous few weeks; if the second time range is the most recent month, the third time range may be the previous few months.

[0164] For example, the above-mentioned multiple time statistical dimensions include weekly statistics or monthly statistics, etc., which is not limited in this application. In another possible implementation, the daily emotional fluctuation records of the target object under multiple time statistical dimensions are output in the form of a calendar, including but not limited to:

[0165] In the form of a calendar table, the target object's daily mood fluctuation records are outputted in each week covered by the second time range and the third time range. The daily mood fluctuation records are used to reflect the target object's mood fluctuations in each time period of the day (for example, accurate to every hour); or,

[0166] In the form of a calendar table, the mood fluctuation statistics are performed on a monthly basis, and the daily mood fluctuation records of the target object in each month covered by the second time range and the third time range are output.

[0167] The first point to be explained is that, in addition to the record of emotional fluctuations, the emotion assessment report also includes the attribute information of the target object, the emotion type and emotion intensity level of the target object, the explanatory information of the emotion recognition result, etc., which is not limited in this application. In addition, the emotion assessment report can be in the form of a combination of pictures and texts, which is also not limited in this application.

[0168] The second point to be explained is that the embodiment of the present application also proposes an early warning mechanism based on artificial intelligence technology, which is used to provide graded early warnings based on emotion recognition results. That is, the early warning information is used to indicate the severity of the emotional problem of the target object. Exemplarily, the principle of graded early warning is that the higher the severity of the emotional problem, the greater the probability that the early warning information will be seen by the target object and the shorter the time it takes, which is not limited in the present application.

[0169] The third point to be explained is that after completing the emotional assessment of the target object, the computer device will also output a personalized intervention plan to the target object. That is, based on the user's emotion recognition results and physical characteristics, a personalized intervention plan in the field of traditional Chinese medicine is generated for the user, and fed back to the user through the human-computer interaction system to help the user better manage his health, as detailed in the following step 204.

[0170] 204. The computer device outputs emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition results and the physical characteristics of the target object.

[0171] In a possible implementation, based on the emotion recognition results and the physical characteristics of the target object, emotion intervention suggestions in the field of traditional Chinese medicine are output, including but not limited to the following methods:

[0172] The scenario demand information of the target object is obtained; wherein the scenario demand information is used to reflect the target object's demand for the self-help intervention scenario. Next, based on the emotion recognition results and the physical characteristics of the target object, an emotion intervention suggestion in the field of traditional Chinese medicine is output in an output mode that matches the scenario demand information.

[0173] For example, during the self-help intervention process, based on the scenario demand information, virtual reality technology is used to create an emotional companion robot for the target object, and emotional intervention suggestions are output to the target object through immersive scenarios, so as to provide the target object with timely, flexible and targeted emotional companionship.

[0174] In another possible implementation, based on the emotion recognition results and the physical characteristics of the target object, an emotion intervention suggestion is output in an output manner that matches the scenario requirement information, including: based on the emotion recognition results, an emotion intervention suggestion that is consistent with the target object's TCM constitution type is output in an output manner that matches the scenario requirement information.

[0175] Exemplarily, TCM constitution types include: balanced constitution, yang deficiency constitution, yin deficiency constitution, qi deficiency constitution, phlegm-damp constitution, damp-heat constitution, blood stasis constitution, special constitution, and qi stagnation constitution, but this application does not limit this.

[0176] In another possible implementation, the emotional intervention suggestions in the field of traditional Chinese medicine include at least one of psychological intervention suggestions (psychotherapy), exercise intervention suggestions (exercise therapy), diet intervention suggestions (diet therapy) or sleep intervention suggestions (sleep therapy). Among them, psychological therapy is mainly based on cognitive growth therapy, which is a traditional Chinese medicine psychotherapy technology that is guided by the traditional Chinese medicine's cultivation of both body and spirit, with the wisdom of Confucianism, Buddhism and Taoism as the core, combined with cognitive therapy and narrative therapy to integrate and innovate.

[0177] In summary, the embodiment of the present application proposes a new medical model, that is, online emotional management of users, such as emotional assessment, early warning and identification. In detail, this method implements emotional assessment of users based on multimodal data collected by wearable devices worn by users under the guidance of basic theories of traditional Chinese medicine. This online emotional assessment method not only makes it possible for users to conduct self-assessment of emotional problems anytime and anywhere, but also effectively improves the efficiency of preventing emotional problems.

[0178] In addition, TCM theory is an effective guiding ideology for the management of emotional problems throughout the entire course of illness. The holistic concept of the prevention and treatment of emotional problems requires a comprehensive assessment of etiology and pathogenesis, macroscopic symptom manifestations and microscopic test results, treatment outcomes, and daily conditioning. Syndrome differentiation and treatment focuses more on the determination of diagnosis, the formation of treatment principles and methods, and the use of traditional Chinese medicine. In addition, there are non-drug therapies that rely on acupoint plastering, massage, aromatherapy, qigong guidance, and other means, which can not only effectively improve the patient's mood, regulate the patient's body functions as a whole, reduce toxic side effects, reduce the patient's personal risks, and improve the patient's treatment compliance, but also reduce the medical expenses and economic losses caused by the treatment of emotional disorders. In summary, the emotional management solution provided in the embodiments of the present application is more suitable for Chinese people to manage their emotions.

[0179] In addition, this method can also output emotion assessment reports and early warning information based on emotion assessment results, so that users can understand the emotional problems they face in a timely and detailed manner, and the human-computer interaction efficiency is high. In addition, this method can also recommend highly personalized emotion intervention suggestions to users based on emotion assessment results and the user's personal physique, so that users can conduct self-help intervention for emotional problems that match their own emotional and physical conditions in a timely manner, and the human-computer interaction effect is good.

[0180] In addition, since wearable devices can collect multimodal data of users in real time, the embodiments of the present application can monitor the user's emotions in a timely manner and output early warning information in a timely manner. For example, an alarm mechanism can be established for some users with more serious emotional problems to contact the users in real time to avoid risks caused by emotional problems. In addition, for some users with more serious emotional problems, the embodiments of the present application can also track and monitor them to form multiple rounds of feedback loops between emotional assessment, early warning and self-help intervention, and generate tracking reports.

[0181] Figure 3 is a flow chart of another emotion management method provided by an embodiment of the present application. The execution subject of the method is a computer device. Taking the user's emotion assessment and intervention after the user initiates a request as an example, see Figure 3 , the method flow provided in the embodiment of the present application includes:

[0182] 301. After the target object initiates an emotion assessment request, the computer device obtains input data of the target object during the human-computer dialogue process.

[0183] In the embodiment of the present application, the target object is Figure 4 The human-computer interaction interface shown in (a) initiates an emotion assessment request. The input data of the target object can be text, audio, or video, which is not limited in this application. That is, the target object can conduct online medical consultation through various methods such as text input, voice call, or video call.

[0184] 302. The computer device obtains the electronic health data of the target object and the multimodal data collected by the wearable device worn by the target object.

[0185] Exemplarily, with user authorization, the computer device obtains electronic health data of target fulfillment from a third-party data platform, which is not limited in this application.

[0186] 303. The computer device performs emotion recognition on the target object based on the multimodal data of the target object and the input data in the human-computer dialogue process to obtain an emotion recognition result.

[0187] Among them, when evaluating the emotion of the target object, it is also possible to combine the input data of the target object during the human-computer dialogue process to achieve a more accurate emotion evaluation of the target object.

[0188] In addition, since the target object may have multiple rounds of input during the human-computer dialogue process, the emotion recognition process can be performed multiple times, wherein the user input data input into the emotion recognition model during each emotion recognition process is different. For example, the user input data of the previous emotion recognition process is a subset of the user input data of the next emotion recognition process. This application does not limit this.

[0189] 304. The computer device outputs response data that matches the input data of the target object based on the emotion recognition result, the electronic health data of the target object and the knowledge base.

[0190] In one possible implementation, the computer device calls a pre-trained human-computer interaction model, and based on the emotion recognition results, the target object's electronic health data and knowledge base, outputs response data that matches the input data of the target object. For example, the computer device inputs the emotion recognition results and electronic health data into the human-computer interaction model, and then the human-computer interaction model uses the pre-established knowledge base to conduct a medical consultation with the target object. Figure 4 (b) to Figure 4 As shown in (d), this human-computer interaction method can fully simulate the real TCM consultation scene, provide users with a natural and interactive question-and-answer experience, and has high human-computer interaction efficiency.

[0191] In another possible implementation, the process of establishing the knowledge base includes:

[0192] 1. Determine the knowledge scope of the field of traditional Chinese medicine, extract key concepts within the knowledge scope; analyze the semantic associations between the extracted key concepts, establish semantic association triples based on the analyzed semantic associations; and construct the field ontology of traditional Chinese medicine within the knowledge scope. Exemplarily, this step uses SINOMED as a reference, traditional Chinese medicine clinical terminology as a framework, and emotional issues as a starting point to form a framework system of domain knowledge, and refers to the language system of traditional Chinese medicine to clarify the commonly used semantic associations in the field knowledge, and then establish semantic association triples and construct the field ontology of traditional Chinese medicine.

[0193] 2. Establish a TCM literature library; wherein the TCM literature library includes a plurality of documents, which are used to treat emotional problems based on TCM knowledge. Exemplarily, these documents include ancient TCM literature and modern basic and clinical research literature, which is not limited in this application.

[0194] 305. The computer device outputs an emotion assessment report and warning information based on the emotion recognition results; and outputs emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition results and the physical characteristics of the target object.

[0195] This step may refer to the above-mentioned step 203 and step 204, which will not be described in detail here.

[0196] In another possible implementation, the embodiment of the present application also provides a tracking and follow-up service to comprehensively record and evaluate the user's disease progression. Accordingly, the method provided by the embodiment of the present application also includes:

[0197] When the severity of the target subject's emotional problems is above a preset threshold, follow-up information is intermittently output to the target subject, and feedback data from the target subject during each follow-up is obtained; based on the feedback data obtained during each follow-up, the emotional intervention suggestions for the target subject are adjusted until the severity of the target subject's emotional problems is below the preset threshold.

[0198] In an embodiment of the present application, the follow-up information is output to the target object at intervals, including: periodically outputting the follow-up information to the target object at fixed time intervals; or, outputting the follow-up information to the target object at intervals at non-fixed time intervals. Among them, the non-fixed time intervals can be preset, such as following up at time intervals of 7 days, 14 days, and 21 days after the initial visit. In addition, the next follow-up time can also be determined based on the feedback data of the target object during this follow-up process, and this application does not limit this.

[0199] In another possible implementation, based on the feedback data obtained during each follow-up, the emotional intervention suggestions for the target object are adjusted, including but not limited to the following methods:

[0200] For any follow-up process, the computing device inputs the feedback data obtained during the follow-up process and the multimodal data obtained within the first time range into the emotion recognition model, and performs emotion recognition on the target object again based on the emotion recognition model. The first time range refers to the period between the last follow-up process and the current follow-up process. Alternatively, the first time range refers to the period between the initial visit process and the first follow-up process.

[0201] Next, the computer device adjusts the target object's emotional intervention suggestions based on the emotion recognition results obtained during this follow-up and the target object's physical characteristics. The emotional intervention suggestions here are the emotional intervention suggestions determined during the previous follow-up or the emotional intervention suggestions determined during the initial visit.

[0202] In summary, the embodiment of the present application proposes a new medical model, that is, online emotional management of users. In detail, the user can actively initiate an emotional assessment request for emotional problems through the terminal, and the emotional management platform side can simulate the real offline Chinese medicine clinic scene after receiving the user's emotional assessment request, and conduct natural and interactive questions and answers with the user. In addition, during the medical consultation process, the embodiment of the present application will, under the guidance of the basic theory of Chinese medicine, conduct emotional assessment on the user based on the multimodal data collected by the wearable device worn by the user and the input data of the user during the human-computer dialogue process. This online emotional assessment method not only makes it possible for users to conduct self-service assessment of emotional problems anytime and anywhere, but also effectively improves the prevention efficiency of emotional problems. In addition, the method can also output emotional assessment reports and early warning information based on the emotional assessment results, so that users can understand the emotional problems they face in a timely and detailed manner, and the human-computer interaction efficiency is high. In addition, the method can also recommend highly personalized emotional intervention suggestions to users based on the emotional assessment results and the user's personal physique, so as to facilitate users to conduct self-service intervention of emotional problems that match their own emotional and physical conditions in a timely manner, and the human-computer interaction effect is good.

[0203] The following introduces the training and optimization process of the emotion management network (used for emotion recognition, early warning and intervention).

[0204] 1. Collect large amounts of data related to emotional issues, and clean and label the collected data.

[0205] In one possible implementation, the collected data includes user data, such as multimodal data from sample subjects, clinically collected TCM diagnosis and treatment cases related to emotional problems, and other data that can reflect the user's emotional state, health status, or TCM diagnosis results. In addition, the collected data also includes knowledge in the field of traditional Chinese medicine, such as symptom information extracted from clinical trials and experimental studies of emotional problems, and question and answer data in the process of human-computer dialogue, which is not limited in this application.

[0206] In another possible implementation, the data cleaning process includes the following steps:

[0207] 1. Data deduplication: Remove duplicate and similar data to ensure that each data point is unique.

[0208] 2. Noise removal: Remove noise and irrelevant information from the data, such as background noise or irrelevant sensor data.

[0209] 3. Outlier detection: Detect outliers through statistical methods and remove or correct them.

[0210] 4. Missing value processing: handle missing data values, such as through interpolation, extrapolation or deletion of missing values.

[0211] 5. Feature selection: used to select features related to emotional issues and remove irrelevant or redundant features.

[0212] Among them, feature selection is the process of removing irrelevant and redundant features, selecting relevant features, and generating better feature subsets, which can reduce the dimension of original data, shorten model training time, and improve model performance. Generally speaking, the feature selection process is divided into four parts, namely, subset generation strategy, subset evaluation function, feature selection stopping criterion, and model verification. That is, first generate candidate feature subsets in the original feature space, then use a certain evaluation function to evaluate the generated candidate feature subsets, and then compare the evaluation results with the feature selection stopping criterion; if the feature selection stopping criterion is met, the generation of feature subsets is terminated; if the feature selection stopping criterion is not met, continue to generate candidate feature subsets until the feature selection stopping criterion is met; finally, the selected feature subsets are verified by the model and the optimal subset is output.

[0213] 6. Data standardization: Standardize the data to a uniform scale for better model training.

[0214] 7. Data conversion: Convert data into a format and type suitable for model training.

[0215] 8. Data labeling: Converting raw data into a format that can be understood and used by machine learning algorithms.

[0216] 9. Dataset division: Divide the dataset into training dataset, validation dataset and test dataset to evaluate the performance and generalization ability of the model.

[0217] 10. Data anonymization and privacy protection: Ensure the privacy and security of data during use and storage, such as desensitizing and encrypting data for storage.

[0218] Among them, data annotation plays a vital role in artificial intelligence models. Before using data such as text, images, or audio for training artificial intelligence models, it is usually necessary to annotate them to convert them into a format that the model can understand. For sentiment assessment services, for text data, it may be necessary to annotate the text data as positive, negative, or neutral sentiment tendencies, or perform more complex annotations depending on the specific situation. For image data, there are many types of annotations, such as point cloud segmentation, continuous frame annotation, key point annotation, panoramic segmentation, entity extraction, instance segmentation, etc. The annotation of audio data involves the transcription and timestamping of speech, including the transcription of specific voices and intonations.

[0219] 2. Model selection and architecture design, and model training based on training data sets.

[0220] When selecting models and designing architectures, a single model or a combination of multiple models can be selected for training according to the specific requirements of the task, and this application does not limit this. In the embodiments of this application, the emotion management network has the capabilities of natural language understanding and generation, knowledge question answering and reasoning, machine translation, and human-computer dialogue.

[0221] In one possible implementation, the emotion management network is composed of a combination of multiple models, which may involve an emotion recognition model for performing emotion recognition tasks, an early warning model for performing early warning tasks, a human-computer dialogue model for performing human-computer dialogue tasks, an intervention suggestion recommendation model for outputting personalized intervention plans, etc. This application does not limit this.

[0222] In addition, it is necessary to design an appropriate network architecture to capture the complex patterns and features in the data. For example, through large model technology, macro physiological characteristics are combined with modern medical indicators to analyze the relationship between the two, and the basic theory of traditional Chinese medicine is applied to classify users and build a comprehensive emotion assessment model.

[0223] In another possible implementation, in terms of model architecture, the emotion management network adopts a deep learning model, such as a recurrent neural network, a long short-term memory network, or a convolutional neural network. Alternatively, the emotion management network adopts a neural network model Transformer architecture based on a self-attention mechanism. Among them, the model consists of multiple stacked encoder layers, each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network, which can capture long-distance dependencies in the text and has good parallel computing performance. Alternatively, the emotion management network adopts a multimodal large model, which is not limited in this application.

[0224] In addition, model selection and architecture design, as well as model training, can also be performed based on the idea of ​​ensemble learning or multi-task learning, which is not limited in this application.

[0225] In another possible implementation, during the model training process, the following training method can be used:

[0226] Big data training: Using large-scale training data sets for training, the emotion management network has better generalization capabilities.

[0227] Distributed training: Distribute data and computing tasks across multiple GPUs (Graphics Processing Units) or servers to speed up the training process.

[0228] Gradient descent optimization algorithm: Use optimization algorithms such as Adam and RMSprop to adjust model parameters to minimize the loss function.

[0229] In addition, during the model training process, you can also consider applying early stopping or Dropout to prevent the model from overfitting.

[0230] 3. Evaluate and optimize the performance of the trained artificial intelligence model for emotion management.

[0231] In the embodiments of the present application, model evaluation and tuning refers to the use of a validation data set or a test data set to regularly evaluate model performance. Among them, the validation data set is used to adjust hyperparameters and select the best model, and the test data set is used to evaluate model performance. Indicators used to evaluate model performance include but are not limited to: accuracy, recall rate, F1 score, etc., which are not limited in this application. Ultimately, based on the performance evaluation results of the model, adjustments such as model parameters, structure or training strategies are made.

[0232] In addition, the trained emotion management network also needs to be continuously updated and optimized based on continuous learning mechanisms and model update strategies to adapt to new data and changes. In addition, for data involving user privacy, technologies such as differential privacy and federated learning can be used to ensure data security. In addition, the identification and intervention of emotional problems not only rely on the results output by the emotion management network, but also need to output explanatory information to users based on interpretability technology so that users can better understand the working principle of the model and the basis for decision-making.

[0233] Next, the trained emotion management network can be deployed to actual application scenarios and continuously optimized and updated to improve the accuracy and stability of the emotion management system.

[0234] In summary, the emotion management scheme provided by the embodiment of the present application is based on emotion recognition technology, early warning mechanism and emotion intervention mechanism, combined with the human-computer interaction system enabled by traditional Chinese medicine, to achieve highly personalized emotion recognition, early warning and intervention, and extremely natural human-computer interaction, not only the accuracy of emotion recognition, early warning and intervention is high, but also the user experience is good. In other words, the embodiment of the present application can improve the accuracy and scientificity of emotion recognition by integrating the basic theory of traditional Chinese medicine and artificial intelligence technology, and can provide users with more comprehensive, more timely, more accurate and more personalized emotion early warning and intervention services based on this. In addition, the human-computer interaction system can simulate the real Chinese medicine consultation scene and provide users with a more natural and real interactive experience. In addition, by integrating and analyzing user data and data on traditional Chinese medicine, it can not only provide a more comprehensive and systematic decision-making basis, but also improve data utilization. In addition, it can also provide users with personalized and personal physique-adapted Chinese medicine intervention suggestions based on the user's emotional state and physical characteristics, improve user satisfaction, and have strong practicality.

[0235] Figure 5 is a structural diagram of an emotion management device provided in an embodiment of the present application. Figure 5 , the device comprises:

[0236] The acquisition module 501 is configured to acquire multimodal data collected by a wearable device worn by a target object; wherein the multimodal data includes physiological indicator data, audio data and environmental data of the current environment of the target object;

[0237] The processing module 502 is configured to perform emotion recognition on the target object based on the multimodal data with reference to basic theoretical knowledge of traditional Chinese medicine, and obtain an emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object;

[0238] The output module 503 is configured to output an emotion assessment report and warning information based on the emotion recognition result; wherein the warning information is used to indicate the severity of the emotional problem of the target object;

[0239] The output module 503 is further configured to output emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition result and the physical characteristics of the target object.

[0240] The embodiment of the present application proposes a new medical model, that is, online emotional management of users, such as emotional assessment, early warning and identification. In detail, this method implements emotional assessment of users based on multimodal data collected by wearable devices worn by users under the guidance of basic theories of traditional Chinese medicine. This online emotional assessment method not only makes it possible for users to conduct self-service assessments of emotional problems anytime and anywhere, but also effectively improves the efficiency of preventing emotional problems. In addition, this method can also output emotional assessment reports and early warning information based on the emotional assessment results, so that users can understand the emotional problems they face in a timely and detailed manner, and the human-computer interaction efficiency is high. In addition, this method can also recommend highly personalized emotional intervention suggestions to users based on the emotional assessment results and the user's personal physique, so as to facilitate users to conduct self-service interventions for emotional problems that match their own emotional and physical conditions in a timely manner, and the human-computer interaction effect is good.

[0241] In a possible implementation, the acquisition module 501 is further configured to acquire input data of the target object during the human-computer dialogue process;

[0242] The acquisition module 501 is further configured to acquire the electronic health data of the target object;

[0243] The output module 503 is further configured to output response data matching the input data based on the emotion recognition result, the electronic health data and the knowledge base.

[0244] In another possible implementation, the processing module 502 is configured to perform emotion recognition on the target object based on the multimodal data and the input data to obtain the emotion recognition result.

[0245] In another possible implementation, the process of establishing the knowledge base includes:

[0246] Determine the knowledge scope of the field of traditional Chinese medicine, extract key concepts within the knowledge scope; analyze the semantic associations between the extracted key concepts, establish semantic association triples based on the analyzed semantic associations; and construct a traditional Chinese medicine field ontology within the knowledge scope;

[0247] Establish a literature database in the field of traditional Chinese medicine; wherein the literature database in the field of traditional Chinese medicine includes a plurality of documents, and the plurality of documents are used to treat emotional problems based on knowledge in the field of traditional Chinese medicine.

[0248] In another possible implementation, the acquisition module 501 is configured to perform at least one of the following:

[0249] Acquire the physiological sign data of the target object according to a first rule; wherein the first rule is used to instruct acquisition of physiological sign data related to TCM diagnosis;

[0250] Acquire the physiological sign data of the target object according to a second rule; wherein the second rule is used to instruct acquisition of physiological sign data affected by emotional changes.

[0251] In another possible implementation, the physiological indicator data of the target object includes at least one of the following:

[0252] the target subject's heart rate;

[0253] The heart rate variability of the target object; wherein the heart rate variability refers to the change in the difference between successive heartbeat cycles;

[0254] the sleep quality of the target subject;

[0255] the activity level of the target subject;

[0256] The physiological stress response of the target object; wherein the physiological stress response is used to measure the physical stress level of the target object;

[0257] the body temperature of the target subject;

[0258] the target object's posture and head movement;

[0259] The target subject's breathing rate.

[0260] In another possible implementation, the output module 503 is configured to obtain the scene requirement information of the target object; and output the emotion intervention suggestion in an output manner matching the scene requirement information based on the emotion recognition result and the physical characteristics of the target object.

[0261] In another possible implementation, the output module 503 is configured to output, based on the emotion recognition result, an emotion intervention suggestion that matches the TCM constitution type of the target object in an output manner that matches the scenario requirement information.

[0262] In another possible implementation, the output module 503 is configured to output the target object's emotion fluctuation record based on the emotion change trend included in the emotion recognition result.

[0263] In another possible implementation, the output module 503 is configured to obtain the historical emotion change trend of the target object; based on the historical emotion change trend and the emotion change trend included in the emotion recognition result, output the daily emotion fluctuation record of the target object in multiple time statistical dimensions in the form of a calendar.

[0264] In another possible implementation, the output module 503 is further configured to output follow-up information to the target object at intervals when the severity of the emotional problem of the target object is above a preset threshold;

[0265] The acquisition module 501 is further configured to acquire feedback data of the target object during each follow-up;

[0266] The processing module 502 is further configured to adjust the emotion intervention suggestion of the target object based on the feedback data obtained during each follow-up process until the severity of the emotional problem of the target object is below the preset threshold.

[0267] In another possible implementation, the processing module 502 is further configured to, for any follow-up process, input the feedback data obtained in the current follow-up process and the multimodal data obtained within the first time range into the emotion recognition model, and perform emotion recognition on the target object again based on the emotion recognition model; based on the emotion recognition results obtained in the current follow-up process and the physical characteristics of the target object, adjust the emotion intervention suggestions for the target object; wherein the first time range refers to the period between the last follow-up process and the current follow-up process.

[0268] In another possible implementation, the processing module 502 is configured to input the multimodal data into an emotion recognition model, perform emotion recognition on the target object based on the emotion recognition model, and obtain the emotion recognition result; wherein the emotion recognition model is an artificial intelligence model that integrates basic theoretical knowledge of traditional Chinese medicine;

[0269] The training process of the emotion recognition model includes:

[0270] Constructing a sample database; wherein the sample database includes clinically collected TCM diagnosis and treatment cases, symptom information extracted from clinical trials and experimental studies of emotional problems, and multimodal data of sample subjects; the TCM diagnosis and treatment cases are related to emotional problems;

[0271] The artificial intelligence model is trained based on the sample database to obtain the emotion recognition model.

[0272] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0273] It should be noted that: the emotion management device provided in the above embodiment only uses the division of the above functional modules as an example when performing emotion management. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the emotion management device provided in the above embodiment and the emotion management method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0274] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. Among them, the computer 600 can be a server. The computer device 600 may have relatively large differences due to different configurations or performances, including one or more processors (Central Processing Units, CPU) 601 and one or more memories 602, wherein the memory 602 stores at least one program code, and the at least one program code is loaded and executed by the processor 601 to implement the above-mentioned emotion management method. Of course, the computer device 600 also has components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The computer device 600 also includes other components for realizing the functions of the device, which will not be repeated here.

[0275] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a program code, and the program code can be executed by a processor in a computer device to perform the above-mentioned emotion management method. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0276] In an exemplary embodiment, a computer program product or a computer program is also provided, which includes a computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned emotion management method.

[0277] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0278] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for emotion management, characterized in that: The method comprises: Acquire multimodal data collected by a wearable device worn by a target object; wherein the multimodal data includes physiological indicator data, audio data, and environmental data of the current environment of the target object; Using basic theoretical knowledge of traditional Chinese medicine as a reference, emotion recognition is performed on the target object based on the multimodal data to obtain an emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object; Based on the emotion recognition result, an emotion assessment report and warning information are output; wherein the warning information is used to indicate the severity of the emotional problem of the target object; Based on the emotion recognition result and the physical characteristics of the target object, emotion intervention suggestions in the field of traditional Chinese medicine are output.

2. The method according to claim 1, characterized in that: The method further comprises: Obtaining input data of the target object during the human-computer dialogue process; Acquiring electronic health data of the target object; Based on the emotion recognition result, the electronic health data and the knowledge base, response data matching the input data is output.

3. The method according to claim 2, characterized in that The performing emotion recognition on the target object based on the multimodal data to obtain an emotion recognition result includes: Based on the multimodal data and the input data, emotion recognition is performed on the target object to obtain the emotion recognition result.

4. The method according to claim 2, characterized in that: The process of establishing the knowledge base includes: Determine the knowledge scope of the field of traditional Chinese medicine, extract key concepts within the knowledge scope; analyze the semantic associations between the extracted key concepts, establish semantic association triples based on the analyzed semantic associations; and construct a traditional Chinese medicine field ontology within the knowledge scope; Establish a literature database in the field of traditional Chinese medicine; wherein the literature database in the field of traditional Chinese medicine includes a plurality of documents, and the plurality of documents are used to treat emotional problems based on knowledge in the field of traditional Chinese medicine.

5. The method according to claim 1, characterized in that The acquiring of multimodal data collected by a wearable device worn by the target object includes at least one of the following: Acquire the physiological sign data of the target object according to a first rule; wherein the first rule is used to instruct acquisition of physiological sign data related to TCM diagnosis; Acquire the physiological sign data of the target object according to a second rule; wherein the second rule is used to instruct acquisition of physiological sign data affected by emotional changes.

6. The method according to claim 1 or 5, characterized in that: The physiological indicator data of the target object includes at least one of the following: the target subject's heart rate; The heart rate variability of the target object; wherein the heart rate variability refers to the change in the difference between successive heartbeat cycles; the sleep quality of the target subject; the activity level of the target subject; The physiological stress response of the target object; wherein the physiological stress response is used to measure the physical stress level of the target object; the body temperature of the target subject; the target object's posture and head movement; The target subject's breathing rate.

7. The method according to claim 1, characterized in that Outputting emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition results and the physical characteristics of the target object includes: Acquire scene requirement information of the target object; Based on the emotion recognition result and the physical characteristics of the target object, the emotion intervention suggestion is output in an output manner that matches the scenario requirement information.

8. The method according to claim 7, characterized in that The step of outputting the emotion intervention suggestion based on the emotion recognition result and the physical characteristics of the target object in an output manner matching the scene requirement information includes: Based on the emotion recognition result, an emotion intervention suggestion consistent with the TCM constitution type of the target object is output in an output manner that matches the scenario requirement information.

9. The method according to claim 1, characterized in that: Outputting an emotion assessment report based on the emotion recognition result includes: Based on the emotion change trend included in the emotion recognition result, the emotion fluctuation record of the target object is output.

10. The method according to claim 9, characterized in that The step of outputting the target object's emotion fluctuation record based on the target object's emotion change trend includes: Obtaining the historical emotion change trend of the target object; Based on the historical emotion change trend and the emotion change trend included in the emotion recognition result, the daily emotion fluctuation record of the target object in multiple time statistical dimensions is output in the form of a calendar.

11. The method according to claim 1, characterized in that When the severity of the emotional problem of the target object is above a preset threshold, the method further includes: outputting follow-up information to the target object at intervals, and obtaining feedback data from the target object during each follow-up; Based on the feedback data obtained during each follow-up, the emotional intervention suggestions for the target object are adjusted until the severity of the emotional problem of the target object is below the preset threshold.

12. The method according to claim 11, characterized in that The step of adjusting the target object's emotional intervention suggestions based on the feedback data obtained during each follow-up visit includes: For any follow-up process, the feedback data obtained in the follow-up process and the multimodal data obtained in the first time range are input into the emotion recognition model, and emotion recognition is performed again on the target object based on the emotion recognition model; Based on the emotion recognition results obtained during this follow-up and the physical characteristics of the target object, adjusting the emotion intervention suggestions for the target object; The first time range refers to the period between the last follow-up process and the current follow-up process.

13. The method according to any one of claims 1 to 12, characterized in that The method of using basic theoretical knowledge of traditional Chinese medicine as a reference and performing emotion recognition on the target object based on the multimodal data to obtain an emotion recognition result includes: Inputting the multimodal data into an emotion recognition model, performing emotion recognition on the target object based on the emotion recognition model, and obtaining the emotion recognition result; wherein the emotion recognition model is an artificial intelligence model integrating basic theoretical knowledge of traditional Chinese medicine; The training process of the emotion recognition model includes: Constructing a sample database; wherein the sample database includes clinically collected TCM diagnosis and treatment cases, symptom information extracted from clinical trials and experimental studies of emotional problems, and multimodal data of sample subjects; the TCM diagnosis and treatment cases are related to emotional problems; The artificial intelligence model is trained based on the sample database to obtain the emotion recognition model.

14. An emotion management device, characterized in that: The device comprises: An acquisition module is configured to acquire multimodal data collected by a wearable device worn by a target object; wherein the multimodal data includes physiological indicator data, audio data, and environmental data of the current environment of the target object; A processing module is configured to use basic theoretical knowledge of traditional Chinese medicine as a reference, perform emotion recognition on the target object based on the multimodal data, and obtain an emotion recognition result; wherein the emotion recognition result includes the traditional Chinese medicine syndrome, emotion type, emotion intensity level and emotion change trend of the target object; An output module is configured to output an emotion assessment report and warning information based on the emotion recognition result; wherein the warning information is used to indicate the severity of the emotional problem of the target object; The output module is further configured to output emotion intervention suggestions in the field of traditional Chinese medicine based on the emotion recognition results and the physical characteristics of the target object.

15. A computer device, characterized in that: The device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the emotion management method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the emotion management method according to any one of claims 1 to 13.

17. A computer program product or a computer program, characterized in that The computer program product or computer program includes a computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the emotion management method as claimed in any one of claims 1 to 13.

Images