Data processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a computer readable storage medium, and belongs to the technical field of data processing. The method comprises the following steps: acquiring multi-modal emotion data of a target object; obtaining a first emotion entropy value according to the multi-modal emotion data of the target object; acquiring brain wave data of the target object; obtaining a second emotion entropy value according to the brain wave data of the target object; obtaining a target emotion entropy value according to the first emotion entropy value and the second emotion entropy value; and performing grade division on the target emotion entropy according to an emotion evaluation criterion to obtain an emotion state grade of the target object, and setting an emotion atmosphere scene according to the emotion state grade. Through emotion label identification of the multi-modal emotion data and the brain wave data, the emotion score of the target object is obtained more accurately, and the ability of adjusting the environment atmosphere is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a data processing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the advancement of medical technology and the deepening of purification engineering concepts, the construction standards of operating rooms, as crucial places in hospitals, are constantly improving. Although digital operating rooms have achieved many breakthroughs in technology, the decoration of operating rooms is monotonous and cold, and it is impossible to accurately adjust the scene atmosphere according to the patient's emotional state. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present disclosure provide a data processing method, device, electronic device and computer-readable storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method, the method comprising: acquiring multimodal emotion data of a target object; obtaining a first emotion entropy value based on the multimodal emotion data of the target object; acquiring brain wave data of the target object; obtaining a second emotion entropy value based on the brain wave data of the target object; obtaining a target emotion entropy value based on the first emotion entropy value and the second emotion entropy value; and grading the target emotion entropy value according to emotion evaluation criteria to obtain the emotion state level of the target object, and setting an emotional atmosphere scene according to the emotion state level.

[0005] In one embodiment, the multimodal emotion data includes a facial expression image, a voice, and a physiological impedance value of a target object, and calculating a first emotion entropy value based on the multimodal emotion data of the target object includes: obtaining a first emotion sub-entropy value based on the facial expression image of the target object, and setting a first weight corresponding to the first emotion sub-entropy value; obtaining a second emotion sub-entropy value based on the voice of the target object, and setting a second weight corresponding to the first emotion sub-entropy value; obtaining a third emotion sub-entropy value based on the physiological impedance value of the target object, and setting a third weight corresponding to the first emotion sub-entropy value; and weighted summing the first emotion sub-entropy value, the first weight, the second emotion sub-entropy value, the second weight, the third emotion sub-entropy value, and the third weight to obtain the first emotion entropy value.

[0006] In one embodiment, a first emotion sub-entropy value is obtained based on a facial expression image of a target object, including: constructing a facial expression data set and constructing a first deep learning neural network based on facial expression features; wherein the facial expression data set includes multiple facial expression images and facial emotion labels corresponding to each facial expression image; a facial emotion model is obtained by training the first deep learning neural network and the facial expression data set; according to the facial emotion model, the facial emotion labels corresponding to the facial expression images of the target object are identified; and the first emotion sub-entropy value of the target object is obtained based on the emotion scores corresponding to different preset facial emotion labels.

[0007] In one embodiment, a second emotion sub-entropy value is obtained based on the sound of a target object, including: constructing a sound data set and constructing a second deep learning neural network based on the sound features; wherein the sound data set includes multiple sound data and a sound emotion label corresponding to each sound data; training a sound emotion model through the second deep learning neural network and the sound data set; identifying the sound emotion label corresponding to the sound data of the target object based on the sound emotion model; and obtaining the second emotion sub-entropy value of the target object based on the emotion scores corresponding to different preset sound emotion labels.

[0008] In one embodiment, a third emotion sub-entropy value is obtained based on the physiological impedance value of the target object, including: constructing a physiological data set and constructing a third deep learning neural network based on physiological characteristics; wherein the physiological data set includes multiple physiological impedance values ​​and a physiological emotion label corresponding to each physiological impedance value; training a physiological emotion model through the third deep learning neural network and the physiological data set; identifying the physiological emotion label corresponding to the physiological data of the target object according to the physiological emotion model; and obtaining the third emotion sub-entropy value of the target object according to the emotion scores corresponding to different preset physiological emotion labels.

[0009] In one embodiment, a second emotion entropy value is obtained based on the brain wave data of the target object, including: constructing a brain wave data set and constructing a fourth deep learning neural network based on the brain wave characteristics; wherein the brain wave data set includes multiple brain wave data signals and a brain wave emotion label corresponding to each brain wave data signal; a brain wave emotion model is trained through the fourth deep learning neural network and the brain wave data set; according to the brain wave emotion model, the brain wave emotion label corresponding to the brain wave data of the target object is identified; and according to the emotion scores corresponding to different preset brain wave emotion labels, the second emotion entropy value of the target object is obtained.

[0010] In one embodiment, the vital sign parameters of the target object are obtained and normalized; a weight value corresponding to each vital sign parameter is set, and the normalized vital sign parameters and the corresponding weight values ​​are weighted summed to obtain a health entropy value; and the health entropy value is graded according to health assessment criteria to obtain the health status level of the target object, and the comprehensive status level of the target object is obtained according to the emotional state level and the health state level, and an emotional atmosphere scene is set according to the comprehensive status level.

[0011] In a second aspect, an embodiment of the present disclosure provides a data processing device, which includes: a first processing module, used to obtain multimodal emotion data of a target object; obtain a first emotion entropy value based on the multimodal emotion data of the target object; a second processing module, used to obtain brain wave data of the target object; obtain a second emotion entropy value based on the brain wave data of the target object; obtain a target emotion entropy value based on the first emotion entropy value and the second emotion entropy value; and a setting module, used to grade the target emotion entropy value according to emotion evaluation criteria, obtain the emotion state level of the target object, and set an emotion atmosphere scene according to the emotion state level.

[0012] In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is run by the processor, the data processing method provided in the first aspect is executed.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, which executes the data processing method provided in the first aspect when the computer program is run on a processor.

[0014] The data processing method provided by the above disclosure and the data processing method provided by the embodiment of the disclosure obtain the multimodal emotional data of the target object; obtain the first emotional entropy value according to the multimodal emotional data of the target object; obtain the brain wave data of the target object; obtain the second emotional entropy value according to the brain wave data of the target object; obtain the target emotional entropy value according to the first emotional entropy value and the second emotional entropy value; and classify the target emotional entropy value according to the emotional evaluation criteria to obtain the emotional state level of the target object, and set the emotional atmosphere scene according to the emotional state level. By identifying the emotional labels of the multimodal emotional data and the brain wave data, the emotional score of the target object is obtained more accurately, and the ability to adjust the environmental atmosphere is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope of protection of the present disclosure. In each of the drawings, similar components are numbered similarly.

[0016] Figure 1 A first flow chart of a data processing method provided by an embodiment of the present disclosure is shown;

[0017] Figure 2 A second flow chart of the data processing method provided by the embodiment of the present disclosure is shown;

[0018] Figure 3 A third flow chart of the data processing method provided by the embodiment of the present disclosure is shown;

[0019] Figure 4 A fourth flow chart of the data processing method provided by the embodiment of the present disclosure is shown;

[0020] Figure 5 A fifth flow chart of the data processing method provided by the embodiment of the present disclosure is shown;

[0021] Figure 6 A sixth flow chart of the data processing method provided by the embodiment of the present disclosure is shown; and

[0022] Figure 7 A schematic diagram of the structure of a data processing device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments.

[0024] The components of the embodiments of the present disclosure generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0025] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present disclosure, are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing items.

[0026] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0027] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present disclosure belong. Terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present disclosure.

[0028] An embodiment of the present disclosure provides a data processing method.

[0029] See also Figure 1 , the data processing method includes steps S110 to S130.

[0030] Step S110, obtaining multimodal emotion data of the target object; and obtaining a first emotion entropy value according to the multimodal emotion data of the target object.

[0031] In the disclosed embodiment, a camera is used to capture the facial expressions of a target object (such as a patient) in an operating room or other wards, a microphone is used to record sound data, and a sensor is used to measure physiological impedances such as skin impedance and body impedance.

[0032] Exemplarily, the collected data is analyzed to extract emotion-related features, such as geometric changes in facial expressions, voice pitch, speech speed, volume, and changes in impedance values, and finally a quantitative first emotion entropy value is obtained, and the emotional state is evaluated by the first emotion entropy value.

[0033] Step S120, obtaining brain wave data of the target object; obtaining a second emotion entropy value according to the brain wave data of the target object; and obtaining a target emotion entropy value according to the first emotion entropy value and the second emotion entropy value.

[0034] In an embodiment of the present disclosure, an electroencephalogram (EEG) or other device is used to record brain wave data of a target object, and emotion-related features are extracted from the brain wave data, such as the power spectral density of different frequency bands, the frequency of occurrence of specific waveforms, etc. The emotional state of the target object is obtained based on the brain wave features, and a second emotion entropy value is calculated. The first emotion entropy value and the second emotion entropy value are weighted or fused to obtain a target emotion entropy value, thereby making the emotion assessment more comprehensive.

[0035] Step S130, classifying the target emotion entropy value according to the emotion evaluation criteria, obtaining the emotion state level of the target object, and setting the emotion atmosphere scene according to the emotion state level.

[0036] In the disclosed embodiment, the emotion assessment result is converted into an understandable emotion state level.

[0037] For example, when the emotional entropy value is 80-100, the patient's current emotion is a first-level emotion, indicating a happy state. When the emotional entropy value is 60-80, the patient's current emotion is a second-level emotion, indicating a peaceful state. When the emotional entropy value is 40-60, the patient's current emotion is a third-level emotion, indicating a tense state. When the emotional entropy value is 20-40, the patient's current emotion is a fourth-level emotion, indicating an anxious state. When the emotional entropy value is 0-20, the patient's current emotion is a fifth-level emotion, indicating a fearful state.

[0038] For example, according to the emotional state level, when the emotional entropy value is the first level, it indicates a happy state, and a starry sky top picture is displayed. When the emotional entropy value is the second level, it indicates a peaceful state, and a small bridge and flowing water picture is displayed. When the emotional entropy value is the third level, it indicates a tense state, and a natural scenery picture is displayed to relieve the patient's nervous mood. When the emotional entropy value is the fourth or fifth level, it indicates anxiety or fear, and a warm and inspirational picture is displayed.

[0039] See also Figure 2 In one embodiment, the multimodal emotion data includes facial expression images, sounds, and physiological impedance values ​​of the target object, and a first emotion entropy value is obtained based on the multimodal emotion data of the target object, including steps S111 to S114.

[0040] Step S111, obtaining a first emotion sub-entropy value according to the facial expression image of the target object, and setting a first weight corresponding to the first emotion sub-entropy value.

[0041] In the disclosed embodiment, the system obtains the facial expression image of the target object through a camera or other image capture device, identifies the facial tag type using a pre-trained deep learning model, and obtains the first emotion sub-entropy value of the target object according to the emotion score table corresponding to different facial expressions. According to the importance of facial expressions in emotion detection, the system assigns a first weight to the first emotion sub-entropy value. Generally, facial expressions are important clues in emotion detection, so the weight corresponding to facial expressions is relatively high.

[0042] Step S112: obtaining a second emotion sub-entropy value according to the sound of the target object, and setting a second weight corresponding to the second emotion sub-entropy value.

[0043] In the disclosed embodiment, the system collects the sound of the target object through a microphone and other devices, and extracts the sound quality, speaking speed, pitch and other features of the sound through a neural network through the collected sound data and a trained sound emotion model. The extracted features are used to output the emotion label corresponding to the sound, and the emotion entropy value table corresponding to the emotion label is matched to obtain a second emotion sub-entropy value. According to the relative importance of the sound in emotion detection, the system assigns a second weight to the second emotion sub-entropy value.

[0044] Step S113, obtaining a third emotion sub-entropy value according to the physiological impedance value of the target object, and setting a third weight corresponding to the third emotion sub-entropy value.

[0045] In the disclosed embodiment, the physiological impedance value of the target object is collected by a wearable device (such as a skin conductance sensor, etc.), the collected physiological impedance value is preprocessed by using a physiological signal processing technology, and the preprocessed physiological impedance value is input into a pre-trained physiological emotion model to evaluate the emotional state of the target object. According to the relative importance of the physiological signal in emotion detection, the system assigns a third weight to the third emotion sub-entropy value.

[0046] Step S114, performing weighted summation on the first emotion sub-entropy value, the first weight, the second emotion sub-entropy value, the second weight, the third emotion sub-entropy value and the third weight to obtain the first emotion entropy value.

[0047] In the disclosed embodiment, the system multiplies the three emotion sub-entropy values ​​from facial expressions, sounds, and physiological signals by their corresponding weights, and then adds the products to obtain a first emotion entropy value. The first emotion entropy value combines the results of multiple emotion detection methods, providing a more comprehensive and accurate emotion assessment, which can determine the current main emotional state of the target object and take corresponding emotion relief measures accordingly.

[0048] See also Figure 3 In one embodiment, step S111 includes steps S1111 to S1114.

[0049] Step S1111, constructing a facial expression data set and constructing a first deep learning neural network according to facial expression features; wherein the facial expression data set includes multiple facial expression images and a facial emotion label corresponding to each facial expression image.

[0050] In the disclosed embodiments, a large number of facial expression images are collected from various sources such as the Internet, public databases, and laboratory shots, a corresponding emotion label (such as happiness, sadness, anger, surprise, fear, disgust, neutrality, etc.) is assigned to each image, and the images are cropped, scaled, normalized, and other processes are performed to ensure that the data format of the input model is consistent and suitable for training.

[0051] For example, since the emotions of patients in the operating room vary greatly and there are more complex and diverse facial expressions, a deep learning architecture for image classification, such as ResNet, VGG, etc., is selected based on facial expressions, which can extract more feature vectors and design its parameters such as the number of layers, convolution kernel size, and activation function.

[0052] Step S1112: training a facial emotion model using a first deep learning neural network and a facial expression dataset.

[0053] In the disclosed embodiment, the preprocessed facial expression dataset is divided into a training set and a validation set. The model is trained using the training set data and its performance is evaluated on the validation set. The weights of the model are optimized through multiple iterations to reduce the prediction error. The model parameters are adjusted according to the performance on the validation set to further improve the accuracy and generalization ability of the model.

[0054] Step S1113, identifying the facial emotion label corresponding to the facial expression image of the target object according to the facial emotion model.

[0055] In an embodiment of the present disclosure, the facial expression image of the target object captured by the camera is input into a trained model, the model processes the input image, and outputs an emotion label and its corresponding probability or confidence, and based on the output probability or confidence, the most likely emotion label is selected as the recognition result.

[0056] Step S1114, obtaining a first emotion sub-entropy value of the target object according to the emotion scores corresponding to different preset facial emotion tags.

[0057] In the embodiment of the present disclosure, an emotion score is defined for each emotion label through an emotion score table set in advance, and the corresponding emotion score is obtained from the emotion score table from the emotion label output by the facial emotion model. The emotion score is the first emotion sub-entropy value.

[0058] See also Figure 4In one embodiment, step S112 includes steps S1121 to S1124.

[0059] Step S1121, constructing a sound data set and constructing a second deep learning neural network based on the sound features; wherein the sound data set includes multiple sound data and a sound emotion label corresponding to each sound data.

[0060] In the disclosed embodiment, a large amount of sound data needs to be collected from different sources such as telephone customer service recordings, online meeting records, and voice messages on social media to ensure the diversity and representativeness of the data, so as to cover people of different genders, ages, cultural backgrounds, and emotional states. Each collected sound file will be judged and labeled with emotions such as "angry", "anxious", "calm", and "happy" based on factors such as the tone, speed, volume, and clarity of pronunciation, as well as keywords or phrases in the sound.

[0061] For example, since the original sound data usually contains a large number of sample points, directly using it for neural network training will result in huge computational complexity and low efficiency. Before building a neural network, it is necessary to determine the sound features, including Mel-frequency cepstral coefficients (MFCCs), fundamental frequency (F0), energy, short-time zero-crossing rate, etc., and extract the above features from the original sound data through signal processing methods. Based on the extracted sound features, a combination of convolutional neural network (CNN) and gated recurrent unit (GRU) is designed to extract the emotional features in the above sound features, and an attention mechanism is used to enhance the model's attention to important sound fragments.

[0062] Step S1122: training a sound emotion model using a second deep learning neural network and a sound data set.

[0063] In the disclosed embodiment, the collected sound data is preprocessed, including noise removal, volume standardization, sampling rate adjustment, etc., to ensure the consistency of the data and the training effect of the model. At the same time, the sound data needs to be segmented and divided into smaller segments for model processing.

[0064] Exemplarily, the preprocessed sound data set is divided into a training set, a validation set, and a test set. The deep learning neural network is trained using the training set data and the validation set data. During the training process, the sound emotion model will continuously adjust its parameters to minimize the value of the loss function, thereby learning how to map sound features to corresponding emotion labels.

[0065] Step S1123, identifying the sound emotion label corresponding to the sound data of the target object according to the sound emotion model.

[0066] In the disclosed embodiment, the sound data of the target object to be identified is input into a trained sound emotion model, relevant sound features are extracted, and the corresponding emotion label is obtained through forward propagation of the network.

[0067] Step S1124, obtaining a second emotion sub-entropy value of the target object according to the emotion scores corresponding to different preset sound emotion tags.

[0068] In the disclosed embodiment, the second emotion sub-entropy value of the target object is also obtained through the emotion score table. This value represents the complexity and intensity of the emotion shown by the target object in the current sound data.

[0069] In one embodiment, step S113 includes: constructing a physiological data set and constructing a third deep learning neural network based on physiological characteristics; wherein the physiological data set includes multiple physiological impedance values ​​and a physiological emotion label corresponding to each physiological impedance value; training a physiological emotion model through the third deep learning neural network and the physiological data set; identifying the physiological emotion label corresponding to the physiological data of the target object according to the physiological emotion model; and obtaining the third emotion sub-entropy value of the target object according to the emotion scores corresponding to different preset physiological emotion labels.

[0070] In the disclosed embodiment, a known AC voltage signal is applied to the human body through an instrument and the current flowing through the human body is measured. The impedance value of the human body at this frequency can be calculated through the ratio of voltage and current. Since the impedance characteristics of human tissue under different emotions are different, physiological impedance parameters such as human skin conductance (GSR) can be collected through medical-grade equipment, and the collected physiological impedance values ​​and corresponding emotional labels are organized into a data set to ensure the accuracy and consistency of the data. Considering the time series characteristics of physiological data, RNN is selected as the third deep learning neural network.

[0071] Exemplarily, the RNN neural network is trained using a sorted physiological data set (measured skin conductance data and emotion labels), and the network weights are adjusted through a back-propagation algorithm to minimize the prediction error, ultimately obtaining a physiological emotion model.

[0072] Exemplarily, the real-time physiological data of the target object is processed using the trained physiological emotion model to identify the current physiological emotion label of the target object, and the third emotion sub-entropy value is obtained according to the identified physiological emotion label.

[0073] See also Figure 5 In one embodiment, step S120 includes steps S121-S124.

[0074] Step S121, constructing a brain wave data set and constructing a fourth deep learning neural network based on the brain wave characteristics; wherein the brain wave data set includes multiple brain wave data signals and a brain wave emotion label corresponding to each brain wave data signal.

[0075] In the disclosed embodiment, multiple brain wave data are collected from multiple subjects in different emotional states through an electroencephalogram (EEG) device, and each brain wave data signal will be marked with a corresponding emotion label, including happiness, sadness, anger, calmness, etc. Design a suitable deep learning neural network model. Based on the temporal dynamics and spatial distribution characteristics in the brain wave signal, the fourth deep learning neural network model is a combination of CNN and RNN to extract emotion-related information from the original data.

[0076] Step S122: training a brain wave emotion model using a fourth deep learning neural network and a brain wave data set.

[0077] In the disclosed embodiment, a constructed brain wave data set is used to train a fourth deep learning neural network structure obtained by combining CNN and RNN. During the training process, the input brain wave data is mapped to the corresponding emotion label. Through multiple iterative training, the network parameters are continuously adjusted until the performance of the model on the training set reaches the preset evaluation indicators.

[0078] Step S123, identifying the brain wave emotion label corresponding to the brain wave data of the target object according to the brain wave emotion model.

[0079] In the disclosed embodiment, after the brainwave emotion model training is completed, the brainwave data of the target object is input into the brainwave emotion model, and an emotion label is output.

[0080] Step S124, obtaining a second emotion entropy value of the target object according to the emotion scores corresponding to different preset brain wave emotion tags.

[0081] In the disclosed embodiment, the second emotion entropy value is obtained according to the identified emotion label by using the emotion score table.

[0082] See also Figure 6 In one embodiment, the method further includes steps S610-S630.

[0083] Step S610, obtaining vital sign parameters of the target object and performing normalization processing.

[0084] In the embodiments of the present disclosure, vital sign parameters are important indicators reflecting the health status of the human body, including but not limited to heart rate, blood pressure, respiratory rate, body temperature, blood oxygen saturation, etc. The vital sign parameters of the target object are obtained in real time through medical-grade equipment or wearable devices. Since the measurement units and ranges of different vital sign parameters may be different, in order to be able to compare and calculate on the same scale, the original data is scaled to a specific interval (such as 0 to 1) to eliminate the impact of the dimension.

[0085] Step S620, setting a weight value corresponding to each vital sign parameter, performing weighted summation on the normalized vital sign parameters and the corresponding weight values ​​to obtain a health entropy value.

[0086] In the disclosed embodiment, different vital sign parameters have different importance for health assessment. A corresponding weight value is set for each parameter according to medical knowledge, expert experience or data analysis, and the normalized vital sign parameters are multiplied by their corresponding weight values, and then all products are added to obtain a health entropy value. The health entropy value reflects the current health status of the target object.

[0087] Step S630, classify the health entropy value according to the health assessment criteria to obtain the health status level of the target object, and obtain the comprehensive status level of the target object according to the emotional status level and the health status level, and set the emotional atmosphere scene according to the comprehensive status level.

[0088] In the disclosed embodiment, the health entropy value is divided into different levels according to the preset health assessment criteria. When the health entropy value is 80-100, the patient's current life health is level one health, indicating the best state. When the health entropy value is 60-80, the patient's current life health is level two health, indicating a good state. When the health entropy value is 40-60, the patient's current life health is level three health, indicating a sub-healthy state. When the health entropy value is 20-40, the patient's current life health is level four health, indicating an early stage of disease. When the health entropy value is 0-20, the patient's current life health is level five health, indicating a disease state. The emotional state level obtained previously is combined with the health state level to obtain the comprehensive state level of the target object. The comprehensive state level takes into account both emotional factors and physiological health factors, and is therefore more comprehensive and accurate. According to the comprehensive status level, the target object's environment is adjusted, including displaying vital signs parameters such as heart rate, body temperature, pulse, blood pressure, and health entropy values, so as to monitor the target object's life and health in real time; and the environment is adjusted through color-changing tempered glass. For example, when the emotional entropy value is level 1, it indicates a happy state, and the display is green. When the emotional entropy value is level 2, it indicates a peaceful state, and the display is light green. When the emotional entropy value is level 3, it indicates a tense state, and the display is blue. When the emotional entropy value is level 4, it indicates anxiety, and the display is yellow. When the emotional entropy value is level 5, it indicates fear, and the display is red.

[0089] The data processing method provided by the embodiment of the present disclosure obtains the multimodal emotional data of the target object, calculates the first emotional entropy value according to the multimodal emotional data of the target object; obtains the brain wave data of the target object, calculates the second emotional entropy value according to the brain wave data of the target object, and obtains the target emotional entropy value according to the first emotional entropy value and the second emotional entropy value; and classifies the target emotional entropy value according to the emotional evaluation criteria to obtain the emotional state level of the target object, and sets the emotional atmosphere scene according to the emotional state level. By identifying the emotional labels of the multimodal emotional data and the brain wave data, the emotional score of the target object is obtained more accurately, and the ability to adjust the environmental atmosphere is improved.

[0090] In addition, an embodiment of the present disclosure provides a data processing device, which is applied to an electronic device.

[0091] like Figure 7 As shown, the data processing device 700 includes: a first processing module 701 , a second processing module 702 and a setting module 703 .

[0092] The first processing module 701 is used to obtain multimodal emotion data of the target object; and obtain a first emotion entropy value according to the multimodal emotion data of the target object.

[0093] The second processing module 702 is used to obtain brain wave data of the target object; obtain a second emotion entropy value according to the brain wave data of the target object; and obtain a target emotion entropy value according to the first emotion entropy value and the second emotion entropy value.

[0094] The setting module 703 is used to classify the target emotion entropy value according to the emotion evaluation criteria, obtain the emotion state level of the target object, and set the emotion atmosphere scene according to the emotion state level.

[0095] Optionally, the first processing module 701 shown is also used to obtain a first emotion sub-entropy value based on the facial expression image of the target object, and set a first weight corresponding to the first emotion sub-entropy value; obtain a second emotion sub-entropy value based on the voice of the target object, and set a second weight corresponding to the second emotion sub-entropy value; obtain a third emotion sub-entropy value based on the physiological impedance value of the target object, and set a third weight corresponding to the third emotion sub-entropy value; and perform weighted summation of the first emotion sub-entropy value, the first weight, the second emotion sub-entropy value, the second weight, the third emotion sub-entropy value and the third weight to obtain the first emotion entropy value.

[0096] Optionally, the first processing module 701 shown is also used to construct a facial expression data set and to construct a first deep learning neural network based on facial expression features; wherein the facial expression data set includes multiple facial expression images and facial emotion labels corresponding to each facial expression image; a facial emotion model is trained through the first deep learning neural network and the facial expression data set; according to the facial emotion model, the facial emotion labels corresponding to the facial expression images of the target object are identified; and according to the emotion scores corresponding to the preset different facial emotion labels, the first emotion sub-entropy value of the target object is obtained.

[0097] Optionally, the first processing module 701 shown is also used to construct a sound data set and to construct a second deep learning neural network based on the sound features; wherein the sound data set includes multiple sound data and a sound emotion label corresponding to each sound data; a sound emotion model is trained through the second deep learning neural network and the sound data set; according to the sound emotion model, the sound emotion label corresponding to the sound data of the target object is identified; and according to the emotion scores corresponding to the preset different sound emotion labels, the second emotion sub-entropy value of the target object is obtained.

[0098] Optionally, the first processing module 701 shown is also used to construct a physiological data set and to construct a third deep learning neural network based on physiological characteristics; wherein the physiological data set includes multiple physiological impedance values ​​and a physiological emotion label corresponding to each physiological impedance value; a physiological emotion model is trained through the third deep learning neural network and the physiological data set; according to the physiological emotion model, the physiological emotion label corresponding to the physiological data of the target object is identified; and according to the emotion scores corresponding to the preset different physiological emotion labels, the third emotion sub-entropy value of the target object is obtained.

[0099] Optionally, the second processing module 702 shown is also used to construct a brain wave data set and to construct a fourth deep learning neural network based on brain wave characteristics; wherein the brain wave data set includes multiple brain wave data signals and a brain wave emotion label corresponding to each brain wave data signal; a brain wave emotion model is trained through the fourth deep learning neural network and the brain wave data set; according to the brain wave emotion model, the brain wave emotion label corresponding to the brain wave data of the target object is identified; and according to the emotion scores corresponding to different preset brain wave emotion labels, a second emotion entropy value of the target object is obtained.

[0100] Optionally, the setting module 703 shown is also used to obtain the vital sign parameters of the target object and perform normalization processing; set the weight value corresponding to each vital sign parameter, and perform weighted summation of the normalized vital sign parameter and the corresponding weight value to obtain the health entropy value; and classify the health entropy value according to the health assessment criteria to obtain the health status level of the target object, and obtain the comprehensive status level of the target object according to the emotional state level and the health status level, and set the emotional atmosphere scene according to the comprehensive status level.

[0101] The data processing device 700 provided in the embodiment of the present disclosure can implement the data processing method provided in the above embodiment, and will not be described again here to avoid repetition.

[0102] The data processing device provided by the embodiment of the present disclosure obtains multimodal emotional data of the target object; obtains a first emotional entropy value according to the multimodal emotional data of the target object; obtains brain wave data of the target object; obtains a second emotional entropy value according to the brain wave data of the target object; obtains a target emotional entropy value according to the first emotional entropy value and the second emotional entropy value; and classifies the target emotional entropy value according to the emotional evaluation criteria to obtain the emotional state level of the target object, and sets the emotional atmosphere scene according to the emotional state level. By identifying the emotional labels of the multimodal emotional data and the brain wave data, the emotional score of the target object is obtained more accurately, and the ability to adjust the environmental atmosphere is improved.

[0103] In addition, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the data processing method provided in the above embodiment is executed.

[0104] The electronic device provided in the embodiment of the present disclosure can execute the steps of the data processing method provided in the above method embodiment, which will not be described again here to avoid repetition.

[0105] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data processing method provided in the above embodiment is implemented.

[0106] In the embodiments of the present disclosure, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0107] The computer-readable storage medium provided in the embodiments of the present disclosure can implement the data processing method provided in the above embodiments, which will not be described again here to avoid repetition.

[0108] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.

[0110] The embodiments of the present disclosure are described above in conjunction with the accompanying drawings, but the present disclosure is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present disclosure, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present disclosure and the claims, all of which are within the protection of the present disclosure.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire multimodal emotion data of a target object; obtain a first emotion entropy value according to the multimodal emotion data of the target object; Acquire brain wave data of a target object; obtain a second emotion entropy value according to the brain wave data of the target object; obtain a target emotion entropy value according to the first emotion entropy value and the second emotion entropy value; and The target emotion entropy value is graded according to the emotion evaluation criteria to obtain the emotion state grade of the target object, and the emotion atmosphere scene is set according to the emotion state grade.

2. The method according to claim 1, characterized in that The multimodal emotion data includes a facial expression image, a sound, and a physiological impedance value of the target object, and the first emotion entropy value is calculated according to the multimodal emotion data of the target object, including: Obtaining a first emotion sub-entropy value according to the facial expression image of the target object, and setting a first weight corresponding to the first emotion sub-entropy value; Obtaining a second emotion sub-entropy value according to the voice of the target object, and setting a second weight corresponding to the second emotion sub-entropy value; Obtaining a third emotion sub-entropy value according to the physiological impedance value of the target object, and setting a third weight corresponding to the third emotion sub-entropy value; and The first emotion entropy value is obtained by weighted summing the first emotion sub-entropy value, the first weight, the second emotion sub-entropy value, the second weight, the third emotion sub-entropy value and the third weight.

3. The method according to claim 2, characterized in that The step of obtaining a first emotion sub-entropy value according to the facial expression image of the target object includes: Constructing a facial expression data set and constructing a first deep learning neural network according to facial expression features; wherein the facial expression data set includes a plurality of facial expression images and a facial emotion label corresponding to each facial expression image; Training a facial emotion model using the first deep learning neural network and the facial expression dataset; According to the facial emotion model, identifying the facial emotion label corresponding to the facial expression image of the target object; and According to the emotion scores corresponding to different preset facial emotion labels, a first emotion sub-entropy value of the target object is obtained.

4. The method according to claim 2, characterized in that: The obtaining a second emotion sub-entropy value according to the sound of the target object includes: Constructing a sound data set and constructing a second deep learning neural network according to the sound features; wherein the sound data set includes a plurality of sound data and a sound emotion label corresponding to each sound data; By using the second deep learning neural network and the sound data set, a sound emotion model is obtained by training; According to the sound emotion model, identifying the sound emotion label corresponding to the sound data of the target object; and According to the emotion scores corresponding to the preset different sound emotion tags, a second emotion sub-entropy value of the target object is obtained.

5. The method according to claim 2, characterized in that: The step of obtaining a third emotion sub-entropy value according to the physiological impedance value of the target object includes: Constructing a physiological data set and constructing a third deep learning neural network according to the physiological characteristics; wherein the physiological data set includes a plurality of physiological impedance values ​​and a physiological emotion label corresponding to each physiological impedance value; By using the third deep learning neural network and the physiological data set, a physiological emotion model is obtained through training; According to the physiological emotion model, identifying the physiological emotion label corresponding to the physiological data of the target object; and According to the emotion scores corresponding to the preset different physiological emotion labels, a third emotion sub-entropy value of the target object is obtained.

6. The method according to claim 1, characterized in that The step of obtaining a second emotion entropy value according to the brain wave data of the target object includes: Constructing a brain wave data set and constructing a fourth deep learning neural network according to the brain wave characteristics; wherein the brain wave data set includes a plurality of brain wave data signals and a brain wave emotion label corresponding to each brain wave data signal; By using the fourth deep learning neural network and the brain wave data set, a brain wave emotion model is trained; According to the brain wave emotion model, identifying the brain wave emotion label corresponding to the brain wave data of the target object; and According to the emotion scores corresponding to the preset different brain wave emotion labels, a second emotion entropy value of the target object is obtained.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining vital sign parameters of the target object and performing normalization processing; Setting a weight value corresponding to each vital sign parameter, performing weighted summation on the normalized vital sign parameters and the corresponding weight values ​​to obtain a health entropy value; and The health entropy value is graded according to the health assessment criteria to obtain the health status grade of the target object, and the comprehensive status grade of the target object is obtained according to the emotional status grade and the health status grade, and the emotional atmosphere scene is set according to the comprehensive status grade.

8. A data processing device, characterized in that: The device comprises: A first processing module, configured to obtain multimodal emotion data of a target object; and obtain a first emotion entropy value according to the multimodal emotion data of the target object; a second processing module, configured to obtain brain wave data of a target object; obtain a second emotion entropy value according to the brain wave data of the target object; obtain a target emotion entropy value according to the first emotion entropy value and the second emotion entropy value; and The setting module is used to classify the target emotion entropy value according to the emotion evaluation criteria, obtain the emotion state level of the target object, and set the emotion atmosphere scene according to the emotion state level.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the data processing method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer program stores a computer program, which executes the data processing method according to any one of claims 1 to 7 when running on a processor.

Images