Patient emotion recognition method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a patient emotion recognition method and device, computer equipment and a storage medium. The method is applied to a data processing device in the intelligent dental chair system, the intelligent dental chair system further comprises a camera, a voice acquisition device, a heart rate sensor and a blood pressure sensor, and the method comprises the steps that face change feature information, voice feature information, heart rate information and blood pressure information of a patient are obtained; inputting the face change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; inputting the voice feature information into a preset second patient emotion recognition model to obtain a second emotion vector; inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; finally, the target emotion of the patient is comprehensively determined according to the first emotion vector, the second emotion vector, the third emotion vector and other information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer equipment and storage medium for identifying patient emotions. Background Art

[0002] Emotions are an intuitive reflection and high-level summary of subjective feelings, inner activities, and external behaviors. In the medical field, in order to improve the patient experience, doctors will combine patient emotions for treatment.

[0003] However, patients rarely actively report their current emotions to doctors during treatment, and doctors need to focus on the treatment process and have difficulty observing patients' emotions. Especially during oral treatment, it is more difficult for doctors to obtain patients' emotions because patients are not able to speak. Patients' emotions cannot be reported in a timely manner, which affects their medical experience. There is an urgent need for a method that can automatically identify patients' emotions. Summary of the invention

[0004] The embodiments of the present application provide a patient emotion recognition method, apparatus, computer equipment and storage medium, which can automatically recognize the patient's emotions.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying patient emotions, the method being applied to a data processing device in an intelligent dental chair system, the intelligent dental chair system further comprising a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrests of the intelligent dental chair, the method comprising: Acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; Inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; Inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; Inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; Determining a first similarity between the first emotion vector and the third emotion vector, and determining a second similarity between the second emotion vector and the third emotion vector; Determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; Determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and determine a second target emotion according to the third emotion vector; Determining whether the first target emotion and the second target emotion are in the same emotion set, where the emotion set includes multiple emotions; If the first target emotion and the second target emotion are in the same emotion set, the target vector is determined according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and based on a preset target emotion recognition model, the target emotion of the patient is determined according to the target vector and the third emotion vector.

[0006] In a second aspect, the embodiment of the present application further provides a patient emotion recognition device, which is deployed in a data processing device of an intelligent dental chair system, wherein the intelligent dental chair system further includes a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the intelligent dental chair, and the patient emotion recognition device includes: a first acquisition unit, configured to acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor, respectively; A first input unit, used for inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; A second input unit, used for inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; A third input unit, used for inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; a first determining unit, configured to determine a first similarity between the first emotion vector and the third emotion vector, and to determine a second similarity between the second emotion vector and the third emotion vector; A second determining unit, configured to determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; a third determining unit, configured to determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and to determine a second target emotion according to the third emotion vector; a judging unit, configured to judge whether the first target emotion and the second target emotion are in the same emotion set, wherein the emotion set includes multiple emotions; The fourth determination unit is used to determine the target vector according to the first weight value, the second weight value, the first emotion vector and the second emotion vector when the first target emotion and the second target emotion are in the same emotion set, and determine the target emotion of the patient according to the target vector and the third emotion vector based on a preset target emotion recognition model.

[0007] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and wherein the program instructions can implement the above method when executed by a processor.

[0009] The embodiments of the present application provide a method, device, computer equipment and storage medium for patient emotion recognition. Among them, the method is applied to a data processing device in an intelligent dental chair system, and the intelligent dental chair system also includes a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the intelligent dental chair, and the method includes: obtaining facial change feature information of the patient through the camera; obtaining voice feature information of the patient through the voice acquisition device; obtaining heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; inputting the voice feature information into a preset second patient emotion recognition model to obtain a second emotion vector; inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; determining the first relationship between the first emotion vector and the third emotion vector similarity, and determine the second similarity between the second emotion vector and the third emotion vector; determine the first weight value and the second weight value according to the first similarity, the second similarity and the preset self-attention weight learning rule; determine the first target emotion according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and determine the second target emotion according to the third emotion vector; judge whether the first target emotion and the second target emotion are in the same emotion set, and the emotion set includes multiple emotions; if the first target emotion and the second target emotion are in the same emotion set, determine the target vector according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and determine the target emotion of the patient according to the target vector and the third emotion vector based on the preset target emotion recognition model. In an embodiment of the present application, when a patient lies on a smart dental chair for oral treatment, the patient's target emotion can be automatically identified by a data processing device in the smart dental chair system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of a process for identifying patient emotions provided in an embodiment of the present application; Figure 2 A schematic diagram of a sub-process of a patient emotion recognition method provided in an embodiment of the present application; Figure 3 A schematic block diagram of a patient emotion recognition device provided in an embodiment of the present application; Figure 4 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0014] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0015] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0016] Embodiments of the present application provide a method, apparatus, computer device, and storage medium for identifying patient emotions.

[0017] The executor of the patient emotion recognition method may be the patient emotion recognition device provided in the embodiment of the present application, or a computer device integrated with the patient emotion recognition device, wherein the patient emotion recognition device may be implemented in hardware or software, and the computer device may be a terminal or a server. Specifically, the terminal or the server is a data processing device in the smart dental chair system, and the smart dental chair system includes at least a camera and a voice acquisition device installed above the smart dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the smart dental chair.

[0018] Figure 1 is a flow chart of the patient emotion recognition method provided in the embodiment of the present application. Figure 1 As shown, the method includes the following steps S110-S190.

[0019] S110, acquiring facial change feature information of the patient through the camera, acquiring voice feature information of the patient through the voice acquisition device, and acquiring heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively.

[0020] In some embodiments, see Figure 2 , specifically, the facial change feature information of the patient is obtained through the following steps: S1101. Acquire the current facial video information of the patient through the camera.

[0021] S1102. Extracting current shape features of key face parts from the current face video information, where the key face parts include nose, eyes, eyebrows, mouth, cheeks and forehead.

[0022] Specifically, the current face video information is collected in real time, and the key parts of the face are identified from the collected current face video information, and the current shape features of each key part of the face are extracted from the video.

[0023] S1103, obtaining the facial change feature information by analyzing the current shape feature and the static shape feature, wherein the static shape feature is the shape feature of the key parts of the face collected in advance when the patient is in a calm state.

[0024] Among them, the static shape feature is the static shape feature collected by the camera when the patient is in a calm state when lying on the smart dental chair before treatment.

[0025] The facial change feature information includes nose change features, eye change features, eyebrow change features, mouth change features, cheek change features and forehead change features. The facial change feature information includes the change amplitude and change direction of each key part of the face. Specifically, the nose change feature includes the nose change amplitude and change direction, the eye change feature includes the eye change amplitude and change direction, the eyebrow change feature includes the eyebrow change amplitude and change direction, the mouth change feature includes the mouth change amplitude and change direction, the cheek change feature includes the cheek change amplitude and change direction, and the forehead change feature includes the forehead change amplitude and change direction.

[0026] In this embodiment, although the patient is undergoing oral treatment and is not convenient to speak, he can make sounds. At this time, the voice collection device will collect these sounds, and the data processing device will obtain the voice feature information of the collected sounds through voice recognition technology.

[0027] In addition, the voice recognition device can also record conversations between patients and doctors before, during and after treatment.

[0028] Furthermore, considering that the heart rate and blood pressure of different patients in a calm state are also different, at this time, it is necessary to obtain the patient's calm heart rate data and calm blood pressure data in a calm state. The heart rate information indicates the difference between the current heart rate data and the calm heart rate data, and the blood pressure information indicates the difference between the current blood pressure data and the calm blood pressure data.

[0029] S120: Input the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector.

[0030] In this embodiment, the first patient emotion recognition model is a pre-trained deep learning model used to identify different emotion vectors according to different facial change feature information.

[0031] S130: Input the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector.

[0032] In this embodiment, the second patient emotion recognition model is a pre-trained deep learning model used to recognize different emotion vectors according to different speech feature information.

[0033] S140: Input the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector.

[0034] In this embodiment, the third patient emotion recognition model is a pre-trained deep learning model used to identify different emotion vectors according to different heart rate information and blood pressure information.

[0035] S150: Determine a first similarity between the first emotion vector and the third emotion vector, and determine a second similarity between the second emotion vector and the third emotion vector.

[0036] In this embodiment, since the emotions obtained through video and audio can be highly disguised and the authenticity of the obtained emotion values ​​is relatively weak, while the emotions obtained through heart rate and blood pressure are more authentic, this embodiment will dynamically allocate the weights of the first emotion value and the second emotion value based on the similarity with the third emotion vector.

[0037] Specifically, in this embodiment, the first similarity and the second similarity are cosine similarities. In this embodiment, the calculation logic of the first similarity and the second similarity can be set in the self-attention weight learning rule.

[0038] S160. Determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule.

[0039] Specifically, the first weight value is determined according to the first weight calculation formula, and the first weight calculation formula is: W1= exp(a1) / [ exp(a1) + exp(a2)]; The second weight value is determined according to the second weight calculation formula, and the second weight calculation formula is: W2= exp(a2) / [ exp(a1) + exp(a2)]; Among them, W1 represents the first weight value, W2 represents the second weight value, a1 represents the first similarity, and a2 represents the second similarity.

[0040] In other embodiments, the present application can also determine a weight sequence corresponding to the current treatment step according to a preset weight allocation rule, wherein the weight sequence includes feature weight values ​​corresponding to each feature, and the first weight value and the second weight value are updated according to the corresponding feature weight values. Specifically, the corresponding feature weight value can be added to the first weight value to update the first weight value, and the step of updating the second weight value is similar to the step of updating the first weight value, and the details are not repeated here.

[0041] Specifically, the current treatment step is the treatment step currently being performed in the target treatment plan, and the target treatment plan includes multiple treatment steps. This plan sets different weight sequences for different treatment steps. For example, some steps have relatively loud noises, and it is difficult to discern the patient's emotions through sound. At this time, it is necessary to reduce the weight corresponding to the second emotion vector. For example, the second weight value is set to 0 at this time.

[0042] The current treatment step is determined by: The current treatment tool information and the current mouth feature information of the patient are obtained through the camera; the current treatment step is determined according to the current treatment tool information, the current mouth feature information and the treatment plan corresponding to the patient.

[0043] Specifically, the current treatment step can be obtained by analyzing the current treatment tool and the patient's current mouth shape.

[0044] In other embodiments, the current treatment step may also be determined based on the doctor's input data.

[0045] S170. Determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and determine a second target emotion according to the third emotion vector.

[0046] Specifically, a target vector is determined according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector; and the first target emotion corresponding to the target vector is determined according to a preset correspondence between vectors and emotions.

[0047] The first weight value is multiplied by the first emotion vector to obtain a first vector, the second weight value is multiplied by the second emotion vector to obtain a second vector, and the first vector and the second vector are added to obtain a target vector.

[0048] S180: Determine whether the first target emotion and the second target emotion are in the same emotion set, where the emotion set includes multiple emotions.

[0049] In this embodiment, multiple different emotions are preset, such as emotion 1, emotion 2, emotion 3, emotion 4, emotion 5 and emotion 6, and the multiple different emotions are divided into set a and set b according to the similarity of the emotions. For example, set a is (emotion 1, emotion 2, emotion 3) and set b is (emotion 4, emotion 5, emotion 6).

[0050] If the first target emotion and the second target emotion are in the same emotion set, it indicates that the first target emotion is authentic; otherwise, the first target emotion is determined to be a disguised emotion. If it is a disguised emotion, emotion recognition is not performed for this period, or the target emotion is calculated only according to the third emotion vector.

[0051] S190. If the first target emotion and the second target emotion are in the same emotion set, the target vector is determined according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and based on a preset target emotion recognition model, the target emotion of the patient is determined according to the target vector and the third emotion vector.

[0052] Specifically, the first weight value is multiplied by the first emotion vector to obtain a first vector, the second weight value is multiplied by the second emotion vector to obtain a second vector, and the first vector and the second vector are added to obtain a target vector.

[0053] In some embodiments, based on a preset target emotion recognition model, determining the target emotion of the patient according to the target vector and the third emotion vector includes: The probability of each preset emotion is calculated based on a preset emotion recognition algorithm, and the preset emotion with the greatest probability is determined as the target emotion, wherein the emotion recognition algorithm is: ; in, is the probability of the current preset emotion, the current recognition period includes n data acquisition moments, i is the i-th data acquisition moment in the current recognition period, To obtain the probability value of according to the target vector at the i-th moment, is the probability value obtained according to the third emotion vector at the i-th moment.

[0054] Furthermore, the smart dental chair system further includes a first pressure sensor installed on a seat cushion of the smart dental chair and a second pressure sensor installed on a backrest of the smart dental chair; before determining the first target emotion according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, the method further includes: Acquire a first pressure feature through the first pressure sensor, and acquire a second pressure feature through the second pressure sensor; input the first pressure feature and the second pressure feature into a preset fourth patient emotion recognition model to obtain a fourth emotion vector; determine a third similarity between the fourth emotion vector and the third emotion vector; At this time, the determining the first weight value and the second weight value according to the first similarity, the second similarity and the preset self-attention weight learning rule includes: determining the first weight value, the second weight value and the third weight value according to the first similarity, the second similarity, the third similarity and the preset self-attention weight learning rule; The determining the first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector comprises: determining the first target emotion according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector; Determining the target vector based on the first weight value, the second weight value, the first emotion vector and the second emotion vector includes: determining the target vector based on the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector and the fourth emotion vector.

[0055] Among them, the fourth patient emotion recognition model is a pre-trained deep learning model used to identify different emotion vectors according to different first stress characteristics and second stress characteristics.

[0056] In this embodiment, the first pressure value and the second pressure value are correlated and analyzed by the fourth patient emotion recognition model to obtain the comfort level of the patient lying on the smart dental chair at this time. Different comfort levels are associated with different emotion vectors, thereby obtaining a fourth emotion vector.

[0057] Furthermore, since different hand movements can also reflect different emotions of the patient, for example, if the patient clenches his fist tightly, it means that the patient is nervous or in high pain, and if the hand is in a relaxed state, it means that the patient is also in a relaxed mood. At this time, before determining the target vector according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector, the method further includes: Acquire the hand motion features of the patient through the camera; input the hand motion features into a preset fifth patient emotion recognition model to obtain a fifth emotion vector; determine a fourth similarity between the fifth emotion vector and the third emotion vector; Determining a first weight value, a second weight value, and a third weight value according to the first similarity, the second similarity, the third similarity, and a preset self-attention weight learning rule, including: determining a first weight value, a second weight value, a third weight value, and a fourth weight value according to the first similarity, the second similarity, the third similarity, the fourth similarity, and a preset self-attention weight learning rule; The determining the first target emotion according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector comprises: determining the first target emotion according to the first weight value, the second weight value, the third weight value, the fourth weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector; The determining of the target vector based on the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector and the fourth emotion vector includes: determining the target vector based on the first weight value, the second weight value, the third weight value, the fourth weight value, the first emotion vector, the second emotion vector, the fourth emotion vector and the fifth emotion vector.

[0058] Among them, the fifth patient emotion recognition model is a pre-trained deep learning model used to identify different emotion vectors based on different hand movement features.

[0059] At this time, the target emotion vector=the first emotion vector×the first weight value+the second emotion vector×the second weight value+the third emotion vector×the third weight value+the fourth emotion vector×the fourth weight value+the fifth emotion vector×the fifth weight value.

[0060] Further, in some embodiments, after determining the target emotion of the patient, the method further comprises: A treatment suggestion is generated according to the target emotion, the target treatment plan corresponding to the patient, and the current treatment step, wherein the treatment suggestion includes an anesthetic dose adjustment suggestion and a treatment rhythm adjustment suggestion.

[0061] For example, the treatment suggestion is displayed on a display screen and a prompt sound is emitted to remind the doctor to check the treatment suggestion. Through this embodiment, the doctor can better adjust the treatment strategy and improve the patient experience.

[0062] Further, in some embodiments, the current treatment step is a treatment step currently being performed in a target treatment plan, and the target treatment plan includes a plurality of treatment steps; after determining the target emotion of the patient according to the target emotion vector, the method further includes: Storing the target emotion in an emotion library corresponding to the patient, and associating the target emotion with the current treatment step; When a treatment end instruction is received, the patient emotions associated with each treatment step are obtained from the emotion library to obtain a set of patient emotions corresponding to each treatment step; According to the patient emotion sets corresponding to each treatment step, the patient comfort score of each treatment step is determined respectively.

[0063] Specifically, the patient emotion set includes multiple emotions, and different emotions correspond to different emotion scores. This embodiment determines the patient comfort score according to the average value of the emotion score of each emotion in the patient emotion set. For example, the patient emotion set of a certain treatment step is (comfortable, comfortable, comfortable, uncomfortable, extremely uncomfortable), the emotion score corresponding to comfort is 10 points, the emotion score corresponding to discomfort is 5 points, and the emotion score corresponding to extreme discomfort is 1 point. At this time, the patient comfort score corresponding to the treatment step = (10+10+10+5+1) ÷ 5 = 7.2 points.

[0064] The patient comfort score of each treatment step can be used to analyze the patient's comfort level at different treatment steps.

[0065] To summarize, in the embodiment of the present application, when a patient lies on a smart dental chair for oral treatment, the patient's target emotion can be automatically identified by the data processing device in the smart dental chair system.

[0066] Figure 3is a schematic block diagram of a patient emotion recognition device 300 provided in an embodiment of the present application. Figure 3 As shown, corresponding to the above patient emotion recognition method, the present application also provides a patient emotion recognition device 300. The patient emotion recognition device 300 includes a unit for executing the above patient emotion recognition method. The patient emotion recognition device 300 can be configured in a data processing device of an intelligent dental chair system. The intelligent dental chair system also includes a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the intelligent dental chair. For details, please refer to Figure 3 The patient emotion recognition device 300 includes a first acquisition unit 301, a first input unit 302, a second input unit 303, a third input unit 304, a first determination unit 305, a second determination unit 306, a third determination unit 307, a judgment unit 308 and a fourth determination unit 309.

[0067] The first acquisition unit 301 is used to acquire the facial change feature information of the patient through the camera, acquire the voice feature information of the patient through the voice acquisition device, and acquire the heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; A first input unit 302, used to input the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; The second input unit 303 is used to input the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; A third input unit 304, used to input the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; A first determining unit 305, configured to determine a first similarity between the first emotion vector and the third emotion vector, and to determine a second similarity between the second emotion vector and the third emotion vector; A second determining unit 306, configured to determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; a third determining unit 307, configured to determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and to determine a second target emotion according to the third emotion vector; A judging unit 308, configured to judge whether the first target emotion and the second target emotion are in the same emotion set, wherein the emotion set includes multiple emotions; The fourth determination unit 309 is used to determine the target vector according to the first weight value, the second weight value, the first emotion vector and the second emotion vector when the first target emotion and the second target emotion are in the same emotion set, and determine the target emotion of the patient according to the target vector and the third emotion vector based on a preset target emotion recognition model.

[0068] To summarize, in the embodiment of the present application, when a patient lies on a smart dental chair for oral treatment, the patient's target emotion can be automatically identified by the data processing device in the smart dental chair system.

[0069] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned patient emotion recognition device and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0070] The patient emotion recognition device can be implemented in the form of a computer program. Figure 4 Runs on the computer device shown.

[0071] See also Figure 4 , Figure 4 400 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 400 is a data processing device in an intelligent dental chair system, and the intelligent dental chair system also includes a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the intelligent dental chair.

[0072] See also Figure 4 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a non-volatile storage medium 403 and an internal memory 404 .

[0073] The non-volatile storage medium 403 may store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, and when the program instructions are executed, the processor 402 may execute a patient emotion recognition method.

[0074] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device 400 .

[0075] The internal memory 404 provides an environment for the operation of the computer program 4032 in the non-volatile storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute a patient emotion recognition method.

[0076] The network interface 405 is used to communicate with other devices over the network. Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 400 to which the solution of the present application is applied. The specific computer device 400 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0077] The processor 402 is used to run the computer program 4032 stored in the memory to implement the following steps: Acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; Inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; Inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; Inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; Determining a first similarity between the first emotion vector and the third emotion vector, and determining a second similarity between the second emotion vector and the third emotion vector; Determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; Determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and determine a second target emotion according to the third emotion vector; Determining whether the first target emotion and the second target emotion are in the same emotion set, where the emotion set includes multiple emotions; If the first target emotion and the second target emotion are in the same emotion set, the target vector is determined according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and based on a preset target emotion recognition model, the target emotion of the patient is determined according to the target vector and the third emotion vector.

[0078] It should be understood that in the embodiment of the present application, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0079] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0080] Therefore, the present application also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps: Acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; Inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; Inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; Inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; Determining a first similarity between the first emotion vector and the third emotion vector, and determining a second similarity between the second emotion vector and the third emotion vector; Determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; Determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and determine a second target emotion according to the third emotion vector; Determining whether the first target emotion and the second target emotion are in the same emotion set, where the emotion set includes multiple emotions; If the first target emotion and the second target emotion are in the same emotion set, the target vector is determined according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and based on a preset target emotion recognition model, the target emotion of the patient is determined according to the target vector and the third emotion vector.

[0081] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0082] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0083] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0084] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0086] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A patient emotion recognition method, characterized in that: The method is applied to a data processing device in an intelligent dental chair system, wherein the intelligent dental chair system further includes a camera and a voice acquisition device installed above the intelligent dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the intelligent dental chair, and the method includes: Acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor respectively; Inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; Inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; Inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; Determining a first similarity between the first emotion vector and the third emotion vector, and determining a second similarity between the second emotion vector and the third emotion vector; Determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; Determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and determine a second target emotion according to the third emotion vector; Determining whether the first target emotion and the second target emotion are in the same emotion set, where the emotion set includes multiple emotions; If the first target emotion and the second target emotion are in the same emotion set, the target vector is determined according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, and based on a preset target emotion recognition model, the target emotion of the patient is determined according to the target vector and the third emotion vector.

2. The method according to claim 1, characterized in that: The step of obtaining the facial change feature information of the patient through the camera includes: Acquiring the current facial video information of the patient through the camera; Extracting current shape features of key parts of the face from the current face video information, wherein the key parts of the face include nose, eyes, eyebrows, mouth, cheeks and forehead; The facial change feature information is obtained by analyzing the current shape feature and the static shape feature, wherein the static shape feature is the shape feature of the key part of the face collected in advance when the patient is in a calm state.

3. The method according to claim 1, characterized in that: The determining a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector includes: Determine a target vector according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector; The first target emotion corresponding to the target vector is determined according to a preset correspondence relationship between the vector and the emotion.

4. The method according to claim 1, characterized in that The smart dental chair system further includes a first pressure sensor installed on a seat cushion of the smart dental chair and a second pressure sensor installed on a backrest of the smart dental chair; before determining the first target emotion according to the first weight value, the second weight value, the first emotion vector and the second emotion vector, the method further includes: Acquire a first pressure characteristic through the first pressure sensor, and acquire a second pressure characteristic through the second pressure sensor; Inputting the first pressure feature and the second pressure feature into a preset fourth patient emotion recognition model to obtain a fourth emotion vector; determining a third similarity between the fourth emotion vector and the third emotion vector; The determining a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule includes: Determine a first weight value, a second weight value, and a third weight value according to the first similarity, the second similarity, the third similarity, and a preset self-attention weight learning rule; The determining a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector includes: Determine the first target emotion according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector; The determining a target vector according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector includes: A target vector is determined according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector.

5. The method according to claim 4, characterized in that Before determining the target vector according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector, the method further includes: Acquiring hand movement characteristics of the patient through the camera; Inputting the hand motion feature into a preset fifth patient emotion recognition model to obtain a fifth emotion vector; determining a fourth similarity between the fifth emotion vector and the third emotion vector; Determining a first weight value, a second weight value, and a third weight value according to the first similarity, the second similarity, the third similarity, and a preset self-attention weight learning rule, including: Determine a first weight value, a second weight value, a third weight value, and a fourth weight value according to the first similarity, the second similarity, the third similarity, the fourth similarity, and a preset self-attention weight learning rule; The determining the first target emotion according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector includes: Determine the first target emotion according to the first weight value, the second weight value, the third weight value, the fourth weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector; The determining a target vector according to the first weight value, the second weight value, the third weight value, the first emotion vector, the second emotion vector, and the fourth emotion vector comprises: The target vector is determined according to the first weight value, the second weight value, the third weight value, the fourth weight value, the first emotion vector, the second emotion vector, the fourth emotion vector, and the fifth emotion vector.

6. The method according to claim 1, characterized in that After determining the target emotion of the patient according to the target vector and the third emotion vector, the method further includes: A treatment suggestion is generated according to the target emotion, the target treatment plan corresponding to the patient, and the current treatment steps, wherein the treatment suggestion includes an anesthetic dose adjustment suggestion and a treatment rhythm adjustment suggestion.

7. The method according to claim 6, characterized in that The current treatment step is a treatment step currently being performed in a target treatment plan, and the target treatment plan includes a plurality of treatment steps; after determining the target emotion of the patient according to the target vector and the third emotion vector, the method further includes: Storing the target emotion in an emotion library corresponding to the patient, and associating the target emotion with the current treatment step; When a treatment end instruction is received, the patient emotions associated with each treatment step are obtained from the emotion library to obtain a set of patient emotions corresponding to each treatment step; According to the patient emotion sets corresponding to each treatment step, the patient comfort score of each treatment step is determined respectively.

8. A patient emotion recognition device, characterized in that: The patient emotion recognition device is deployed in the data processing device of the smart dental chair system. The smart dental chair system also includes a camera and a voice acquisition device installed above the smart dental chair, a heart rate sensor and a blood pressure sensor installed on the armrest of the smart dental chair. The patient emotion recognition device includes: a first acquisition unit, configured to acquire facial change feature information of the patient through the camera, acquire voice feature information of the patient through the voice acquisition device, and acquire heart rate information and blood pressure information of the patient through the heart rate sensor and the blood pressure sensor, respectively; A first input unit, used for inputting the facial change feature information into a preset first patient emotion recognition model to obtain a first emotion vector; A second input unit, used for inputting the speech feature information into a preset second patient emotion recognition model to obtain a second emotion vector; A third input unit, used for inputting the heart rate information and the blood pressure information into a preset third patient emotion recognition model to obtain a third emotion vector; a first determining unit, configured to determine a first similarity between the first emotion vector and the third emotion vector, and to determine a second similarity between the second emotion vector and the third emotion vector; A second determining unit, configured to determine a first weight value and a second weight value according to the first similarity, the second similarity and a preset self-attention weight learning rule; a third determining unit, configured to determine a first target emotion according to the first weight value, the second weight value, the first emotion vector, and the second emotion vector, and to determine a second target emotion according to the third emotion vector; a judging unit, configured to judge whether the first target emotion and the second target emotion are in the same emotion set, wherein the emotion set includes multiple emotions; The fourth determination unit is used to determine the target vector according to the first weight value, the second weight value, the first emotion vector and the second emotion vector when the first target emotion and the second target emotion are in the same emotion set, and determine the target emotion of the patient according to the target vector and the third emotion vector based on a preset target emotion recognition model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The computer device is a data processing device in the smart dental chair system, and the smart dental chair system also includes a camera and a voice acquisition device installed above the smart dental chair, a heart rate sensor and a blood pressure sensor installed on the armrests of the smart dental chair. When the processor executes the computer program, it implements the patient emotion recognition method described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the patient emotion recognition method as described in any one of claims 1 to 7.

Images