Emotion recognition method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a kind of mood recognition method, device, electronic equipment and computer readable storage medium, wherein method includes: when virtual scene is in mood recognition mode, judge whether the virtual character in the virtual scene is in the visible state of camera;If the virtual character is in the visible state of camera, read the target identification information associated with the virtual character;The target emotion type corresponding to the target identification information is determined as the emotion type of the virtual character.This way, without spending a lot of cost in advance to train neural network model with emotion recognition function, for the emotion recognition of the virtual character of virtual scene, read the emotion type recorded in advance to confirm the real-time emotion type of virtual character faster.This way, the emotion recognition of virtual character can be realized at lower cost and efficiently, it is convenient to use the emotion identified in simulation to do more personalized docking service.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of simulation, and in particular to an emotion recognition method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] Emotion recognition based on facial expressions is very important in intelligent robots, intelligent voice dialog, and other scenarios, and is conducive to recognizing the emotions of users to provide timely psychological comfort and other measures. For emotion recognition based on facial expression recognition, existing technologies use a neural network-based model recognition method, that is, a large number of facial expression samples of different emotions are obtained using artificial intelligence to train a facial expression recognition model, and the intelligent model with facial expression recognition capability trained is used to recognize user emotions. However, this recognition is inefficient and costly for simulation projects.

[0003] Therefore, there is an urgent need for a scheme for emotion recognition in simulation projects at a lower cost. SUMMARY

[0004] To solve the above technical problems, the embodiments of the present application provide an emotion recognition method, device, electronic device, and computer readable storage medium.

[0005] In a first aspect, the embodiments of the present application provide an emotion recognition method, which comprises:

[0006] If the virtual scene is in an emotion recognition mode, it is determined whether a virtual character in the virtual scene is in a visible state of a camera.

[0007] If the virtual character is in the visible state of the camera, target identification information associated with the virtual character is read.

[0008] A target emotion type corresponding to the target identification information is determined as an emotion type of the virtual character.

[0009] According to a specific embodiment of the present application, before the step of determining whether the virtual character in the virtual scene is in the visible state of the camera, the method further comprises:

[0010] Pre-made facial expression display resources of multiple types of emotions are prepared.

[0011] Identification information of various types of emotions and facial expression display resources are associated and stored.

[0012] Available emotion types are configured for the virtual character.

[0013] According to a target emotion type required by the virtual character to be displayed in real time, an expression display resource corresponding to the target emotion type is called to realize expression display of the virtual character, wherein the target emotion type is any one of available emotion types of the virtual character.

[0014] According to an embodiment of the present application, the step of determining whether the virtual character in the virtual scene is in the visible state of the camera includes:

[0015] determining whether the real-time distance between the camera and the virtual character is within a preset distance range;

[0016] If the real-time distance between the camera and the virtual character is within the preset distance range, it is determined that the virtual character is in the visible state of the camera.

[0017] According to an embodiment of the present application, the step of determining whether the real-time distance between the camera and the virtual character is within a preset distance range includes:

[0018] determining whether the face-enclosing sphere of the virtual character is between the near clipping plane and the far clipping plane of the camera;

[0019] If the face-enclosing sphere of the virtual character is between the near clipping plane and the far clipping plane of the camera, it is determined that the camera and the face-enclosing sphere of the virtual character are within the preset distance range.

[0020] According to an embodiment of the present application, before the step of determining whether the real-time distance between the camera and the virtual character is within a preset distance range, the method further includes:

[0021] determining whether the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy an intersection state;

[0022] If the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state, the step of determining whether the real-time distance between the camera and the virtual character is within a preset distance range is performed;

[0023] If the view frustum of the camera and the face-enclosing sphere of the virtual character do not satisfy the intersection state, it is directly determined that the virtual character in the virtual scene is in the invisible state of the camera.

[0024] According to an embodiment of the present application, the step of determining whether the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy an intersection state includes:

[0025] An octree is used to perform a space acceleration algorithm to determine whether the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state.

[0026] According to an embodiment of the present application, before the step of determining the target emotion type corresponding to the target identification information as the emotion type of the virtual character, the method further comprises:

[0027] verifying the target emotion type corresponding to the target identification information according to the actual facial image of the virtual character;

[0028] if the verification is passed, performing the step of determining the target emotion type as the emotion type of the virtual character;

[0029] if the verification is not passed, feeding back the indication information of emotion recognition failure of the virtual character.

[0030] In a second aspect, the embodiments of the present application further provide an emotion recognition device, which comprises:

[0031] a judging module, configured to judge whether a virtual character in a virtual scene is in a visible state of a camera if the virtual scene is in an emotion recognition mode;

[0032] a reading module, configured to read target identification information associated with the virtual character if the virtual character is in the visible state of the camera;

[0033] a determining module, configured to determine a target emotion type corresponding to the target identification information as an emotion type of the virtual character.

[0034] According to an embodiment of the present application, the emotion recognition device further comprises a configuration module, configured to:

[0035] pre-produce expression display resources of multiple types of emotions;

[0036] store identification information of various types of emotions and the expression display resources in association;

[0037] configure available emotion types for the virtual character;

[0038] according to a target emotion type required by the virtual character to be displayed in real time, call expression display resources corresponding to the target emotion type to realize expression display of the virtual character, wherein the target emotion type is any type in the available emotion types of the virtual character.

[0039] According to an embodiment of the present application, the judging module is configured to:

[0040] judge whether a real-time distance between the camera and the virtual character is within a preset distance range;

[0041] If the real-time distance between the camera and the virtual role is within the preset distance range, it is determined that the virtual role is in a visible state of the camera.

[0042] According to an embodiment of the present application, the determining module is configured to:

[0043] determine whether the face bounding sphere of the virtual role is between the near clipping plane and the far clipping plane of the camera;

[0044] If the face bounding sphere of the virtual role is between the near clipping plane and the far clipping plane of the camera, it is determined that the camera and the face bounding sphere of the virtual role are within the preset distance range.

[0045] According to an embodiment of the present application, the determining module is configured to:

[0046] determine whether the frustum of the camera and the face bounding sphere of the virtual role satisfy an intersection state;

[0047] If the frustum of the camera and the face bounding sphere of the virtual role satisfy the intersection state, the step of determining whether the real-time distance between the camera and the virtual role is within the preset distance range is performed.

[0048] If the frustum of the camera and the face bounding sphere of the virtual role do not satisfy the intersection state, it is directly determined that the virtual role in the virtual scene is in an invisible state of the camera.

[0049] According to an embodiment of the present application, the determining module is configured to:

[0050] perform a space acceleration algorithm by an octree to determine whether the frustum of the camera and the face bounding sphere of the virtual role satisfy the intersection state.

[0051] According to an embodiment of the present application, the determining module is configured to:

[0052] verify the target emotion type corresponding to the target identification information according to the actual face image of the virtual role;

[0053] If the verification is passed, the step of determining the target emotion type as the emotion type of the virtual role is performed.

[0054] If the verification is not passed, an indication information of emotion recognition failure for the virtual role is fed back.

[0055] In a third aspect, the embodiments of the present application also provide an electronic device, including a memory and a processor, the memory stores a computer program, and the computer program performs the emotion recognition method in any one of the first aspect when the processor runs.

[0056] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium storing a computer program, which, when executed on a processor, performs the mood recognition method of any one of the first aspect.

[0057] The mood recognition method, device, electronic device and computer readable storage medium provided by the present application can determine whether a virtual character in a virtual scene is in a visible state of a camera when the virtual scene is in a mood recognition mode; read target identification information associated with the virtual character if the virtual character is in the visible state of the camera; and determine a target mood type corresponding to the target identification information as a mood type of the virtual character. In this way, without pre-training a neural network model with mood recognition function at a large cost, the mood recognition of the virtual character in the virtual scene can read a pre-recorded mood type to quickly confirm the real-time mood type of the virtual character. In this way, the mood recognition of the virtual character can be realized at a low cost and efficiently, and the recognized mood can be used for more personalized docking services in simulation. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of protection of the present application. In the various drawings, similar components are denoted by similar reference numerals.

[0059] Figure 1 A flowchart of a mood recognition method provided by an embodiment of the present application is shown;

[0060] Figure 2 An emotion type switching diagram related to the mood recognition method provided by an embodiment of the present application is shown;

[0061] Figure 3 An intersection diagram of a camera view cone and a virtual character related to the mood recognition method provided by an embodiment of the present application is shown;

[0062] Figure 4 A structural diagram of a mood recognition device provided by an embodiment of the present application is shown;

[0063] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0064] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all.

[0065] The components of the application embodiments generally described and illustrated in the figures herein can be arranged and designed in a wide variety of different configurations. Therefore, the following detailed description of the application, as represented in the figures, is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. The described embodiments of the application are to be considered in a descriptive sense only and not for purposes of limitation. Based on the embodiments of the application, those skilled in the art will appreciate that many equivalents can be made without departing from the scope of the application.

[0066] Hereinafter, the terms "include", "have", and their conjugates, used in the various embodiments of the application, merely indicate the presence of the features, numbers, steps, operations, elements, components, or combinations thereof, and do not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof.

[0067] In addition, the terms "first", "second", "third", and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0068] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the application belong. The terms (such as terms defined in a generally used dictionary) will be interpreted as having the same meaning as the context in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the application.

[0069] Embodiment 1

[0070] Referring to Figure 1 , a flowchart of an emotion recognition method provided by the embodiments of the application is shown. As shown in Figure 1 , the method mainly includes the following steps:

[0071] Step S101, if the virtual scene is in the emotion recognition mode, it is judged whether the virtual character in the virtual scene is in the visible state of the camera;

[0072] The emotion recognition method provided in the embodiments of the present application is applied to a virtual scene. The virtual scene is a virtual simulation scene built based on simulation software, and the virtual scene has space, buildings and virtual roles. The number of virtual roles can be one or more. The virtual role is a virtual person simulating a person object in a real scene. In order to provide better care or dialogue service for the virtual role, the virtual scene can be set to switch to an emotion recognition mode. In the emotion recognition mode, the emotion types of the virtual roles in the virtual scene can be recognized.

[0073] It should be noted that the execution subject of the scheme provided in the embodiments of the present application can be the device itself for building the virtual scene, or can be specifically limited to a camera in the virtual scene, that is, the emotion type of the virtual role in the visible range is recognized in the angle of the camera for collecting the view of a specific area in the virtual scene. In this case, to be closer to the actual scene, if the virtual role is to be recognized for emotion, the virtual role needs to be limited to the visible state of the camera, that is, the camera can collect the facial features of the virtual role.

[0074] According to a specific embodiment of the present application, before the step of judging whether the virtual role in the virtual scene is in the visible state of the camera, the method further comprises:

[0075] Preparation of expression display resources of various types of emotions;

[0076] Association and storage of identification information of various types of emotions and expression display resources;

[0077] Configuration of available emotion types for the virtual role;

[0078] According to the target emotion type that the virtual role needs to display in real time, the expression display resource corresponding to the target emotion type is called to realize the expression display of the virtual role, wherein the target emotion type is any one of the available emotion types of the virtual role.

[0079] Before emotion recognition, the virtual role needs to be configured with displayable emotion types and expression display resources, that is, the virtual role can display multiple expressions. The specific process is as follows:

[0080] First, pre-create facial expression display resources for different types of emotions. There are many common methods for creating facial expression display resources. For example, one method is to continuously switch the facial texture of the virtual character, similar to GIF (Graphics Interchange Format), to display indexed color images in Hypertext Markup Language. Alternatively, one can use a scheme called BlendShape for blending deformations; see the general usage examples for details, which will not be elaborated further.

[0081] Secondly, the created emoji display resources are associated with and stored along with the corresponding emotion types.

[0082] The data can be stored in a configuration table, as shown in Table 1, which lists various emotions.

[0083] Table 1

[0084] Expression ID Expression display resource Emotion type 1 ModelA_Facial Expression0.asset Expressionless 2 ModelA_Facial Expression1.asset Whispering cry 3 ModelA_Facial Expression2.asset Wailing cry 4 ModelA_Facial Expression3.asset A little sad 5 ModelA_Facial Expression4.asset Dumb smile 6 ModelA_Facial Expression5.asset Mad laugh

[0085] Once the emoji display resources are created, configure a table to associate emoji IDs with their corresponding emoji resources and emotional states. Alternatively, you can directly use the emoji ID without configuring the emotional state separately. In this case, the ID should highlight the uniqueness of the emotional state; for example, the crying ID should be separated into "crying softly" (ID string) and "crying loudly" (ID string). If there is a separate emotional state, then the ID can be any type of expression; the emotional type can help provide unique distinguishing information.

[0086] Next, the available emotion types are configured for the virtual character to enable the display of the virtual character's expressions.

[0087] The system switches the facial animations applied to the virtual character based on different emotion IDs. For example... Figure 2 As shown, the virtual character can be controlled to switch between various types of emotional expressions. A member variable is created for the virtual character to store the expression ID. Then, an emotion state variable is created for the virtual character to record the currently playing expression ID. When the virtual character's face enters the recognition process, it can be queried and recognized; the expression ID variable is read during recognition.

[0088] Define the emotion type that the virtual character needs to display in real time as the target emotion type. Then, when the target emotion type needs to be displayed, the expression display resource corresponding to the target emotion type can be called to realize the expression display of the virtual character. The target emotion type can be any of the available emotion types of the virtual character. At the same time, store the target identification information corresponding to the target emotion type, such as the expression ID, so that the target identification information can be directly read when identifying the real-time emotion type of the virtual character.

[0089] Step S102: If the virtual character is visible to the camera, then read the target identification information associated with the virtual character;

[0090] The electronic device determines whether the virtual character is visible to the camera, triggering the emotion recognition process only when the virtual character is visible. As the virtual character displays the target emotion type in real time, it simultaneously records the target identifier information corresponding to that emotion type. Therefore, when the specific triggering process is initiated, the target identifier information associated with the virtual character can be directly read to obtain the target emotion type corresponding to that identifier.

[0091] According to one specific embodiment of this application, the step of determining whether a virtual character in the virtual scene is visible to the camera may include:

[0092] Determine that the real-time distance between the camera and the virtual character is within a preset distance range;

[0093] If the real-time distance between the camera and the virtual character is within a preset distance range, then the virtual character is determined to be visible to the camera.

[0094] like Figure 3 As shown, from the perspective of the person or virtual camera corresponding to the executing entity, the virtual character's emotions can only be recognized when the real-time distance between the camera and the virtual character is relatively close. Specifically, a preset distance range under critical conditions can be provided, meaning that the virtual character can only be considered visible to the camera and capable of emotion recognition when the real-time distance between the camera and the virtual character is within the preset distance range.

[0095] Furthermore, the step of determining that the real-time distance between the camera and the virtual character is within a preset distance range may include:

[0096] Determine whether the bounding sphere of the virtual character's face is located between the near and far view planes of the camera;

[0097] If the facial bounding sphere of the virtual role is between the near clipping plane and the far clipping plane of the camera, it is determined that the camera and the facial bounding sphere of the virtual role are within the preset distance range.

[0098] According to another specific embodiment of the present application, before the step of determining whether the real-time distance between the camera and the virtual role is within the preset distance range, the method further comprises:

[0099] determining whether the frustum of the camera and the facial bounding sphere of the virtual role satisfy an intersection state;

[0100] If the frustum of the camera and the facial bounding sphere of the virtual role satisfy the intersection state, the step of determining whether the real-time distance between the camera and the virtual role is within the preset distance range is performed.

[0101] If the frustum of the camera and the facial bounding sphere of the virtual role do not satisfy the intersection state, it is directly determined that the virtual role in the virtual scene is in an invisible state of the camera.

[0102] If it is determined whether to enter the emotion recognition process, in addition to satisfying a closer distance, the relative position between the virtual role and the camera can also be limited, that is, the image information of the virtual role that can be collected by the camera is limited. As shown in Figure 3 the intersection relationship between the frustum of the camera in the virtual scene and the facial bounding sphere of the virtual role is determined to determine whether the face of the virtual role is in a visible state in the AI camera. The specific determination scheme is to directly detect the intersection relationship between the frustum and the facial bounding sphere, and then further determine that the entire bounding sphere is in the frustum and is close enough to the near clipping plane of the virtual camera to recognize the expression. Only the expression completely between the near clipping plane and the far clipping plane can enter the recognition process.

[0103] Further, the step of determining whether the frustum of the camera and the facial bounding sphere of the virtual role satisfy an intersection state comprises:

[0104] An octree is used for spatial acceleration algorithm to determine whether the frustum of the camera and the facial bounding sphere of the virtual role satisfy an intersection state.

[0105] The octree is used for scene management in 3D space, and can quickly know the position of an object in a 3D scene, or detect whether the object collides with other objects and whether the object is in a visible range. The spatial acceleration algorithm is used for the octree to determine the intersection of the identifiable bounding sphere, thereby improving the speed of intersection determination.

[0106] In step S103, the target emotion type corresponding to the target identification information is determined as the emotion type of the virtual role.

[0107] The virtual character records the emotion type of the current emotion in real time when switching the emotion type. The electronic device can directly read the target identification information of the virtual character to find the corresponding target emotion type, and determine the target emotion type as the emotion type of the virtual character.

[0108] Of course, in other embodiments, if the execution subject of the current emotion recognition method is different from the execution subject of the emotion type record of the virtual character, the execution subject of the emotion recognition method can also send an emotion type acquisition request to the identified virtual character. After receiving the emotion type acquisition request, the execution subject of the emotion record of the virtual character queries the real-time emotion type, defines it as the target emotion type, and feeds back the target emotion type to the electronic device for emotion recognition. In this way, the electronic device can record the emotion of the virtual character as the target emotion type, and complete the emotion recognition of the virtual character.

[0109] In addition, considering that there can be multiple virtual characters in a virtual scene, when performing emotion recognition, the emotions of each virtual character can be identified in turn through traversal. Specifically, when the facial expression of the virtual character enters the intersection region, a list containing the containment relationship of each virtual character is obtained, the list is traversed one by one, the distance is judged, and then the expression ID associated with each virtual character is read one by one to obtain a mapping from the role to the expression ID. In this way, the expression of the identified virtual character can be used as needed.

[0110] On the basis of the above embodiment, according to a specific embodiment of the present application, before the step of determining the target emotion type corresponding to the target identification information as the emotion type of the virtual character, the method further comprises:

[0111] Verifying the target emotion type corresponding to the target identification information according to the actual facial image of the virtual character;

[0112] If the verification is passed, the step of determining the target emotion type as the emotion type of the virtual character is performed;

[0113] If the verification is not passed, feedback is given to the indication information of the failure of the emotion recognition of the virtual character.

[0114] This embodiment adds a link of verifying the target emotion type of the determined virtual character. The camera can capture the actual facial image of the virtual character, and verify according to the actual facial image. Specifically, the expressions corresponding to different types of emotions are different, but the expressions corresponding to the same type of emotion are the same or similar.

[0115] The electronic device can pre-store various types of emotion corresponding expression images, and after receiving the actual facial image of the virtual character, find the reference emotion type corresponding to the expression image with the highest matching degree by iteration comparison, and compare whether the reference emotion type is the target emotion type. If the reference emotion type obtained by image comparison is the target emotion type, the verification is passed, and the emotion of the virtual character is recorded as the target emotion type, and if the reference emotion type is not the target emotion type, it is considered that the verification is not passed, and the indication information of the emotion recognition failure can be fed back.

[0116] The scheme of adding the actual facial image of the virtual character to find the reference emotion type and then verify the target emotion type can further improve the accuracy of emotion recognition, and then feed back the optimized emotion recognition scheme.

[0117] The emotion recognition method provided in the present application determines whether the virtual character in the virtual scene is in the visible state of the camera when the virtual scene is in the emotion recognition mode, and reads and determines the target emotion type of the virtual character if the virtual character is in the visible state of the camera. In this way, without pre-training a neural network model with emotion recognition function at a large cost, the emotion recognition of the virtual character in the virtual scene is performed according to the pre-recorded emotion type, and the real-time emotion type is recorded. In this way, the emotion recognition of the virtual character can be realized at a low cost and efficiently, and the recognized emotion can be used for more personalized docking services in simulation.

[0118] Embodiment 2

[0119] Referring to Figure 4 The present application also provides a structure diagram of an emotion recognition device 400. Corresponding to the above-mentioned method embodiment, as shown in Figure 4 The emotion recognition device 400 mainly includes:

[0120] The judgment module 401 is configured to determine whether the virtual character in the virtual scene is in the visible state of the camera if the virtual scene is in the emotion recognition mode.

[0121] The reading module 402 is configured to read the target identification information associated with the virtual character if the virtual character is in the visible state of the camera.

[0122] The determination module 403 is configured to determine the target emotion type corresponding to the target identification information as the emotion type of the virtual character.

[0123] According to a specific embodiment of the present application, the emotion recognition device 400 further includes a configuration module configured to:

[0124] Preparation of various types of emotion expression display resources;

[0125] The identification information of various types of emotions is stored in association with expression display resources;

[0126] The virtual character is configured with available emotion types;

[0127] According to a target emotion type that the virtual character needs to display in real time, an expression display resource corresponding to the target emotion type is called to realize expression display of the virtual character, wherein the target emotion type is any one of the available emotion types of the virtual character.

[0128] According to an embodiment of the present application, the judging module 401 is configured to:

[0129] Judge whether the real-time distance between the camera and the virtual character is within a preset distance range;

[0130] If the real-time distance between the camera and the virtual character is within the preset distance range, it is determined that the virtual character is in a visible state of the camera.

[0131] According to an embodiment of the present application, the judging module 401 is configured to:

[0132] Judge whether the face-enclosing sphere of the virtual character is between the near clipping plane and the far clipping plane of the camera;

[0133] If the face-enclosing sphere of the virtual character is between the near clipping plane and the far clipping plane of the camera, it is determined that the camera and the face-enclosing sphere of the virtual character are within the preset distance range.

[0134] According to an embodiment of the present application, the judging module 401 is configured to:

[0135] Judge whether the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy an intersection state;

[0136] If the view frustum of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state, the step of judging whether the real-time distance between the camera and the virtual character is within a preset distance range is executed;

[0137] If the view frustum of the camera and the face-enclosing sphere of the virtual character do not satisfy the intersection state, it is directly determined that the virtual character in the virtual scene is in an invisible state of the camera.

[0138] According to an embodiment of the present application, the judging module 401 is configured to:

[0139] The space acceleration algorithm is performed by using an octree to determine whether the view cone of the camera and the face-enclosing sphere of the virtual character satisfy an intersection state.

[0140] According to one specific embodiment of the present application, the determination module 403 is configured to:

[0141] verify the target emotion type corresponding to the target identification information according to the actual face image of the virtual character;

[0142] If the verification is passed, the step of determining the target emotion type as the emotion type of the virtual character is performed.

[0143] If the verification is not passed, the indication information of the emotion recognition failure of the virtual character is fed back.

[0144] The emotion recognition device provided by the present application determines whether the virtual character in the virtual scene is in the visible state of the camera when the virtual scene is in the emotion recognition mode; reads the target identification information associated with the virtual character if the virtual character is in the visible state of the camera; and determines the target emotion type corresponding to the target identification information as the emotion type of the virtual character. In this way, without pre-training a neural network model with emotion recognition function, the emotion recognition of the virtual character in the virtual scene can be realized by reading the pre-recorded emotion type to quickly confirm the real-time emotion type of the virtual character. In this way, the emotion recognition of the virtual character can be realized at a low cost and efficiently, and the recognized emotion can be used for more personalized docking services in simulation. The specific implementation process of the emotion recognition device provided by the present application can refer to the specific recognition process of the emotion recognition method provided by the above-mentioned embodiments, which will not be repeated here.

[0145] Embodiment 3

[0146] In addition, the present disclosure provides an electronic device, including a memory and a processor, the memory stores a computer program, and the computer program executes the emotion recognition method provided by the above-mentioned method embodiment 1 when the processor runs.

[0147] Specifically, as shown in Figure 5 The electronic device 500 provided by the present embodiment includes:

[0148] The electronic device 500 provided by the present embodiment includes a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511, etc. Those skilled in the art can understand that Figure 5The electronic device structure shown in the figure does not constitute a limitation on the electronic device, which can include more or fewer components than shown, or combine certain components, or arrange different components. In the embodiments of the present application, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, and a pedometer, etc.

[0149] It should be understood that in the embodiments of the present application, the radio frequency unit 501 can be used for receiving and transmitting signals in the process of transmitting information or calling. Specifically, after receiving the downlink data from the base station, the processor 510 processes it. In addition, the uplink data is sent to the base station. Generally, the radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 501 can also communicate with the network and other devices through a wireless communication system.

[0150] The electronic device provides wireless broadband Internet access for users through the network module 502, such as helping users to send and receive emails, browse web pages, and access streaming media, etc.

[0151] The audio output unit 503 can convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output it as a sound. Moreover, the audio output unit 503 can also provide audio output related to a specific function performed by the electronic device 500 (for example, a call signal receiving sound, a message receiving sound, etc.). The audio output unit 503 includes a speaker, a buzzer, and a receiver, etc.

[0152] The input unit 504 is used to receive audio or video signals. The input unit 504 can include a graphics processor (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes image data of a still picture or a video obtained by an image capture electronic device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be played on the display unit 506. The image frame processed by the graphics processor 5041 can be stored in the memory 509 (or other storage medium) or transmitted via the radio frequency unit 501 or the network module 502. The microphone 5042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 501 in the case of a telephone call mode.

[0153] The electronic device 500 further includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 5061 and / or the backlight when the electronic device 500 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and when at rest, can detect the magnitude and direction of gravity, and can be used to identify the electronic device posture (such as screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. The sensor 505 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, which will not be described here.

[0154] The display unit 506 is used for video playing information input by the user or information provided to the user. The display unit 506 can include a display panel 5061, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), and the like.

[0155] The user input unit 507 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071, also known as a touch screen, can collect user touch operations (such as user operations using a finger, a stylus, or any suitable object or accessory on or near the touch panel 5071) on or near it. The touch panel 5071 can include a touch detection electronic device and a touch controller. The touch detection electronic device detects the touch position of the user and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection electronic device, converts it into touch coordinates, and sends it to the processor 510, receives commands from the processor 510 and executes them. In addition, the touch panel 5071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 5071, the user input unit 507 can also include other input devices 5072. Specifically, the other input devices 5072 can include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, on-off buttons, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0156] Further, the touch panel 5071 can be overlaid on the display panel 5061, and when the touch panel 5071 detects a touch operation thereon or nearby, it transmits to the processor 510 to determine the type of touch event, and then the processor 510 provides corresponding visual output on the display panel 5061 according to the type of touch event. Although in the above embodiment, the touch panel 5071 and the display panel 5061 are implemented as two independent components to realize the input and output functions of the electronic device, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to realize the input and output functions of the electronic device, which is not limited here. Figure 5

[0157] The interface unit 508 is an interface for connecting external electronic devices to the electronic device 500. For example, the external electronic devices can include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting an electronic device having an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, and the like. The interface unit 508 can be used to receive input (e.g., data information, power, and the like) from external electronic devices and transmit the received input to one or more elements within the electronic device 500, or can be used to transmit data between the electronic device 500 and external electronic devices.

[0158] The memory 509 can be used to store software programs and various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 509 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0159] The processor 510 is the control center of the electronic device, connects all parts of the electronic device through various interfaces and lines, executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 509 and calling data stored in the memory 509, and thus monitors the whole electronic device. The processor 510 can include one or more processing units; preferably, the processor 510 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 510.

[0160] ​The electronic device 500 can further include a power supply 511 (such as a battery) for supplying power to various components. Preferably, the power supply 511 can be logically connected to the processor 510 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management.

[0161] In addition, the electronic device 500 includes some function modules that are not shown here and will not be described here.

[0162] The electronic device provided in the embodiment can determine whether the virtual character in the virtual scene is in a visible state of the camera when the virtual scene is in the emotion recognition mode; if the virtual character is in the visible state of the camera, read target identification information associated with the virtual character; and determine a target emotion type corresponding to the target identification information as an emotion type of the virtual character. In this way, without pre-training a neural network model with emotion recognition function at a large cost, the pre-recorded emotion type is read to quickly confirm the real-time emotion type of the virtual character for emotion recognition of the virtual character in the virtual scene. In this way, the emotion recognition of the virtual character can be realized at a low cost and efficiently, and the recognized emotion can be used for more personalized docking services in simulation. The specific implementation process of the electronic device provided in the embodiment can refer to the specific implementation process of the emotion recognition method provided in the above embodiment, which will not be described here.

[0163] Embodiment 4

[0164] In addition, the present application also provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the emotion recognition method provided in the above embodiment.

[0165] In the embodiment, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0166] The computer readable storage medium provided in the embodiment can implement the emotion recognition method shown in embodiment 1, and will not be described here to avoid repetition.

[0167] It should be noted that, in the present document, the terms "comprises / comprising" or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0168] From the above description of the embodiments, it can be clear to those skilled in the art that the above-mentioned example methods can be realized by means of software plus a necessary general hardware platform, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the form of a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0169] The embodiments of the present application are described above in combination with the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative rather than limiting, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A method of emotion recognition, characterized by, The method comprises: If the virtual scene is in the emotion recognition mode, it is judged whether a virtual character in the virtual scene is in a visible state of a camera; If the virtual character is in the visible state of the camera, target identification information associated with the virtual character is read; A target emotion type corresponding to the target identification information is determined as an emotion type of the virtual character; Before the step of determining the target emotion type as the emotion type of the virtual character, the method further comprises: According to an actual facial image of the virtual character, the target emotion type corresponding to the target identification information is verified; If the verification is passed, the step of determining the target emotion type as the emotion type of the virtual character is executed.

2. The emotion recognition method of claim 1, wherein, Before the step of judging whether the virtual character in the virtual scene is in the visible state of the camera, the method further comprises: Preparation of expression display resources of multiple types of emotions; Storage of identification information of various types of emotions and expression display resources in association; Configuration of available emotion types for the virtual character; According to a target emotion type required by the virtual character to be displayed in real time, expression display resources corresponding to the target emotion type are called to realize expression display of the virtual character, wherein the target emotion type is any type in the available emotion types of the virtual character. 3.The emotion recognition method of claim 1, wherein, The step of judging whether the virtual character in the virtual scene is in the visible state of the camera comprises: Judgment of whether a real-time distance between the camera and the virtual character is within a preset distance range; If the real-time distance between the camera and the virtual character is within the preset distance range, it is determined that the virtual character is in the visible state of the camera.

4. The emotion recognition method of claim 3, wherein, The step of judging whether the real-time distance between the camera and the virtual character is within the preset distance range comprises: Judgment of whether a face-enclosing sphere of the virtual character is between a near clipping plane and a far clipping plane of the camera; If the face-enclosing sphere of the virtual character is between the near clipping plane and the far clipping plane of the camera, it is determined that the camera and the face-enclosing sphere of the virtual character are within the preset distance range.

5. The emotion recognition method of claim 3, wherein, Before the step of judging whether the real-time distance between the camera and the virtual character is within the preset distance range, the method further comprises: Judgment of whether a view cone of the camera and a face-enclosing sphere of the virtual character satisfy an intersection state; If the view cone of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state, the step of judging whether the real-time distance between the camera and the virtual character is within the preset distance range is executed; If the view cone of the camera and the face-enclosing sphere of the virtual character do not satisfy the intersection state, it is directly determined that the virtual character in the virtual scene is in an invisible state of the camera.

6. The emotion recognition method of claim 5, wherein, The step of judging whether the view cone of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state comprises: Using an octree to perform a space acceleration algorithm to judge whether the view cone of the camera and the face-enclosing sphere of the virtual character satisfy the intersection state.

7. The emotion recognition method of any one of claims 1-6, characterized in that, The method further comprises: If the check fails, feedback is provided to the virtual character of an indication that emotion recognition has failed.

8. An emotion recognition apparatus characterized by comprising: The emotion recognition device comprises: a judgment module configured to, if the virtual scene is in an emotion recognition mode, judge whether a virtual character in the virtual scene is in a visible state of a camera; a reading module configured to, if the virtual character is in the visible state of the camera, read target identification information associated with the virtual character; a determination module configured to check a target emotion type corresponding to the target identification information according to an actual facial image of the virtual character; if the check passes, a step of determining the target emotion type as an emotion type of the virtual character is performed; the target emotion type corresponding to the target identification information is determined as the emotion type of the virtual character.

9. An electronic device, comprising: A computer program product, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed on the processor, performs the emotion recognition method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program product, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed on the processor, performs the emotion recognition method of any one of claims 1 to 7.

Images