Content display method and device, electronic equipment and storage medium

Abstract

The invention discloses a content display method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of obtaining a current emotion type of a first user corresponding to a currently displayed first virtual image; and driving the face of the first virtual image to be switched and displayed from the first expression to a second expression corresponding to the emotion type. Thus, by driving the face of the first virtual image to display the second expression corresponding to the emotion type, the emotion type of the current first user in a real scene can be better conveyed, that is, richer and stronger expression of the virtual image in emotion is enhanced, and then the immersion and experience of the user are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a content display method, device, electronic device and storage medium. Background Art

[0002] In order to enhance the user's interest in a video call, video conference, or video recording process, special effects may be introduced automatically or manually by the user to the virtual image in the video.

[0003] In the related art, the common scheme for introducing special effects is generally to change the background of the virtual image, or to change the clothing or hairstyle of the virtual image. However, there is still the problem that the displayed virtual image is not rich and strong enough in emotional expression, which leads to a certain degree of lack of immersion and experience for users. Summary of the invention

[0004] The present application proposes a content display method, device, electronic device and storage medium to enhance the emotional expression of a virtual image.

[0005] In a first aspect, an embodiment of the present application provides a content display method, which is applied to a first electronic device, and the method includes: obtaining an emotion type of a first user corresponding to a first virtual image currently displayed; and driving the face of the first virtual image to switch from a first expression to a second expression corresponding to the emotion type.

[0006] In a second aspect, an embodiment of the present application provides a content display device, which is applied to a first electronic device, and the device includes: an emotion acquisition module and a driving module. The emotion acquisition module is used to acquire the emotion type of the first user corresponding to the currently displayed first virtual image; the driving module is used to drive the face of the first virtual image to switch from a first expression to a second expression corresponding to the emotion type.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the above method.

[0009] In the solution provided by the present application, the current emotion type of the first user corresponding to the currently displayed first virtual image is obtained; the face of the first virtual image is driven to switch from the first expression to the second expression corresponding to the emotion type. In this way, by driving the face of the first virtual image to display the second expression corresponding to the emotion type, the emotion type of the current first user in the real scene can be better conveyed, that is, the virtual image is enhanced to be more rich and more intense in emotional expression, thereby improving the user's immersion and experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A scenario diagram showing an application scenario of a content display method provided in an embodiment of the present application.

[0012] Figure 2 An interface schematic diagram of a display interface provided in an embodiment of the present application is shown.

[0013] Figure 3 A flow chart of a content display method provided in an embodiment of the present application is shown.

[0014] Figure 4 Shown are schematic diagrams of various first expressions provided in the embodiments of the present application.

[0015] Figure 5 A background schematic diagram of various first backgrounds provided by the embodiments of the present application is shown.

[0016] Figure 6 An interface schematic diagram of another display interface provided in an embodiment of the present application is shown.

[0017] Figure 7 A flow chart of a content display method provided in another embodiment of the present application is shown.

[0018] Figure 8 An interface schematic diagram of another display interface provided in an embodiment of the present application is shown.

[0019] Fig. 9 An interface schematic diagram of yet another display interface provided in an embodiment of the present application is shown.

[0020] Fig.10 A flow chart of a content display method provided in yet another embodiment of the present application is shown.

[0021] Fig.11 Shows Fig.10 A flowchart of the sub-steps of step S410 in one implementation manner.

[0022] Fig.12 Component schematic diagrams of various virtual components provided in embodiments of the present application.

[0023] Fig.13 A schematic diagram of the effect of superimposing multiple virtual components provided in an embodiment of the present application.

[0024] Fig.14 A flow chart of a content display method provided in yet another embodiment of the present application is shown.

[0025] Fig.15 An interface schematic diagram of an expression modification interface provided in an embodiment of the present application is shown.

[0026] Fig.16 A schematic diagram of an expression parameter interface provided in an embodiment of the present application is shown.

[0027] Fig.17 An interface schematic diagram of another expression modification interface provided in an embodiment of the present application is shown.

[0028] Fig.18 It is a block diagram of a content display device provided according to an embodiment of the present application.

[0029] Fig.19 It is a block diagram of an electronic device according to an embodiment of the present application for executing the content display method according to an embodiment of the present application.

[0030] Fig. 20 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing the content display method according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings of the present application, multiple operations that appear in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The sequence numbers of operations such as S110, S120, etc. are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or less operations, and these operations may be executed in sequence or in parallel. And, the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0033] The inventor proposes a content display method, device, electronic device and storage medium, which obtains the emotion type of a first user corresponding to a currently displayed first virtual image; and drives the face of the first virtual image to switch from a first expression to a second expression corresponding to the emotion type. The content display method provided in the embodiment of the present application is described in detail below.

[0034] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of a content display method provided in an embodiment of the present application. The application scenario includes: a first electronic device 110. The first electronic device 110 includes, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, augmented reality (AR) glasses, AR head-mounted display devices, mixed reality (MR) glasses, MR head-mounted display devices, virtual reality (VR) glasses, and VR head-mounted display devices.

[0035] The first electronic device 110 has a content display function and can display a first virtual image corresponding to a first user, wherein the first user is Figure 1 For a user holding the first electronic device 110, the display interface of the first electronic device 110 may be as follows: Figure 2 As shown, Figure 2The first virtual image 120 in the image is the first virtual image corresponding to the first user. Based on this, the first electronic device 110 can obtain the emotion type of the first user corresponding to the currently displayed first virtual image 120; and drive the face of the first virtual image 120 to switch from the first expression to the second expression corresponding to the emotion type.

[0036] Please refer to Figure 3 , Figure 3 A flow chart of a content display method provided in an embodiment of the present application is applied to a first electronic device. Figure 3 The content display method provided in the embodiment of the present application is described in detail. The content display method may include the following steps:

[0037] Step S210: Obtain the emotion type of the first user corresponding to the currently displayed first virtual image.

[0038] In this embodiment, the emotion of the first user does not remain constant but changes with the passage of time. Therefore, in order to make the currently displayed first virtual image more rich and intense in emotional expression, the current emotion type of the first virtual image can be obtained, and the expression of the first virtual image can be adjusted accordingly based on the current emotion type, so that the expression of the first virtual image can better reflect the current emotion type of the first user. Among them, the emotion type can be happiness, surprise, anger, sadness, fear, loneliness, seriousness, embarrassment, depression or anxiety; of course, happiness can also be subdivided into two emotion types, very happy and generally happy, according to different degrees. The same applies to other types of emotion types, which will not be repeated here.

[0039] In some embodiments, the current emotion type of the first user can be obtained by acquiring a user image of the first user in the current real environment, and inputting the user image into a pre-trained emotion recognition model for identification. The emotion recognition model can be pre-trained based on a large number of training samples, and the training samples can include user images under different emotions, and each user image can be labeled with the emotion type corresponding to the user. The scene recognition model can be a neural network model, a decision tree model, etc., which are not limited here. In this way, the emotion type of the first user can be quickly and accurately identified through the emotion recognition model. The user image can be a facial image of the user, or a full-body image including a face and limbs. In this way, the user's actions can be further analyzed in combination with the image of the limbs, and the user's emotion type can be more accurately identified. This embodiment does not limit this.

[0040] In other embodiments, if it is detected that the first user is speaking, the audio data currently sent by the first user can be obtained, the tone recognition can be performed on the audio data, and the current emotion type of the first user can be determined based on the tone recognition result. Among them, a pre-trained tone recognition model can be used for recognition. Of course, other tone recognition algorithms can also be used for tone recognition, and this embodiment does not limit this. In this way, when the user image of the first user cannot be collected, the current emotion type of the first user can also be accurately known through the audio data sent by the user.

[0041] In some other embodiments, if it is detected that the first user is speaking, and the user image of the first user in the current real environment can be obtained, the user image and audio data of the first user can be obtained, and based on the user image, the current emotion type of the first user is identified to obtain a first emotion recognition result; at the same time, based on the audio data, the current emotion type of the first user is identified to obtain a second emotion recognition result. Further, the first emotion recognition result and the second emotion recognition result are fused to obtain the current emotion type of the first user. In this way, the emotion type of the first user is determined by both the user image and the audio data of the user, which can greatly improve the recognition accuracy of the emotion type.

[0042] It can be understood that the application scenarios of the content display method of this embodiment include but are not limited to video call scenarios, video conference scenarios, video recording scenarios, and emoticon recording scenarios. The first virtual image can be an Omoji virtual image selected by the user or the default Omoji virtual image of the first electronic device. When the Omoji special effects display function is turned on in the first electronic device, the first electronic device can display the first virtual image in the current display interface.

[0043] Step S220: driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type.

[0044] In this embodiment, a mapping relationship between expressions and emotion types may be pre-stored, in which each of the multiple preset expressions corresponds to at least one emotion type. Based on this, after determining the emotion type of the first user, a preset expression corresponding to the emotion type of the first user may be determined from the multiple preset expressions based on the mapping relationship as the second expression.

[0045] For example, Figure 4 A schematic diagram of a plurality of preset expressions is shown, and the plurality of preset expressions may include Figure 4In the expression A, expression B, expression C, expression D and expression E, the multiple emotion types may include happiness, sadness, anger, surprise and embarrassment. Among them, expression A corresponds to the emotion type "happy", expression B corresponds to the emotion type "sadness", expression C corresponds to the emotion type "anger", expression D corresponds to the emotion type "surprise", and expression E corresponds to the emotion type "embarrassment". Based on this, if it is detected that the current emotion type of the first user is the emotion type "happy", expression A corresponding to the emotion type "happy" can be determined as the second expression.

[0046] Furthermore, after determining the second expression that matches the current emotion type of the first user, the face of the first virtual image corresponding to the first user can be driven to switch from the first expression to the second expression corresponding to the emotion type. In this way, by driving the face of the first virtual image to display the second expression corresponding to the emotion type of the first user, the emotion type of the first user in the real scene can be better conveyed, that is, the virtual image is more emotionally rich and more intensely expressed; and, especially in the video call scene, the user's sense of immersion and experience can be improved, thereby greatly improving the fun of the video call.

[0047] In some embodiments, if it is determined that the first electronic device is currently in the automatic expression switching mode, the face of the first virtual image can be directly driven to switch from the first expression to the second expression corresponding to the emotion type.

[0048] In other embodiments, if it is determined that the first electronic device is currently in a non-automatic expression switching mode, a first prompt message is displayed, and the first prompt message is used to prompt the first user to confirm whether the expression of the first virtual image needs to be switched; if a confirmation operation based on the first prompt message input is received, the face of the first virtual image is driven to switch from the first expression to the second expression corresponding to the emotion type.

[0049] In some embodiments, in order to further enhance the rich emotional expression of the virtual image, a first background corresponding to the emotional type of the first user may be determined from a plurality of first backgrounds as a target background; and the current display background where the first virtual image is located may be switched to the target background. That is, the first electronic device may pre-store a mapping relationship between backgrounds and emotional types, in which each of the plurality of first backgrounds corresponds to at least one emotional type. Based on this, after determining the emotional type of the first user, the first background corresponding to the emotional type of the first user may be determined from a plurality of first backgrounds based on the mapping relationship as the target background. In this way, not only the facial expression of the first virtual image will change with the emotional changes of the first user, but also the background where the first virtual image is located in the current display interface will change with the emotional changes of the first user; that is, the expression of the first virtual image in the current display interface and the current display background of the first virtual image both match the emotional type of the first user, further enhancing the richer and stronger emotional expression of the virtual image, thereby further enhancing the user's sense of immersion and experience.

[0050] For example, Figure 5 A background diagram showing a plurality of first backgrounds, wherein the plurality of first backgrounds may include Figure 4 In the background A, background B, background C and background D, the multiple emotion types may include depression, sadness, loneliness and happiness. Among them, background A corresponds to the emotion type "depression", background B corresponds to the emotion type "sadness", background C corresponds to the emotion type "loneliness", and background D corresponds to the emotion type "happy". Based on this, if the current emotion type is detected as the emotion type "happy", the background D corresponding to the emotion type "happy" can be determined as the target background; Figure 6 As shown, Figure 6 It shows the current display interface of the first electronic device when the emotion type of the first user is the emotion type "happy". The facial expression of the first virtual image 120 in the current display interface is expression A corresponding to "happy", and the target background of the first virtual image 120 is background D corresponding to "happy".

[0051] In this embodiment, the first electronic device obtains the current emotion type of the first user corresponding to the currently displayed first virtual image, and then drives the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type. This can better convey the emotion type of the first user in the real scene, that is, improve the richer and stronger emotional expression of the virtual image; and, especially in the video call scene, can improve the user's immersion and experience, thereby greatly improving the fun of the video call.

[0052] Please refer to Figure 7 , Figure 7 A flow chart of a content display method provided in another embodiment of the present application is applied to a first electronic device. Figure 7 The content display method provided in the embodiment of the present application is described in detail. The content display method may include the following steps:

[0053] Step S310: If the first electronic device is currently in a first expression driving mode, obtain an emotion type of a first user corresponding to the first virtual image currently displayed.

[0054] In this embodiment, a plurality of expression drive modes may be provided in the first electronic device, and the plurality of expression drive modes include at least a first expression drive mode and a second expression drive mode. Among them, the first expression drive mode is used to drive the face of the first virtual image to display a second expression corresponding to the emotion type according to the current emotion type of the first user; in the first expression drive mode, the second expression corresponding to the emotion type may be an expression that just reflects the emotion type, or an expression that is moderately exaggerated for the emotion type. By displaying a moderately exaggerated expression, the fun of the virtual image display may be improved to a certain extent, and this embodiment does not impose any restrictions on this. The second expression drive mode is used to drive the face of the first virtual image to display an expression that is the same or similar to the current expression according to the current expression of the first user.

[0055] Based on this, the first electronic device may be provided with a manual switching function and an automatic switching function for the mode. Optionally, the manual switching function corresponds to a mode switching button, and the user can input a mode switching operation through the mode switching button to switch the first expression drive mode to the second expression drive mode, or switch the second expression drive mode to the second expression drive mode. In this way, the user can simply and quickly adjust the expression drive mode of the first electronic device to the actual desired expression drive mode through the mode switching button.

[0056] Optionally, in the automatic switching function, the target operating time period corresponding to each mode can be configured, and then the first electronic device can determine the mode corresponding to the target operating time period at the current time according to the current time, and use the mode corresponding to the target operating time period at the current time as the current expression driving mode of the first electronic device. In this way, the user can set different operating time periods by himself, and control the first electronic device to use different expression driving modes in different operating time periods to meet the expression driving requirements in different operating time periods.

[0057] Optionally, in the automatic switching function, the CPU utilization interval corresponding to each expression drive mode can also be configured, and then the first electronic device can determine the CPU utilization interval in which the current CPU utilization is located based on the detected current CPU utilization, as the target utilization interval; and the expression drive mode corresponding to the target utilization interval is used as the current expression drive mode of the first electronic device. In this way, the first electronic device can adjust the expression drive mode that matches the CPU utilization in real time according to the CPU utilization implemented by itself, so as to ensure the effect of expression drive on the virtual image.

[0058] In some implementations, if the first electronic device is currently in the first expression driving mode, the emotion type of the first user corresponding to the currently displayed first virtual image can be first obtained. The specific implementation of how to obtain the current emotion type of the first user can refer to the content of the aforementioned embodiment, which will not be repeated here.

[0059] Step S320: determining a driving parameter corresponding to the second expression from a plurality of expression driving parameters as a target driving parameter, wherein different expression driving parameters correspond to different expressions.

[0060] Optionally, a plurality of expression driving parameters may be pre-stored in the first electronic device, each expression driving parameter corresponds to a preset expression mentioned in the above embodiment, that is, each preset expression is different; wherein the expression driving parameter is used to drive the expression of the virtual image, for example, to drive the position and posture of the facial features of the virtual image to change, so as to display different expressions. Based on this, after determining the emotion type of the first user, the driving parameter corresponding to the second expression corresponding to the emotion type can be determined from the plurality of expression driving parameters as the driving parameter for driving the facial expression of the first virtual image. That is to say, after determining the emotion type, the target driving parameter for driving the face of the first virtual image to display the second expression corresponding to the emotion type can be quickly obtained, without the need to calculate the driving parameter in real time, thus reducing the consumption of computing resources and the power consumption of the first electronic device, while also ensuring the real-time acquisition of the target driving parameter, thereby ensuring the real-time driving of the face of the first virtual image to display the second expression.

[0061] It can be understood that making facial expressions is a continuous action. Accordingly, the expression driving parameters corresponding to each preset expression also include the driving parameters for the virtual image in each frame of the virtual image during the process of completing the continuous action.

[0062] Step S330: According to the target driving parameter, driving the face of the first virtual image to switch from the first expression to the second expression.

[0063] In some embodiments, the first electronic device includes virtual images with multiple appearances, for example, virtual images with fatter faces and virtual images with thinner faces. It is understandable that due to different facial contours, different positions and sizes of facial features, even if the virtual images with different appearances are driven to display the same expression, the required expression driving parameters will be different. Therefore, multiple sub-driving parameters can be set in the expression driving parameters corresponding to each first expression, and the multiple sub-driving parameters correspond to the virtual images of multiple information one by one.

[0064] Based on this, from the multiple sub-driving parameters in the target driving parameters, the sub-driving parameters corresponding to the image of the first virtual image are obtained as the target sub-driving parameters; and further according to the target sub-driving parameters, the face of the first virtual image is driven to display the second expression. In this way, it is possible to avoid the use of the same expression driving parameters to drive virtual images of different images with the same expression, resulting in inconsistent expression display and other problems.

[0065] In some embodiments, the first virtual image is a three-dimensional virtual image. Before driving the facial expression of the first virtual image, the current head posture data of the first user can also be obtained; based on the current head posture data, the head driving parameters for the first virtual image are determined. Finally, according to the head driving parameters, the head position and head posture of the first virtual image are driven, and based on the target driving parameters, the face of the first virtual image is driven to switch from the first expression to the second expression. In other words, the first virtual image in the present application can be a three-dimensional virtual image, which can not only drive the facial expression of the three-dimensional first virtual image, but also drive the head of the first virtual image to move with the head of the first user according to the movement of the head of the first user. In this way, the realism of the virtual image display can be improved, thereby improving the user experience and immersion.

[0066] For example, if only the head posture of the first user changes, for example, turning the head to the left, and the emotion does not change, the first electronic device drives the head of the first virtual image 120 to change from Figure 8 The state shown changes to Fig. 9 At the same time, the first electronic device drives the face of the first virtual image to switch from the first expression to the second expression based on the target driving parameter. Figure 8 and Fig. 9 The face of the first virtual image in the video all displays the second expression.

[0067] In other embodiments, if the first electronic device is currently in the second expression driving mode, the expression information of the first user corresponding to the currently displayed first virtual image is obtained; based on the expression information, the first expression driving parameter for the first virtual image is generated; according to the first expression driving parameter, the face of the first virtual image is driven to switch from the first expression to the expression corresponding to the first expression driving parameter. In other words, in the second expression driving mode, the expression of the first virtual image can change in real time with the expression of the first user. For example, if the first user frowns, the first virtual image also displays the corresponding frowning special effect. In this way, the expression of the first virtual image is driven in real time to correspond to the expression of the first user in the real scene, which improves the realism of the displayed first virtual image and also improves the user's interactive use experience.

[0068] In some other embodiments, if the first electronic device is currently in the third expression driving mode, at least one virtual thumbnail image of the multiple virtual thumbnail images is displayed in the target display area of ​​the current display interface, wherein the multiple virtual thumbnail images are generated based on the multiple third expressions and the first virtual image, and the multiple virtual thumbnail images correspond to the multiple third expressions one by one. Based on this, the user can manually select the currently required second expression based on the multiple virtual thumbnail images of different expressions displayed in the target display area, and switch the expression of the first virtual image. Among them, the multiple third expressions can be the same as the multiple preset expressions of the aforementioned price increase, or they can be different expressions, and this embodiment does not limit this.

[0069] In some embodiments, in a video call scenario, that is, a scenario in which a video call connection is established between the first electronic device and the second electronic device, after step S320, the first electronic device may send the target drive parameter to the second electronic device, so that the second electronic device drives the face of the first virtual image in the current display interface of the second electronic device to switch from the first expression to the second expression according to the target drive parameter. In this way, the target drive parameter obtained according to the emotion of the first user can be sent to the second electronic device at the other end of the video call in a timely manner, so that the second electronic device can synchronously display the first virtual image making the second expression, that is, the second user corresponding to the second electronic device can know the current emotion type of the first user through the second expression of the displayed first virtual image, so as to better realize video conversation communication.

[0070] In this embodiment, the first electronic device, when in the first expression driving mode, will obtain the current emotion type of the first user corresponding to the currently displayed first virtual image; and according to the target driving parameter, drive the face of the first virtual image to switch from the first expression to the second expression. In this way, in the first expression driving mode, by driving the face of the first virtual image to display the second expression corresponding to the emotion type, the current emotion type of the first user in the real scene can be better conveyed, that is, the virtual image is more emotionally rich and more intensely expressed; and, especially in the video call scene, the user's sense of immersion and experience can be improved, thereby greatly improving the fun of the video call.

[0071] Please refer to Fig.10 , Fig.10 A flowchart of a content display method provided in another embodiment of the present application is applied to a first electronic device. Fig.10 The content display method provided in the embodiment of the present application is described in detail. The content display method may include the following steps:

[0072] Step S410: Obtaining the emotion type of the first user corresponding to the displayed first virtual image at every target time duration.

[0073] In this embodiment, the target duration may be a preset value, such as 1 second or 3 seconds. Of course, the value of the target duration may be adjusted by the user according to actual needs, and this embodiment does not limit this.

[0074] Based on this, the first electronic device can detect the emotion type of the first user corresponding to the first virtual image at target time intervals, that is, detect the user's emotions at regular intervals. It can be understood that since the user's emotions generally do not change significantly in a short period of time, detecting the first user's emotions at target time intervals can save the computing resources and power consumption of the first electronic device to a certain extent.

[0075] In some embodiments, see Fig.11 , step S410 may include the following contents from step S411 to step S412:

[0076] Step S411: acquiring user data of the first user at every target duration, wherein the current user data includes at least one of facial image data and audio data.

[0077] Step S412: Determine the current emotion type of the first user according to the user data.

[0078] In this manner, the emotion type of the first user is analyzed in real time according to at least one of the acquired facial image data and audio data at every target time period. Specifically, how to determine the emotion type of the first user according to at least one of the acquired facial image data and audio data can be referred to the contents of the aforementioned embodiment, which will not be described in detail here.

[0079] Step S420: driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type.

[0080] In this embodiment, the specific implementation of step S420 can refer to the content of the above-mentioned embodiment, which will not be repeated here.

[0081] In some embodiments, the user data includes the audio data, and after obtaining the user data of the first user, the audio data is converted into text data; if it is determined that the text data contains a target keyword, a virtual component corresponding to the target keyword is obtained as the target virtual component, and the target keyword is any first keyword among multiple first keywords, and the multiple first keywords correspond one-to-one to multiple virtual components; the target virtual component is displayed at the target position in the display interface of the first electronic device.

[0082] The audio data can be converted into corresponding text data by using Automatic Speech Recognition (ASR) technology; or the audio data can be converted into text by using a pre-trained text recognition model to obtain the corresponding text data. Based on this, in order to improve the fun of the virtual image display, multiple first keywords can be pre-set, and each first keyword corresponds to a virtual component, such as Fig.12 As shown, the first keyword "love" corresponds to virtual component 1 "heart pattern", the first keyword "money" corresponds to virtual component 2 "starry eyes", the first keyword "happy" corresponds to virtual component 3 "fireworks", etc., and no further examples are given here. It can be understood that the multiple first keywords and the multiple virtual components can be factory settings of the first electronic device, or can be set by the user according to his or her actual needs during subsequent use. This embodiment does not limit this. Furthermore, after the text data is acquired, keyword recognition can be performed on the text data to determine whether the text data contains any of the multiple first keywords, and then the target virtual component is added to the first virtual image according to the keyword recognition result. For example, Fig.13 As shown, Fig.13Schematic diagrams showing virtual component 1, virtual component 2 and virtual component 3 added to the face of the first virtual image are shown respectively. In this way, the facial expression of the first virtual image is driven accordingly in combination with the current emotion type of the first user and the audio data sent. At the same time, the addition of the target virtual component greatly improves the realism and fun of the display of the first virtual image, thereby making the virtual image more emotionally richer and more intensely expressive, improving the user's sense of immersion and experience.

[0083] In some embodiments, since the second expression corresponds to a certain display time, if the display time is less than the aforementioned target time, the first electronic device can drive the first virtual image to display a specified expression after driving the first virtual image to display the second expression and before the next emotion detection time is reached, so as to ensure the display consistency of the facial expression of the first virtual image. The specified expression can be a pre-set smiling expression or other expressions with a lower degree of exaggeration determined according to the current emotion type. For example, if the second expression is a laughing expression, after the first virtual image displays the laughing expression, since the first user's emotions change little in a short period of time, at this time, the first user is very likely to still be in a happy mood. At this time, the first virtual image can be driven to display a smiling expression with a lower degree of exaggeration than the big and small expressions.

[0084] In this embodiment, the first electronic device will obtain the emotion type of the first user corresponding to the currently displayed first virtual image at target durations, and drive the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type. In this way, by driving the face of the first virtual image to display the second expression, the emotion type of the first user in the real scene can be better conveyed, that is, the virtual image is more emotionally rich and more intensely expressed, while also improving the user's sense of immersion and experience, thereby greatly improving the fun of the video call; and, because the user's emotion type is obtained once per target duration, the computing resources and power consumption of the first electronic device are saved to a certain extent, avoiding the unnecessary consumption of computing resources and power consumption caused by emotion recognition for each frame of the user image collected.

[0085] Please refer to Fig.14 , Fig.14 A flowchart of a content display method provided in yet another embodiment of the present application is applied to a first electronic device. Fig.14 The content display method provided in the embodiment of the present application is described in detail. The content display method may include the following steps:

[0086] Step S510: Obtain the emotion type of the first user corresponding to the currently displayed first virtual image.

[0087] Step S520: driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type.

[0088] In this embodiment, the specific implementation of step S510 to step S520 can refer to the content of the above-mentioned embodiment, which will not be repeated here.

[0089] Step S530: In response to the expression modification operation input to the first virtual image, the second expression displayed on the face of the first virtual image is correspondingly adjusted.

[0090] In this embodiment, after driving the face of the first virtual image to display the second expression, the first electronic device also provides an expression modification function, that is, the first user can check whether the second expression of the first virtual image meets the expression type he wants. If not, the first user can make corresponding adjustments to the second expression of the first virtual image based on the expression modification function. In this way, by providing the expression modification function, the user can edit and adjust the expression to his satisfaction, thereby improving the user experience and fun.

[0091] In some embodiments, at least one virtual thumbnail image among a plurality of virtual thumbnail images is displayed in a target display area in the current display interface of the first electronic device, and the plurality of virtual thumbnail images are generated based on the plurality of third expressions and the first virtual image, and the plurality of virtual thumbnail images correspond to the plurality of third expressions one by one. In response to an image selection operation input based on the target display area, the facial expression of the first virtual image is switched from the second expression to the third expression corresponding to the selected virtual thumbnail image. In this manner, if the first user is not satisfied with the corresponding second expression automatically displayed according to the current emotion type, for example, the current emotion type of the first user is "sad", but the first user does not want the other party to notice that he is very sad during the video call; at this time, the third expression that the first user wants can be selected through the image selection operation to be displayed, for example, the third expression corresponding to "happy", thereby effectively protecting the user's emotional privacy when the user does not want the call counterpart to notice the current emotion, and further meeting the user's personalized needs.

[0092] For example, Fig.15As shown, the current display interface of the first electronic device displays the first virtual image 120, the target display area 130, the virtual thumbnail image 131 in the target display area 130, and the virtual thumbnail image 132. The first user can select the virtual thumbnail image 131 through the image selection operation in the target display area 130, and at this time, the first electronic device can switch the facial expression of the first virtual image 120 from the second expression to the facial expression of the virtual thumbnail image 131.

[0093] In some other embodiments, in response to an expression modification operation inputted for the first virtual image, a plurality of expression parameters for adjusting the expression of the first virtual image are displayed; in response to a parameter selection operation for a target expression parameter among the plurality of expression parameters, the second expression displayed on the face of the first virtual image is correspondingly adjusted based on the target expression parameter. Since different users have different degrees of exaggeration required for the expression of the first virtual image, for example, some users may want the expression of the first virtual image to be as exaggerated as possible to increase the fun of the first virtual image; for another example, some users may want the expression of the first virtual image to be as peaceful as possible to be closer to the expression in the real scene. Based on this, the first electronic device can provide a parameter adjustment function, so that the user can make a more detailed adjustment to the second expression displayed by the first virtual image based on the parameter adjustment function, so that the second expression of the displayed first virtual image is more in line with the expression required by the first user.

[0094] Specifically, the user can input an expression modification operation for the first virtual image to trigger the first electronic device to display multiple expression parameters for adjusting the expression of the first virtual image. The input method of the expression modification operation includes but is not limited to input methods such as clicking on the screen, gesture input, and voice input. Further, after the multiple expression parameters are displayed, the user can select a target expression parameter from the multiple expression parameters to adjust the facial expression of the first virtual image corresponding to the target expression parameter.

[0095] Exemplarily, the first electronic device responds to the expression modification operation, such as Fig.16 The designated display area 140 in the current display interface shown in FIG. 1 displays multiple expression parameters, for example, expression parameters corresponding to the mouth, eyes, and eyebrows; further, if the target expression parameter is an expression parameter corresponding to the mouth, correspondingly, the first electronic device can display the expression parameters in the following manner: Fig.17The parameter adjustment control 141 and the parameter adjustment range control 142 are displayed in the designated display area 140 in the current display interface shown. At this time, the user can adjust the exaggeration of the mouth under the second expression by dragging the position of the parameter adjustment control 141 in the direction from small to large of the parameter adjustment range control 142. Taking the second expression of "smile" as an example, the greater the exaggeration, the greater the smile arc of the mouth, and vice versa. The principles of other types of second expressions are similar and will not be repeated here.

[0096] In other embodiments, the user may also, during the stage of pre-setting multiple third expressions, make more detailed adjustments to the multiple third expressions that are factory set for the multiple first electronic devices through the expression modification operation described in this embodiment, and save the adjusted multiple third expressions so as to subsequently drive the face of the first virtual image to display corresponding expressions based on the recognized emotion type.

[0097] In this embodiment, the current emotion type of the first user corresponding to the currently displayed first virtual image is obtained, and the face of the first virtual image is driven to switch from the first expression to the second expression corresponding to the emotion type; in response to the expression modification operation input to the first virtual image, the second expression displayed on the face of the first virtual image is adjusted accordingly. It can be seen that the first electronic device, by providing the expression modification function, allows the user to edit and adjust the expression that he or she is satisfied with to be displayed on the face of the first virtual image, thereby improving the user's interactive experience and the fun of the virtual image display.

[0098] Please refer to Fig.18 , which shows a structural block diagram of a content display device 600 provided in an embodiment of the present application, which is applied to a first electronic device. The device 600 may include: an emotion acquisition module 610 and a driving module 620.

[0099] The emotion acquisition module 610 is used to acquire the emotion type of the first user corresponding to the currently displayed first virtual image.

[0100] The driving module 620 is used to drive the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type.

[0101] In some embodiments, the emotion acquisition module 610 may be specifically configured to acquire the emotion type of the first user corresponding to the first virtual image currently displayed if the first electronic device is currently in the first expression driving mode.

[0102] In this manner, the driving module 620 may include: a driving parameter determination unit and a driving unit. The driving parameter determination unit may be used to determine a driving parameter corresponding to the second expression from a plurality of expression driving parameters as a target driving parameter, and different expression driving parameters correspond to different expressions. The driving unit may be used to drive the face of the first virtual image to switch from the first expression to the second expression according to the target driving parameter.

[0103] In some other embodiments, the content display device 600 may further include: an expression acquisition module and a drive parameter generation module. The expression acquisition module may be used to acquire the current expression information of the first user corresponding to the currently displayed first virtual image if the first electronic device is currently in the second expression drive mode. The drive parameter generation module may be used to generate a first expression drive parameter for the first virtual image based on the current expression information. The drive module 630 may be specifically used to drive the face of the first virtual image to display an expression corresponding to the first expression information according to the first expression drive parameter.

[0104] In some embodiments, the first virtual image is a three-dimensional virtual image, and the content display device 600 may further include: a head posture acquisition module and a head drive parameter determination module. The head posture acquisition module may be used to acquire the current head posture data of the first user before driving the face of the first virtual image to switch from the first expression to the second expression according to the target drive parameter. The head drive parameter determination module may be used to determine the head drive parameters for the first virtual image based on the current head posture data. The drive module 620 may be specifically used to drive the head position and head posture of the first virtual image according to the head drive parameters, and drive the face of the first virtual image to switch from the first expression to the second expression based on the target drive parameters.

[0105] In some embodiments, a video call connection is established between the first electronic device and the second electronic device, and the content display device 600 may further include: a parameter sending module. The parameter sending module may be used to send the target driving parameter to the second electronic device after determining the driving parameter corresponding to the second expression from the plurality of expression driving parameters as the target driving parameter, so that the second electronic device drives the face of the first virtual image in the current display interface of the second electronic device to switch from the first expression to the second expression according to the target driving parameter.

[0106] In some implementations, the emotion acquisition module 610 may be specifically configured to: acquire the emotion type of the first user corresponding to the displayed first virtual image at every target duration.

[0107] In this manner, the emotion acquisition module 610 may include: a user data acquisition unit and an emotion type determination unit. The user data acquisition unit may be configured to acquire user data of the first user at each target duration, wherein the user data includes at least one of face image data and audio data. The emotion type determination unit may be configured to determine the emotion type of the first user based on the user data.

[0108] In this manner, the current user data includes the audio data, and the content display device 600 may further include: a text conversion module, a virtual component acquisition module, and a virtual component display module. Among them, the text conversion module can be used to convert the audio data into text data after the user data of the first user is acquired. The virtual component acquisition module can be used to acquire the virtual component corresponding to the target text as the target virtual component if it is determined that the text data contains a target text, and the target text is any first text among a plurality of first texts, and the plurality of first texts correspond one-to-one to a plurality of virtual components. The virtual component display module can be used to display the target virtual component at a target position in the display interface of the first electronic device.

[0109] In some embodiments, the content display device 600 may further include an expression modification module. The expression modification module may be configured to, after driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type, respond to an expression modification operation input to the first virtual image and adjust the second expression displayed on the face of the first virtual image accordingly.

[0110] In this manner, at least one virtual thumbnail image among a plurality of virtual thumbnail images is displayed in a target display area in a current display interface of the first electronic device, the plurality of virtual thumbnail images are generated based on a plurality of third expressions and the first virtual image, and the plurality of virtual thumbnail images correspond one to one with the plurality of third expressions. The expression modification module can be specifically used to: in response to an image selection operation input based on the target display area, switch the facial expression of the first virtual image from the second expression to the third expression corresponding to the selected virtual thumbnail image.

[0111] In this manner, the expression modification module may further include: a parameter display unit and a parameter adjustment unit. The parameter display unit may be used to display a plurality of expression parameters for adjusting the expression of the first virtual image in response to an expression modification operation input to the first virtual image. The parameter adjustment unit may be used to make corresponding adjustments to the second expression displayed on the face of the first virtual image based on the target expression parameter in response to a parameter selection operation for a target expression parameter among the plurality of expression parameters.

[0112] In some embodiments, the content display device 600 may further include: a background determination module and a background switching module. The background determination module may be used to determine a first background corresponding to the emotion type from a plurality of first backgrounds as a target background after acquiring the emotion type of the first user corresponding to the currently displayed first virtual image. The background switching module may be used to switch the currently displayed background of the first virtual image to the target background.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0114] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0115] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0116] In summary, the first electronic device obtains the current emotion type of the first user corresponding to the currently displayed first virtual image; and drives the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type. In this way, by driving the face of the first virtual image to display the second expression corresponding to the emotion type, the emotion type of the current first user in the real scene can be better conveyed, that is, the virtual image is enhanced to be more rich and more intense in emotional expression, thereby improving the user's immersion and experience.

[0117] The following will be combined Fig.19 An electronic device provided by the present application is described.

[0118] Reference Fig.19 , Fig.19The structural block diagram of an electronic device 700 provided in an embodiment of the present application is shown, and the above method provided in an embodiment of the present application can be executed by the electronic device 700. The electronic device can be an electronic terminal with a data processing function, and the electronic terminal includes but is not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3) player, and an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) player, etc.

[0119] The electronic device 700 in the embodiment of the present application may include one or more of the following components: a processor 701, a memory 702, and one or more applications, wherein the one or more applications may be stored in the memory 702 and configured to be executed by one or more processors 701, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0120] The processor 701 may include one or more processing cores. The processor 701 uses various interfaces and lines to connect various parts of the entire electronic device 700, and executes various functions and processes data of the electronic device 700 by running or executing instructions, programs, code sets or instruction sets stored in the memory 702, and calling data stored in the memory 702. Optionally, the processor 701 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 701 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem can also be integrated into the processor 701 and implemented separately through a communication chip.

[0121] The memory 702 may include a random access memory (RAM) or a read-only memory (ROM). The memory 702 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as the various corresponding relationships described above) created by the electronic device 700 during use.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0123] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.

[0124] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0125] Please refer to Fig. 20 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0126] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step of the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0127] In some embodiments, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the steps in the above-mentioned method embodiments.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A content display method, characterized in that: Applied to a first electronic device, the method includes: Obtaining an emotion type of a first user corresponding to a currently displayed first virtual image; The face of the first virtual image is driven to switch from a first expression to a second expression corresponding to the emotion type.

2. The method according to claim 1, characterized in that The obtaining the emotion type of the first user corresponding to the currently displayed first virtual image includes: If the first electronic device is currently in the first expression driving mode, obtaining an emotion type of the first user corresponding to the first virtual image currently displayed; The step of driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type includes: Determine a driving parameter corresponding to the second expression from a plurality of expression driving parameters as a target driving parameter, wherein different expression driving parameters correspond to different expressions; According to the target driving parameter, the face of the first virtual image is driven to switch from the first expression to the second expression.

3. The method according to claim 2, characterized in that The first virtual image is a three-dimensional virtual image. Before driving the face of the first virtual image to switch from the first expression to the second expression according to the target driving parameter, the method further includes: Acquire current head posture data of the first user; Determining head drive parameters for the first virtual image based on the current head pose data; The step of driving the face of the first virtual image to switch from the first expression to the second expression according to the target driving parameter includes: The head position and head posture of the first virtual image are driven according to the head driving parameters, and the face of the first virtual image is driven to switch from the first expression to the second expression based on the target driving parameters.

4. The method according to claim 2, characterized in that: A video call connection is established between the first electronic device and the second electronic device. After determining the driving parameter corresponding to the second expression from the plurality of expression driving parameters as the target driving parameter, the method further includes: The target driving parameter is sent to the second electronic device, so that the second electronic device drives the face of the first virtual image in the current display interface of the second electronic device to switch from the first expression to the second expression according to the target driving parameter.

5. The method according to claim 1, characterized in that The method further comprises: If the first electronic device is currently in the second expression driving mode, acquiring expression information of the first user corresponding to the currently displayed first virtual image; Based on the expression information, generating a first expression driving parameter for the first virtual image; According to the first expression driving parameter, the face of the first virtual image is driven to switch from the first expression to the expression corresponding to the first expression driving parameter.

6. The method according to claim 1, characterized in that The obtaining the emotion type of the first user corresponding to the currently displayed first virtual image includes: At every target time period, an emotion type of the first user corresponding to the displayed first virtual image is obtained.

7. The method according to claim 6, characterized in that The acquiring the emotion type of the first user corresponding to the displayed first virtual image at every target time period includes: At each target time interval, obtaining user data of the first user, the user data including at least one of face image data and audio data; An emotion type of the first user is determined according to the user data.

8. The method according to claim 7, characterized in that The user data includes the audio data. After acquiring the user data of the first user, the method further includes: Converting the audio data into text data; If it is determined that the text data contains a target keyword, a virtual component corresponding to the target keyword is obtained as the target virtual component, wherein the target keyword is any first keyword among a plurality of first keywords, and the plurality of first keywords correspond to a plurality of virtual components one by one; The target virtual component is displayed at a target position in the display interface of the first electronic device.

9. The method according to claim 1, characterized in that: After driving the face of the first virtual image to switch from the first expression to the second expression corresponding to the emotion type, the method further includes: In response to an expression modification operation input to the first virtual image, the second expression displayed on the face of the first virtual image is adjusted accordingly.

10. The method according to claim 9, characterized in that At least one virtual thumbnail image of a plurality of virtual thumbnail images is displayed in a target display area in a current display interface of the first electronic device, the plurality of virtual thumbnail images are generated based on a plurality of third expressions and the first virtual image, and the plurality of virtual thumbnail images correspond one to one to the plurality of third expressions; In response to the expression modification operation input to the first virtual image, the second expression displayed on the face of the first virtual image is correspondingly adjusted, including: In response to an image selection operation input based on the target display area, the facial expression of the first virtual image is switched from the second expression to a third expression corresponding to the selected virtual thumbnail image.

11. The method according to claim 9, characterized in that In response to the expression modification operation input to the first virtual image, the second expression displayed on the face of the first virtual image is correspondingly adjusted, including: In response to an expression modification operation input to the first virtual image, displaying a plurality of expression parameters for adjusting the expression of the first virtual image; In response to a parameter selection operation for a target expression parameter among the plurality of expression parameters, the second expression displayed on the face of the first virtual image is correspondingly adjusted based on the target expression parameter.

12. The method according to any one of claims 1 to 11, characterized in that: After acquiring the emotion type of the first user corresponding to the currently displayed first virtual image, the method further includes: Determine a first background corresponding to the emotion type from a plurality of first backgrounds as a target background; The current display background of the first virtual image is switched to the target background.

13. A content display device, characterized in that: Applied to a first electronic device, the device comprises: An emotion acquisition module, used to acquire an emotion type of a first user corresponding to a currently displayed first virtual image; The driving module is used to drive the face of the first virtual image to switch from a first expression to a second expression corresponding to the emotion type.

14. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 12.

Images