Display method and device for virtual image, storage medium and electronic device

Abstract

The present invention discloses a method and device for displaying a virtual image, a storage medium, and an electronic device. Among them, the method includes: identifying a target action currently demonstrated by a host object in a live broadcast screen; obtaining multiple part pictures that match a target virtual image associated with the host object, where each part picture is respectively used to display a key body part of the target virtual image; respectively adjusting the multiple part pictures according to the action parameters of the target action to obtain multiple adjusted part pictures; assembling the multiple adjusted part pictures to display the target virtual image performing the target action. By adopting the above technical solution, the problem of too high production cost in the process of displaying the target virtual image performing the target action based on a 3D model in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular, to a method and device for displaying a virtual image, a storage medium, and an electronic device. Background Art

[0002] With the development of network technology and the live broadcast industry, virtual anchors, as a new form of live broadcast, have successfully attracted the public's attention to network live broadcasts. In the prior art, human body pose driving requires artists to manually build a 3D human body model. For example, a person is photographed from multiple angles through a mobile phone or a camera to obtain more information points, and the information points are fused with the key points in the virtual image to generate the target image of the virtual anchor.

[0003] However, the method of designing a new virtual anchor image based on a 3D model not only requires a large number of pictures of specific target people for training, but also can only drive specific target tasks, resulting in the problem of too high production costs in the process of displaying a target virtual image performing a target action based on the 3D model.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide a method and device for displaying a virtual image, a storage medium, and an electronic device, so as to at least solve the problem of too high production costs in the process of displaying a target virtual image performing a target action based on a 3D model.

[0006] According to an aspect of an embodiment of the present invention, a method for displaying a virtual image is provided, including: identifying a target action currently demonstrated by a host object in a live broadcast screen; obtaining a plurality of part pictures matching a target virtual image associated with the host object, where each part picture is respectively used to display a body key part of the target virtual image; adjusting the plurality of part pictures respectively according to the action parameters of the target action to obtain a plurality of adjusted part pictures; and assembling the plurality of adjusted part pictures to display the target virtual image performing the target action.

[0007] Optionally, the adjusting the plurality of part pictures respectively according to the action parameters of the target action to obtain a plurality of adjusted part pictures includes: obtaining the attitude angles of each body key part of the host object when performing the target action from the action parameters; and adjusting the rotation angle of the part picture corresponding to the body key part according to the attitude angle of the body key part to obtain the adjusted part picture of the body key part.

[0008] Optionally, adjusting the rotation angle of the part image corresponding to the body key part according to the posture angle of the body key part to obtain the adjusted part image of the body key part includes: when the body key part is the trunk part, obtaining the current inclination angle of the trunk part of the host object as the trunk posture angle of the target virtual image; rotating and adjusting the trunk part image matched with the trunk part of the target virtual image according to the trunk posture angle to obtain the adjusted trunk part image;

[0009] When the body key part is the arm part, obtaining the current first arm inclination angle of the first arm object in the arm part of the host object as the first arm posture angle of the target virtual image, and obtaining the current second arm inclination angle of the second arm object in the arm part of the host object as the second arm posture angle of the target virtual image; rotating and adjusting the first arm image matched with the first arm object of the target virtual image according to the first arm posture angle, and rotating and adjusting the second arm image matched with the second arm object of the target virtual image according to the second arm posture angle to obtain the adjusted arm part image;

[0010] When the body key part is the leg part, obtaining the current first leg inclination angle of the first leg object in the leg part of the host object as the first leg posture angle of the target virtual image, and obtaining the current second leg inclination angle of the second leg object in the host object's leg part as the second leg posture angle of the target virtual image; rotating and adjusting the first leg image matched with the first leg object of the target virtual image according to the first leg posture angle, and rotating and adjusting the second leg image matched with the second leg object of the target virtual image according to the second leg posture angle to obtain the adjusted leg part image.

[0011] Optionally, adjusting the rotation angle of the part image corresponding to the body key part according to the posture angle of the body key part to obtain the adjusted part image of the body key part includes: when the body key part is the head part, obtaining the current rotation angle of the head part of the host object as the head rotation angle of the target virtual image; rotating and adjusting the head part image matched with the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part image.

[0012] Optionally, before rotating and adjusting the head part picture matching the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture, it further includes: identifying the opening and closing states of the eyes and the mouth in the head part of the anchor object; determining, from the eye sequence diagram of the target virtual image, the target eye part picture that matches the opening and closing state of the eyes of the anchor object, where the eye sequence diagram includes pictures of the eyes of the target virtual image in different opening and closing states; determining, from the mouth sequence diagram of the target virtual image, the target mouth part picture that matches the opening and closing state of the mouth of the anchor object, where the mouth sequence diagram includes pictures of the mouth of the target virtual image in different opening and closing states; generating a head part picture that matches the head part of the target virtual image based on the target eye part picture and the target mouth part picture.

[0013] Optionally, the assembling the multiple adjusted part pictures to display the target virtual image performing the target action includes: determining the connection points of the multiple adjusted part pictures respectively; connecting the connection points to assemble the multiple adjusted part pictures to obtain the target virtual image performing the target action.

[0014] According to another aspect of the present invention, there is also provided a display device for a virtual phenomenon, including: an identification unit for identifying the target action currently demonstrated by the anchor object in the live broadcast screen; a first acquisition unit for acquiring multiple part pictures matching the target virtual image associated with the anchor object, where each part picture is respectively used to display a body key part of the target virtual image; an adjustment unit for respectively adjusting the multiple part pictures according to the action parameters of the target action to obtain multiple adjusted part pictures; a processing unit for assembling the multiple adjusted part pictures to display the target virtual image performing the target action.

[0015] Optionally, the adjustment unit further includes: an acquisition module for acquiring the posture angles of each body key part of the anchor object when performing the target action from the action parameters; an adjustment module for adjusting the rotation angle of the part picture corresponding to the body key part according to the posture angle of the body key part to obtain the adjusted part picture of the body key part.

[0016] Through the embodiments of the present invention, identify the target action currently demonstrated by the host object in the live broadcast screen; obtain multiple part pictures that match the target virtual image associated with the host object, where each part picture is respectively used to display a key body part of the target virtual image; adjust the multiple part pictures respectively according to the action parameters of the target action to obtain multiple adjusted part pictures; assemble the multiple adjusted part pictures to display the target virtual image performing the target action. That is to say, according to the action parameters of the target action demonstrated by the host object, adjust the multiple 2D part pictures obtained that match the target virtual image associated with the host object, and then assemble the multiple adjusted 2D part pictures. That is, only by transmitting the picture including the target virtual image from the server to the client, the target virtual image can be driven to be displayed according to the target action. This avoids the problem of excessively high costs caused by manually creating 3D models, improves the timeliness of the display of the target virtual image, enriches the diversity of virtual images, and further enhances the interest of the live broadcast. Description of the Drawings

[0017] The drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a schematic diagram of the application scenario environment of an optional method for displaying a virtual image according to an embodiment of the present invention;

[0019] Figure 2 is a flowchart of an optional method for displaying a virtual image according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an optional host object and virtual image according to an embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of each part of an optional disassembled target virtual image according to an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of each part of another optional disassembled target virtual image according to an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of the human key points of an optional host object according to an embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of the adjustment of the leg object of an optional host object according to an embodiment of the present invention;

[0025] Figure 8It is a schematic diagram of an optional method for generating a picture of the head part of a target virtual image according to an embodiment of the present invention;

[0026] Figure 9 It is a schematic diagram of an optional action switching of an anchor object according to an embodiment of the present invention;

[0027] Figure 10 It is a schematic diagram of an optional adjustment of connection points of a picture of a target virtual image part according to an embodiment of the present invention;

[0028] Figure 11 It is a schematic diagram of an optional display of a target virtual image performing a target action according to an embodiment of the present invention;

[0029] Figure 12 It is a structural block diagram of a display device for a virtual image according to an embodiment of the present invention. Detailed implementation manners

[0030] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] According to one aspect of an embodiment of the present invention, a method for displaying a virtual object is provided. Optionally, as an optional implementation manner, the above method for displaying a virtual object may but is not limited to be applied to a virtual image generation system in a hardware environment as shown in Figure 1 The virtual image generation system may include, but is not limited to, an anchor device 102, a network 104, a server 106, a database 108, and a viewing device 110. A target client (such as Figure 1The live broadcast interface shown, where the target client can be the host version client of a live broadcast platform). The viewing device 110 includes a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen is used to display the live broadcast interface of the host client (such as Figure 1 the live broadcast interface of a host client shown); it is also used to provide a human-computer interaction interface to receive human-computer interaction operations for users to use the live broadcast software for network live broadcasts. The processor is used to generate interaction instructions in response to the above human-computer interaction operations and send the interaction instructions to the server 106. The memory is used to store relevant attribute data, such as interface special effect information of the live broadcast interface, different virtual gift information of the live broadcast platform, etc. The host device 102 also includes a human-computer interaction screen, a processor, and a memory. Among them, the human-computer interaction screen is used to display the live broadcast interface of the viewer client.

[0033] The specific process is as follows: Step S102, identify the target action currently shown by the host object in the live broadcast picture; then, as in Step S104, obtain multiple part pictures that match the target virtual image associated with the host object, where each part picture is used to display a key body part of the target virtual image. The server 106 will execute Steps S108 - S116, adjust the multiple part pictures separately according to the action parameters of the target action to obtain multiple adjusted part pictures; assemble the multiple adjusted part pictures; and display the target virtual image performing the target action.

[0034] As another optional implementation manner, when the host device 102 has relatively strong computing and processing capabilities, the above Steps S108 - S116 can also be completed by the host device 102. Here, for example, this embodiment does not make any limitations in this regard.

[0035] In this embodiment, a method for displaying a virtual image is provided. Figure 2 It is a flowchart of the method for displaying a virtual image according to an embodiment of the present invention, and this process includes the following steps:

[0036] Step S202, identify the target action currently shown by the host object in the live broadcast picture;

[0037] Step S204, obtain multiple part pictures that match the target virtual image associated with the host object, where each part picture is used to display a key body part of the target virtual image;

[0038] Step S206, adjust the multiple part pictures separately according to the action parameters of the target action to obtain multiple adjusted part pictures;

[0039] Step S208, assemble the multiple adjusted part pictures to display the target virtual image performing the target action.

[0040] In this embodiment, the target virtual image is described by taking the second-generation image in the live broadcast room as an example. Specifically, as Figure 3 shown, the art design has pre-given a virtual image 302 associated with the current state of the host object 301. During the live broadcast, the virtual image 302 will adjust the actions of one or more parts that match the target action of the host object 301 according to the actions of the host object 301, that is, drive the virtual image 302 to display and execute the target action of the host object 301. For example: when the host object 301 raises the right arm, the virtual image 302 also raises the right arm; when the host object 301 shakes the head, the virtual image 302 also shakes the head, etc.

[0041] In order to drive the target virtual image to display and execute the target action of the host object 301, first, it is necessary to Figure 3 split the pre-given virtual image 302 into the following main parts: head, torso, left arm, right arm, left leg, right leg, etc. The head is further subdivided, including eyes, mouth, nose, ears, and hair, etc. Among them, the nose and ears do not need to be driven. The face pictures are pasted with the face, eyes, mouth, and hair in fixed places. Among them, the eyes and mouth are sequence diagrams for driving blinking and the mouth.

[0042] As an alternative embodiment, the pictures of each part of the disassembled target virtual image are shown in Figure 4. The main body parts of the target virtual image: head, torso, left arm, right arm, left leg, and right leg, etc. are respectively split into one picture, while the eyes and mouth can be split into sequence diagrams in different opening and closing states.

[0043] It should be noted that the pictures of the head, torso, left arm, right arm, left leg, and right leg of the disassembled target virtual image are stored in Image Set 1 in the server, while the sequence diagrams of the eyes and mouth are stored in Image Set 2 in the server, that is, the storage locations of each picture of the disassembled body part and the sequence diagrams of the eyes and mouth in the server are different.

[0044] As another alternative embodiment, the pictures of each part of the disassembled target virtual image are as Figure 5 shown. Obtain the sequence diagrams of each part of the disassembled virtual image 302 in different postures and angles. As Figure 5 shown in (a) in Figure 5 are N sequence diagrams of the torso part composed of the front image of the torso part and the side image rotated by a certain angle; as Figure 5 shown in (b) in Figure 5As shown in (d), there are Q right-leg sequence diagrams composed of images of the right leg being lifted, lowered, or rotated at different angles. Among them, N, M, P, and Q are positive integers greater than or equal to 1. There is no limit on the values of N, M, P, and Q. The four values can all be equal, some can be equal, or all can be unequal.

[0045] Further, during the live broadcast, when it is recognized that the action of the host object 301 in the picture changes, for example, the right arm changes from hanging down to being lifted, the right-arm sequence diagram of the virtual image is searched for in the server and transmitted to the client. According to the action parameters of the target action of the host object 301 (such as the angle of the lifted right arm being 90 degrees), one or more pictures in the right-arm sequence diagram of the virtual image 302 (such as image 3-1 in the right-arm sequence diagram) are adjusted so that the lifted angle in the adjusted right-arm image of the virtual image 302 is also 90 degrees; the adjusted right-arm pictures of the virtual image 302 are assembled, and the target virtual image performing the right-arm lift of 90 degrees can be displayed, ensuring the action consistency between the target virtual image and the host object.

[0046] Through the above embodiments provided by this application, according to the action parameters of the target action shown by the host object, multiple 2D part pictures matching the target virtual image associated with the host object are adjusted, and then the adjusted multiple 2D part pictures are assembled. That is, only by transmitting the picture including the target virtual image from the server to the client, the target virtual image can be driven to be displayed according to the target action. This avoids the problem of too high cost caused by manually creating 3D models and improves the timeliness of the display of the target virtual image.

[0047] In an optional embodiment, the above steps adjust multiple part pictures respectively according to the action parameters of the target action to obtain multiple adjusted part pictures, including:

[0048] Obtain the posture angles of each body key part of the host object when performing the target action from the action parameters;

[0049] According to the posture angles of the body key parts, adjust the rotation angles of the part pictures corresponding to the body key parts to obtain the adjusted part pictures of the body key parts.

[0050] In the above-mentioned embodiment of the present application, the method for recognizing the target action shown by the host object in the live broadcast picture includes, but is not limited to, recognizing the postures of each body key part of the host object when performing the target action, such as the body tilt angle and the head tilt angle of the host object. Specifically, through the limb recognition of the host object, the key points of the torso, head, and limbs are obtained, and the body tilt angle is calculated through the spatial movement distance of the torso key points; the head tilt angle is calculated through the spatial movement distance of the head key points, etc.

[0051] As an alternative implementation, the distribution of the body key points of the host object in the live broadcast screen is as Figure 6 shown. (1) is the head key point, (2), (3), (8), and (9) form the body key points, (6) and (7) form the key points of the left arm, (4) and (5) form the key points of the right arm, (11), (13) form the key points of the left leg, and (10), (12) form the key points of the right leg. Among them, the key points are used to indicate the connection points for driving the relevant parts of the body to perform the target actions.

[0052] Calculate the torso tilt angle through the spatial movement distances of the body key points (2), (3), (8), and (9), and then perform a rotation transformation on multiple torso images in the torso image sequence of the virtual image according to the torso tilt angle to obtain the adjusted torso picture.

[0053] Based on the same principle, calculate the head tilt angle through the spatial movement distance of the head key point (1), and then perform a rotation transformation on multiple head images in the head image sequence of the virtual image according to the head tilt angle to obtain the adjusted head picture; calculate the left arm tilt angle through the spatial movement distances of the left arm key points (6) and (7), and then perform a rotation transformation on multiple left arm images in the left arm image sequence of the virtual image according to the left arm tilt angle to obtain the adjusted left arm picture; for the adjusted pictures of the right arm, left leg, and right leg, the acquisition method can refer to the implementation processes of the above-mentioned torso, head, and left arm, which will not be elaborated here.

[0054] Through the above embodiments provided by the present application, through pose recognition, obtain the spatial movement distances of the key points of the key parts of the host object after movement, calculate the tilt angles of the corresponding parts through each key point, and thus perform a rotation transformation on multiple pictures of the virtual image to obtain the adjusted part pictures. Ensure the consistency between the adjusted part pictures and the target actions, and improve the timeliness of the target virtual image displaying and executing the target actions.

[0055] In an alternative embodiment, the above-mentioned adjusting the rotation angle of the part picture corresponding to the body key part according to the pose angle of the body key part to obtain the adjusted part picture of the body key part includes:

[0056] When the body key part is the torso part, obtain the current tilt angle of the torso part of the host object as the torso pose angle of the target virtual image; perform a rotation adjustment on the torso part picture matching the torso part of the target virtual image according to the torso pose angle to obtain the adjusted torso part picture;

[0057] When the key body part is the arm part, obtain the current first arm tilt angle of the first arm object in the arm part of the host object as the first arm pose angle of the target virtual image, and obtain the current second arm tilt angle of the second arm object in the arm part of the host object as the second arm pose angle of the target virtual image; rotate and adjust the first arm picture matched with the first arm object of the target virtual image according to the first arm pose angle, and rotate and adjust the second arm picture matched with the second arm object of the target virtual image according to the second arm pose angle to obtain the adjusted arm part picture;

[0058] When the key body part is the leg part, obtain the current first leg tilt angle of the first leg object in the leg part of the host object as the first leg pose angle of the target virtual image, and obtain the current second leg tilt angle of the second leg object of the host object as the second leg pose angle of the target virtual image; rotate and adjust the first leg picture matched with the first leg object of the target virtual image according to the first leg pose angle, and rotate and adjust the second leg picture matched with the second leg object of the target virtual image according to the second leg pose angle to obtain the adjusted leg part picture.

[0059] In this embodiment, the implementation methods for adjusting the part pictures of the target virtual image for different key body parts of the host object are described respectively. Specifically, when the key body part is the torso part, the tilt angle θ1 of the torso is calculated through the torso key points (2), (3), (8), (9) as shown in Figure 6 ; select the torso picture from the image set 1 shown in Figure 4 , or select the torso part picture 1-2 that matches the current torso tilt angle of the host object from the torso part sequence diagram in Figure 5 , and rotate and adjust the picture 1-2 according to the tilt angle θ1 to obtain the adjusted torso part picture (1-2).

[0060] As an optional implementation method, for Figure 5 the torso part sequence diagram in is a continuous multi-frame torso part picture with different postures. When selecting the torso part picture of the target virtual image that matches the torso part posture of the host object, it may include but is not limited to the following methods:

[0061] 1) At a frequency of selecting one frame every other frame, select one (or multiple) torso part Figure 5 from the torso sequence diagram in, and according to the torso tilt angle θ1, for the torso part Figure 1-1 Figure 1-1 ​Perform rotation adjustment to obtain the adjusted torso image (1-1); then calculate the current torso tilt angle θ2 of the host object, and select one or more torso parts from the torso sequence diagram in Figure 5 Figure 1-2 , and perform rotation adjustment on the torso part according to the torso tilt angle θ2 Figure 1-2 to obtain the adjusted torso image (1-2); and so on, track and drive the target virtual image at a frequency of every other frame;

[0062] 2) At a frequency of every s frames (a positive integer where 1 < s < N), select one or more torso bitmaps from the torso sequence diagram in Figure 5 , and adjust the torso bitmap according to the torso tilt angle of the host object at the current moment to obtain the adjusted torso image;

[0063] 3) At a frequency of every time interval t, select one or more torso bitmaps from the torso sequence diagram in Figure 5 , and adjust the torso bitmap according to the torso tilt angle of the host object at the current moment to obtain the adjusted torso image.

[0064] Similarly, when the body key part is the hip part, through pose recognition, obtain the tilt angle θ3 of the hip part of the host object, and then select one or more hip part images of the target virtual object from the hip sequence diagram shown in Figure 5 according to any of the above selection methods, and perform rotation adjustment on the hip part image according to the tilt angle θ3 to obtain the adjusted hip part image.

[0065] It should be noted that since the hip part image contains two hip objects, there are two ways to obtain the tilt angle θ3 of the hip part of the host object through pose recognition: one is to directly recognize the hip part as a whole, and the recognized tilt angle is directly used as the final tilt angle θ3 of the hip part; the other is to divide the hip part into a first hip object and a second hip object, and respectively recognize the tilt angle θ4 of the first hip object and the tilt angle θ5 of the second hip object; then respectively perform rotation adjustment on the first hip image and the second hip image according to the tilt angles θ4 and θ5, and obtain the finally adjusted hip part image based on the adjusted first hip image and the adjusted second hip image.

[0066] Similarly, when the body key part is the leg part, the leg part of the host object includes a first leg object and a second leg object, referring to the adjustment method of the above hip part image. Specifically, the following steps:

[0067] S3-1. Obtain the first leg tilt angle θ6 of the first leg object and the second leg angle θ7 of the second leg object in the right leg of the host object at the current moment through pose recognition.

[0068] S3-2. Respectively obtain the leg part pictures containing the first leg object and the second leg object from the Figure 5 right leg image sequence diagram, specifically as shown in Figure 6 .

[0069] S3-3. For the first leg object 601 in picture (a), perform rotation adjustment according to the tilt angle θ6; for the second leg object 602, perform rotation adjustment according to the tilt angle θ7.

[0070] S3-3. Based on the adjusted first leg object 602 and the adjusted second leg object 604, obtain the adjusted leg picture as shown in (b).

[0071] Through the above embodiments provided by the present application, when selecting the part pictures of the target virtual image that match different body parts of the host object, different dimensions of selection methods are set, enriching the diversity of virtual images and meeting different demand-driven methods. When rotating and adjusting the part pictures of the target virtual image according to the tilt angle or pose angle of different parts, different adjustment methods are set according to the structural characteristics of different body parts, improving the flexibility of the generation method of the target virtual image and also enhancing the consistency between the generated target image and the host object.

[0072] As an optional implementation, the above-mentioned method of adjusting the rotation angle of the part picture corresponding to the body key part according to the pose angle of the body key part to obtain the adjusted part picture of the body key part includes:

[0073] In the case where the body key part is the head part, obtain the current rotation angle of the head part of the host object as the head rotation angle of the target virtual image.

[0074] Perform rotation adjustment on the head part picture matching the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture.

[0075] In this embodiment, the adjustment methods for the part pictures of the target virtual image when the body key parts are the trunk part, the hip part, and the leg part respectively in the above embodiment can be referred to.

[0076] Specifically, when the body key part is the head part, according to Figure 6The moving distance of the head key point (1) therein is used to calculate the rotation angle θ8 of the head part of the host object at the current moment. According to the selection method described in the above embodiment, select the torso picture from the image set 1 shown in Figure 4 or select one or more head part pictures (such as selecting picture 2-2) from the head sequence images in Figure 5 . Rotate and adjust picture 2-2 according to the rotation angle θ8 to obtain the adjusted head part picture.

[0077] As an alternative embodiment, before rotating and adjusting the head part picture matching the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture, it further includes:

[0078] Identifying the opening and closing states of the eyes and the opening and closing states of the mouth in the head part of the host object;

[0079] Determining a target eye part picture matching the opening and closing state of the eyes of the host object from the eye sequence diagrams of the target virtual image, where the eye sequence diagrams include pictures of the eyes of the target virtual image in different opening and closing states;

[0080] Determining a target mouth part picture matching the opening and closing state of the mouth of the host object from the mouth sequence diagrams of the target virtual image, where the mouth sequence diagrams include pictures of the mouth of the target virtual image in different opening and closing states;

[0081] Generating a head part picture matching the head part of the target virtual image based on the target eye part picture and the target mouth part picture.

[0082] In this embodiment, since the head part of the host object includes eyes and a mouth, and the eyes and the mouth can change different opening and closing states. Therefore, when selecting the head sequence diagram of the target virtual image, it is necessary to first determine the target eye part picture of the target virtual image from the head sequence images as shown in Figure 8 according to the opening and closing state of the eyes of the host object at the previous moment. For example, in picture 3-1, both eyes are in the open state, in picture 3-2, one eye is in the open state and the other eye is in the closed state, and in picture 3-R, both eyes are in the closed state.

[0083] Similarly, according to the opening and closing state of the mouth of the host object at the current moment, determine the target mouth part picture of the target virtual image from the head sequence images as shown in Figure 8 . For example, the mouth in picture 2-1 indicates the mouth shape in a smiling state, and it can be determined that it is in the open state, and the mouth in picture 2-M indicates the mouth shape in a sad state, and it can be determined that it is in the closed state.

[0084] Further, according to the eye sequence of the target virtual image Figure 3-1 , 3-2…3-R and the mouth sequence Figure 4-1 , 4-2…4-T, the following steps are adopted to generate the head part pictures 5-1, 5-2…5-U that match the head part of the target virtual image:

[0085] S1, extract features from the eye sequence Figure 3-1 , 3-2…3-R to obtain the first feature corresponding to the eye sequence diagram;

[0086] S2, extract features from the mouth sequence Figure 4-1 , 4-2…4-T to obtain the second feature corresponding to the mouth sequence diagram;

[0087] S3, fuse the first feature and the second feature to generate the head part picture that matches the head part of the target virtual image.

[0088] Through the above embodiments provided by the present application, according to the opening and closing states of the eyes and mouth of the recognized host object, the target eye part picture and the target mouth part picture that match the opening and closing states of the eyes and mouth of the host object are determined from the body part sequence diagrams of the target virtual image. According to the eye part sequence diagram and the mouth part sequence diagram of the virtual image, a face image that displays the execution of the actions synchronized with the eyes and mouth of the host object is generated, achieving the technical effect of face driving for the target virtual image.

[0089] As an alternative implementation manner, the above-mentioned assembling of multiple adjusted part pictures to display the target virtual image performing the target action includes:

[0090] Determine the connection points of each of the multiple adjusted part pictures;

[0091] Connect the connection points to assemble the multiple adjusted part pictures to obtain the target virtual image performing the target action.

[0092] Specifically, when it is recognized that the target action of the host object is to raise the right arm and turn the head, as Figure 9 shown, determine the sequence diagrams of the head and the right arm from the part sequence diagrams of the target virtual image. Assume that one head sequence Figure 5-1 and one right arm sequence Figure 6-1 are respectively determined, as Figure 10As shown in the figure. According to the head rotation angle of the host object, the head connection point A0 in Figure 5-1 is adjusted to the connection point A1 in Figure 5-2; according to the right arm tilt angle of the host object, the rotation angles B0 and C0 of the connection points in Figure 6-1 are adjusted to the connection points B1 and C1 in Figure 6-2; the adjusted connection points A1, B1, and C1 are connected to obtain the virtual image 1102 as shown in Figure 11 the figure.

[0093] It should be noted that connecting the adjusted connection points means connecting the adjusted part connection points to the pre-set (art design) torso connection points respectively. For example, assuming that the connection points include the head connection point Z1 and the right arm connection point Z2, connect Figure 11 the adjusted head connection point A1 in to the head connection point Z1 in the torso connection points, and connect the adjusted right arm connection point B1 to the right arm connection point Z2 in the torso connection points. With other part connection points remaining unchanged, by the corresponding connection between A1 and Z1, the adjusted Figure 5-2 is pasted onto the torso picture; by the corresponding connection between B1 and Z2, the adjusted Figure 6-2 is pasted onto the torso picture; thus, the assembly of the adjusted part pictures can be completed, and the target virtual image 1102 that performs the action of raising the right arm and turning the head can be displayed.

[0094] It can be understood that the adjusted part pictures are determined by the number of human body parts involved in the target action of the recognized host object and the amplitude of the action transformation. Therefore, in this embodiment, the number of the adjusted part pictures is not limited.

[0095] Through the above embodiments provided by the present application, by rotating and splicing the sequence diagrams of the head, torso, left arm, right arm, etc. of the target virtual image, a target virtual image that can be driven by limbs and face is reassembled. It avoids the problem of excessively high costs caused by manually making 3D models. And based on the rotation splicing and assembly of two-dimensional images to generate the target virtual image, it saves the transmission time of 3D model data from the server to the client, improves the timeliness of the target virtual image to display and execute the target action, and further enriches the diversity of live interaction.

[0096] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0097] In this embodiment, a display device for a virtual image is further provided. This device is used to implement the above embodiments and preferred implementation methods, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0098] Figure 12 is a structural block diagram of a display device for a virtual image according to an embodiment of the present invention. This device includes:

[0099] An identification unit 1202, configured to identify a target action currently demonstrated by a host object in a live broadcast screen;

[0100] A first acquisition unit 1204, configured to acquire multiple part pictures that match a target virtual image associated with the host object, where each part picture is respectively used to display a key body part of the target virtual image;

[0101] An adjustment unit 1206, configured to respectively adjust the multiple part pictures according to the action parameters of the target action to obtain multiple adjusted part pictures;

[0102] A processing unit 1208, configured to assemble the multiple adjusted part pictures to display the target virtual image performing the target action.

[0103] Optionally, the above adjustment unit 1206 further includes:

[0104] An acquisition module, configured to acquire the pose angles of each key body part of the host object when performing the target action from the action parameters;

[0105] An adjustment module, configured to adjust the rotation angle of the part picture corresponding to the key body part according to the pose angle of the key body part to obtain an adjusted part picture of the key body part.

[0106] Optionally, the above adjustment module further includes:

[0107] The first adjustment sub-module is used to obtain the current tilt angle of the trunk part of the host object as the trunk posture angle of the target virtual image when the key body part is the trunk part; rotate and adjust the trunk part picture matched with the trunk part of the target virtual image according to the trunk posture angle to obtain the adjusted trunk part picture;

[0108] The second adjustment sub-module is used to obtain the current first arm tilt angle of the first arm object in the arm part of the host object as the first arm posture angle of the target virtual image, and obtain the current second arm tilt angle of the second arm object in the arm part of the host object as the second arm posture angle of the target virtual image when the key body part is the arm part; rotate and adjust the first arm picture matched with the first arm object of the target virtual image according to the first arm posture angle, and rotate and adjust the second arm picture matched with the second arm object of the target virtual image according to the second arm posture angle to obtain the adjusted arm part picture;

[0109] The third adjustment sub-module is used to obtain the current first leg tilt angle of the first leg object in the leg part of the host object as the first leg posture angle of the target virtual image, and obtain the current second leg tilt angle of the second leg object of the host object as the second leg posture angle of the target virtual image when the key body part is the leg part; rotate and adjust the first leg picture matched with the first leg object of the target virtual image according to the first leg posture angle, and rotate and adjust the second leg picture matched with the second leg object of the target virtual image according to the second leg posture angle to obtain the adjusted leg part picture.

[0110] Optionally, adjusting the rotation angle of the part picture corresponding to the key body part according to the posture angle of the key body part to obtain the adjusted part picture of the key body part includes:

[0111] The first acquisition sub-module is used to obtain the current rotation angle of the head part of the host object as the head rotation angle of the target virtual image when the key body part is the head part;

[0112] The fourth adjustment sub-module is used to rotate and adjust the head part picture matched with the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture.

[0113] Optionally, before rotating and adjusting the head part picture matched with the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture, it further includes:

[0114] An identification sub-module, configured to identify the opening and closing states of the eyes and the opening and closing states of the mouth in the head part of the host object;

[0115] A first determination sub-module, configured to determine, from the eye sequence diagram of the target virtual image, a target eye part picture that matches the opening and closing state of the eyes of the host object, where the eye sequence diagram includes pictures of the eyes of the target virtual image in different opening and closing states;

[0116] A second determination sub-module, configured to determine, from the mouth sequence diagram of the target virtual image, a target mouth part picture that matches the opening and closing state of the mouth of the host object, where the mouth sequence diagram includes pictures of the mouth of the target virtual image in different opening and closing states;

[0117] A generation sub-module, configured to generate a head part picture that matches the head part of the target virtual image based on the target eye part picture and the target mouth part picture.

[0118] Optionally, the above-mentioned assembling of multiple adjusted part pictures to display the target virtual image performing the target action includes:

[0119] A determination module, configured to determine the connection points of the multiple adjusted part pictures respectively;

[0120] A connection module, configured to connect the connection points to assemble the multiple adjusted part pictures to obtain the target virtual image performing the target action.

[0121] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0122] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing the following steps:

[0123] S1, identifying the target action currently shown by the host object in the live broadcast screen;

[0124] S2, obtaining multiple part pictures that match the target virtual image associated with the host object, where each part picture is respectively used to display a key body part of the target virtual image;

[0125] S3, respectively adjusting the multiple part pictures according to the action parameters of the target action to obtain multiple adjusted part pictures;

[0126] S4, assembling the multiple adjusted part pictures to display the target virtual image performing the target action.

[0127] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memory (ROM), random access memory (RAM), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0128] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated herein.

[0129] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0130] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0131] S1, identify the target action currently demonstrated by the host object in the live broadcast screen;

[0132] S2, obtain multiple part images that match the target virtual image associated with the host object, where each part image is respectively used to display a key body part of the target virtual image;

[0133] S3, respectively adjust the multiple part images according to the action parameters of the target action to obtain multiple adjusted part images;

[0134] S4, assemble the multiple adjusted part images to display the target virtual image performing the target action.

[0135] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0136] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated herein.

[0137] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0138] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for displaying an avatar, characterized in that, Including: Identifying a target action currently demonstrated by a host object in a live video stream; Obtaining multiple part images that match a target virtual image associated with the host object, where each of the part images is respectively used to display a key body part of the target virtual image; Obtaining the posture angles of each key body part of the host object when performing the target action according to the action parameters of the target action; Adjusting the rotation angles of the part images corresponding to the key body parts according to the posture angles of the key body parts to obtain multiple adjusted part images of the key body parts; Determining connection points of the multiple adjusted part images, connecting the connection points, and assembling the multiple adjusted part images to obtain the target virtual image performing the target action.

2. The method according to claim 1, wherein The adjusting the rotation angles of the part images corresponding to the key body parts according to the posture angles of the key body parts to obtain multiple adjusted part images of the key body parts includes: When the key body part is the trunk part, obtaining the current inclination angle of the trunk part of the host object as the trunk posture angle of the target virtual image; rotating and adjusting the trunk part image that matches the trunk part of the target virtual image according to the trunk posture angle to obtain an adjusted trunk part image; When the key body part is the arm part, obtaining the current first arm inclination angle of a first arm object in the arm part of the host object as the first arm posture angle of the target virtual image, and obtaining the current second arm inclination angle of a second arm object in the arm part of the host object as the second arm posture angle of the target virtual image; rotating and adjusting the first arm image that matches the first arm object of the target virtual image according to the first arm posture angle, and rotating and adjusting the second arm image that matches the second arm object of the target virtual image according to the second arm posture angle to obtain an adjusted arm part image; When the key body part is the leg part, obtaining the current first leg inclination angle of a first leg object in the leg part of the host object as the first leg posture angle of the target virtual image, and obtaining the current second leg inclination angle of a second leg object in the leg part of the host object as the second leg posture angle of the target virtual image; rotating and adjusting the first leg image that matches the first leg object of the target virtual image according to the first leg posture angle, and rotating and adjusting the second leg image that matches the second leg object of the target virtual image according to the second leg posture angle to obtain an adjusted leg part image.

3. The method according to claim 1, wherein The adjusting the rotation angles of the part images corresponding to the key body parts according to the posture angles of the key body parts to obtain multiple adjusted part images of the key body parts includes: When the critical body part is the head part, obtain the current rotation angle of the head part of the host object as the head rotation angle of the target virtual image; Rotate and adjust the head part picture matching the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture.

4. The method according to claim 3, wherein Before rotating and adjusting the head part picture matching the head part of the target virtual image according to the head rotation angle to obtain the adjusted head part picture, it further includes: Identify the opening and closing states of the eyes and the opening and closing states of the mouth in the head part of the host object; Determine a target eye part picture matching the opening and closing state of the eyes of the host object from the eye sequence diagram of the target virtual image, where the eye sequence diagram includes pictures of the eyes of the target virtual image in different opening and closing states; Determine a target mouth part picture matching the opening and closing state of the mouth of the host object from the mouth sequence diagram of the target virtual image, where the mouth sequence diagram includes pictures of the mouth of the target virtual image in different opening and closing states; Generate a head part picture matching the head part of the target virtual image based on the target eye part picture and the target mouth part picture.

5. A display device for a virtual avatar, characterized in that, It includes: An identification unit for identifying the target action currently shown by the host object in the live broadcast screen; A first acquisition unit for acquiring multiple part pictures matching the target virtual image associated with the host object, where each part picture is used to display a critical body part of the target virtual image; An adjustment unit for obtaining the posture angles of each critical body part of the host object when performing the target action according to the action parameters of the target action; rotating the rotation angle of the part picture corresponding to the critical body part according to the posture angle of the critical body part to obtain multiple adjusted part pictures of the critical body part; A processing unit for determining the connection points of the multiple adjusted part pictures, connecting the connection points, and assembling the multiple adjusted part pictures to obtain the target virtual image performing the target action.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where the program, when running, executes the method described in any one of claims 1 to 4.

7. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is set to execute the method described in any one of claims 1 to 4 through the computer program.

Images