2d live image generation method and device, storage medium, and electronic device

Abstract

The present disclosure relates to a 2D live image generation method and device, a storage medium and an electronic device, and relates to the technical field of computers. The method comprises: acquiring a real-time anchor face image in a live broadcast process, and determining a real-time expression feature parameter corresponding to the real-time anchor face image; based on the real-time expression feature parameter, a target 2D expression map matching the real-time expression feature parameter is acquired from a preset 2D expression map library; the acquired target 2D expression map is attached to a preset virtual image base map, and a virtual anchor image in the live broadcast process is generated in real time. The present disclosure realizes the live broadcast effect of a 2D cartoon character, reduces the cost, and improves the degree of freedom of a user during live broadcast.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method and apparatus for generating 2D live streaming images, a storage medium, and an electronic device. Background Technology

[0002] Currently, virtual avatar-based live streaming is a new trend in live streaming. In virtual avatar-based live streaming, users can create a virtual 3D avatar by using provided materials such as clothing, hairstyle, and outfit. In the live streaming room, motion capture technology is used to capture the user's facial movements, thereby driving the virtual 3D avatar to make the same performance.

[0003] However, creating and maintaining virtual 3D avatar-related materials requires significant investment. Furthermore, the created materials may not meet user preferences, limiting user freedom during live streams.

[0004] It should be noted that the information in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a 2D live streaming image generation method, a 2D live streaming image generation device, a storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the problems of high cost and low user freedom in related technologies based on virtual 3D images due to limitations and defects in related technologies.

[0006] According to one aspect of this disclosure, a method for generating 2D live-streaming avatars is provided, comprising: Acquire real-time facial images of the anchor during the live broadcast, and determine the real-time facial expression feature parameters corresponding to the real-time facial images of the anchor; Based on the real-time facial expression feature parameters, a target 2D facial expression image matching the real-time facial expression feature parameters is obtained from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained by 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. The acquired target 2D emoticon texture is attached to the preset virtual avatar base image to generate the virtual avatar of the anchor during the live broadcast in real time.

[0007] In one exemplary embodiment of this disclosure, determining the real-time facial expression feature parameters corresponding to the real-time anchor's facial image includes: The eyes and mouth included in the real-time anchor's facial image are obtained, and the real-time expression feature parameters of the left eye, the right eye, and the mouth are determined.

[0008] In one exemplary embodiment of this disclosure, before obtaining the target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: 2D expression texture samples and face deformer models are obtained. The face deformer model is trained using each 2D expression sample and the corresponding expression feature parameters to obtain the expression texture model.

[0009] In one exemplary embodiment of this disclosure, before obtaining the target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: The 2D facial expression map sample is segmented to obtain a segmented image, and the face position in the segmented image is obtained; The face in the segmented image is cropped according to the face position to obtain a face image; The face image and the facial feature parameters are input into the facial texture model to obtain the 2D facial texture.

[0010] In one exemplary embodiment of this disclosure, the step of segmenting the 2D expression map sample to obtain a segmented image and obtaining the face position in the segmented image includes: Obtain an anime character segmentation network and an anime face detection network; The anime character segmentation network is used to detect anime characters in the 2D facial expression texture samples and to segment the anime characters to obtain the segmented image; The face detection network described above is used to detect faces in the segmented image to obtain the face locations in the segmented image.

[0011] In one exemplary embodiment of this disclosure, the step of inputting the face image and expression feature parameters into the expression mapping model to obtain the 2D expression map includes: The face image and the facial feature parameters are input into the facial texture model to obtain a facial transformation image; The facial expression transformation image is cropped to obtain the 2D facial expression texture.

[0012] In one exemplary embodiment of this disclosure, cropping the facial expression transformation image to obtain the 2D facial expression texture includes: A key point detection model for anime faces is obtained, and the key point detection model for anime faces is used to detect the expression change image to obtain the left eye region, right eye region, and mouth region in the expression change image. The left eye region, the right eye region, and the mouth region in the expression transformation image are cropped to obtain a 2D expression texture for the left eye, a 2D expression texture for the right eye, and a 2D expression texture for the mouth.

[0013] In one exemplary embodiment of this disclosure, obtaining a target 2D emoticon image that matches the real-time emoticon feature parameters from a preset 2D emoticon image library based on the real-time emoticon feature parameters includes: Based on the real-time expression feature parameters of the left eye, a 2D expression texture of the left eye corresponding to the real-time expression feature parameters of the left eye is obtained; Based on the real-time expression feature parameters of the right eye, a 2D expression texture of the right eye corresponding to the real-time expression feature parameters of the right eye is obtained; Based on the real-time facial expression feature parameters, a 2D facial expression texture map corresponding to the real-time facial expression feature parameters is obtained.

[0014] In one exemplary embodiment of this disclosure, before attaching the acquired target 2D emoticon texture to a preset virtual avatar base image, the method further includes: Obtain a virtual avatar image and the facial expression closure parameter from the facial expression feature parameters, segment the virtual avatar image to obtain a segmented image, and obtain the face position in the segmented image; The face in the segmented image is cropped according to the face position to obtain a face image; The face image and the fully closed expression parameters are input into the expression mapping model to obtain a fully closed expression face. The fully closed expression face is then pasted back into the segmented image to obtain the virtual image base map.

[0015] In an exemplary embodiment of this disclosure, before obtaining a target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: The real-time body posture of the anchor during the live broadcast is obtained, and the real-time body posture parameters corresponding to the real-time body posture of the anchor are determined. Based on the real-time body posture parameters, a target 2D body texture that matches the body posture parameters is obtained from a 2D body texture library.

[0016] In one exemplary embodiment of this disclosure, before obtaining a 2D body texture matching the body posture parameters from a preset 2D body texture map based on the real-time body posture parameters, the method further includes: Obtain a body texture model, input the segmented image and preset body control parameters into the body texture model to obtain a 2D body texture.

[0017] In one exemplary embodiment of this disclosure, the method further includes: The 2D body texture, which matches the real-time body posture parameters, is attached to the preset virtual avatar base map.

[0018] According to one aspect of this disclosure, a 2D live streaming avatar generation apparatus is provided, comprising: The real-time facial expression feature parameter acquisition module is used to acquire real-time facial images of the anchor during the live broadcast and determine the real-time facial expression feature parameters corresponding to the real-time facial images of the anchor. The facial expression matching module is used to obtain a target 2D facial expression image that matches the real-time facial expression feature parameters from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained by 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. The virtual avatar generation module is used to attach the acquired target 2D expression texture to the preset virtual avatar base image to generate the virtual avatar of the anchor during the live broadcast in real time.

[0019] According to one aspect of this disclosure, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the 2D live streaming image generation method described in any of the exemplary embodiments above.

[0020] According to one aspect of this disclosure, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to execute the 2D live streaming image generation method described in any of the above exemplary embodiments by executing the executable instructions.

[0021] This disclosure provides a method for generating a 2D live stream avatar. The method involves acquiring a real-time facial image of the streamer during a live stream and determining the real-time facial expression feature parameters corresponding to the image. Based on these parameters, a target 2D facial expression map matching the parameters is obtained from a pre-set 2D facial expression map library. The library includes multiple 2D facial expression maps pre-generated using a facial expression map model, which is trained using 2D facial expression map samples with different facial expression feature parameters. Each 2D facial expression map is used to simulate a specific facial expression. The acquired target 2D facial expression map is then attached to a pre-set virtual avatar base image to generate the streamer's virtual avatar in real-time. On the one hand, it supports inputting virtual avatar images and generating virtual avatar background images based on these images. During live streaming, the real-time facial expression feature parameters of the streamer can be determined. By generating the streamer's virtual avatar using the target 2D emoticon texture corresponding to the real-time facial expression feature parameters and the virtual avatar background image, it solves the problem in related technologies where the materials for creating virtual 3D avatars may not meet the user's preferences, resulting in low user freedom and improving user freedom during live streaming. On the other hand, after obtaining the streamer's real-time facial expression feature parameters, a target 2D emoticon texture matching the real-time facial expression feature parameters can be obtained from a preset 2D emoticon image library. This solves the problem in related technologies where a large amount of cost is required to create virtual 3D avatar materials, thus reducing costs.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 The flowchart illustrates a method for generating a 2D live stream character according to an exemplary embodiment of the present disclosure.

[0025] Figure 2 The flowchart illustrates a method for generating a 2D live avatar according to an example embodiment of the present disclosure, prior to obtaining a target 2D emoticon image that matches real-time emoticon feature parameters from a preset 2D emoticon image library.

[0026] Figure 3The flowchart illustrates a method for segmenting a 2D facial expression map sample to obtain a segmented image and acquiring the face position in the segmented image, according to an exemplary embodiment of the present disclosure.

[0027] Figure 4 The flowchart illustrates a method for inputting a face image and facial feature parameters into an expression mapping model to obtain a 2D expression map, according to an example embodiment of the present disclosure.

[0028] Figure 5 The flowchart illustrates a method for cropping an expression transformation image to obtain a 2D expression texture according to an example embodiment of the present disclosure.

[0029] Figure 6 The flowchart illustrates a method for obtaining a target 2D emoticon image that matches the real-time emoticon feature parameters from a preset 2D emoticon image library, based on real-time emoticon feature parameters, according to an example embodiment of the present disclosure.

[0030] Figure 7 The flowchart illustrates a method for generating a 2D live avatar according to an example embodiment of the present disclosure, before attaching the acquired target 2D emoticon texture to a preset virtual avatar base map.

[0031] Figure 8 The flowchart illustrates a method for generating a 2D live stream character according to an example embodiment of the present disclosure, prior to obtaining a target 2D emoticon image that matches the real-time emoticon feature parameters from a preset 2D emoticon image library based on the real-time emoticon feature parameters.

[0032] Figure 9 The diagram schematically illustrates a block diagram of a 2D live streaming image generation apparatus according to an exemplary embodiment of the present disclosure.

[0033] Figure 10 An electronic device for implementing a 2D live streaming image generation method according to an exemplary embodiment of the present invention is illustrated. Detailed Implementation

[0034] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make the invention more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention may be practiced with one or more of these specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the invention.

[0035] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] This exemplary embodiment first provides a 2D live-streaming avatar generation method. This method can run on a device terminal, which may include a PC, a mobile device, etc. Of course, those skilled in the art can also run the method disclosed herein on other platforms as needed, and this exemplary embodiment does not impose any special limitations on this. (See references) Figure 1 As shown, the image generation method may include the following steps: Step S110. Obtain the real-time facial image of the anchor during the live broadcast, and determine the real-time facial expression feature parameters corresponding to the real-time facial image of the anchor; Step S120. Based on the real-time facial expression feature parameters, obtain a target 2D facial expression image that matches the real-time facial expression feature parameters from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained using 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. Step S130. Attach the acquired target 2D expression texture to the preset virtual image base image to generate the virtual image of the anchor during the live broadcast in real time.

[0037] The aforementioned 2D live stream avatar generation method involves acquiring a real-time facial image of the streamer during the live stream, determining the real-time facial expression feature parameters corresponding to the real-time facial image, and based on the real-time facial expression feature parameters, acquiring a target 2D facial expression texture that matches the real-time facial expression feature parameters from a preset 2D facial expression texture library. The 2D facial expression texture library includes multiple 2D facial expression textures pre-generated using a facial expression texture model, which is trained using 2D facial expression texture samples with different facial expression feature parameters. Each 2D facial expression texture is used to simulate a facial expression. The acquired target 2D facial expression texture is then attached to a preset virtual avatar base image to generate the streamer's virtual avatar in real time during the live stream. On the one hand, it supports inputting virtual avatar images and generating virtual avatar background images based on these images. During live streaming, it can determine the real-time facial expression feature parameters of the streamer, and generate the streamer's virtual avatar using the target 2D emoticon texture corresponding to the real-time facial expression feature parameters and the virtual avatar background image. This solves the problem in related technologies where the materials for creating virtual 3D avatars may not meet user preferences, resulting in low user freedom, and improves user freedom during live streaming. On the other hand, after obtaining the streamer's real-time facial expression feature parameters, it can obtain target 2D emoticon textures that match the real-time facial expression feature parameters from a preset 2D emoticon image library. This solves the problem in related technologies where a large amount of cost is required to create virtual 3D avatar materials, and reduces costs.

[0038] The following provides a detailed explanation and description of each step involved in the 2D live streaming image generation method of the exemplary embodiments of this disclosure.

[0039] First, the application scenarios and purposes of the exemplary embodiments of this disclosure will be explained and described. Specifically, the exemplary embodiments of this disclosure can drive virtual character images to make the same movements based on the facial movements of live streaming users, thereby achieving the live streaming effect of 2D anime characters. The main research focuses on how to reduce product costs and increase user freedom of choice in live streaming based on virtual avatars.

[0040] In this disclosure, based on the real-time facial expressions of the broadcaster, after obtaining the real-time facial image of the broadcaster, the real-time facial expression feature parameters corresponding to the real-time facial image of the broadcaster are determined. The real-time facial expression feature parameters are matched in a preset 2D facial expression texture library to obtain a target 2D facial expression texture corresponding to the real-time facial expression feature parameters. The target 2D facial expression texture is then attached to the virtual image base map to generate the virtual image of the broadcaster in real time during the live broadcast.

[0041] Next, steps S110-S130 will be further explained and described.

[0042] In step S110, a real-time facial image of the anchor is acquired during the live broadcast, and the real-time facial expression feature parameters corresponding to the real-time facial image of the anchor are determined.

[0043] In this example embodiment, when a broadcaster is live streaming on a client, a real-time facial image of the broadcaster is acquired through the user device's camera. Based on the acquired real-time facial image, the broadcaster's real-time facial expression feature parameters can be obtained. Determining the real-time facial expression feature parameters corresponding to the real-time broadcaster's facial image includes: The eyes and mouth included in the real-time anchor's facial image are obtained, and the real-time expression feature parameters of the left eye, the right eye, and the mouth are determined.

[0044] Specifically, after acquiring a real-time image of the anchor's face, the states of the anchor's left eye, right eye, and mouth can be obtained to acquire real-time facial expression feature parameters. That is, real-time facial expression feature parameters for the left eye, right eye, and mouth are obtained. These real-time facial expression feature parameters can be represented by values ​​from 0 to 1; when the anchor's left eye is fully closed, the left eye real-time facial expression feature parameter can be represented by 0; when the anchor's right eye is fully open, the right eye real-time facial expression feature parameter can be represented by 1; when the anchor's mouth is half-open, the mouth real-time facial expression feature parameter can be represented by 0.5. In this example embodiment, no specific limitations are imposed on these parameters.

[0045] In step S120, based on the real-time facial expression feature parameters, a target 2D facial expression image matching the real-time facial expression feature parameters is obtained from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained using 2D facial expression image samples with different facial expression feature parameters, and each 2D facial expression image is used to simulate a facial expression.

[0046] In this example embodiment, after obtaining the real-time facial expression feature parameters of the anchor, a target 2D facial expression texture can be obtained by matching the anchor's real-time facial expression feature parameters in a preset 2D facial expression texture library.

[0047] Generating 2D emojis using emoji models involves massive computation. If this method were built into the streamer's phone, it would place high demands on the phone's performance, increasing the barrier to entry for users. Deploying the same method to a server would require real-time calculations and transformations for each streamer, increasing server costs and introducing instability due to network latency. Therefore, a pre-built 2D emoji library is stored as a resource package on the streamer's client, reducing latency and stuttering during live streams, lowering costs, and improving the efficiency of 2D emoji matching.

[0048] When generating 2D facial expressions using a facial expression mapping model, the facial expression feature parameters input to the model can be multiple sets of parameters. These parameters include the state parameters of the left eye, right eye, and mouth, which can be represented by 0-1. Each set of facial expression feature parameters has a different state parameter. When the state parameters of the left eye, right eye, and mouth are all 0, it indicates that the eyes and mouth are closed. When the state parameters of the left eye, right eye, and mouth are all 1, it indicates that the eyes and mouth are fully open.

[0049] In this example embodiment, before obtaining the target 2D emoji image matching the real-time emoji feature parameters from a preset 2D emoji image library, the method further includes: 2D expression texture samples and face deformer models are obtained. The face deformer model is trained using each 2D expression sample and the corresponding expression feature parameters to obtain the expression texture model.

[0050] Specifically, a face deformer model and 2D expression map samples can be obtained. Using each sample in the 2D expression map samples and the expression feature parameters corresponding to each 2D expression sample, the face deformer model is trained to obtain an expression map model. The face deformer model, Face Morpher, can change facial expressions by generating another image containing modifications to the 2D expression map. These changes are combined with the 2D expression map through an alpha mask, which is also generated by the face deformer itself.

[0051] In this example embodiment, reference Figure 2 As shown, before obtaining the target 2D emoticon image that matches the real-time emoticon feature parameters from the preset 2D emoticon image library, the method may further include: Step S210. Segment the 2D expression map sample to obtain a segmented image, and obtain the face position in the segmented image; Step S220. Cropping the face in the segmented image according to the face position to obtain a face image; Step S230. Input the face image and the expression feature parameters into the expression mapping model to obtain the 2D expression mapping.

[0052] The following will further explain and illustrate steps S210-S230. Specifically, firstly, the 2D expression map samples are preprocessed. This preprocessing includes segmenting the 2D expression map samples to obtain segmented images and obtaining the face positions in the segmented images. When segmenting the 2D expression map samples, an anime character segmentation network can be used to detect anime characters in the 2D expression map samples and segment the detected anime characters. After obtaining the segmented images, an anime character face detection network is used to detect the faces of the anime characters in the segmented images, obtaining the face positions of the anime characters in the segmented images. After obtaining the face positions, the faces in the segmented images are cropped according to the face positions to obtain face images. The face image and expression feature parameters are then input into the expression map model to obtain the 2D expression map.

[0053] The anime character segmentation network can be a high-precision binary image segmentation network, designed to segment objects with high accuracy from natural images using extremely fine-grained labels for annotation. A novel baseline network, IS-Net, is introduced to reduce overfitting by enforcing direct feature synchronization in a high-dimensional feature space. Additionally, a new manual correction measure, HCE, is designed to measure the degree of barrier between model predictions and real-world applications by calculating the human intervention required to correct error regions. The anime face detection network can be based on LBP (Local Binary Patterns), HOG (Histogram of Oriented Gradients), or SSD (Single Shot MultiBox Detector). In this example embodiment, no specific limitation is made to the anime face detection network.

[0054] Further reference Figure 3 As shown, the step of segmenting the 2D facial expression map sample to obtain a segmented image and then obtaining the face position in the segmented image may include: Step S310. Obtain the anime character segmentation network and the anime face detection network; Step S320. Use the anime character segmentation network to detect anime characters in the 2D facial expression texture samples and segment the anime characters to obtain the segmented image; Step S330. Use the animation face detection network to detect faces in the segmented image to obtain the face positions in the segmented image.

[0055] The following will further explain and illustrate steps S310-S330. Specifically, firstly, an anime character segmentation network and an anime face detection network are obtained. The anime face segmentation network is used to detect anime characters in 2D expression texture samples and segment the detected anime characters to obtain segmented images. After obtaining the segmented images, the anime face detection network is used to detect the faces of the anime characters in the segmented images to obtain the face positions in the segmented images. The anime character segmentation network can be a high-precision bisection image segmentation network, which aims to segment high-precision objects from natural images and uses extremely fine-grained labels for annotation. A novel baseline network, IS-Net, is introduced to reduce overfitting by enforcing direct feature synchronization in a high-dimensional feature space. In addition, a new artificial correction quantity, HCE, is designed to measure the degree of barrier between model prediction and real-world application by calculating the human intervention required to correct error regions. The anime face detection network can be based on LBP (Local Binary Patterns), HOG (Histogram of Oriented Gradient), or SSD (Single Shot MultiBox Detector). In this example embodiment, no specific limitation is made on the anime face detection network.

[0056] In this example embodiment, after obtaining the face image from the 2D expression map sample, the face image and expression feature parameters can be input into the expression map model, for reference. Figure 4 As shown, the step of inputting the face image and facial feature parameters into the facial texture model to obtain the 2D facial texture may include: Step S410. Input the face image and the expression feature parameters into the expression mapping model to obtain the expression transformation image; Step S420. Crop the expression transformation image to obtain the 2D expression texture.

[0057] The following will further explain and illustrate steps S410 and S420. Specifically, the face image and expression feature parameters are input into the expression mapping model. Through this expression mapping model, an expression transformation image of the face image corresponding to the input expression feature parameters is obtained. Since there are multiple sets of expression feature parameters, there are also multiple expression transformation images. In each expression transformation image, the expression of the anime character is different. This difference can be reflected in the different closing states of the anime character's left eye, right eye, and mouth in each expression transformation image.

[0058] After obtaining the facial expression transformation image, it can be cropped to obtain images of the anime character's eyes and closed mouth, thus obtaining a 2D facial expression texture. (Reference) Figure 5 As shown, cropping the facial expression transformation image to obtain a 2D facial expression texture can include: Step S510. Obtain the anime face key point detection model, and use the anime face key point detection model to detect the expression change image to obtain the left eye region, right eye region and mouth region in the expression change image; Step S520. Cropping the left eye region, the right eye region, and the mouth region in the expression transformation image to obtain a left eye 2D expression texture, a right eye 2D expression texture, and a mouth 2D expression texture.

[0059] The following will further explain and illustrate steps S510 and S520. Specifically, firstly, an anime face keypoint detection model is obtained. This model is used to detect facial keypoints in the expression transformation image, obtaining the facial keypoints of the anime character in the expression transformation image. Based on the facial keypoints, the keypoints of the left eye region, right eye region, and mouth region are connected to obtain the boxes for the left eye region, right eye region, and mouth region in the transformation image. These boxes are then cropped to obtain multiple expression maps. That is, each expression map includes a left eye 2D expression map, a right eye 2D expression map, and a mouth 2D expression map. In the left eye 2D expression map, the expression feature parameters corresponding to each expression map are different; in the right eye 2D expression map, the expression feature parameters corresponding to each expression map are different; and in the mouth 2D expression map, the expression parameters corresponding to each expression map are different.

[0060] The facial landmark detection model can be the PFLD (Practical Facial Landmark Detector) model. In this PFLD model, features at three different scales are fused to enhance the model's expressive power. The backbone of this network structure uses Mobilenet-v2, and a sub-network is introduced during the training process of PFLD to supervise the training of the PFLD network model. The input of this sub-network is not training data, but the intermediate output of the PFLD main network Mobilenet-v2.

[0061] In this example embodiment, after generating 2D emoticon textures using an emoticon texture model, a 2D emoticon texture library can be generated based on the generated 2D emoticon textures. Once a preset 2D emoticon texture library is obtained, matching can be performed within the preset 2D emoticon texture library based on real-time emoticon feature parameters, with reference to... Figure 6 As shown, based on the real-time facial expression feature parameters, obtaining a target 2D facial expression texture that matches the real-time facial expression feature parameters from a preset 2D facial expression texture library may include: Step S610. Based on the real-time expression feature parameters of the left eye, obtain the 2D expression texture of the left eye corresponding to the real-time expression feature parameters of the left eye; Step S620. Based on the real-time expression feature parameters of the right eye, obtain the 2D expression texture of the right eye corresponding to the real-time expression feature parameters of the right eye; Step S630. Obtain a 2D mouth expression texture corresponding to the real-time mouth expression feature parameters based on the real-time mouth expression feature parameters.

[0062] The following will further explain and illustrate steps S610-S630. Specifically, based on the real-time facial expression feature parameters of the anchor's left eye, a matching process is performed in a preset 2D facial expression image library to determine the facial expression feature parameter that is closest to the real-time facial expression feature parameters of the left eye, and a 2D facial expression texture for the left eye is determined based on this closest facial expression feature parameter; similarly, based on the real-time facial expression feature parameters of the anchor's right eye, a matching process is performed in a preset 2D facial expression image library to determine the facial expression feature parameter that is closest to the real-time facial expression feature parameters of the right eye, and a 2D facial expression texture for the right eye is determined based on this closest facial expression feature parameter; and similarly, based on the real-time facial expression feature parameters of the anchor's mouth, a matching process is performed in a preset 2D facial expression image library to determine the facial expression feature parameter that is closest to the real-time facial expression feature parameters of the mouth, and a 2D facial expression texture for the mouth is determined based on this closest facial expression feature parameter. For example, in a live stream, if the streamer's real-time facial expression feature parameter for the left eye is 0.1, the real-time facial expression feature parameter for the right eye is 0.8, and the real-time facial expression feature parameter for the mouth is 0, then the 2D facial expression texture library can be used to select the facial expression texture with the closest facial expression feature parameter of 0.1 for the left eye, the facial expression texture with the closest facial expression feature parameter of 0.8 for the right eye, and the facial expression texture with the closest facial expression feature parameter of 0 for the mouth.

[0063] In step S130, the acquired target 2D expression texture is attached to the preset virtual image base image to generate the virtual image of the anchor during the live broadcast in real time.

[0064] In this example embodiment, the preset virtual avatar background image can be an image generated based on the virtual avatar image. The virtual avatar image can be an image uploaded by the broadcaster after entering the live broadcast room. The uploaded image can be an image generated using AI drawing technology, or it can be a 2D emoticon sticker sample. In this example embodiment, no specific limitation is made.

[0065] Before attaching the acquired target 2D emoticon texture to the preset virtual avatar base image, a virtual avatar base image can be generated based on the virtual avatar image, for reference. Figure 7 As shown, before attaching the acquired target 2D facial expression texture to the preset virtual avatar base image, the method may further include: Step S710. Obtain the virtual avatar image and the facial expression full closure parameter in the facial expression feature parameters, segment the virtual avatar image to obtain a segmented image, and obtain the face position in the segmented image; Step S720. Cropping the face in the segmented image according to the face position to obtain a face image; Step S730. Input the face image and the expression closure parameters into the expression mapping model to obtain a face with a fully closed expression. Then, paste the face with a fully closed expression back into the segmented image to obtain the virtual image base map.

[0066] The following will further explain and illustrate steps S710-S730. Specifically, for the virtual avatar background image displayed in the live broadcast room, the eyes and mouth of the virtual avatar in the background image should be closed. Therefore, the parameters for fully closed expressions in the expression feature parameters can be obtained. After obtaining the fully closed expression parameters, the virtual avatar background image is obtained based on the fully closed expression parameters and the expression mapping model. That is, the state of the eyes and mouth of the character in the virtual avatar image is modified to be closed by the expression mapping model. Obtaining the virtual avatar background image through the expression mapping model may include: first, segmenting the virtual avatar image to obtain a segmented image, and obtaining the face position in the segmented image; then, cropping the face in the segmented image according to the face position to obtain a face image; finally, inputting the face image and the fully closed expression parameters into the expression mapping model to obtain a fully closed expression face, and pasting the fully closed expression face back into the segmented image to obtain the virtual avatar background image.

[0067] In this example embodiment, the virtual avatar's body can also be moved according to the broadcaster's body posture, with reference to... Figure 8 As shown, before obtaining a target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method may further include: Step S810. Obtain the real-time body posture of the anchor during the live broadcast and determine the real-time body posture parameters corresponding to the real-time body posture of the anchor. Step S820. Based on the real-time body posture parameters, obtain a target 2D body texture that matches the body posture parameters from the 2D body texture library.

[0068] The following will further explain and illustrate steps S810 and S820. Specifically, when a broadcaster is live streaming on a client, the real-time body posture of the broadcaster is acquired through the user's device camera. After acquiring the real-time body posture, the broadcaster's real-time body posture parameters can be determined based on the real-time body posture. Based on the real-time body posture parameters, a target 2D body texture matching the body posture parameters is obtained from a 2D body texture library. The 2D body texture can be obtained through a body texture model, and the 2D body texture library includes multiple 2D body textures with different postures. Before obtaining the 2D body texture matching the body posture parameters from a preset 2D body texture library based on the real-time body posture parameters, the method further includes: Obtain a body texture model, input the segmented image and body control parameters into the body texture model to obtain a 2D body texture.

[0069] Specifically, first, a body texture model is obtained. This model can be a body rotator model. The segmented image and body control parameters are input into the body texture model to obtain a 2D body texture. The body control parameters are multiple sets, meaning that multiple 2D body textures can be generated through this model. These parameters can be represented by numbers between 0 and 1, with different parameters representing different degrees of body tilt. The body rotator model is a complementary algorithm combining a viewpoint synthesis algorithm and an adversarial neural network (ANN). The viewpoint synthesis algorithm calculates an appearance flow, which knows which pixel in the input the color of each output pixel comes from. It excels at preserving the texture of the original image, generating detailed results, but it is not good at displaying parts that are only revealed after body rotation. The image generated by the ANN is relatively blurry, but it can fill in parts that were not visible before rotation. Multiple 2D body textures result in a 2D body texture library, which can then be stored as a resource package in the broadcaster's client.

[0070] After obtaining the 2D body texture from the body texture model, the method further includes: The 2D body texture, which matches the real-time body posture parameters, is attached to the preset virtual avatar base map.

[0071] Specifically, the 2D body texture library can be matched with the real-time body posture parameters of the streamer to obtain a 2D body texture that matches the real-time body posture parameters. After obtaining the matching 2D body texture, the 2D body texture is attached to the preset virtual image base.

[0072] The 2D live avatar generation method provided in this exemplary embodiment has at least the following advantages: Firstly, it supports user input of virtual avatar images, and drives the user-input virtual avatar image to perform the same expression based on the real-time facial expression feature parameters of the anchor, thus improving the user's freedom. Secondly, it generates 2D facial expression textures based on an expression texture model and 2D body textures based on a body texture model. When the anchor is live streaming in the live room and real-time facial expressions or body postures of the anchor are obtained, real-time facial expression feature parameters can be determined based on the real-time facial expressions or real-time body posture parameters. Then, the real-time facial expression feature parameters are used for matching to obtain the target 2D facial expression texture or the real-time body posture parameters are used for matching to obtain the target 2D body texture. This solves the problem of needing to invest a lot of cost to create virtual 3D avatar materials in related technologies, thus reducing costs. Thirdly, after generating the 2D facial expression texture library and the 2D body texture library, the 2D facial expression texture library and the 2D body texture library are stored in the anchor's client in the form of a resource package, reducing server costs and reducing instability caused by network latency during live streaming.

[0073] This disclosure also provides an apparatus for generating 2D live streaming images, as illustrated in the example embodiments. Figure 9 As shown, it may include: a real-time facial expression feature parameter acquisition module 910, an facial expression texture matching module 920, and a virtual avatar generation module 930. Among them: The real-time facial expression feature parameter acquisition module 910 is used to acquire real-time facial images of the anchor during the live broadcast and determine the real-time facial expression feature parameters corresponding to the real-time facial images of the anchor. The facial expression matching module 920 is used to obtain a target 2D facial expression image that matches the real-time facial expression feature parameters from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained by 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. The virtual avatar generation module 930 is used to attach the acquired target 2D expression texture to the preset virtual avatar base image to generate the virtual avatar of the anchor in real time during the live broadcast.

[0074] The specific details of each module in the aforementioned 2D live streaming image generation device have been described in detail in the corresponding 2D live streaming image generation method, so they will not be repeated here.

[0075] In one exemplary embodiment of this disclosure, determining the real-time facial expression feature parameters corresponding to the real-time anchor's facial image includes: The eyes and mouth included in the real-time anchor's facial image are obtained, and the real-time expression feature parameters of the left eye, the right eye, and the mouth are determined.

[0076] In one exemplary embodiment of this disclosure, before obtaining the target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: 2D expression texture samples and face deformer models are obtained. The face deformer model is trained using each 2D expression sample and the corresponding expression feature parameters to obtain the expression texture model.

[0077] In one exemplary embodiment of this disclosure, before obtaining the target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: The 2D facial expression map sample is segmented to obtain a segmented image, and the face position in the segmented image is obtained; The face in the segmented image is cropped according to the face position to obtain a face image; The face image and the facial feature parameters are input into the facial texture model to obtain the 2D facial texture.

[0078] In one exemplary embodiment of this disclosure, the step of segmenting the 2D expression map sample to obtain a segmented image and obtaining the face position in the segmented image includes: Obtain an anime character segmentation network and an anime face detection network; The anime character segmentation network is used to detect anime characters in the 2D facial expression texture samples and to segment the anime characters to obtain the segmented image; The face detection network described above is used to detect faces in the segmented image to obtain the face locations in the segmented image.

[0079] In one exemplary embodiment of this disclosure, the step of inputting the face image and expression feature parameters into the expression mapping model to obtain the 2D expression map includes: The face image and the facial feature parameters are input into the facial texture model to obtain a facial transformation image; The facial expression transformation image is cropped to obtain the 2D facial expression texture.

[0080] In one exemplary embodiment of this disclosure, cropping the facial expression transformation image to obtain the 2D facial expression texture includes: A key point detection model for anime faces is obtained, and the key point detection model for anime faces is used to detect the expression change image to obtain the left eye region, right eye region, and mouth region in the expression change image. The left eye region, the right eye region, and the mouth region in the expression transformation image are cropped to obtain a 2D expression texture for the left eye, a 2D expression texture for the right eye, and a 2D expression texture for the mouth.

[0081] In one exemplary embodiment of this disclosure, obtaining a target 2D emoticon image that matches the real-time emoticon feature parameters from a preset 2D emoticon image library based on the real-time emoticon feature parameters includes: Based on the real-time expression feature parameters of the left eye, a 2D expression texture of the left eye corresponding to the real-time expression feature parameters of the left eye is obtained; Based on the real-time expression feature parameters of the right eye, a 2D expression texture of the right eye corresponding to the real-time expression feature parameters of the right eye is obtained; Based on the real-time facial expression feature parameters, a 2D facial expression texture map corresponding to the real-time facial expression feature parameters is obtained.

[0082] In one exemplary embodiment of this disclosure, before attaching the acquired target 2D emoticon texture to a preset virtual avatar base image, the method further includes: Obtain a virtual avatar image and the facial expression closure parameter from the facial expression feature parameters, segment the virtual avatar image to obtain a segmented image, and obtain the face position in the segmented image; The face in the segmented image is cropped according to the face position to obtain a face image; The face image and the fully closed expression parameters are input into the expression mapping model to obtain a fully closed expression face. The fully closed expression face is then pasted back into the segmented image to obtain the virtual image base map.

[0083] In an exemplary embodiment of this disclosure, before obtaining a target 2D emoticon image matching the real-time emoticon feature parameters from a preset 2D emoticon image library, the method further includes: The real-time body posture of the anchor during the live broadcast is obtained, and the real-time body posture parameters corresponding to the real-time body posture of the anchor are determined. Based on the real-time body posture parameters, a target 2D body texture that matches the body posture parameters is obtained from a 2D body texture library.

[0084] In one exemplary embodiment of this disclosure, before obtaining a 2D body texture matching the body posture parameters from a preset 2D body texture map based on the real-time body posture parameters, the method further includes: Obtain a body texture model, input the segmented image and preset body control parameters into the body texture model to obtain a 2D body texture.

[0085] In one exemplary embodiment of this disclosure, the method further includes: The 2D body texture, which matches the real-time body posture parameters, is attached to the preset virtual avatar base map.

[0086] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0087] Furthermore, although the steps of the method in this invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0088] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above-described method is also provided.

[0089] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0090] The following reference Figure 10 To describe an electronic device 1000 according to this embodiment of the present invention. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0091] like Figure 10As shown, the electronic device 1000 is manifested in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: at least one processing unit 1010, at least one storage unit 1020, a bus 1030 connecting different system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0092] The storage unit stores program code that can be executed by the processing unit 1010, causing the processing unit 1010 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1010 can perform, as follows: Figure 1 The steps shown are as follows: S110: Acquire a real-time facial image of the anchor during the live broadcast and determine the real-time facial expression feature parameters corresponding to the real-time anchor facial image; S120: Based on the real-time facial expression feature parameters, acquire a target 2D facial expression texture that matches the real-time facial expression feature parameters from a preset 2D facial expression texture library, wherein the 2D facial expression texture library includes multiple 2D facial expression textures pre-generated by a facial expression texture model, the facial expression texture model is trained by 2D facial expression texture samples with different facial expression feature parameters, and each 2D facial expression texture is used to simulate a facial expression; S130: Attach the acquired target 2D facial expression texture to a preset virtual image base image to generate the anchor virtual image during the live broadcast in real time.

[0093] Storage unit 1020 may include readable media in the form of volatile storage units, such as random access memory (RAM) 10201 and / or cache memory 10202, and may further include read-only memory (ROM) 10203.

[0094] Storage unit 1020 may also include a program / utility 10204 having a set (at least one) program module 10205, such program module 10205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0095] Bus 1030 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0096] Electronic device 1000 can also communicate with one or more external devices 1100 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems.

[0097] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0098] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0099] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0100] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0101] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0102] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF (Radio Frequency), etc., or any suitable combination thereof.

[0103] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0104] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0105] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

Claims

1. A method for generating 2D live-streaming avatars, characterized in that, include: Acquire real-time facial images of the anchor during the live broadcast, and determine the real-time facial expression feature parameters corresponding to the real-time facial images of the anchor; Based on the real-time facial expression feature parameters, a target 2D facial expression image matching the real-time facial expression feature parameters is obtained from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained by 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. The process involves acquiring a virtual avatar image and the facial expression closure parameter from the facial expression feature parameters; segmenting the virtual avatar image to obtain a segmented image; obtaining the face position in the segmented image; cropping the face in the segmented image based on the face position to obtain a face image; inputting the face image and the facial expression closure parameter into the facial expression mapping model to obtain a facial expression closure face; pasting the facial expression closure face back into the segmented image to obtain a virtual avatar base image; and attaching the acquired target 2D facial expression texture to the preset virtual avatar base image to generate the virtual avatar of the anchor during the live broadcast in real time.

2. The 2D live streaming image generation method according to claim 1, characterized in that, The step of determining the real-time facial expression feature parameters corresponding to the real-time anchor's facial image includes: The eyes and mouth included in the real-time anchor's facial image are obtained, and the real-time expression feature parameters of the left eye, the right eye, and the mouth are determined.

3. The 2D live streaming image generation method according to claim 2, characterized in that, Before obtaining the target 2D expression map that matches the real-time expression feature parameters from the preset 2D expression map library, the method further includes: 2D expression texture samples and face deformer models are obtained. The face deformer model is trained using each 2D expression sample and the corresponding expression feature parameters to obtain the expression texture model.

4. The 2D live streaming image generation method according to claim 2, characterized in that, Before obtaining the target 2D expression map that matches the real-time expression feature parameters from the preset 2D expression map library, the method further includes: The 2D facial expression map sample is segmented to obtain a segmented image, and the face position in the segmented image is obtained; The face in the segmented image is cropped according to the face position to obtain a face image; The face image and the facial feature parameters are input into the facial texture model to obtain the 2D facial texture.

5. The 2D live streaming image generation method according to claim 4, characterized in that, The step of segmenting the 2D facial expression map sample to obtain a segmented image and obtaining the face position in the segmented image includes: Obtain an anime character segmentation network and an anime face detection network; The anime character segmentation network is used to detect anime characters in the 2D facial expression texture samples and to segment the anime characters to obtain the segmented image; The face detection network described above is used to detect faces in the segmented image to obtain the face locations in the segmented image.

6. The 2D live streaming image generation method according to claim 5, characterized in that, The step of inputting the facial image and facial feature parameters into the facial texture model to obtain the 2D facial texture includes: The face image and the facial feature parameters are input into the facial texture model to obtain a facial transformation image; The facial expression transformation image is cropped to obtain the 2D facial expression texture.

7. The 2D live streaming image generation method according to claim 6, characterized in that, The 2D expression texture is obtained by cropping the expression transformation image, including: A key point detection model for anime faces is obtained, and the key point detection model for anime faces is used to detect the expression change image to obtain the left eye region, right eye region, and mouth region in the expression change image. The left eye region, the right eye region, and the mouth region in the expression transformation image are cropped to obtain a 2D expression texture for the left eye, a 2D expression texture for the right eye, and a 2D expression texture for the mouth.

8. The 2D live streaming image generation method according to claim 7, characterized in that, Based on the real-time facial expression feature parameters, a target 2D facial expression texture matching the real-time facial expression feature parameters is obtained from a preset 2D facial expression texture library, including: Based on the real-time expression feature parameters of the left eye, a 2D expression texture of the left eye corresponding to the real-time expression feature parameters of the left eye is obtained; Based on the real-time expression feature parameters of the right eye, a 2D expression texture of the right eye corresponding to the real-time expression feature parameters of the right eye is obtained; Based on the real-time facial expression feature parameters, a 2D facial expression texture map corresponding to the real-time facial expression feature parameters is obtained.

9. The 2D live streaming image generation method according to claim 4, characterized in that, Before obtaining a target 2D expression texture that matches the real-time expression feature parameters from a preset 2D expression texture library, the method further includes: The real-time body posture of the anchor during the live broadcast is obtained, and the real-time body posture parameters corresponding to the real-time body posture of the anchor are determined. Based on the real-time body posture parameters, a target 2D body texture that matches the body posture parameters is obtained from a 2D body texture library.

10. The 2D live streaming image generation method according to claim 9, characterized in that, Before obtaining a 2D body texture matching the real-time body pose parameters from a preset 2D body texture map, the method further includes: Obtain a body texture model, input the segmented image and preset body control parameters into the body texture model to obtain a 2D body texture.

11. The 2D live streaming image generation method according to claim 10, characterized in that, The method further includes: The 2D body texture, which matches the real-time body posture parameters, is attached to the preset virtual avatar base map.

12. A 2D live streaming image generation device, characterized in that, include: The real-time facial expression feature parameter acquisition module is used to acquire real-time facial images of the anchor during the live broadcast and determine the real-time facial expression feature parameters corresponding to the real-time facial images of the anchor. The facial expression matching module is used to obtain a target 2D facial expression image that matches the real-time facial expression feature parameters from a preset 2D facial expression image library. The 2D facial expression image library includes multiple 2D facial expression images pre-generated by a facial expression image model. The facial expression image model is trained by 2D facial expression image samples with different facial expression feature parameters. Each 2D facial expression image is used to simulate a facial expression. The virtual avatar generation module is used to acquire a virtual avatar image and the facial expression closure parameter from the facial expression feature parameters; segment the virtual avatar image to obtain a segmented image; acquire the face position in the segmented image; crop the face in the segmented image according to the face position to obtain a face image; input the face image and the facial expression closure parameter into the facial expression mapping model to obtain a facial expression closure face; paste the facial expression closure face back into the segmented image to obtain a virtual avatar base image; and attach the acquired target 2D facial expression texture to the preset virtual avatar base image to generate the virtual avatar of the anchor during the live broadcast in real time.

13. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processing unit, it implements the 2D live streaming image generation method according to any one of claims 1-11.

14. An electronic device, characterized in that, include: Processing unit; as well as A storage unit for storing the executable instructions of the processing unit; The processing unit is configured to execute the 2D live streaming image generation method according to any one of claims 1-11 by executing the executable instructions.

Images