Synthesis, presentation method and system of virtual avatar

Abstract

Embodiments of the present application provide a virtual image synthesis and display method and system, computer device and computer readable storage medium, which comprise: receiving an original picture uploaded by a user through a client; performing face recognition on the original picture to obtain face key point information; synthesizing a related picture of a virtual image based on the original picture; and packaging the face key point information and the related picture of the virtual image to obtain model data of the virtual image. Thus, the user can quickly generate a virtual image by uploading an original picture, such as an animation character picture or a game character picture, to meet the personalized demands of the user, and the production cost of the virtual image is low, and the virtual image does not need to be limited to existing component materials.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system, computer device, and computer-readable storage medium for synthesizing and displaying virtual images. Background Technology

[0002] Virtual avatars are characters that do not exist in reality but can exist in works such as TV series, comics, and games; they are fictional characters within these works. Current technologies for creating virtual avatars based on images mostly involve labeling each virtual avatar component. After recognizing the image, a component from a resource library is returned for each component, and then all components are pieced together for display. Examples include Live2D and 3D Avatar virtual avatar creation technologies. Live2D generates a virtual avatar through a series of continuous images and character modeling; 3D Avatar assembles a virtual avatar based on existing component resources. However, traditional methods suffer from high production costs, are limited by existing component resources, and cannot meet users' personalized needs. Summary of the Invention

[0003] The purpose of this application is to provide a method, system, computer device, and computer-readable storage medium for synthesizing and displaying virtual images, in order to solve the following problems: the production cost of virtual images is high, and they are limited by existing component materials, which cannot meet the personalized needs of users.

[0004] One aspect of this application provides a method for synthesizing virtual avatars, including:

[0005] Receive raw images uploaded by users via the client;

[0006] Perform face recognition on the original image to obtain facial key point information;

[0007] Based on the original image, related images of the virtual avatar are synthesized;

[0008] The facial key point information and related images of the virtual avatar are packaged to obtain the model data of the virtual avatar.

[0009] Optionally, the step of packaging the facial key point information and related images of the virtual avatar to obtain the model data of the virtual avatar includes:

[0010] The images related to the virtual avatar are encrypted to obtain encrypted image data;

[0011] The facial key point information and image encryption data are packaged to obtain the model data of the virtual image.

[0012] Optionally, the related images for synthesizing the virtual avatar based on the original image include:

[0013] The original image is segmented to obtain a foreground region image;

[0014] The virtual avatar is synthesized from the foreground area image.

[0015] Optionally, before the step of performing face recognition on the original image to obtain facial key point information, the method further includes:

[0016] Check if the original image dimensions meet the requirements;

[0017] If the size of the original image does not meet the requirements, then return to the step of receiving the original image uploaded by the user through the client.

[0018] Optionally, the method further includes:

[0019] The model data of the virtual avatar is stored in a cloud server, and the storage address is sent to the client.

[0020] Optionally, the images related to the virtual avatar include images showing changes in head posture, eyes, and mouth.

[0021] One aspect of this application provides a method for displaying a virtual avatar, including:

[0022] Obtain user's facial micro-expression data;

[0023] In response to a selection operation applied to one or more original images, the model data of the virtual image corresponding to the target image is loaded.

[0024] The model data of the virtual avatar is driven by the facial micro-expression data to display a virtual avatar that matches the user's expression on the graphical user interface.

[0025] Optionally, the step of driving the model data of the virtual avatar based on the facial micro-expression data to display the virtual avatar adapted to the user's expressions on the graphical user interface includes:

[0026] The model data of the virtual avatar is analyzed to obtain facial key point information and related images of the virtual avatar;

[0027] Based on the facial micro-expression data and the facial key point information, multiple pose images of the virtual image are determined from the relevant images of the virtual image;

[0028] Based on multiple pose images of the virtual avatar, a virtual avatar adapted to the user's facial expressions is displayed on the graphical user interface.

[0029] Optionally, the images related to the virtual avatar include images showing changes in head posture, eyes, and mouth; the posture images of the virtual avatar include images showing head offset, eye opening and closing, and mouth opening and closing.

[0030] The step of displaying a virtual avatar adapted to the user's facial expressions on the graphical user interface based on multiple pose images of the virtual avatar includes:

[0031] The head offset image, eye opening and closing image, and mouth opening and closing image are synthesized to obtain the final image of the virtual image;

[0032] Based on the final image of the virtual avatar, a virtual avatar adapted to the user's facial expressions is displayed on the graphical user interface.

[0033] Optionally, acquiring facial micro-expression data includes:

[0034] Capture user images using a camera device;

[0035] The user image is converted into facial micro-expression data.

[0036] Optionally, loading the model data of the virtual image corresponding to the target image includes:

[0037] Obtain the storage address of the virtual avatar's model data in the cloud server;

[0038] Based on the storage address, the model data of the virtual image corresponding to the target image is loaded from the cloud server.

[0039] One aspect of this application provides a virtual avatar display system, including a virtual avatar synthesis module and a virtual avatar display module, wherein...

[0040] The virtual avatar synthesis module is used to receive the original image uploaded by the user through the client; perform facial recognition on the original image to obtain facial key point information; synthesize related images of the virtual avatar based on the original image; and package the facial key point information and the related images of the virtual avatar to obtain the model data of the virtual avatar.

[0041] The virtual avatar display module is used to acquire the user's facial micro-expression data; respond to the selection operation of a target image in one or more original images, load the model data of the virtual avatar corresponding to the target image; and drive the model data of the virtual avatar based on the facial micro-expression data to display the virtual avatar on the graphical user interface.

[0042] One aspect of this application provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the virtual image synthesis and display method described above.

[0043] One aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by at least one processor to implement the steps of the virtual image synthesis and display method described above when the at least one processor executes the computer program.

[0044] The virtual avatar synthesis and display method, system, device, and computer-readable storage medium provided in this application embodiment enable users to quickly generate virtual avatars by uploading original images, such as anime character images or game character images, to meet users' personalized needs. Moreover, the production cost of virtual avatars is low and is not limited by existing component materials. Attached Figure Description

[0045] Figure 1 This diagram illustrates an application environment for a method of synthesizing and displaying virtual images according to an embodiment of this application.

[0046] Figure 2 A flowchart illustrating a method for synthesizing a virtual image according to Embodiment 1 of this application is shown schematically;

[0047] Figure 3 A schematic diagram of a graphical user interface according to Embodiment 1 of this application is shown.

[0048] Figure 4 A flowchart illustrating a method for displaying a virtual avatar according to Embodiment 2 of this application is shown schematically.

[0049] Figure 5 A block diagram of a virtual avatar synthesis system according to Embodiment 3 of this application is shown schematically;

[0050] Figure 6 A block diagram schematically illustrates a virtual avatar synthesis apparatus according to Embodiment 4 of this application;

[0051] Figure 7 A block diagram schematically illustrates a virtual avatar display device according to Embodiment 5 of this application; and

[0052] Figure 8 The illustration shows a schematic diagram of the hardware architecture of a computer device suitable for implementing a method for synthesizing and displaying virtual images according to Embodiment Six of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0054] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0055] In existing technologies, most solutions for creating virtual avatars based on images involve labeling each virtual avatar component, recognizing the image, returning a component from a resource library for each component, and then piecing all the components together for display. Traditional solutions suffer from high virtual avatar production costs and are limited by existing component resources, failing to meet users' personalized needs.

[0056] In view of this, this application aims to propose a method for synthesizing and displaying virtual avatars based on user-uploaded original images. The method for synthesizing virtual avatars includes: receiving original images uploaded by users through a client; performing facial recognition on the original images to obtain facial key point information; synthesizing related images of the virtual avatar based on the original images; and packaging the facial key point information and the related images of the virtual avatar to obtain model data of the virtual avatar. The method for displaying virtual avatars includes: acquiring facial micro-expression data; responding to a selection operation on one or more original images of a target image, loading model data of the virtual avatar corresponding to the target image; and driving the model data of the virtual avatar based on the facial micro-expression data to display the virtual avatar on a graphical user interface. Thus, users can quickly generate virtual avatars by uploading original images, such as anime character images, satisfying their personalized needs. Furthermore, the production cost of virtual avatars is low and not limited by existing component materials.

[0057] This application provides several embodiments to further illustrate the synthesis and display schemes of virtual images, as detailed below.

[0058] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0059] The following is a definition of the terminology used in this application:

[0060] AI SDK: short for Artificial Intelligence Software Development Kit, is a mobile development toolkit provided by the Artificial Intelligence Department. It can convert facial micro-expression data and AI image virtual images into virtual images of the user's current expression.

[0061] AI Server: A cloud service that can generate AI-generated virtual avatars based on anime images.

[0062] Euler angles: Euler angles consist of yaw, pitch, and roll angles. Yaw angle: left and right rotation; Pitch angle: up and down rotation; Roll angle: rotational rotation.

[0063] Figure 1 The illustration shows an environmental application diagram according to an embodiment of this application. For example... Figure 1 As shown:

[0064] Computer device 10000 can connect to client 30000 via network 20000.

[0065] Computer device 10000 can provide services such as network debugging, or return the results of virtual image synthesis and display to client 30000.

[0066] Computer equipment 10000 may be located in a data center, such as a single site, or distributed across different geographical locations (e.g., multiple sites). Computer equipment 10000 may provide services via one or more networks 20000. Network 20000 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, agent devices, and / or similar devices. Network 20000 may include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, combinations thereof, etc. Network 20000 may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0067] Computer device 10000 can be implemented by one or more compute nodes. One or more compute nodes can include virtualized compute instances. Virtualized compute instances can include virtual machines, such as emulations of computer systems, operating systems, servers, etc. Compute nodes can load virtual machines based on virtual images and / or other data defining specific software used for emulation (e.g., operating systems, dedicated applications, servers). As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more compute nodes. A hypervisor can be implemented to manage the use of different virtual machines on the same compute node.

[0068] Client 30000 can be configured to access the content and services of computer device 10000. Client 30000 can include any type of electronic device, such as mobile devices, tablets, laptops, workstations, virtual reality devices, gaming devices, set-top boxes, digital streaming media devices, vehicle terminals, smart TVs, etc.

[0069] The client 30000 can output (e.g., display, render, present) the results of the synthesis and display of virtual avatars to the user.

[0070] The following will describe a network debugging solution through several embodiments. This solution can be implemented using computer equipment 10000.

[0071] Example 1

[0072] Figure 2 A flowchart illustrating a method for synthesizing a virtual avatar according to Embodiment 1 of this application is shown schematically. It includes steps S200-S206, wherein...

[0073] Step S200: Receive the original image uploaded by the user through the client;

[0074] In this embodiment, an entry point for creating a virtual avatar is provided through the client, allowing users to upload original images. As an example, such as... Figure 3 As shown, the entry point for creating a virtual avatar can include two controls: a random avatar control and an image upload control. When a user clicks the image upload control, they are redirected to the device's photo album page. On the album page, the user can select a desired image as the original image and upload it to the AI ​​server via the client. Alternatively, the user can click the random avatar control. Upon receiving this click, the AI ​​server can use a random image to synthesize the virtual avatar.

[0075] Step S202: Perform face recognition on the original image to obtain facial key point information;

[0076] In this embodiment of the invention, the AI ​​server can perform face recognition on the original image to obtain facial key point information. This facial key point information includes the coordinates of several key points on the face, such as the coordinates of the eyes and mouth. In specific implementations, existing face recognition algorithms can be used, such as the Fisherfaces algorithm and the Haar Cascade algorithm. This embodiment of the invention does not impose specific limitations on this approach.

[0077] Step S204: Synthesize related images of the virtual image based on the original image;

[0078] The images related to the virtual avatar may include changes in head posture, changes in eyes, changes in mouth, etc.

[0079] In this embodiment, the original image is input into a pre-trained virtual avatar synthesis model, which then outputs related images of the virtual avatar. As an example, the output images may include 729 images showing changes in head pose, 36 images showing changes in eyes, and 12 images showing changes in mouth, thus allowing the synthesized virtual avatar to include a wide variety of facial poses, making it more vivid and lifelike.

[0080] Step S206: Package the facial key point information and related images of the virtual image to obtain the model data of the virtual image.

[0081] In this embodiment, after synthesizing the relevant images of the virtual avatar, the model data of the virtual avatar is obtained by packaging the facial key point information and the relevant images of the virtual avatar, so that the client can obtain the model data of the virtual avatar to display the virtual avatar.

[0082] The following are several optional embodiments for optimizing the synthesis and display method of the virtual image, as detailed below:

[0083] In a preferred embodiment of this application, step S206 may include the following steps: encrypting the relevant images of the virtual avatar to obtain encrypted image data; and packaging the facial key point information and the encrypted image data to obtain the model data of the virtual avatar.

[0084] In this embodiment, encrypted image data is obtained by encrypting the images related to the virtual avatar. For example, the images can be converted into binary data to prevent them from being visible to the user. In another example, the images can also be encrypted by converting them into TXT text format. Furthermore, other methods can be used to encrypt the images related to the virtual avatar, and this embodiment does not limit this approach.

[0085] In a preferred embodiment of this application, step S204 may include the following steps: performing foreground segmentation on the original image to obtain a foreground region image; and synthesizing a related image of the virtual character based on the foreground region image.

[0086] In this embodiment, a foreground region map is obtained by performing foreground segmentation on the original image. Then, the foreground region map is input into a pre-trained virtual character synthesis model, and the relevant images of the virtual character are output.

[0087] In a preferred embodiment of this application, before step S202, the following steps may be included: detecting whether the size of the original image meets the requirements; if the size of the original image does not meet the requirements, then returning to the step of receiving the original image uploaded by the user through the client.

[0088] In this embodiment, the size of the original image is checked to see if it meets the requirements. If the size of the original image does not meet the requirements, the process returns to step S200. If the size of the original image meets the requirements, the process proceeds to step S202.

[0089] In one example, facial feature recognition can also be performed on the original image. If no facial features can be recognized in the original image, the original image is determined to be unacceptable, and the process returns to the step of receiving the original image uploaded by the user through the client.

[0090] In a preferred embodiment of this application, the method further includes:

[0091] The model data of the virtual avatar is stored in a cloud server, and the storage address is sent to the client.

[0092] In this embodiment, the cloud server is used to store some user data. After generating the model data of the virtual avatar, the model data of the virtual avatar can be stored in the cloud server and the storage address can be sent to the client so that the client can load the model data of the virtual avatar from the cloud server according to the storage address.

[0093] Example 2

[0094] Figure 4 A flowchart illustrating a method for displaying a virtual avatar according to Embodiment 2 of this application is shown schematically. It includes steps S400-S404, wherein...

[0095] Step S400: Obtain the user's facial micro-expression data;

[0096] Facial micro-expression data is used to describe a person's facial expressions. Facial expressions are a form of body language; changes in facial muscles can generate a wide variety of rich facial expressions that can express an individual's mindset and emotions. Using facial micro-expression data to drive virtual avatars can make the displayed virtual avatars more lifelike.

[0097] Step S402: In response to a selection operation applied to one or more original images, load the model data of the virtual image corresponding to the target image;

[0098] In this embodiment, the original image is a user-uploaded image. A virtual avatar synthesis scheme can be used to synthesize related images of the virtual avatar from the original image, and then package them to obtain the virtual avatar's model data. In this embodiment, the user can select a target image from one or more original images. After detecting the user's selection, in response to the selection, the model data of the virtual avatar corresponding to the target image is loaded.

[0099] Step S404: Based on the facial micro-expression data, drive the model data of the virtual image to display a virtual image that matches the user's expression on the graphical user interface.

[0100] In this embodiment, model data of virtual avatars is driven by facial micro-expression data to display virtual avatars that match the user's expressions on the graphical user interface, making the displayed virtual avatars more vivid.

[0101] The following are several optional embodiments for optimizing the synthesis and display method of the virtual image, as detailed below:

[0102] In a preferred embodiment of this application, step S404 may include the following steps: parsing the model data of the virtual avatar to obtain facial key point information and related images of the virtual avatar; determining multiple pose images of the virtual avatar from the related images of the virtual avatar based on the facial micro-expression data and the facial key point information; and displaying a virtual avatar adapted to the user's expression on the graphical user interface based on the multiple pose images of the virtual avatar.

[0103] In this embodiment, facial key point information and related images of the virtual image can be obtained by parsing the model data of the virtual image. Then, based on the facial micro-expression data and facial key point information, multiple pose images of the virtual image are determined from the related images of the virtual image. Based on the multiple pose images of the virtual image, the virtual image adapted to the user's expression is displayed on the graphical user interface.

[0104] In a preferred embodiment of this application, the relevant images of the virtual avatar include a relevant change map of the head posture, a relevant change map of the eyes, and a relevant change map of the mouth; the posture images of the virtual avatar include a head offset map, an eye opening and closing map, and a mouth opening and closing map.

[0105] In one example, the images related to the virtual avatar may include 729 images showing changes in head posture, 36 images showing changes in eyes, and 12 images showing changes in mouth. Based on facial micro-expression data and the facial key point information, the required head offset image, eye opening / closing image, and mouth opening / closing image can be determined. In a specific implementation, the corresponding head offset image can be found based on the Euler angle coefficients of the micro-expression, the corresponding eye opening / closing image can be found based on the eye coefficients of the micro-expression, and the corresponding mouth opening / closing image can be found based on the mouth coefficients of the micro-expression.

[0106] In a preferred embodiment of this application, the step of displaying a virtual image adapted to the user's facial expressions on the graphical user interface based on multiple pose images of the virtual image includes: synthesizing the head offset image, eye opening and closing image, and mouth opening and closing image to obtain a final image of the virtual image; and displaying a virtual image adapted to the user's facial expressions on the graphical user interface based on the final image of the virtual image.

[0107] In this embodiment, the final image of the virtual avatar is obtained by synthesizing the head offset image, the eye opening and closing image, and the mouth opening and closing image. Then, the virtual avatar is driven according to the final image of the virtual avatar to display a virtual avatar that matches the user's expression on the graphical user interface.

[0108] In a preferred embodiment of this application, step S400 may include the following steps: acquiring a user image through a camera device; and converting the user image into facial micro-expression data.

[0109] In this embodiment, the user's image is captured by calling the camera device of the terminal device, and then the face recognition algorithm converts the user image into facial micro-expression data. For example, SenseTime's face recognition function can be used to convert the user image into facial micro-expression data. In specific implementation, the face recognition algorithm can be set as needed, and this embodiment does not limit it.

[0110] In a preferred embodiment of this application, step S402 includes the following steps: obtaining the storage address of the model data of the virtual image in the cloud server; and loading the model data of the virtual image corresponding to the target image from the cloud server according to the storage address.

[0111] In this embodiment, the storage address of the virtual avatar's model data in the cloud server is obtained, and then the model data of the virtual avatar corresponding to the target image is loaded from the cloud server based on the storage address.

[0112] Example 3

[0113] Figure 5 The diagram illustrates a block diagram of a virtual avatar synthesis system according to Embodiment 3 of this application. The system apparatus for the virtual avatar can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in the embodiments of this application.

[0114] like Figure 5 As shown, the virtual avatar synthesis system 500 may include the following modules:

[0115] The virtual avatar synthesis module 510 is used to receive the original image uploaded by the user through the client; perform face recognition on the original image to obtain facial key point information; synthesize related images of the virtual avatar based on the original image; and package the facial key point information and the related images of the virtual avatar to obtain the model data of the virtual avatar.

[0116] The virtual avatar display module 520 is used to acquire the user's facial micro-expression data; respond to the selection operation of a target image in one or more original images, load the model data of the virtual avatar corresponding to the target image; and drive the model data of the virtual avatar based on the facial micro-expression data to display the virtual avatar on the graphical user interface.

[0117] Example 4

[0118] Figure 6 The diagram schematically illustrates a block diagram of a virtual avatar synthesis apparatus according to Embodiment 4 of this application. This virtual avatar synthesis apparatus can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in the embodiments of this application.

[0119] like Figure 6 As shown, the virtual avatar synthesis device 600 may include the following modules:

[0120] The original image receiving module 610 is used to receive original images uploaded by users through the client;

[0121] The face recognition module 620 is used to perform face recognition on the original image to obtain facial key point information;

[0122] The virtual avatar synthesis module 630 is used to synthesize related images of a virtual avatar based on the original image;

[0123] The model data packaging module 640 is used to package the facial key point information and related images of the virtual image to obtain the model data of the virtual image.

[0124] In a preferred embodiment of this application, the model data packaging module 640 includes:

[0125] The image encryption submodule is used to encrypt the relevant images of the virtual image to obtain encrypted image data;

[0126] The model data packaging submodule is used to package the facial key point information and image encryption data to obtain the model data of the virtual image.

[0127] In a preferred embodiment of this application, the virtual avatar synthesis module 630 includes:

[0128] The foreground segmentation submodule is used to segment the original image to obtain a foreground region image;

[0129] The virtual avatar synthesis submodule is used to synthesize relevant images of the virtual avatar based on the foreground area image.

[0130] In a preferred embodiment of this application, it further includes:

[0131] The size detection module is used to detect whether the size of the original image meets the requirements.

[0132] The re-acquisition image module is used to return to the step of receiving the original image uploaded by the user through the client if the size of the original image does not meet the requirements.

[0133] In a preferred embodiment of this application, the method further includes:

[0134] The model data storage module is used to store the model data of the virtual image in a cloud server and send the storage address to the client.

[0135] In a preferred embodiment of this application, the relevant images of the virtual avatar include relevant change images of the head posture, relevant change images of the eyes, and relevant change images of the mouth.

[0136] Example 5

[0137] Figure 7 The diagram schematically illustrates a block diagram of a virtual avatar display device according to Embodiment 5 of this application. The virtual avatar synthesis device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in the embodiments of this application.

[0138] like Figure 7 As shown, the virtual avatar display device 700 may include the following modules:

[0139] The micro-expression acquisition module 710 is used to acquire the user's facial micro-expression data;

[0140] The model data loading module 720 is used to load the model data of the virtual image corresponding to the target image in response to a selection operation applied to one or more original images.

[0141] The virtual avatar display module 730 is used to drive the model data of the virtual avatar based on the facial micro-expression data, so as to display a virtual avatar that matches the user's expression on the graphical user interface.

[0142] In a preferred embodiment of this application, the virtual avatar display module 730 includes:

[0143] The model data parsing submodule is used to parse the model data of the virtual image to obtain facial key point information and related images of the virtual image;

[0144] The pose image determination submodule is used to determine multiple pose images of the virtual image from related images of the virtual image based on the facial micro-expression data and the facial key point information;

[0145] The virtual avatar display submodule is used to display a virtual avatar that matches the user's facial expressions on the graphical user interface based on multiple pose images of the virtual avatar.

[0146] In a preferred embodiment of this application, the relevant images of the virtual avatar include a relevant change map of the head posture, a relevant change map of the eyes, and a relevant change map of the mouth; the posture images of the virtual avatar include a head offset map, an eye opening and closing map, and a mouth opening and closing map.

[0147] In a preferred embodiment of this application, the virtual avatar display submodule includes:

[0148] The image synthesis unit is used to synthesize the head offset image, eye opening and closing image, and mouth opening and closing image to obtain the final image of the virtual image;

[0149] The virtual avatar display unit is used to display a virtual avatar that matches the user's facial expressions on the graphical user interface based on the final image of the virtual avatar.

[0150] In a preferred embodiment of this application, the micro-expression acquisition module 710 includes:

[0151] The image acquisition submodule is used to acquire user images via a camera device;

[0152] The image conversion submodule is used to convert the user image into facial micro-expression data.

[0153] In a preferred embodiment of this application, the model data loading module 720 includes:

[0154] The storage address acquisition submodule is used to acquire the storage address of the model data of the virtual image in the cloud server;

[0155] The model data loading submodule is used to load the model data of the virtual image corresponding to the target image from the cloud server according to the storage address.

[0156] Example 6

[0157] Figure 8 This illustration schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a method for synthesizing and displaying virtual avatars according to Embodiment Six of this application. In this embodiment, the computer device 10000 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. For example, it may be a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including independent FEN servers, or server clusters composed of multiple servers), etc. Figure 8 As shown, the computer device 10000 includes, but is not limited to, at least the following: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked to each other via a system bus. Wherein:

[0158] The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for virtual character synthesis and display methods. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0159] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0160] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0161] It should be pointed out that, Figure 8 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0162] In this embodiment, the method for synthesizing and displaying the virtual image stored in the memory 10010 can be further divided into one or more program modules and executed by one or more processors (processor 10020 in this embodiment) to complete the embodiment of this application.

[0163] Example 7

[0164] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the method for synthesizing and displaying virtual images in the embodiments.

[0165] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the computer-readable storage medium can also include both internal storage units and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code for the synthesis and display methods of virtual images in this embodiment. Furthermore, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0166] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0167] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for synthesizing virtual images, characterized in that, include: Receive raw images uploaded by users via the client; Perform face recognition on the original image to obtain facial key point information; Based on the original image, related images of the virtual avatar are synthesized; The facial key point information and related images of the virtual avatar are packaged to obtain the model data of the virtual avatar; The images related to the virtual avatar include images showing changes in head posture, eyes, and mouth. The images related to the synthesis of the virtual avatar based on the original image include: The original image is segmented to obtain a foreground region image; The virtual avatar is synthesized from the foreground area image; The method further includes: Acquire facial micro-expression data; The model data of the virtual avatar is driven by the acquired facial micro-expression data to display a virtual avatar that is adapted to the micro-expressions.

2. The method for synthesizing virtual images according to claim 1, characterized in that, The process of packaging the facial key point information and related images of the virtual avatar to obtain the model data of the virtual avatar includes: The images related to the virtual avatar are encrypted to obtain encrypted image data; The facial key point information and image encryption data are packaged to obtain the model data of the virtual image.

3. The method for synthesizing virtual images according to claim 1, characterized in that, Before the step of performing face recognition on the original image to obtain facial key point information, the method further includes: Check if the original image dimensions meet the requirements; If the size of the original image does not meet the requirements, then return to the step of receiving the original image uploaded by the user through the client.

4. The method for synthesizing virtual images according to claim 1, characterized in that, The method further includes: The model data of the virtual avatar is stored in a cloud server, and the storage address is sent to the client.

5. The method for synthesizing virtual images according to claim 1, characterized in that, The model data of the virtual avatar is driven by the acquired facial micro-expression data to display a virtual avatar adapted to the micro-expressions, including: In response to a selection operation applied to one or more original images, the model data of the virtual image corresponding to the target image is loaded. The model data of the virtual image corresponding to the target image is driven by the facial micro-expression data to display the virtual image adapted to the micro-expression on the graphical user interface.

6. The method for synthesizing virtual images according to claim 5, characterized in that, The process of driving the model data of the virtual avatar corresponding to the target image based on the facial micro-expression data to display the virtual avatar adapted to the micro-expression on the graphical user interface includes: The model data of the virtual avatar is analyzed to obtain facial key point information and related images of the virtual avatar; Based on the facial micro-expression data and the facial key point information, multiple pose images of the virtual image are determined from the relevant images of the virtual image; Based on multiple pose images of the virtual avatar, a virtual avatar adapted to the micro-expressions is displayed on the graphical user interface.

7. The method for synthesizing virtual images according to claim 6, characterized in that, The images related to the virtual avatar include images showing changes in head posture, eyes, and mouth; the images showing the virtual avatar's posture include images showing head offset, eyes opening and closing, and mouth opening and closing. The step of displaying a virtual avatar adapted to the micro-expressions on the graphical user interface based on multiple pose images of the virtual avatar includes: The head offset image, eye opening and closing image, and mouth opening and closing image are synthesized to obtain the final image of the virtual image; Based on the final image of the virtual avatar, a virtual avatar adapted to the micro-expressions is displayed on the graphical user interface.

8. The method for synthesizing virtual images according to claim 5, characterized in that, The acquisition of facial micro-expression data includes: Capture user images using a camera device; The user image is converted into facial micro-expression data.

9. The method for synthesizing virtual images according to claim 5, characterized in that, The loading of the model data of the virtual image corresponding to the target image includes: Obtain the storage address of the virtual avatar's model data in the cloud server; Based on the storage address, the model data of the virtual image corresponding to the target image is loaded from the cloud server.

10. A system for synthesizing virtual avatars, characterized in that, It includes a virtual avatar creation module and a virtual avatar display module, among which, The virtual avatar synthesis module is used to receive original images uploaded by users through a client; perform facial recognition on the original images to obtain facial key point information; synthesize related images of the virtual avatar based on the original images; and package the facial key point information and the related images of the virtual avatar to obtain model data of the virtual avatar. The related images of the virtual avatar include related change maps of head posture, eyes, and mouth. The step of synthesizing related images of the virtual avatar based on the original images includes: performing foreground segmentation on the original images to obtain foreground region images; and synthesizing related images of the virtual avatar based on the foreground region images. The virtual avatar display module is used to acquire facial micro-expression data; based on the acquired facial micro-expression data, it drives the model data of the virtual avatar to display a virtual avatar adapted to the micro-expressions.

11. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for synthesizing virtual images according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the method for synthesizing a virtual image as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method for synthesizing virtual images as described in any one of claims 1 to 9.

Images