Virtual image generation method and related device

By obtaining user facial images and 3D face models, combining stylized parameters to generate virtual images, and allowing users to adjust stylized parameters in real time, the problem of personalized generation and stylized adjustment of virtual images in the existing technology is solved, and the user experience is improved .

WO2024245248A1PCT designated stage expired Publication Date: 2024-12-05HUAWEI TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/095838
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-02
Filing Date
2024-05-28
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The prior art is difficult to generate personalized virtual images, and it is impossible to adjust the stylization degree of virtual images in real time according to user preferences, which affects the user experience.

Method used

By obtaining the user's facial image and 3D face model, combining stylized parameters, virtual images are generated and displayed, and users can adjust stylized parameters in real time to realize the virtual images from closer to real faces to more stylized. adjust.

Benefits of technology

It realizes the generation of personalized virtual images based on user preferences, improves the user experience, and enables users to customize virtual images based on stylized parameters through real-time preview function to meet the needs of different users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024095838_05122024_PF_FP_ABST
    Figure CN2024095838_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed in the embodiments of the present application are a virtual image generation method and a related device. An electronic device may acquire a first picture comprising a human face; then, the electronic device acquires a first 3D human face model and a first stylization parameter which is selected by a user and is related to the first picture; and next, the electronic device may generate and display a first virtual image according to the first picture, the first 3D human face model and the first stylization parameter, wherein the first stylization parameter indicates the extent of the effect of the first picture and the first 3D human face model on the first virtual image. By using the embodiments of the present application, a user can adjust a generated virtual image between "closer to the real face" and "more stylized" by means of adjusting stylization parameters, so as to meet the preferences of different users, thereby improving the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Virtual image generation method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 2, 2023, with application number 202310651361.1, and priority to the Chinese patent application entitled “Virtual Image Generation Method and Related Equipment”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a method for generating a virtual image and related equipment. Background Art

[0003] The concept of the metaverse has been gaining popularity in recent years, and related products have exploded onto the market. "Digital humans" and "avatars" have become hot topics in this field. Avatar generation has been marketed as a key feature of these products. Avatars can represent a portion of a user's personality, and many users seek personalized avatars. How to generate personalized avatars based on user preferences for electronic devices like mobile phones and computers is a current and future research topic.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a virtual image generation method and related equipment, which can generate personalized virtual images according to user preferences and improve user experience.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a virtual image, which is applied to an electronic device, and the method includes: the electronic device obtains a first image including a human face; the electronic device obtains a first 3D face model; the electronic device obtains a first stylization parameter related to the first image selected by a user; the electronic device generates and displays a first virtual image based on the first image, the first 3D face model, and the first stylization parameter; the first stylization parameter indicates the proportion of influence of the first image and the first 3D face model on the first virtual image.

[0007] In the embodiments of the present application, based on stylization parameters input by the user, the facial features of the person are fused with a stylized 3D face model to generate an avatar, which can then be displayed on a user interface for easy browsing. Since different stylization parameters generated by the user result in different avatars, the user can adjust the stylization parameters to adjust the generated avatar between "closer to a real face" and "more stylized," thereby satisfying different user preferences and improving the user experience.

[0008] In some embodiments, before the electronic device obtains the first stylization parameters related to the first image selected by the user, it also includes: the electronic device generates and displays an initial virtual image based on the first image, the first 3D face model, and the default stylization parameters; the default stylization parameters indicate the proportion of influence of the first image and the first 3D face model on the initial virtual image.

[0009] In the embodiments of the present application, developers can preset a set of stylization parameters (i.e., default stylization parameters). The electronic device can first fuse the facial features of the person with the stylized 3D face model based on the preset stylization parameters to generate an initial avatar, and can display this initial avatar in the user interface. The user can preview the initial avatar and then adjust the stylization parameters until a avatar that meets the user's preferences is generated, thereby improving the user experience.

[0010] In some embodiments, the electronic device generates and displays an initial virtual image based on a first image, a first 3D face model, and default stylization parameters, including: the electronic device generates a real face model based on the first image and the first 3D face model; the electronic device generates a first face model based on the real face model, the first 3D face model, and the default stylization parameters; the electronic device generates the initial virtual image based on the first face model and displays a first user interface, wherein the initial virtual image is displayed in the first user interface.

[0011] In the embodiment of the present application, the real face model is actually obtained by migrating the facial features of a 2D human face to the topological structure of a stylized 3D face model. The facial features of the real face model are determined by the facial features of the human face in the portrait photo, and it is impossible to make the real face model contain more or fewer stylized features (such as more cartoon-like or more comic-like). Therefore, the real face model and the stylized 3D face model are fused based on the default stylization parameters, so that the 3D virtual image generated based on the fusion result can be adjusted between "closer to the real face" and "more stylized", meeting the preferences of different users and improving the user experience.

[0012] In some embodiments, an electronic device obtains a first image including a face, including: the electronic device displays a second user interface, the second user interface includes a first preview area and a first control, the first preview area displays an image including a face from a camera of the electronic device; the electronic device receives an operation acting on the first control, and saves the image displayed in the first preview area as the first image.

[0013] In an embodiment of the present application, an electronic device may display a user interface, which may include a display area and a first control. The display area may display image information of a person captured by a camera of the electronic device, and the first control may trigger the electronic device to save the image of the person currently captured by the camera. Upon receiving an operation on the first control, the electronic device may save the image of the person currently captured by the camera as the first image, thereby obtaining facial feature information of the person, facilitating the subsequent generation of a personalized virtual avatar for the user and enhancing the user experience.

[0014] In some embodiments, an electronic device obtains a first image including a face, including: the electronic device displays a third user interface; the electronic device receives an operation acting on the third user interface and displays a fourth user interface, the fourth user interface displays one or more images, the one or more images including a thumbnail of the first image; the electronic device receives an operation acting on the thumbnail of the first image and obtains the first image.

[0015] In an embodiment of the present application, a user can select an image from the gallery of the electronic device as the first image to obtain facial feature information of the character, thereby facilitating the subsequent generation of a personalized virtual image of the user and improving the user experience.

[0016] In some embodiments, before the electronic device obtains the first 3D face model, the method also includes: the electronic device displays a fifth user interface, the fifth user interface displays indication information of one or more 3D face models, the one or more 3D face models including the first 3D face model; the electronic device receives the indication information acting on the first 3D face model.

[0017] In an embodiment of the present application, multiple 3D face models can be stored in advance in the electronic device, and different 3D face models have different styles. For example, the multiple 3D face models may include comic face models, cartoon face models, exaggerated face models, etc. The user can select one from the multiple 3D face models as the first 3D face model according to his or her preferences, so as to facilitate the subsequent generation of a virtual image that meets the user's preferences and enhance the user experience.

[0018] In some embodiments, the electronic device generates and displays a first virtual image based on a first image, a first 3D face model, and a first stylization parameter, including: the electronic device generates a second face model based on a real face model, the first 3D face model, and the first stylization parameter; the electronic device generates a first virtual image based on the second face model, and displays the first virtual image in a first user interface.

[0019] In an embodiment of the present application, the electronic device can, based on the user-defined degree of stylization, re-integrate the real face model with the stylized 3D face model to obtain a new virtual image, and can display the new virtual image on the electronic device, achieving a real-time preview effect for the user, thereby improving the user experience.

[0020] In some embodiments, the electronic device obtains a first stylization parameter related to a first image selected by a user, including: the electronic device receives a first operation acting on a first user interface, the first operation being used to adjust the stylization parameter; and in response to the first operation, determines the first stylization parameter selected by the user.

[0021] In an embodiment of the present application, the user can freely adjust the degree of stylization (i.e., stylization parameters) on the user interface of the electronic device, and then the electronic device can, based on the user-defined degree of stylization, re-integrate the real face model with the stylized 3D face model to obtain a new virtual image, and the new virtual image can be displayed on the electronic device, achieving the effect of real-time preview for the user, thereby improving the user experience.

[0022] In some embodiments, the first user interface includes a slider control, and the first operation is an operation of sliding the slider control to adjust the stylization parameter.

[0023] In an embodiment of the present application, a user can change the value of a stylized parameter by sliding a slider on the user interface of an electronic device, thereby obtaining a virtual image that meets the user's preferences and improving the user experience.

[0024] In some embodiments, the method also includes: the electronic device receives an adjustment operation acting on the first user interface for the first virtual image, adjusts the facial features of the first virtual image, and obtains an updated first virtual image; the electronic device displays the updated first virtual image on the first user interface.

[0025] In the application embodiment, the user can also perform secondary editing on the generated virtual image to obtain a more personalized virtual image that meets the user's preferences, thereby improving the user experience.

[0026] In some embodiments, the electronic device generates a first virtual image based on a second face model, including: the electronic device renders the second face model to obtain the first virtual image.

[0027] In an embodiment of the present application, the generated face model can be rendered and the rendering result can be presented on the user interface of the electronic device, so that the user can browse the virtual image on the electronic device, thereby improving the user experience.

[0028] In some embodiments, the electronic device generates a first virtual image based on the second face model, including: the electronic device obtains a default 3D material, renders the second face model based on the default 3D material, and obtains the first virtual image, wherein the default 3D material includes one or more of the limb features and clothing features of the 3D character model.

[0029] In an embodiment of the present application, default 3D materials can be obtained first, such as the limb features, clothing features and other information of the 3D character model, and then the face model can be rendered based on the default 3D materials, thereby obtaining a more complete virtual image and improving the user experience.

[0030] In a second aspect, an embodiment of the present application provides an electronic device, which includes a first processing unit, and the first processing unit is used to: obtain a first image including a face; obtain a first 3D face model; obtain a first stylization parameter related to the first image selected by a user; generate and display a first virtual image based on the first image, the first 3D face model, and the first stylization parameter; the first stylization parameter indicates the proportion of influence of the first image and the first 3D face model on the first virtual image.

[0031] In a third aspect, an electronic device is provided, comprising: a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method as in the first aspect or any one of the embodiments of the first aspect.

[0032] In a fourth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, cause the electronic device to execute a method as in the first aspect or any one of the embodiments of the first aspect.

[0033] In a fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute the method of the first aspect or any one of the embodiments of the first aspect.

[0034] In a sixth aspect, a chip system is provided, comprising at least one processor for implementing the method of the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0036] FIG1a is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

[0037] FIG1 b is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0038] FIG1c is a schematic diagram of the software structure of another electronic device provided in an embodiment of the present application.

[0039] Figures 2a-2i are a set of user interfaces displayed on an electronic device provided in an embodiment of the present application.

[0040] FIG3 is a flow chart of a method for generating a virtual image provided in an embodiment of the present application.

[0041] 4a-4f are schematic diagrams of a set of virtual image generation method processes provided in an embodiment of the present application.

[0042] FIG5 is a flow chart of another method for generating a virtual image provided in an embodiment of the present application.

[0043] FIG6 is a schematic diagram of an application scenario of a virtual image provided in an embodiment of the present application.

[0044] FIG7 is a flow chart of another method for generating a virtual image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0046] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0047] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] The present invention provides a method for generating an avatar, which can be applied to electronic devices, including but not limited to smartphones, tablet personal computers (Tablet PCs), handheld computers, wearable electronic devices, personal computers (PCs), and desktop computers.

[0049] This virtual image generation method can be applied to scenarios such as games, virtual social networking, and 3D model design.

[0050] First, several basic concepts involved in this application are introduced as follows:

[0051] Stylization: Stylization refers to caricature, cartoon or exaggeration.

[0052] Stylized base model: A 3D face model with a completely comic, cartoon, or exaggerated effect, usually created by a 3D model designer using a 3D model creation tool.

[0053] In the virtual image generation method provided in the present application, the electronic device can transfer the facial features of the face in the portrait photo to the topological structure of the stylized 3D face model to generate an initial 3D face model; then, the initial 3D face model and the stylized base model are fused to obtain the final 3D face model; finally, the final 3D face model is rendered as a 3D virtual image.

[0054] In this application, the initial 3D face model is actually obtained by migrating the facial features of a 2D face to the topological structure of a stylized 3D face model. The facial features of the initial 3D face model are determined by the facial features of the face in the portrait photo, and it is impossible to make the initial 3D face model contain more or fewer stylized features (such as more cartoon-like, more comic-like, etc.). Therefore, the initial 3D face model is fused with the stylized 3D face model, so that the 3D virtual image generated based on the fusion result can be adjusted between "closer to the real face" and "more stylized", meeting the preferences of different users and improving the user experience.

[0055] The electronic device receives a user operation to adjust the stylization degree of the 3D virtual image, and then re-merges the initial 3D face model and the stylized base model with the new stylization degree according to the user operation to obtain a new 3D face model, and then renders the new 3D face model into a new 3D virtual image.

[0056] In this application, users can freely adjust the degree of stylization on the user interface of the electronic device, and then the electronic device can fuse the initial 3D face model and the vertex data of the stylized base model based on the user-defined degree of stylization to obtain a new 3D virtual image, and the new 3D virtual image can be displayed on the electronic device, achieving the effect of real-time preview for the user, thereby improving the user experience.

[0057] Before introducing the method provided by the embodiment of the present application in detail, the electronic device provided by the embodiment of the present application is first introduced.

[0058] FIG. 1 a shows a schematic structural diagram of an electronic device 100 .

[0059] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a 3D camera module 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0060] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), a modem processor, an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, etc. Among them, different processing units can be independent devices or integrated into one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110.

[0062] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0063] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly access the memory. This avoids repeated accesses, reduces the waiting time of processor 110, and thus improves the efficiency of electronic device 100.

[0064] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0065] Among them, the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the 3D camera module 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the 3D camera module 193 communicate via the CSI interface to implement the camera function (also known as the photo function) of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0066] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative description and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0067] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and render graphics, such as rendering virtual characters displayed on display screen 194. Processor 110 may include one or more GPUs that execute instructions to generate or modify display information.

[0068] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0069] The electronic device 100 can realize the camera function through a 3D camera module 193, an ISP, a video codec, a GPU, a display screen 194, an application processor AP, a neural network processor NPU, etc.

[0070] The 3D camera module 193 can be used to capture color image data of the subject. The ISP can be used to process the color image data captured by the 3D camera module 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be provided in the 3D camera module 193.

[0071] In some embodiments, the 3D camera module 193 may be composed of a color camera module.

[0072] In some embodiments, the photosensitive element of the camera of the color camera module can be a charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV.

[0073] In some embodiments, the electronic device 100 may include one or N 3D camera modules 193, where N is a positive integer greater than 1. Specifically, the electronic device 100 may include one front-facing 3D camera module 193 and one rear-facing 3D camera module 193. The front-facing 3D camera module 193 may generally be used to capture color image data of the user facing the display screen 194, while the rear-facing 3D camera module 193 may be used to capture color image data of the subject (e.g., a person, scenery, etc.) that the user is facing.

[0074] In some embodiments, the CPU or GPU or NPU in the processor 110 can process the color image data collected by the 3D camera module 193. In some embodiments, the NPU can identify the color image data collected by the 3D camera module 193 (specifically the color camera module) through a neural network algorithm based on the skeleton point recognition technology, such as a convolutional neural network algorithm (CNN), to determine the skeleton points of the photographed person. The CPU or GPU can also run a neural network algorithm to determine the skeleton points of the photographed person based on the color image data. In some embodiments, the CPU or GPU or NPU can also be used to migrate the facial features of the face in the portrait photo to the topological structure of the stylized 3D face model to generate an initial 3D face model; then, the initial 3D face model and the stylized base model are fused to obtain the final 3D face model; finally, the final 3D face model is rendered as a 3D virtual image. This will be described in detail later and will not be repeated here.

[0075] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0076] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, data such as music, photos, and videos can be stored on the external memory card.

[0077] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can run the above instructions stored in the internal memory 121, so that the electronic device 100 executes the virtual image generation method provided in some embodiments of the present application, as well as various functional applications and data processing. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data (such as photos, etc.) created during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0078] The electronic device 100 exemplarily shown in FIG1a can display the various user interfaces described in the following embodiments through the display screen 194. The electronic device 100 can detect touch operations in various user interfaces through the touch sensor 180K, such as click operations in various user interfaces (such as touch operations on icons, double-click operations), and for example, upward or downward sliding operations in various user interfaces, or operations of performing circle gestures, etc. In some embodiments, the electronic device 100 can detect motion gestures performed by the user holding the electronic device 100, such as shaking the electronic device, through a gyroscope sensor 180B, an acceleration sensor 180E, etc. In some embodiments, the electronic device 100 can detect non-touch gesture operations through a 3D camera module 193 (such as a 3D camera).

[0079] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0080] FIG1 b is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0081] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0082] The application layer can include a series of application packages.

[0083] As shown in FIG1b , the application package may include smart photo taking application, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0084] Among them, the smart photo application can be used to generate the user's virtual image. The smart photo application can receive user interactive operations on the graphical interface, such as the user uploading a portrait photo, the user adjusting the stylization parameters, the user manually pinching the face, etc. The smart photo application can be a system application or a third-party application. The functions of the smart photo application have been clearly described here and in the subsequent method embodiments. Its name does not constitute a limitation of this application. For example, the smart photo application can also be called "3D face pinching", "3D character construction", etc., which are not limited here.

[0085] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0086] As shown in FIG1b , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0087] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0088] Content providers are used to store and retrieve data and make it accessible to applications. The data may include images, videos, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0089] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0090] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0091] The resource manager provides various resources for applications, such as localized images, strings, icons, layout files, video files, and so on.

[0092] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0093] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0094] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0095] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0096] The system library can include multiple functional modules, such as surface manager, media library, 3D engine module (such as OpenGL ES), 2D graphics engine (such as SGL), model generation module, etc.

[0097] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0098] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, G.264, MP3, AAC, AMR, JPG, PNG, etc.

[0099] 3D engine module, used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing.

[0100] A 2D graphics engine is a drawing engine for 2D drawings.

[0101] The model generation module is used to receive user-input portrait photos and stylized parameters and output the stylized 3D face pinching result.

[0102] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0103] In some embodiments, the model generation module and the 3D engine module in the present application will be described in detail in combination with FIG. 1c .

[0104] The model generation module (Avatar SDK) can be used to receive user-input portrait images and stylized parameters, and output a stylized face model. The Avatar SDK can include three functional units, namely a real 3D face reconstruction module, a vertex data weighted fusion module, and a stylized base model topology migration module. Among them, the real 3D face reconstruction module can take a portrait image as input, and output a real face model through a 3D face reconstruction algorithm. Based on the real face model, the vertex data of the real face model can be obtained, and the vertex data of the real face model can be cached, so that the user can achieve the purpose of real-time adjustment and preview when adjusting the stylized parameters. The vertex data weighted fusion module can take the vertex data of the real face model, the vertex data of the target stylized base model, and the stylized parameters set by the user / developer as input, and output the stylized 3D face model through the weighted fusion of vertex data. The stylized base model topology migration module relies on the topological structure of the target stylized base model. Through the deformation algorithm, the topological structure of the real face model generated by the real 3D face reconstruction module is made consistent with the topological structure of the target stylized base model. That is, the vertex data of the real face model corresponds one-to-one with the vertex data of the target stylized base model.

[0105] The 3D engine module is used to parse the results output by the Avatar SDK, and in response to the update of the manual adjustment results of the user, perform real-time rendering and frame generation, and feed back the rendering results (i.e., the virtual image) to the smart photo application for the user to preview the virtual image effect. In some embodiments, the 3D engine module can perform real-time rendering and frame generation of the vertex data of the stylized 3D face model output by the Avatar SDK, thereby enabling a virtual image preview in the smart photo application interface. In some embodiments, the 3D engine module can obtain manual adjustment information input by the user from the smart photo application, which manual adjustment information may include but is not limited to bone coefficients, material index values, etc., and perform real-time rendering and frame generation based on the vertex data and manual adjustment information of the stylized 3D face model output by the Avatar SDK, thereby enabling a virtual image preview in the smart photo application interface.

[0106] The following describes the user interface provided by the embodiments of the present application.

[0107] FIG. 2 a - FIG. 2 i are a set of user interfaces displayed on the electronic device 100 provided in an embodiment of the present application.

[0108] FIG. 2 a exemplarily shows an exemplary user interface 40 for displaying installed applications on the electronic device 100 .

[0109] As shown in Figure 2a, the user interface 40 displays one or more installed application icons, such as the user interface 40 displays smart photography 301, gallery 302, clock, calendar, memo, file management, email, music, calculator, Huawei video, sports health, weather, browser, smart life, settings, and recorder.

[0110] As shown in Figure 2a, the electronic device 100 can receive and respond to the user's touch operation on the smart photo 301 icon of the user interface 40 (i.e., the operation of opening the smart photo application), start the smart photo application, and display the user interface 41 as shown in Figure 2b on the display screen 194 of the electronic device 100. The user interface 41 may include a character creation control 303.

[0111] As shown in FIG2b , electronic device 100 may receive and respond to a user touch operation on create character control 303 of user interface 41, and then display user interface 42 as shown in FIG2c on display screen 194. User interface 42 may include target display area 304, photo control 305, and template display area 306. Target display area 304 may be used to display a stylized template selected by the user; template display area 306 may be used to display information indicating multiple selectable stylized templates.

[0112] As shown in Figure 2c, the electronic device 100 can receive and respond to the user's operation of selecting a target stylized template from multiple stylized templates. For example, if the user touches the indication information of the first stylized template in the template display area 306, the electronic device 100 can receive and respond to the user's touch operation on the indication information of the first stylized template, and the electronic device 100 can determine the first stylized template as the target stylized template.

[0113] In some embodiments, user interface 42 may be a fifth user interface.

[0114] As shown in FIG. 2 d , the electronic device 100 may display the target stylized template in the target display area 304 of the user interface 42 .

[0115] As shown in Figure 2d, the user can touch the photo control 305 on the user interface 42. The electronic device 100 can receive and respond to the user's touch operation on the photo control 305, start the camera of the electronic device 100, and display the user interface 43 shown in Figure 2e on the display screen 194. The user interface 43 can include an upload picture control 307, a photo control 308 and a preset display area 309. The preset display area 309 can be used to display the character image captured by the camera in real time.

[0116] In some embodiments, the user interface 43 may be a second user interface, or the user interface 43 may be a third user interface.

[0117] As shown in FIG2e , the electronic device 100 receives and responds to a user's touch operation on the photo control 308 of the user interface 43, saves the image of the person captured by the current camera, and determines the image of the person as the first image. The first image includes real facial features, which may include the face shape and facial features of the person.

[0118] As shown in FIG2e , the electronic device 100 receives and responds to a user touch operation on the upload picture control 307 of the user interface 43. The electronic device 100 launches the gallery 302 application and displays the user interface 44 shown in FIG2f on the display screen 194. The user interface 44 can be used to display thumbnails of multiple pictures stored in the electronic device 100. Further, the electronic device 100 can receive and respond to a user operation of selecting a target picture from multiple pictures, such as a user selecting the thumbnail of the first picture in the gallery. The electronic device 100 can then use the first picture as a first image. The first image includes real human facial features. The real human facial features can include facial shape features, facial features, etc.

[0119] In some embodiments, user interface 44 may be a fourth user interface.

[0120] In some embodiments, after electronic device 100 acquires a first image, default stylization parameters, and a target stylization template, it may generate an initial avatar based on the first image, default stylization parameters, and target stylization template. The method for generating the initial avatar will be described in detail later and is not detailed here. After generating the initial avatar, electronic device 100 may display user interface 45, as shown in FIG2g , on display screen 194. User interface 45 may include the initial avatar and preset controls 310 for adjusting stylization parameters. Preset controls for adjusting stylization parameters may include, but are not limited to, a progress bar, an input box, or a disc-style sliding button. FIG2g uses a progress bar as an example.

[0121] In some embodiments, Figure 2g may be a first user interface.

[0122] It should be noted that the stylization parameter can be understood as the ratio of the target stylized master model's features to the avatar's features. The stylization parameter can be continuously adjusted within the range of [0, 100%]. The default stylization parameter can be a value preset by the developer. The avatar can be understood as a 3D facial model obtained by fusing the real-life facial features with the target stylized master model's features according to the stylization parameters. For example, the avatar's facial features are obtained by fusing the character's facial features with the target stylized master model's facial features according to the stylization parameters. The initial avatar can be understood as a 3D facial model obtained by fusing the real-life facial features with the target stylized master model's features according to the default stylization parameters.

[0123] As shown in Figure 2g, electronic device 100 can receive and respond to a user's operation on preset control 310 of user interface 45 to determine user-defined stylization parameters. If preset control 310 is a progress bar, the user operation can be sliding the progress bar; if preset control 310 is an input box, the user operation can be inputting a stylization parameter value; if preset control 310 is a disc-shaped sliding button, the user operation can be sliding the disc-shaped sliding button. Furthermore, electronic device 100 generates an updated avatar based on the user-defined stylization parameters, the first image, and the target stylization template, and displays the updated avatar on user interface 45. How to regenerate the avatar will be described in detail later and will not be elaborated on here.

[0124] In some embodiments, the user's operation on the preset control 310 of the user interface 45 may be the first operation.

[0125] As shown in Figure 2h , a user can slide on a preset control 310 (using a progress bar as an example) of the user interface 45 . Different positions on the progress bar correspond to different stylized parameters. Sliding the progress bar to the left decreases the stylized parameter, while sliding the progress bar to the right increases the stylized parameter. For example, in Figure 2h (a), if the user slides to 10%, the stylized parameter is 10%; in Figure 2h (b), if the user slides to 50%, the stylized parameter is 50%; and in Figure 2h (c), if the user slides to 100%, the stylized parameter is 100%.

[0126] In the present application, as shown in FIG2h , the user can arbitrarily adjust the stylization parameters on the user interface 45 , and can display the virtual image based on different stylization parameters in real time on the user interface 45 , thereby enabling the user to freely adjust between the real face proportions and the stylized face proportions, thereby improving the user experience.

[0127] As shown in FIG2i , the user interface 45 further includes a display area 311 and a display area 312. The display area 311 includes multiple local feature controls, such as an eyebrow control, a nose control, a clothing control, and a headwear control. A user can view local feature materials by clicking on a local feature control. For example, if a user clicks on an eyebrow control, the electronic device can display multiple local feature materials (such as multiple eyebrow materials) on the display area 312. The user can then select one of the multiple materials, such as the first eyebrow, and the electronic device can then adjust the shape of the avatar's eyebrows based on the shape of the first eyebrow, thereby regenerating an updated avatar.

[0128] Next, the virtual image generation method provided by the embodiment of the present application will be described in conjunction with Figures 3 and 4a-4f, and is described in detail as follows:

[0129] S101: The electronic device starts a first application.

[0130] Specifically, the first application can be used to generate a virtual image for the user, and the first application can be the smart photo application in Figure 1b above. The smart photo application can be a system application or a third-party application.

[0131] For example, the electronic device 100 may receive and, in response to a user's touch operation on the smart photo taking 301 icon on the user interface 40 (ie, an operation to open the smart photo taking application), launch the first application.

[0132] S102: Acquire a first image containing a person through a first application.

[0133] Specifically, the first image includes real facial features, which may include two-dimensional facial shape features, two-dimensional facial features, etc. of the person.

[0134] The first image may be one or more pictures, one or more animated images, or one or more videos.

[0135] The first image can be obtained by real-time shooting by the electronic device, or can be obtained from a gallery stored in the electronic device.

[0136] In some embodiments, as shown in FIG2e , the electronic device 100 may receive and respond to a user's touch operation on the photo control 308 of the user interface 43 , save a person image captured by the current camera, and determine the person image as the first image.

[0137] In other embodiments, as shown in FIG2e , the electronic device 100 receives and responds to a user touch operation on the upload image control 307 of the user interface 43. The electronic device 100 launches the gallery 302 application and displays the user interface 44 shown in FIG2f on the display screen 194. The user interface 44 can be used to display thumbnails of multiple images stored in the electronic device 100. Furthermore, the electronic device 100 can receive and respond to a user operation of selecting a target image from multiple images, such as a user operation of selecting the thumbnail of the first image in the gallery. The electronic device 100 can then use the first image as the first image.

[0138] S103: Determine the first 3D base model selected by the user.

[0139] Specifically, the first 3D base model (also referred to as the first 3D face model) can be the target stylized base model mentioned above, that is, a stylized 3D character model designed by a modeler, such as a standard face shape / facial features designed and produced by a modeler in a cartoonized, comic, or exaggerated manner.

[0140] In some embodiments, the electronic device may display a user interface provided by a first application that displays multiple templates, detect a user operation of selecting a template on the user interface, and determine the first 3D template selected by the user based on the user operation. For example, as shown in FIG2c , the electronic device 100 may receive and respond to a user operation of selecting a target stylized template from multiple stylized templates. For example, if a user touches the indication information of the first stylized template in the template display area 306 , the electronic device 100 may receive and respond to the user's touch operation on the indication information of the first stylized template, and the electronic device 100 may determine the first stylized template as the target stylized template (i.e., the first 3D template).

[0141] In some embodiments, S103 may be performed first and then S102.

[0142] Optionally, at S104, the electronic device may display a user interface for displaying 3D materials provided by the first application, detect a user operation of selecting a material on the user interface, and determine the 3D material selected by the user based on the user operation. The 3D material includes but is not limited to hairstyles, clothing, jewelry, etc.

[0143] S105: Generate a real face model according to the first image and the topological structure of the first 3D base model.

[0144] Specifically, the real face model is actually obtained by transferring the facial features of the face in the first image to the topological structure of the first 3D base model. The facial features of the real face model are determined by the facial features of the face in the first image.

[0145] First, the first image is preprocessed to obtain a face area image.

[0146] Specifically, as shown in FIG4a , the electronic device can pre-process the first image (i.e., a portrait image) input by the user, such as detecting the face region in the photo using a face detection algorithm. The electronic device then extracts facial key points from the region and uses a face alignment algorithm to normalize and correct the deflection angle of the face, thereby outputting an aligned face region image. The face region image includes facial features and facial features that are closer to the person's actual facial features.

[0147] Then, the face area image is processed based on the first 3D base model and a preset algorithm to obtain a real face model.

[0148] Specifically, as shown in Figure 4b, the electronic device can process the facial region image based on the classic 3D Morphable Models (3DMM) technology principles to reconstruct a true 3D face, obtain a true face model, and cache this data. To ensure that the generated true face model is consistent with the topological structure of the first 3D base model, a non-rigid deformation algorithm can be used to perform topological migration on the 3DMM data.

[0149] It should be noted that the topological structure of a 3D model (such as the first 3D primitive or the real face model) represents the layout, structure, and connections of the points, lines, and surfaces in the 3D model. Because the real face model and the target stylized primitive share the same topological structure, the facial semantic information expressed by the points, lines, and surfaces in the real face model (for example, the point, line, and surface index values ​​corresponding to the nose region) corresponds one-to-one with the target stylized primitive. For example, the vertex data of the real face model corresponds one-to-one with the vertex data of the target stylized primitive.

[0150] It should also be noted that the vertex data of the target stylized base model includes the facial features of the target stylized base model, facial features (such as forehead, eyebrows, eyes, nose, lips, etc.). The vertex data of the real face model includes the real facial features of the person, such as face shape features, facial features, etc.

[0151] S106 , generating a first face model according to the real face model and the first 3D base model based on default stylization parameters.

[0152] Specifically, the stylization parameter is the ratio of the first 3D base model features to the virtual image features, and the stylization parameter can be continuously adjusted within the range of [0, 100%]. The default stylization parameter can be a value preset in advance by the developer. The first face model can indicate the vertex data of the initial virtual image. The facial features of the first face model are between the facial features of the real face model and the facial features of the first 3D base model. The initial virtual image can be the result of electronic device rendering based on the real face model and the first 3D base model, and is used to display the eyes, nose, mouth, skin and other features of the stylized character on the user interface.

[0153] In some embodiments, the first 3D face model indicates a target stylized base model, and the first face model indicates an initial 3D face model generated based on the first image and the target stylized base model.

[0154] For example, as shown in Figure 4c, based on the vertex data of the real face model and the vertex data of the target stylized base model (i.e., the first 3D base model), the vertex data is weighted fused according to the default stylization parameters, and the 3D face structure of the default stylization parameters (i.e., the first face model) is output. The stylization parameters are used as weight values ​​to balance the proportion of real face and stylized face features. The vertex data weighted fusion formula is shown in Formula 1. stylozed =w*Mesh target +(1-w)*Mesh real (Formula 1)

[0155] Meshtarget represents the vertex data of the target stylized base model, Meshreal represents the vertex data of the real face model, Meshstylized represents the vertex data of the stylized 3D face model (such as the first face model), and w represents the stylization parameter.

[0156] S107: Generate an initial virtual image based on the first face model.

[0157] Specifically, vertex data is obtained based on the first face model, parsed and loaded into the 3D scene, and the vertex data is used to replace the facial vertex data of the default virtual image in the 3D scene (such as the virtual image corresponding to the first 3D model) to obtain an initial virtual image. The initial virtual image can use the bone parameters, expression base parameters, and texture material properties of the default virtual image.

[0158] In some embodiments, the vertex data of the first facial model is parsed and loaded into the 3D scene, replacing the facial vertex data of the default avatar in the 3D scene (e.g., the avatar corresponding to the first 3D model). Simultaneously, based on the 3D material input by the user, the avatar's hairstyle, clothing, jewelry, etc. are determined from a 3D material library to generate an initial avatar. The 3D material library stores multiple hairstyle models, clothing models, jewelry models, etc. for avatars.

[0159] For example, as shown in FIG2g and FIG4d, the electronic device can render a frame in real time and display the initial virtual image on the display screen 194 for the user to preview, so that the user can view the generated personalized virtual image in real time, thereby improving the user experience.

[0160] It should be noted that the higher the default stylization parameter setting, the higher the stylization degree of the initial virtual image (such as the higher the cartoon degree), that is, the more similar the initial virtual image is to the target stylized base model; the lower the default stylization parameter setting, the lower the stylization degree of the initial virtual image (such as the lower the cartoon degree), that is, the more similar the initial virtual image is to the real human face model.

[0161] In this application, the real face model is actually obtained by migrating the facial features of a 2D human face to the topological structure of the first 3D base model. The facial features of the real face model are determined by the facial features of the human face in the portrait photo, and it is impossible to make the real face model contain more or less stylized features (such as more cartoon-like or more comic-like). Therefore, the real face model is fused with the first 3D base model, so that the virtual image generated based on the fusion result can be adjusted between "closer to the real face" and "more stylized", meeting the preferences of different users and improving the user experience.

[0162] S108: Receive a user operation for adjusting stylization parameters, and determine new stylization parameters according to the user operation.

[0163] Specifically, the user operations used to adjust the stylized parameters can be of various types, such as operations on the user interface (touch operations, drag operations), voice commands, air operation commands, etc., which are not specifically limited in this application.

[0164] For example, as shown in FIG2g , the electronic device 100 can receive and respond to a user sliding operation on a progress bar (i.e., preset control 310) of the user interface 45 to determine user-defined stylization parameters, i.e., new stylization parameters. As shown in FIG2h , the user can perform a sliding operation on the progress bar of the user interface 45 , with different positions on the progress bar corresponding to different stylization parameters. Sliding the progress bar to the left decreases the stylization parameters, while slidring the progress bar to the right increases the stylization parameters. For details, please refer to the description of FIG2h above, which will not be repeated here.

[0165] S109 , generating a second face model based on the real face model and the first 3D base model and the new stylized parameters.

[0166] Specifically, vertex data of a real face model and vertex data of a first 3D base model can be obtained, and the new stylization parameters can be used as weight values. The vertex data of the real face model and the vertex data of the first 3D base model can be re-weighted and fused using the above formula to obtain a second face model. The second face model can indicate the vertex data of the updated virtual image. The facial features of the second face model are between the facial features of the real face model and the facial features of the first 3D base model. The difference between the second face model and the first face model is the different proportion of the first 3D base model features, and therefore the corresponding virtual image is also different.

[0167] For example, as shown in FIG4e, the user can preview the initial avatar and then modify the stylization parameters using a slider on the user interface (user interface 45 shown in FIG2g) to balance the stylization ratio of the avatar. After receiving the new stylization parameters, the electronic device re-merges the real face model with the target style base model and refreshes the output of the new face model.

[0168] It should be noted that when the slider is dragged to the right, the stylization parameter increases, and the stylization degree of the updated virtual image is higher (such as the higher the cartoon degree), that is, the updated virtual image is more similar to the target stylized base model; when the slider is dragged to the left, the stylization parameter decreases, and the stylization degree of the updated virtual image is lower (such as the lower the cartoon degree), that is, the updated virtual image is more similar to the real face model.

[0169] S110: Generate an updated virtual image based on the second face model.

[0170] Specifically, the vertex data of the second face model is parsed and loaded into the 3D scene, and the facial vertex data of the default virtual image in the 3D scene (such as the virtual image corresponding to the first 3D model) is replaced to obtain an updated virtual image. The updated virtual image can use the bone parameters, expression base parameters, and texture material properties of the default virtual image.

[0171] In some embodiments, the updated virtual image may be the first virtual image.

[0172] In some embodiments, the vertex data of the second facial model is parsed and loaded into the 3D scene, replacing the facial vertex data of the default avatar in the 3D scene (e.g., the avatar corresponding to the first 3D model). Simultaneously, based on the 3D material input by the user, the avatar's hairstyle, clothing, jewelry, etc. are determined from the 3D material library to generate an updated avatar.

[0173] For example, as shown in Figure 4e, the electronic device can display an updated avatar in real time on the user interface. The user can input a single facial photo and stylization parameters. The system calls an algorithm to generate a 3D facial model. The system then fuses it with the target stylized base model based on the stylization parameters to obtain a stylized 3D facial model. The result is then parsed and loaded, and rendered as a frame to the user interface for preview. Furthermore, the user can adjust and modify the stylization parameters based on the preview results. After receiving the new stylization parameters, the system again fuses the 3D facial model with the target style base model, refreshes the output of the new stylized 3D facial model, and provides real-time feedback to the user interface, thereby generating a personalized avatar that meets the user's preferences and improves the user experience.

[0174] In this application, as shown in Figure 4f, users can adjust stylization parameters by sliding, allowing real-time previews of avatars with varying degrees of stylized deformation. Specifically, users can freely adjust stylization parameters on the user interface of an electronic device. The electronic device then fuses the real face model with the first 3D base model based on the user-defined stylization parameters to generate a new 3D avatar. The new 3D avatar can then be displayed on the electronic device, achieving a real-time preview effect for the user, enhancing the user experience.

[0175] Optional S111: receiving a user operation for adjusting facial features, and adjusting the facial features of the updated virtual image.

[0176] Specifically, the electronic device can receive and respond to a user's manual adjustment operation (i.e., a user operation to adjust facial features) and obtain manual adjustment information, including but not limited to updated bone coefficients, updated expression base coefficients, updated clothing and accessory models, updated texture and material properties, etc. The electronic device can then regenerate an updated avatar based on the second face model and the manual adjustment information.

[0177] Exemplarily, as shown in FIG2i , the user interface 45 further includes a display area 311 and a display area 312. The display area 311 includes multiple local feature controls, such as an eyebrow control, a nose control, a clothing control, a headwear control, etc. The user can view local feature materials by clicking on a local feature control. For example, if the user clicks on the eyebrow control, the electronic device can display multiple local feature materials (such as multiple eyebrow materials) on the display area 312. The user can then select one of the multiple materials, such as the first eyebrow, and the electronic device can then adjust the eyebrow shape of the avatar based on the shape of the first eyebrow, thereby regenerating an updated avatar.

[0178] The electronic device can display a manual adjustment and status interface, which can be used to manually adjust the local position of the virtual image's face through the bones and expression bases bound in advance by the modeler. It can also be used to customize the appearance (including hairstyle, hair color, glasses, clothes, etc.) in the material library prepared in advance by the modeler.

[0179] In this application, users can also perform secondary editing on the generated 3D virtual image on the user interface, such as manually adjusting the eyebrows, nose, mouth, face shape, etc. of the 3D virtual image to generate a more personalized virtual image that meets the user's preferences.

[0180] 5 , which describes how various modules within an electronic device cooperate to implement a virtual image method.

[0181] S201: Start a smart photo-taking application, which receives a first image uploaded or taken by a user and a first 3D base model selected by the user.

[0182] After the electronic device receives the user's operation to open the smart photo application, it starts the smart photo application. For the specific implementation of S201 and S202, please refer to the detailed introduction of S101 in Figure 3 above.

[0183] S202: The smart camera application receives and responds to a first request initiated by a user, and sends a first image to a model generation module.

[0184] In some embodiments, the first request includes a first 3D base model.

[0185] S203: The model generation module pre-processes the first image to obtain a face area image.

[0186] For the specific implementation of S203 , please refer to the detailed description of S105 in FIG. 3 .

[0187] S204: The model generation module generates a real face model according to the face region image and the topological structure of the first 3D base model.

[0188] For the specific implementation of S203 , please refer to the detailed description of S105 in FIG. 3 .

[0189] S205: The model generation module determines and saves vertex data of the real face model.

[0190] S206 , the model generation module generates a first face model according to the vertex data of the real face model and the vertex data of the first 3D base model and based on default stylization parameters.

[0191] For the specific implementation of S206 , please refer to the detailed description of S106 in FIG. 3 .

[0192] S207: The model generation module sends the first face model to the 3D engine module.

[0193] S208: The 3D engine module renders the first face model into a frame to obtain an initial virtual image.

[0194] S209: The 3D engine module sends the initial virtual image to the smart camera application.

[0195] S210: The smart camera application displays an initial virtual image on a display screen of the electronic device.

[0196] For the specific implementation of S207-S210, please refer to the detailed introduction of S107 in Figure 3 above.

[0197] S211 : The smart camera application receives a user operation for adjusting stylization parameters and determines the stylization parameters.

[0198] The user may adjust the stylized parameters on the user interface (the user interface shown in FIG. 2 g ), and then the smart photo taking application executes S211 .

[0199] S212: The smart camera application sends stylized parameters to the model generation module.

[0200] S213, the model generation module determines new stylization parameters; generates a second face model based on the new stylization parameters according to the vertex data of the real face model and the vertex data of the first 3D base model.

[0201] For the specific implementation of S211 - S213 , please refer to the detailed description of S108 and S109 in FIG. 3 .

[0202] S214: The model generation module sends the second face model to the 3D engine module.

[0203] S215 , the 3D engine module renders a frame of the second face model to obtain an updated virtual image.

[0204] S216: The 3D engine module sends the updated virtual image to the smart camera application.

[0205] S217, the smart camera application displays the updated virtual image through the display screen of the electronic device.

[0206] For the specific implementation of S214-S217, please refer to the detailed introduction of S110 in Figure 3 above.

[0207] S218, the smart camera application sends facial adjustment information to the 3D engine module.

[0208] The user may initiate a user operation of adjusting facial features on the user interface, and then the smart camera application may execute S218.

[0209] S219, the 3D engine module adjusts the facial features of the updated virtual image based on the facial adjustment information.

[0210] S220: The 3D engine module sends the adjusted virtual image to the smart camera application.

[0211] S221, the smart camera application displays the adjusted virtual image through the display screen of the electronic device.

[0212] For the specific implementation of S218-S221, please refer to the detailed introduction of S111 in Figure 3 above.

[0213] In some embodiments, the avatar generated by the avatar generation method provided by this application can be applied to scenarios such as games, virtual social networking, and 3D model design as shown in Figure 6. For example, the characters in the game can be replaced by the avatar generated by this application, and the characters in the virtual social networking can be replaced by the avatar generated by this application, etc.

[0214] Please refer to FIG7 . Next, the process of another virtual image generation method provided by the embodiment of the present application will be described in conjunction with FIG7 . The details are as follows:

[0215] S301: The electronic device obtains a first image including a human face.

[0216] For the specific implementation of S301, please refer to the detailed introduction of S102 in Figure 3 above.

[0217] S302: The electronic device obtains a first 3D face model.

[0218] For the specific implementation of S302, please refer to the detailed description of S103 in FIG. 3 above.

[0219] S303: The electronic device obtains a first stylization parameter related to the first image selected by the user.

[0220] For the specific implementation of S303 , please refer to the detailed description of S108 in FIG. 3 .

[0221] S304: The electronic device generates and displays a first virtual image based on the first image, the first 3D face model, and the first stylization parameter.

[0222] Specifically, the first stylization parameter indicates the influence ratio of the first image and the first 3D face model on the first virtual image. For the specific implementation of S304, please refer to the detailed description of S105, S109 and S110 in Figure 3 above.

[0223] In the embodiments of the present application, based on stylization parameters input by the user, the facial features of the person are fused with a stylized 3D face model to generate an avatar, which can then be displayed on a user interface for easy browsing. Since different stylization parameters generated by the user result in different avatars, the user can adjust the stylization parameters to adjust the generated avatar between "closer to a real face" and "more stylized," thereby satisfying different user preferences and improving the user experience.

[0224] In some embodiments, the electronic device obtains default stylization parameters and generates N initial virtual images based on the first image, the default stylization parameters, and N 3D primitives (N is an integer greater than or equal to 1), and displays the N initial virtual images on the display screen 194 of the electronic device. Furthermore, the electronic device receives and responds to a user's operation of selecting a target initial virtual image from the N initial virtual images, and displays the target initial virtual image and a progress bar for adjusting the stylization parameters on the display screen 194. The electronic device can then receive and respond to a user's sliding operation on the progress bar to determine new stylization parameters, and regenerate the virtual image based on the new stylization parameters, the first image, and the target stylization primitive (i.e., the 3D primitive corresponding to the target initial virtual image), and display the regenerated virtual image on the display screen 194.

[0225] The present application provides an electronic device, which includes a first processing unit, the first processing unit being configured to: obtain a first image including a human face; obtain a first 3D face model; obtain a first stylization parameter selected by a user and associated with the first image; generate and display a first virtual image based on the first image, the first 3D face model, and the first stylization parameter; the first stylization parameter indicating a proportion of influence of the first image and the first 3D face model on the first virtual image.

[0226] The present application provides a computer storage medium, characterized in that the computer storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned virtual image generation methods.

[0227] An embodiment of the present application provides an electronic device comprising a processor configured to support the electronic device in implementing the corresponding functions of any of the aforementioned methods for generating an avatar. The electronic device may further comprise a memory coupled to the processor and storing program instructions and data necessary for the electronic device. The electronic device may further comprise a communication interface for communicating with other devices or a communication network.

[0228] This application provides a chip system that includes a processor for supporting an electronic device in implementing the aforementioned functions, such as generating or processing information involved in the aforementioned avatar generation method. In one possible design, the chip system also includes a memory for storing program instructions and data necessary for the electronic device. The chip system can be composed of a single chip or include a chip and other discrete components.

[0229] The present application provides a computer program, characterized in that the computer program includes instructions, which, when executed by a computer, enable the computer to execute the above-mentioned virtual image generation method.

[0230] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0231] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0233] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, the functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0235] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0236] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a virtual image, characterized in that: Applied to electronic equipment, the method comprises: The electronic device acquires a first image including a face; The electronic device acquires a first 3D face model; The electronic device acquires a first stylization parameter related to the first image selected by a user; The electronic device generates and displays a first virtual image based on the first image, the first 3D face model, and the first stylization parameter; the first stylization parameter indicates the influence ratio of the first image and the first 3D face model on the first virtual image.

2. The method according to claim 1, characterized in that: Before the electronic device acquires the first stylization parameter related to the first image selected by the user, the method further includes: The electronic device generates and displays an initial virtual image based on the first image, the first 3D face model, and default stylization parameters; the default stylization parameters indicate the influence ratio of the first image and the first 3D face model on the initial virtual image.

3. The method according to claim 2, characterized in that: The electronic device generates and displays an initial virtual image according to the first image, the first 3D face model, and default stylization parameters, including: The electronic device generates a real face model according to the first image and the first 3D face model; The electronic device generates a first face model according to the real face model, the first 3D face model, and the default stylization parameters; The electronic device generates the initial virtual image according to the first face model and displays a first user interface, in which the initial virtual image is displayed.

4. The method according to any one of claims 1 to 3, characterized in that: The electronic device acquires a first image including a face, including: The electronic device displays a second user interface, the second user interface includes a first preview area and a first control, the first preview area displays an image containing a human face from a camera of the electronic device; The electronic device receives an operation applied to the first control and saves the image displayed in the first preview area as the first image.

5. The method according to any one of claims 1 to 3, characterized in that: The electronic device acquires a first image including a face, including: The electronic device displays a third user interface; The electronic device receives an operation performed on the third user interface and displays a fourth user interface, wherein the fourth user interface displays one or more images, and the one or more images include a thumbnail of the first image; The electronic device receives an operation performed on a thumbnail of the first image and acquires the first image.

6. The method according to any one of claims 1 to 5, characterized in that: Before the electronic device acquires the first 3D face model, the method further includes: The electronic device displays a fifth user interface, wherein the fifth user interface displays indication information of one or more 3D face models, wherein the one or more 3D face models include the first 3D face model; The electronic device receives instruction information acting on the first 3D face model.

7. The method according to any one of claims 3 to 6, characterized in that: The electronic device generates and displays a first virtual image according to the first image, the first 3D face model, and the first stylized parameter, including: The electronic device generates a second face model according to the real face model, the first 3D face model, and the first stylization parameter; The electronic device generates the first virtual image according to the second face model and displays the first virtual image in the first user interface.

8. The method according to any one of claims 3 to 7, characterized in that: The electronic device acquires a first stylization parameter related to the first image selected by a user, including: The electronic device receives a first operation acting on the first user interface, where the first operation is used to adjust a stylization parameter; In response to the first operation, the first stylization parameter selected by the user is determined.

9. The method according to claim 7, characterized in that: The first user interface includes a slide bar control, and the first operation is an operation of sliding the slide bar control to adjust a stylized parameter.

10. The method according to any one of claims 3 to 9, characterized in that: The method further comprises: The electronic device receives the adjustment operation on the first user interface for the first virtual image, adjusts the facial features of the first virtual image, and obtains an updated first virtual image; The electronic device displays the updated first virtual image on the first user interface.

11. The method according to any one of claims 7 to 10, characterized in that: The electronic device generates the first virtual image according to the second face model, including: The electronic device renders the second face model to obtain the first virtual image.

12. The method according to any one of claims 7 to 10, characterized in that: The electronic device generates the first virtual image according to the second face model, including: The electronic device obtains a default 3D material, and renders the second face model based on the default 3D material to obtain the first virtual image, wherein the default 3D material includes one or more of body features and clothing features of the 3D character model.

13. An electronic device, characterized in that: The electronic device includes a first processing unit, which is used to: obtain a first image including a face; obtain a first 3D face model; obtain a first stylization parameter related to the first image selected by a user; generate and display a first virtual image based on the first image, the first 3D face model, and the first stylization parameter; the first stylization parameter indicates the proportion of influence of the first image and the first 3D face model on the first virtual image.

14. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-9.

15. A chip system, characterized in that: The chip system includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected via lines. Instructions are stored in the at least one memory. When the instructions are executed by the processor, the method described in any one of claims 1 to 12 is implemented.

16. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method as claimed in any one of claims 1 to 12.

17. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are run on an electronic device, the electronic device executes the method as claimed in any one of claims 1 to 12.

Citation Information

Patent Citations

  • Virtual image generation method, device and equipment, and readable storage medium

    CN108510437A

  • Three-dimensional virtual image generation method and device, electronic equipment and storage medium

    CN112541963A

  • Virtual image generation method and device, electronic equipment and storage medium

    CN113240778A

  • Image processing method, image processing system and electronic equipment

    CN114266695A

  • Face pinching processing method

    CN114549293A