A method, apparatus, device, and storage medium for generating a virtual face.

Abstract

This invention discloses a method, apparatus, device, and storage medium for generating a virtual face. The method includes acquiring a real face image; generating multiple partial sub-images corresponding to the real face image, each including predefined facial elements; determining the type of facial elements in each partial sub-image based on an image feature set corresponding to each partial sub-image and a geometric feature set of facial elements; determining target standard face-shaping parameters corresponding to each facial element in the real face image based on the type of facial elements in each partial sub-image and preset standard face-shaping parameters corresponding to each type of facial element; and forming a virtual face matching the real face image based on the target standard face-shaping parameters corresponding to each facial element in the real face image. The technical solution of this invention achieves accurate identification of facial features, accurate classification of facial features, and precise generation of virtual faces.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and more particularly to convolutional neural network technology, and especially to a method, apparatus, device and storage medium for generating virtual faces. Background Technology

[0002] With the development of computer technology and the gaming industry, role-playing games are attracting more and more gamers. To meet users' needs for customizing their character's face, a character creation feature is usually added when creating a virtual character. This feature allows players to adjust the facial features and shape of their virtual character, helping them achieve a customized appearance.

[0003] Current game character customization is achieved by acquiring facial image features. Specifically, this involves extracting color, texture, shape, and spatial relationship features from facial images to identify facial features, obtain customization parameters, and then customize the game character's face.

[0004] In the process of realizing this invention, the inventors discovered the following defects in the prior art: by simply obtaining image features from real face images to identify facial features and generate face-shaping parameters, the generation of virtual faces is inaccurate due to the inaccurate identification of facial features in real face images. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for generating virtual faces, so as to achieve accurate identification of facial features, accurate classification of facial features, and precise generation of virtual faces.

[0006] In a first aspect, embodiments of the present invention provide a method for generating a virtual face, the method comprising:

[0007] Acquire a real human face image and generate multiple local sub-images corresponding to the real human face image. Each local sub-image corresponds to a set facial element on the human face.

[0008] Based on the image feature set corresponding to each local sub-image and the geometric feature set of facial elements, determine the type of facial elements in each local sub-image;

[0009] Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of facial element, the target standard face-shaping parameters corresponding to each facial element in the real face image are determined.

[0010] A virtual face matching the real face image is formed based on the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0011] Secondly, embodiments of the present invention also provide a virtual face generation apparatus, the apparatus comprising:

[0012] The facial element generation module acquires a real face image and generates multiple local sub-images corresponding to the real face image. Each local sub-image corresponds to a set facial element on the face.

[0013] The facial element type determination module determines the type of facial elements in each local sub-image based on the image feature set corresponding to each local sub-image and the facial element geometric feature set.

[0014] The standard face-shaping parameter determination module is used to determine the target standard face-shaping parameters corresponding to each face element in the real face image based on the type of the face element in each local sub-image and the preset standard face-shaping parameters corresponding to each type of each face element.

[0015] The virtual face formation module is used to form a virtual face that matches the real face image based on the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0017] One or more processors;

[0018] Storage device for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual face generation method as described in any embodiment of the present invention.

[0020] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the virtual face generation method as described in any embodiment of the present invention.

[0021] This invention employs a technical means to acquire a real face image and generate multiple local sub-images corresponding to the real face image, each containing pre-defined facial elements. Based on the image feature set corresponding to each local sub-image and the geometric feature set of the facial elements, the type of each facial element in the local sub-image is determined. Based on the type of each facial element in the local sub-image and preset standard face-shaping parameters corresponding to each type of each facial element, target standard face-shaping parameters corresponding to each facial element in the real face image are determined. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, a virtual face matching the real face image is formed. In the process of generating the virtual face, the classification of real face facial elements and face-shaping parameters are comprehensively considered. This solves the problem of inaccurate virtual face generation caused by inaccurate recognition of facial features in real face images in existing technologies, achieving accurate recognition of facial features, accurate classification of facial features, and precise generation of virtual faces. Attached Figure Description

[0022] Figure 1 The flowchart below illustrates a method for generating a virtual face, as provided in Embodiment 1 of the present invention.

[0023] Figure 2A A flowchart is provided for another method of generating a virtual face in Embodiment 2 of the present invention;

[0024] Figure 2B The flowchart illustrates a specific application scenario of the virtual face generation method provided in Embodiment 2 of the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of a virtual face generation device provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it.

[0028] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention, not the entire structure. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] The terms "first" and "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units may include steps or units not listed, but rather steps or units not listed.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a method for generating a virtual face according to Embodiment 1 of the present invention. This embodiment is applicable to situations where a virtual face is generated based on a real face image. The method can be executed by a virtual face generation device, which can be implemented through software and / or hardware, and is generally integrated into a client or server with image processing capabilities. (Reference) Figure 1 The method specifically includes the following steps:

[0032] S110. Obtain a real face image and generate multiple local sub-images corresponding to the real face image, each local sub-image corresponding to a set facial element on the face.

[0033] The real-life facial image can serve as the basis for generating the virtual face; for example, it could be a facial image actually captured by a user using a camera. A local sub-image can refer to a portion of the overall face image to be classified, including a specific facial element. Figure 1 Generally, these are proper subsets of real human face images, and there may be some overlap between different local sub-images. For example, a local sub-image including the facial contour may overlap with a local sub-image including the eyes. Defined facial elements can refer to facial features that affect a person's appearance, such as the facial contour, eyes, nose, eyebrows, and mouth.

[0034] Optionally, a real human face image can be used to generate multiple corresponding local sub-images based on the set facial elements on the face.

[0035] In an optional embodiment of the present invention, generating multiple local sub-images corresponding to a real face image may include: annotating multiple facial key points in the real face image; determining target facial key points corresponding to each facial element in the real face image from among the annotated facial key points according to the set of facial key points corresponding to different facial elements; and forming local sub-images corresponding to each facial element in the real face image according to the target facial key points corresponding to each facial element in the real face image.

[0036] Among them, facial landmarks can refer to the points of facial features. Target facial landmarks can refer to the facial landmarks corresponding to the facial elements in the image to be classified.

[0037] Optionally, key points can be annotated on the real face image to obtain multiple face key points; then, based on the set of face key points corresponding to different facial elements, target face key points corresponding to each facial element in the real face image can be determined from the annotated face key points; based on the target face key points corresponding to each facial element in the real face image, local sub-images corresponding to each facial element in the real face image can be formed.

[0038] In a specific example, based on the existing GBDT (One Millisecond Face Alignment with an Ensemble of Regression Trees) algorithm, 68 facial keypoints can be annotated in the input face image, such as facial contour points, corner points of the eyes, center points of the eyes, and corner points of the mouth. Facial keypoints at different locations are numbered with different indices. Furthermore, by establishing a set of facial keypoints corresponding to different facial elements, for example, the facial contour corresponds to a set of keypoints numbered 1 to 17, and the eyes correspond to a set of keypoints numbered 37 to 46, etc., and then by combining this set of facial keypoints with the facial keypoints annotated in the face image to be classified, the image regions containing different facial elements in the face image to be classified can be divided, thus forming the aforementioned local sub-images.

[0039] S120. Based on the image feature set corresponding to each local sub-image and the geometric feature set of facial elements, determine the type of facial elements in each local sub-image.

[0040] The image feature set includes multiple image features corresponding to facial elements in the local sub-image; the image features may include at least one of the following: color features, texture features, shape features, and spatial relationship features. The facial element geometric feature set includes multiple facial element geometric features corresponding to facial elements in the local sub-image, wherein the facial element geometric features may include at least one of the following: the scale of the facial element (typically, length or width, etc.), tilt angle, angle between keypoints, and edge curvature, etc.

[0041] In this way, multiple optional types can be set for each facial element in advance to cover different morphological features of different facial elements. For example, for the facial element of face shape, multiple optional face types can be set, such as round face shape, oval face shape, egg-shaped face shape, inverted egg-shaped face shape, square face shape, rectangular face shape, trapezoidal face shape, inverted trapezoidal face shape, diamond face shape, and pentagonal face shape.

[0042] Accordingly, based on at least one image feature and at least one geometric feature of the face element included in a certain local sub-image, the type to which the face element belongs can be determined from among the selectable face types that match the face element.

[0043] Optionally, a matching standard image feature set and a face element set feature set can be pre-set for each type of each face element. Then, after obtaining the image feature set corresponding to the face element in a certain local sub-image and the face element geometric feature set, the type most similar to the face element in the local sub-image is obtained by calculating the similarity, and this type is taken as the type to which the face element in the local sub-image belongs.

[0044] In an optional embodiment of the present invention, determining the type of facial element in each local sub-image based on the image feature set corresponding to each local sub-image and the facial element geometric feature set may include: obtaining the target facial element corresponding to the currently processed target local sub-image; obtaining a target facial element classification model that matches the target facial element from all pre-trained facial element classification models, and inputting the target local sub-image into the target facial element classification model; extracting the matching image feature set and facial element geometric feature set from the target local sub-image through the facial element classification model, and determining the type of the target facial element in the target local sub-image based on the image feature set and facial element geometric feature set.

[0045] Here, the target local subgraph can refer to a specific local subgraph that needs to be processed from all the acquired local subgraphs. The target facial element can refer to the facial elements contained within the target local subgraph. The facial element classification model can be a pre-trained model used to determine the type of a facial element; the facial element classification model can be obtained through at least one round of training using standardized facial elements. Alternatively, the target facial element classification model can refer to the facial element classification model corresponding to a specific facial element determined from all facial element classification models.

[0046] Specifically, the corresponding facial elements can be obtained from the currently processed target local sub-image and identified as target facial elements. Among all pre-trained facial element classification models, the target facial element classification model corresponding to the target facial element can be obtained. Then, the currently processed target local sub-image can be input into the obtained target facial element classification model. Furthermore, the target facial element classification model can extract the corresponding image feature set and facial element geometric feature set from the target local sub-image, and determine the type of the target facial element in the target local sub-image based on the extracted image feature set and facial element geometric feature set.

[0047] In another optional embodiment of the present invention, the target facial element classification model may specifically include: an image feature extraction network, a facial element geometric feature extraction network, a feature fusion network, and a classification network; the image feature extraction network is used to extract image feature sets in multiple dimensions based on the input target local sub-image; the facial element geometric feature extraction network is used to extract facial element geometric feature sets corresponding to the target facial elements in the input target local sub-image; the feature fusion network is used to concatenate the image feature sets and the facial element geometric feature sets to obtain a fused feature set; and the classification network is used to determine the type of the target facial element in the target local sub-image based on the fused feature set.

[0048] The image feature extraction network can be a convolutional neural network based on an attention mechanism, such as ResNet50 (residual network). The fused feature set can refer to the facial element features obtained by concatenating and fusing image features with facial geometric features.

[0049] In an optional embodiment of the present invention, an image feature set can be extracted from the input target local sub-image in multiple dimensions according to the image feature extraction network in the target facial element classification model; a facial element geometric feature set can be extracted from the input target local sub-image according to the facial element geometric feature extraction network; further, the extracted image feature set and facial element geometric feature set can be concatenated by a feature fusion network to obtain a fused feature set; then, a classification network can be used to classify the type of the target facial element in the target local sub-image according to the obtained fused feature set.

[0050] The advantage of this setup is that when the image feature extraction network extracts image features, the geometric features of the texture structure are weakened due to the convolution and pooling layers, which can hinder the recognition of facial elements. This solution can neutralize the weakening by using the geometric features of facial elements, thereby improving the recognition accuracy.

[0051] S130. Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of each facial element, determine the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0052] Standard face-shaping parameters refer to face-shaping parameters that are pre-created by the face-shaping tool based on predefined standardized face element types, each corresponding to a specific type of standardized face element. Target standard face-shaping parameters refer to standard face-shaping parameters selected from the preset standard face-shaping parameters that correspond to the type of face elements in a real human face image.

[0053] Optionally, the target standard face-shaping parameters corresponding to each facial element in the real face image can be determined based on the type of facial elements in each local sub-image of the real face image and the preset standard face-shaping parameters.

[0054] S140. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, a virtual face matching the real face image is formed.

[0055] Optionally, the target standard face-shaping parameters corresponding to each facial element in the determined real face image can be assembled, and the face-shaping tool can generate a virtual face that matches the real face image.

[0056] The technical solution of this embodiment acquires a real face image and generates multiple local sub-images corresponding to the real face image, each containing facial elements. Based on the image feature set corresponding to each local sub-image and the geometric feature set of the facial elements, the type of the facial elements in each local sub-image is determined. Based on the type of the facial elements in each local sub-image and preset standard face-shaping parameters corresponding to each type of each facial element, target standard face-shaping parameters corresponding to each facial element in the real face image are determined. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, a virtual face matching the real face image is formed. In the process of generating a virtual face, the classification of real face facial elements and face-shaping parameters are comprehensively considered. This solves the problem of inaccurate virtual face generation caused by inaccurate recognition of facial features in real face images in existing technologies, achieving accurate recognition of facial features, accurate classification of facial features, and precise generation of virtual faces.

[0057] Example 2

[0058] Figure 2A This is a flowchart illustrating another method for generating a virtual face according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment preferably adds further operations after acquiring a real face image and generating multiple local sub-images corresponding to the real face image, with each local sub-image corresponding to a set facial element on the face. Preferably, the operation of forming a virtual face matching the real face image based on target standard face-shaping parameters corresponding to each facial element in the real face image is further refined. (Reference) Figure 2A The method specifically includes the following steps:

[0059] S210. Obtain a real face image and generate multiple local sub-images corresponding to the real face image, each local sub-image corresponding to a set facial element on the face.

[0060] S220. Generate a set of geometric features of facial elements corresponding to each local sub-image.

[0061] Optionally, each local sub-image of a real face image can generate a corresponding set of geometric features of facial elements.

[0062] In an optional embodiment of the present invention, generating a set of facial element geometric features corresponding to each local sub-image may include: inputting each local sub-image into a pre-trained facial element geometric feature extraction network to obtain a set of facial element geometric features corresponding to each local sub-image.

[0063] Specifically, each local sub-image of a real face image can be input into a pre-trained facial element geometric feature extraction network to extract the facial element geometric feature set corresponding to each local sub-image.

[0064] S230. Based on the geometric feature sets of each facial element, determine the associated facial element description information corresponding to the real face image.

[0065] Among them, the associated facial element description information can refer to the information describing the influence and connection between various facial elements in a real face image. The associated facial element description information can include the relative positional relationship and / or size ratio relationship between multiple facial elements in a real face image.

[0066] Specifically, based on the acquired geometric feature sets of each facial element, associated facial element description information corresponding to the real face image can be determined. This includes personalized information such as the distance between the nose and mouth, the tilt angle of the nose relative to the face, and the ratio of nose length to face length. The advantage of this approach is that by acquiring associated facial element description information, it is possible to effectively distinguish real face images that belong to the same facial element type but differ in overall facial appearance due to differences in geometric features, relative positions, and / or size ratios. This maximizes the extraction of personalized information from different faces and avoids generating identical virtual faces.

[0067] S240. Based on the image feature set corresponding to each local sub-image and the geometric feature set of facial elements, determine the type of facial elements in each local sub-image.

[0068] S250. Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of each facial element, determine the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0069] S260. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, and the associated facial element description information, a virtual face matching the real face image is formed.

[0070] In this embodiment, the target standard face-shaping parameters corresponding to each facial element in a real face image can be assembled and combined with the associated facial element description information to generate a virtual face that matches the real face image by the face-shaping tool.

[0071] The technical solution of this embodiment involves acquiring a real face image and generating multiple local sub-images corresponding to the real face image, including defined facial features; then generating a set of geometric features of facial elements corresponding to each local sub-image; determining the associated facial element description information corresponding to the real face image based on each set of geometric features; determining the type of facial elements in each local sub-image based on the image feature set corresponding to each local sub-image and the set of geometric features of facial elements; and finally determining the type of facial elements in each local sub-image based on the type of facial elements in each local sub-image and the preset values ​​for each facial element. The standard face-shaping parameters corresponding to different types are used to determine the target standard face-shaping parameters corresponding to each facial element in a real face image. Based on the target standard face-shaping parameters and the associated facial element description information, a technical means of forming a virtual face that matches the real face image is formed. When generating the geometric features and set of facial elements, the associated facial element description information is further obtained. This solves the problem that the existing technology is inaccurate in generating virtual faces due to the inaccurate recognition of facial features in real face images. It achieves the effect of accurately recognizing facial features, accurately classifying facial features, and accurately generating virtual faces.

[0072] For example, the following is a specific application scenario of the virtual face generation method.

[0073] Figure 2B This is a flowchart for this specific application scenario. Game players upload real facial images, and a pre-trained facial element classification model identifies the type of each facial element in the real image. Then, a pre-built facial element sculpting parameter library is searched to obtain target facial element sculpting parameters corresponding to the types of facial elements in the real image. Finally, a sculpting tool assembles the target facial element sculpting parameters and renders the player character's face (i.e., a virtual face), allowing the player to save their character's facial information and complete the game character sculpting process.

[0074] Specifically, step 1: Standardize the facial elements in advance. Referring to Table 1, according to the standardized classification of each facial element, create facial sculpting parameters that correspond one-to-one with each category of each facial element using the facial sculpting tool (these facial sculpting parameters are used to render game character facial elements similar to the corresponding facial elements using the corresponding facial sculpting tool).

[0075] Step 2: Create a character face element sculpting parameter library by taking the face sculpting parameter set of each facial element obtained from Step 1.

[0076] Step 3: Players upload their own or their favorite facial photos, and a pre-trained facial element classification model identifies the type of each facial element in the uploaded image.

[0077] Step 4: Based on the type of each facial element obtained in Step 3, retrieve the facial element sculpting parameter library for the game character created in Step 2, and determine the facial element sculpting parameters of the target game character that correspond to the real face in the uploaded photo.

[0078] Step 5: Assemble the facial elements of the target game character obtained in Step 4, and then use the face sculpting tool to render and generate the game player character's face.

[0079]

[0080]

[0081] Example 3

[0082] Figure 3 This is a schematic diagram of a virtual face generation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device may include: a face element generation module 310, a face element type determination module 320, a standard face-shaping parameter determination module 330, and a virtual face formation module 340, wherein:

[0083] The facial element generation module 310 acquires a real face image and generates multiple local sub-images corresponding to the real face image. Each local sub-image corresponds to a set facial element on the face.

[0084] The facial element type determination module 320 determines the type of facial elements in each local sub-image based on the image feature set corresponding to each local sub-image and the facial element geometric feature set.

[0085] The standard face-shaping parameter determination module 330 is used to determine the target standard face-shaping parameters corresponding to each face element in the real face image based on the type of the face element in each local sub-image and the preset standard face-shaping parameters corresponding to each type of each face element.

[0086] The virtual face forming module 340 is used to form a virtual face that matches the real face image based on the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0087] The technical solution of this embodiment acquires a real face image and generates multiple local sub-images corresponding to the real face image, each containing facial elements. Based on the image feature set corresponding to each local sub-image and the geometric feature set of the facial elements, the type of the facial elements in each local sub-image is determined. Based on the type of the facial elements in each local sub-image and preset standard face-shaping parameters corresponding to each type of each facial element, target standard face-shaping parameters corresponding to each facial element in the real face image are determined. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, a virtual face matching the real face image is formed. In the process of generating a virtual face, the classification of real face facial elements and face-shaping parameters are comprehensively considered. This solves the problem of inaccurate virtual face generation caused by inaccurate recognition of facial features in real face images in existing technologies, achieving accurate recognition of facial features, accurate classification of facial features, and precise generation of virtual faces.

[0088] Optionally, the above-mentioned device further includes a module for determining associated facial element description information, used to, after acquiring a real face image and generating multiple local sub-images corresponding to the real face image, each local sub-image corresponding to a set facial element on the face:

[0089] Generate a set of geometric features of facial elements corresponding to each local sub-image;

[0090] Based on the geometric feature sets of each facial element, the associated facial element description information corresponding to the real face image is determined. The associated facial element description information includes: the relative positional relationship and / or size ratio relationship between multiple facial elements in the real face image.

[0091] The virtual face formation module 340 can be specifically used to: form a virtual face that matches the real face image based on the target standard face-shaping parameters corresponding to each facial element in the real face image and the description information of the associated facial elements.

[0092] Optionally, in the above-mentioned device, the face element type determination module 320 can be used for:

[0093] Retrieve the target face element corresponding to the currently processed target local subgraph;

[0094] Among all pre-trained facial element classification models, obtain the target facial element classification model that matches the target facial element, and input the target local sub-image into the target facial element classification model;

[0095] The facial element classification model extracts matching image feature sets and facial element geometric feature sets from the target local sub-image, and determines the type of the target facial element in the target local sub-image based on the image feature sets and facial element geometric feature sets.

[0096] Optionally, in the above-mentioned device, the target facial element classification model specifically includes: an image feature extraction network, a facial element geometric feature extraction network, a feature fusion network, and a classification network;

[0097] The image feature extraction network is used to extract image feature sets in multiple dimensions based on the input target local sub-image;

[0098] The facial element geometric feature extraction network is used to extract the facial element geometric feature set corresponding to the target facial element from the input target local sub-image;

[0099] A feature fusion network is used to concatenate the image feature set and the facial element geometric feature set to obtain a fused feature set.

[0100] A classification network is used to determine the type of the target facial elements in the target local sub-image based on the fused feature set.

[0101] Optionally, the associated facial element description information determination module in the above-mentioned device can be used for:

[0102] Each local sub-image is input into a pre-trained facial element geometric feature extraction network to obtain the facial element geometric feature set corresponding to each local sub-image.

[0103] Optionally, the geometric features of the facial elements may include at least one of the following: the scale of the facial elements, the tilt angle, the angle between key points, and the edge curvature.

[0104] Optionally, in the above-mentioned device, the face element generation module 310 can be used for:

[0105] Multiple facial landmarks were annotated in the real face image;

[0106] Based on the set of facial key points corresponding to different facial elements, the target facial key points corresponding to each facial element in the real face image are determined from the annotated facial key points.

[0107] Based on the target facial key points corresponding to each facial element in the real facial image, a local sub-image corresponding to each facial element in the real facial image is formed.

[0108] The virtual face generation apparatus provided in this embodiment of the invention can execute the virtual face generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0109] Example 4

[0110] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 4 A block diagram is shown of an electronic device 412 suitable for implementing embodiments of the present invention. Figure 4 The electronic device 412 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0111] like Figure 4 As shown, electronic device 412 is represented in the form of a general-purpose computing device. The components of electronic device 412 may include, but are not limited to: one or more processors 416, storage device 428, for storing one or more computer programs, and bus 418 connecting different system components (including storage device 428 and processor 416).

[0112] Bus 418 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0113] Electronic device 412 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 412, including volatile and non-volatile media, removable and non-removable media.

[0114] Storage device 428 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 430 and / or cache memory 432. Electronic device 412 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 434 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc-read-only memory (CD-ROM), a digital video disc-read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 418 via one or more data media interfaces. Storage device 428 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0115] A program 436 having at least one set of program modules 426 may be stored in, for example, a storage device 428. Such program modules 426 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 426 typically perform the functions and / or methods described in the embodiments of the present invention.

[0116] Electronic device 412 can also communicate with one or more external devices 414 (e.g., keyboard, pointing device, camera, display 424, etc.), and with one or more devices that enable a user to interact with the electronic device 412, and / or with any device that enables the electronic device 412 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 422. Furthermore, electronic device 412 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN)) and / or public networks, such as the Internet) via network adapter 420. As shown, network adapter 420 communicates with other modules of electronic device 412 via bus 418. It should be understood that, although... Figure 4Not shown, other hardware and / or software modules may be used in conjunction with electronic device 412, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) systems, tape drives, and data backup storage systems.

[0117] The processor 416 executes various functional applications and data processing by running one or more computer programs stored in the storage device 428, such as implementing the virtual face generation method described in any embodiment of the present invention, which may include:

[0118] Acquire a real human face image and generate multiple local sub-images corresponding to the real human face image. Each local sub-image corresponds to a set facial element on the human face.

[0119] Based on the image feature set corresponding to each local sub-image and the geometric feature set of facial elements, determine the type of facial elements in each local sub-image;

[0120] Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of facial element, the target standard face-shaping parameters corresponding to each facial element in the real face image are determined.

[0121] A virtual face matching the real face image is formed based on the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0122] Example 5

[0123] Embodiment 5 of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program is used to perform the virtual face generation method described in any embodiment of the present invention. This method may include:

[0124] Acquire a real human face image and generate multiple local sub-images corresponding to the real human face image. Each local sub-image corresponds to a set facial element on the human face.

[0125] Based on the image feature set corresponding to each local sub-image and the geometric feature set of facial elements, determine the type of facial elements in each local sub-image;

[0126] Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of facial element, the target standard face-shaping parameters corresponding to each facial element in the real face image are determined.

[0127] A virtual face matching the real face image is formed based on the target standard face-shaping parameters corresponding to each facial element in the real face image.

[0128] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0130] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0131] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as Python, MATLAB, C, or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0132] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for generating a virtual face, characterized in that, include: A real face image is acquired, and multiple local sub-images corresponding to the real face image are generated. Each local sub-image corresponds to a set facial element on the face. The local sub-image is a local image of the entire face image to be classified, including a specific facial element. Generate a set of geometric features of facial elements corresponding to each local sub-image; Based on the geometric feature sets of each facial element, the associated facial element description information corresponding to the real face image is determined. The associated facial element description information includes: the relative positional relationship and / or size ratio relationship between multiple facial elements in the real face image. Retrieve the target face element corresponding to the currently processed target local subgraph; Among all pre-trained facial element classification models, a target facial element classification model matching the target facial element is obtained, and the target local sub-image is input into the target facial element classification model; wherein, the facial element classification model is obtained by training at least one round using standardized facial elements; The facial element classification model extracts matching image feature sets and facial element geometric feature sets from the target local sub-image, and determines the type of the target facial element in the target local sub-image based on the image feature sets and facial element geometric feature sets. Based on the type of facial elements in each local sub-image and the preset standard face-shaping parameters corresponding to each type of facial element, the target standard face-shaping parameters corresponding to each facial element in the real face image are determined. Based on the target standard face-shaping parameters corresponding to each facial element in the real face image, and the description information of the associated facial elements, a virtual face matching the real face image is formed. The target facial element classification model specifically includes: an image feature extraction network, a facial element geometric feature extraction network, a feature fusion network, and a classification network; The image feature extraction network is used to extract image feature sets in multiple dimensions based on the input target local sub-image; The facial element geometric feature extraction network is used to extract the facial element geometric feature set corresponding to the target facial element from the input target local sub-image; A feature fusion network is used to concatenate the image feature set and the facial element geometric feature set to obtain a fused feature set. A classification network is used to determine the type of the target facial elements in the target local sub-image based on the fused feature set.

2. The method according to claim 1, characterized in that, Generate a set of geometric features of facial elements corresponding to each local subimage, including: Each local sub-image is input into a pre-trained facial element geometric feature extraction network to obtain the facial element geometric feature set corresponding to each local sub-image.

3. The method according to any one of claims 1-2, characterized in that, The geometric features of the facial elements include at least one of the following: the scale of the facial elements, the tilt angle, the angle between key points, and the edge curvature.

4. The method according to claim 1, characterized in that, Generate multiple local sub-images corresponding to the real face image, including: Multiple facial landmarks were annotated in the real face image; Based on the set of facial key points corresponding to different facial elements, the target facial key points corresponding to each facial element in the real face image are determined from the annotated facial key points. Based on the target facial key points corresponding to each facial element in the real facial image, a local sub-image corresponding to each facial element in the real facial image is formed.

5. A device for generating a virtual human face, characterized in that, include: The facial element generation module acquires a real face image and generates multiple local sub-images corresponding to the real face image. Each local sub-image corresponds to a set facial element on the face. The local sub-image is a local image of the entire face image to be classified, including a specific facial element. The associated facial element description information determination module is used to generate a set of facial element geometric features corresponding to each local sub-image; and to determine the associated facial element description information corresponding to the real face image based on each set of facial element geometric features. The associated facial element description information includes: the relative positional relationship and / or size ratio relationship between multiple facial elements in the real face image. A facial element type determination module is used to obtain the target facial element corresponding to the currently processed target local sub-image; among all pre-trained facial element classification models, a target facial element classification model matching the target facial element is obtained, and the target local sub-image is input into the target facial element classification model; wherein, the facial element classification model is obtained by training at least one round using standardized facial elements; the facial element classification model extracts matching image feature sets and facial element geometric feature sets from the target local sub-image, and determines the type of the target facial element in the target local sub-image based on the image feature sets and facial element geometric feature sets; The standard face-shaping parameter determination module is used to determine the target standard face-shaping parameters corresponding to each face element in the real face image based on the type of the face element in each local sub-image and the preset standard face-shaping parameters corresponding to each type of each face element. The virtual face formation module is used to form a virtual face that matches the real face image based on the target standard face-shaping parameters corresponding to each facial element in the real face image and the description information of the associated facial elements. The target facial element classification model specifically includes: an image feature extraction network, a facial element geometric feature extraction network, a feature fusion network, and a classification network; The image feature extraction network is used to extract image feature sets in multiple dimensions based on the input target local sub-image; The facial element geometric feature extraction network is used to extract the facial element geometric feature set corresponding to the target facial element from the input target local sub-image; A feature fusion network is used to concatenate the image feature set and the facial element geometric feature set to obtain a fused feature set. A classification network is used to determine the type of the target facial elements in the target local sub-image based on the fused feature set.

6. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual face generation method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for generating virtual faces as described in any one of claims 1-4.

Images