Image generation method and device, equipment and computer storage medium

Abstract

The invention belongs to the technical field of intelligent household appliances, and particularly relates to an image generation method and device, equipment and a computer storage medium. According to the method, the face contour and the face image of the image are acquired, and the face image is analyzed and processed, so that the head and face composite image with relatively high similarity with the image is acquired. According to the method, the skin color reconstruction of the head part in the image can be realized, the skin color reconstruction of the neck part in the image can also be realized, the consistency of the face skin color and the neck skin color during reconstruction is further ensured, the image which is the same as the picture uploaded by the user is generated, and the use experience of the user is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of smart home appliances, and specifically relates to an image generation method, device, equipment and computer storage medium. Background Art

[0002] Usually, when reconstructing the head, the 3D reconstruction model only focuses on the reconstruction of the head part, but not the reconstruction of the torso part. Therefore, when reconstructing the input photo, there will be a problem of inconsistency between the skin color of the face and the skin color of the torso such as the neck.

[0003] For example, suppose there is a photo of a person's whole body. When people choose to use a 3D reconstruction model to analyze and process the photo, the 3D reconstruction model will pay more attention to the reconstruction of the head part in the photo and generate the same skin color as the head part in the photo. For the torso part of the human body in the photo, the 3D reconstruction model will choose to use the existing bone model to determine the model of the torso and choose to use the skin color in the existing torso model. Therefore, the result produced in this way may lead to inconsistent overall skin color.

[0004] Therefore, the current problem that needs to be solved is how to ensure that the facial skin color and the neck skin color are consistent during reconstruction, so as to generate images similar to the photos uploaded by the user, thereby improving the user experience. Summary of the invention

[0005] The present application provides an image generation method, apparatus, device and computer storage medium, which can not only realize the reconstruction of the skin color of the head part in the image, but also realize the reconstruction of the skin color of the neck part in the image, thereby ensuring the consistency of the facial skin color and the skin color of the neck part during reconstruction, thereby generating an image identical to the photo uploaded by the user and improving the user experience.

[0006] In a first aspect, the present application provides an image generation method, comprising:

[0007] Acquire a three-dimensional facial model of an image and a facial image, wherein the three-dimensional facial model is used to indicate a facial contour in the image;

[0008] Determining, based on the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image;

[0009] generating, according to the first skin color, a target head and neck model matching the first skin color;

[0010] A head-face composite image corresponding to the image is generated according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model.

[0011] Optionally, generating a target head and neck model matching the first skin color according to the first skin color includes:

[0012] Acquire a plurality of candidate head and neck models, wherein the plurality of candidate head and neck models have different head and neck skin colors;

[0013] Using face masking technology, obtaining a skin color filling value of a second skin color from the first skin color, where the second skin color is the skin color corresponding to the left face area and the right face area in the facial image;

[0014] Determine a target head and neck skin color according to the second skin color and the head and neck skin colors corresponding to the multiple candidate head and neck models;

[0015] determining a target head and neck texture according to the target head and neck skin color and the second skin color;

[0016] According to the target head and neck texture, a candidate head and neck model corresponding to the target head and neck skin color is filled to obtain a target head and neck model.

[0017] Optionally, generating a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model includes:

[0018] Determining a side face texture of the facial image according to the first texture;

[0019] generating a facial composite image according to the side face texture, the first texture and the first skin color;

[0020] The face composite image, the three-dimensional face model and the target head and neck model are combined to generate the head and face composite image.

[0021] Optionally, determining the side face texture of the facial image according to the first texture includes:

[0022] Determine, based on the first texture, a second texture corresponding to the front face area and a third texture corresponding to the side face area;

[0023] The third texture is repaired according to the second texture to obtain a side face texture.

[0024] Optionally, the three-dimensional facial model and facial image of the acquired image include:

[0025] Acquire the image uploaded by the user, and detect a face offset value of the image, where the face offset value is used to indicate the orientation and angle of the face in the image;

[0026] Determining whether the face offset value is within a preset offset range;

[0027] When the facial offset value is within the preset offset range, a three-dimensional facial model and a facial image of the image are acquired.

[0028] Optionally, the method for determining whether the face offset value is within a preset offset range further includes:

[0029] When the facial offset value is not within the preset offset range, a GAN network model is used to perform a restoration process on the posture of the image to obtain a target image, wherein the facial offset value of the target image is within the preset offset range;

[0030] A three-dimensional facial model and a facial image of the target image are obtained.

[0031] Optionally, after generating a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model, the method further includes:

[0032] Get the synthetic torso request sent by the user;

[0033] Performing restoration processing on a torso image in the image according to the target head and neck texture and the image to obtain a target torso image;

[0034] A whole-body composite image corresponding to the image is generated according to the head-face composite image and the target torso image.

[0035] In a second aspect, the present application provides an image generating device, comprising:

[0036] An acquisition module, used to acquire a three-dimensional facial model of an image and a facial image, wherein the three-dimensional facial model is used to indicate a facial contour in the image;

[0037] a determination module, configured to determine, based on the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image;

[0038] A generating module, configured to generate, according to the first skin color, a target head and neck model matching the first skin color;

[0039] The generation module is further used to generate a head-face composite image corresponding to the image based on the target head and neck model, the first texture, the first skin color and the three-dimensional face model.

[0040] Optionally, the acquisition module is further used to acquire a plurality of candidate head and neck models, wherein the plurality of candidate head and neck models have different head and neck skin colors;

[0041] The acquisition module is further used to acquire a skin color filling value of a second skin color from the first skin color by using a face masking technology, where the second skin color is the skin color corresponding to the left face area and the right face area in the facial image;

[0042] The determination module is further configured to determine a target head and neck skin color according to the second skin color and the head and neck skin colors corresponding to the plurality of candidate head and neck models;

[0043] The determination module is further used to determine the target head and neck texture according to the target head and neck skin color and the second skin color;

[0044] The device further comprises: a processing module;

[0045] The processing module is used to fill the candidate head and neck model corresponding to the target head and neck skin color according to the target head and neck texture;

[0046] The determination module is specifically used to obtain the target head and neck model after the candidate head and neck model corresponding to the target head and neck skin color is filled.

[0047] Optionally, the determination module is further used to determine the side face texture of the facial image according to the first texture;

[0048] The generating module is further used to generate a facial composite image according to the side face texture, the first texture and the first skin color;

[0049] The generating module is specifically used to generate the head-face composite image by combining the composite face image, the three-dimensional face model and the target head and neck model.

[0050] Optionally, the determination module is further used to determine, based on the first texture, a second texture corresponding to the front face area and a third texture corresponding to the side face area;

[0051] The processing module is specifically used to perform repair processing on the third texture according to the second texture to obtain a side face texture.

[0052] Optionally, the acquisition module is also used for the image uploaded by the user;

[0053] The device also includes: a detection module;

[0054] The detection module is used to detect a face offset value of the image, where the face offset value is used to indicate the orientation and angle of the face in the image;

[0055] The device further comprises: a judgment module;

[0056] The judging module is used to judge whether the face offset value is within a preset offset range;

[0057] The acquisition module is specifically configured to acquire the three-dimensional facial model and the facial image of the image when the facial offset value is within the preset offset range.

[0058] Optionally, the processing module is further configured to use a GAN network model to perform a restoration process on the posture of the image when the facial offset value is not within the preset offset range;

[0059] The determination module is further configured to obtain a target image after performing a posture restoration process on the image using a GAN network model, wherein the facial offset value of the target image is within the preset offset range;

[0060] The acquisition module is specifically used to acquire the three-dimensional facial model and facial image of the target image.

[0061] Optionally, the acquisition module is further used to acquire a synthetic torso request sent by a user;

[0062] The processing module is further used to perform repair processing on the torso image in the image according to the target head and neck texture and the image, so as to obtain a target torso image;

[0063] The generation module is further used to generate a full-body composite image corresponding to the image based on the head-face composite image and the target torso image.

[0064] In a third aspect, the present application provides an image generating device, comprising:

[0065] Memory;

[0066] processor;

[0067] Wherein, the memory stores computer-executable instructions;

[0068] The processor executes the computer-executable instructions stored in the memory to implement the image generating method as described in the first aspect and various possible implementations of the first aspect.

[0069] In a fourth aspect, the present application provides a computer storage medium having computer execution instructions stored thereon, wherein the computer execution instructions are executed by a processor to implement the image generation method as described in the first aspect and various possible implementations of the first aspect.

[0070] The image generation method provided by the present application obtains the facial contour and facial image of the image, and analyzes and processes the facial image, thereby obtaining a head-face composite image with a high degree of similarity to the image. This method can not only reconstruct the skin color of the head part in the image, but also reconstruct the skin color of the neck part in the image, thereby ensuring the consistency of the facial skin color and the skin color of the neck part during reconstruction, thereby generating an image that is the same as the photo uploaded by the user and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0072] Figure 1 The process of the image generation method provided in this application Figure 1 ;

[0073] Figure 2 The process of the image generation method provided in this application Figure 2 ;

[0074] Figure 3 The process of the image generation method provided in this application Figure 3 ;

[0075] Figure 4 It is a structural schematic diagram of the image generating device provided by the present application;

[0076] Figure 5 It is a structural schematic diagram of the image generating device provided by this application.

[0077] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0079] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein, for example.

[0080] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0081] Usually, when using a 3D reconstruction model to reconstruct an image, the 3D reconstruction model pays more attention to the reconstruction of the head part and relatively ignores the reconstruction of the torso part. This situation leads to a common problem that when the input photo is reconstructed in 3D, there is a discrepancy between the skin color of the face and the skin color of the torso parts such as the neck.

[0082] For example, suppose there is a photo of a person's whole body. When people choose to use a 3D reconstruction model to analyze and process the photo, the 3D reconstruction model will pay more attention to the reconstruction of the head part in the photo and generate the same skin color as the head part in the photo. For the torso part of the human body in the photo, the 3D reconstruction model will choose to use the existing bone model to determine the model of the torso and choose to use the skin color in the existing torso model. Therefore, the result produced in this way may lead to inconsistent overall skin color.

[0083] Therefore, how to ensure that the facial skin color and the neck skin color are consistent during reconstruction, and then generate an image similar to the photo uploaded by the user, thereby avoiding defects that affect user use, is the problem that needs to be solved at present.

[0084] In view of the above problems, the present application provides an image generation method. The method obtains the facial contour and facial image of the image, and analyzes and processes the facial image, thereby obtaining a head-face composite image with a high similarity to the image. The method can not only reconstruct the skin color of the head part in the image, but also reconstruct the skin color of the neck part in the image, thereby ensuring the consistency of the facial skin color and the neck skin color during reconstruction, thereby generating an image identical to the photo uploaded by the user and improving the user experience.

[0085] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0086] Figure 1 The process of the image generation method provided in the embodiment of the present application Figure 1 .like Figure 1 The image generation method provided in this embodiment includes:

[0087] S101: Acquire a three-dimensional facial model of an image and a facial image, wherein the three-dimensional facial model is used to indicate a facial contour in the image.

[0088] The three-dimensional facial model is used to indicate the facial contour in the image; and the facial image is used to indicate the facial posture in the image.

[0089] The method of obtaining the three-dimensional facial model and the facial image of the image can be, for example, obtained through a DECA model, a Pix2Pix model, or a Neural Style Transfer model. This application is not limited to this. Here, the process of obtaining the three-dimensional facial model and the facial image of the image using the DECA model is described in detail.

[0090] The DECA model is a deep learning model for generating high-quality 3D facial models, which can obtain a 3D facial model identical to the facial image as well as the facial image from a 2D facial image.

[0091] The DECA model first detects and locates key points of 2D facial images. It uses a face detection algorithm to detect the position of the face in the input image, and uses a key point location algorithm to obtain the exact position of key points in the facial image, such as eyes, nose, mouth, etc.; secondly, based on the accurate position of the key points, a neural network is used to generate a three-dimensional facial shape model; then, the neural network is used to obtain the texture information of the face and generate a facial image similar to the image; finally, an iterative optimization method is used to adjust the parameters of the model so that the generated three-dimensional facial model can better fit the input 2D facial image, thereby obtaining a more accurate three-dimensional facial model and facial image.

[0092] Optionally, in order to ensure that the brightness corresponding to the left face area and the brightness corresponding to the right face area in the image are evenly distributed, before obtaining the three-dimensional facial model and facial image of the image, the parameters of the two-dimensional gamma function can be adjusted using the distribution characteristics of the illumination components to adjust the brightness of the left face area and the brightness of the right face area.

[0093] It can be understood that, first, the distribution characteristics of the illumination components can be used to analyze the brightness of the left and right face areas in the image; secondly, the parameters of the two-dimensional gamma function can be adjusted according to the analysis results; finally, the adjusted two-dimensional gamma function is applied to the image to achieve uniform distribution of the brightness of the left and right face areas in the image.

[0094] S102: Determine, according to the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image.

[0095] The first texture is used to indicate textures corresponding to multiple regions in the facial image, and the first skin color is used to indicate facial skin color corresponding to the facial image.

[0096] It can be understood that the multiple regions of the facial image can be divided into, for example, a forehead region, a cheekbone region, an area around the eyes, a nose and nose wing region, and an area around the lips.

[0097] The forehead area is generally a large flat area, and its skin texture is usually finer and denser; the cheekbone area is generally located above the cheek, and it usually has diagonal wrinkles or lines due to facial muscle movement and gravity; the skin texture of the area around the eyes is usually thin and delicate, and there may be fine lines or wrinkles, especially when the skin around the eyes ages; the nose and nose wing area usually has fine lines along the horizontal nose line and around the nose wing, and these lines sometimes deepen, such as due to enlarged pores caused by excessive oil secretion; and the area around the lips may have lines such as corner mouth lines and lip lines.

[0098] In this step, multiple areas of the facial image and a first skin color of the facial image can be determined based on the facial image, and then first textures corresponding to the multiple areas of the facial image can be determined based on the multiple areas of the facial image.

[0099] For example, when the area of ​​the facial image is determined to be the cheekbone area, the corresponding texture is a diagonal texture.

[0100] S103: Generate a target head and neck model matching the first skin color according to the first skin color.

[0101] The target head and neck model is used to indicate a head and neck model that matches the facial skin color in the facial image.

[0102] In this step, a target head and neck model matching the facial skin color in the facial image may be generated according to the facial skin color corresponding to the facial image.

[0103] It can be understood that there is a corresponding relationship between the skin color of the target head and neck model and the facial skin color of the facial image. After the facial skin color in the facial image is determined, it is necessary to generate a target head and neck model that matches the facial skin color based on the facial skin color.

[0104] For example, suppose there is a facial image in which the facial skin color of the person in the image is brown. Then the formula is used to generate a head and neck model that matches the skin color in the facial image, that is, the skin color of the head and neck model is also brown.

[0105] S104: Generate a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model.

[0106] Among them, the head and face composite image is used to present the three-dimensional effect of the image.

[0107] In this step, a head-face composite image corresponding to the image can be generated based on a head and neck model that matches the facial skin color in the facial image, textures corresponding to multiple areas in the facial image, facial skin color corresponding to the facial image, and facial contours in the image.

[0108] It can be understood that in order to make the facial skin color and neck skin color in the generated stereoscopic image consistent, it is necessary to jointly generate a head and face composite image corresponding to the image based on the target head and neck model, the first texture, the first skin color and the three-dimensional facial model.

[0109] The image generation method provided in this embodiment obtains the three-dimensional facial model and facial image of the image, and performs analysis processing on the facial image to obtain the first texture and the first skin color corresponding to multiple regions of the facial image, and then obtains the target head and neck model with a high degree of matching with the first skin color according to the first skin color, and then generates the head and face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color and the three-dimensional facial model. This method avoids the defect of inconsistency between the facial skin color and the neck skin color during reconstruction, generates an image identical to the photo uploaded by the user, and then ensures the use of the user, thereby improving the user experience.

[0110] Figure 2 The process of the image generation method provided in the embodiment of the present application Figure 2 .like Figure 2 As shown, this embodiment is Figure 1 Based on the embodiment, the image generation method is described in detail. The image generation method provided in this embodiment includes:

[0111] S201: Acquire an image uploaded by a user, and detect a facial offset value of the image.

[0112] The method of acquiring the image uploaded by the user may be, for example, acquiring the image from a photographing device associated with the 3D reconstruction model, or acquiring the image from an application associated with the 3D reconstruction model. This application does not limit this.

[0113] It is understandable that after obtaining the image uploaded by the user, the image uploaded by the user can be analyzed and processed to avoid deformed defects in the side face during reconstruction, which would lead to poor reconstruction effect.

[0114] In this step, for example, face recognition technology can be used to analyze and process the image uploaded by the user to obtain a facial offset value corresponding to the image. The specific implementation process may include, for example: the face recognition technology first pre-processes the image uploaded by the user, including image size adjustment, contrast enhancement and other operations; then a face detection algorithm is used to detect the position and area of ​​the face from the image uploaded by the user, and based on the face detection result, a face alignment operation is performed to align the face to a standard position and posture to reduce the influence of facial expressions and postures; secondly, a face recognition algorithm is used to extract a facial offset value from the aligned face image; finally, a facial offset reference model is used to compare the facial offset value extracted from the image uploaded by the user with the model, so as to obtain a facial offset value corresponding to the image, and this offset value may include an offset of the facial contour, an offset of the eye position, an offset of the mouth shape, etc.

[0115] S202: Determine whether the face offset value is within a preset offset range, if so, execute step S203, if not, execute step S204.

[0116] The preset offset range may be, for example, an offset range pre-set in face recognition technology, or an offset range set according to an actual scenario. The present application does not limit the preset offset range.

[0117] The face offset value is used to indicate the orientation and angle of the face in the face image.

[0118] The purpose of judging whether the facial offset value is within the preset offset range is to determine whether the facial posture in the image has defects.

[0119] If the facial offset value is within the preset offset range, it indicates that the facial posture in the image has no flaws, and therefore a three-dimensional facial model and a facial image of the image are obtained.

[0120] If the facial offset value is not within the preset offset range, it indicates that the facial posture in the image is flawed. Therefore, it is necessary to use a GAN network model to repair the posture of the image to obtain a target image.

[0121] For example, the maximum offset value of the pre-set facial image may be 5 pixels in the vertical direction and 3 pixels in the horizontal direction, while the facial image uploaded by the user has an offset value of 2 pixels in the vertical direction and 1 pixel in the horizontal direction. Therefore, a three-dimensional facial model and a facial image of the facial image may be obtained.

[0122] S203: Acquire a three-dimensional facial model and a facial image of the image.

[0123] Step S203 is similar to the above step S201 and will not be described again.

[0124] S204: Using a GAN network model to perform a posture restoration process on the image to obtain a target image.

[0125] Among them, the GAN network model is a deep learning model, whose Chinese name is Generative Adversarial Network, which consists of a generator and a discriminator. The generator is a network used to generate new samples, which receives a random noise vector as input and generates samples similar to the training data through a series of operations; the discriminator is a network used to classify samples, which receives samples from the generator and real training data and classifies them to distinguish between real samples and generated samples as accurately as possible.

[0126] The target image is used to represent the highest similarity with the facial image, and the facial offset value of the target image is within a preset offset range.

[0127] In this step, when it is determined that the facial image has defects, the GAN network model can be used to repair the posture of the facial image to obtain a target image with the highest similarity to the facial image.

[0128] It can be understood that the specific implementation steps of using the GAN network model to repair the posture of a facial image are: first, obtain a facial image with posture defects; then use the generator to repair the facial image so that it looks like it has a normal posture; secondly, use the discriminator to determine whether the generated image is a real image or a repaired image; finally, by repeatedly training the generator and the discriminator, the GAN network model can gradually learn the rules of posture repair and generate high-quality repaired images.

[0129] Optionally, after obtaining the target image, the 3D facial model and facial image of the target image may be obtained. Since the facial image is obtained after performing a posture restoration process on the image, the 3D facial model and facial image of the image may also be obtained.

[0130] S205: Determine, according to the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image.

[0131] Step S205 is similar to the above step S102 and will not be described again.

[0132] S206: Generate a target head and neck model matching the first skin color according to the first skin color.

[0133] Step S206 is similar to the above step S103 and will not be described again.

[0134] S207: Determine the side face texture of the facial image according to the first texture.

[0135] Different facial images correspond to different side face textures. After the first texture of the facial image is determined, the side face texture of the facial image can be determined based on the first texture.

[0136] It can be understood that after the first texture of the facial image, a more accurate side face texture can be determined for the facial image based on the first texture to ensure that the facial composite image is more accurate.

[0137] Specifically, the specific implementation method of determining the side face texture of the facial image based on the first texture may be, for example: determining the second texture corresponding to the front face area and the third texture corresponding to the side face area based on the first texture; and repairing the third texture based on the second texture to obtain the side face texture.

[0138] The second texture is used to indicate the texture corresponding to the frontal face area in the facial image, and the third texture is used to indicate the texture corresponding to the side face area in the facial image.

[0139] The first texture is associated with the second texture and the third texture. After the first texture is determined, the second texture and the third texture can be determined based on the first texture.

[0140] In this step, the second texture and the third texture corresponding to the first texture may be determined according to the first texture, and then the third texture may be repaired according to the second texture to ensure that the third texture is the same as the second texture.

[0141] It can be understood that the specific implementation process of repairing the third texture based on the second texture may include, for example, first, using an image enhancement algorithm to amplify the acquired front face texture to make it clearer and richer; secondly, using an edge diffusion method to diffuse the front face texture along the measured edge to achieve the repair of the side face texture; finally, smoothing the repaired side face texture to make the transition of the side face texture more natural.

[0142] S208: Generate a facial composite image according to the side face texture, the first texture and the first skin color.

[0143] The facial composite image is used to indicate that the facial composite image has the same texture and skin color as the facial image.

[0144] It can be understood that an image similar to the facial image can be generated based on the side face texture, the first texture and the first skin color, and the texture and skin color of the image are the same as the texture and skin color of the facial image.

[0145] In this step, the side face texture may be repaired based on the first texture, and then an image similar to the facial image may be generated based on the first texture, the side face texture and the first skin color.

[0146] S209: Generate the head-face composite image by combining the composite face image, the three-dimensional face model and the target head and neck model.

[0147] Before generating the head-face composite image, the face composite image, the three-dimensional face model and the target head and neck model may be determined first to ensure that the generated stereoscopic image is more similar to the portrait in the image.

[0148] It can be understood that after generating a composite head and face image, it can be determined that the composite head and face image includes not only a facial image but also a neck image, which not only ensures the consistency of the facial skin color and the neck skin color during reconstruction, but also improves the user experience.

[0149] S210: Obtain a synthetic torso request sent by a user.

[0150] The request for synthesizing the torso refers to applying the acquired facial skin color to the torso.

[0151] The method of obtaining the synthetic torso request sent by the user may be, for example, obtaining the request through an application associated with the three-dimensional reconstruction model.

[0152] The purpose of obtaining the synthetic torso request sent by the user is to make the skin color of the generated torso image consistent with that of the head and face image, so as to improve the user experience.

[0153] S211: Performing restoration processing on a torso image in the image according to the target head and neck texture and the image to obtain a target torso image.

[0154] The target torso image is used to indicate that the skin color is the same as that of the head and face composite image.

[0155] In this step, after the target head and neck texture is determined, the torso image in the image can be repaired according to the target head and neck texture.

[0156] It is understandable that in order to ensure that the skin color difference of the generated full-body image is not too big, the torso image in the image may be repaired according to the target head and neck texture, thereby obtaining the target torso image.

[0157] For example, when a user stands in the sun, the ultraviolet rays in the sun will stimulate the melanocytes in the skin to produce melanin, which makes the skin darker. Since the user's face is usually more exposed to the sun, the face is easily affected by ultraviolet rays and darkens. In contrast, other parts of the body may be covered by clothing, hats or other shielding objects, thereby reducing sunlight exposure and making the body skin color relatively lighter.

[0158] Therefore, after the target head and neck texture is determined, the torso image can be repaired according to the target head and neck texture to ensure that the skin color difference of the generated full-body image is not too large.

[0159] S212: Generate a full-body composite image corresponding to the image based on the head-face composite image and the target torso image.

[0160] Among them, the full-body synthetic image is used to represent that the facial skin color corresponding to the generated image is consistent with the torso skin color.

[0161] It can be understood that after the head-face composite image is determined, a target torso image can be generated based on the head-face image, and then a corresponding full-body composite image can be generated based on the head-face composite image and the target torso image.

[0162] The image generation method provided by the present application obtains a three-dimensional facial model and a facial image of the image when the facial offset value of the image uploaded by the user meets the preset offset value range, and analyzes and processes the facial image to obtain a first texture and a first skin color corresponding to the facial image, and then obtains the side face texture of the facial image, thereby generating a head-face composite image corresponding to the image. This method can not only realize the reconstruction of the skin color of the head part in the image, but also realize the reconstruction of the skin color of the neck part in the image, thereby ensuring the consistency of the facial skin color and the skin color of the neck part during reconstruction, thereby generating an image identical to the photo uploaded by the user and improving the user experience.

[0163] Figure 3 The process of the image generation method provided in the embodiment of the present application Figure 3 .like Figure 3 As shown, this embodiment is Figure 2 Based on the embodiment, generating a target head and neck model matching the first skin color according to the first skin color is described in detail. The image generating method provided in this embodiment includes:

[0164] S301: Acquire multiple candidate head and neck models, where the multiple candidate head and neck models have different head and neck skin colors.

[0165] The method of obtaining the multiple candidate head and neck models may be, for example, obtaining them from a data storage in the three-dimensional reconstruction model, or obtaining them from an application associated with the three-dimensional reconstruction model. This application does not limit this.

[0166] It is understandable that there is an association between the multiple candidate head and neck models and the head and neck skin colors corresponding to the multiple candidate head and neck models. After the candidate head and neck model is determined, the head and neck skin color matching the candidate head and neck model can be obtained.

[0167] The execution time of this step is not limited to before step S301, and it can also be executed after step S302. The specific execution time depends on the actual situation.

[0168] S302: using face masking technology to obtain a skin color filling value of a second skin color from the first skin color.

[0169] The second skin color is the skin color corresponding to the left face area and the right face area in the facial image.

[0170] Face masking technology is a technology used to process and analyze facial images.

[0171] It can be understood that in the face mask, for example, the pixel values ​​of the left and right local areas in the facial image can be set to 0, and the pixel values ​​of other areas can be set to 1, so as to obtain the skin color filling values ​​of the left and right local areas.

[0172] In this step, for example, face masking technology can be used to determine the skin colors corresponding to the left and right face areas in the facial image from the facial image, so that the determined target head and neck skin color is consistent with the skin colors corresponding to the left and right face areas.

[0173] S303: Determine a target head and neck skin color according to the second skin color and the head and neck skin colors corresponding to the multiple candidate head and neck models.

[0174] The target head and neck skin color is used to represent the consistency of the skin color corresponding to the left face area and the right face area.

[0175] After determining the head and neck skin colors corresponding to multiple candidate head and neck models, the multiple head and neck skin colors can be matched with the skin colors of the left and right local areas in the facial image to further determine the head and neck skin colors similar to the skin colors of the left and right local areas in the facial image.

[0176] It can be understood that the skin colors of the head and neck corresponding to the multiple candidate models are associated with the skin colors of the left and right local areas in the facial image. After the skin colors of the left and right local areas in the facial image are determined, the skin colors of the head and neck corresponding to the left and right local areas can be matched from the multiple candidate models according to the skin colors of the left and right local areas, which is equivalent to determining the candidate head and neck models.

[0177] S304: Determine a target head and neck texture according to the target head and neck skin color and the second skin color.

[0178] The target head and neck texture is used to indicate the similarity with the texture corresponding to the left and right local areas in the facial image.

[0179] The target head and neck texture of the candidate head and neck model may be jointly determined based on the target head and neck skin color and the skin colors of the left and right local areas in the facial image.

[0180] It can be understood that in order to make the texture of the candidate head and neck model consistent with the texture in the facial image, it is necessary to determine the target head and neck texture of the candidate head and neck model based on the target head and neck skin color and the second skin color.

[0181] Specifically, according to the target head and neck skin color and the second skin color, the specific implementation method of determining the target head and neck texture can be determined by, for example, the following formula: U head =qI m +(1-q)U m

[0182] Among them, U head is the target head and neck texture, q is the weight coefficient, and q is a constant, I m is the skin color of the left and right local areas in the facial image, U m Targets head and neck skin tone.

[0183] S305: Filling the candidate head and neck model corresponding to the target head and neck skin color according to the target head and neck texture to obtain the target head and neck model.

[0184] Among them, after the target head and neck texture is determined, the candidate head and neck model corresponding to the target head and neck skin color can be filled according to the target head and neck texture to obtain the target head and neck model.

[0185] It can be understood that in order to make the facial texture of the target head and neck model consistent with the facial texture of the facial image, it is necessary to fill the candidate head and neck model corresponding to the target head and neck skin color after the target head and neck texture is determined, so as to obtain the target head and neck model.

[0186] The image generation method provided by the present application, after obtaining multiple candidate head and neck models, uses face masking technology to obtain the skin color corresponding to the left and right face areas in the facial image, and determines the target head and neck skin color with the highest similarity to the skin color from the head and neck skin colors corresponding to the multiple candidate head and neck models, and then determines the target head and neck texture, and fills the target head and neck texture into the candidate head and neck model corresponding to the target head and neck skin color, thereby obtaining the target head and neck model. This method not only ensures the consistency of the reconstructed neck skin color with the skin color in the image, but also improves the user experience.

[0187] Figure 4 is a schematic diagram of the structure of the image generation device provided by this application. Figure 4 As shown, the present application provides an image generating device, the image generating device 400 includes:

[0188] An acquisition module 401 is used to acquire a three-dimensional face model of an image and a face image, wherein the three-dimensional face model is used to indicate a face contour in the image;

[0189] A determination module 402, configured to determine, based on the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image;

[0190] A generating module 403 is used to generate a target head and neck model matching the first skin color according to the first skin color;

[0191] The generating module 403 is further configured to generate a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color and the three-dimensional face model.

[0192] Optionally, the acquisition module 401 is further used to acquire multiple candidate head and neck models, where the multiple candidate head and neck models have different head and neck skin colors;

[0193] The acquisition module 401 is further used to acquire a skin color filling value of a second skin color from the first skin color by using a face masking technology, where the second skin color is the skin color corresponding to the left face area and the right face area in the facial image;

[0194] The determination module 402 is further configured to determine a target head and neck skin color according to the second skin color and the head and neck skin colors corresponding to the plurality of candidate head and neck models;

[0195] The determination module 402 is further configured to determine a target head and neck texture according to the target head and neck skin color and the second skin color;

[0196] The device further includes: a processing module 404;

[0197] The processing module 404 is used to fill the candidate head and neck model corresponding to the target head and neck skin color according to the target head and neck texture;

[0198] The determination module 402 is specifically used to obtain the target head and neck model after the candidate head and neck model corresponding to the target head and neck skin color is filled.

[0199] Optionally, the determination module 402 is further configured to determine a side face texture of the facial image according to the first texture;

[0200] The generating module 403 is further configured to generate a composite facial image according to the side face texture, the first texture and the first skin color;

[0201] The generating module 403 is specifically configured to generate the head-face composite image by combining the composite facial image, the three-dimensional facial model and the target head and neck model.

[0202] Optionally, the determination module 402 is further configured to determine, based on the first texture, a second texture corresponding to the front face region and a third texture corresponding to the side face region;

[0203] The processing module 404 is specifically configured to perform repair processing on the third texture according to the second texture to obtain a side face texture.

[0204] Optionally, the acquisition module 401 is also used for the image uploaded by the user;

[0205] The device further comprises: a detection module 405;

[0206] The detection module 405 is used to detect a face offset value of the image, where the face offset value is used to indicate the direction and angle of the face in the image;

[0207] The device further includes: a determination module 406;

[0208] The judging module 406 is used to judge whether the face offset value is within a preset offset range;

[0209] The acquisition module 401 is specifically configured to acquire the three-dimensional facial model and the facial image of the image when the facial offset value is within the preset offset range.

[0210] Optionally, the processing module 404 is further configured to use a GAN network model to perform a restoration process on the posture of the image when the facial offset value is not within the preset offset range;

[0211] The determination module 402 is further configured to obtain a target image after performing a posture restoration process on the image using a GAN network model, wherein the facial offset value of the target image is within the preset offset range;

[0212] The acquisition module 401 is specifically used to acquire the three-dimensional facial model and facial image of the target image.

[0213] Optionally, the acquisition module 401 is further used to acquire a synthetic torso request sent by a user;

[0214] The processing module 404 is further used to perform restoration processing on the torso image in the image according to the target head and neck texture and the image, so as to obtain a target torso image;

[0215] The generating module 403 is further configured to generate a full-body composite image corresponding to the image according to the composite head-face image and the target torso image.

[0216] Figure 5 Schematic diagram of the structure of the image generation device provided by this application. Figure 5 As shown, the present application provides an image generating device, and the image generating device 500 includes: a receiver 501, a transmitter 502, a processor 503 and a memory 504.

[0217] Receiver 501, used for receiving instructions and data;

[0218] A transmitter 502, used for sending instructions and data;

[0219] Memory 504, used to store computer-executable instructions;

[0220] The processor 503 is used to execute the computer-executable instructions stored in the memory 504 to implement the various steps performed by the image generation method in the above embodiment. For details, please refer to the relevant description in the above-mentioned image generation method embodiment.

[0221] Optionally, the memory 504 may be independent or integrated with the processor 403 .

[0222] When the memory 504 is independently provided, the electronic device further includes a bus for connecting the memory 504 and the processor 503 .

[0223] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image generation method performed by the above-mentioned image generation device is implemented.

[0224] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0225] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image generation method, characterized in that: include: Acquire a three-dimensional facial model of an image and a facial image, wherein the three-dimensional facial model is used to indicate a facial contour in the image; Determining, based on the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image; generating, according to the first skin color, a target head and neck model matching the first skin color; A head-face composite image corresponding to the image is generated according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model.

2. The method according to claim 1, characterized in that The step of generating a target head and neck model matching the first skin color according to the first skin color comprises: Acquire a plurality of candidate head and neck models, wherein the plurality of candidate head and neck models have different head and neck skin colors; Using face masking technology, obtaining a skin color filling value of a second skin color from the first skin color, where the second skin color is the skin color corresponding to the left face area and the right face area in the facial image; Determine a target head and neck skin color according to the second skin color and the head and neck skin colors corresponding to the multiple candidate head and neck models; determining a target head and neck texture according to the target head and neck skin color and the second skin color; According to the target head and neck texture, a candidate head and neck model corresponding to the target head and neck skin color is filled to obtain a target head and neck model.

3. The method according to claim 1, characterized in that The step of generating a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model includes: Determining a side face texture of the facial image according to the first texture; generating a facial composite image according to the side face texture, the first texture and the first skin color; The face composite image, the three-dimensional face model and the target head and neck model are combined to generate the head and face composite image.

4. The method according to claim 3, characterized in that The step of determining the side face texture of the facial image according to the first texture includes: Determine, based on the first texture, a second texture corresponding to the front face area and a third texture corresponding to the side face area; The third texture is repaired according to the second texture to obtain a side face texture.

5. The method according to claim 1, characterized in that The three-dimensional facial model and facial image of the acquired image include: Acquire the image uploaded by the user, and detect a face offset value of the image, where the face offset value is used to indicate the orientation and angle of the face in the image; Determining whether the face offset value is within a preset offset range; When the facial offset value is within the preset offset range, a three-dimensional facial model and a facial image of the image are acquired.

6. The method according to claim 5, characterized in that The method further comprises: When the facial offset value is not within the preset offset range, a GAN network model is used to perform a restoration process on the posture of the image to obtain a target image, wherein the facial offset value of the target image is within the preset offset range; A three-dimensional facial model and a facial image of the target image are obtained.

7. The method according to claim 2, characterized in that After generating a head-face composite image corresponding to the image according to the target head and neck model, the first texture, the first skin color, and the three-dimensional face model, the method further includes: Get the synthetic torso request sent by the user; Performing restoration processing on a torso image in the image according to the target head and neck texture and the image to obtain a target torso image; A whole-body composite image corresponding to the image is generated according to the head-face composite image and the target torso image.

8. An image generating device, characterized in that: include: An acquisition module, used to acquire a three-dimensional facial model of an image and a facial image, wherein the three-dimensional facial model is used to indicate a facial contour in the image; a determination module, configured to determine, based on the facial image, a first texture and a first skin color corresponding to a plurality of regions of the facial image; A generating module, configured to generate, according to the first skin color, a target head and neck model matching the first skin color; The generation module is further used to generate a head-face composite image corresponding to the image based on the target head and neck model, the first texture, the first skin color and the three-dimensional face model.

9. An image generating device, characterized in that: include: Memory; processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the image generation method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are used to implement the image generation method according to any one of claims 1 to 7 when executed by a processor.

Images