Method, device and equipment for reconstructing and repairing human face skin color in image and medium

Abstract

The invention relates to the technical field of image processing and financial science and technology, and discloses a face skin color reconstruction and restoration method, device, equipment and medium in an image, and the method comprises the steps: obtaining a face video of a pre-selected model, extracting face images in the face video, obtaining a face image set, and storing the face image set in a database; identifying face key points of each image in the face image set, performing face alignment on the images in the face image set according to the face key points to obtain a pre-processed face image, performing face reconstruction on the pre-processed face image by using a pre-trained face reconstruction model to obtain a reconstructed face image, and performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image, and performing color weighted mean fusion on the converted face image and the converted reconstructed image based on a color channel to obtain a target face skin color.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology and financial technology, and in particular to a method, device, equipment and medium for reconstructing and restoring the skin color of a face in an image. Background Art

[0002] Digital human is the product of the integration of information science and life science. It uses information science methods to simulate the human body at different levels of form and function. It is suitable for business scenarios such as digital human live streaming and anchor face reconstruction. In addition, it is also used in the field of virtual customer service, for example, setting up a virtual financial service consulting customer service in financial software. With the continuous advancement of science and technology, people have higher and higher requirements for the texture and skin color accuracy of face models.

[0003] However, in current digital human applications, the facial color of real people or digital people in live broadcast reconstruction is very different from that of the real person, resulting in poor synthesis effect. Therefore, a method needs to be proposed to make the reconstruction synthesis effect closer to that of real people. Summary of the invention

[0004] The present invention provides an artificial intelligence method, device, equipment and medium for reconstructing and restoring the skin color of a human face in an image, the main purpose of which is to solve the problem that the existing method for reconstructing and restoring the skin color of a human face in an image has poor effect.

[0005] In a first aspect, a method for reconstructing and restoring the skin color of a face in an image is provided, comprising:

[0006] Obtain a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set;

[0007] Identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image;

[0008] Performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image;

[0009] Performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image;

[0010] The converted face image and the transformed reconstructed image are subjected to color weighted mean fusion based on color channels to obtain the target face skin color.

[0011] In a second aspect, a device for reconstructing and restoring the skin color of a face in an image is provided, comprising:

[0012] An image acquisition module is used to acquire a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set;

[0013] An image processing module, used for identifying facial key points of each image in the facial image set, and performing facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image;

[0014] A face reconstruction module, used to perform face reconstruction on the pre-processed face image using a pre-trained face reconstruction model to obtain a reconstructed face image;

[0015] The color conversion module is used to perform color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image.

[0016] The color fusion module is used to perform color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for reconstructing and restoring the skin color of a face in the above-mentioned image when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method for reconstructing and repairing the skin color of a face in the above-mentioned image are implemented.

[0019] In the scheme implemented by the above-mentioned facial skin color reconstruction and restoration method, device, computer equipment and storage medium, a facial video of a pre-selected model can be obtained through a client, a facial picture in the facial video can be extracted to obtain a facial image set, facial key points of each image in the facial image set can be identified, and the images in the facial image set can be aligned according to the facial key points to obtain a pre-processed facial image, and the pre-processed facial image can be reconstructed using a pre-trained facial reconstruction model to obtain a reconstructed facial image, and the pre-processed facial image and the reconstructed facial image can be color-space converted to obtain a converted facial image and a converted reconstructed image, and the converted facial image and the converted reconstructed image can be color-weighted mean fused based on the color channel to obtain the target facial skin color. In the present invention, a reconstructed facial image with a target facial skin color is obtained by obtaining a facial image from a video, identifying and aligning facial key points, reconstructing the face, converting the color space and fusion the color, which can effectively improve the effect of facial reconstruction and make it closer to the skin color of real people. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0021] Figure 1 2 is a schematic diagram of an application environment of a method for reconstructing and restoring face skin color in an image according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a method for reconstructing and restoring the skin color of a face in an image in one embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a structure of a device for reconstructing and restoring the skin color of a face in an image in one embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0025] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The method for reconstructing and restoring the skin color of a human face in an image provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the client obtains a face video of a preselected model, extracts a face picture from the face video, obtains a face image set, identifies the face key points of each image in the face image set, performs face alignment on the images in the face image set according to the face key points, obtains a preprocessed face image, performs face reconstruction on the preprocessed face image using a pre-trained face reconstruction model, obtains a reconstructed face image, performs color space conversion on the preprocessed face image and the reconstructed face image, obtains a converted face image and a converted reconstructed image, performs color weighted mean fusion on the converted face image and the converted reconstructed image based on the color channel, and obtains the target face skin color. The efficiency of business process execution can be effectively improved. Among them, the client can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0028] See also Figure 2 As shown, Figure 2 A flowchart of a method for reconstructing and restoring the skin color of a face in an image provided by an embodiment of the present invention includes the following steps:

[0029] S1. Obtain a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set.

[0030] In the embodiment of the present invention, the face video is a video with a total frame number of not less than 5000 frames.

[0031] In the embodiment of the present invention, extracting the face picture in the face video to obtain the face image set includes:

[0032] Decoding the face video to obtain decoded video data;

[0033] Extracting the decoded video data frame by frame to obtain frame data;

[0034] The frame data is output according to a preset output format to obtain a set of face images.

[0035] In detail, the decoding process of the face video to obtain decoded video data is to convert the compressed video data into an uncompressed image sequence, which can be decoded by a decoding tool (such as FFmpeg, VLCMediaPlayer, OpenCV, etc.).

[0036] Specifically, the decoding tool can read the compressed data encoded in the video file, and then restore this data into a series of independent, uncompressed image frames through a series of algorithms and calculations. These image frames are the basic elements that make up the video. They are arranged in chronological order and show the dynamic changes in the video. In this way, the video data that was originally compressed for storage and transmission is converted into image data that can be further processed by computer vision algorithms, which facilitates subsequent face analysis, feature extraction and other related operations.

[0037] In detail, the preset output format includes the format of the picture (such as jpeg, png, etc.) and the resolution of the picture.

[0038] In the embodiment of the present invention, a face video of a pre-selected model is obtained, and a face picture in the face video is extracted to obtain a face image set, thereby improving the efficiency of subsequently obtaining pre-processed face images.

[0039] S2. Identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image.

[0040] In the embodiment of the present invention, identifying the facial key points of each image in the facial image set refers to identifying the contours of the facial features of each image in the facial image set.

[0041] In detail, identifying the facial key points of each image in the face image collection means carefully analyzing these images in order to accurately locate and extract the contours of the facial features in each face. This process involves using specific algorithms, such as deep learning-based facial key point detection technology, to determine the exact position and shape of facial features such as eyebrows, eyes, nose, mouth, etc. By identifying these key points of each face in the image collection, a detailed facial feature map of each face can be constructed, which is crucial for subsequent facial expression analysis, face alignment, face recognition and many other applications. This step ensures that the facial features of each person can be accurately grasped when processing face images.

[0042] In the embodiment of the present invention, the step of identifying the facial key points of each image in the facial image set includes:

[0043] Performing face detection on each image in the face image set to obtain face frame data;

[0044] Extracting an accurate face image according to the face frame data;

[0045] Performing histogram equalization processing on the precise face image to obtain a balanced face image;

[0046] The equalized face image is matched with facial features to obtain facial key points.

[0047] In detail, the face detection is performed on each image in the face image set to obtain face frame data, which is to use a face recognition algorithm (such as Adaboost learning algorithm) to realize face detection. The face frame data is a rectangular frame containing the position and size of the face. The Adaboost algorithm builds a strong classifier by combining multiple weak classifiers to improve the accuracy of face detection. During the detection process, the algorithm scans the image, finds the area that meets the facial features, and generates a face frame data for each detected face. These face frame data are not just simple marks, they contain specific information about the position and size of each face, usually in the form of a rectangular frame, which accurately defines the range of the face in the image. The coordinates and size of this rectangular frame are crucial for subsequent face analysis steps, such as key point detection, attribute analysis, and face alignment. In this way, it can be ensured that when processing face images, all operations are performed on the face area, thereby improving the pertinence and efficiency of the processing.

[0048] In detail, extracting the accurate face image according to the face frame data refers to cropping the original image according to the face frame data.

[0049] In detail, the histogram equalization is an image processing technology used to enhance the contrast of an image. The histogram equalization is a technology widely used in the field of image processing, and its core purpose is to enhance the contrast of an image by adjusting the histogram distribution of an image. This method makes the histogram distribution of an image more uniform by redistributing the pixel values ​​of the image, thereby improving the overall contrast of the image and making the details of the image more clearly visible. Specifically, histogram equalization creates a cumulative distribution function by counting the frequency of occurrence of each pixel value in the image, and then maps each pixel value according to this distribution function to achieve the redistribution of pixel values. In this way, pixel values ​​that are too concentrated at certain brightness levels will be dispersed to a wider range of brightness levels, while those brightness levels that appear less frequently in the image will be enhanced. In this way, histogram equalization can significantly enhance the details of dark areas in the image, while not overexposing the details of bright areas, so that the visual effect of the entire image is improved. This technology is particularly suitable for images with low contrast due to undesirable lighting conditions, making them more visually balanced and vivid.

[0050] In detail, the facial features of the equalized face image are matched to obtain facial key points. The contours of the facial features in the equalized face image are identified by an edge recognition algorithm, and then matched with a pre-established facial feature contour database to confirm the key points of the facial feature positions in the equalized face image.

[0051] In detail, by applying the edge recognition algorithm, the contours of the facial features in the equalized facial image can be effectively identified. This process involves the detection and analysis of each edge in the image to extract the specific shape and position of the facial features such as the eyes, nose, and mouth. After identifying these contours, they are then matched with a pre-established facial contour database. This database stores a large amount of standardized facial contour information as a reference for judgment and confirmation. Through this matching process, the key points of the facial features in the equalized facial image can be accurately located. These key points not only help further image processing and analysis, but also provide important basic data for subsequent applications such as face recognition and expression analysis. In this way, the accurate recognition and positioning of facial features is ensured, thereby improving the accuracy and effect of subsequent processing.

[0052] In the embodiment of the present invention, the face alignment of the images in the face image set according to the face key points to obtain the preprocessed face image refers to aligning the images in the face image set with a preset face template. This process means that the facial features in each image, such as the positions of the eyes, nose and mouth, will be used to match a preset face template. The face template is a standard face model that contains the expected positions and distribution of facial features. Through the alignment operation, the face in each image can be adjusted so that its features correspond to the features in the template, thereby achieving standardization of the posture, size and position of the face. In this way, no matter how the faces in the original images are tilted or rotated, after alignment, they will be converted into a unified posture, which is convenient for subsequent processing and analysis. This step is crucial to ensure the consistency and accuracy of facial images during comparison, recognition or other analysis.

[0053] In the embodiment of the present invention, the step of performing face alignment on the images in the face image set according to the face key points to obtain the preprocessed face image includes:

[0054] Constructing a transformation matrix according to the facial key points;

[0055] Mapping the images in the face image set to a preset face template by aligning face key points using the transformation matrix to obtain a mapped face image;

[0056] The mapped face image is normalized to obtain an aligned face image.

[0057] In detail, the preset face template is a standard face image or a standard template containing key point positions. This template will be used as the alignment target.

[0058] In detail, the normalization process for the mapped facial images is to ensure that all aligned facial images are consistent in terms of brightness, contrast, etc.

[0059] In an embodiment of the present invention, by identifying the facial key points of each image in the facial image set, the images in the facial image set are face aligned according to the facial key points to obtain a preprocessed facial image, thereby improving the efficiency of subsequent facial reconstruction.

[0060] S3. Reconstruct the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image.

[0061] In an embodiment of the present invention, the face reconstruction model is a model based on the DeepFaceLab algorithm.

[0062] In detail, the DeepFaceLab is a face swap algorithm based on deep learning technology. It realizes accurate recognition and replacement of faces in source videos by training neural network models. The algorithm builds complex neural networks through a deep learning framework, which can learn and understand the characteristics and structure of faces. During the training process, the neural network learns how to accurately identify faces from source videos through a large amount of data, including the location of facial features, facial expressions and feature points. Once the model training is completed, DeepFaceLab can use this learned knowledge to accurately map the facial features in the source video to the target video to achieve face replacement. This process involves highly complex image processing and pattern recognition technology, so that the replaced face looks natural and seamless visually, and can maintain consistency in expression, lighting conditions and other visual factors. This ability of DeepFaceLab gives it a wide range of application potential in filmmaking, game development, virtual reality and other fields.

[0063] In the embodiment of the present invention, the training of the face reconstruction model requires obtaining more than 5,000 modal face images as source data, and more than 90,000 face images containing various expressions and postures as target data, using VAE (variational autoencoder) as a generator, and utilizing two Inter layers (latent spaces) to decouple the features of the source data and the target data. The Inter layer is then used to cooperate with the encoder to express the features of the source data, and the Inter layer is used to adapt the features of the target data, and the training of the face reconstruction model is achieved through face reconstruction and implicit alignment.

[0064] In detail, the VAE (Variational Autoencoder) is a deep generative model that achieves efficient representation and generation of data by introducing probability distribution and reparameterization techniques.

[0065] In an embodiment of the present invention, the step of performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image includes:

[0066] Initializing a preset three-dimensional face model using the face reconstruction model to obtain an initialized face model;

[0067] Adjust the key point positions of the initialized face model according to the face key points to obtain an adjusted face model;

[0068] Mapping the preprocessed face image to the adjusted face model to obtain a mapped face model;

[0069] Acquire the facial image of the mapped facial model to obtain a reconstructed facial image.

[0070] In the embodiment of the present invention, the face reconstruction is performed on the preprocessed face image by using a pre-trained face reconstruction model to obtain a reconstructed face image, thereby improving the efficiency of subsequent color space conversion.

[0071] S4. Perform color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image.

[0072] In the embodiment of the present invention, the color space conversion of the pre-processed face image and the reconstructed face image to obtain the converted face image and the converted reconstructed image refers to converting the image from the RBG color space to the LAB color space so as to obtain the converted face image and the converted reconstructed image. The specific meaning of this conversion is to convert the color representation of the image from the RGB color space to the LAB color space. The RGB color space represents the color based on the combination of three colors: red (R), green (G) and blue (B), while the LAB color space is a more complex color model that divides the color information into brightness (L*) and two color channels (a and b), where a represents the color range from green to red, and b represents the color range from blue to yellow. Through this conversion, the color information of the image can be represented in a way that is more in line with human visual perception, thereby providing higher accuracy and effect in subsequent image processing and analysis. The use of the LAB color space helps to maintain color consistency under different lighting conditions, so that the final generated face image and reconstructed image are more natural and realistic in visual effect.

[0073] In detail, the LAB color space is a color space defined by the International Commission on Illumination (CIE) in 1976. The construction of this color space is based on experimental data on color perception by the human visual system. The purpose is to provide a device-independent color representation method that can more accurately measure the differences between colors. The LAB color space consists of three coordinates: L represents brightness, a and b represent the opposite dimensions of color, where the a axis represents the range from red to green, and the b* axis represents the range from yellow to blue. This color space is designed to simulate the human eye's perception of color, making color changes more visually uniform and intuitive. Therefore, the LAB color space is very useful in color management and applications that require precise color control, such as image processing and other fields.

[0074] In detail, the LAB color space is developed from the early CIE XYZ color space. Although CIE XYZ is also a standardized color space, its coordinate values ​​are not directly related to human visual perception. The LAB color space is more in line with the human eye's perception of color differences, so that the color distance calculated in this space can better correspond to the color differences perceived by the human eye.

[0075] Furthermore, the CIE XYZ color space is a standard color space defined by the International Commission on Illumination (CIE) in 1931. It is constructed based on experimental data and mathematical models to describe the human visual system's perception of color. Its purpose is to provide an objective and standardized way to represent color.

[0076] In the embodiment of the present invention, the color space conversion of the preprocessed face image and the reconstructed face image to obtain the converted face image and the converted reconstructed image means first converting the image from the RGB color space to the CIEXYZ color space, and then converting from the CIE XYZ color space to the LAB color space. Through this series of color space conversions, the converted face image and the converted reconstructed image obtained can more realistically reflect the color information of the face, and provide a more accurate color basis for subsequent image analysis, editing and synthesis.

[0077] In the embodiment of the present invention, the color space conversion of the preprocessed face image and the reconstructed face image to obtain the converted face image and the converted reconstructed image includes:

[0078] Converting the preprocessed face image and the reconstructed face image into CIE XYZ color space to obtain color values ​​in CIEXYZ color space;

[0079] Calculate target brightness, target red-green axis components, and target yellow-blue axis components according to the CIE XYZ color space color value;

[0080] The preprocessed face image and the reconstructed face image are converted in color space according to the target brightness, the target red-green axis component, and the target yellow-blue axis component to obtain a converted face image and a converted reconstructed image.

[0081] In the embodiment of the present invention, by performing color space conversion on the preprocessed face image and the reconstructed face image, a converted face image and a converted reconstructed image are obtained, thereby improving the efficiency of subsequent color fusion.

[0082] S5. Performing color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color.

[0083] In the embodiment of the present invention, the color-weighted mean fusion of the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color includes:

[0084] Calculating the difference between the brightness of the converted face image and the brightness mean to obtain a first difference, and calculating the ratio of the brightness mean square error of the converted reconstructed image to the brightness mean square error of the converted face image to obtain a first ratio;

[0085] Calculating the product of the first difference and the first ratio to obtain a first product, and calculating the sum of the first product and the brightness mean of the transformed and reconstructed image to obtain a fused brightness;

[0086] Calculate the difference between the red-green axis component of the converted face image and the mean of the red-green axis component to obtain a second difference, calculate the ratio of the mean square error of the red-green axis component of the converted reconstructed image to the mean square error of the red-green axis component of the converted face image to obtain a second ratio;

[0087] Calculate the product of the second difference and the second ratio to obtain a second product, calculate the sum of the second product and the mean of the red and green axis components of the transformed and reconstructed image to obtain a fused red and green axis component;

[0088] Calculate the difference between the blue-yellow axis component of the converted face image and the mean of the blue-yellow axis component to obtain a third difference, and calculate the ratio of the mean square error of the blue-yellow axis component of the converted and reconstructed image to the mean square error of the blue-yellow axis component of the converted face image to obtain a third ratio;

[0089] Calculate the product of the third difference and the third ratio to obtain a third product, calculate the sum of the third product and the mean of the blue-yellow axis component of the transformed and reconstructed image to obtain a fused blue-yellow axis component;

[0090] The target face skin color is constructed according to the fused brightness, the fused red-green axis component, and the fused blue-yellow axis component.

[0091] In detail, the red-green axis component represents the change of color in the red-green direction. When the red-green axis component is positive, it means that the color is biased towards red, and when the red-green axis component is negative, it means that the color is biased towards green. This component belongs to a dimension in the color space and is used to distinguish and quantify the specific position of the color sample in the spectrum range from red to green.

[0092] In detail, the blue-yellow axis component represents the change of color in the blue-yellow direction. When the blue-yellow axis component is a positive value, it means that the color tends to be blue, and when the blue-yellow axis component is a negative value, it means that the color tends to be yellow.

[0093] In the embodiment of the present invention, the target face skin color is obtained by performing color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel, so that the target face skin color is closer to the real face color.

[0094] It can be seen that this patent proposes to perform color fusion between the source face image and the reconstructed face image, thereby improving the accuracy of the face color and making the visual effect closer to that of a real person. In detail, through a series of steps including acquiring a face image from a video, performing face key point recognition and alignment, face reconstruction, color space conversion and color fusion, a reconstructed face image with the target face skin color is obtained, thereby effectively improving the effect of face reconstruction and making it closer to the skin color of a real person.

[0095] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0096] In one embodiment, a device for reconstructing and restoring the skin color of a human face in an image is provided, and the device for reconstructing and restoring the skin color of a human face in an image corresponds one-to-one to the method for reconstructing and restoring the skin color of a human face in an image in the above embodiment. Figure 3 As shown, the facial skin color reconstruction and restoration device in the image includes an image acquisition module 101, an image processing module 102, a face reconstruction module 103, a color conversion module 104 and a color fusion module 105. The detailed description of each functional module is as follows:

[0097] An image acquisition module 101 is used to acquire a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set;

[0098] An image processing module 102 is used to identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a pre-processed facial image;

[0099] A face reconstruction module 103 is used to perform face reconstruction on the pre-processed face image using a pre-trained face reconstruction model to obtain a reconstructed face image;

[0100] The color conversion module 104 is used to perform color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image.

[0101] The color fusion module 105 is used to perform color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color.

[0102] In one embodiment, the image acquisition module 101, when extracting the face image in the face video to obtain the face image set, is used to:

[0103] Decoding the face video to obtain decoded video data;

[0104] Extracting the decoded video data frame by frame to obtain frame data;

[0105] The frame data is output according to a preset output format to obtain a set of face images.

[0106] In one embodiment, the image processing module 102, when performing the identification of facial key points of each image in the facial image set, is used to:

[0107] Performing face detection on each image in the face image set to obtain face frame data;

[0108] Extracting an accurate face image according to the face frame data;

[0109] Performing histogram equalization processing on the precise face image to obtain a balanced face image;

[0110] The equalized face image is matched with facial features to obtain facial key points.

[0111] In one embodiment, the image processing module 102, when performing face alignment on the images in the face image set according to the face key points to obtain the pre-processed face image, is used to:

[0112] Constructing a transformation matrix according to the facial key points;

[0113] Mapping the images in the face image set to a preset face template by aligning face key points using the transformation matrix to obtain a mapped face image;

[0114] The mapped face image is normalized to obtain an aligned face image.

[0115] In one embodiment, the face reconstruction module 103, when performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image, is used to:

[0116] Initializing a preset three-dimensional face model using the face reconstruction model to obtain an initialized face model;

[0117] Adjust the key point positions of the initialized face model according to the face key points to obtain an adjusted face model;

[0118] Mapping the preprocessed face image to the adjusted face model to obtain a mapped face model;

[0119] Acquire the facial image of the mapped facial model to obtain a reconstructed facial image.

[0120] In one embodiment, the color conversion module 104, when performing color space conversion on the preprocessed face image and the reconstructed face image to obtain the converted face image and the converted reconstructed image, is used to:

[0121] Converting the preprocessed face image and the reconstructed face image into CIE XYZ color space to obtain color values ​​in CIEXYZ color space;

[0122] Calculate target brightness, target red-green axis components, and target yellow-blue axis components according to the CIE XYZ color space color value;

[0123] The preprocessed face image and the reconstructed face image are converted in color space according to the target brightness, the target red-green axis component, and the target yellow-blue axis component to obtain a converted face image and a converted reconstructed image.

[0124] In one embodiment, the color fusion module 105, when performing the color-weighted mean fusion of the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color, is used to:

[0125] Calculating the difference between the brightness of the converted face image and the brightness mean to obtain a first difference, and calculating the ratio of the brightness mean square error of the converted reconstructed image to the brightness mean square error of the converted face image to obtain a first ratio;

[0126] Calculating the product of the first difference and the first ratio to obtain a first product, and calculating the sum of the first product and the brightness mean of the transformed and reconstructed image to obtain a fused brightness;

[0127] Calculate the difference between the red-green axis component of the converted face image and the mean of the red-green axis component to obtain a second difference, calculate the ratio of the mean square error of the red-green axis component of the converted reconstructed image to the mean square error of the red-green axis component of the converted face image to obtain a second ratio;

[0128] Calculate the product of the second difference and the second ratio to obtain the second product, calculate the second The present invention provides a device for reconstructing and repairing the skin color of a face in an image. Based on the basic idea of ​​the tree model, the parent node of the tree is used as the main operating unit, and its left and right child nodes are used as the basic branches for executing success / failure. According to the operation status of the parent node (success or failure), the left and right child nodes are automatically selected. The left and right child nodes are respectively subtrees that meet the same design structure. Through the nesting of the tree, the originally complex jump process is converted into a selection of tree nodes with a clear nested structure. Combined with the processing of the iterator, complex logic that meets different jump conditions can be flexibly designed, because the change of the node does not change the stability of the tree structure itself. Compared with the traditional if-else structure, the tree mode proposed in the embodiment of the present invention can effectively maintain the structural stability of the code, improve the maintainability of the code, and greatly improve the processing efficiency of complex processes.

[0129] For the specific limitations of the device for reconstructing and repairing the skin color of a human face in an image, please refer to the limitations of the intelligent question-answering method above, which will not be repeated here. Each module in the above-mentioned device for reconstructing and repairing the skin color of a human face in an image can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0130] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a method for reconstructing and repairing the skin color of a face in an image.

[0131] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a method for reconstructing and repairing the skin color of a face in an image

[0132] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0133] Obtain a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set;

[0134] Identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image;

[0135] Performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image;

[0136] Performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image;

[0137] The converted face image and the transformed reconstructed image are subjected to color weighted mean fusion based on color channels to obtain the target face skin color.

[0138] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0139] Obtain a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set;

[0140] Identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image;

[0141] Performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image;

[0142] Performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image;

[0143] The converted face image and the transformed reconstructed image are subjected to color weighted mean fusion based on color channels to obtain the target face skin color.

[0144] After obtaining the data environment fault processing instruction, the data environment fault processing tree structure process is executed to obtain the process execution result. It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device can be referred to the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0146] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0147] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0148] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0149] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0150] Finally, it should be noted that if software tools or components other than those of the company appear in the application embodiments, they are only used for illustration and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for reconstructing and restoring facial skin color in an image, characterized in that: include: Obtain a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set; Identify facial key points of each image in the facial image set, and perform facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image; Performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image; Performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image; The converted face image and the transformed reconstructed image are subjected to color weighted mean fusion based on color channels to obtain the target face skin color.

2. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The step of extracting the face picture from the face video to obtain a face image set includes: Decoding the face video to obtain decoded video data; Extracting the decoded video data frame by frame to obtain frame data; The frame data is output according to a preset output format to obtain a set of face images.

3. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The identifying facial key points of each image in the facial image set includes: Performing face detection on each image in the face image set to obtain face frame data; Extracting an accurate face image according to the face frame data; Performing histogram equalization processing on the precise face image to obtain a balanced face image; The equalized face image is matched with facial features to obtain facial key points.

4. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The step of performing face alignment on the images in the face image set according to the face key points to obtain a preprocessed face image comprises: Constructing a transformation matrix according to the facial key points; Mapping the images in the face image set to a preset face template by aligning face key points using the transformation matrix to obtain a mapped face image; The mapped face image is normalized to obtain an aligned face image.

5. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The method of performing face reconstruction on the preprocessed face image using a pre-trained face reconstruction model to obtain a reconstructed face image includes: Initializing a preset three-dimensional face model using the face reconstruction model to obtain an initialized face model; Adjust the key point positions of the initialized face model according to the face key points to obtain an adjusted face model; Mapping the preprocessed face image to the adjusted face model to obtain a mapped face model; Acquire the facial image of the mapped facial model to obtain a reconstructed facial image.

6. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The color space conversion of the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image includes: Converting the preprocessed face image and the reconstructed face image into a CIE XYZ color space to obtain a color value in the CIE XYZ color space; Calculate target brightness, target red-green axis components, and target yellow-blue axis components according to the CIE XYZ color space color value; The preprocessed face image and the reconstructed face image are converted in color space according to the target brightness, the target red-green axis component, and the target yellow-blue axis component to obtain a converted face image and a converted reconstructed image.

7. The method for reconstructing and restoring facial skin color in an image according to claim 1, characterized in that: The step of performing color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color includes: Calculating the difference between the brightness of the converted face image and the brightness mean to obtain a first difference, and calculating the ratio of the brightness mean square error of the converted reconstructed image to the brightness mean square error of the converted face image to obtain a first ratio; Calculating the product of the first difference and the first ratio to obtain a first product, and calculating the sum of the first product and the brightness mean of the transformed and reconstructed image to obtain a fused brightness; Calculate the difference between the red-green axis component of the converted face image and the mean of the red-green axis component to obtain a second difference, calculate the ratio of the mean square error of the red-green axis component of the converted reconstructed image to the mean square error of the red-green axis component of the converted face image to obtain a second ratio; Calculate the product of the second difference and the second ratio to obtain a second product, calculate the sum of the second product and the mean of the red and green axis components of the transformed and reconstructed image to obtain a fused red and green axis component; Calculate the difference between the blue-yellow axis component of the converted face image and the mean of the blue-yellow axis component to obtain a third difference, and calculate the ratio of the mean square error of the blue-yellow axis component of the converted and reconstructed image to the mean square error of the blue-yellow axis component of the converted face image to obtain a third ratio; Calculate the product of the third difference and the third ratio to obtain a third product, calculate the sum of the third product and the mean of the blue-yellow axis component of the transformed and reconstructed image to obtain a fused blue-yellow axis component; The target face skin color is constructed according to the fused brightness, the fused red-green axis component, and the fused blue-yellow axis component.

8. A device for reconstructing and restoring the skin color of a human face in an image, characterized in that: include: An image acquisition module is used to acquire a face video of a pre-selected model, extract a face image from the face video, and obtain a face image set; An image processing module, used for identifying facial key points of each image in the facial image set, and performing facial alignment on the images in the facial image set according to the facial key points to obtain a preprocessed facial image; A face reconstruction module, used to perform face reconstruction on the pre-processed face image using a pre-trained face reconstruction model to obtain a reconstructed face image; A color conversion module, used for performing color space conversion on the preprocessed face image and the reconstructed face image to obtain a converted face image and a converted reconstructed image; The color fusion module is used to perform color weighted mean fusion on the converted face image and the transformed reconstructed image based on the color channel to obtain the target face skin color.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for reconstructing and restoring the skin color of a human face in an image as claimed in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for reconstructing and restoring the skin color of a face in an image as claimed in any one of claims 1 to 7 are implemented.

Images