Training method, device and equipment capable of enhancing face stereo model

Abstract

The invention discloses a training method, device and equipment capable of enhancing a face stereo model. The method comprises the following steps: a model comprises a bone phase segmentation model M1 and an enhancement model M2, namely a fusion module; the method comprises the steps that a data set is constructed, the data set comprises a plurality of figure images and real labels corresponding to the images, and the real labels comprise mask labels of concave-convex areas of five sense organs, mask labels of concave-convex areas of facial contours and facial muscle contour trend diagrams; for each image in the data set, jointly inputting the image and the facial muscle contour trend graph into a bone phase segmentation model M1, and guiding the bone phase segmentation model M1 by the facial muscle contour trend graph to predict Mask of a five-sense-organ concave-convex region and Mask of a facial contour concave-convex region of the corresponding image; the five-sense-organ concave-convex area corresponds to the five-sense-organ concave-convex area, and the face contour concave-convex area corresponds to the face contour concave-convex area. The whole process is automatic, the requirement for manual intervention is reduced from image input to result image output, and the processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of face processing, and in particular to a training method, device and equipment capable of enhancing a face stereoscopic model. Background Art

[0002] Previous manual facial stereo enhancement methods mainly rely on manual operations by professional technicians and have at least one of the following limitations:

[0003] (1) Significant time consumption: Manually processing each face image requires a lot of time, especially when processing large-scale image datasets, its efficiency is significantly reduced.

[0004] (2) Lack of consistency: Due to differences in operations among different technicians, it is often difficult to maintain consistency in processing results, which affects the standardization and repeatability of image processing.

[0005] (3) High technical threshold: Manual stereo enhancement requires the operator to have professional skills and rich experience, which makes it difficult for novices to quickly master and effectively apply it.

[0006] (4) Expensive labor costs: Manual processing methods require a lot of human resources, thereby increasing the overall cost of the project.

[0007] These limitations indicate that the development of automated facial stereo enhancement technology is of great significance for improving efficiency, reducing costs, and ensuring consistency and standardization of processing results. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a training method, device and equipment that can enhance the three-dimensional model of the face, which can solve at least one technical problem mentioned in the background technology.

[0009] According to one aspect of the present invention, a training method for a facial stereoscopic model is provided, wherein the model includes a bone segmentation model M1 and an enhancement model M2, i.e., a fusion module; the method includes:

[0010] Construct a data set, which includes several person images and real labels corresponding to the images, and the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend maps;

[0011] For each image in the data set, the image and the facial muscle contour trend map are input into the bone phase segmentation model M1. The facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the facial features concave and convex area and the mask of the facial contour concave and convex area of ​​the corresponding image;

[0012] For each image, the mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour are input into the enhancement model M2 together. The enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour, and outputs an enhanced image and the corresponding enhanced area mask;

[0013] For each enhanced image and enhanced region mask of the image, the enhanced image and the enhanced region mask are input into the fusion module, the image and the enhanced image are fused based on the enhanced region mask, and the result image is output;

[0014] A loss value is calculated based on the result image and the original image, and the model is trained based on the loss value.

[0015] In the above technical solution, a deep learning training framework for image processing is provided, especially for enhancing the convex and concave surfaces of facial muscles in character images. The model consists of a bone segmentation model M1 and an enhancement model M2. The bone segmentation model M1 is responsible for predicting the concave and convex areas of facial features and facial contours in the image. It uses the facial muscle contour trend map as guiding information to improve the accuracy of the prediction. The enhancement model M2 receives the original image and the masks of the concave and convex areas of facial features and facial contours generated by the M1 model, and brightens or darkens the corresponding areas of the original image (that is, the bright area guides the enhancement model to enhance the corresponding strength, and the dark area guides the enhancement model to weaken the corresponding strength) to highlight the three-dimensional sense of the muscles. The fusion module fuses the enhanced image with the original image according to the enhanced area mask to generate the final output image. In this method, a data set containing character images and their corresponding real labels is collected. The real labels include masks of concave and convex areas of facial features, masks of concave and convex areas of facial contours, and facial muscle contour trend maps. Each image and its corresponding facial muscle contour trend map are input into the M1 model to predict the masks of the concave and convex areas of the facial features and the concave and convex areas of the facial contour. The M2 model is used to brighten or darken the original image according to the mask generated by the M1 model to generate an enhanced image and its mask. The enhanced image is fused with the original image through the fusion module to generate the final output image. The loss value between the result image and the original image is calculated, and this loss value is used to train and optimize the entire model. By using the facial muscle contour trend map as a guide, the model can more accurately identify and enhance the convex and concave surfaces of the face, thereby improving the three-dimensional sense and realism of the image. The entire process is automated, from image input to output result map, reducing the need for manual intervention and improving processing efficiency. The model is trained based on a large amount of annotated data and can learn complex image features and enhancement patterns, which is more flexible and accurate than traditional manual adjustment methods. By processing the facial features and facial contours separately, the model can retain the details and texture of the image while enhancing the contrast of the image. The use of the fusion module ensures a smooth transition between the enhanced image and the original image, avoiding obvious boundaries between the enhanced and non-enhanced areas.

[0016] In some embodiments, the facial muscle contour trend map is obtained by:

[0017] Anisotropic filtering is used to obtain the muscle contour information and contour direction of the face of the character in the character image, and a contour trend map is generated.

[0018] In the above technical solution, anisotropic filtering is an image processing technology that can smooth an image while maintaining edge features. This method is particularly suitable for enhancing specific structural features in an image, such as facial muscle contours. In a specific implementation, anisotropic filtering is first applied to a character image to highlight muscle contours. During the filtering process, the algorithm adjusts the diffusion process according to the local gradient characteristics of the image, so that the diffusion is weakened near the edge, thereby retaining edge information. The processed image can more clearly show the contour trend of the facial muscles. The generated contour trend map can provide important information for subsequent image enhancement models, helping the model to more accurately identify and process the convex and concave areas of the face. The advantage of this method is that it can automatically extract key muscle contour information from the original image, provide precise guidance for image enhancement, and thus improve the efficiency and effect of image processing.

[0019] In some embodiments, the image is fused with the enhanced image based on the enhanced region mask, and the result image is outputted, and the fusion is performed using the following formula:

[0020] I f =M⊙I1+(1-M)⊙I2

[0021] In the formula, I f represents the result image, M represents the mask, and its value range is [0, 1], I1 represents the image, and I2 represents the enhanced image.

[0022] In the above technical solution, two images I1 and I2 are weighted combined according to mask M. I1 is the original image, I2 is the enhanced image, and M is the enhanced region mask. Mask M is a binary image with a value range between [0, 1]. Each pixel value in the mask determines whether the pixel at the corresponding position is selected from I1 or I2. If a pixel value in mask M is 1, the result image I f The corresponding pixels in will come entirely from I1 (the original image). If a pixel value in mask M is 0, then the result image I f The corresponding pixels in will come entirely from I2 (the enhanced image). The operation ⊙ represents an element-by-element multiplication, also known as the Hadamard product. This method allows the degree of fusion between the two images to be flexibly controlled by adjusting the value of the mask M, which can accurately locate the area that needs to be selected from the original image I1 and the area that needs to be retained in the enhanced image I2. Since the value range of the mask M is continuous, this method can produce a natural and smooth transition effect, especially in the mask edge area.

[0023] In some embodiments, a loss value is calculated based on the result image and the original image, and the model is trained based on the loss value, specifically:

[0024] Calculate the VGG loss for the result image and the original image to obtain the first loss value;

[0025] Calculate the GAN loss for the result image and the original image to obtain the second loss value;

[0026] The model is trained based on the first loss value and the second loss value.

[0027] In the above technical solution, VGG Loss (perceptual loss, used in this case to measure the content difference of the image): the first loss value is obtained by calculating VGG loss on the result image and the original image. VGG loss is a feature extraction method based on deep learning, which calculates the loss by comparing the feature representations of two images in the pre-trained VGG network. GAN Loss (adversarial loss, used in this case to measure the generation quality of the image): the second loss value is obtained by calculating GANloss on the result image and the original image. GAN loss is usually used to train a generative adversarial network (GAN), in which the discriminator is used to distinguish between real images and generated images, and the generator tries to generate images that can deceive the discriminator. Through this adversarial process, the quality of the generated image can be improved to make it closer to the real image. Combining these two loss values ​​can simultaneously consider the perceptual quality and generation quality of the image, thereby optimizing the model more comprehensively. During the training process, the parameters of the model are adjusted according to the gradients of the two losses to minimize the difference between the result image and the original image and improve the generation effect of the model. This method of combining multiple loss functions can improve the robustness and adaptability of the model, enabling it to achieve good performance in different application scenarios.

[0028] In some embodiments, a loss value is calculated based on the result image and the original image, and the model is trained based on the loss value, specifically:

[0029] Calculate the VGG loss for the result image and the original image to obtain the first loss value;

[0030] Calculate the GAN loss for the result image and the original image to obtain the second loss value;

[0031] The contrast value of the corresponding area in the result image is calculated based on the mask of the concave and convex area of ​​the facial features and the mask of the concave and convex area of ​​the facial contour;

[0032] The model is trained based on the first loss value, the second loss value, and the contrast value.

[0033] In the above technical solution, based on the Mask of the concave and convex areas of the facial features and the Mask of the concave and convex areas of the facial contour, the contrast value of the corresponding area in the result image is calculated. This step is to ensure that the model not only generates an image similar to the original image as a whole, but also maintains or enhances the contrast in the local area, thereby improving the visual impact and three-dimensional sense of the image. Combining the above three indicators to train the model, this method can ensure that the model is optimized in many aspects, including overall image quality, local contrast enhancement, and adversarial generation capabilities, so that the model can learn more comprehensively how to generate high-quality result images. By considering the perceptual quality, generation quality and local contrast of the image, the model can more accurately capture and enhance the key features of the image; and it helps to balance the training process of the model, avoid overfitting, and improve the adaptability of the model to different types of images, which helps to generate images that are more visually realistic, have stronger contrast and rich details.

[0034] According to another aspect of the present invention, there is provided a model capable of enhancing the three-dimensionality of the face, characterized in that it is trained by the above-mentioned training method for removing the model of the blooper; the model comprises a bone phase segmentation model M1 and an enhancement model M2, i.e., a fusion module.

[0035] In the above technical solution, the model is trained using the above method to realize automatic processing of face images, which can quickly and accurately enhance the three-dimensional sense of the face while maintaining the natural beauty of the image. This model overcomes the limitations of previous manual three-dimensional enhancement methods and can significantly improve processing efficiency while maintaining high precision. It should be noted that the principles and effects of each step have been described above and will not be explained in detail here.

[0036] According to another aspect of the present invention, a training device for removing a goof object model is provided, comprising:

[0037] Acquisition module, constructing a data set, the data set includes several person images and real labels corresponding to the images, the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend map;

[0038] The bone phase segmentation module is used to input the image and the facial muscle contour trend map into the bone phase segmentation model M1 for each image in the data set. The facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the facial features concave and convex area and the mask of the facial contour concave and convex area of ​​the corresponding image;

[0039] The enhancement module is used for inputting the image and the corresponding facial features concave-convex area mask and facial contour concave-convex area mask of each image into the enhancement model M2. The enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the facial features concave-convex area mask and the facial contour concave-convex area mask, and outputs an enhanced image and a corresponding enhanced area mask;

[0040] A fusion module is used for inputting the enhanced image and the enhanced region mask of each image into the fusion module, fusing the image with the enhanced image based on the enhanced region mask, and outputting a result image;

[0041] A training module is used to calculate a loss value based on the result image and the original image, and train the model based on the loss value.

[0042] In the above technical scheme, in order to better use the above method, the present application proposes a training device for removing the model of the object with the gaffe, and each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0043] According to another aspect of the present invention, there is provided a training device for removing a goof object model, comprising:

[0044] at least one processor and a memory communicatively coupled to the at least one processor;

[0045] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0046] In the above technical solution, in order to better run and process the method, the above method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0047] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program, wherein the computer program implements the above method when executed by a processor.

[0048] In the above technical solution, in order to better run and use the method, the above method is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a flow chart of an embodiment of a training method for enhancing a facial stereoscopic model according to the present invention;

[0051] Figure 2 It is a schematic diagram of a training framework of an embodiment of a training method capable of enhancing a facial stereoscopic model according to the present invention;

[0052] Figure 3 It is an original image of an embodiment of a training method for enhancing a facial stereoscopic model according to the present invention;

[0053] Figure 4 It is a schematic diagram of Maska1 of an embodiment of a training method capable of enhancing a facial stereoscopic model of the present invention;

[0054] Figure 5 2 is a schematic diagram of Maska2 of an embodiment of a training method for enhancing a facial stereoscopic model according to the present invention;

[0055] Figure 6 Schematic diagram of Mask D of an embodiment of a training method capable of enhancing a facial stereoscopic model according to the present invention;

[0056] Figure 7 It is a schematic diagram of a difference graph J of an embodiment of a training method capable of enhancing a facial stereoscopic model according to the present invention;

[0057] Figure 8 It is a schematic diagram of the final result of an embodiment of a training method capable of enhancing a facial stereoscopic model according to the present invention;

[0058] Fig. 9 It is a schematic diagram of actual reasoning steps of an embodiment of a training method capable of enhancing a facial stereoscopic model according to the present invention;

[0059] Fig.10 It is a structural schematic diagram of an embodiment of a training device for removing a goof object model according to the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0061] The present invention provides a training method, a device and equipment for enhancing a facial stereoscopic model, which can solve at least one technical problem mentioned in the background technology.

[0062] Embodiment 1

[0063] See also Figure 1 A training method for a facial stereoscopic model capable of enhancing the facial stereoscopic model includes a bone segmentation model M1 and an enhancement model M2, i.e., a fusion module; the method includes:

[0064] S1. Construct a data set, which includes several person images and real labels corresponding to the images, wherein the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend maps;

[0065] S2, for each image in the data set, the image and the facial muscle contour trend map are input into the bone phase segmentation model M1, and the facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the concave and convex area of ​​the facial features and the mask of the concave and convex area of ​​the facial contour of the corresponding image;

[0066] S3, for each image, the mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour, input the image and the corresponding mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour into the enhancement model M2, and the enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the masks of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour, and outputs an enhanced image and the corresponding enhanced area mask;

[0067] S4, for each enhanced image and enhanced region mask of the image, the enhanced image and the enhanced region mask are input into the fusion module, the image and the enhanced image are fused based on the enhanced region mask, and the result image (such as Figure 4 );

[0068] S5. Calculate a loss value based on the result image and the original image, and train the model based on the loss value.

[0069] In this embodiment, a deep learning training framework for image processing is provided, in particular, for enhancing the convex and concave surfaces of facial muscles in character images. The model consists of a bone segmentation model M1 and an enhancement model M2. The bone segmentation model M1 is responsible for predicting the concave and convex areas of facial features and facial contours in the image. It uses the facial muscle contour trend map as guidance information to improve the accuracy of the prediction. The enhancement model M2 receives the original image and the mask of the concave and convex areas of facial features and facial contours generated by the M1 model, and brightens or darkens the corresponding areas of the original image to highlight the three-dimensional sense of the muscles. The fusion module fuses the enhanced image with the original image according to the enhanced area mask to generate the final output image. In this method, a data set containing character images and their corresponding true labels is collected. The true labels include masks of concave and convex areas of facial features, masks of concave and convex areas of facial contours, and facial muscle contour trend maps. Each image and its corresponding facial muscle contour trend map are input into the M1 model to predict the masks of concave and convex areas of facial features and facial contours. The M2 model is used to brighten or darken the original image according to the mask generated by the M1 model to generate an enhanced image and its mask. The enhanced image is fused with the original image through the fusion module to generate the final output image. The loss value between the result image and the original image is calculated, and this loss value is used to train and optimize the entire model. By using the facial muscle contour trend map as a guide, the model can more accurately identify and enhance the convexity and concavity of facial muscles, thereby improving the three-dimensional sense and realism of the image. The entire process is automated, from image input to output result map, reducing the need for manual intervention and improving processing efficiency. The model is trained based on a large amount of annotated data and can learn complex image features and enhancement patterns, which is more flexible and accurate than traditional manual adjustment methods. By processing the concave and convex areas of facial features and facial contours separately, the model can retain the details and texture of the image while enhancing the contrast of the image. The use of the fusion module ensures a smooth transition between the enhanced image and the original image, avoiding obvious boundaries between the enhanced area and the non-enhanced area.

[0070] In this embodiment, the facial muscle contour trend map is obtained by the following method: anisotropic filtering is used to obtain the muscle contour information and contour trend of the face of the character in the character image, and a contour trend map is generated. Anisotropic filtering is an image processing technology that can smooth the image while maintaining edge features. This method is particularly suitable for enhancing specific structural features in the image, such as facial muscle contours. In a specific implementation, anisotropic filtering is first applied to the character image to highlight the muscle contour. During the filtering process, the algorithm adjusts the diffusion process according to the local gradient characteristics of the image, so that the diffusion is weakened near the edge, thereby retaining the edge information. The image processed in this way can more clearly show the contour trend of the facial muscles. The generated contour trend map can provide important information for the subsequent image enhancement model, helping the model to more accurately identify and process the convex and concave areas of the facial muscles. The advantage of this method is that it can automatically extract key muscle contour information from the original image, provide accurate guidance for image enhancement, and thus improve the efficiency and effect of image processing.

[0071] In this embodiment, the image is fused with the enhanced image based on the enhanced region mask, and the result image is output. The fusion is performed using the following formula:

[0072] I f =M⊙I1+(1-M)⊙I2

[0073] In the formula, I f represents the result image, M represents the mask, and its value range is [0, 1]. I1 represents the image, and I2 represents the enhanced image. The two images I1 and I2 are weighted combined according to the mask M. I1 is the original image, I2 is the enhanced image, and M is the enhanced area mask. Mask M is a binary image, and its value range is [0, 1]. Each pixel value in the mask determines whether the pixel at the corresponding position is selected from I1 or I2. If a pixel value in the mask M is 1, then the result image I1 is f The corresponding pixels in will come entirely from I1 (the original image). If a pixel value in mask M is 0, then the result image I f The corresponding pixels in will come entirely from I2 (the enhanced image). The operation ⊙ represents an element-by-element multiplication, also known as the Hadamard product. This method allows the degree of fusion between the two images to be flexibly controlled by adjusting the value of the mask M, which can accurately locate the area that needs to be selected from the original image I1 and the area that needs to be retained in the enhanced image I2. Since the value range of the mask M is continuous, this method can produce a natural and smooth transition effect, especially in the mask edge area.

[0074] In this embodiment, a loss value is calculated based on the result image and the original image, and the model is trained based on the loss value. Specifically:

[0075] Calculate the VGG loss for the result image and the original image to obtain the first loss value;

[0076] Calculate the GAN loss for the result image and the original image to obtain the second loss value;

[0077] The model is trained based on the first loss value and the second loss value.

[0078] In this embodiment, VGG Loss (perceptual loss, used in this case to measure the content difference of the image): the first loss value is obtained by calculating VGG loss on the result image and the original image. VGG loss is a feature extraction method based on deep learning, which calculates the loss by comparing the feature representations of two images in the pre-trained VGG network. GANLoss (adversarial loss, used in this case to measure the generation quality of the image): the second loss value is obtained by calculating GAN loss on the result image and the original image. GANloss is usually used to train generative adversarial networks (GANs), in which the discriminator is used to distinguish between real images and generated images, and the generator tries to generate images that can deceive the discriminator. Through this adversarial process, the quality of the generated image can be improved to make it closer to the real image. Combining these two loss values ​​can simultaneously consider the perceptual quality and generation quality of the image, thereby optimizing the model more comprehensively. During the training process, the parameters of the model will be adjusted according to the gradients of the two losses to minimize the difference between the result image and the original image and improve the generation effect of the model. This method of combining multiple loss functions can improve the robustness and adaptability of the model, enabling it to achieve good performance in different application scenarios.

[0079] In this embodiment, a loss value is calculated based on the result image and the original image, and the model is trained based on the loss value. Specifically:

[0080] Calculate the VGG loss for the result image and the original image to obtain the first loss value;

[0081] Calculate the GAN loss for the result image and the original image to obtain the second loss value;

[0082] The contrast value of the corresponding area in the result image is calculated based on the mask of the concave and convex area of ​​the facial features and the mask of the concave and convex area of ​​the facial contour;

[0083] The model is trained based on the first loss value, the second loss value, and the contrast value.

[0084] In this embodiment, based on the Mask of the concave and convex areas of the facial features and the Mask of the concave and convex areas of the facial contour, the contrast value of the corresponding area in the result image is calculated. This step is to ensure that the model not only generates an image similar to the original image as a whole, but also maintains or enhances the contrast in the local area, thereby improving the visual impact and three-dimensional sense of the image. Combining the above three indicators for training the model, this method can ensure that the model is optimized in many aspects, including overall image quality, local contrast enhancement, and adversarial generation capabilities, so that the model can learn more comprehensively how to generate high-quality result images. By considering the perceptual quality, generation quality, and local contrast of the image, the model can more accurately capture and enhance the key features of the image; and it helps to balance the training process of the model, avoid overfitting, and improve the adaptability of the model to different types of images, which helps to generate images that are more visually realistic, have stronger contrast, and are rich in details.

[0085] In order to better understand the present invention, each step will be described below:

[0086] See also Figure 2 , model training steps are detailed:

[0087] During the training process, two models were trained, namely bone segmentation model M1 and enhancement model M2.

[0088] Both models are ResNET structures. They are open source models. The input and output are modified to match the process requirements. It should be noted that the specific modification method can be set according to actual needs, and will not be explained here.

[0089] Input image A: Input a facial image, which will be sent to the bone segmentation model M1 after preprocessing. Preprocessing refers to the face being turned straight, aligned and cropped to the same standard.

[0090] Bone segmentation model M1: The M1 model receives image A and outputs two masks: a1 and a2. a1 is guided by the three-dimensional part of the facial features, and a2 is guided by the three-dimensional part of the outer contour.

[0091] Input enhancement model M2: Input a1, a2 and original image A into the enhancement model M2.

[0092] Enhanced model M2 output: The M2 model outputs two parts of results:

[0093] C: The enhanced facial image has increased three-dimensionality.

[0094] D: The corresponding enhanced area Mask.

[0095] Fusion module: Use Mask D to fuse the enhanced image C with the original image A to obtain the final result image R.

[0096] Loss calculation:

[0097] VGG Loss: Calculate VGG loss for the result image R and the original image A to measure the content difference of the image.

[0098] GAN Loss: Calculate the GAN loss for the result image R and the original image A to measure the quality of image generation.

[0099] Gradient feedback and model optimization: The calculated loss gradient is fed back to the M1 and M2 models for optimization.

[0100] Model saving: When the effect is good and the loss tends to be stable, save models M1 and M2 for subsequent reasoning.

[0101] Through the above steps, the present invention can effectively enhance the facial image to make it more three-dimensional, and the training method can ensure the effect and stability of the model.

[0102] See also Fig. 9 , the actual reasoning steps are described in detail:

[0103] During the training process, two models were trained, namely the bone segmentation model M1 and the enhancement model M2, which will be used in the reasoning process.

[0104] Input image: Input an image A with a relatively flat face (see Figure 3 ).

[0105] Bone segmentation model (M1):

[0106] Input: Image A.

[0107] Output: Two Masks, a1 (see Figure 4 ) and a2 (see Figure 5 ).

[0108] a1: The part of the face that needs to be brightened.

[0109] a2: The part of the face that needs to be darkened.

[0110] Enhanced Model (M2):

[0111] Input: Image A and Mask a1, a2.

[0112] Output:

[0113] C: Image with face enhanced in three-dimensionality.

[0114] D: Mask of the enhanced area. (See Figure 6 )

[0115] Fusion Processing:

[0116] Use Mask D to fuse the result image C with the original image A to obtain the final result image R.

[0117] Difference calculation:

[0118] Calculate the difference between the original image A and the result image R, and get the difference image J (see Figure 7 ).

[0119] High resolution processing:

[0120] Upscale the difference map J to the high-resolution image K.

[0121] Apply the difference to the high-resolution image L to get the final high-resolution image (see Figure 8 ).

[0122] Embodiment 2

[0123] A model capable of enhancing the three-dimensionality of the face is trained by a training method for removing a model of a goof object as described in one of the embodiments. The model includes a bone segmentation model M1, an enhancement model M2, i.e., a fusion module

[0124] In this embodiment, the model is trained using the method described in one of the embodiments to achieve automated processing of facial images, which can quickly and accurately enhance the three-dimensional sense of the face while maintaining the natural beauty of the image. The model overcomes the limitations of previous manual three-dimensional enhancement methods and can significantly improve processing efficiency while maintaining high precision. It should be noted that the principles and effects of each step involved in the model have been described above and will not be described in detail here.

[0125] Embodiment 3

[0126] See also Fig.10 , a training device for removing a goof object model, comprising

[0127] Acquisition module, constructing a data set, the data set includes several person images and real labels corresponding to the images, the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend map;

[0128] The bone phase segmentation module is used to input the image and the facial muscle contour trend map into the bone phase segmentation model M1 for each image in the data set. The facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the facial features concave and convex area and the mask of the facial contour concave and convex area of ​​the corresponding image;

[0129] The enhancement module is used for inputting the image and the corresponding facial features concave-convex area mask and facial contour concave-convex area mask of each image into the enhancement model M2. The enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the facial features concave-convex area mask and the facial contour concave-convex area mask, and outputs an enhanced image and a corresponding enhanced area mask;

[0130] A fusion module is used for inputting the enhanced image and the enhanced region mask of each image into the fusion module, fusing the image with the enhanced image based on the enhanced region mask, and outputting a result image;

[0131] A training module is used to calculate a loss value based on the result image and the original image, and train the model based on the loss value.

[0132] In this embodiment, in order to better use the method described in one of the embodiments, the present application proposes a training device for removing the model of the object that is out of place. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0133] Embodiment 4

[0134] A training device for removing a goof object model, comprising:

[0135] at least one processor and a memory communicatively coupled to the at least one processor;

[0136] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0137] In this embodiment, in order to better run and process the method described in one of the embodiments, the above method is stored in a memory, and a processor is used to execute the stored method. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0138] Embodiment 5

[0139] A computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0140] In this embodiment, in order to better run and use the method described in one of the embodiments, the method is stored in a computer-readable storage medium and implemented by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0141] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A training method capable of enhancing a facial stereoscopic model, characterized in that: The model includes a bone phase segmentation model M1 and an enhancement model M2, i.e., a fusion module; and the method includes: Construct a data set, which includes several person images and real labels corresponding to the images, and the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend maps; For each image in the data set, the image and the facial muscle contour trend map are input into the bone phase segmentation model M1. The facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the facial features concave-convex area and the facial contour concave-convex area of ​​the corresponding image; the mask of the facial features concave-convex area and the facial contour concave-convex area of ​​the facial features concave-convex area are predicted. For each image, the mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour are input into the enhancement model M2 together. The enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the mask of the concave-convex area of ​​the facial features and the mask of the concave-convex area of ​​the facial contour, and outputs an enhanced image and the corresponding enhanced area mask; For each enhanced image and enhanced region mask of the image, the enhanced image and the enhanced region mask are input into the fusion module, the image and the enhanced image are fused based on the enhanced region mask, and the result image is output; A loss value is calculated based on the result image and the original image, and the model is trained based on the loss value.

2. A method for training a facial stereoscopic model as claimed in claim 1, characterized in that: The facial muscle contour trend map is obtained in the following way: Anisotropic filtering is used to obtain the muscle contour information and contour direction of the face of the character in the character image, and a contour trend map is generated.

3. A method for training a facial stereoscopic model as claimed in claim 1, characterized in that: The image is fused with the enhanced image based on the enhanced region mask, and the result image is output. The fusion is performed using the following formula: I f =M⊙I1+(1-M)⊙I2 In the formula, I f represents the result image, M represents the mask, and its value range is [0, 1], I1 represents the image, and I2 represents the enhanced image.

4. A method for training a facial stereoscopic model as claimed in claim 1, characterized in that: A loss value is calculated based on the result image and the original image, and the model is trained based on the loss value, specifically: Calculate VGGloss for the result image and the original image to obtain the first loss value; Calculate GANloss for the result image and the original image to obtain the second loss value; The model is trained based on the first loss value and the second loss value.

5. A method for training a facial stereoscopic model as claimed in claim 1, characterized in that: A loss value is calculated based on the result image and the original image, and the model is trained based on the loss value, specifically: Calculate VGGloss for the result image and the original image to obtain the first loss value; Calculate GANloss for the result image and the original image to obtain the second loss value; The contrast value of the corresponding area in the result image is calculated based on the mask of the concave and convex area of ​​the facial features and the mask of the concave and convex area of ​​the facial contour; The model is trained based on the first loss value, the second loss value, and the contrast value.

6. A model capable of enhancing the three-dimensionality of a face, characterized in that: The model is trained by a training method for removing bloopers as described in any one of claims 1 to 5; the model includes a bone phase segmentation model M1 and an enhancement model M2, i.e., a fusion module.

7. A training device for removing a goof object model, characterized in that: include Acquisition module, constructing a data set, the data set includes several person images and real labels corresponding to the images, the real labels include mask labels of concave and convex areas of facial features, mask labels of concave and convex areas of facial contours, and facial muscle contour trend map; The bone phase segmentation module is used to input the image and the facial muscle contour trend map into the bone phase segmentation model M1 for each image in the data set. The facial muscle contour trend map guides the bone phase segmentation model M1 to predict the mask of the facial features concave and convex area and the mask of the facial contour concave and convex area of ​​the corresponding image; The enhancement module is used for inputting the image and the corresponding facial features concave-convex area mask and facial contour concave-convex area mask of each image into the enhancement model M2. The enhancement model M2 performs corresponding brightening or darkening operations on the corresponding areas of the original image based on the facial features concave-convex area mask and the facial contour concave-convex area mask, and outputs an enhanced image and a corresponding enhanced area mask; A fusion module is used for inputting the enhanced image and the enhanced region mask of each image into the fusion module, fusing the image with the enhanced image based on the enhanced region mask, and outputting a result image; A training module is used to calculate a loss value based on the result image and the original image, and train the model based on the loss value.

8. A training device for removing a goof object model, characterized in that: include: at least one processor and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as claimed in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images