A method, device, equipment and storage medium for enhancing face image

Abstract

The present disclosure relates to a facial image enhancement method, device, equipment and storage medium, the method comprising: based on the facial attributes of the facial image to be enhanced, determining the corresponding facial category as the target category. From the image enhancement models corresponding to each facial category, determining the image enhancement model corresponding to the target category as the target model; each image enhancement model is trained based on the facial image samples of the facial category to which it corresponds. The facial image to be enhanced is input into the target model, and after enhancement processing, a facial enhanced image corresponding to the facial image to be enhanced is obtained. Since the present disclosure uses different image enhancement models to perform enhancement processing on facial images of different categories respectively, the image enhancement model can perform enhancement processing according to the image enhancement requirements of the facial images of the corresponding categories. For each facial image, a facial enhanced image that meets its enhancement requirements as much as possible can be obtained, thereby improving the facial image enhancement effect.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for enhancing facial images. Background Art

[0002] At present, image enhancement technology is widely used in the field of image and video processing, mainly including image denoising, sharpening, deblurring, color enhancement, super-resolution and other technologies. Image enhancement technology can enhance the entire image or only enhance a certain area in the image. Face image enhancement usually means first detecting the area where the face is located in the image, and then enhancing the area where the face is located separately.

[0003] In the current face image enhancement methods, the same image enhancement method is used for all face images. However, different categories of face images have different characteristics and may have different image enhancement processing requirements. If the same image enhancement method is used, it is obviously unable to meet the enhancement processing requirements of different categories of face images.

[0004] Therefore, how to achieve enhanced processing of face images of different categories is a technical problem that urgently needs to be solved. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a facial image enhancement method, device, equipment and storage medium.

[0006] In a first aspect, the present disclosure provides a method for enhancing a facial image, the method comprising:

[0007] Based on the facial attributes of the facial image to be enhanced, determining the face category corresponding to the facial image to be enhanced as the target category;

[0008] Determine, from the image enhancement models corresponding to the respective face categories, the image enhancement model corresponding to the target category as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself;

[0009] The face image to be enhanced is input into the target model, and after being enhanced by the target model, a face enhanced image corresponding to the face image to be enhanced is obtained.

[0010] In an optional implementation manner, determining, based on the facial attributes of the facial image to be enhanced, the face category corresponding to the facial image to be enhanced as the target category includes:

[0011] Inputting the face image to be enhanced into the image classification model, and obtaining the classification result corresponding to the face image to be enhanced after classification processing by the image classification model; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples;

[0012] Based on the classification result, a face category corresponding to the face image to be enhanced is determined as a target category.

[0013] In an optional implementation, before determining the image enhancement model corresponding to the target category from the image enhancement models corresponding to the various face categories as the target model, the method further includes:

[0014] Constructing a face image sample set corresponding to the first face category; the face image sample set includes noise-free face image samples and noisy face image samples;

[0015] The preset machine model is trained using the face image samples corresponding to the first face category to obtain an image enhancement model corresponding to the first face category.

[0016] In an optional implementation manner, the step of constructing a face image sample set corresponding to the first face category includes:

[0017] Obtain a noise-free face image sample corresponding to the first face category;

[0018] Degrading the noise-free face image samples to obtain noisy face image samples corresponding to the first face category;

[0019] Based on the noisy face image samples and the noise-free face image samples, a face image sample set corresponding to the first face category is constructed.

[0020] In an optional implementation, the step of degrading the noise-free face image samples to obtain noisy face image samples corresponding to the first face category includes:

[0021] Determine, from the degradation processing methods corresponding to the respective face categories, a degradation processing method corresponding to the first face category; wherein the degradation processing method includes at least one method of increasing image noise;

[0022] The noise-free face image samples are degraded using a degradation method corresponding to the first face category to obtain noisy face image samples corresponding to the first face category.

[0023] In an optional implementation, the step of inputting the face image to be enhanced into the image classification model and obtaining the classification result corresponding to the face image to be enhanced after the image classification model has been classified, further includes:

[0024] De-noising the face image to be enhanced with a preset de-noising strength to obtain a de-noised face image;

[0025] Accordingly, the face image to be enhanced is input into the image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained, including:

[0026] The denoised face image is input into an image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained.

[0027] In an optional implementation, the facial attributes include at least one of age, gender, appearance, and race.

[0028] In a second aspect, the present disclosure provides a facial image enhancement device, the device comprising:

[0029] A first determination module, configured to determine, based on the facial attributes of the facial image to be enhanced, a face category corresponding to the facial image to be enhanced as a target category;

[0030] A second determination module is used to determine, from the image enhancement models corresponding to the respective face categories, an image enhancement model corresponding to the target category as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself;

[0031] The enhancement processing module is used to input the face image to be enhanced into the target model, and obtain a face enhanced image corresponding to the face image to be enhanced after the enhancement processing of the target model.

[0032] In an optional implementation manner, the first determining module includes:

[0033] A classification submodule, used to input the face image to be enhanced into the image classification model, and obtain the classification result corresponding to the face image to be enhanced after the classification processing of the image classification model; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples;

[0034] The first determination submodule is used to determine, based on the classification result, a face category corresponding to the face image to be enhanced as a target category.

[0035] In an optional implementation, the device further includes:

[0036] A construction module, used to construct a face image sample set corresponding to the first face category; the face image sample set includes noise-free face image samples and noisy face image samples;

[0037] The training module is used to train a preset machine model using a face image sample set corresponding to the first face category to obtain an image enhancement model corresponding to the first face category.

[0038] In an optional implementation manner, the building block includes:

[0039] An acquisition submodule, used to acquire a noise-free face image sample corresponding to a first face category;

[0040] A first degradation processing submodule, configured to perform degradation processing on the noise-free face image samples to obtain noisy face image samples corresponding to the first face category;

[0041] A construction submodule is used to construct a face image sample set corresponding to the first face category based on the noisy face image samples and the noise-free face image samples.

[0042] In an optional implementation manner, the first degradation processing submodule includes:

[0043] A second determination submodule is used to determine the degradation processing method corresponding to the first face category from the degradation processing methods corresponding to the various face categories; wherein the degradation processing method includes at least one method of increasing image noise;

[0044] The first degradation processing submodule is used to degrade the noise-free face image samples using the degradation processing method corresponding to the first face category to obtain noisy face image samples corresponding to the first face category.

[0045] In an optional implementation, the device further includes:

[0046] A denoising module, used for denoising the face image to be enhanced with a preset denoising intensity to obtain a denoised face image;

[0047] Accordingly, the classification submodule is specifically used for:

[0048] The denoised face image is input into an image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained.

[0049] In an optional implementation, the facial attributes include at least one of age, gender, appearance, and race.

[0050] In a third aspect, the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements any of the methods described above.

[0051] In a fourth aspect, the present disclosure provides a device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the methods described above is implemented.

[0052] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0053] The disclosed embodiment provides a method for enhancing a facial image. First, based on the facial attributes of a facial image to be enhanced, a facial category corresponding to the facial image to be enhanced is determined as a target category. Then, from the image enhancement models corresponding to each facial category, an image enhancement model corresponding to the target category is determined as a target model; wherein each image enhancement model is trained based on a facial image sample of the facial category corresponding to itself. Finally, the facial image to be enhanced is input into the target model, and after being enhanced by the target model, a facial enhanced image corresponding to the facial image to be enhanced is obtained. The disclosed embodiment uses different image enhancement models to enhance facial images of different categories respectively. Since the image enhancement models of various categories are trained based on the facial image sample sets of the corresponding categories, the image enhancement models can enhance the image enhancement requirements of the facial images of the corresponding categories. For each facial image, the disclosed embodiment can obtain a facial enhanced image that meets its enhancement requirements as much as possible, thereby improving the facial image enhancement effect and avoiding the problem of poor generalization of facial image enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1A flowchart of a facial image enhancement method provided by an embodiment of the present disclosure;

[0057] Figure 2 A degradation processing flow chart provided in an embodiment of the present disclosure;

[0058] Figure 3 A schematic diagram of the structure of a facial image enhancement device provided in an embodiment of the present disclosure;

[0059] Figure 4 A schematic diagram of the structure of a facial image enhancement device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0062] A face image is an image that shows a face and the displayed face occupies a large proportion of the entire image. Face image enhancement refers to enhancing the quality of face images and improving the display effect of face images by performing enhancement processing such as denoising, sharpening, deblurring, color enhancement, and detail generation on face images. Face image enhancement is also known as face image restoration.

[0063] Face image enhancement is usually used to repair faces in old photos and videos or faces in photos and videos with poor face quality. Face image enhancement methods can currently be implemented based on network models. Specifically, a large number of face image samples are used to train the network model to obtain a trained network model, and then the network model is used to enhance the face image to obtain an enhanced face image.

[0064] However, in the current face image enhancement methods, since the network model is trained based on face image samples of various categories, the face image enhancement processing method learned by the trained network model is a method of enhancing the common features of face images of various categories. For example, for male face images and female face images, the current face image enhancement method uses the same enhancement processing method for both, but in fact, men and women have different needs for face enhancement processing. Women may not only need whitening, filters, acne removal, and freckle removal, but also need to adjust the shape and angle of the face, eyes, nose, etc., while men have relatively fewer needs for face processing. Therefore, using the same network model to perform the same enhancement processing on face images of various categories, it is obvious that the generalization of face image enhancement is not good, and the expected enhancement effect cannot be achieved for face images of various categories.

[0065] Therefore, the present disclosure provides a method for enhancing a facial image. First, based on the facial attributes of the facial image to be enhanced, the facial category corresponding to the facial image to be enhanced is determined as the target category. Then, from the image enhancement models corresponding to each facial category, the image enhancement model corresponding to the target category is determined as the target model; wherein each image enhancement model is trained based on the facial image sample of the facial category corresponding to itself. Finally, the facial image to be enhanced is input into the target model, and after the enhancement processing of the target model, the facial enhanced image corresponding to the facial image to be enhanced is obtained. The embodiment of the present disclosure uses different image enhancement models to enhance facial images of different categories. Since the image enhancement models of various categories are trained based on the facial image sample sets of the corresponding categories, the facial images of the corresponding categories are enhanced using the image enhancement models of the corresponding categories. For each facial image, a facial enhanced image that meets its enhancement requirements as much as possible can be obtained, thereby improving the enhancement effect of the facial image and avoiding the problem of poor generalization of facial image enhancement.

[0066] Based on this, the present disclosure provides a method for enhancing a face image. Figure 1 , Figure 1 A flow chart of a facial image enhancement method provided in an embodiment of the present disclosure, wherein the facial image enhancement method includes:

[0067] S101: Based on the face attributes of the face image to be enhanced, determine the face category corresponding to the face image to be enhanced as the target category.

[0068] In the embodiment of the present disclosure, before determining the face category corresponding to the face image to be enhanced, the face image to be enhanced is first divided into face categories based on face attributes. The face attributes may include gender, age, race, appearance, etc. That is, the embodiment of the present disclosure may divide the face image to be enhanced into face categories based on at least one attribute of gender, age, race, and appearance.

[0069] Taking facial attributes including gender and age as an example, facial images can be divided into two categories according to gender and male and female, and can be divided into three categories according to age, namely children and teenagers, young and middle-aged people, and the elderly. For children and the elderly, age is more characteristic than gender, so there is no need to distinguish gender. Therefore, combining the attributes of gender and age, facial images can be determined into four categories, namely children and teenagers, young and middle-aged men, young and middle-aged women, and the elderly.

[0070] In the disclosed embodiment, based on the above-mentioned face category classification method and based on the face attributes of the face image to be enhanced, the face category corresponding to the face image to be enhanced can be determined as the target category.

[0071] In an optional implementation, a deep learning method may be used to determine the face category corresponding to the face image to be enhanced. Specifically, an image classification model may be used to classify the face image to be enhanced to obtain a classification result, and then the face category corresponding to the face image to be enhanced is determined based on the classification result. Specifically, the face image to be enhanced is first input into the image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples. Then, based on the classification result, the face category corresponding to the face image to be enhanced is determined as the target category.

[0072] Prior to this, first construct face image training samples with category labels for training the image classification model. Then, use a large number of face image training samples with category labels to train the image classification model to complete the training of the image classification model. The image classification model can be implemented based on Resnet50 residual network and the like.

[0073] In another optional implementation, feature analysis may be performed on the face image to be enhanced based on the face attributes of the face image to be enhanced to determine the face category corresponding to the face image to be enhanced.

[0074] In order to improve the accuracy of determining the face category corresponding to the face image to be enhanced, the embodiment of the present disclosure may perform a slight image denoising process on the face image to be enhanced before determining the face category corresponding to the face image to be enhanced, so as to remove the noise with higher intensity on the face image to be enhanced, so as to avoid the influence of the noise with higher intensity on the face category determination of the face image to be enhanced. Specifically, the face image to be enhanced is denoised with a preset denoising intensity. Among them, the preset denoising intensity can be set to a weaker denoising intensity. Taking the NL-Mean (non-local mean) denoising algorithm as an example, when the NL-Mean denoising algorithm is used to denoise the face image to be enhanced, the function of the fastNlMeansDenoisingColored function in opencv is to denoise the color image, and the filter strength parameters h and hForColorComponents are recommended to be set to about 3.

[0075] In practical applications, the face image to be enhanced is subjected to denoising processing with a preset denoising intensity, and after obtaining the denoised face image, the denoised face image is input into the image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained. Since the embodiment of the present disclosure performs slight image denoising processing on the face image to be enhanced, it is possible to remove the strong noise on the face image to be enhanced, thereby avoiding the influence of the strong noise on the face category determination of the face image to be enhanced, and improving the accuracy of the face category determination.

[0076] S102: Determine, from the image enhancement models corresponding to the respective face categories, an image enhancement model corresponding to the target category as the target model; wherein each image enhancement model is trained based on face image samples of the face category corresponding to itself.

[0077] In practical applications, before using the image enhancement model to enhance the face image to be enhanced, first, for each face category, a corresponding image enhancement model is constructed, and face image samples corresponding to each category are constructed respectively. Then, the face image samples corresponding to each category are used to train the image enhancement model of the corresponding category.

[0078] In an optional implementation, a large number of face images can be collected for constructing face image samples. Specifically, for each face image collected, the face category corresponding to the face image is first detected, such as belonging to the category of young or middle-aged males, and then the face image is used to construct a face image sample set of the face category.

[0079] In order to improve the processing capability of the image enhancement model and thereby improve the enhancement effect on facial images, the disclosed embodiment can construct a high-quality facial image sample set and a low-quality facial image sample set for each category, respectively, and use the high-quality facial image sample set and the low-quality facial image sample set to train the image enhancement model simultaneously, so that the trained image enhancement model can also obtain a good enhancement effect on low-quality facial images.

[0080] In practical applications, high-quality face images refer to high-definition face images with less noise and clearer images, that is, noise-free face image samples. After obtaining the high-quality face image sample set corresponding to each category, the high-quality face image sample set can be used to construct the low-quality face image sample set corresponding to the category.

[0081] In an optional implementation, in a method of constructing a low-quality face image sample set of a corresponding category based on the high-quality face image sample set of each category, the face image samples in the high-quality face image sample set of each category are downgraded to obtain degraded face image samples, which are used to construct the low-quality face image sample set of that category, i.e., noisy face image samples.

[0082] Since different categories of face images have different requirements for face enhancement effects, the disclosed embodiments use different degradation processing methods for different categories of high-quality face image sample sets. After determining the degradation processing methods corresponding to each face category, the disclosed embodiments can determine the corresponding degradation processing methods from the degradation processing methods corresponding to each face category, and then use the corresponding degradation processing methods to degrade the face image samples in the high-quality face image sample sets of each category, and finally obtain the low-quality face image sample sets corresponding to each category.

[0083] Taking the first face category as an example, the degradation processing method corresponding to the first face category is determined from the degradation processing methods corresponding to each face category; wherein the degradation processing method includes at least one method of increasing image noise, and then, the noise-free face image sample is degraded using the degradation processing method corresponding to the first face category to obtain the noisy face image sample corresponding to the first face category. Then, based on the noisy face image samples and the noise-free face image samples, a face image sample set corresponding to the first face category is constructed. The face image sample set includes noise-free face image samples and noisy face image samples. Finally, the face image samples corresponding to the first face category are used to train the preset machine model to obtain the image enhancement model corresponding to the first face category.

[0084] Take the four categories of young children, young and middle-aged men, young and middle-aged women, and the elderly as examples. Figure 2As shown, it is a flowchart of degradation processing provided by an embodiment of the present disclosure. Among them, since the facial images of the young and middle-aged male categories do not have high requirements for the enhancement effect, therefore, when generating a low-quality facial image sample set, the image samples in the high-quality facial image sample set of the young and middle-aged male categories are first resized, that is, the image samples are downsampled to low-resolution images, and then resampled back to the original resolution images. Then, the image samples that have been resized are processed with Jpeg strong noise, that is, high-intensity Jpeg noise is added to the image samples to obtain image samples processed with Jpeg noise, which are used to construct a low-quality facial image sample set of the young and middle-aged male categories. For the image samples in the high-quality facial image sample set of the middle-aged male categories, the resize processing is first performed, and then the image samples that have been resized are processed with Jpeg medium-intensity noise to obtain image samples processed with Jpeg noise, which are used to construct a low-quality facial image sample set of the middle-aged male categories.

[0085] Since the facial images of young and middle-aged women and the elderly have higher requirements for the enhancement effect, more degradation processing steps are required when generating a low-quality facial image sample set. Specifically, for the image samples in the high-quality facial image sample set of young and middle-aged women, the image samples are first resized and then Gaussian noise processed so that the trained image enhancement model can remove the acne on the face, and then Jpeg strong noise processing is performed, and finally the image samples processed with Jpeg noise are obtained, which are used to construct the low-quality facial image sample set of young and middle-aged women. Similarly, for the image samples in the high-quality facial image sample set of the elderly category, the image samples are first resized and then Gaussian blur processed so that the trained image enhancement model can remove the wrinkles on the face, and then medium-intensity Jpeg noise processing is performed, and finally the image samples processed with Jpeg noise are obtained, which are used to construct the low-quality facial image sample set of the elderly category.

[0086] In another optional implementation, low-quality facial image sample sets of each category may also be obtained by directly collecting low-quality facial images. Other methods of obtaining low-quality facial image sample sets will not be described in detail in the embodiments of the present disclosure.

[0087] In the disclosed embodiment, after obtaining high-quality face image sample sets and low-quality face image sample sets of each category, the image enhancement models corresponding to the corresponding categories are trained using the high-quality face image sample sets and low-quality face image sample sets of each category to obtain trained image enhancement models corresponding to each category. After determining the face category corresponding to the face image to be enhanced as the target category, the image enhancement model corresponding to the target category is determined from the image enhancement models corresponding to each face category as the target model for performing enhancement processing on the face image to be enhanced.

[0088] S103: Inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after being enhanced by the target model.

[0089] In the embodiment of the present disclosure, after determining the face category corresponding to the face image to be enhanced, further determine the image enhancement model corresponding to the face category, that is, the target model, and then use the target model to perform image enhancement processing on the face image to be enhanced. The image enhancement model can be implemented by using various deep learning models.

[0090] Among them, image enhancement processing is performed on the face image to be enhanced, and enhancement processing such as denoising, sharpening, deblurring, color enhancement, and detail generation can be performed on the face image to be enhanced to enhance the quality of the face image to be enhanced and improve the display effect of the face image to be enhanced.

[0091] In the face image enhancement method provided by the embodiment of the present disclosure, first, based on the face attributes of the face image to be enhanced, the face category corresponding to the face image to be enhanced is determined as the target category. Then, from the image enhancement models corresponding to each face category, the image enhancement model corresponding to the target category is determined as the target model; wherein each image enhancement model is trained based on the face image sample of the face category corresponding to itself. Finally, the face image to be enhanced is input into the target model, and after the enhancement processing of the target model, the face enhanced image corresponding to the face image to be enhanced is obtained. The embodiment of the present disclosure uses different image enhancement models to enhance face images of different categories. Since the image enhancement models of various categories are trained based on the face image sample set of the corresponding category, the face images of the corresponding category are enhanced by using the image enhancement model of the corresponding category. For each face image, a face enhanced image that meets its enhancement requirements as much as possible can be obtained, thereby improving the enhancement effect of the face image and avoiding the problem of poor generalization of face image enhancement.

[0092] Corresponding to the above method implementation, the present disclosure also provides a face image enhancement device, referring to Figure 3, is a schematic diagram of the structure of a face image enhancement device provided by an embodiment of the present disclosure, the device comprising:

[0093] A first determination module 301 is used to determine the face category corresponding to the face image to be enhanced as the target category based on the face attributes of the face image to be enhanced;

[0094] The second determination module 302 is used to determine the image enhancement model corresponding to the target category from the image enhancement models corresponding to the face categories, as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself;

[0095] The enhancement processing module 303 is used to input the face image to be enhanced into the target model, and obtain a face enhanced image corresponding to the face image to be enhanced after the enhancement processing of the target model.

[0096] In an optional implementation, a machine model may be used to classify the face image to be enhanced to improve the accuracy of the classification result. Therefore, the first determination module includes:

[0097] A classification submodule, used to input the face image to be enhanced into the image classification model, and obtain the classification result corresponding to the face image to be enhanced after the classification processing of the image classification model; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples;

[0098] The first determination submodule is used to determine, based on the classification result, a face category corresponding to the face image to be enhanced as a target category.

[0099] In an optional implementation, the machine model can be trained based on noisy face image samples and noise-free face image samples at the same time to obtain a better image enhancement model. Therefore, the device also includes:

[0100] A construction module, used to construct a face image sample set corresponding to the first face category; the face image sample set includes noise-free face image samples and noisy face image samples;

[0101] The training module is used to train a preset machine model using a face image sample set corresponding to the first face category to obtain an image enhancement model corresponding to the first face category.

[0102] In an optional implementation, a noisy image sample can be constructed based on a noise-free face image sample. To this end, the construction module includes:

[0103] An acquisition submodule, used to acquire a noise-free face image sample corresponding to a first face category;

[0104] A first degradation processing submodule, configured to perform degradation processing on the noise-free face image samples to obtain noisy face image samples corresponding to the first face category;

[0105] A construction submodule is used to construct a face image sample set corresponding to the first face category based on the noisy face image samples and the noise-free face image samples.

[0106] In an optional implementation, in order to improve the processing capability of the image enhancement model, the first degradation processing submodule includes:

[0107] A second determination submodule is used to determine the degradation processing method corresponding to the first face category from the degradation processing methods corresponding to the various face categories; wherein the degradation processing method includes at least one method of increasing image noise;

[0108] The first degradation processing submodule is used to degrade the noise-free face image samples using the degradation processing method corresponding to the first face category to obtain noisy face image samples corresponding to the first face category.

[0109] In an optional implementation manner, in order to improve the accuracy of the classification result, the device further includes:

[0110] A denoising module, used for denoising the face image to be enhanced with a preset denoising intensity to obtain a denoised face image;

[0111] Accordingly, the classification submodule is specifically used for:

[0112] The denoised face image is input into an image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained.

[0113] In an optional implementation, the facial attributes include at least one of age, gender, appearance, and race.

[0114] In the face image enhancement device provided by the embodiment of the present disclosure, first, based on the face attributes of the face image to be enhanced, the face category corresponding to the face image to be enhanced is determined as the target category. Then, from the image enhancement models corresponding to each face category, the image enhancement model corresponding to the target category is determined as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself. Finally, the face image to be enhanced is input into the target model, and after the enhancement processing of the target model, the face enhanced image corresponding to the face image to be enhanced is obtained. Since the embodiment of the present disclosure uses different image enhancement models to perform enhancement processing on face images of different categories respectively, the image enhancement model can perform enhancement processing according to the image enhancement requirements of the face images of the corresponding categories. For each face image, the embodiment of the present disclosure can obtain a face enhanced image that meets its enhancement requirements as much as possible, thereby improving the face image enhancement effect.

[0115] In addition, the present disclosure also provides a face image enhancement device, see Figure 4 As shown, it may include:

[0116] Processor 401, memory 402, input device 403 and output device 404. The number of processors 401 in the face image enhancement device can be one or more. Figure 4 In some embodiments of the present invention, the processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means, wherein: Figure 4 The example of connecting through bus is taken in the following.

[0117] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing of the face image enhancement device by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The input device 403 can be used to receive input digital or character information, and generate signal input related to user settings and function control of the face image enhancement device.

[0118] Specifically in this embodiment, the processor 401 will load the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 will run the applications stored in the memory 402, thereby realizing the various functions of the above-mentioned face image enhancement device.

[0119] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0120] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A facial image enhancement method, characterized in that: The method comprises: Based on the facial attributes of the facial image to be enhanced, determining the face category corresponding to the facial image to be enhanced as the target category; Determine, from the image enhancement models corresponding to the respective face categories, the image enhancement model corresponding to the target category as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself; Inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after being enhanced by the target model; Before determining the image enhancement model corresponding to the target category from the image enhancement models corresponding to the face categories as the target model, the method further includes: Constructing a face image sample set corresponding to the first face category; the face image sample set includes noise-free face image samples and noisy face image samples; Using the face image samples corresponding to the first face category, training a preset machine model to obtain an image enhancement model corresponding to the first face category; The step of constructing a face image sample set corresponding to the first face category includes: Obtain a noise-free face image sample corresponding to the first face category; Degrading the noise-free face image samples to obtain noisy face image samples corresponding to the first face category; Based on the noisy face image samples and the noise-free face image samples, construct a face image sample set corresponding to the first face category; The step of degrading the noise-free face image samples to obtain noisy face image samples corresponding to the first face category includes: Determine, from the degradation processing methods corresponding to the respective face categories, a degradation processing method corresponding to the first face category; wherein the degradation processing method includes at least one method of increasing image noise; The noise-free face image samples are degraded using a degradation method corresponding to the first face category to obtain noisy face image samples corresponding to the first face category.

2. The method according to claim 1, characterized in that The step of determining, based on the face attributes of the face image to be enhanced, a face category corresponding to the face image to be enhanced as a target category includes: Inputting the face image to be enhanced into the image classification model, and obtaining the classification result corresponding to the face image to be enhanced after classification processing by the image classification model; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples; Based on the classification result, a face category corresponding to the face image to be enhanced is determined as a target category.

3. The method according to claim 2, characterized in that The method further includes: inputting the face image to be enhanced into the image classification model, and obtaining the classification result corresponding to the face image to be enhanced after the image classification model has been classified and processed. De-noising the face image to be enhanced with a preset de-noising strength to obtain a de-noised face image; Accordingly, the face image to be enhanced is input into the image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained, including: The denoised face image is input into an image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained.

4. The method according to claim 1, characterized in that: The facial attributes include at least one of age, gender, appearance and race.

5. A facial image enhancement device, characterized in that: The device comprises: A first determination module, configured to determine, based on the facial attributes of the facial image to be enhanced, a face category corresponding to the facial image to be enhanced as a target category; A second determination module is used to determine, from the image enhancement models corresponding to the respective face categories, an image enhancement model corresponding to the target category as the target model; wherein each image enhancement model is trained based on the face image samples of the face category corresponding to itself; An enhancement processing module, used for inputting the face image to be enhanced into the target model, and obtaining a face enhanced image corresponding to the face image to be enhanced after the enhancement processing of the target model; The device also includes: A construction module, used to construct a face image sample set corresponding to the first face category; the face image sample set includes noise-free face image samples and noisy face image samples; A training module, configured to train a preset machine model using a face image sample set corresponding to the first face category to obtain an image enhancement model corresponding to the first face category; The building blocks include: An acquisition submodule, used to acquire a noise-free face image sample corresponding to a first face category; A first degradation processing submodule, configured to perform degradation processing on the noise-free face image samples to obtain noisy face image samples corresponding to the first face category; A construction submodule, configured to construct a face image sample set corresponding to the first face category based on the noisy face image samples and the noise-free face image samples; The first degradation processing submodule includes: A second determination submodule is used to determine the degradation processing method corresponding to the first face category from the degradation processing methods corresponding to the various face categories; wherein the degradation processing method includes at least one method of increasing image noise; The first degradation processing submodule is used to degrade the noise-free face image samples using the degradation processing method corresponding to the first face category to obtain noisy face image samples corresponding to the first face category.

6. The device according to claim 5, characterized in that The first determining module includes: A classification submodule, used to input the face image to be enhanced into the image classification model, and obtain the classification result corresponding to the face image to be enhanced after the classification processing of the image classification model; wherein the image classification model is trained based on face image samples with category labels, and the category labels are determined based on the face attributes of the face image samples; The first determination submodule is used to determine, based on the classification result, a face category corresponding to the face image to be enhanced as a target category.

7. The device according to claim 6, characterized in that The device also includes: A denoising module, used for denoising the face image to be enhanced with a preset denoising intensity to obtain a denoised face image; Accordingly, the classification submodule is specifically used for: The denoised face image is input into an image classification model, and after classification processing by the image classification model, a classification result corresponding to the face image to be enhanced is obtained.

8. The device according to claim 5, characterized in that The facial attributes include at least one of age, gender, appearance and race.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 4.

10. A device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

Images