A method, device and equipment for removing facial oiliness based on CNN

Abstract

The present invention discloses a method, device, equipment and storage medium for removing facial gloss based on CNN, which comprises: obtaining an input image to be processed; preprocessing the image to be processed to obtain a first face image; calculating a face mask and a noise mask of the first face image; inputting the first face image and the noise mask into a pre-trained gloss removal model to perform face gloss removal processing to obtain a target face image. Without complex parameter adjustment, the facial gloss of a portrait can be removed quickly and accurately, improving the visual perception effect of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device and equipment for removing facial oiliness based on CNN. Background Art

[0002] Staying up late, excessive fatigue and irregular work and rest schedules can lead to endocrine disorders and oily and greasy faces. Some specific circumstances (such as lighting) can also result in photos with oily faces. Based on the photos obtained above, when retouching the photos, the face of the portrait will have obvious oiliness and scattered highlights, affecting the overall beauty of the photo. How to say goodbye to oily faces and have a high-end texture with one click is a difficult problem that the industry wants to solve. Oiliness is mainly distributed on the forehead, cheeks, philtrum, chin, nose and around the corners of the eyes. The oiliness is spread out in large areas in flakes, with irregular shapes, uneven distribution, a liquid feel, and the highlight area is bright white.

[0003] At present, the solution for removing oily gloss from the face in the industry is based on the traditional algorithm, which detects the oily area of ​​the face through brightness statistics and then uses a fixed threshold to adjust the oily area of ​​the face. Using this type of method to remove oily gloss requires constantly adjusting the judgment threshold of the oily area according to different usage scenarios and different lighting environments. For complex scenarios in actual needs, the application of traditional algorithms requires setting different parameters for adjustment, which is cumbersome. Summary of the invention

[0004] In view of this, the purpose of the present invention is to propose a facial oily shine removal method, device and equipment based on CNN, aiming to solve the problem that the existing oily shine removal method needs to continuously adjust the judgment threshold according to different situations, resulting in cumbersome operation.

[0005] To achieve the above object, the present invention provides a method for removing facial oiliness based on CNN, the method comprising:

[0006] Get the input image to be processed;

[0007] Preprocessing the image to be processed to obtain a first face image;

[0008] Calculating a face mask and a noise mask of the first face image;

[0009] The first face image and the noise mask are input into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image.

[0010] Preferably, the preprocessing of the image to be processed to obtain the first face image includes:

[0011] Performing face point detection on the image to be processed, and calculating the face circumscribed frame of the image to be processed based on the face points;

[0012] An affine transformation matrix of the image to be processed is calculated according to the face circumscribed frame, and face correction and cropping are performed to obtain a first face image.

[0013] Preferably, calculating the face mask and the noise mask of the first face image includes:

[0014] Calculating the face mask of the first face image by acquiring face points of the first face image;

[0015] According to the face mask, the noise mask having the same size as the face mask is randomly generated from the area in the first face image where the pixel value is greater than 0.

[0016] Preferably, the step of inputting the first face image and the noise mask into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image includes:

[0017] The corresponding inverse matrix is ​​calculated based on the affine transformation matrix, and the result image obtained after the oily face removal processing in the oily face removal model is mapped to the image to be processed through the inverse matrix to obtain the target face image.

[0018] Preferably, the method further comprises:

[0019] Determine whether the facial lighting type of the first facial image is an oily type.

[0020] Preferably, the oil-removing model is constructed based on a GAN network, and the oil-removing model includes a generation network and a discrimination network, wherein the generation network is constructed using a coding and decoding structure, and the discrimination network includes a multi-scale discriminator.

[0021] Preferably, the oil-removing model includes a texture protection module provided in the generation network for extracting facial texture of a face area in an image.

[0022] To achieve the above object, the present invention also provides a facial oily shine removal device based on CNN, the device comprising:

[0023] An acquisition unit, used for acquiring an input image to be processed;

[0024] A preprocessing unit, used for preprocessing the image to be processed to obtain a first face image;

[0025] A calculation unit, used for calculating a face mask and a noise mask of the first face image;

[0026] The oil-glare removal unit is used to input the first face image and the noise mask into a pre-trained oil-glare removal model to perform face oil-glare removal processing to obtain a target face image.

[0027] In order to achieve the above objectives, the present invention also proposes a device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a CNN-based facial oiliness removal method as described in the above embodiment.

[0028] In order to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a CNN-based facial oiliness removal method as described in the above embodiment.

[0029] Beneficial effects:

[0030] The above scheme calculates the first face image obtained after preprocessing the image to be processed, obtains the face mask and the noise mask, and inputs them into the pre-trained degreasing model to remove the face greasy light, and obtains the target face image. The degreasing model is trained based on the CNN network structure, and does not require complex parameter adjustment. It can quickly and accurately remove the facial greasy light of the portrait and improve the visual perception effect of the picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of a process of removing facial oiliness based on CNN provided by one embodiment of the present invention.

[0033] Figure 2 A schematic diagram of a face mask provided by an embodiment of the present invention.

[0034] Figure 3 A schematic diagram of a noise mask provided by an embodiment of the present invention.

[0035] Figure 4 A schematic diagram of the structure of a generation network provided in one embodiment of the present invention.

[0036] Figure 5 A schematic diagram of a network structure of a generation network provided in one embodiment of the present invention.

[0037] Figure 6 A schematic structural diagram of a facial oily shine removal device based on CNN provided in one embodiment of the present invention.

[0038] The realization of the purpose, functional features and advantages of the invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0041] The content of the present invention is described in detail below with reference to embodiments.

[0042] The existing solution for removing oiliness from the face is not perfect and still has the following difficulties:

[0043] (1) Although some existing solutions can remove facial gloss, in order to ensure that the texture of the glossy area of ​​the face is not lost, the de-greasing image is merged with the original image in the form of linear overlay. Although this can reduce the loss of texture details in the de-greasing area, it will cause the resulting image to not remove the gloss cleanly. (2) In addition, existing solutions cannot accurately distinguish between the highlight, matte and glossy areas of the face, resulting in the incorrect removal of highlight and matte areas. In summary, although existing photo editing software can remove gloss according to the actual situation of the user's picture, there are still cases where the gloss is not removed cleanly or the texture of the glossy area is lost after de-greasing or the skin color is excessively uneven. Currently, there is no method that can effectively remove the gloss from the face of the portrait while ensuring the texture of the face.

[0044] Based on this, the present invention proposes a CNN-based facial oiliness removal method, which does not require complex parameter adjustment and can quickly and accurately remove facial oiliness from portraits while ensuring skin color texture, ensuring good robustness in various complex scenes.

[0045] Reference Figure 1 The figure is a schematic flow chart of a facial oiliness removal method based on CNN provided by an embodiment of the present invention.

[0046] In this embodiment, the method includes:

[0047] S11, obtaining an input image to be processed.

[0048] S12, preprocessing the image to be processed to obtain a first face image.

[0049] S13, calculating a face mask and a noise mask of the first face image.

[0050] S14, inputting the first face image and the noise mask into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image.

[0051] In this embodiment, in order to improve the learning ability of the oil-removing model and the stability of the model, when preparing training data, the image is not directly used as the network input, but the image is preprocessed: including face detection, extraction of face points and face external frames, and straightening of the extracted face, so as to ensure that the input face is straight and the cropped face accounts for a higher proportion in the image, which can provide more effective information for the network. Specifically, the face point FcaePoint in the image Img is detected by using the currently mature CNN-based face detection and face alignment technology, and the maximum external moment of the face in the image is calculated based on the face point FcaePoint. The external moment is used to calculate the affine transformation matrix F of the face relative to the image Img, the corrected face image FaceImg is cropped from the image Img, and the corresponding face point FacePointWarp of the face image after rotation correction is obtained.

[0052] Furthermore, the face point FacePointWarp of the corrected face image is used to calculate the single-channel face mask FaceMask corresponding to the cropped face image. The role of the face mask FaceMask is to improve the training efficiency of the subsequent CNN network. The face mask FaceMask is as follows: Figure 2As shown. Further, for each area in the face image where the pixel value is greater than 0, a random noise mask NosieMask with the same scale as the face mask FaceMask is randomly generated. The noise mask NosieMask is as follows: Figure 3 In this embodiment, a random noise mask NosieMask generated based on the effective area of ​​the face mask FaceMask is added and concatenated with the input face image FaceImg as the fourth channel. The role of the noise mask is to improve the richness of the input model data, improve the robustness of the model, and improve the skin texture and clarity of the final model result, thereby ensuring that the output result will not have a smearing feeling.

[0053] Furthermore, the face image FaceImg is input into the gloss type recognition module for recognition processing to distinguish between highlight / matte / gloss types. By distinguishing images with gloss problems, paired supervision data can directly obtain corresponding gloss removal results, which has good robustness for various scenes.

[0054] The oil-removing model is constructed based on a GAN network, and the oil-removing model includes a generation network and a discrimination network. The generation network is constructed using a coding and decoding structure, and the discrimination network includes a multi-scale discriminator.

[0055] The oil-removing model includes a texture protection module in the generation network for extracting facial texture of a face area in an image.

[0056] In this embodiment, a general full convolutional network is constructed, which uses random resolution input in the scale range of 320-640 (which can effectively improve the network's ability to cope with images of different resolutions). The degreasing model includes a generative network and a discriminative network, wherein the generative network input channels are 4 channels, namely, the face image FaceImg cropped after rotation correction, and NosieMask generated based on the face mask FaceMask, and the two images are concatenated together as input. Among them, FaceImg and NosieMask need to be normalized to the range of (-1, 1).

[0057] Furthermore, the generative network adopts an encoding-decoding structure. The upsampling of the decoding part adopts a combination of nearest neighbor upsampling + convolutional layer, and the activation function of the output layer is Tanh, see Figure 4As shown. In order to improve the training efficiency of the model, the traditional generative network will add a normalization layer to the network structure. In order to improve the robustness of the model to different complex scenes, this embodiment cancels all BN layers and reduces the negative impact of fixed normalization values ​​on the effect. That is, by removing the BN layer, the problem of uneven color in the area after degreasing can be effectively improved.

[0058] Considering that the images processed by the traditional CNN-based de-glossing algorithm have unclear textures and excessive uneven glossy areas, this method adds a texture protection module (TPM) between the encoding and decoding layers, such as Figure 5 Schematic diagram of the network structure shown. The facial texture of the face area is a high-frequency global feature. The texture protection module can extract the global high-frequency features of the image. The scale of the front and back input feature maps of TPM remains unchanged and is combined with the local features before entering the decoding network. By adding a texture protection module and combining the global high-frequency features with the local features, the texture of the face area can be effectively protected to prevent excessive uneven skin color in the de-greasy area. This embodiment is applicable to any full convolution-generative adversarial network structure. Here, taking the network structure in Table 1 below as an example, a generative network is constructed. The input size of the network is 384, and the negative Slope of LeakyReLU is 0.3.

[0059] Furthermore, the discriminator network uses a multi-scale Discriminator to distinguish true and false images of different resolutions. This embodiment uses two scale discriminators to distinguish images of different resolutions, 384x384 and 192x192, and can be directly downsampled by MaxPooling.

[0060]

[0061]

[0062] Table 1

[0063] In addition, the training data is obtained by collecting 50,000 images of users of different ages, different facial postures, different lighting environments, and different genders (multi-pose training data such as side faces can effectively improve the robustness of the model). By collecting portrait data of users of different ages, different facial postures, and different genders with oily faces, the traditional oil removal algorithm is used to generate supervision data. 47,000 of them are training sets and 3,000 are test sets. In order to obtain a more robust network, multiple gains are added to the data during training, such as affine transformation, text or object occlusion, exposure, contrast, saturation adjustment, mosaic, etc.

[0064] Furthermore, Adam is used as the optimization algorithm for network training, the learning rate of the generator network is 0.0002, and the learning rate of the discriminator network is 0.0001. The total loss of the model training is expressed as:

[0065]

[0066] Among them, α, β, and γ represent the weights of the corresponding loss function. α=1, β=0.5, γ=0.5.

[0067]

[0068] Among them, tanh is the hyperbolic tangent processing, Output represents the network output image, and GroundTruth is the target image.

[0069] In addition, selecting L1_loss can ensure the color brightness of the image. L1_loss is a linear loss function. In order to ensure that the details of the high-frequency area of ​​the image are not lost, before calculating L1_loss, the output image Output and GroundTruth are respectively subjected to Tanh operation, which can ensure that the image has a higher weight in the high-frequency area. In addition, in order to ensure the perceptual similarity of the image, the perceptual loss based on VGG19 is introduced: L Perc ; And in order to ensure the authenticity of the image, the adversarial loss Gan_Loss is used to minimize the distance between the color estimation of the real image and the image color distribution output by the generated network. Among them, Loss function representing the discriminator output at two different resolutions.

[0070]

[0071] Among them, Ladv (Generation Adversarial Loss) is a common adversarial loss function, D represents the discriminator mentioned above, x represents real data (i.e., GroundTruth), z represents the data generated by the network, px represents the distribution of real data samples, and pz is the same. E represents expectation.

[0072] Furthermore, the preprocessing of the image to be processed to obtain a first face image includes:

[0073] S12-1, performing face point detection on the image to be processed, and calculating a face circumscribed frame of the image to be processed based on the face points;

[0074] S12-2, calculating the affine transformation matrix of the image to be processed according to the face circumscribed frame, and performing face correction and cropping to obtain a first face image.

[0075] Furthermore, the calculating the face mask and the noise mask of the first face image includes:

[0076] S13-1, calculating the face mask of the first face image by acquiring face points of the first face image;

[0077] S13-2, based on the face mask, randomly generate the noise mask of the same size as the face mask from the area in the first face image where the pixel value is greater than 0.

[0078] Furthermore, the step of inputting the first face image and the noise mask into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image includes:

[0079] The corresponding inverse matrix is ​​calculated based on the affine transformation matrix, and the result image obtained after the oily face removal processing in the oily face removal model is mapped to the image to be processed through the inverse matrix to obtain the target face image.

[0080] In this embodiment, the de-greasy model is obtained after the above training is completed. In specific applications, for an original user image, the above-mentioned data preprocessing method is first used to crop the corrected first face image, and the corresponding face mask is generated by obtaining the face points of the first face image, and then a random noise mask is generated according to the effective area of ​​the face mask, and the cropped first face image and the noise mask are cancate into 4 channels and input into the trained de-greasy model. The final output result of the network is the target face image after de-greasy.

[0081] Furthermore, after preprocessing the image to be processed to obtain the first face image, the method further includes:

[0082] Determine whether the facial lighting type of the first facial image is an oily type.

[0083] In this embodiment, facial highlights and matte are both normal conditions of facial makeup. Traditional algorithms for removing oily shine mostly directly process the oily areas of the face, but do not distinguish the oily shine from the highlight / matte types in the makeup, resulting in incorrect processing of the highlight / matte areas. This embodiment adds an oily shine type recognition module, and uses the currently more mature CNN-based classification method for the three types of facial lighting conditions of oily shine / matte / highlight to identify and process the cropped first face image, that is, it determines that the face of the first face image is of the oily type before processing it through the oily shine removal model. This can effectively distinguish the highlight / matte / oily shine types, prevent incorrect processing, and effectively remove the oily shine.

[0084] In summary, this embodiment can effectively distinguish between highlight / matte / oily types and effectively remove oiliness by designing a full-convolution network-based oiliness removal model and a special training process. It is an end-to-end facial oiliness removal method. In addition, the traditional method uses Gaussian or other low-pass filtering schemes to process the oily area and then superimpose it back to the original image, which will cause the loss of texture details in the oily area and excessive uneven skin color. The present solution uses the full face image plus the noise image as input, and directly outputs the result image without superimposing the original image, thereby avoiding problems such as unclean oiliness removal. In addition, a texture protection module TPM is added to the middle part of the encoding-decoding network, and the global high-frequency texture features of the image are extracted and combined with the local features of the residual module. This can ensure that the continuous processing of the facial area will not cause excessive uneven skin color, and at the same time improve the texture details of the model output and improve clarity.

[0085] Reference Figure 6 FIG. 1 is a schematic structural diagram of a facial oily shine removal device based on CNN provided in one embodiment of the present invention.

[0086] In this embodiment, the device 90 includes:

[0087] An acquisition unit 91 is used to acquire an input image to be processed;

[0088] A preprocessing unit 92, configured to preprocess the image to be processed to obtain a first face image;

[0089] A calculation unit 93, configured to calculate a face mask and a noise mask of the first face image;

[0090] The gloss removal unit 94 is used to input the first face image and the noise mask into a pre-trained gloss removal model to perform gloss removal processing on the face to obtain a target face image.

[0091] Furthermore, the pre-processing unit 92 further includes:

[0092] A bounding box calculation unit, used for performing face point detection on the image to be processed, and calculating a face bounding box of the image to be processed based on the face points;

[0093] The correction and cropping unit is used to calculate the affine transformation matrix of the image to be processed according to the face circumscribed frame, and perform face correction and cropping to obtain a first face image.

[0094] Furthermore, the calculation unit 93 further includes:

[0095] A face mask calculation unit, configured to calculate the face mask of the first face image by acquiring face points of the first face image;

[0096] The noise mask calculation unit is used to randomly generate the noise mask of the same size as the face mask from the area in the first face image where the pixel value is greater than 0 according to the face mask.

[0097] Furthermore, the gloss removal unit 94 is also used for:

[0098] The corresponding inverse matrix is ​​calculated based on the affine transformation matrix, and the result image obtained after the oily face removal processing in the oily face removal model is mapped to the image to be processed through the inverse matrix to obtain the target face image.

[0099] Furthermore, the device 90 comprises:

[0100] The type judgment unit is used to judge whether the facial lighting type of the first facial image is an oily type.

[0101] Furthermore, the degreasing model is constructed based on a GAN network, and the degreasing model includes a generation network and a discrimination network, wherein the generation network is constructed using a coding and decoding structure, and the discrimination network includes a multi-scale discriminator.

[0102] Furthermore, the oil-removing model includes a texture protection module in the generation network for extracting facial texture of a face area in an image.

[0103] Each unit module of the device 90 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the above corresponding steps for details.

[0104] The embodiment of the present invention further provides a device, which includes the CNN-based facial oily shine removal device as described above, wherein the CNN-based facial oily shine removal device can be used Figure 6 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0105] The device includes: a mobile phone, a digital camera, a tablet computer, or other devices with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0106] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0107] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0108] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel, and optionally, the display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event, and then the processor provides a corresponding visual output on the display panel according to the type of touch event.

[0109] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not installed in a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The CNN-based facial oil removal method is shown. The computer-readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0110] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, equipment embodiment and storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] Furthermore, in this document, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0112] The above description shows and describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention, through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A facial oily shine removal method based on CNN, characterized in that: The method comprises: Get the input image to be processed; Preprocessing the image to be processed to obtain a first face image; Calculating a face mask and a noise mask of the first face image; the calculating the face mask and the noise mask of the first face image includes: Calculating the face mask of the first face image by acquiring face points of the first face image; According to the face mask, randomly generate the noise mask of the same size as the face mask from the area in the first face image where the pixel value is greater than 0; The first face image and the noise mask are input into a pre-trained degreasing model to perform face degreasing processing to obtain a target face image; the degreasing model is constructed based on a GAN network, and the degreasing model includes a generating network and a discriminant network, wherein the generating network is constructed using a coding and decoding structure, the discriminant network includes a multi-scale discriminator, and the network structure of the generating network includes an input layer, convolution, LeakyReLU, residual block, TPM, upsampling, and activation layer.

2. The facial oiliness removal method based on CNN according to claim 1, characterized in that: The preprocessing of the image to be processed to obtain a first face image includes: Performing face point detection on the image to be processed, and calculating the face circumscribed frame of the image to be processed based on the face points; An affine transformation matrix of the image to be processed is calculated according to the face circumscribed frame, and face correction and cropping are performed to obtain a first face image.

3. The facial oiliness removal method based on CNN according to claim 2, characterized in that: The step of inputting the first face image and the noise mask into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image includes: The corresponding inverse matrix is ​​calculated based on the affine transformation matrix, and the result image obtained after the oily face removal processing in the oily face removal model is mapped to the image to be processed through the inverse matrix to obtain the target face image.

4. The facial oiliness removal method based on CNN according to claim 1, characterized in that: The method further comprises: Determine whether the facial lighting type of the first facial image is an oily type.

5. The method for removing facial shine based on CNN according to claim 1, characterized in that: The degreasing model includes a texture protection module in the generation network for extracting facial texture of a face area in an image.

6. A facial oily shine removal device based on CNN, characterized in that: The device comprises: An acquisition unit, used for acquiring an input image to be processed; A preprocessing unit, used for preprocessing the image to be processed to obtain a first face image; A calculation unit, used to calculate a face mask and a noise mask of the first face image; the calculation unit further includes: A face mask calculation unit, configured to calculate the face mask of the first face image by acquiring face points of the first face image; a noise mask calculation unit, configured to randomly generate the noise mask of the same size as the face mask from the area in the first face image where the pixel value is greater than 0 according to the face mask; The oil-removal unit is used to input the first face image and the noise mask into a pre-trained oil-removal model to perform oil-removal processing on the face to obtain a target face image; the oil-removal model is constructed based on a GAN network, and the oil-removal model includes a generation network and a discriminant network, wherein the generation network is constructed using a coding and decoding structure, the discriminant network includes a multi-scale discriminator, and the network structure of the generation network includes an input layer, convolution, LeakyReLU, residual block, TPM, upsampling, and activation layer.

7. A device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a facial oiliness removal method based on CNN as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of a CNN-based facial shine removal method as described in any one of claims 1 to 5.

Images