Face image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides a facial image processing method, device, electronic device and storage medium, the method comprising: performing frequency division and partitioning processing on a facial image to be processed, determining a facial organ area and a facial skin area in the facial image, and determining a target area that belongs to the facial skin area but does not belong to the facial organ area from the facial image, and removing the intermediate frequency component in the target area to obtain a target facial image. The facial image contains information of all frequency bands, namely, low-frequency, intermediate-frequency and high-frequency information, wherein the intermediate frequency component corresponds to information of uneven skin color in the facial image, and the high-frequency component corresponds to skin texture in the facial image. According to the scheme of the present disclosure, only the intermediate frequency component in the target area is removed, that is, the facial blemishes in the target area are weakened while retaining the skin texture, and the realism of the skin is retained to the greatest extent, and the facial organ area is not blurred, thereby ensuring the clarity of the facial organ area.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a face image processing method, device, electronic device and storage medium. Background Art

[0002] When users take images or videos of people's faces, the real face skin is extremely uneven due to lighting or the skin's own reasons. For example, the face may have spots, moles, acne, etc., which seriously affect the beauty of the face. Therefore, many applications involving image or video processing (such as camera apps, live broadcast apps, photo editing apps, etc.) include beauty functions, which can beautify photos or videos to improve the beauty of the face.

[0003] Most of the skin smoothing algorithms in related technologies will make the facial skin become almost pure color, with a serious smearing feeling, which is extremely unrealistic. Summary of the invention

[0004] The present disclosure provides a facial image processing method, device, electronic device and storage medium to at least solve the problem of severe distortion of facial skin color processing in related technologies. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a face image processing method is provided, comprising:

[0006] Determine the facial organ area and the facial skin area from the face image to be processed;

[0007] Determine a target area in the face image to be processed that belongs to the facial skin area and does not belong to the facial organ area;

[0008] The target face image is obtained by removing the intermediate frequency component contained in the target area, wherein the intermediate frequency component contains information representing the uneven skin color part of the face image.

[0009] In a possible implementation manner of the first aspect, the step of removing the intermediate frequency component contained in the target area to obtain the target face image includes:

[0010] Acquire the intermediate frequency component contained in the face image to be processed;

[0011] The target face image is obtained by filtering out the intermediate frequency components contained in the target area.

[0012] In another possible implementation manner of the first aspect, the step of removing the intermediate frequency component contained in the target area to obtain the target face image includes:

[0013] The target face image is obtained by acquiring the intermediate frequency component contained in the target area and filtering out the intermediate frequency component in the target area.

[0014] In another possible implementation of the first aspect, the step of filtering out intermediate frequency components contained in the target area to obtain the target face image includes:

[0015] Obtaining a target pixel value of the target area according to the original pixel value in the target area and the pixel value corresponding to the intermediate frequency component of the face image to be processed;

[0016] The target face image is obtained according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area.

[0017] In yet another possible implementation of the first aspect, the step of obtaining the target facial image according to the original pixel values ​​of the facial image to be processed and the target pixel values ​​of the target area includes:

[0018] For any pixel point in the target area, the original pixel value corresponding to the pixel point is replaced with the target pixel value to obtain the target face image.

[0019] In another possible implementation of the first aspect, the step of obtaining the target facial image according to the original pixel values ​​of the facial image to be processed and the target pixel values ​​of the target area includes:

[0020] For any pixel point in the target area, replacing the original pixel value of the pixel point with the target pixel value to obtain a first face image;

[0021] The pixel values ​​corresponding to the same pixel point in the face image to be processed and the first face image are fused according to a set fusion ratio to obtain the target face image.

[0022] In yet another possible implementation manner of the first aspect, a process of acquiring an intermediate frequency component contained in an image includes:

[0023] Downsampling the image to obtain a first downsampled image;

[0024] Filtering the first down-sampled image using an edge-preserving filtering algorithm to obtain a first filtered image;

[0025] Downsampling the first downsampled image to obtain a second downsampled image, and filtering the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image;

[0026] The difference between the pixel value of the first filtered image and the pixel value of the second filtered image is calculated to obtain the intermediate frequency component.

[0027] In another possible implementation of the first aspect, the step of determining the facial organ region from the face image to be processed includes:

[0028] Positioning facial key points of the facial image to be processed;

[0029] The facial key points of the face image to be processed and the facial key points of the standard face mask image are matched one by one to obtain a face mask image corresponding to the face image to be processed, wherein the gray value of each pixel in the face mask image represents the probability that the pixel is a facial organ.

[0030] In another possible implementation of the first aspect, the step of determining the facial skin area from the face image to be processed includes:

[0031] Skin color detection is performed on the face image to be processed to obtain a skin color mask image, and the grayscale value of each pixel in the skin color mask image represents the probability that the pixel is skin.

[0032] According to a second aspect of an embodiment of the present disclosure, there is provided a face image processing device, comprising:

[0033] A facial organ region determination module is configured to determine the facial organ region from the face image to be processed;

[0034] A skin region determination module is configured to determine a facial skin region from a face image to be processed;

[0035] A target region determination module is configured to determine a target region in the face image to be processed that belongs to the facial skin region and does not belong to the facial organ region;

[0036] The skin smoothing processing module is configured to remove the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven facial skin color part in the face image.

[0037] In a possible implementation of the second aspect, the skin smoothing treatment module includes:

[0038] A first intermediate frequency acquisition submodule is configured to execute acquisition of the intermediate frequency component contained in the face image to be processed;

[0039] The intermediate frequency filtering submodule is configured to filter out the intermediate frequency components in the target area to obtain the target face image.

[0040] In another possible implementation of the second aspect, the skin smoothing treatment module includes:

[0041] A second intermediate frequency acquisition submodule is configured to acquire the intermediate frequency components contained in the target area;

[0042] The intermediate frequency filtering submodule is configured to filter out the intermediate frequency components in the target area to obtain the target face image.

[0043] In yet another possible implementation of the second aspect, the intermediate frequency filtering submodule includes:

[0044] A target pixel determination submodule is configured to obtain a target pixel value of the target area according to an original pixel value in the target area and a pixel value corresponding to an intermediate frequency component of the face image to be processed;

[0045] The target face image acquisition submodule is configured to obtain the target face image according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area.

[0046] In yet another possible implementation of the second aspect, the target face image obtaining submodule is specifically configured to:

[0047] For any pixel point in the target area, the original pixel value corresponding to the pixel point is replaced with the target pixel value to obtain the target face image.

[0048] In another possible implementation of the second aspect, the target face image obtaining submodule includes:

[0049] A target area image determination submodule is configured to replace an original pixel value of any pixel point in the target area with the target pixel value to obtain a first face image;

[0050] The image fusion submodule is configured to fuse the pixel values ​​corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion ratio to obtain the target face image.

[0051] In another possible implementation manner of the second aspect, the first intermediate frequency acquisition submodule includes:

[0052] A first down-sampling submodule is configured to perform down-sampling on the face image to be processed to obtain a first down-sampled image;

[0053] A first filtering submodule is configured to perform filtering on the first down-sampled image using an edge-preserving filtering algorithm to obtain a first filtered image;

[0054] A second filtering submodule is configured to perform downsampling on the first downsampled image to obtain a second downsampled image, and filter the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image;

[0055] The first calculation submodule is configured to calculate the difference between the pixel value of the first filtered image and the pixel value of the second filtered image to obtain the intermediate frequency component.

[0056] In another possible implementation manner of the second aspect, the facial organ region determination module includes:

[0057] A key point positioning submodule is configured to perform facial key point positioning on the face image to be processed;

[0058] The face mask image acquisition submodule is configured to perform a one-to-one correspondence between the face key points of the face image to be processed and the face key points of the standard face mask image, so as to obtain a face mask image corresponding to the face image to be processed, wherein the grayscale value of each pixel in the face mask image represents the probability that the pixel is a facial organ.

[0059] In another possible implementation manner of the second aspect, the skin area determination module is specifically configured to:

[0060] Skin color detection is performed on the face image to be processed to obtain a skin color mask image, and the grayscale value of each pixel in the skin color mask image represents the probability that the pixel is skin.

[0061] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0062] processor;

[0063] a memory for storing instructions executable by the processor;

[0064] Wherein, the processor is configured to execute the instructions to implement the facial image processing method as described in any one of the first aspects.

[0065] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the facial image processing method described in any one of the first aspects.

[0066] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, which, when executed on a data processing device, is suitable for executing a program for initializing the facial image processing method described in any possible implementation manner of the first aspect.

[0067] The face image processing method provided by the embodiment of the present disclosure performs frequency division and partitioning processing on the face image to be processed, determines the facial organ area and the facial skin area in the face image, and determines the target area that belongs to the facial skin area but does not belong to the facial organ area from the face image. The target face image is obtained by removing the intermediate frequency component in the target area. The face image contains information of all frequency bands, namely low frequency, intermediate frequency and high frequency information, wherein the intermediate frequency component corresponds to the information of the uneven skin color part in the face image, and the high frequency component corresponds to the skin texture in the face image. Only the intermediate frequency component in the target area is removed, that is, the facial blemishes in the target area are weakened while retaining the skin texture, thereby retaining the realism of the skin to the greatest extent. Moreover, the facial organ area is not blurred, so that the clarity of the facial organ area is improved.

[0068] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0070] Figure 1 is a flowchart of a method for processing a face image according to an exemplary embodiment;

[0071] Figure 2 is a flow chart showing a process of obtaining an intermediate frequency component in a face image to be processed according to an exemplary embodiment;

[0072] Figure 3 is a flowchart of another method for processing a face image according to an exemplary embodiment;

[0073] Figure 4 is a schematic diagram of a standard face mask image according to an exemplary embodiment;

[0074] Figure 5 is a block diagram of a face image processing device according to an exemplary embodiment;

[0075] Figure 6 is a block diagram of a skin smoothing processing module according to an exemplary embodiment;

[0076] Figure 7 It is a block diagram of another facial image processing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0077] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0078] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0079] Figure 1 is a flowchart of a method for processing a face image according to an exemplary embodiment. Figure 1 As shown, the face image processing method is used in an APP (Application) involving face image processing, wherein the APP involving face image processing can be run in a terminal device, such as a smart phone, a tablet computer, a smart camera, etc. The method includes the following steps.

[0080] In S110, a facial organ region and a facial skin region are determined from the face image to be processed.

[0081] The facial image to be processed may be a picture or video image taken by an APP involving facial image processing, or may be a picture or video image taken by other loaded APPs.

[0082] After obtaining the face image to be processed, the facial key point detection model can be used to obtain the area where the facial organs in the face image are located, that is, the facial organ area. For example, the eyes, eyebrows, nose wings, bridge of the nose, mouth, jaw line, lip pits, facial contours, etc. The facial organ area is the area that needs to be protected, that is, the area that does not need to be processed for skin color uniformity.

[0083] In one embodiment of the present disclosure, a skin color detection algorithm may be used to perform skin color detection on a face image to be processed to obtain a skin color mask image, wherein the grayscale value of each pixel of the skin color mask image represents the probability that the pixel is facial skin.

[0084] Among them, the skin color mask image is represented by grayscale values. The grayscale value corresponding to each pixel represents the probability value of the pixel position belonging to the skin color area. The probability value is a value in the range of 0 to 1. For example, when the grayscale value is 0.2, it means that the probability of the pixel point belonging to the skin color area is 20%. When the grayscale value is 0, it means that the pixel point is not a point in the skin color area. When the grayscale value is 1, it means that the pixel point is a point in the skin color area.

[0085] In S120, a target area in the face image to be processed that belongs to the facial skin area and does not belong to the facial organ area is determined.

[0086] Some facial organ areas overlap with facial skin areas, such as the nose wings and the bridge of the nose. Therefore, any pixel in a face image may have the following three situations:

[0087] 1) Belong to the facial organ area but not the facial skin area;

[0088] 2) Belongs to both the facial organ area and the facial skin area;

[0089] 3) Belongs to the facial skin area but not the facial organ area.

[0090] The case 3) mentioned above requires skin tone uniformity treatment, and the area that needs to be protected is the case 1) and 2) mentioned above, that is, the area that belongs to the facial skin area but not the facial organ area is the target area.

[0091] In S130, the intermediate frequency components contained in the target area are removed to obtain a target face image.

[0092] In an image, high-frequency components refer to parts of the image where the brightness / grayscale changes dramatically, such as edges and contours; low-frequency components refer to parts of the image where the brightness / grayscale changes slowly, such as large blocks of color; other parts are medium-frequency components, such as uneven skin color, spots, moles or acne, or facial blemishes.

[0093] If the high-frequency part in the target area is directly filtered out, the skin texture will be removed, making the skin look seriously smeared and extremely unreal. In order to solve this technical problem, the embodiment of the present disclosure does not directly blur the area corresponding to the third case (i.e., the target area), but only removes the mid-frequency component in the target area, retains the high-frequency component and the low-frequency component, and obtains the target face image.

[0094] Among them, the face image to be processed contains low, medium and high frequency information. After removing the medium frequency component in the target area, the high and low frequency information of the target area is retained. Moreover, the area containing facial organs is not processed. Therefore, this method weakens facial blemishes while retaining skin texture, and retains the realism of the skin and the clarity of facial organs to the greatest extent.

[0095] In a possible implementation of the present disclosure, Figure 2 As shown, S130 may include the following steps:

[0096] In S131, the intermediate frequency component in the entire face image is obtained.

[0097] In S132, the intermediate frequency components in the target area are removed to obtain a target face image.

[0098] This implementation method first obtains the intermediate frequency components contained in the entire face image, and then removes the intermediate frequency components contained in the target area. In this way, the following process of obtaining the intermediate frequency components can be directly executed in parallel on the pixels contained in the entire face image, thereby improving the image processing speed.

[0099] like Figure 2 As shown, the process of obtaining the intermediate frequency components contained in the entire face image to be processed may include the following steps:

[0100] In S1311, the face image to be processed is down-sampled to obtain a first down-sampled image.

[0101] Downsampling an image means reducing the feature dimension and retaining effective information, which is equivalent to reducing the image to generate a thumbnail of the corresponding image. It can be seen that downsampling an image can reduce the feature dimension of the image and thus increase the image processing speed.

[0102] In S1312, the first down-sampled image is filtered using an edge-preserving filtering algorithm to obtain a first filtered image.

[0103] Edge-preserving filtering algorithm refers to a special type of filtering method that can effectively preserve the edge information in the image during the smoothing filtering process. For example, surface filtering and guided filtering are both edge-preserving filtering algorithms. Edge-preserving filtering algorithm mainly processes the high-frequency components in the face image, that is, it filters out some high-frequency components and retains the medium and low-frequency components.

[0104] In the embodiment of the present disclosure, the blur radius of the edge-preserving filtering algorithm can be set to a larger value, such as 20 pixels. Of course, it is set according to the size of the image (such as the number of pixels contained). The larger the image, the larger the corresponding blur radius. On the contrary, the smaller the image, the corresponding blur radius can be set to a smaller value. For example, for a 720*1080 image, the blur radius can be set to 20 pixels. Of course, other units can also be used and will not be repeated here.

[0105] The blur radius refers to the size of the blur value from the center point of the object on the layer, that is, denoising and filtering the area within the blur radius to make the filtered image smoother and flatter.

[0106] In S1313, the first down-sampled image is down-sampled to obtain a second down-sampled image, and the second down-sampled image is filtered using a low-pass filtering algorithm to obtain a second filtered image.

[0107] The second down-sampled image obtained by further down-sampling the first down-sampled image has a lower feature dimension, which further improves the image processing efficiency.

[0108] After low-pass filtering the image, the low-frequency components in the image are retained, and the medium and high frequency components in the image are filtered. For example, mean filtering is a common low-pass filtering method. Mean filtering is a method of taking the average value of the target pixel and the surrounding pixels and then filling it back into the target pixel to achieve the filtering purpose.

[0109] In S1314, the difference between the pixel value of the first filtered image and the pixel value of the second filtered image is calculated to obtain the intermediate frequency component.

[0110] As mentioned above, the first filtered image retains the information of the intermediate frequency and low frequency, and the second filtered image retains the low frequency component. Therefore, by subtracting the pixel value of the second filtered image from the pixel value of the first filtered image, the intermediate frequency component of the entire face image can be obtained. This method uses edge-preserving filtering and low-pass filtering, and then subtracts the two filtering results to obtain the intermediate frequency component in the image. The processing process is simple and the intermediate frequency component in the image can be accurately extracted.

[0111] In another possible implementation of the present disclosure, the intermediate frequency component of the target area in the face image may be first obtained, and then the intermediate frequency component of the target area is filtered out to obtain the target face image. Figure 2 The difference of the implementation shown is that this implementation only processes the image belonging to the target area to obtain the intermediate frequency component, and further filters the intermediate frequency component of the target area. Obtaining the intermediate frequency component contained in the target area also needs to go through the processing shown in S1311 to S1314, but the processing object is only the image of the target area instead of the entire face image.

[0112] This implementation method directly obtains the intermediate frequency components contained in the target area rather than the intermediate frequency components contained in the entire image. Therefore, the intermediate frequency components obtained by this method are more accurate and the filtered intermediate frequency components are more targeted. Therefore, the final image processing effect is better.

[0113] After obtaining the intermediate frequency component contained in the entire face image or the target area, the intermediate frequency component is converted to the time domain to obtain the pixel value corresponding to the intermediate frequency component. The intermediate frequency component in the target area can be filtered out by subtracting the pixel value corresponding to the intermediate frequency component from the pixel value of the target area to obtain the target pixel value of the target area. Then, the target face image is obtained according to the original pixel value of the face image to be processed and the target pixel value of the target area.

[0114] In one possible implementation, the original pixel values ​​of each pixel in the target area of ​​the entire face image can be directly replaced with the corresponding target pixel values, and the pixel values ​​of the non-target area in the face image remain unchanged, thereby obtaining the target face image. This implementation method only requires replacing the pixel points in the target area of ​​the entire image with the target pixel values, and other pixel points do not need to be processed. The processing process is simple, and the image processing speed is improved while ensuring the image processing effect.

[0115] In another possible implementation, each pixel point in the target area of ​​the face image to be processed is replaced with the target pixel value, and the pixel value of the non-target area remains unchanged to obtain the first face image; then the first face image and the face image to be processed are fused at a certain fusion ratio to obtain the target face image. The fusion ratio can be set by the user according to actual needs. This implementation allows the user to adjust the skin smoothing degree (i.e., fusion ratio) according to their own needs, and the skin smoothing processing results are more flexible and diversified.

[0116] The facial image processing method provided in this embodiment performs frequency division and zoning processing on the facial image to be processed, determines the facial organ area and the facial skin area in the facial image, and determines the target area that belongs to the facial skin area but does not belong to the facial organ area from the facial image. The target facial image is obtained by removing the intermediate frequency component in the target area. The facial image contains information of all frequency bands, namely, low frequency, intermediate frequency and high frequency information, wherein the intermediate frequency component corresponds to the information of the uneven skin color part in the facial image, and the high frequency component corresponds to the skin texture in the facial image. Only the intermediate frequency component in the target area is removed, that is, the facial blemishes in the target area are weakened while retaining the skin texture, thereby retaining the realism of the skin to the greatest extent. Moreover, the facial organ area is not blurred, so that the clarity of the facial organ area is improved.

[0117] In order to more intuitively understand the implementation process of the face image processing method provided by the present disclosure, Figure 3 Detailed introduction, Figure 3 The present invention is a processing flow chart of another facial image processing method according to an exemplary embodiment. The method includes the following steps.

[0118] In S210, facial key points are located on the input image.

[0119] The facial key point detection model is used to locate the facial key points of the input image InputImage, and the position areas of the eyes, eyebrows, nose wings, nose bridge, mouth, jaw line, lip pits, facial contours, etc. on the face image to be processed are obtained.

[0120] In S220, the facial key points of the input image and the facial key points of the standard face mask image are matched one by one to obtain a face mask image corresponding to the input image.

[0121] The key points of the face corresponding to InputImage are matched one by one with the key points of the standard face mask image. For example, the key points of the eyes in the standard face mask image are mapped to the key points of the eyes in the InputImage image. Furthermore, for the area composed of multiple key points in the face key point image of InputImage (such as a triangular area composed of three key points), it is replaced with the pixel value of the same key point area in the standard face mask image to obtain the face mask image corresponding to InputImage.

[0122] This step completes the warp mapping of the standard face mask material image to InputImage, that is, the face mask image OrganMask of InputImage is obtained. The gray value of each pixel in OrganMask represents the probability that the pixel belongs to a facial organ. Therefore, OrganMask can be used to identify the facial organ area and non-facial organ area of ​​the face image.

[0123] Figure 4 The following is a schematic diagram of a standard face mask image, where the black area is the non-facial organ area of ​​the standard face, and the white area is the facial organ area of ​​the standard face. Figure 4 Similar images, except that the positions of the facial organs of OrganMask may be different from the positions of the organs of InputImage.

[0124] In S230, skin color detection is performed on the input image to obtain a skin color mask image of the input image.

[0125] The skin color detection algorithm is used to perform facial skin detection on InputImage to obtain the facial skin color area and obtain the skin color mask image SkinMask.

[0126] Among them, SkinMask is represented by grayscale values, each grayscale value represents the probability that the point belongs to the facial skin area, and the probability value is a value in the range of 0 to 1. For example, when the grayscale value is 0.2, it means that the probability that the pixel point belongs to the skin color area is 20%. When the grayscale value is 0, it means that the pixel point is not a point in the skin color area. When the grayscale value is 1, it means that the pixel point is a point in the skin color area.

[0127] It should be noted that the present disclosure does not limit the order of obtaining the OrganMask and obtaining the SkinMask.

[0128] In S240, the input image is downsampled by a first preset coefficient to obtain DsImage, and surface filtering is performed on DsImage to obtain SurfaceBlurImage.

[0129] In this embodiment, the first preset coefficient can be set according to actual application requirements, for example, it can be set to 2, that is, 2 times downsampling is performed. The edge-preserving filtering method filters out high-frequency components in the image, such as skin texture.

[0130] In S250 , DsImage is downsampled by a second preset coefficient, and the sampled image is low-pass filtered to obtain BlurImage.

[0131] The low-pass filter removes the medium and high frequency components in the image and retains the low frequency components in the image. The second preset coefficient can be set according to actual needs, for example, 2 times. BlurImage is the second filtered image mentioned above.

[0132] In S260 , the difference between SurfaceBlurImage and BlurImage is calculated to obtain DiffImage.

[0133] That is, DiffImage=SurfaceBlurImage-BlurImage, where DiffImage mainly includes the intermediate frequency components that need to be removed, such as uneven skin color blocks, spots, moles, etc.

[0134] In S270, the intermediate frequency components in the target area are removed according to the OrganMask and the SkinMask to obtain the target face image.

[0135] The target area is the area in the face image that belongs to the facial skin area and does not belong to the facial organ area.

[0136] In one embodiment of the present disclosure, the process of obtaining the first face image TempImage according to the following formula is as follows:

[0137] TempImage=InputImage–DiffImage*SkinMask*(1.0-OrganMask) (Formula 1)

[0138] In formula 1, DiffImage is the intermediate frequency component to be removed, SkinMask represents the probability of InputImage belonging to facial skin, OrganMask represents the probability of InputImage belonging to facial organs, and (1.0-OrganMask) represents the probability of InputImage not belonging to facial organs;

[0139] According to Formula 1, if SkinMask=0, it means that the area is not the facial skin area, then DiffImage*SkinMask*(1.0-OrganMask)=0, therefore, TempImage=InputImage, that is, this area is not processed.

[0140] If OrganMask=1, it means that the area belongs to the facial organ area, 1.0-OrganMask=0, DiffImage*SkinMask*(1.0-OrganMask)=0, therefore, TempImage=InputImage, that is, this area is not processed.

[0141] Therefore, according to the above analysis, the meaning of Formula 1 is to remove the intermediate frequency components in the input image that belong to the facial skin area but not the facial organ area (i.e., the target area), that is, to remove the uneven skin color blocks, spots, moles, etc. in the facial skin area but not the facial organ area.

[0142] TempImage is further fused with InputImage according to Formula 2 to obtain the final image output.

[0143] OutputImage=mix(InputImage,TempImage,BlurAlpha)

[0144] =InputImage*(1-BlurAlpha)+TempImage*BlurAlpha (Formula 2)

[0145] In Formula 2, OutputImage represents the final output target face image, BlurAlpha represents the degree of skin smoothing processing, which can be adjusted by the user, and BlurAlpha is the set fusion ratio mentioned above.

[0146] For example, if BlurAlpha = 0, it means that the input image is not subjected to skin smoothing, then OutputImage = InputImage; if BlurAlpha = 1, OutputImage = TempImage, that is, the TempImage obtained in the previous step is directly used as the final target face image. For another example, if BlurAlpha = 0.5, then OutputImage = InputImage*0.5+TempImage*0.5, that is, for any pixel, it is composed of 50% of the InputImage at that pixel and 50% of the TempImage at that pixel.

[0147] The face image processing method provided in this embodiment removes the intermediate frequency components only from the area of ​​the face image that belongs to the skin area and does not belong to the facial organ area. The facial blemishes in the target area are weakened while retaining the skin texture in the face image, thereby retaining the realism of the skin to the greatest extent. In addition, this embodiment can also fuse the input image and the image after the skin smoothing processing according to the skin smoothing processing degree set by the user to obtain the final target face image, so as to realize the skin smoothing processing of different skin smoothing degrees on the input image, and the skin smoothing processing is more flexible.

[0148] Figure 5 FIG. 1 is a block diagram of a facial image processing device according to an exemplary embodiment. Figure 5 The device includes a facial organ region determination module 110, a skin region determination module 120, a target region determination module 130 and a skin smoothing processing module 140.

[0149] The facial organ region determining module 110 is configured to determine the facial organ region from the face image to be processed.

[0150] In a possible implementation, the facial organ region determination module includes: a key point positioning submodule and a face mask image acquisition submodule.

[0151] The key point positioning submodule is configured to perform facial key point positioning on the face image to be processed.

[0152] The face mask image acquisition submodule is configured to perform a one-to-one correspondence between the face key points of the face image to be processed and the face key points of the standard face mask image, so as to obtain the face mask image corresponding to the face image to be processed.

[0153] Among them, the grayscale value of each pixel in the face mask image represents the probability that the pixel is a facial organ.

[0154] The skin region determination module 120 is configured to determine the facial skin region from the face image to be processed.

[0155] The skin region determination module 120 is specifically used to perform skin color detection on the face image to be processed to obtain a skin color mask image, wherein the gray value of each pixel in the skin color mask image represents the probability that the pixel is skin.

[0156] The target region determination module 130 is configured to determine a target region in the face image to be processed that belongs to the facial skin region and does not belong to the facial organ region.

[0157] The skin smoothing processing module 140 is configured to remove the intermediate frequency components contained in the target area to obtain the target face image, where the intermediate frequency components contain information representing the uneven skin color of the face in the face image.

[0158] In one embodiment of the present disclosure, the intermediate frequency components contained in the entire face image are first obtained, and then the intermediate frequency components in the target area are filtered out, such as Figure 6 As shown, the skin smoothing processing module 140 includes a first intermediate frequency acquisition submodule 141 and an intermediate frequency filtering submodule 142 .

[0159] The first intermediate frequency acquisition submodule 141 is configured to acquire the intermediate frequency component contained in the face image to be processed.

[0160] In one possible implementation, Figure 6 As shown, the first intermediate frequency acquisition submodule 141 includes:

[0161] The first down-sampling submodule 1411 is configured to perform down-sampling on the face image to be processed to obtain a first down-sampled image.

[0162] The first filtering submodule 1412 is configured to perform filtering on the first down-sampled image using an edge-preserving filtering algorithm to obtain a first filtered image.

[0163] The second filtering submodule 1413 is configured to perform downsampling on the first downsampled image to obtain a second downsampled image, and filter the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image.

[0164] The first calculation submodule 1414 is configured to calculate the difference between the pixel value of the first filtered image and the pixel value of the second filtered image to obtain the intermediate frequency component.

[0165] The intermediate frequency filtering submodule 142 is configured to filter out the intermediate frequency components in the target area to obtain a target face image.

[0166] In one possible implementation, Figure 6 As shown, the intermediate frequency filtering submodule 142 includes:

[0167] The target pixel determination submodule 1421 is configured to obtain a target pixel value of the target area according to the original pixel value in the target area and the pixel value corresponding to the intermediate frequency component of the face image to be processed.

[0168] The target facial image obtaining submodule 1422 is configured to obtain the target facial image according to the original pixel values ​​of the facial image to be processed and the target pixel values ​​of the target area.

[0169] In a possible implementation, the target face image obtaining submodule 1422 is specifically configured to: for any pixel point in the target area, replace the original pixel value corresponding to the pixel point with the target pixel value to obtain the target face image.

[0170] In another possible implementation, the target face image obtaining submodule 1422 includes: a target area image determining submodule and an image fusion submodule;

[0171] The target area image determination submodule is configured to replace the original pixel value of any pixel point in the target area with the target pixel value to obtain a first face image.

[0172] The image fusion submodule is configured to fuse the pixel values ​​corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion ratio to obtain a target face image.

[0173] In another embodiment of the present disclosure, the intermediate frequency component of the target area in the face image can be first obtained, and then the intermediate frequency component of the target area is filtered out to obtain the target face image. The process of obtaining the intermediate frequency component contained in the target area is basically the same as the process of obtaining the intermediate frequency component of the entire image, but the processing object is only the image of the target area instead of the entire face image, which will not be repeated here.

[0174] The face image processing device provided in this embodiment performs frequency division and partitioning processing on the face image to be processed, determines the facial organ area and the facial skin area in the face image, and determines the target area that belongs to the facial skin area but does not belong to the facial organ area from the face image. The target face image is obtained by removing the intermediate frequency component in the target area. The face image contains information of all frequency bands, namely low-frequency, intermediate frequency and high-frequency information, wherein the intermediate frequency component corresponds to the information of the uneven skin color part in the face image, and the high-frequency component corresponds to the skin texture in the face image. Only the intermediate frequency component in the target area is removed, that is, the facial blemishes in the target area are weakened while retaining the skin texture, thereby retaining the realism of the skin to the greatest extent. Moreover, the facial organ area is not blurred, so that the clarity of the facial organ area is improved.

[0175] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0176] On the other hand, the present disclosure further provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the above-mentioned face image processing methods. Figure 7 As shown, the electronic device includes at least one memory 210, at least one processor 220 and a bus 230. The processor 220 communicates with the memory 210 via the bus 230.

[0177] The memory 210 stores instructions, and the processor 220 calls the instructions in the memory 210 to execute the above-mentioned face image processing method. The electronic device in this article can be a mobile intelligent terminal (for example, a smart phone, a tablet computer, a smart camera, etc.) or a PC, etc.

[0178] In an exemplary embodiment, the present disclosure further provides a storage medium including instructions, such as a memory 210 including instructions, and the above instructions can be executed by a processor 220 of an electronic device to complete the above facial image processing method.

[0179] Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0180] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0181] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A facial image processing method, characterized in that: include: Determine the facial organ area and the facial skin area from the face image to be processed; Determine a target area in the face image to be processed that belongs to the facial skin area and does not belong to the facial organ area; Removing the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven skin color portion of the face image; The process of obtaining the intermediate frequency component includes: Downsampling the image to obtain a first downsampled image; Filtering the first down-sampled image using an edge-preserving filtering algorithm to obtain a first filtered image; Downsampling the first downsampled image to obtain a second downsampled image, and filtering the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image; The intermediate frequency component is obtained by subtracting the pixel value of the second filtered image from the pixel value of the first filtered image.

2. The facial image processing method according to claim 1, characterized in that: The step of removing the intermediate frequency components contained in the target area to obtain the target face image comprises: Acquire the intermediate frequency component contained in the face image to be processed; The target face image is obtained by filtering out the intermediate frequency components contained in the target area.

3. The facial image processing method according to claim 1, characterized in that: The step of removing the intermediate frequency components contained in the target area to obtain the target face image comprises: The target face image is obtained by acquiring the intermediate frequency component contained in the target area and filtering out the intermediate frequency component in the target area.

4. The facial image processing method according to claim 2 or 3, characterized in that: The step of filtering out the intermediate frequency components contained in the target area to obtain the target face image comprises: Obtaining a target pixel value of the target area according to the original pixel value in the target area and the pixel value corresponding to the intermediate frequency component of the face image to be processed; The target face image is obtained according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area.

5. The facial image processing method according to claim 4, characterized in that: The step of obtaining the target face image according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area comprises: For any pixel point in the target area, the original pixel value corresponding to the pixel point is replaced with the target pixel value to obtain the target face image.

6. The facial image processing method according to claim 4, characterized in that: The step of obtaining the target face image according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area comprises: For any pixel point in the target area, replacing the original pixel value of the pixel point with the target pixel value to obtain a first face image; The pixel values ​​corresponding to the same pixel point in the face image to be processed and the first face image are fused according to a set fusion ratio to obtain the target face image.

7. The facial image processing method according to claim 1, characterized in that: The step of determining the facial organ region from the face image to be processed comprises: Positioning facial key points of the facial image to be processed; The facial key points of the face image to be processed and the facial key points of the standard face mask image are matched one by one to obtain a face mask image corresponding to the face image to be processed, and the grayscale value of each pixel in the face mask image represents the probability that the pixel is a facial organ.

8. The facial image processing method according to claim 1, characterized in that: The step of determining the facial skin area from the face image to be processed comprises: Skin color detection is performed on the face image to be processed to obtain a skin color mask image, and the grayscale value of each pixel in the skin color mask image represents the probability that the pixel is skin.

9. A facial image processing device, characterized in that: include: A facial organ region determination module is configured to determine the facial organ region from the face image to be processed; A skin region determination module is configured to determine a facial skin region from a face image to be processed; A target region determination module is configured to determine a target region in the face image to be processed that belongs to the facial skin region and does not belong to the facial organ region; a skin smoothing processing module, configured to remove the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven skin color of the face in the face image; The process of obtaining the intermediate frequency component includes: Downsampling the image to obtain a first downsampled image; Filtering the first down-sampled image using an edge-preserving filtering algorithm to obtain a first filtered image; Downsampling the first downsampled image to obtain a second downsampled image, and filtering the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image; The intermediate frequency component is obtained by subtracting the pixel value of the second filtered image from the pixel value of the first filtered image.

10. The face image processing device according to claim 9, characterized in that: The skin smoothing treatment module comprises: A first intermediate frequency acquisition submodule is configured to execute acquisition of the intermediate frequency component contained in the face image to be processed; The intermediate frequency filtering submodule is configured to filter out the intermediate frequency components in the target area to obtain the target face image.

11. The face image processing device according to claim 9, characterized in that: The skin smoothing treatment module comprises: A second intermediate frequency acquisition submodule is configured to acquire the intermediate frequency components contained in the target area; The intermediate frequency filtering submodule is configured to filter out the intermediate frequency components in the target area to obtain the target face image.

12. The facial image processing device according to claim 10 or 11, characterized in that: The intermediate frequency filtering submodule includes: A target pixel determination submodule is configured to obtain a target pixel value of the target area according to an original pixel value in the target area and a pixel value corresponding to an intermediate frequency component of the face image to be processed; The target face image acquisition submodule is configured to obtain the target face image according to the original pixel values ​​of the face image to be processed and the target pixel values ​​of the target area.

13. The facial image processing device according to claim 12, characterized in that: The target face image acquisition submodule is specifically used for: For any pixel point in the target area, the original pixel value corresponding to the pixel point is replaced with the target pixel value to obtain the target face image.

14. The facial image processing device according to claim 12, characterized in that: The target face image acquisition submodule includes: A target area image determination submodule is configured to replace an original pixel value of any pixel point in the target area with the target pixel value to obtain a first face image; The image fusion submodule is configured to fuse the pixel values ​​corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion ratio to obtain the target face image.

15. The facial image processing device according to claim 9, characterized in that: The facial organ region determination module comprises: A key point positioning submodule is configured to perform facial key point positioning on the face image to be processed; The face mask image acquisition submodule is configured to perform a one-to-one correspondence between the face key points of the face image to be processed and the face key points of the standard face mask image, so as to obtain a face mask image corresponding to the face image to be processed, wherein the grayscale value of each pixel in the face mask image represents the probability that the pixel is a facial organ.

16. The facial image processing device according to claim 9, characterized in that: The skin area determination module is specifically used for: Skin color detection is performed on the face image to be processed to obtain a skin color mask image, and the grayscale value of each pixel in the skin color mask image represents the probability that the pixel is skin.

17. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the facial image processing method as described in any one of claims 1 to 8.

18. A storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the facial image processing method as described in any one of claims 1 to 8.

Images