Image Processing Method, System, Device and Storage Medium

Abstract

The present application discloses an image processing method, system, device and storage medium. The method includes: performing a flaw mask processing on a face image to obtain a black flaw mask and a white flaw mask, and determining a black flaw removal degree layer and a white flaw removal degree layer according to the face image; performing a brightening curve processing on the face image to obtain a brightening layer and performing a darkening curve processing on the face image to obtain a darkening layer; determining a black spot removal layer according to the black flaw mask and the black flaw removal degree layer, and determining a white spot removal layer according to the white flaw mask and the white flaw removal degree layer; superimposing the face image, the black spot removal layer and the brightening layer to obtain a black spot-free layer, and superimposing the black spot-free layer, the white spot removal layer and the darkening layer to obtain a flaw-free layer; obtaining a target image according to the flaw-free layer, thereby improving the image processing effect.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, system, device and storage medium. Background Art

[0002] Portrait photo retouching technology is widely used in advertising, magazines, photoshoots, and photography. With the rapid advancement of technology, automatic photo retouching has been incorporated into mainstream beauty apps. With the development of material and spiritual life, people's expectations for photo retouching are becoming increasingly sophisticated. The original crude skin smoothing effect using blurring is no longer able to meet people's rising aesthetic standards. Currently, portrait skin smoothing and blemish removal mainly rely on various filtering methods and image filling techniques. However, these methods only focus on smoothness and flatness, ignoring details such as skin texture. This results in a severe "Photoshop effect" and cannot meet the current needs of high-pixel selfies and commercial retouching scenarios. Summary of the Invention

[0003] The embodiments of the present application aim to solve the problem of blurred and lost skin texture during portrait skin resurfacing and blemish removal by providing an image processing method, system, device, and storage medium.

[0004] The present invention provides an image processing method, which includes:

[0005] Performing blemish mask processing on the face image to obtain a black blemish mask and a white blemish mask, and determining a black blemish removal degree layer and a white blemish removal degree layer according to the face image;

[0006] Performing a brightening curve process on the face image to obtain a brightening layer and performing a darkening curve process on the face image to obtain a darkening layer;

[0007] Determining a black spot removal layer according to the black blemish mask and the black blemish removal degree layer, and determining a white spot removal layer according to the white blemish mask and the white blemish removal degree layer;

[0008] Superimposing the face image, the dark spot removal layer, and the brightening layer to obtain a dark spot-free layer, and superimposing the dark spot-free layer, the white spot removal layer, and the darkening layer to obtain a flawless layer;

[0009] A target image is obtained according to the flawless layer.

[0010] In one embodiment, the step of performing defect mask processing on the facial image to obtain a black defect mask and a white defect mask includes:

[0011] Performing inversion processing on the face image to obtain an inverted face image;

[0012] Perform blemish masking on the face image to obtain a black blemish mask, and perform blemish masking on the inverted face image to obtain a white blemish mask;

[0013] Among them, the process of the blemish masking includes:

[0014] Obtain the blue channel image of the face image and the skin color mask after binarizing the face image;

[0015] Perform high-contrast retention processing on the blue channel image for a preset number of times to obtain a high-contrast layer after each high-contrast retention processing; among them, the radius coefficients used for each high-contrast retention processing are different;

[0016] Perform binarization processing on the high-contrast layer after each high-contrast retention processing to obtain the corresponding binarized image;

[0017] Multiply the union result obtained by union processing of the binarized images with the skin color mask to obtain a blemish binarized mask corresponding to the face image;

[0018] Perform connected component detection processing on the blemish binarized mask to obtain a blemish mask.

[0019] In one embodiment, the step of performing connected component detection processing on the blemish binarized mask to obtain a blemish mask includes:

[0020] Delete the connected components in the blemish binarized mask whose aspect ratio is greater than a preset value to obtain the blemish mask;

[0021] Alternatively, delete the connected components in the blemish binarized mask where any one of the length, width or area does not meet the first preset condition to obtain the blemish mask;

[0022] Alternatively, determine the average brightness difference between the connected component part or the non-connected component part in the blemish binarized mask, and delete the connected components whose average brightness of the connected component part or the average brightness difference of the non-connected component part does not meet the second preset condition to obtain the blemish mask.

[0023] In one embodiment, the determining the black blemish removal degree layer and the white blemish removal degree layer according to the face image includes:

[0024] Mix the blue channel image of the face image and the green channel image of the face image to obtain a mixed single-channel layer;

[0025] Perform high-contrast retention processing on the mixed single-channel layer to obtain a high-contrast layer;

[0026] The high-contrast layer is superimposed and enhanced to obtain a black defect removal degree layer, and the high-contrast layer is inverted, and the high-contrast layer after the inversion processing is superimposed and enhanced to obtain a white defect removal degree layer.

[0027] In one embodiment, the steps of determining the black spot removal layer according to the black blemish mask and the black blemish removal degree layer, and determining the white spot removal layer according to the white blemish mask and the white blemish removal degree layer include:

[0028] Performing edge softening processing and superposition processing on the black defect mask and the white defect mask respectively to obtain corresponding superposition-processed black defect mask and white defect mask;

[0029] The black blemish mask after superimposition processing is multiplied by the black blemish removal degree layer to obtain a black spot removal layer, and the white blemish mask after superimposition processing is multiplied by the white blemish removal degree layer to obtain a white spot removal layer.

[0030] In one embodiment, the step of performing edge softening processing and superimposing processing on the black defect mask and the white defect mask respectively to obtain the superimposed black defect mask and the white defect mask comprises:

[0031] Performing Gaussian blur processing and color filter overlay processing on the black defect mask and the white defect mask respectively, to obtain corresponding black defect mask and white defect mask with softened edges;

[0032] The black defect mask and the edge-softened black defect mask are superimposed to obtain a superimposed black defect mask, and the white defect mask and the edge-softened white defect mask are superimposed to obtain a superimposed white defect mask.

[0033] In one embodiment, the step of obtaining a target image based on the flawless layer includes:

[0034] Determine a second black defect removal degree layer corresponding to the defect-free layer;

[0035] performing black shadow detection processing on the second black defect removal level layer to obtain a black shadow removal layer;

[0036] Performing brightening curve processing on the flawless layer to obtain a flawless brightened layer;

[0037] The target image is obtained by superimposing the flawless brightening layer, the flawless layer and the black shadow removal layer.

[0038] In addition, to achieve the above object, the present application further provides an image processing system, which includes:

[0039] A first determination module, configured to perform defect masking processing on a face image to obtain a black defect mask and a white defect mask, and determine a black defect removal degree layer and a white defect removal degree layer according to the face image;

[0040] A brightening or darkening processing module, configured to perform brightening curve processing on the face image to obtain a brightening layer and perform darkening curve processing on the face image to obtain a darkening layer;

[0041] A second determination module, configured to determine a black spot removal layer according to the black defect mask and the black defect removal degree layer, and determine a white spot removal layer according to the white defect mask and the white defect removal degree layer;

[0042] An overlay module, configured to overlay the face image, the black spot removal layer, and the brightening layer to obtain a black spot-free layer, and overlay the black spot-free layer, the white spot removal layer, and the darkening layer to obtain a defect-free layer;

[0043] A third determination module, configured to obtain a target image according to the defect-free layer.

[0044] In addition, to achieve the above object, the present application further provides an image processing device, which includes: a memory, a processor, and an image processing program stored on the memory and executable on the processor. When the image processing program is executed by the processor, the steps of the above image processing method are implemented.

[0045] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the above image processing method are implemented.

[0046] In the technical solution of an image processing method, system, device, and storage medium provided in the embodiments of the present application, a blemish mask processing is performed on a face image to obtain a black blemish mask and a white blemish mask, and a black blemish removal degree layer and a white blemish removal degree layer are determined according to the face image; a brightening curve processing is performed on the face image to obtain a brightening layer and a darkening curve processing is performed on the face image to obtain a darkening layer; a black spot removal layer is determined according to the black blemish mask and the black blemish removal degree layer, and a white spot removal layer is determined according to the white blemish mask and the white blemish removal degree layer; the face image, the black spot removal layer, and the brightening layer are superimposed to obtain a black spot-free layer, and the black spot-free layer, the white spot removal layer, and the darkening layer are superimposed to obtain a blemish-free layer; a target image is obtained according to the blemish-free layer. Due to the "hyperbola skin smoothing" process, by creating a brightening layer and a darkening layer, and automatically removing blemishes according to the detection layer combined with the removal degree, brightening the dark areas and darkening the bright areas, blemish removal and skin smoothing are performed, while not losing the skin texture and smoothing the bright and dark parts of the skin. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic structural diagram of an image processing device related to the solution of the embodiments of the present application;

[0048] Figure 2 It is a schematic flowchart of the first embodiment of the image processing method of the present application;

[0049] Figure 3 It is a functional module diagram of the image processing system of the present application.

[0050] The realization, functional features, and advantages of the purpose of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above accompanying drawings are only diagrams of one embodiment and not all of the invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To solve the problem of skin texture blurring and loss during portrait skin smoothing and blemish removal processes, this application proposes an image processing method. By performing blemish masking on a face image, a black blemish mask and a white blemish mask are obtained, and a black blemish removal degree layer and a white blemish removal degree layer are determined based on the face image; the face image is processed with a brightening curve to obtain a brightening layer and processed with a darkening curve to obtain a darkening layer; a black spot removal layer is determined based on the black blemish mask and the black blemish removal degree layer, and a white spot removal layer is determined based on the white blemish mask and the white blemish removal degree layer; the face image, the black spot removal layer, and the brightening layer are superimposed to obtain a black spot-free layer, and the black spot-free layer, the white spot removal layer, and the darkening layer are superimposed to obtain a blemish-free layer; the target image is obtained based on the blemish-free layer. Due to the "hyperbola skin smoothing" process, by creating a brightening layer and a darkening layer, and automatically removing blemishes according to the detection layer combined with the removal degree, brightening the dark areas and darkening the bright areas, blemish removal and skin smoothing are performed, while not losing skin texture and smoothing the bright and dark parts of the skin.

[0052] To better understand the above technical solution, the exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] As Figure 1 shown, Figure 1 It is a schematic structural diagram of the hardware operating environment related to the solution of the embodiment of this application.

[0054] It should be noted that Figure 1 it can be a schematic structural diagram of the hardware operating environment of an image processing device.

[0055] As Figure 1As shown in the figure, the image processing device may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to implement the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art can understand that Figure 1 the structure of the image processing device shown in

[0057] does not constitute a limitation on the image processing device, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Figure 1 As shown in

[0058] the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and an image processing program. Among them, the operating system is a program for managing and controlling the hardware and software resources of the image processing device, and for running the image processing program and other software or programs. Figure 1 In the image processing device shown in

[0059] the user interface 1003 is mainly used to connect to a terminal and perform data communication with the terminal; the network interface 1004 is mainly used to connect to a background server and perform data communication with the background server; the processor 1001 may be used to call the image processing program stored in the memory 1005.

[0060] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are performed:

[0061] Perform defect masking processing on the face image to obtain a black defect mask and a white defect mask, and determine a black defect removal degree layer and a white defect removal degree layer according to the face image;

[0062] Perform a brightening curve processing on the face image to obtain a brightening layer and perform a darkening curve processing on the face image to obtain a darkening layer;

[0063] Determine a black spot removal layer based on the black flaw mask and the black flaw removal degree layer, and determine a white spot removal layer based on the white flaw mask and the white flaw removal degree layer;

[0064] Overlay the face image, the black spot removal layer, and the brightening layer to obtain a black spot-free layer, and overlay the black spot-free layer, the white spot removal layer, and the darkening layer to obtain a flaw-free layer;

[0065] Obtain a target image based on the flaw-free layer.

[0066] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:

[0067] Perform an inversion process on the face image to obtain an inverted face image;

[0068] Perform a flaw mask process on the face image to obtain a black flaw mask, and perform a flaw mask process on the inverted face image to obtain a white flaw mask;

[0069] Among them, the process of the flaw mask process includes:

[0070] Obtain the blue channel image of the face image and the skin color mask after the face image is binarized;

[0071] Perform a high-contrast retention process on the blue channel image for a preset number of times to obtain a high-contrast layer after each high-contrast retention process; among them, the radius coefficients used for each high-contrast retention process are different;

[0072] Perform a binarization process on the high-contrast layer after each high-contrast retention process to obtain a corresponding binarized image;

[0073] Multiply the union result obtained by the union process of each binarized image by the skin color mask to obtain a flaw binarized mask corresponding to the face image;

[0074] Perform a connected component detection process on the flaw binarized mask to obtain a flaw mask.

[0075] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:

[0076] Delete the connected components in the flaw binarized mask whose aspect ratio is greater than a preset value to obtain the flaw mask;

[0077] Alternatively, delete the connected components in the flaw binarized mask whose length, width, or area does not meet the first preset condition to obtain the flaw mask;

[0078] Alternatively, determine the average luminance difference of the connected region part or the non-connected region part in the defect binary mask, and delete the connected regions where the average luminance of the connected region part or the average luminance difference of the non-connected region part does not meet the second preset condition, to obtain the defect mask.

[0079] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:

[0080] Mix the blue channel image of the face image and the green channel image of the face image to obtain a mixed single-channel layer;

[0081] Perform high-contrast retention processing on the mixed single-channel layer to obtain a high-contrast layer;

[0082] Perform overlay processing and enhancement processing on the high-contrast layer to obtain a black defect removal degree layer, and perform inversion processing on the high-contrast layer, and perform overlay processing and enhancement processing on the inverted high-contrast layer to obtain a white defect removal degree layer.

[0083] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:

[0084] Perform edge softening processing and overlay processing on the black defect mask and the white defect mask respectively to obtain the corresponding black defect mask and white defect mask after overlay processing;

[0085] Multiply the black defect mask after overlay processing by the black defect removal degree layer to obtain a black spot removal layer, and multiply the white defect mask after overlay processing by the white defect removal degree layer to obtain a white spot removal layer.

[0086] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed:

[0087] Perform Gaussian blur processing and screen overlay processing on the black defect mask and the white defect mask respectively in sequence to obtain the corresponding black defect mask and white defect mask after edge softening;

[0088] Perform overlay processing on the black defect mask and the black defect mask after edge softening to obtain the black defect mask after overlay processing, and perform overlay processing on the white defect mask and the white defect mask after edge softening to obtain the white defect mask after overlay processing.

[0089] When the processor 1001 calls the image processing program stored in the memory 1005, the following operations are also performed: determining a second black defect removal degree layer corresponding to the defect-free layer;

[0090] Performing a black shadow detection process on the second black defect removal degree layer to obtain a black shadow removal layer;

[0091] Performing a brightening curve process on the defect-free layer to obtain a defect-free brightening layer;

[0092] Overlaying the defect-free brightening layer, the defect-free layer, and the black shadow removal layer to obtain a target image.

[0093] The technical solution of the present application will be introduced below by way of embodiments.

[0094] As Figure 2 shown, in the first embodiment of the present application, the image processing method of the present application includes the following steps:

[0095] Step S110, performing a defect mask process on a face image to obtain a black defect mask and a white defect mask, and determining a black defect removal degree layer and a white defect removal degree layer according to the face image.

[0096] In this embodiment, the face image is an image obtained by performing face positioning on an input image to be processed. Among them, the specific method for obtaining the face image is: after obtaining the input image, obtaining face point information by detecting face key points; then obtaining a face rectangular image including the entire head through the face point information; and cropping the input image according to the face rectangular image to obtain the image texture within the face rectangle, and thus the face image can be obtained.

[0097] The process of the present application for determining the black defect mask and the white defect mask according to the face image is essentially: determining a skin color mask according to the face image, and then obtaining the black defect mask and the white defect mask according to the skin color mask and the face image. Among them, the process of determining the skin color mask according to the face image is: after obtaining the face image, obtaining the face width through the face key point information. Taking the face image as the input, obtaining the skin color mask within the face image through a skin color segmentation model; the specific method is to pre-obtain a large number of portrait pictures marked with skin colors, perform model training using deep learning methods, and use the trained model to predict the skin color area of the input image, which can better distinguish the skin, facial features, hair, and background. After obtaining the skin color mask, obtaining the black defect mask and the white defect mask according to the skin color mask and the face image.

[0098] In one embodiment, performing a defect mask process on a face image to obtain a black defect mask and a white defect mask specifically includes the following steps:

[0099] Step S111: Perform an inversion process on the face image to obtain the face image after the inversion process.

[0100] Step S112: Perform a defect mask process on the face image to obtain a black defect mask, and perform a defect mask process on the face image after the inversion process to obtain a white defect mask.

[0101] In this embodiment, the processing processes of the black defect mask and the white defect mask are similar. The main difference is that the white defect mask needs to perform an inversion process on the face image, and then perform a defect mask process based on the face image after the inversion process to obtain the white defect mask. The black defect mask can be directly obtained by performing a defect mask process on the face image.

[0102] Among them, the specific steps of performing a defect mask process on the face image include:

[0103] Step S1121: Obtain the blue channel image of the face image and the skin color mask after the face image is binarized.

[0104] In this embodiment, the obtained skin color mask is binarized to obtain the skin color mask after the face image is binarized. Then, the blue channel image in the face image is taken.

[0105] Step S1122: Perform a high-contrast retention process on the blue channel image for a preset number of times to obtain a high-contrast layer after each high-contrast retention process; among them, the radius coefficients used for each high-contrast retention process are different.

[0106] In this embodiment, when performing a high-contrast retention process on the blue channel image for a preset number of times, a high-contrast layer after each high-contrast retention process is obtained. Among them, the radius coefficients used for each high-contrast retention process are different. The radius coefficient is determined according to the face width Wface obtained from the above-mentioned face key points. Three radii from large to small are calculated by multiplying the above-mentioned face width Wface by the coefficients 0.04, 0.02, and 0.01 respectively; the three radii are used to perform a high-contrast retention process on the detection image respectively, so as to obtain a high-contrast retention layer after each high-contrast retention process. In other embodiments, the preset number can be determined according to actual development requirements.

[0107] Step S1123: Perform a binarization process on each high-contrast layer after the high-contrast retention process to obtain a corresponding binarized image.

[0108] In this embodiment, after obtaining the high-contrast retention layer, the three obtained high-contrast layers are respectively binarized with a unified threshold to obtain corresponding binarized images.

[0109] Step S1124: Multiply the union result obtained by union processing of each of the binary images with the skin color mask to obtain a binary mask of defects corresponding to the face image.

[0110] In this embodiment, after obtaining the binary images, the union result is obtained by performing union on the above-obtained 3 binary images. This union result is to combine the 3 binary images into one binary image. Multiply this union result with the skin color mask after binary processing of the face image to take the intersection, and finally obtain a binary mask of defects within the skin area.

[0111] Step S1125: Perform connected component detection processing on the binary mask of defects to obtain a defect mask.

[0112] In this embodiment, after obtaining the binary mask of defects through the above method, perform connected component detection processing on the binary mask of defects. The specific method is to traverse all detected connected components, exclude all connected components that do not conform to the defect characteristics, and only retain the connected components that conform to the defect characteristics as the final black defect mask.

[0113] In one embodiment, performing connected component detection processing on the binary mask of defects to obtain a black defect mask specifically includes: deleting the connected components in the binary mask of defects whose aspect ratio is greater than a preset value to obtain a black defect mask; or deleting the connected components in the binary mask of defects where any one of the length, width, or area does not meet the first preset condition to obtain a black defect mask; or determining the average brightness difference between the connected component part and the non-connected component part in the binary mask of defects, and deleting the connected components where the average brightness of the connected component part or the average brightness difference of the non-connected component part does not meet the second preset condition to obtain a black defect mask.

[0114] Optionally, it is also possible to exclude the connected components with an aspect ratio greater than the first preset value. Such areas are more likely to be face edge shadows, hair, etc. Optionally, the first preset condition is that the connected component where any one of the length, width, or area is greater than a fixed preset value, and when it does not meet the first preset condition, the corresponding connected component is deleted. Such areas are more likely to be nose wing shadows, etc. Optionally, the second preset condition is the average brightness difference between the connected component part and the non-connected component part in the connected component rectangle, and exclude the connected components with an average brightness difference greater than a fixed value to ensure that personal characteristic information such as moles and warts is not removed.

[0115] It should be noted that the above preset value, first preset value, fixed preset value, and fixed value are different and can be set according to actual situations.

[0116] In the technical solution of this embodiment, the connected regions that do not conform to the defect characteristics are excluded, so as to obtain the connected regions that conform to the defect characteristics as the final black defect mask.

[0117] In one embodiment, determining the black defect removal degree layer and the white defect removal degree layer according to the face image specifically includes the following steps:

[0118] Step S2111, mixing the blue channel image of the face image and the green channel image of the face image to obtain a mixed single-channel layer.

[0119] In this embodiment, the black defect removal degree layer is also divided into the black defect removal degree layer and the white defect removal degree layer. Specifically, calculating the black defect removal degree layer includes: obtaining the blue channel image and the green channel image in the face image. Mixing the blue channel image of the person and the green channel image to obtain a mixed single-channel layer. Among them, the mixed single-channel layer is calculated by using the formula Imgface.b*Imgface.g / 2, where b represents the blue channel image and g represents the green channel image.

[0120] Step S2112, performing a high-contrast retention process on the mixed single-channel layer to obtain a high-contrast layer.

[0121] In this embodiment, after obtaining the mixed single-channel layer, a high-contrast retention process is performed on the mixed single-channel layer, and the radius is calculated by the formula r = Wface / 50 to obtain a high-contrast layer.

[0122] Step S2113, performing an overlay process and an enhancement process on the high-contrast layer to obtain the black defect removal degree layer, and performing an inversion process on the high-contrast layer, and performing an overlay process and an enhancement process on the inverted high-contrast layer to obtain the white defect removal degree layer.

[0123] In this embodiment, the high-contrast layer is then superimposed with itself in the "hard light mode" to obtain the superimposed layer 1, and the superimposed layer 1 is then superimposed with the superimposed layer 1 in the second "hard light mode" to obtain the superimposed layer. Among them, the hard light overlay formula is:

[0124]

[0125] where p is a pixel.

[0126] Finally, the superimposed layer is enhanced according to the image enhancement formula: Boost = (75 - blendlayer) / 89 to finally obtain the black defect removal degree layer, where blendlayer in the above formula is the superimposed layer.

[0127] In this embodiment, the process of calculating the white flaw removal degree layer is the same as that of calculating the black flaw removal degree layer. The high-contrast layer in the black flaw removal degree layer is inverted, and then the inverted high-contrast layer is successively subjected to the first "hard light mode" overlay process, the second "hard light mode" overlay process, and enhancement process in the same manner to obtain the white flaw removal degree layer.

[0128] In the technical solution of this embodiment, the black flaw removal degree layer and the white flaw removal degree layer are obtained by processing the face image.

[0129] Step S120, perform a brightening curve process on the face image to obtain a brightening layer and perform a darkening curve process on the face image to obtain a darkening layer.

[0130] In this embodiment, perform a brightening curve process on the face image to obtain a brightening layer, and perform a darkening curve process on the face image to obtain a darkening layer. The brightening layer and the darkening layer are used as the upper layers for black and white spot removal overlay; the brightening curve parameters are determined by three points (0, 0), (120, 146), and (255, 255), and the darkening parameters are determined by three points (0, 0), (136, 118), and (255, 255).

[0131] Step S130, determine the black spot removal layer according to the black flaw mask and the black flaw removal degree layer, and determine the white spot removal layer according to the white flaw mask and the white flaw removal degree layer.

[0132] In this embodiment, after obtaining the black flaw mask and the white flaw mask in the above manner, further determine the black spot removal layer according to the black flaw mask and the black flaw removal degree layer, and determine the white spot removal layer according to the white flaw mask and the white flaw removal degree layer.

[0133] In one embodiment, determining the black spot removal layer according to the black flaw mask and the black flaw removal degree layer, and determining the white spot removal layer according to the white flaw mask and the white flaw removal degree layer specifically includes the following steps:

[0134] Step S131, perform edge softening and overlay processing on the black flaw mask and the white flaw mask respectively to obtain the corresponding black flaw mask and white flaw mask after overlay processing.

[0135] In this embodiment, edge softening and overlay processing are performed on the black defect mask, so as to obtain the corresponding black defect mask after overlay processing. And, edge softening and overlay processing are performed on the white defect mask, so as to obtain the corresponding white defect mask after overlay processing.

[0136] In one embodiment, the steps of separately performing edge softening and overlay processing on the black defect mask and the white defect mask to obtain the corresponding black defect mask and white defect mask after overlay processing are specifically as follows:

[0137] Step S1311, perform Gaussian blur processing and screen overlay processing on the black defect mask and the white defect mask in sequence to obtain the corresponding black defect mask and white defect mask after edge softening;

[0138] Step S1312, perform overlay processing on the black defect mask and the black defect mask after edge softening to obtain the black defect mask after overlay processing, and perform overlay processing on the white defect mask and the white defect mask after edge softening to obtain the white defect mask after overlay processing.

[0139] In this embodiment, Gaussian blur is performed on the black defect mask, and the radius calculation formula is r = Wface / 125 to obtain the black defect mask after Gaussian blur; then, the black defect mask after Gaussian blur is subjected to self-screen overlay to obtain the black defect mask after edge softening. Among them, the screen overlay formula is softmask = 1 - (1 - blurmask) * (1 - blurmask), where blurmask is the black defect mask after Gaussian blur and softmask is the black defect mask after edge softening; then, the initial black defect mask (i.e., the black defect mask before Gaussian blur processing) and the black defect mask after edge softening are overlaid in the lighten mode with an opacity of 80%, that is, spotmask = max(softmask, mask * 0.8), where max represents taking the maximum value, to obtain the black defect mask after overlay processing.

[0140] Optionally, the method for obtaining the white defect mask after overlay processing according to the white defect mask is the same as the method for obtaining the black defect mask after overlay processing for the above black defect mask, and will not be elaborated here.

[0141] Step S132, multiply the black defect mask after overlay processing by the black defect removal degree layer to obtain the black spot removal layer, and multiply the white defect mask after overlay processing by the white defect removal degree layer to obtain the white spot removal layer.

[0142] In this embodiment, the black flaw removal degree layer and the white flaw removal degree layer are obtained through the above method. Multiply the superimposed black flaw mask obtained in the above process by the black flaw removal degree layer to obtain the black spot removal layer. Multiply the superimposed white flaw mask obtained in the above process by the white flaw removal degree layer to obtain the white spot removal layer.

[0143] Step S140, superimpose the face image, the black spot removal layer, and the lightening layer to obtain a black spot-free layer, and superimpose the black spot-free layer, the white spot removal layer, and the darkening layer to obtain a flaw-free layer.

[0144] In this embodiment, after obtaining the white spot removal layer, the black spot removal layer, the lightening layer, and the darkening layer, remove the black spots. Removing the black spots specifically includes the following steps:

[0145] Use the following formula for layer superimposition with the face image as the bottom layer, the lightening layer as the upper layer, and the black spot removal layer as the mask:

[0146] Result = Imgface * (1 - blackspotmask) + Imglighten * blackspotmask;

[0147] Where Imgface is the face image, blackspotmask is the black spot removal layer, and Imglighten is the lightening layer, to obtain a black spot-free layer.

[0148] After obtaining the black spot-free layer, use the black spot-free layer as the bottom layer, the darkening layer as the upper layer, and the white spot removal layer as the mask for layer superimposition to obtain a flaw-free layer.

[0149] In this embodiment, the lightening and darkening layers are made through preset parameters, and the flaws are automatically removed according to the detection layer combined with the removal degree, smoothing the bright and dark parts of the skin without losing the skin texture.

[0150] Step S150, obtain the target image according to the flaw-free layer.

[0151] In this embodiment, obtain the white flaw removal degree layer according to the obtained flaw-free layer; then perform black shadow detection. This time, mainly detect the areas in the skin that are not flat and have large light-colored shadows. Through the black shadow detection process on the white flaw removal degree layer, obtain the result layer. Finally, paste the result layer back onto the input image according to the positioning rectangular frame of the face image to obtain the final effect image.

[0152] In one embodiment, obtaining the target image according to the flaw-free layer specifically includes the following steps:

[0153] Step S151: Determine the second black flaw removal degree layer corresponding to the flawless layer.

[0154] In this embodiment, the obtained flawless layer is repeatedly returned to execute the step of calculating the removal degree of black spot flaws, and the second black flaw removal degree layer is obtained.

[0155] Step S152: Perform black shadow detection processing on the second black flaw removal degree layer to obtain a black shadow removal layer.

[0156] In this embodiment, perform binarization processing on the second black flaw removal degree layer, perform connected component detection on the binarized second black flaw removal degree layer, and then exclude connected components that are relatively too large, too narrow and long, and the effective areas of the connected components are too scattered. The process of connected component detection processing can refer to the above process of connected component detection processing of the flaw binarization mask, and will not be elaborated here.

[0157] In this embodiment, after performing connected component detection on the second black flaw removal degree layer, a connected component layer after connected component detection and retention is obtained. Multiply the retained connected component layer after exclusion with the skin color mask and the second black flaw removal degree layer corresponding to the flaw removal layer in sequence to obtain a black shadow removal layer.

[0158] Step S153: Perform brightening curve processing on the flawless layer to obtain a brightened flawless layer.

[0159] In this embodiment, brighten the flawless layer in the same manner and with the same parameters as above to obtain a brightened flawless layer.

[0160] Step S154: Overlay the brightened flawless layer, the flawless layer, and the black shadow removal layer to obtain a target image.

[0161] In this embodiment, use the flawless layer as the bottom layer, the brightened flawless layer as the upper layer, and the black shadow removal layer as the mask, and perform layer overlay according to the formula: Result = Imgface * (1 - blackspotmask) + Imglighten * blackspotmask to obtain a result layer. Finally, paste the result layer back onto the input image according to the positioning rectangular frame of the face image to obtain the final effect image.

[0162] According to the above technical solution, the technical solution of this application adopts a retouching process similar to "hyperbola skin smoothing" in commercial retouching. First, automatically detect fine flaws, then calculate the flaw removal intensity of the skin through an image enhancement process, and finally, through a series of layer overlay methods of brightening and darkening curves, achieve the effects of removing flaws and smoothing the skin, while also retaining the personal characteristic information of the user himself.

[0163] An embodiment of the present application provides an embodiment of an image processing method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0164] Based on the same inventive concept, Figure 3 This is a functional block diagram of the image processing system of the present application. The present application also provides an image processing system, and the image processing system includes:

[0165] A first determination module 10 is configured to perform a defect mask processing on a face image to obtain a black defect mask and a white defect mask, and determine a black defect removal degree layer and a white defect removal degree layer according to the face image. In one embodiment, the first determination module 10 is configured to perform an inversion processing on the face image to obtain an inverted face image; perform a defect mask processing on the face image to obtain a black defect mask, and perform a defect mask processing on the inverted face image to obtain a white defect mask; wherein, the process of the defect mask processing includes: obtaining a blue channel image of the face image and a skin color mask after the face image is binarized; performing a high-contrast retention processing on the blue channel image for a preset number of times to obtain a high-contrast layer after each high-contrast retention processing; wherein, the radius coefficient used for each high-contrast retention processing is different; performing a binarization processing on each high-contrast layer after the high-contrast retention processing to obtain a corresponding binarized image; multiplying the union result obtained by union processing of each binarized image with the skin color mask to obtain a defect binarized mask corresponding to the face image; performing a connected component detection processing on the defect binarized mask to obtain a defect mask. In one embodiment, the first determination module 10 is configured to delete a connected component in the defect binarized mask whose aspect ratio is greater than a preset value to obtain the defect mask; or, delete a connected component in the defect binarized mask whose length, width or area does not meet a first preset condition to obtain the defect mask; or, determine an average brightness difference between a connected component part or a non-connected component part in the defect binarized mask, and delete a connected component whose average brightness of the connected component part or the average brightness difference of the non-connected component part does not meet a second preset condition to obtain the defect mask. In one embodiment, the first determination module 10 is configured to perform a mixing processing on the blue channel image of the face image and the green channel image of the face image to obtain a mixed single-channel layer; perform a high-contrast retention processing on the mixed single-channel layer to obtain a high-contrast layer; perform an overlay processing and an enhancement processing on the high-contrast layer to obtain a black defect removal degree layer, and perform an inversion processing on the high-contrast layer, and perform an overlay processing and an enhancement processing on the inverted high-contrast layer to obtain a white defect removal degree layer.

[0166] A brightening or darkening processing module 20 for performing a brightening curve processing on the face image to obtain a brightening layer and performing a darkening curve processing on the face image to obtain a darkening layer;

[0167] A second determination module 30 for determining a black spot removal layer according to the black flaw mask and the black flaw removal degree layer, and determining a white spot removal layer according to the white flaw mask and the white flaw removal degree layer; In one embodiment, the second determination module 30 is configured to perform edge softening processing and superposition processing on the black flaw mask and the white flaw mask respectively to obtain a corresponding superposed black flaw mask and white flaw mask; multiplying the superposed black flaw mask by the black flaw removal degree layer to obtain a black spot removal layer, and multiplying the superposed white flaw mask by the white flaw removal degree layer to obtain a white spot removal layer. In one embodiment, the second determination module 30 is configured to perform Gaussian blur processing and screen overlay processing on the black flaw mask and the white flaw mask respectively in sequence to obtain a corresponding edge-softened black flaw mask and white flaw mask; performing superposition processing on the black flaw mask and the edge-softened black flaw mask to obtain a superposed black flaw mask, and performing superposition processing on the white flaw mask and the edge-softened white flaw mask to obtain a superposed white flaw mask.

[0168] A superposition module 40 for superposing the face image, the black spot removal layer and the brightening layer to obtain a black spot-free layer, and superposing the black spot-free layer, the white spot removal layer and the darkening layer to obtain a flaw-free layer;

[0169] A third determination module 50 for obtaining a target image according to the flaw-free layer. In one embodiment, the third determination module 50 is configured to determine a second black flaw removal degree layer corresponding to the flaw-free layer; performing black shadow detection processing on the second black flaw removal degree layer to obtain a black shadow removal layer; performing a brightening curve processing on the flaw-free layer to obtain a flaw-free brightening layer; superposing the flaw-free brightening layer, the flaw-free layer and the black shadow removal layer to obtain a target image.

[0170] The specific implementation manner of the image processing system of this application is basically the same as each embodiment of the above image processing method, and will not be elaborated here.

[0171] Based on the same inventive concept, an embodiment of this application also provides a storage medium, the storage medium stores an image processing program, and when the image processing program is executed by a processor, it implements each step of the above-mentioned image processing method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0172] Since the storage medium provided in the embodiments of the present application is the storage medium used to implement the methods in the embodiments of the present application, based on the methods introduced in the embodiments of the present application, those skilled in the art can understand the specific structure and variations of the storage medium, and thus will not be elaborated herein. Any storage medium used in the methods of the embodiments of the present application falls within the scope of protection of the present application.

[0173] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0175] These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device that implements the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0177] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0178] Although the preferred embodiments of the present application have been described, additional changes and modifications can be made by those skilled in the art once they learn the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Perform inversion processing on the face image to obtain an inverted face image; perform defect mask processing on the face image to obtain a black defect mask, and perform defect mask processing on the inverted face image to obtain a white defect mask; wherein the defect mask processing process includes: obtaining a blue channel image of the face image and a skin color mask after binarization processing of the face image; performing high contrast retention processing on the blue channel image for a preset number of times to obtain a high contrast layer after each high contrast retention processing; wherein the radius coefficient used in each high contrast retention processing is different; performing binarization processing on the high contrast layer after each high contrast retention processing to obtain a corresponding binarized image; multiplying the union result obtained by the union processing of each of the binarized images by the skin color mask to obtain a defect binarized mask corresponding to the face image; performing connected domain detection processing on the defect binarized mask to obtain a defect mask; Mixing the blue channel image of the facial image and the green channel image of the facial image to obtain a mixed single-channel layer; performing high-contrast retention processing on the mixed single-channel layer to obtain a high-contrast layer; performing superposition processing and enhancement processing on the high-contrast layer to obtain a black blemish removal degree layer; performing inversion processing on the high-contrast layer, and superposition processing and enhancement processing on the inverted high-contrast layer to obtain a white blemish removal degree layer; performing brightening curve processing on the facial image to obtain a brightening layer and performing darkening curve processing on the facial image to obtain a darkening layer; Determining a black spot removal layer according to the black blemish mask and the black blemish removal degree layer, and determining a white spot removal layer according to the white blemish mask and the white blemish removal degree layer; Superimposing the face image, the dark spot removal layer, and the brightening layer to obtain a dark spot-free layer, and superimposing the dark spot-free layer, the white spot removal layer, and the darkening layer to obtain a flawless layer; A target image is obtained according to the flawless layer.

2. The image processing method according to claim 1, wherein: The step of performing connected domain detection on the defect binary mask to obtain the defect mask comprises: Deleting connected domains with aspect ratios greater than a preset value in the defect binary mask to obtain the defect mask; Alternatively, the connected domains in the defect binary mask whose length, width or area do not meet the first preset condition are deleted to obtain the defect mask; Alternatively, the average brightness difference of the connected domain part or the non-connected domain part in the defect binary mask is determined, and the connected domains whose average brightness of the connected domain part or the average brightness difference of the non-connected domain part does not meet the second preset condition are deleted to obtain the defect mask.

3. The image processing method according to claim 1, wherein: The steps of determining the black spot removal layer according to the black blemish mask and the black blemish removal degree layer, and determining the white spot removal layer according to the white blemish mask and the white blemish removal degree layer include: Performing edge softening processing and superposition processing on the black defect mask and the white defect mask respectively to obtain corresponding superposition-processed black defect mask and white defect mask; The black blemish mask after superimposition processing is multiplied by the black blemish removal degree layer to obtain a black spot removal layer, and the white blemish mask after superimposition processing is multiplied by the white blemish removal degree layer to obtain a white spot removal layer.

4. The image processing method according to claim 3, wherein: The step of performing edge softening processing and superimposing processing on the black defect mask and the white defect mask respectively to obtain the superimposed black defect mask and the white defect mask comprises: Performing Gaussian blur processing and color filter overlay processing on the black defect mask and the white defect mask respectively, to obtain corresponding black defect mask and white defect mask with softened edges; The black defect mask and the edge-softened black defect mask are superimposed to obtain a superimposed black defect mask, and the white defect mask and the edge-softened white defect mask are superimposed to obtain a superimposed white defect mask.

5. The image processing method according to claim 1, wherein: The step of obtaining a target image based on the flawless layer comprises: Determine a second black defect removal degree layer corresponding to the defect-free layer; performing black shadow detection processing on the second black defect removal level layer to obtain a black shadow removal layer; Performing brightening curve processing on the flawless layer to obtain a flawless brightened layer; The target image is obtained by superimposing the flawless brightening layer, the flawless layer and the black shadow removal layer.

6. An image processing system, characterized in that: The image processing system comprises: The first determination module is used to perform inversion processing on the face image to obtain the face image after inversion processing; perform defect mask processing on the face image to obtain a black defect mask, and perform defect mask processing on the face image after inversion processing to obtain a white defect mask; wherein, the process of the defect mask processing includes: obtaining the blue channel image of the face image and the skin color mask after the face image is binarized; performing high contrast retention processing on the blue channel image for a preset number of times to obtain a high contrast layer after each high contrast retention processing; wherein, the radius coefficient used for each high contrast retention processing is different; performing binarization processing on the high contrast layer after each high contrast retention processing to obtain the corresponding binarization image; multiplying the union result obtained by the union processing of each of the binary images by the skin color mask to obtain a defect binary mask corresponding to the face image; performing connected domain detection processing on the defect binary mask to obtain a defect mask; mixing the blue channel image of the face image and the green channel image of the face image to obtain a mixed single-channel layer; performing high contrast retention processing on the mixed single-channel layer to obtain a high contrast layer; superimposing and enhancing the high contrast layer to obtain a black defect removal degree layer, and inverting the high contrast layer, and superimposing and enhancing the inverted high contrast layer to obtain a white defect removal degree layer; a brightening or darkening processing module, configured to perform brightening curve processing on the face image to obtain a brightened layer and perform darkening curve processing on the face image to obtain a darkened layer; a second determining module, configured to determine a black spot removal layer according to the black blemish mask and the black blemish removal degree layer, and to determine a white spot removal layer according to the white blemish mask and the white blemish removal degree layer; a superposition module, configured to superimpose the facial image, the dark spot removal layer, and the brightening layer to obtain a dark spot-free layer, and to superimpose the dark spot-free layer, the white spot removal layer, and the darkening layer to obtain a flawless layer; A third determining module, configured to obtain a target image based on the flawless layer; The step of performing defect mask processing on the facial image to obtain a black defect mask and a white defect mask includes: determining a skin color mask according to the facial image, and obtaining a black defect mask and a white defect mask according to the skin color mask and the facial image.

7. An image processing device, characterized in that The image processing device includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

Images