Portrait photo skin color processing method and device and server

Abstract

The embodiment of the invention provides a portrait photo skin color processing method and device and a server, and relates to the technical field of image processing. The method comprises the steps of obtaining an original human image photo; performing skin segmentation on the original human image photo to obtain a skin area mask; processing the original human image photo and the skin area mask by adopting a preset skin color brightness unified network to obtain a skin brightness mask of the original human image photo; obtaining a target color mean value of the skin area in the original human image photo according to the original human image photo and the skin area mask; and generating a target skin color unified effect picture corresponding to the original human image picture according to the original human image picture, the skin brightness masking picture and the target color mean value of the skin area. Therefore, the skin color unification problem is decoupled into separate processing of the skin brightness unification module and the color unification module, mutual influence of the color and the brightness is avoided, the processing effect is improved, the processing efficiency is improved, and the image processing cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a portrait photo skin color processing method, device and server. BACKGROUND

[0002] At present, in the field of portrait photo retouching, skin color unification under uneven illumination / color is a very basic but very important function. Due to the influence of camera illumination, exposure and environmental light, it is often easy to have problems such as uneven skin color brightness, too large difference between dark and bright skin colors, etc. The commonly used method in the industry is manual Photoshop retouching, which has high labor cost and time cost, and cannot meet the needs of customers and the market.

[0003] Some solutions automatically perform white balance on the skin area of a portrait image. Since only a traditional color white balance solution is used, some scene exposure and color cast problems are solved, but the color effect and brightness unification of skin color cannot be solved, resulting in unsatisfactory processing effect.

[0004] Therefore, there is an urgent need for a skin color processing method that is efficient and has good processing effect. SUMMARY

[0005] The present application aims to solve the problems of unsatisfactory processing effect in the prior art by providing a portrait photo skin color processing method, device and server.

[0006] To achieve the above-mentioned purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0007] In a first aspect, the embodiments of the present application provide a portrait photo skin color processing method, which comprises:

[0008] obtaining an original portrait photo;

[0009] performing skin segmentation on the original portrait photo to obtain a skin region mask;

[0010] processing the original portrait photo and the skin region mask using a preset skin color brightness unification network to obtain a skin brightness mask graph of the original portrait photo;

[0011] obtaining a target color mean value of the skin region in the original portrait photo according to the original portrait photo and the skin region mask;

[0012] generating a target skin color unification effect graph corresponding to the original portrait photo according to the original portrait photo, the skin brightness mask graph and the target color mean value of the skin region.

[0013] Optionally, the method further comprises:

[0014] According to the skin region mask, performing brightness statistics on the skin region in the original portrait photo to obtain a brightness median value of the skin region.

[0015] According to the brightness median value and the skin region mask, determining each target skin pixel in the skin region with a brightness greater than the brightness median value from the original portrait photo.

[0016] According to the color value of each target skin pixel in the original portrait photo, determining a target color mean value of the skin region.

[0017] Optionally, the method further comprises:

[0018] According to the color value of each target skin pixel in the original portrait photo, calculating a first color mean value of the skin region in a first color space.

[0019] Converting the first color mean value in the first color space to a second color space to obtain a color mean value in the second color space as the target color mean value of the skin region.

[0020] Optionally, the method further comprises:

[0021] Mixing the original portrait photo and the skin brightness mask to obtain an initial skin brightness uniform effect image.

[0022] According to the target color mean value and the initial skin brightness uniform effect image, performing skin color processing on the original portrait photo to generate the target skin color uniform effect image.

[0023] Optionally, the method further comprises:

[0024] According to the target color mean value, performing color adjustment on the initial skin brightness uniform effect image to obtain an adjusted skin color uniform effect image.

[0025] The original portrait photo and the adjusted skin color unified effect image are mixed using a preset skin color unified intensity parameter to generate the target skin color unified effect image.

[0026] Optionally, the target color mean is a color mean in a second color space; and performing color adjustment on the initial skin color uniform effect image according to the target color mean to obtain an adjusted skin color uniform effect image includes:

[0027] According to the skin area mask, traverse the initial skin brightness uniform effect map to obtain a first color value of each skin pixel in the skin area in the first color space;

[0028] converting a first color value of each skin pixel in the first color space into the second color space to obtain a second color value of each skin pixel in the second color space;

[0029] Recombining the hue average value in the target color mean with the saturation value and the lightness value in the second color value of each skin pixel to generate a target color value for each skin pixel in the second color space;

[0030] converting the target color value of each skin pixel in the second color space into the first color space to obtain the target color value of each skin pixel in the first color space;

[0031] The adjusted skin color unified effect image is generated according to the target color value of each skin pixel in the first color space.

[0032] Optionally, the preset skin color brightness unified network is trained in the following manner:

[0033] Generate a sample data set based on multiple sample portrait photos, the sample data set comprising: multiple data pairs, each data pair comprising: one of the sample portrait photos and corresponding multiple sample skin region masks, and multiple sample skin brightness uniform effect maps; different sample portrait photos contain at least different portrait skin regions;

[0034] The sample data set is used to train the initial skin color brightness unified network until a preset stopping iteration condition is met, thereby obtaining the preset skin color brightness unified network.

[0035] Optionally, generating a sample data set based on a plurality of sample portrait photos includes:

[0036] Performing skin segmentation on the sample portrait photo to obtain a corresponding sample skin area mask;

[0037] Adjusting the skin brightness of the sample portrait photo to obtain a sample skin brightness uniform effect image corresponding to the sample portrait photo;

[0038] A data pair is constructed according to the sample portrait photo, the corresponding sample skin area mask, and the corresponding sample skin brightness uniform effect map.

[0039] In a second aspect, an embodiment of the present application provides a device for processing skin color in a portrait photo, the device comprising:

[0040] An acquisition module is used to obtain original portrait photos;

[0041] a segmentation module, configured to perform skin segmentation on the original portrait photo to obtain a skin region mask;

[0042] a processing module, configured to process the original portrait photo and the skin region mask using a preset skin color brightness unified network to obtain a skin brightness mask image of the original portrait photo;

[0043] an acquisition module, configured to acquire a target color mean value of the skin area in the original portrait photo according to the original portrait photo and the skin area mask;

[0044] A generation module is used to generate a target skin color uniform effect map corresponding to the original portrait photo based on the original portrait photo, the skin brightness mask map and the target color mean of the skin area.

[0045] In a third aspect, an embodiment of the present application provides a server comprising: a processor and a storage medium, wherein the processor and the storage medium are communicatively connected via a bus, the storage medium stores program instructions executable by the processor, and the processor calls the program stored in the storage medium to execute the steps of the portrait photo skin color processing method as described in any one of the first aspects.

[0046] Compared with the prior art, this application has the following beneficial effects:

[0047] The embodiment of the present application provides a portrait photo skin color processing method, device and server, the method comprises the following steps: obtaining an original portrait photo; performing skin segmentation on the original portrait photo to obtain a skin region mask; processing the original portrait photo and the skin region mask by using a preset skin color brightness uniform network to obtain a skin brightness mask graph of the original portrait photo; obtaining a target color mean value of the skin region in the original portrait photo according to the original portrait photo and the skin region mask; and generating a target skin color uniform effect graph corresponding to the original portrait photo according to the original portrait photo, the skin brightness mask graph and the target color mean value of the skin region. Thus, the skin color uniform problem is decoupled into skin brightness uniform and color uniform, which are processed separately, the influence of color and brightness on each other is avoided, the processing effect is improved, the processing efficiency is improved, and the image processing cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0049] Figure 1 A flowchart of a portrait photo skin color processing method provided by the embodiment of the present application;

[0050] Figure 2 A flowchart of a method for obtaining a target color mean value of a skin region in an original portrait photo according to the original portrait photo and a skin region mask provided by the embodiment of the present application;

[0051] Figure 3 A flowchart of a method for determining a target color mean value of a skin region according to color values of each target skin pixel in an original portrait photo provided by the embodiment of the present application;

[0052] Figure 4 A flowchart of a method for generating a target skin color uniform effect graph corresponding to an original portrait photo according to the original portrait photo, a skin brightness mask graph and a target color mean value of a skin region provided by the embodiment of the present application;

[0053] Figure 5 A flowchart of a method for generating a target skin color uniform effect graph according to a target color mean value and an initial skin brightness uniform effect graph provided by the embodiment of the present application;

[0054] Figure 6 A flowchart of a method for obtaining an adjusted skin color uniform effect graph according to a target color mean value provided by the embodiment of the present application;

[0055] Figure 7 A flowchart of a training method of a skin color brightness unification network provided by an embodiment of the present application is shown in the figure;

[0056] Figure 8 A structure diagram of a Pix2Pix network provided by an embodiment of the present application is shown in the figure;

[0057] Figure 9 A flowchart of a method for generating a sample data set according to a sample portrait photo provided by an embodiment of the present application is shown in the figure;

[0058] Figure 10 A schematic diagram of a portrait photo skin color processing device provided by an embodiment of the present application is shown in the figure;

[0059] Figure 11 A schematic diagram of a server provided by an embodiment of the present application is shown in the figure.

[0060] Icon: 1001-first acquisition module, 1002-segmentation module, 1003-processing module, 1004-second acquisition module, 1005-generation module, 1101-processor, 1102-storage medium. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0063] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0064] In addition, if the terms "first", "second", etc. are used, they are only used for differentiation description, and cannot be understood as indicating or implying relative importance.

[0065] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0066] The method for processing skin color of a portrait photo provided by the embodiment of the application is explained and described below by way of specific examples. Figure 1 The flowchart of the method for processing skin color of a portrait photo provided by the embodiment of the application. The execution subject of the method can be a server, which can be a device with a computing and processing function, such as a desktop computer, a notebook computer, a smart phone, etc. As shown in the figure, the method comprises the following steps. Figure 1 The method comprises the following steps.

[0067] S101, obtaining an original portrait photo.

[0068] A user inputs an original portrait photo in an image processing software on an electronic device. By way of example, the original portrait photo is an 8K or 16K high-resolution portrait photo, and the photo format is bgra32 format.

[0069] S102, performing skin segmentation on the original portrait photo to obtain a skin region mask.

[0070] The original portrait photo is subjected to binaryzation processing to obtain an image mask corresponding to the original portrait photo, and the image mask is segmented to obtain a skin region mask and a background region mask.

[0071] By way of example, the image mask is a black-and-white binary image, black pixels represent background region pixels, and white pixels represent skin region pixels. In this embodiment, the white pixels are mainly focused on. Specifically, any one of Unet / PspNet / BiSeNet, etc. can be used to implement skin segmentation, and Segment Anything, etc. can also be used to implement skin segmentation.

[0072] The image mask is sliced using a preset size to obtain a plurality of skin region masks.

[0073] By way of example, taking the width of the original portrait photo as W, the height as H, and the preset size as 384, the size of each skin region mask obtained by slicing is 384x384, and the image mask of the original portrait photo is segmented into a plurality of skin region masks with a size of 384x384. Specifically, W is segmented according to the preset size, and then H is segmented according to the preset size, thereby implementing segmentation of the portrait photo. The total number of skin region masks can be calculated using the following formula (1):

[0074] N_w = ceil(W / 384)

[0075] N_h = ceil(H / 384) (1)

[0076] N = N_w x N_h

[0077] Wherein, N is the total number of slices, N_w is the number of wide partitions, and N_h is the number of high partitions.

[0078] It should be noted that, regardless of width or height, there are cases where the preset size cannot be divided by an integer, and the last piece obtained by division is certainly smaller than or equal to the preset size, which will cause the edge part of the portrait photo to be smaller than 384x384 in size after division. At this time, the slices that are less than 384x384 can be completed to 384x384, and the completed area can be filled with black to distinguish the skin area.

[0079] S103, using a preset skin color brightness uniformity network to process the original portrait photo and the skin area mask to obtain a skin brightness mask graph of the original portrait photo.

[0080] Since the preset skin color brightness uniformity network can directly process the original portrait photo and the skin area mask, the performance and effect problems of high-resolution skin brightness uniformity are effectively solved, and the skin brightness mask graph obtained can retain the natural texture features.

[0081] Moreover, the original portrait photo is split into multiple skin area masks for processing, and the resolution of each skin area mask remains unchanged, and the resolution of the entire image is also not affected. After processing by the preset skin color brightness uniformity network, the skin brightness mask graph still maintains the clarity of the original image, avoiding the loss of clarity caused by image scaling. At the same time, all skin area masks can be processed in parallel, which can improve the image processing efficiency.

[0082] For example, the preset skin color brightness uniformity network is a trained Pix2Pix network.

[0083] S104, obtaining a target color mean value of the skin area in the original portrait photo according to the original portrait photo and the skin area mask.

[0084] In addition to using the preset skin color brightness uniformity network for skin color uniformity processing, color uniformity processing is also required.

[0085] S105, generating a target skin color uniformity effect graph corresponding to the original portrait photo according to the original portrait photo, the skin brightness mask graph, and the target color mean value of the skin area.

[0086] Wherein, the skin brightness mask graph realizes brightness uniformity, and the target color mean value realizes color uniformity. The skin color uniformity problem is decoupled into two modules of skin brightness uniformity and color uniformity for separate processing, avoiding the mutual influence of color and brightness.

[0087] The skin color uniformity precision is improved by slicing first and then directly processing through the preset skin color brightness uniformity network. The processing efficiency is improved through parallel processing of multiple slices, the image processing of an ultra-high resolution large image is realized, and the application range of image processing is expanded. Manual retouching is not required, and the image processing cost is reduced.

[0088] To sum up, in the embodiment, an original portrait photo is acquired; skin segmentation is performed on the original portrait photo to obtain a skin region mask; a preset skin color brightness uniformity network is used to process the original portrait photo and the skin region mask to obtain a skin brightness mask graph of the original portrait photo; a target color mean value of the skin region in the original portrait photo is acquired according to the original portrait photo and the skin region mask; and a target skin color uniformity effect graph corresponding to the original portrait photo is generated according to the original portrait photo, the skin brightness mask graph, and the target color mean value of the skin region. Thus, the skin color uniformity problem is decoupled into two modules, skin brightness uniformity and color uniformity, which are processed separately, the influence of color on brightness and the influence of brightness on color are avoided, the processing effect is improved, the processing efficiency is improved, and the image processing cost is reduced.

[0089] On the basis of another embodiment of the application, the application further provides a method for acquiring a target color mean value of a skin region in an original portrait photo according to the original portrait photo and a skin region mask. Figure 2 A flowchart of a method for acquiring a target color mean value of a skin region in an original portrait photo according to the original portrait photo and a skin region mask provided by an embodiment of the application is shown in FIG. 10. Figure 2 As shown in FIG. 10, the acquiring of the target color mean value of the skin region in the original portrait photo according to the original portrait photo and the skin region mask in S104 includes the following steps.

[0090] S201. Skin region brightness statistics are performed on the skin region in the original portrait photo according to the skin region mask to obtain a brightness median value of the skin region.

[0091] The specific statistical method is as follows: assuming that the width and height of the image are W and H, each pixel position (i, j) in the skin region is traversed, the corresponding skin brightness value is L(i, j), the skin region brightness histogram array is Hist

[256] = {0}, and the total number of skin region pixels is sum_skin. The following formulas (2), (3), (4), and (5) are calculated.

[0092] L(i, j) = (R(i, j) + G(i, j) + B(i, j)) / 3 (2)

[0093] Hist[L(i, j)]++ (3)

[0094] The skin region brightness histogram C(j), j ∈ [0, 255] is calculated.

[0095]

[0096] Traverse the cumulative histogram C(j), j∈[0,255], to calculate the median T:

[0097] T = j, C(j) > sum_skin / 2 && C(j-1)≤sum_skin / 2 (5)

[0098] S202, determine each target skin pixel in the skin region of the original portrait photo whose brightness is greater than the brightness median according to the brightness median and the skin region mask.

[0099] Determine the pixel in the skin region whose brightness is greater than the brightness median as the target skin pixel.

[0100] S203, determine the target color mean value of the skin region according to the color value of each target skin pixel in the original portrait photo.

[0101] The color value of the target skin pixel represents the color with high enough brightness, and the target color mean value is obtained by using these color values, which is used for subsequent skin color unification processing, so that the skin color unification effect is better.

[0102] In summary, in the embodiment, the brightness of the skin region in the original portrait photo is counted according to the skin region mask to obtain the brightness median of the skin region; each target skin pixel in the skin region of the original portrait photo whose brightness is greater than the brightness median is determined according to the brightness median and the skin region mask; and the target color mean value of the skin region is determined according to the color value of each target skin pixel in the original portrait photo. Thus, the target color mean value with high brightness is obtained.

[0103] On the basis of another embodiment of the present application, the present application further provides a method for determining the target color mean value of the skin region according to the color value of each target skin pixel in the original portrait photo. Figure 3 A flowchart of a method for determining the target color mean value of the skin region according to the color value of each target skin pixel in the original portrait photo provided by the embodiment of the present application is shown in FIG. 3. Figure 3 As shown in FIG. 3, the determination of the target color mean value of the skin region according to the color value of each target skin pixel in the original portrait photo in S203 includes:

[0104] S301, calculate the first color mean value of the skin region in the first color space according to the color value of each target skin pixel in the original portrait photo.

[0105] The first color mean value of each target skin pixel is obtained by mean calculation according to the color value of each target skin pixel. The first color mean value of the skin region in the first color space is obtained by mean calculation according to the first color mean value of each target skin pixel.

[0106] The first color space is RGB (Red, Green, and Blue) color space.

[0107] The first color mean value is calculated according to the following formula (6):

[0108]

[0109] The pixel value of any point (i, j) in the original portrait photo is P_ij(R, G, B), and the first color mean value is RGB(R_mean, G_mean, B_mean).

[0110] S302, convert the first color mean value in the first color space to the second color space to obtain the color mean value in the second color space as the target color mean value of the skin region.

[0111] The second color space is HSV (Hue, Saturation, and Value) color space.

[0112] The first color mean value is converted to the second color space according to the following formula (7):

[0113] T_HSV(H_mean, S_mean, V_mean) = RGB2HSV(R_mean, G_mean, B_mean) (7)

[0114] The first color mean value is RGB(R_mean, G_mean, B_mean), and the color mean value in the second color space is T_HSV(H_mean, S_mean, V_mean).

[0115] In summary, in this embodiment, the first color mean value of the skin region in the first color space is calculated according to the color value of each target skin pixel in the original portrait photo. The first color mean value in the first color space is converted to the second color space to obtain the color mean value in the second color space as the target color mean value of the skin region. Thus, the conversion from the first color mean value to the second color space is realized.

[0116] On the basis of another embodiment of the present application, the present application further provides a method for generating a target skin color uniform effect picture corresponding to an original portrait picture according to the original portrait picture, a skin brightness mask picture and a target color mean value of a skin area. Figure 4 A flowchart of a method for generating a target skin color uniform effect picture corresponding to an original portrait picture according to the original portrait picture, a skin brightness mask picture and a target color mean value of a skin area is provided in an embodiment of the present application. As shown in FIG. 5, the method comprises the following steps. Figure 4 The step of generating a target skin color uniform effect picture corresponding to an original portrait picture according to the original portrait picture, a skin brightness mask picture and a target color mean value of a skin area in S105 comprises the following steps.

[0117] S401, mixing according to the original portrait picture and the skin brightness mask picture to obtain an initial skin brightness uniform effect picture.

[0118] The specific mixing method is shown in the following formula (8):

[0119]

[0120] Wherein, a is the RGB component value of the original portrait picture, b is the RGB component value, and y is the RGB component value of the initial skin color uniform effect picture. The RGB component value ranges from 0 to 255;

[0121] Mixing the original portrait picture and the skin brightness mask picture to obtain an initial skin brightness uniform effect picture.

[0122] S402, performing skin color processing on the original portrait picture according to the target color mean value and the initial skin brightness uniform effect picture to generate a target skin color uniform effect picture.

[0123] Wherein, the target color mean value realizes color uniformity, and the initial skin brightness uniform effect picture realizes brightness uniformity. The generated target skin color uniform effect picture realizes color uniformity and brightness uniformity.

[0124] In summary, in the embodiment, the original portrait picture and the skin brightness mask picture are mixed to obtain an initial skin brightness uniform effect picture; the original portrait picture is subjected to skin color processing according to the target color mean value and the initial skin brightness uniform effect picture to generate a target skin color uniform effect picture. Thus, the target skin color uniform effect picture realizes color uniformity and brightness uniformity.

[0125] On the basis of another embodiment of the present application, the present application further provides a method for generating a target skin color uniform effect picture according to a target color mean value and an initial skin brightness uniform effect picture. Figure 5A flowchart of a method for generating a target skin color uniform effect picture according to a target color mean value and an initial skin brightness uniform effect picture is provided in an embodiment of the present application. As shown in Figure 5 The target skin color uniform effect picture is generated by performing skin color processing on the original portrait picture according to the target color mean value and the initial skin brightness uniform effect picture in S402, including:

[0126] S501, color adjustment is performed on the initial skin brightness uniform effect picture according to the target color mean value to obtain an adjusted skin color uniform effect picture.

[0127] The target color mean value achieves color uniformity, and the initial skin brightness uniform effect picture achieves brightness uniformity. The adjusted skin color uniform effect picture achieves color uniformity and brightness uniformity.

[0128] S502, the original portrait picture and the adjusted skin color uniform effect picture are mixed using a preset skin color uniform intensity parameter to generate the target skin color uniform effect picture.

[0129] The specific mixing method can be shown in the following formula (9):

[0130] D = D0 * k + S * (1-k) (9)

[0131] Wherein, D is the pixel RGB value of the target skin color uniform effect picture, D0 is the pixel RGB value of the adjusted skin color uniform effect picture, S is the pixel RGB value of the original portrait picture, k is the skin color uniform intensity parameter, and the range of k is [0, 1].

[0132] In summary, in the embodiment, color adjustment is performed on the initial skin brightness uniform effect picture according to the target color mean value to obtain an adjusted skin color uniform effect picture, and the original portrait picture and the adjusted skin color uniform effect picture are mixed using a preset skin color uniform intensity parameter to generate the target skin color uniform effect picture. Thus, the target skin color uniform effect picture achieves color uniformity and brightness uniformity.

[0133] On the basis of another embodiment of the present application, the present application further provides a method for obtaining an adjusted skin color uniform effect picture according to a target color mean value. Figure 6 A flowchart of a method for obtaining an adjusted skin color uniform effect picture according to a target color mean value is provided in an embodiment of the present application. As shown in Figure 6 The target color mean value is a color mean value in a second color space; and color adjustment is performed on the initial skin brightness uniform effect picture according to the target color mean value in S501 to obtain an adjusted skin color uniform effect picture, including:

[0134] S601, traversing the initial skin brightness uniform effect picture according to the skin region mask, to obtain the first color value of each skin pixel in the skin region in the first color space.

[0135] Traversing each skin pixel of the initial skin brightness uniform effect picture, to obtain the first color value of each skin pixel in the first color space, that is, the RGB value P_ij(R, G, B) of each skin pixel.

[0136] S602, converting the first color value of each skin pixel in the first color space to the second color space to obtain the second color value of each skin pixel in the second color space.

[0137] The conversion mode is similar to the above embodiment, which will not be repeated here.

[0138] S603, reorganizing the hue average value in the target color mean value and the saturation value and the lightness value in the second color value of each skin pixel to generate the target color value of each skin pixel in the second color space.

[0139] Reorganize the hue average value in the target color mean value and the saturation value and the lightness value in the second color value of each skin pixel to form a new set of target color values in the second color space. That is, to obtain each skin pixel with updated hue value. To realize the hue value uniformity of each skin pixel.

[0140] S604, converting the target color value of each skin pixel in the second color space to the first color space to obtain the target color value of each skin pixel in the first color space.

[0141] The conversion mode is similar to the above embodiment, which will not be repeated here.

[0142] S605, generating an adjusted skin color uniform effect picture according to the target color value of each skin pixel in the first color space.

[0143] Assigning the target color value of each skin pixel in the first color space to the picture to obtain the skin color uniform effect picture.

[0144] Thus, both the brightness uniformity is realized through the initial skin brightness uniform effect picture, and the color uniformity is realized through the hue average value in the target color mean value.

[0145] In summary, in the embodiment, the target color mean value is a color mean value in a second color space; according to the skin region mask, the initial skin brightness uniform effect picture is traversed to obtain first color values of each skin pixel in the skin region in a first color space; the first color values of each skin pixel in the first color space are converted to the second color space to obtain second color values of each skin pixel in the second color space; the hue average value in the target color mean value and the saturation value and the lightness value in the second color value of each skin pixel are reorganized to generate a target color value of each skin pixel in the second color space; the target color value of each skin pixel in the second color space is converted to the first color space to obtain a target color value of each skin pixel in the first color space; and the adjusted skin color uniform effect picture is generated according to the target color value of each skin pixel in the first color space. Thus, brightness uniformity is realized through the initial skin brightness uniform effect picture, and color uniformity is realized through the hue average value in the target color mean value.

[0146] On the basis of the above-mentioned embodiments, the application further provides a training method of a skin color brightness uniform network. Figure 7 A flowchart of a training method of a skin color brightness uniform network provided by the embodiments of the application is shown in FIG. 6. Figure 7 As shown in FIG. 6, the preset flaw removal network is trained in the following manner:

[0147] S701, generating a sample data set according to a plurality of sample portrait photos.

[0148] The sample data set comprises a plurality of data pairs, each data pair comprising a sample portrait photo and a plurality of sample skin region masks and a plurality of sample skin brightness uniform effect pictures corresponding to the sample portrait photo, and there are at least different portrait skin regions in different sample portrait photos.

[0149] The sample portrait photo is an original portrait, the sample skin region mask is a mask in the original portrait, and the sample skin brightness uniform effect picture is a picture after brightness uniformity.

[0150] S702, training an initial skin color brightness uniform network using the sample data set until a preset stop iteration condition is met to obtain a preset skin color brightness uniform network.

[0151] The sample portrait photo and the plurality of sample skin region masks corresponding to the sample portrait photo are input into the initial skin color brightness uniform network to obtain an output skin brightness uniform effect picture, and the closer the output skin brightness uniform effect picture is to the sample skin brightness uniform effect picture in the same data pair, the more likely the preset skin color brightness uniform network meeting the preset stop iteration condition is obtained.

[0152] For example, the initial skin color brightness uniform network is a Pix2Pix network. Figure 8This is a schematic diagram of the structure of a Pix2Pix network provided in an embodiment of the present application. Figure 8 As shown in the figure, the input sample portrait photo and the corresponding multiple sample skin area masks are concentrated, then channel amplified through convolution + Relu (Rectified Linear Unit), followed by down-convolution + Relu and down-sampling Maxpooling. In the down-convolution + down-sampling, multiple convolutions are repeated and down-sampled to the bottom of the Unet (U-shaped network). Then, multiple upsampling and convolution operations are performed in the corresponding up-convolution + up-sampling operations. At the same time, each layer is horizontally fused with the up-sampled results of the next layer (for multimodal data fusion). Finally, after multiple convolutions, the output effect image is output.

[0153] The generator loss uses VGG loss and L2 loss. The preset stopping condition is that the generator loss value is less than a first preset threshold and the discriminator loss value is less than a second preset threshold.

[0154] Therefore, the deep learning GAN method is used to construct a fixed-size skin color brightness unification network Pix2Pix to achieve skin color brightness unification, thereby improving the efficiency of skin color unification.

[0155] In summary, in this embodiment, a sample dataset is generated based on multiple sample portrait photos. The sample dataset includes multiple data pairs, each of which includes: a sample portrait photo, multiple corresponding sample skin region masks, and multiple sample skin brightness unification effect maps; different sample portrait photos contain at least different portrait skin regions; and an initial skin color brightness unification network is trained using the sample dataset until a preset stop iteration condition is met, thereby obtaining a preset skin color brightness unification network. Thus, a fixed-size skin color brightness unification network, Pix2Pix, is constructed using a deep learning GAN approach to achieve skin color brightness unification, improving skin color unification efficiency.

[0156] Based on another embodiment of the present application, the present application also provides a method for generating a sample data set based on sample portrait photos. Figure 9 The present application provides a flow chart of a method for generating a sample data set based on sample portrait photos. Figure 9 As shown, generating a sample data set based on the sample portrait photos in S701 includes:

[0157] S801: Perform skin segmentation on the sample portrait photo to obtain a corresponding sample skin area mask.

[0158] The implementation method of skin segmentation on the sample portrait photo is similar to the implementation method of skin segmentation on the original portrait photo in the above embodiment, and the preset sizes of the two slices are the same, which will not be described here.

[0159] S802, skin brightness adjustment is performed on the sample portrait photo to obtain a sample skin brightness uniform effect picture corresponding to the sample portrait photo.

[0160] The sample portrait photo is subjected to skin brightness adjustment by using a preset retouching tool, and manual screening and correction (non-skin areas are screened out) to make the skin brightness in the portrait photo uniform. For example, the preset retouching tool can be Photoshop software, and the skin brightness adjustment here can be manual operation of the preset retouching tool or automatic processing of the preset retouching tool.

[0161] S803, a data pair is constructed according to the sample portrait photo, the corresponding sample skin region mask, and the corresponding sample skin brightness uniform effect picture.

[0162] In summary, in the embodiment, the sample portrait photo is subjected to skin segmentation to obtain a corresponding sample skin region mask, the sample portrait photo is subjected to skin brightness adjustment to obtain a sample skin brightness uniform effect picture corresponding to the sample portrait photo, and a data pair is constructed according to the sample portrait photo, the corresponding sample skin region mask, and the corresponding sample skin brightness uniform effect picture. Thus, a plurality of training data pairs are accurately constructed.

[0163] The following describes the portrait photo skin color processing device, server, and storage medium provided by the present application, and the specific implementation process and technical effects are described above. The following will not be described again.

[0164] Figure 10 A schematic diagram of a portrait photo skin color processing device provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the device includes: Figure 10

[0165] A first acquisition module 1001 is configured to acquire an original portrait photo.

[0166] A segmentation module 1002 is configured to perform skin segmentation on the original portrait photo to obtain a skin region mask.

[0167] A processing module 1003 is configured to perform processing on the original portrait photo and the skin region mask by using a preset skin color brightness uniform network to obtain a skin brightness mask picture of the original portrait photo.

[0168] A second acquisition module 1004 is configured to acquire a target color mean value of a skin region in the original portrait photo according to the original portrait photo and the skin region mask.

[0169] ​The generating module 1005 is configured to generate a target skin color uniform effect picture corresponding to the original portrait picture according to the original portrait picture, the skin brightness mask picture, and the target color mean value of the skin region.

[0170] Further, the second obtaining module 1004 is specifically configured to statistically obtain a brightness median value of the skin region in the original portrait picture according to the skin region mask; determine each target skin pixel in the skin region with a brightness greater than the brightness median value from the original portrait picture according to the brightness median value and the skin region mask; and determine the target color mean value of the skin region according to color values of each target skin pixel in the original portrait picture.

[0171] Further, the second obtaining module 1004 is specifically configured to calculate a first color mean value of the skin region in a first color space according to color values of each target skin pixel in the original portrait picture; and convert the first color mean value in the first color space to a second color space to obtain a color mean value in the second color space as the target color mean value of the skin region.

[0172] Further, the generating module 1005 is specifically configured to mix the original portrait picture and the skin brightness mask picture to obtain an initial skin brightness uniform effect picture; and perform skin color processing on the original portrait picture according to the target color mean value and the initial skin brightness uniform effect picture to generate the target skin color uniform effect picture.

[0173] Further, the generating module 1005 is specifically configured to perform color adjustment on the initial skin brightness uniform effect picture according to the target color mean value to obtain an adjusted skin color uniform effect picture; and mix the original portrait picture and the adjusted skin color uniform effect picture by using a preset skin color uniform intensity parameter to generate the target skin color uniform effect picture.

[0174] Further, the generating module 1005 is specifically configured to take the target color mean value as a color mean value in a second color space; traverse the initial skin brightness uniform effect picture according to the skin region mask to obtain first color values of each skin pixel in the skin region in a first color space; convert the first color values of each skin pixel in the first color space to the second color space to obtain second color values of each skin pixel in the second color space; recombine a hue average value in the target color mean value and a saturation value and a brightness value in the second color value of each skin pixel to generate a target color value of each skin pixel in the second color space; convert the target color value of each skin pixel in the second color space to the first color space to obtain a target color value of each skin pixel in the first color space; and generate the adjusted skin color uniform effect picture according to the target color value of each skin pixel in the first color space.

[0175] Further, the generating module 1005 is further used for generating a sample data set according to the plurality of sample portrait photos, the sample data set comprising: a plurality of data pairs, each data pair comprising: a sample portrait photo and a plurality of sample skin region masks corresponding to the sample portrait photo, and a plurality of sample skin brightness uniform effect pictures; there are at least different portrait skin regions in different sample portrait photos; and the initial skin color brightness uniform network is trained by using the sample data set until a preset stop iteration condition is met, to obtain a preset skin color brightness uniform network.

[0176] Further, the generating module 1005 is further used for performing skin segmentation on the sample portrait photo to obtain a corresponding sample skin region mask, performing skin brightness adjustment on the sample portrait photo to obtain a sample skin brightness uniform effect picture corresponding to the sample portrait photo, and constructing a data pair according to the sample portrait photo, the corresponding sample skin region mask, and the corresponding sample skin brightness uniform effect picture.

[0177] Figure 11 A schematic diagram of a server is provided in the embodiments of the present application. The server can be a device with computing processing function.

[0178] The server comprises a processor 1101 and a storage medium 1102. The processor 1101 and the storage medium 1102 are connected through a bus.

[0179] The storage medium 1102 is used for storing a program, and the processor 1101 invokes the program stored in the storage medium 1102 to execute the above-mentioned method embodiments. The specific implementation manners and technical effects are similar, and will not be described here.

[0180] Optionally, the present application also provides a storage medium comprising a program, which is used for executing the above-mentioned method embodiments when executed by a processor. In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. For example, the above-mentioned device embodiments are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0181] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiments of the present application.

[0182] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function unit.

[0183] The integrated unit realized in the form of software function unit can be stored in a storage medium. The software function unit stored in the storage medium includes a plurality of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

Claims

1. A method of processing skin color of a portrait photo, characterized by, The method comprises: obtaining an original portrait photo; performing skin segmentation on the original portrait photo to obtain a skin region mask; processing the original portrait photo and the skin region mask by using a preset skin color brightness uniformity network to obtain a skin brightness mask of the original portrait photo; obtaining a target color mean value of a skin region in the original portrait photo according to the original portrait photo and the skin region mask; generating a target skin color uniformity effect picture corresponding to the original portrait photo according to the original portrait photo, the skin brightness mask and the target color mean value of the skin region.

2. The method of claim 1, wherein, The method comprises: performing brightness statistics on the skin region in the original portrait photo according to the skin region mask to obtain a brightness median value of the skin region; determining each target skin pixel in the skin region with a brightness greater than the brightness median value from the original portrait photo according to the brightness median value and the skin region mask; determining the target color mean value of the skin region according to color values of the each target skin pixel in the original portrait photo.

3. The method of claim 2, wherein, The method comprises: calculating a first color mean value of the skin region in a first color space according to the color values of the each target skin pixel in the original portrait photo; converting the first color mean value in the first color space to a second color space to obtain a color mean value in the second color space as the target color mean value of the skin region.

4. The method of claim 1, wherein, The method comprises: mixing the original portrait photo and the skin brightness mask to obtain an initial skin brightness uniformity effect picture; performing skin color processing on the original portrait photo according to the target color mean value and the initial skin brightness uniformity effect picture to generate the target skin color uniformity effect picture.

5. The method of claim 4, wherein, The method comprises: adjusting the color of the initial skin brightness uniformity effect picture according to the target color mean value to obtain an adjusted skin color uniformity effect picture; mixing the original portrait photo and the adjusted skin color uniformity effect picture by using a preset skin color uniformity intensity parameter to generate the target skin color uniformity effect picture.

6. The method of claim 5, wherein, The target color mean value is a color mean value in a second color space; and the method of adjusting the color of the initial skin brightness uniformity effect picture according to the target color mean value to obtain an adjusted skin color uniformity effect picture comprises: According to the skin region mask, the initial skin brightness uniform effect picture is traversed to obtain first color values of each skin pixel in the skin region in a first color space; The first color values of the each skin pixel in the first color space are converted to the second color space to obtain second color values of the each skin pixel in the second color space; The hue average value in the target color mean value is recombined with the saturation value and the lightness value in the second color values of the each skin pixel to generate target color values of the each skin pixel in the second color space; The target color values of the each skin pixel in the second color space are converted to the first color space to obtain target color values of the each skin pixel in the first color space; According to the target color values of the each skin pixel in the first color space, the adjusted skin color uniform effect picture is generated.

7. The method of claim 1, wherein, The preset skin color brightness uniform network is trained in the following manner: A sample data set is generated according to a plurality of sample portrait photos, the sample data set includes: a plurality of data pairs, each data pair includes: one of the sample portrait photos and a corresponding plurality of sample skin region masks, a plurality of sample skin brightness uniform effect pictures; there are at least different portrait skin regions in different sample portrait photos; The initial skin color brightness uniform network is trained using the sample data set until a preset stop iteration condition is met to obtain the preset skin color brightness uniform network.

8. The method of claim 7, wherein, The sample data set is generated according to a plurality of sample portrait photos, including: Skin segmentation is performed on the sample portrait photos to obtain corresponding sample skin region masks; Skin brightness adjustment is performed on the sample portrait photos to obtain sample skin color uniform effect pictures corresponding to the sample portrait photos; A data pair is constructed according to the sample portrait photos, the corresponding sample skin region masks, and the corresponding sample skin brightness uniform effect pictures.

9. A portrait photo skin color processing apparatus characterized by comprising: The device includes: An acquisition module is configured to acquire an original portrait photo; A segmentation module is configured to perform skin segmentation on the original portrait photo to obtain a skin region mask; A processing module is configured to process the original portrait photo and the skin region mask using a preset skin color brightness uniform network to obtain a skin brightness mask of the original portrait photo; An acquisition module is configured to acquire a target color mean value of a skin region in the original portrait photo according to the original portrait photo and the skin region mask; A generation module is configured to generate a target skin color uniform effect picture corresponding to the original portrait photo according to the original portrait photo, the skin brightness mask, and the target color mean value of the skin region.

10. A server, characterized by It includes: A processor and a storage medium, the processor and the storage medium are communicatively connected through a bus, the storage medium stores program instructions executable by the processor, and the processor calls the program stored in the storage medium to execute the steps of the portrait photo skin color processing method according to any one of claims 1 to 8.

Images