Portrait skin color adjustment method, device, electronic device and storage medium

Abstract

This application relates to a method, device, electronic device, and storage medium for adjusting skin color in portraits. The method includes: obtaining an initial portrait image to be adjusted; determining multiple skin color category probabilities that the initial portrait image belongs to multiple predetermined skin color categories; calculating the mapped pixel value of each pixel in the initial portrait image based on the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories; and determining a skin color beautification image based on the mapped pixel value of each pixel. This method enables more flexible and accurate adaptive skin color adjustment for a variety of skin color conditions.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for adjusting skin color of a portrait. Background Art

[0002] Image beautification technology can enhance portraits in images, making them more aesthetically pleasing to the human eye. This process often requires adjusting the skin tone. Conventional beautification typically only whitens and brightens skin tones. However, due to racial and individual differences, skin tones can vary significantly. Using a uniform whitening process may not achieve the desired beautification effect for some skin tones.

[0003] One solution is to use different skin tone adjustment strategies based on the region in which the beauty product is used. For example, in Asia, a brighter, more rosy adjustment might be used, while in Africa, a darker, brighter adjustment might be used to suit the skin tones of darker people. However, this approach doesn't always accurately match the user's skin tone, and remains inflexible and inaccurate.

[0004] Therefore, there is still room for improvement in the existing skin color adjustment methods. Summary of the Invention

[0005] Based on this, it is necessary to provide a portrait skin color adjustment method, device, electronic device and storage medium that can flexibly and accurately adjust the skin color in a portrait in order to address the above technical problems.

[0006] A method for adjusting skin color of a portrait, the method comprising:

[0007] Obtaining an initial portrait image to be adjusted;

[0008] determining a plurality of skin color category probabilities that the initial portrait image belongs to a plurality of predetermined skin color categories;

[0009] Calculating a mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0010] A skin color beautification image is determined based on the mapped pixel value of each pixel.

[0011] In one embodiment, the calculation of the pixel value after mapping of each pixel in the initial portrait image based on the multiple skin color category probabilities and the multiple skin color mapping relationship tables corresponding to the multiple skin color categories includes: obtaining the multiple skin color mapping relationship tables corresponding to the multiple skin color categories; calculating the original pixel value of each pixel in the initial portrait image and obtaining the multiple skin color mapping values ​​mapped according to the multiple skin color mapping relationship tables; and performing weighted summation of the multiple skin color mapping values ​​corresponding to each pixel using the multiple skin color category probabilities as weights to determine the pixel value after mapping of each pixel.

[0012] In one embodiment, determining the skin color beautification image based on the pixel value after mapping each pixel includes: determining the skin probability of each pixel in the initial portrait image belonging to skin; based on the skin probability of each pixel and the pixel value after mapping each pixel, determining the pixel value of each pixel after skin color beautification, thereby determining the skin color beautification image.

[0013] In one embodiment, each of the plurality of skin color mapping relationship tables records a mapping relationship table from pixel values ​​before skin color mapping to pixel values ​​after skin color mapping under a corresponding skin color category.

[0014] In one embodiment, the pixel values ​​in each of the multiple skin color mapping relationship tables are represented in the RGB color space; in each of the skin color mapping relationship tables in the RGB color space, the R values, G values, and B values ​​of the multiple pixel values ​​before skin color mapping are recorded, as well as the R values, G values, and B values ​​of the pixel values ​​before each skin color mapping and after skin color mapping is performed.

[0015] In one embodiment, the R value, G value, and B value of the pixel value before skin color mapping in the skin color relationship mapping table are respectively compressed by a predetermined multiple.

[0016] In one embodiment, the pixel values ​​in each of the multiple skin color mapping relationship tables are represented in the YUV color space; each of the skin color mapping relationship tables in the YUV color space records multiple Y values ​​before skin color mapping, and the Y value of each Y value before skin color mapping after skin color mapping is performed, and records multiple Cb values ​​and Cr values ​​before skin color mapping, and the Cb value and Cr value of each Cb value and Cr value before skin color mapping after skin color mapping is performed.

[0017] In one embodiment, the Cb value and the Cr value of the pixel value before skin color mapping in the skin color relationship mapping table are respectively limited to a predetermined Cb value range and a predetermined Cr value range.

[0018] A portrait skin color adjustment device, comprising:

[0019] An image acquisition module, used to acquire an initial portrait image to be adjusted;

[0020] a skin color category probability determination module, configured to determine multiple skin color category probabilities that the initial portrait image belongs to multiple predetermined skin color categories;

[0021] a mapped pixel value determination module, configured to calculate a mapped pixel value of each pixel in the initial portrait image based on the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0022] The skin color beautification image determination module is used to determine the skin color beautification image based on the pixel value after the mapping of each pixel.

[0023] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Obtaining an initial portrait image to be adjusted;

[0025] determining a plurality of skin color category probabilities that the initial portrait image belongs to a plurality of predetermined skin color categories;

[0026] Calculating a mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0027] A skin color beautification image is determined based on the mapped pixel value of each pixel.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0029] Obtaining an initial portrait image to be adjusted;

[0030] determining a plurality of skin color category probabilities that the initial portrait image belongs to a plurality of predetermined skin color categories;

[0031] Calculating a mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0032] A skin color beautification image is determined based on the mapped pixel value of each pixel.

[0033] The above-mentioned portrait skin color adjustment method, device, electronic device and storage medium comprehensively utilize multiple skin color mapping relationship tables corresponding to multiple predetermined skin color categories based on the multiple skin color category probabilities of the detected portrait image, rather than just utilizing a single skin color mapping relationship table. This makes it possible to more smoothly and comprehensively adjust the portrait skin color based on the detected skin color condition of the portrait, so that the skin color adjustment can flexibly adapt to various skin color conditions and more accurately and adaptively adjust to various skin color conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A diagram showing an application environment of a portrait skin color adjustment method according to an embodiment;

[0035] Figure 2 1 is a flow chart of a method for adjusting skin color of a portrait in one embodiment;

[0036] Figure 3 1 is a flow chart of a method for adjusting skin color of a portrait in one embodiment;

[0037] Figure 4 1 is a flow chart of a method for adjusting skin color of a portrait in one embodiment;

[0038] Figure 5 Schematic diagram of a three-dimensional skin color mapping relationship table in an RGB color space in one embodiment;

[0039] Figure 6 is a structural block diagram of a portrait skin color adjustment device in one embodiment;

[0040] Figure 7 FIG. 1 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] There are two types of skin color adjustment technologies. The first type does not distinguish the skin color of the portraits in the image, and directly makes a uniform color change to the entire image or the skin color area in the image. The change is usually achieved by curve adjustment, that is, adjusting the brightness, chroma, and saturation curves in the image to achieve brightening, redness, and whitening effects; the second type can simply distinguish between people of different skin colors, that is, first determine whether the portrait in the current processed image belongs to the white race, the yellow race, or the dark race, and then select different skin color adjustment curves to achieve different skin color changes for different races.

[0043] The first type of algorithm uses a uniform skin tone curve for different skin tones, failing to achieve differentiated beautification effects and, therefore, failing to meet the beauty needs of different people. The second type of algorithm can, to a certain extent, meet the personalized processing requirements for different skin tones, but lacks flexibility. In real-world scenarios, due to individual skin color differences and changes in lighting conditions, some images contain skin tones that fall between different skin tones. Forcing a single curve into the image still fails to achieve the desired effect.

[0044] In response to the defects in the above-mentioned skin color adjustment technology, the present application provides a portrait skin color adjustment method, which can flexibly and accurately provide adaptive skin color adjustment for different skin color differences.

[0045] The portrait skin color adjustment method provided in this application can be applied to Figure 1 In the electronic device 100 shown. The electronic device 100 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The electronic device 100 may execute the portrait skin color adjustment method of any embodiment of the present application to adjust the skin color of the initial portrait image to be adjusted, and output a skin color beautification image after the skin color adjustment. The electronic device 100 may include a camera, and the electronic device 100 may obtain the initial portrait image to be adjusted by shooting in real time with the camera. The electronic device 100 may also include a display screen, and the electronic device 100 may execute the portrait skin color adjustment method of the embodiment of the present application on the initial portrait image to be adjusted in real time to obtain a skin color beautification image, and display the obtained skin color beautification image on the display screen for the user to review.

[0046] In one embodiment, Figure 2 As shown, a portrait skin color adjustment method is provided, which is applied to Figure 1 Taking the electronic device 100 in FIG. 1 as an example, the method includes the following steps S210-S240:

[0047] Step S210: obtaining an initial portrait image to be adjusted.

[0048] The initial portrait image is an image that needs skin color adjustment. The initial portrait image can be, for example, an image captured by the electronic device 100 or an external device, and the initial portrait image can contain a portrait. Each pixel in the initial portrait image can have a corresponding original pixel value Value color .

[0049] Step S220 , determining multiple skin color category probabilities that the initial portrait image belongs to multiple predetermined skin color categories.

[0050] For example, four typical skin color categories can be predetermined, namely white skin color, yellow skin color, brown skin color, and black skin color. It will be understood that the above skin color categories are only examples, and in other examples, more, fewer, or different skin color categories can also be set.

[0051] The skin color category probability that the skin color of the portrait in the initial portrait image belongs to each of the predetermined multiple skin color categories can be calculated by a neural network algorithm. For example, in this step, the initial portrait image can be input into a pre-trained skin color classification neural network model, and the skin color classification neural network model will output the skin color category probability that the skin color of the portrait in the initial portrait image belongs to each skin color category. The skin color classification neural network model can be trained using a training set of multiple groups of portrait image-skin color category pairs including various skin color categories. For example, in the example with the above-mentioned four typical skin color categories, after the initial portrait image is input into the pre-trained skin color classification neural network model, the skin color classification neural network model can output the skin color category probability p of the initial portrait image belonging to white skin color, p respectively. White , the probability p of skin color belonging to yellow skin color Yellow , the probability of skin color belonging to brown skin color p Brown , and the probability p of belonging to the black skin color category Black In some embodiments, the sum of the skin color category probabilities of the various skin colors can be made 1, so as to facilitate the subsequent use of these skin color category probabilities as weights to perform weighted calculations on the mapped pixel values. For example, in the example with the four typical skin color categories mentioned above, there can be p White +p Yellow +p Brown +p Black =1.

[0052] In some alternative embodiments, a skin mask region may be first detected and extracted from the initial portrait image using a neural network algorithm. Alternatively, instead of inputting the initial portrait image into a pre-trained skin color classification neural network model, only the skin mask region image may be input into the pre-trained skin color classification neural network model. The skin color classification neural network model may be trained using a training set of multiple skin mask region image-skin color category pairs including various skin color categories. This allows for more accurate determination of the skin color category probability that the skin color of the person in the initial portrait image belongs to each of a plurality of predetermined skin color categories.

[0053] Those skilled in the art will appreciate that the skin color classification neural network model can be implemented using any suitable neural network model with classification functionality, which is not detailed herein.

[0054] Step S230 , calculating the mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and the multiple skin color mapping relationship tables corresponding to the multiple skin color categories.

[0055] In this step, the multiple skin color category probabilities can be used as weights and multiple skin color mapping relationship tables corresponding to the multiple skin color categories can be comprehensively utilized to calculate the mapped pixel value of each pixel in the initial portrait image.

[0056] In one embodiment, step S230 includes: obtaining multiple skin color mapping relationship tables corresponding to multiple skin color categories; calculating the original pixel value of each pixel in the initial portrait image and obtaining multiple skin color mapping values ​​mapped according to the multiple skin color mapping relationship tables; and performing weighted summation of the multiple skin color mapping values ​​corresponding to each pixel using the multiple skin color category probabilities as weights to determine the pixel value of each pixel after mapping.

[0057] Before executing step S230, M corresponding nonlinear skin color mapping relationship tables can be established and stored for M skin color categories in advance, so as to prepare for searching and using these M skin color mapping relationship tables in this step S230, where M is a positive integer. Each of the M skin color mapping relationship tables records a mapping relationship table from the pixel value before skin color mapping to the pixel value after skin color mapping under a corresponding skin color category, that is, it records the different pixel values ​​before skin color mapping, and how many pixel values ​​should be mapped accordingly after the mapping process with skin color beautification effect. For example, in the case where the above four typical skin color categories are set, a skin color mapping relationship table LUT corresponding to white skin color can be established respectively. White , skin color mapping table LUT corresponding to yellow skin color Yellow , skin color mapping table LUT corresponding to brown skin color Brown , and the skin color mapping table LUT corresponding to black skin color Black For each skin color category of a portrait image, each pixel in the skin color area can achieve a skin color beautification effect after being mapped using the skin color mapping table. The specific settings of the pixel values ​​before and after the mapping can be debugged by professional artists.

[0058] Thus, for example, in the case of setting the above four typical skin color categories, in this step, the original pixel value Value of each pixel in the image can be respectively calculated. color , according to the skin color mapping table LUT White , LUT Yellow , LUT Brown and LUTs Black , mapping to obtain the corresponding four skin color mapping values ​​NewValue White 、NewValue Yellow、NewValue Brown , and NewValue Black :

[0059] NewValue White =LUT White (Value color ) Formula 1

[0060] NewValue Yellow =LUT Yellow (Value color ) Formula 2

[0061] NewValue Brown =LUT Brown (Value color ) Formula 3

[0062] NewValue Black =LUT Black (Value color ) Formula 4

[0063] Then, the skin color category probability p White 、p Yellow 、p Brown , and p Black is the weight, for each pixel Value color The corresponding four skin color mapping values ​​NewValue White 、NewValue Yellow 、NewValue Brown , and NewValue Black Perform weighted summation to determine the value of each pixel color The mapped fused pixel value NewValue blend for:

[0064] NewValue blend

[0065] =p White *NewValue White +p Yellow *NewValue Yellow +p Brown *NewValue Brown +p Black *NewValue Black Formula 5

[0066] Step S240 : determining a skin color beautified image based on the mapped pixel value of each pixel.

[0067] In one embodiment, the pixel value of each pixel after mapping can be directly used as the pixel value of each pixel in the skin color beautification image, thereby directly determining the skin color beautification image for output.

[0068] In another embodiment, the pixel values ​​of each pixel in the skin color beautification image may be adjusted differently depending on whether each pixel belongs to skin. Figure 3 and Figure 4 As shown, in this embodiment, step S240 may include steps S241-S242:

[0069] Step S241 : determining the skin probability of each pixel in the initial portrait image belonging to skin.

[0070] In this step, the skin mask area can be detected from the initial portrait image by a neural network algorithm to determine the skin probability of each pixel in the initial portrait image belonging to the skin accordingly. For example, in this step, the initial portrait image can be input into a pre-trained skin detection neural network model, and the skin detection neural network model will output the skin probability of each pixel in the initial portrait image belonging to the skin. For example, after the initial portrait image is input into a pre-trained skin color classification neural network model, the skin color classification neural network model can output the skin probability of each pixel at each pixel position (x, y) in the initial portrait image belonging to the skin.

[0071] Those skilled in the art will appreciate that the skin detection neural network model can be implemented using any suitable neural network model. For example, the U-NET network structure disclosed in the technical document Ronneberger O, Fischer P, Brox TU-Net: Convolutional Networks for Biomedical Image Segmentation [J]. Springer, Cham, 2015 can be used to detect skin areas from an input initial portrait image and output a probability distribution of each pixel in the initial portrait image belonging to skin.

[0072] Step S242 : Based on the skin probability of each pixel and the pixel value after mapping of each pixel, the pixel value of each pixel after skin color beautification is determined, thereby determining a skin color beautification image.

[0073] In this step, the skin probability of each pixel can be used as a weight to perform a weighted summation on the original pixel value and the mapped pixel value of each pixel, so that when the skin probability of the pixel is greater, the pixel value of the pixel after skin beautification is closer to the mapped pixel value, and when the skin probability of the pixel is smaller, the pixel value of the pixel after skin beautification is closer to the original pixel value. When the skin probability of the pixel is maximum (for example, 1), the pixel value of the pixel after skin beautification is equal to the mapped pixel value, and when the skin probability of the pixel is minimum (for example, 0), the pixel value of the pixel after skin beautification is equal to the original pixel value.

[0074] For example, in the case of setting the above four typical skin color categories, in this step, the skin probability of each pixel belonging to skin can be combined Based on the original pixel value of each pixel color And the mapped pixel value NewValue blend , get the pixel value NewValue after skin color beautification of each pixel Beauty :

[0075]

[0076] Among them, the skin probability Can be From the above formula, it can be seen that when the skin probability is within the range of When the pixel value after skin color beautification is NewValue Beauty Equal to the mapped pixel value NewValue blend , when the skin probability is When the pixel value after skin color beautification is NewValue Beauty Equal to the original pixel value Value color This allows for skin beautification to be performed on only the skin area in the image without affecting the display effect of the image outside the skin area.

[0077] In the above-mentioned embodiments of the present application, before step S230, it is necessary to establish M corresponding skin color mapping relationship tables for M skin color categories in advance. Each skin color mapping relationship table records the mapping relationship table from the pixel value before skin color mapping to the pixel value after skin color mapping under the corresponding skin color category. In each skin color mapping table, N pixel values ​​before skin color mapping can be recorded, as well as the pixel value of each pixel value before skin color mapping after skin color mapping is performed. Therefore, for any pixel to be mapped, the pixel value before mapping corresponding to the pixel value of the pixel to be mapped can be found from the skin color mapping table, and then the pixel value of the pixel value before mapping after skin color mapping is performed can be found from the table as the pixel value after mapping of the pixel to be mapped. The pixel values ​​in the skin color mapping table can be expressed in various color spaces. For example, the pixel values ​​before and after mapping can be expressed in color spaces such as RGB, CMY / CMYK, HSI, HSV, HSB, Lab, and YUV.

[0078] In one embodiment, the pixel values ​​in each skin color relationship mapping table of the plurality of skin color mapping relationship tables are represented in an RGB color space.

[0079] In the RGB color space, each pixel value can include the values ​​of three channels: red (R), green (G), and blue (B), and each of the R, G, and B values ​​has an original value range of 0 to 255. In the skin color mapping relationship table under the RGB color space, the R, G, and B values ​​of N1 pixel values ​​before skin color mapping are recorded, as well as the R, G, and B values ​​of each pixel value before skin color mapping and the R, G, and B values ​​of the pixel value after skin color mapping are performed. N1 is a positive integer. For any pixel to be mapped, the original pixel value Value of the pixel can be used. color The R value, G value and B value of the pixel value before mapping are used as the R value, G value and B value of the pixel value before mapping, and the R value, G value and B value of the pixel value after mapping corresponding to the R value, G value and B value of the pixel value before mapping are found in the skin color mapping relationship table.

[0080] Furthermore, in one embodiment, the R value, G value, and B value of the pixel value before skin color mapping in each skin color relationship mapping table may be compressed by a predetermined multiple, respectively.

[0081] The aforementioned compression of the R, G, and B values ​​by predetermined multiples refers to compressing the original value range of the R, G, and B values ​​from 0 to 255 by a predetermined multiple X to a compressed value range of 0 to [255 / X], where X>1 and [] represents rounding down. Within the original value range of 0 to 255, each skin color mapping table can have 256 different values ​​for the R, G, and B values. Thus, the entire skin color mapping table needs to record 256*256*256=16777216 pixel values ​​before mapping and their corresponding 16777216 pixel values ​​after mapping. This will require more memory and increase computational costs. After geometrically compressing the predetermined multiple X to the compression value range of 0 to [255 / X], the R value, G value and B value in each skin color mapping relationship table can have ([255 / X]+1) different values, so the entire skin color mapping relationship table only needs to record ([255 / X]+1)*([255 / X]+1)*([255 / X]+1) pixel values ​​before mapping and their corresponding ([255 / X]+1)*([255 / X]+1)*([255 / X]+1) pixel values ​​after mapping, which can reduce the length of the skin color mapping relationship table and improve computing efficiency. After the above compression, for any pixel to be mapped, the original pixel value Value of the pixel to be mapped can be first recorded. color Compress to the same predetermined multiple X to the compressed pixel value [Value color / X], and then use the compressed pixel value [Value color / X] as the pixel value before mapping [Value color / X] and find the pixel value before mapping from the skin color mapping table [Value color / X] The mapped pixel value corresponding to the pixel value. It can be understood that the mapped pixel value does not need to be compressed, but its R value, G value and B value can be kept in the value range of 0 to 255, so that the mapped pixel value found can be directly used as a regular RGB pixel value.

[0082] For example, after the R value, G value and B value three channels of the pixel value before skin color mapping in the RGB color space are compressed by 4 times, the 0-255 value range of the R value, G value and B value of the pixel value before skin color mapping can be converted to the 0-63 value range, and the following can be obtained: Figure 5 The three-dimensional skin color mapping relationship table LUT[B][G][R] shown in FIG, wherein, Figure 5Figure 2 shows a cross-sectional diagram of the B value at a given RG value in the RGB 3D space of the pixel values ​​before skin color mapping in the 3D skin color mapping table, where b1 to b64 represent 64 B values ​​within the range of 0 to 63. The length of the skin color mapping table is 64*64*64=262144, meaning it contains 262144 R, G, and B values ​​before mapping and their corresponding 262144 R, G, and B values ​​after mapping.

[0083] In the above embodiment of the RGB color space, although the input values ​​of the three RGB channels are appropriately compressed to save storage and computing costs, the accuracy of the color mapping transformation is reduced, and the length of the lookup table is still long.

[0084] In another embodiment, the pixel values ​​in each skin color relationship mapping table in the plurality of skin color mapping relationship tables are represented in a YUV color space.

[0085] In the YUV color space (specifically also known as the YCbCr color space), each pixel value can include three channel values: brightness (Y), blue component (Cb) and red component (Cr), and each of the Y, Cb and Cr values ​​has an original value range of 0 to 255, where the Y value determines the brightness of the pixel, and the Cb and Cr values ​​jointly determine the chromaticity of the pixel. In the YUV color space, the Y value can be mapped and transformed separately, while the Cb and Cr values ​​can be mapped and transformed together. That is, in the skin color mapping relationship table under the YUV color space, the Y values ​​before N2 skin color mapping and the Y value of each skin color mapping before skin color mapping are recorded, and the Cb and Cr values ​​before N3 skin color mapping are recorded, as well as the Cb and Cr values ​​of each skin color mapping before skin color mapping and after skin color mapping are performed. N2 and N3 are positive integers. For any pixel to be mapped, the original pixel value Value of the pixel can be used. color The Y value of the pixel value before mapping is used as the Y value of the pixel value before mapping, and the Y value of the pixel value after mapping corresponding to the Y value of the pixel value before mapping is found from the skin color mapping relationship table, and the original pixel value Value of the pixel is used. color The Cb value and Cr value of the pixel value before mapping are used as the Cb value and Cr value of the pixel value before mapping, and the Cb value and Cr value of the pixel value after mapping corresponding to the Cb value and Cr value of the pixel value before mapping are looked up from the skin color mapping relationship table, so as to determine the (Y, Cb, Cr) value of the pixel value after mapping of the pixel.

[0086] Furthermore, in one embodiment, the Cb value and the Cr value of the pixel value before skin color mapping in each skin color relationship mapping table are respectively limited to a predetermined Cb value range and a predetermined Cr value range.

[0087] Among them, the above Cb value range of a < Cb < b and Cr value range of c < Cr < d can be set as the possible distribution range of human skin color. For colors other than skin color, no color transformation is required. That is, when the Cb value of the current pixel exceeds the Cb value range and / or the Cr value exceeds the Cr value range, it can be determined that the pixel does not belong to the skin, and it can be directly determined that the pixel value after skin color mapping is equal to the original pixel value before skin color mapping, without performing skin color transformation processing on it. In the original value range of 0 to 255, the Y value, Cb value, and Cr value in each skin color mapping relationship table can each have 256 different values. Therefore, the entire skin color mapping relationship table needs to record 256 + 256 * 256 = 65792 pixel values before mapping and their corresponding 65792 pixel values after mapping. After limiting the Cb value and Cr value within the ranges of a < Cb < b and c < Cr < d, the Y value in each skin color mapping relationship table can have 256 values, the Cb value can have (b - a - 1) values, and the Cr value can have (d - c - 1) values. Therefore, the entire skin color mapping relationship table only needs to record (256 + (b - a - 1) * (d - c - 1)) pixel values before mapping and their corresponding (256 + (b - a - 1) * (d - c - 1)) pixel values after mapping. Compared with the skin color mapping relationship table in the above RGB color space, this significantly reduces the length of the skin color mapping relationship table, greatly reduces the computational complexity, and does not compress the color space, with higher color transformation accuracy and smoother color changes.

[0088] For example, based on the distribution characteristics of human skin color, the Cb value range and Cr value range can be set as 77 < Cb < 127 and 133 < Cr < 173 respectively. Then the actual value range of the Cb value that needs to be transformed has only 49 values, and the value range of the Cr value has only 39 values; in the YUV color space, the Y channel corresponding to the image brightness can be processed separately. Therefore, the total length of the skin color mapping relationship table is 256 + 49 * 39 = 2167, that is, this table records 256 Y values before mapping and their corresponding 256 Y values after mapping, as well as 1911 Cb values and Cr values before mapping and their corresponding 1911 Cb values and Cr values after mapping.

[0089] In the above method for adjusting the skin color of a portrait, according to multiple skin color categories of the detected portrait image, multiple skin color mapping relationship tables corresponding to the predetermined multiple skin color categories are comprehensively utilized, rather than simply using a single skin color mapping relationship table. Thus, it is possible to more smoothly adjust the skin color of the portrait according to the skin color condition of the detected portrait, and comprehensively adjust the skin color of the portrait, enabling the skin color adjustment to flexibly adapt to various different skin color conditions and more accurately perform adaptive skin color adjustment for various different skin color conditions.

[0090] It should be understood that although Figure 2-4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0091] In one embodiment, Figure 6 As shown, a portrait skin color adjustment device 600 is provided, comprising: an image acquisition device 610, a skin color category probability determination module 620, a mapping pixel value determination module 630, and a skin color beautification image determination module 640, wherein:

[0092] An image acquisition module 610 is used to acquire an initial portrait image to be adjusted;

[0093] a skin color category probability determination module 620, configured to determine multiple skin color category probabilities that the initial portrait image belongs to multiple predetermined skin color categories;

[0094] a mapped pixel value determination module 630 for calculating a mapped pixel value of each pixel in the initial portrait image based on the multiple skin color category probabilities and multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0095] The skin color beautification image determination module 640 is configured to determine a skin color beautification image based on the mapped pixel value of each pixel.

[0096] The specific definition of the portrait skin color adjustment device 600 can be found in the definition of the portrait skin color adjustment method above and will not be repeated here. Each module in the portrait skin color adjustment device 600 can be implemented in whole or in part via software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within an electronic device in hardware form, or stored in memory within the electronic device in software form, allowing the processor to call and execute the corresponding operations of each module.

[0097] In one embodiment, an electronic device is provided, whose internal structure diagram can be as follows: Figure 7As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for adjusting the skin color of a portrait is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0098] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0099] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0100] Obtaining an initial portrait image to be adjusted;

[0101] determining a plurality of skin color category probabilities that the initial portrait image belongs to a plurality of predetermined skin color categories;

[0102] Calculate the mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and the multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0103] Based on the mapped pixel value of each pixel, a skin color beautification image is determined.

[0104] In other embodiments, when the processor executes the computer program, it further implements the steps of the portrait skin color adjustment method of any of the above embodiments.

[0105] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0106] Obtaining an initial portrait image to be adjusted;

[0107] determining a plurality of skin color category probabilities that the initial portrait image belongs to a plurality of predetermined skin color categories;

[0108] Calculate the mapped pixel value of each pixel in the initial portrait image according to the multiple skin color category probabilities and the multiple skin color mapping relationship tables corresponding to the multiple skin color categories;

[0109] Based on the mapped pixel value of each pixel, a skin color beautification image is determined.

[0110] In other embodiments, when the computer program is executed by the processor, the steps of the portrait skin color adjustment method in any of the above embodiments are also implemented.

[0111] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0112] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for adjusting skin color of a portrait, the method comprising: Obtaining an initial portrait image to be adjusted; determining a skin color category probability that the initial portrait image belongs to each of a plurality of predetermined skin color categories, thereby obtaining a plurality of skin color category probabilities corresponding to the plurality of skin color categories; Using the multiple skin color category probabilities as weights and using multiple skin color mapping relationship tables corresponding to the multiple skin color categories, calculate the mapped pixel value of each pixel in the initial portrait image; Determining a skin probability of each pixel in the initial portrait image belonging to skin; Combined with the skin probability of each pixel belonging to skin Based on the original pixel value of each pixel color And the mapped pixel value NewValue blend Determine the pixel value NewValue after skin color beautification of each pixel according to the following formula Beauty , thereby determining the skin color beautification image: Among them, the skin probability exist within the skin probability range.

2. The method according to claim 1, characterized in that The step of calculating the mapped pixel value of each pixel in the initial portrait image using the multiple skin color category probabilities as weights and the multiple skin color mapping relationship tables corresponding to the multiple skin color categories includes: Obtaining multiple skin color mapping relationship tables corresponding to the multiple skin color categories; Calculating a plurality of skin color mapping values ​​obtained by mapping the original pixel value of each pixel in the initial portrait image according to the plurality of skin color mapping relationship tables; The multiple skin color category probabilities are used as weights to perform weighted summation on the multiple skin color mapping values ​​corresponding to each pixel to determine the mapped pixel value of each pixel.

3. The method according to claim 1, characterized in that The skin probability of each pixel in the initial portrait image belonging to skin is determined by inputting the initial portrait image into a pre-trained skin detection neural network model.

4. The method according to claim 1, wherein Each of the plurality of skin color mapping relationship tables records a mapping relationship table from a pixel value before skin color mapping to a pixel value after skin color mapping under a corresponding skin color category.

5. The method according to any one of claims 1 to 4, characterized in that The pixel value in each of the plurality of skin color mapping relationship tables is expressed in an RGB color space; In each of the skin color mapping relationship tables under the RGB color space, the R values, G values, and B values ​​of multiple pixel values ​​before skin color mapping, as well as the R values, G values, and B values ​​of each pixel value before skin color mapping and the R values, G values, and B values ​​of the pixel value after skin color mapping are performed are recorded.

6. The method according to claim 5, characterized in that The R value, G value, and B value of the pixel value before skin color mapping in the skin color relationship mapping table are respectively compressed by a predetermined multiple.

7. The method according to any one of claims 1 to 4, characterized in that The pixel value in each of the plurality of skin color mapping relationship tables is expressed in a YUV color space; In each of the skin color mapping relationship tables under the YUV color space, multiple Y values ​​before skin color mapping and the Y value of each skin color mapping after skin color mapping are recorded, and multiple Cb values ​​and Cr values ​​before skin color mapping are recorded, as well as the Cb values ​​and Cr values ​​of each skin color mapping before skin color mapping and the Cb values ​​and Cr values ​​of each skin color mapping after skin color mapping are performed.

8. The method according to claim 7, characterized in that The Cb value and the Cr value of the pixel value before skin color mapping in the skin color relationship mapping table are respectively limited to a predetermined Cb value range and a predetermined Cr value range.

9. The method according to any one of claims 1 to 4, characterized in that The plurality of skin color categories include white skin color, yellow skin color, brown skin color, and black skin color.

10. The method according to any one of claims 1 to 4, characterized in that The sum of the multiple skin color category probabilities corresponding to the multiple skin color categories is 1.

11. The method according to any one of claims 1 to 4, characterized in that The skin color category probability that the initial portrait image belongs to each of a plurality of predetermined skin color categories is determined by inputting the initial portrait image into a pre-trained skin color classification neural network model.

12. A portrait skin color adjustment device, characterized in that: The device comprises: An image acquisition module, used to acquire an initial portrait image to be adjusted; a skin color category probability determination module, configured to determine a skin color category probability that the initial portrait image belongs to each of a plurality of predetermined skin color categories, thereby obtaining a plurality of skin color category probabilities corresponding to the plurality of skin color categories; a mapped pixel value determination module, configured to calculate a mapped pixel value of each pixel in the initial portrait image based on the multiple skin color category probabilities as weights and according to multiple skin color mapping relationship tables corresponding to the multiple skin color categories; Skin color beautification image determination module, used to combine the skin probability of each pixel belonging to skin Based on the original pixel value of each pixel color And the mapped pixel value NewValue blend Determine the pixel value NewValue after skin color beautification of each pixel according to the following formula Beauty , thereby determining the skin color beautification image: Among them, the skin probability exist within the skin probability range.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images