Image processing method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses an image processing method, which comprises the steps of acquiring a first pixel value of an image, determining a central pixel value from the first pixel value based on the distribution of the first pixel value, determining a standard region of the central pixel value, determining a first representative pixel value of a second region from the central pixel value based on the central pixel value and the standard region, and determining a translation amount corresponding to the first representative pixel value, adjusting the second pixel value according to the translation amount corresponding to the first representative pixel value, and obtaining and outputting a processed image. The embodiment of the invention further provides an image processing device, electronic equipment and a computer storage medium.

Description

Technical Field

[0001] The present application relates to a technology for processing the skin color of a portrait in an image, and more particularly to an image processing method, device, electronic device and computer storage medium. Background Art

[0002] Photography is an extremely extensive application that is both practical and artistic. Photography is also a key function and key technology of various terminals (including smart mobile terminals). Portrait shooting is one of the most important parts of photography and is also a key function of smart mobile terminal imaging.

[0003] Generally, people are very sensitive to the skin color of portraits in images and have various preferences. Portrait brightness, Auto White Balance (AWB), and skin color enhancement processing all have a key impact on portrait skin color. Portrait skin color optimization has high requirements for portrait brightness processing, accuracy and suitability of AWB, and accuracy of portrait skin color preference enhancement.

[0004] In the related art, the expected skin color value is obtained through a neural network model, and then the color of the skin color area of ​​the portrait is adjusted to the determined expected skin color value. The expected skin color value obtained by using the above neural network model is not accurate enough, resulting in poor processing effect on the skin color of the portrait. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, electronic device and computer storage medium, which can improve the processing effect of the skin color of a portrait in an image.

[0006] The technical solution of this application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0008] Obtaining a first pixel value of the image; wherein the first pixel value is: a pixel value of a second area in the first area in a first color space; the first area is a skin color area of ​​a portrait in the image; and the second area is a skin color area of ​​a face of the portrait in the first area;

[0009] determining a central pixel value from the first pixel values ​​based on a distribution of the first pixel values;

[0010] Determine a standard area of ​​the central pixel value; wherein the standard area is an area in a coordinate system formed by components included in the central pixel value;

[0011] Based on the central pixel value and the standard area, determining a first representative pixel value of the second area from the central pixel value, and determining a translation amount corresponding to the first representative pixel value;

[0012] According to the translation amount corresponding to the first representative pixel value, the second pixel value is adjusted to obtain and output a processed image; wherein the second pixel value is: the pixel value of the first area in the first color space.

[0013] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0014] An acquisition module is used to acquire a first pixel value of an image; wherein the first pixel value is: a pixel value of a second area in a first area in a first color space; the first area is a skin color area of ​​a portrait in the image; and the second area is a skin color area of ​​a face of the portrait in the first area;

[0015] A first determining module, configured to determine a central pixel value from the first pixel values ​​based on a distribution of the first pixel values;

[0016] A second determination module is used to determine a standard area of ​​the central pixel value; wherein the standard area is an area in a coordinate system formed by components included in the central pixel value;

[0017] A third determination module is used to determine a first representative pixel value of the second area from the central pixel value based on the central pixel value and the standard area, and determine a translation amount corresponding to the first representative pixel value;

[0018] The output module is used to adjust the second pixel value according to the translation amount corresponding to the first representative pixel value, obtain and output the processed image; wherein the second pixel value is: the pixel value of the first area in the first color space.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a storage medium storing instructions executable by the processor; the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in one or more of the above embodiments is executed.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0021] The embodiments of the present application provide an image processing method, device, electronic device and computer storage medium, which obtain a first pixel value of an image, where the first pixel value is: a pixel value of a second area in a first color space, where the first area is a skin color area of ​​a portrait in the image, and the second area is a skin color area of ​​a face of the portrait in the first area, and based on the distribution of the first pixel value, a central pixel value is determined from the first pixel value, and a standard area of ​​the central pixel value is determined, and based on the central pixel value and the standard area, the standard area is an area in a coordinate system composed of components included in the central pixel value, a first representative pixel value of the second area is determined from the central pixel value, and a translation amount corresponding to the first representative pixel value is determined, and the second pixel value is adjusted according to the translation amount corresponding to the first representative pixel value to obtain and output a processed image, where the second pixel value is : pixel values ​​of the first area in the first color space; that is, in the embodiment of the present application, the distribution of pixel values ​​of the above-mentioned facial skin color area is used to determine the central pixel value, the standard area of ​​the central pixel value is determined, and then the first representative pixel value can be determined based on the central pixel value and its standard area, so that the determined first representative pixel value can better represent the area in the image where the facial skin color needs to be optimized, and the translation amount corresponding to the first representative pixel value is determined, and an adjustment amount can be determined for the first representative pixel value, and then the portrait skin color area is adjusted based on the translation amount corresponding to the first representative pixel value. Since the determined first representative central pixel value can represent the area in the image where the facial skin color needs to be optimized, the translation amount corresponding to the first representative pixel value makes the adjustment of the portrait skin color area to the portrait more accurate, thereby improving the effect of portrait skin color optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic flow chart of an optional image processing method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of an example of an optional image processing system provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of an example of an optional image processing method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of an optional example of an image containing a portrait provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of an example of an optional standard area provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of an example of an optional standard area translation amount provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of an optional image processing device provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0031] In the related art, in the optimization of skin color of portraits on mobile devices, usually at the end of the image signal processing (ISP) link, an image in the format of red (Red, R), green (Green, G), blue (Blue, B) or brightness (Luminance or Luma, Y) and chrominance (Chrominance or Chroma, UV) is input, and the color of the skin color area of ​​the image portrait is adjusted to the expected value, wherein the expected value is output based on a neural network model, etc.

[0032] In addition, in the optimization of portrait skin color, the AWB of the face skin color in the Raw domain of the ISP link can also be used. Usually, based on the statistical information of the face skin color area, generally the statistical values ​​of R / G and B / G at each grid position in the face area, the color temperature value is obtained by comparing the statistical value with the calibrated color temperature table, and the portrait skin color area is processed as a whole based on the color temperature value. However, this method is not accurate enough and cannot handle facial mixed light well.

[0033] For skin enhancement in the post-processing stage, the pixel values ​​of the skin color area of ​​the portrait are usually adjusted in the tail module of the ISP link. Usually, the expected value is set in the RGB domain, or in the hue (H), saturation (S) and value (V) space or brightness (L) and color component (ab) space.

[0034] However, due to the use of statistical values ​​of the entire skin color area, uniform adjustment of the skin color area, and difficulty in generalizing the expected value setting strategy, inaccurate skin color optimization areas, negative optimization of local areas, and negative optimization scenarios may occur. It can be seen that the use of the above methods leads to poor results in skin color optimization.

[0035] In order to solve the technical problem that the skin color optimization effect of portraits is not good, the embodiment of the present application provides an image processing method. Figure 1 A flowchart of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the image processing method may include:

[0036] S101: Acquire a first pixel value of an image;

[0037] In an embodiment of the present application, a first pixel value of an image is obtained, wherein the image is an image including a portrait, and the image can be obtained from an imaging unit and can also be obtained from a storage unit. Here, the embodiment of the present application does not make any specific limitation on this.

[0038] For the image acquired from the imaging unit, in addition to the pixel values ​​of the image, it also includes other image-related data, such as face detection frame, face key points, face attributes, portrait skin mask, portrait instance segmentation, and other information output by the ISP pre-order module, based on which an image including a portrait can be acquired. For the image acquired from the storage unit, the acquired image can be processed to obtain image-related data, so that an image including a portrait can be acquired based on this.

[0039] In addition, the number of human portraits in the above image may be one, or two or more, and this is not specifically limited in the embodiments of the present application.

[0040] The first pixel value is: the pixel value of the second area in the first area in the first color space; the first area is the skin color area of ​​the portrait in the image; the second area is the skin color area of ​​the face of the portrait in the first area. That is to say, the skin color area of ​​the portrait in the above image is recorded as one area, called the first area, the skin color area of ​​the face in the skin color area of ​​the portrait is recorded as one area, called the second area, and the first pixel value is the pixel value of the skin color area of ​​the face.

[0041] When there is one portrait in the above image, the portrait corresponds to the first area and the second area. When there are two portraits, each portrait corresponds to the first area and the second area. It can be seen that the first area, the second area and the portrait correspond one-to-one, that is, each portrait has the first area and the second area.

[0042] In an embodiment of the present application, the first area and the second area of ​​a portrait in the image can be used for the image, and the above-mentioned image processing method can be used for the first area and the second area of ​​each portrait. A preset number of portraits can also be selected from the portraits in the image, and the above-mentioned image processing can be performed on the first area and the second area of ​​the preset number of portraits. Here, the embodiment of the present application does not make any specific limitations on this.

[0043] It should be noted that the above-mentioned first color space can be any color space. For example, the color space can be a Lab color space, an RGB color space, an HSV color space, or a brightness / lightness (L) chroma / chroma (C) hue / hue (H) color space. Here, the embodiment of the present application does not make any specific limitations on this.

[0044] The above-mentioned first pixel value is the pixel value of the second area in the first color space. The pixel value can be one, two or three components of the first component of the first color space, the second component of the first color space and the third component of the first color space. Here, the embodiment of the present application does not make any specific limitation on this.

[0045] In addition, the above-mentioned first pixel value can also be all or part of the pixel values ​​of the second area in the first color space, for example, the value obtained by downsampling the second area of ​​the image. Here, the embodiment of the present application does not make any specific limitations on this.

[0046] S102: determining a central pixel value from the first pixel values ​​based on the distribution of the first pixel values;

[0047] After the first pixel value is obtained through S101, in S102, the central pixel value can be determined from the first pixel values ​​based on the distribution of the first pixel values. Here, the distribution of the first pixel values ​​can be statistically analyzed, and the central pixel value can be selected from the first pixel values ​​based on the statistical results.

[0048] Here, a histogram can be used to perform statistics on the distribution of the first pixel value. For example, when the first pixel value is one component, a one-dimensional histogram is used to perform statistics on the distribution of the first pixel value. When the first pixel value is two components, a two-dimensional histogram is used to perform statistics on the distribution of the first pixel value. When the first pixel value is three components, a three-dimensional histogram is used to perform statistics on the distribution of the first pixel value.

[0049] Based on the statistical results, the distribution of the first pixel values ​​can be known, for example, which intervals have a larger proportion of first pixel values ​​and which intervals have a smaller proportion of first pixel values. Based on the size of the proportion, the central pixel value can be selected from the first pixel values. The central pixel value is the pixel value with the highest degree of optimization demand among the first pixel values, that is, the skin color area that most needs to be optimized, which may include one area or two or more areas.

[0050] Here, a sliding window method can be used to slide the pixel values ​​of the statistical results to select the middle value of the pixel value of the sliding window corresponding to the maximum value of the sum of the statistical values ​​in the sliding window, and the pixel value corresponding to the maximum value of the statistical value ratio of the pixel value can also be selected. Here, the embodiment of the present application does not make specific limitations on this.

[0051] It should be noted that the above-mentioned central pixel value can be one, or can be two or more, and this is not specifically limited in the embodiment of the present application.

[0052] S103: Determine a standard area of ​​central pixel value;

[0053] After the central pixel value is determined by S102, the standard area of ​​the central pixel value needs to be determined in S103, wherein the standard area is the area under the coordinate system composed of the components included in the central pixel value. In other words, a coordinate system can be formed using the components included in the central pixel value. For example, when the central pixel value includes one component, the X-axis of the coordinate system is the component. When the central pixel value is two components, the coordinate system is a two-dimensional coordinate system in which the X-axis is one component and the Y-axis is another component. When the central pixel value is three components, the coordinate system is a three-dimensional coordinate system in which the X-axis is the first component, the Y-axis is the second component, and the Z-axis is the third component. Here, the embodiment of the present application does not make specific limitations on this.

[0054] It should be noted that each central pixel value has a corresponding standard area, which can be determined based on the region to which the central pixel value belongs, wherein the region to which the central pixel value belongs can be the country to which the portrait in the second region belongs, or it can be determined based on the image parameters of the area corresponding to the central pixel value, or it can be determined based on the region to which the central pixel value belongs and the image parameters of the area corresponding to the central pixel value. Then, the standard area determined based on different regions and / or image parameters is different; here, the embodiment of the present application does not make specific limitations on this.

[0055] In addition, when the above-mentioned central pixel value includes one component, its corresponding standard area is an interval on the X-axis, when the above-mentioned central pixel value includes two components, its corresponding standard area is a plane area in a two-dimensional coordinate system, and when the above-mentioned central pixel value includes three components, its corresponding standard area is a spatial area in a three-dimensional coordinate system. Here, the embodiment of the present application does not make specific limitations on this.

[0056] S104: Based on the central pixel value and the standard area, determine a first representative pixel value of the second area from the central pixel value, and determine a translation amount corresponding to the first representative pixel value;

[0057] After determining the standard area corresponding to the central pixel value in S103, in S104, based on the central pixel value and the standard area, the first representative pixel value of the second area can be determined from the central pixel value, and the translation amount corresponding to the first representative pixel value can be determined.

[0058] When the number of the central pixel values ​​is one, the central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the first representative pixel value is determined based on the central pixel value and the standard area.

[0059] When the number of the above-mentioned central pixel values ​​is two or more, based on each central pixel value and the standard area of ​​each central pixel value, the translation amount corresponding to each central pixel value can be determined, and then, based on the translation amount corresponding to each central pixel value, a central pixel value is selected from all the central pixel values ​​as the first representative pixel value, and the translation amount corresponding to the selected central pixel value is used as the translation amount corresponding to the first representative pixel value.

[0060] The first representative pixel value is a value selected from the central pixel value, which is an area in the image and can represent the skin color area in the second area that needs to be optimized the most.

[0061] S105: adjusting the second pixel value according to the translation amount corresponding to the first representative pixel value, and obtaining and outputting a processed image;

[0062] After determining the first representative pixel value and the translation amount corresponding to the first representative pixel value through the above S104, in S105, the second pixel value needs to be adjusted according to the translation amount corresponding to the first representative pixel value to obtain and output a processed image, wherein the second pixel value is: the pixel value of the first area in the first color space.

[0063] That is, after knowing the translation amount corresponding to the first representative pixel value, the pixel value of the skin color area of ​​the portrait in the first color space is adjusted, so that a processed image can be obtained and output.

[0064] Among them, for the above-mentioned adjustment, based on the pixel value of the skin color area of ​​the portrait in the first color space and the translation amount corresponding to the first representative pixel value, it can be determined in the coordinate system formed by the components included in the central pixel value that the pixel value of the skin color area of ​​the portrait in the first color space is translated according to the translation amount corresponding to the first representative pixel value to obtain the translated value, and the pixel value of the skin color area of ​​the portrait in the first color space is adjusted to the translated value, so as to obtain a processed image, and then the processed image can be outputted using an output unit such as a display.

[0065] It should be noted that, for each portrait in the image, the skin color of each portrait is processed using the above-mentioned image processing method, so as to obtain a processed image.

[0066] In order to determine the central pixel value from the first pixel value, in an optional embodiment, S102 may include:

[0067] Performing statistics on the distribution of the first pixel value to obtain a statistical result;

[0068] Sliding on the pixel value of the statistical result according to a preset sliding window and a preset sliding step size to obtain a first sliding window;

[0069] A sliding window is selected from the first sliding window based on the value of the first sliding window, and a central pixel value is determined from pixel values ​​of the selected sliding window.

[0070] It can be understood that in statistically analyzing the distribution of the first pixel values ​​to obtain statistical results, the first pixel values ​​can be statistically analyzed in the form of a histogram to obtain a histogram of the first pixel values. For example, the first pixel values ​​are grouped into 101 groups, the boundary of the first group is [0, 1), the boundary of the second group is [1, 2), and so on. The boundary of the 100th group is set to [99, 100), the boundary of the 101st group is [100, 101), and so on. In this way, the frequency distribution of the pixel values ​​can be known from the histogram of the first pixel values.

[0071] Then, slide on the pixel value of the statistical result according to the preset sliding window and the preset sliding step size, for example, the preset sliding window is 21, and the preset sliding step size is 1, so that multiple first sliding windows can be obtained, and the value of each first sliding window is calculated, wherein the value of the first sliding window is the sum of the statistical values ​​of the first sliding windows. In other words, the sum of the statistical values ​​of each first sliding window is calculated.

[0072] Finally, based on the value of the first sliding window, a sliding window is selected from the first sliding window, and the central pixel value is determined from the pixel values ​​of the selected sliding window. For example, a value greater than a preset threshold value is selected from the values ​​of the first sliding window, and the value of the preset position of the selected sliding window is used as the central pixel value.

[0073] In this way, a suitable sliding window can be selected through statistical methods and sliding window methods, and then the central pixel value can be selected from the suitable sliding window. The above method is used so that the central pixel value can represent the skin color area that needs to be optimized in the face area, and a suitable reference area is determined for skin color optimization.

[0074] With respect to the above statistics on the distribution of the first pixel value, in an optional embodiment, performing statistics on the distribution of the first pixel value to obtain the statistical result may include:

[0075] Selecting from the second region an area where the first component of the first pixel value falls within a preset component interval;

[0076] Statistics are performed on the distribution of the second component of the first pixel value of the selected area and the third component of the first pixel value of the selected area to obtain statistical results.

[0077] It can be understood that in order to make the statistical results more accurate for the case where the central pixel value contains the second component and the third component, here, the area where the first component of the first pixel value falls into the preset component interval is first selected from the second area. In this way, the area most likely to be optimized is screened out from the second area, and then the second component and the third component are counted in the screened area to obtain the statistical results.

[0078] In this way, the above-screened area is used for statistics, and the central pixel value is determined by sliding the window based on the statistical result, so that the central pixel value can be determined more accurately.

[0079] For the case where two central pixel values ​​are determined, in an optional embodiment, selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining the central pixel value from the pixel values ​​of the selected sliding window include:

[0080] Selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining a first central pixel value from pixel values ​​of the selected sliding window;

[0081] Determine the statistical results other than the preset pixel value interval in the statistical results as partial statistical results;

[0082] Sliding on the pixel values ​​of the partial statistical results according to the preset sliding window and the preset sliding step size to obtain a second sliding window;

[0083] Selecting a sliding window from the second sliding window based on the value of the second sliding window, and determining a second central pixel value from pixel values ​​of the selected sliding window;

[0084] The first central pixel value and the second central pixel value are determined as central pixel values.

[0085] It can be understood that a sliding window can be selected from the first sliding window based on the value of the first sliding window, and the first central pixel value can be determined from the pixel values ​​of the selected sliding window. Then, the statistical results except for the preset pixel value interval in the statistical results are determined as partial statistical results, wherein the preset pixel value interval is: an interval of preset pixel values ​​floating up and down based on the first central pixel value.

[0086] That is to say, the pixel values ​​outside the preset pixel value range based on the first central pixel value are obtained in the same manner as the above-mentioned first sliding window to obtain a second sliding window, and then a sliding window is selected from the second sliding window based on the value of the second sliding window, and the second central pixel value is determined from the pixel values ​​of the selected sliding window, wherein the value of the second sliding window is the sum of the statistical values ​​of the second sliding window, which is similar to the method for the value of the first sliding window and will not be repeated here.

[0087] Finally, the determined first central pixel value and the second central pixel value are both used as central pixel values.

[0088] In this way, two central pixel values ​​are determined in the above manner, which provides multiple reference areas for further determining the first representative pixel value, thereby improving the accuracy of the first representative pixel value.

[0089] In order to determine the central pixel value in the case where there is only one central pixel value, in an optional embodiment, a sliding window is selected from the first sliding window based on the value of the first sliding window, and the central pixel value is determined from the pixel values ​​of the selected sliding window, including:

[0090] The middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the first sliding window is determined as the central pixel value.

[0091] It can be understood that after determining the value of the first sliding window, the maximum value is selected from these values, that is, the value with the highest frequency of distributed pixel values, the sliding window corresponding to the maximum value is selected, and the middle value of the selected sliding window is used as the central pixel value.

[0092] In this way, since the maximum value in the value of the first sliding window is the area with the highest pixel value frequency distribution, selecting the central pixel value here is more accurate and can better represent the area that needs to be optimized in the facial skin color area, providing a favorable reference point for skin color optimization.

[0093] In order to determine the central pixel value in the case where there are only two central pixel values, in an optional embodiment, a sliding window is selected from the first sliding window based on the value of the first sliding window, and the first central pixel value is determined from the pixel values ​​of the selected sliding window, including:

[0094] Determine the middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the first sliding window as the first central pixel value;

[0095] Selecting a sliding window from the second sliding window based on the value of the second sliding window, and determining a second central pixel value from pixel values ​​of the selected sliding window, comprising:

[0096] The middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the second sliding window is determined as the second central pixel value.

[0097] It can be understood that, similar to the above-mentioned method of determining a central pixel value, after determining the value of the first sliding window, the maximum value is selected from these values, that is, the value with the largest frequency of distributed pixel values, the sliding window corresponding to the maximum value is selected, and the middle value of the selected sliding window is used as the first central pixel value.

[0098] After determining the value of the second sliding window, the maximum value is selected from these values, that is, the value with the highest frequency of distributed pixel values, the sliding window corresponding to the maximum value is selected, and the middle value of the selected sliding window is used as the second central pixel value.

[0099] In this way, since the maximum value in the value of the first sliding window is the area with the most pixel value frequency distribution, and the maximum value in the value of the second sliding window is the area with the most pixel value frequency distribution, and the two areas are separated by a certain distance, selecting the first central pixel value and the second central pixel value here is more comprehensive and accurate, and can more comprehensively characterize the areas in the facial skin color area that need to be optimized, providing two favorable reference points for skin color optimization, and laying the foundation for further determining the first representative pixel value.

[0100] With respect to the above first pixel value, in an optional embodiment, the first pixel value includes one or more of the following:

[0101] A first component of the first pixel value, a second component of the first pixel value, and a third component of the first pixel value.

[0102] It can be understood that the first pixel value generally includes three components, and the first pixel value can be one, two or more than two components among the first component, the second component and the third component.

[0103] In this way, one, two or more of the three components can be used as the basis to determine the central pixel value, and then determine the standard area of ​​the central pixel value and the translation amount corresponding to the first representative pixel value, which is conducive to skin color optimization to improve the effect of skin color optimization.

[0104] In order to determine the standard area of ​​the central pixel value, in an optional embodiment, S103 may include:

[0105] A standard area for the central pixel value is determined according to the region to which the portrait in the second area belongs and / or image parameters corresponding to the central pixel value.

[0106] It can be understood that in order to determine the standard area of ​​the central pixel value, standard areas corresponding to different regions and / or different image parameters are stored locally in advance. Then, after knowing the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value, the standard area of ​​the central pixel value can be determined.

[0107] Among them, the region to which the portrait in the above-mentioned second area belongs can be determined by face recognition technology to which region the face in the second area belongs, and the image parameters corresponding to the center pixel value are determined by the image parameters of the image area corresponding to the pixel values ​​of the sliding window with the center pixel value as the middle value. Therefore, the standard region of the center pixel value can be determined based on the region to which the portrait in the second area belongs, and the standard region of the center pixel value can be determined based on the image parameters corresponding to the center pixel value. The standard region of the center pixel value can also be determined based on the region to which the portrait in the second area belongs and the image parameters corresponding to the center pixel value. Here, the embodiments of the present application do not make specific limitations on this.

[0108] In this way, the standard area of ​​the central pixel value is determined by the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value, so that the determined standard area is more in line with the requirements of the portrait in the second area for skin color optimization, which is conducive to improving the effect of skin color optimization.

[0109] In addition, in order to improve the accuracy of the standard area of ​​the central pixel value, in an optional embodiment, determining the standard area of ​​the central pixel value according to the region to which the portrait of the second area belongs and / or the image parameter corresponding to the central pixel value may include:

[0110] determining the number of light sources in the second area;

[0111] Based on the number of light sources in the second area, and according to the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value, a standard area of ​​the central pixel value is determined.

[0112] It is understandable that the number of light sources in the second area can be determined first. Due to the different numbers of light sources, the number of standard areas for determining the central pixel value is also different according to the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value.

[0113] Here, for a light source, one central pixel value determines a standard area, and two central pixel values ​​can select one central pixel value to determine a standard area.

[0114] For the case of two or more light sources, each central pixel value determines a corresponding standard area, so the standard area for each central pixel value can be determined, wherein the central pixel value corresponds to the standard area one by one.

[0115] For each central pixel value, the standard area of ​​the central pixel value is determined according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value.

[0116] In this way, when determining the central pixel value, the number of light sources in the second area is taken into consideration, so that the determined standard area of ​​the central pixel value is more accurate, which is beneficial to improving the effect of skin color optimization.

[0117] For a case where there are two central pixel values, in order to determine the number of light sources in the second area, in an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value, and determining the number of light sources in the second area includes:

[0118] Determine a first proportion of the sum of the statistical values ​​of the sliding window with the second central pixel value as the intermediate value to the total number of the first pixel values;

[0119] When the first proportion is less than the preset proportion, determining the number of light sources to be one;

[0120] When the first proportion is greater than or equal to the preset proportion, the number of light sources is determined to be two.

[0121] It can be understood that the sum of the statistical values ​​of the sliding window with the second central pixel value as the middle value is first calculated, that is, the value of the second sliding window with the second central pixel value as the middle value, and the proportion of this value to the total number of first pixel values ​​is calculated, which is recorded as the first proportion.

[0122] The first proportion is compared with the preset proportion. After comparison, if the first proportion is less than the preset proportion, it means that the proportion of the second central pixel value is small, and it can be considered that this is a single light source scene, so the number of light sources is determined to be one. If the first proportion is greater than or equal to the preset proportion, it means that the proportion of the second central pixel value is large, and it can be considered that this is a dual light source scene, so the number of light sources is determined to be two.

[0123] In this way, by determining the number of light sources by the proportion of the sliding window value corresponding to the second central pixel value, the precise number of light sources can be determined for the image, which helps to determine the accurate standard area, thereby improving the skin color optimization effect.

[0124] For a case where there are two central pixel values ​​and one light source, in order to determine the standard area corresponding to the central pixel value, in an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value; based on the number of light sources in the second area, according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value, determining the standard area of ​​the central pixel value includes:

[0125] When the number of light sources in the second area is one, the standard area of ​​the central pixel value is determined according to the region to which the portrait in the second area belongs, the color temperature value of the second area, and the brightness value corresponding to the first central pixel value.

[0126] It can be understood that when it is determined that the number of light sources in the second area is one, the region to which the portrait in the second area belongs is obtained, the color temperature value of the second area is obtained, and the brightness value corresponding to the first central pixel value is obtained, wherein the brightness value corresponding to the first central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the first central pixel value as the middle value. In other words, the average value of the brightness values ​​of the image area corresponding to the pixel values ​​of the first sliding window with the first central pixel value as the middle value is calculated and used as the brightness value corresponding to the first central pixel value.

[0127] Then, the corresponding standard area is searched in the standard areas corresponding to different regions and / or different image parameters that are pre-stored locally, and is used as the standard area for the central pixel value.

[0128] In this way, determining the standard area of ​​the central pixel value for the case of a single light source is conducive to determining the first representative pixel value and the translation amount corresponding to the first representative pixel value, laying a foundation for skin color optimization.

[0129] For a case where there are two central pixel values ​​and two light sources, in order to determine the standard area corresponding to the central pixel value, in an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value; based on the number of light sources in the second area, according to the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value, determining the standard area of ​​the central pixel value includes:

[0130] When the number of light sources in the second area is two, determining a first standard area for the first central pixel value according to the area to which the portrait in the second area belongs, the color temperature value corresponding to the first central pixel value, and the brightness value corresponding to the first central pixel value;

[0131] Determine a second standard area for the second central pixel value according to the area to which the portrait in the second area belongs, the color temperature value corresponding to the first central pixel value, and the brightness value corresponding to the second central pixel value;

[0132] A first standard region for the first central pixel value and a second standard region for the second central pixel value are determined as standard regions for the central pixel values.

[0133] It can be understood that when it is determined that the number of light sources in the second area is two, the area to which the portrait of the second area belongs is obtained, the color temperature value corresponding to the first central pixel value and the color temperature value corresponding to the second central pixel value are obtained, and the brightness value corresponding to the first central pixel value and the brightness value corresponding to the second central pixel value are obtained, wherein the brightness value corresponding to the first central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the first central pixel value as the middle value, and the brightness value corresponding to the second central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the second central pixel value as the middle value. In other words, the average value of the brightness values ​​of the image area corresponding to the pixel values ​​of the first sliding window with the first central pixel value as the middle value is calculated and used as the brightness value corresponding to the first central pixel value, and the average value of the brightness values ​​of the image area corresponding to the pixel values ​​of the first sliding window with the second central pixel value as the middle value is calculated and used as the brightness value corresponding to the second central pixel value.

[0134] For the first central pixel value, a corresponding standard area is searched in the standard areas corresponding to different regions and / or different image parameters that are pre-stored locally, as the first standard area for the first central pixel value; for the second central pixel value, a corresponding standard area is searched in the standard areas corresponding to different regions and / or different image parameters that are pre-stored locally, as the second standard area for the second central pixel value, and the two standard areas are used as standard areas for central pixel values.

[0135] In this way, for the dual-light source scene, a standard area is determined for each central pixel value, thereby determining multiple areas as standard areas, providing more data basis for determining the first representative pixel value, which is conducive to determining a more accurate first representative pixel value.

[0136] In order to determine the color temperature value corresponding to the first central pixel value and the color temperature value corresponding to the second central pixel value, in an optional embodiment, the method further includes:

[0137] Determine the sum of the color temperature value of the second area and the preset color temperature value as the color temperature value corresponding to the first central pixel value;

[0138] The difference between the color temperature value of the second area and the preset color temperature value is determined as the color temperature value corresponding to the second central pixel value.

[0139] It can be understood that, usually in the case of dual light sources, a preset color temperature value is stored locally. For example, the preset color temperature value can be 2000K. The sum of the acquired color temperature value of the second area and the preset color temperature value is used as the color temperature value corresponding to the first central pixel value, and the difference between the acquired color temperature value of the second area and the preset color temperature value is used as the color temperature value corresponding to the first central pixel value.

[0140] In this way, different color temperature values ​​are determined for different central pixel values, so that the determined color temperature value is more accurate, thereby making the determined standard area of ​​each central pixel value more accurate, which is conducive to determining a more accurate first representative pixel value.

[0141] For a single light source, in order to determine the first representative pixel value and the translation amount corresponding to the first representative pixel value, in an optional embodiment, S104 may include:

[0142] When the number of the light source in the second area is one, determining the first central pixel value as the first representative pixel value;

[0143] When the first representative pixel value is within the range of the standard area, determining that the translation amount corresponding to the first representative pixel value is zero;

[0144] When the first representative pixel value is outside the range of the standard area, the translation amount corresponding to the first representative pixel value is determined according to a first vector in a coordinate system formed by components included in the central pixel value.

[0145] It can be understood that when the light source of the two areas is determined to be a single light source, the first central pixel value is directly determined as the first representative pixel value, and then it is determined whether the first representative pixel value is within the range of the standard area. If so, it means that the skin color of the first representative pixel value is the standard skin color and does not need to be optimized. Therefore, the translation amount corresponding to the first representative pixel value is determined to be zero, that is, no skin color optimization is required.

[0146] If not, it is necessary to use the first vector in the coordinate system formed by the components included in the center pixel value to determine the translation amount corresponding to the first representative pixel value, wherein the first vector is: the vector from the first representative pixel value to the preferred center point of the standard area. In other words, the translation amount corresponding to the first representative pixel value is determined by using the vector from the first representative pixel value to the preferred center point of the standard area.

[0147] It should be noted that, using the above-mentioned first vector, one translation amount can be determined as the translation amount corresponding to the first representative pixel value, or two translation amounts can be determined as the translation amounts corresponding to the first representative pixel value. Here, the embodiment of the present application does not make any specific limitations on this.

[0148] In this way, by determining the position of the first representative pixel value in the standard area under a single light source in the above manner, the translation amount corresponding to the first representative pixel value can be accurately determined, which is beneficial to improving the effect of skin color optimization.

[0149] In order to determine the translation amount corresponding to the first representative pixel value under a single light source, in an optional embodiment, determining the translation amount corresponding to the first representative pixel value according to a first vector in a coordinate system formed by components included in the central pixel value includes:

[0150] The second vector in the coordinate system is determined as the translation amount corresponding to the first representative pixel value.

[0151] It can be understood that the second vector in the coordinate system can be determined as the translation amount corresponding to the first representative pixel value; wherein the second vector is: the vector from the first representative pixel value to the intersection point; the intersection point is: the intersection point of the line connecting the first representative pixel value and the preferred center point of the standard area and the boundary of the standard area. In other words, the vector of the standard area displacement in the first vector is used as the translation amount corresponding to the first representative pixel value, and the translation amount can be called the correction translation amount.

[0152] In this way, by determining the above-mentioned correction shift amount and using the correction shift amount to process the above-mentioned second pixel value, the skin color in the second area can be corrected, thereby improving the effect of skin color correction.

[0153] Further, in order to determine the translation amount corresponding to the first representative pixel value under a single light source, in an optional embodiment, a first vector in a coordinate system composed of components included in the central pixel value determines the translation amount of the first representative pixel value, including:

[0154] Determine the second vector in the coordinate system as the first translation amount corresponding to the first representative pixel value;

[0155] Determine the third vector in the coordinate system as the second translation amount corresponding to the first representative pixel value;

[0156] The translation amount corresponding to the first representative pixel value is determined according to the first translation amount and the second translation amount.

[0157] It can be understood that the second vector in the coordinate system is determined as the first translation corresponding to the first representative pixel value, and the third vector in the coordinate system is also determined as the second translation corresponding to the first representative pixel value, wherein the second vector is: the vector from the first representative pixel value to the intersection point; the intersection point is: the intersection point of the line connecting the first representative pixel value and the preference center point of the standard area and the boundary of the standard area, and the third vector is: the vector from the intersection point to the preference center point of the standard area. In other words, the vector outside the standard area in the first vector is taken as the first translation amount, which can be called the correction translation amount, and the vector within the standard area in the first vector is taken as the second translation amount, which can be called the preference translation amount, thereby determining two translation amounts, and then determining the translation amount corresponding to the first representative pixel value based on these two translation amounts.

[0158] In this way, the translation amount corresponding to the first representative pixel value can be determined from the two aspects of correction and preference, so that the determined translation amount corresponding to the first representative pixel value is more comprehensively considered, thereby making the processed image more in line with the user's preference.

[0159] In order to determine the translation amount corresponding to the first representative pixel value, in an optional embodiment, determining the translation amount corresponding to the first representative pixel value according to the first translation amount and the second translation amount includes:

[0160] The sum of the first translation amount and the second translation amount is determined as the translation amount corresponding to the first representative pixel value.

[0161] It can be understood that the sum of the first translation and the second translation, that is, the first vector, can be directly determined as the translation corresponding to the first representative pixel value. Of course, the first translation and the second translation can also be weighted summed to determine the translation corresponding to the first representative pixel value. Here, the embodiment of the present application does not make any specific limitations on this.

[0162] In this way, by performing the deviation correction translation and the preference translation on the skin color area of ​​the portrait, the skin color area optimization of the portrait is made more accurate, thereby improving the effect of the skin color optimization.

[0163] In a dual-light source scenario, in order to determine the translation amount corresponding to each central pixel value, in an optional embodiment, based on the central pixel value and the standard area, determining the first representative pixel value of the second area from the central pixel value, and determining the translation amount corresponding to the first representative pixel value, includes:

[0164] When the number of light sources in the second area is two, determining a translation amount corresponding to the first central pixel value according to the first central pixel value and a first standard area of ​​the first central pixel value;

[0165] determining a translation amount corresponding to the second central pixel value according to the second central pixel value and a second standard area of ​​the second central pixel value;

[0166] Based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, a first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined.

[0167] It can be understood that for the dual-light source scene, for the first central pixel value, the translation amount corresponding to the first central pixel value is determined based on the first central pixel value and the first standard area of ​​the first central pixel value. This is similar to determining the translation amount in the above-mentioned single-light source scene, and will not be repeated here.

[0168] For the second central pixel value, the translation amount corresponding to the second central pixel value is determined according to the second central pixel value and the second standard area of ​​the second central pixel value. Here, it is similar to determining the translation amount in the above single light source scene, and will not be repeated here.

[0169] Then, after obtaining the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, based on these two values, a pixel value can be selected from the first central pixel value and the second central pixel value as the first representative pixel value, and the translation amount corresponding to the first representative pixel value can be determined.

[0170] It should be noted that the translation amount corresponding to the above-mentioned first central pixel value can be one translation amount or two translation amounts, and the translation amount corresponding to the above-mentioned second central pixel value can be one translation amount or two translation amounts. Here, the embodiment of the present application does not make any specific limitations on this.

[0171] In this way, the corresponding translation amount can be determined for each central pixel value in a dual-light source scenario, so that the first representative pixel value and the translation amount corresponding to the first representative pixel value can be accurately determined, which helps to improve the effect of skin color optimization.

[0172] In a dual-light source scenario, in order to determine an accurate first representative pixel value and a translation amount corresponding to the first representative pixel value, in an optional embodiment, based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined, including:

[0173] Selecting the maximum value of the absolute value of the translation amount corresponding to the first central pixel value and the absolute value of the translation amount corresponding to the second central pixel value;

[0174] The central pixel value corresponding to the maximum value is determined as the first representative pixel value;

[0175] The translation amount corresponding to the maximum value is determined as the translation amount corresponding to the first representative pixel value.

[0176] It can be understood that the absolute value of the translation corresponding to the first central pixel value is calculated first, and the absolute value of the translation corresponding to the second central pixel value is calculated, and then the two absolute values ​​are compared, the maximum value is selected, and the central pixel value corresponding to the maximum value is determined as the first representative pixel value, and the translation corresponding to the maximum value is determined as the translation corresponding to the first representative pixel value.

[0177] In this way, the central pixel value corresponding to the maximum absolute value of the translation amount is taken as the first representative pixel value, and the translation amount corresponding to the maximum value is determined as the translation amount corresponding to the first representative pixel value. That is, the central pixel value that most needs to be optimized is taken as the first representative pixel value, which helps to improve the optimization of the portrait skin color.

[0178] For a dual-light source scene, in an optional embodiment, the translation amount corresponding to the first central pixel value includes: a first translation amount corresponding to the first central pixel value and a second translation amount corresponding to the first central pixel value; the translation amount corresponding to the second central pixel value includes: a first translation amount corresponding to the second central pixel value and a second translation amount corresponding to the second central pixel value.

[0179] It can be understood that by using the above-mentioned single light source to determine the translation amount corresponding to the first central pixel value, the first translation amount corresponding to the first central pixel value can be determined, that is, the correction translation amount corresponding to the first central pixel value; the second translation amount corresponding to the first central pixel value can be determined, that is, the preferred translation amount corresponding to the first central pixel value; the first translation amount corresponding to the second central pixel value can be determined, that is, the correction translation amount corresponding to the second central pixel value; the first translation amount corresponding to the second central pixel value can be determined, that is, the preference translation amount corresponding to the second central pixel value.

[0180] In this way, each central pixel value has a correction translation amount and a preference translation amount, which helps to select a more accurate first representative pixel value.

[0181] Each central pixel value has a correction translation amount and a preference translation amount. In order to determine the first representative pixel value and the translation amount corresponding to the first representative pixel value, in an optional embodiment, based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined, including:

[0182] When the absolute value of the first translation amount corresponding to the first central pixel value is equal to the absolute value of the first translation amount corresponding to the second central pixel value, selecting the maximum value of the absolute value of the second translation amount corresponding to the first central pixel value and the absolute value of the second translation amount corresponding to the second central pixel value;

[0183] The central pixel value corresponding to the maximum value is determined as the first representative pixel value;

[0184] The translation amount corresponding to the maximum value is determined as the translation amount corresponding to the first representative pixel value.

[0185] It can be understood that the absolute value of the first translation corresponding to the first central pixel value is compared with the absolute value of the first translation corresponding to the second central pixel value. If they are not equal, the central pixel value corresponding to the maximum value is selected as the first representative pixel value, and the translation corresponding to the central pixel value corresponding to the maximum value is selected as the translation corresponding to the first representative pixel value.

[0186] When the absolute value of the first translation corresponding to the first central pixel value is equal to the absolute value of the first translation corresponding to the second central pixel value, compare the absolute value of the second translation corresponding to the first central pixel value with the absolute value of the second translation corresponding to the second central pixel value. If they are not equal, select the central pixel value corresponding to the maximum value as the first representative pixel value, and the translation corresponding to the central pixel value corresponding to the maximum value as the translation corresponding to the first representative pixel value.

[0187] In this way, by comparing the absolute value of the first translation corresponding to the first central pixel value with the absolute value of the first translation corresponding to the second central pixel value, and comparing the absolute value of the second translation corresponding to the first central pixel value with the absolute value of the second translation corresponding to the second central pixel value, the area that most needs to be optimized between the first central pixel value and the second central pixel value can be selected as the first representative pixel value, thereby improving the accuracy of the first representative pixel value.

[0188] Furthermore, each central pixel value has a correction translation amount and a preference translation amount. In order to determine the first representative pixel value and the translation amount corresponding to the first representative pixel value, in an optional embodiment, based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined, including:

[0189] When the absolute value of the first translation amount corresponding to the first central pixel value is equal to the absolute value of the first translation amount corresponding to the second central pixel value, and the absolute value of the second translation amount corresponding to the first central pixel value is equal to the absolute value of the second translation amount corresponding to the second central pixel value, determining a second proportion of the total number of first pixel values ​​occupied by the sum of statistical values ​​of the sliding window with the first central pixel value as the intermediate value;

[0190] Determine a first proportion of the sum of the statistical values ​​of the sliding window with the second central pixel value as the intermediate value to the total number of the first pixel values;

[0191] According to the magnitude relationship between the first proportion and the second proportion, a first representative pixel value is determined from the first central pixel value and the second central pixel value, and a translation amount corresponding to the first representative pixel value is determined.

[0192] It can be understood that when the absolute value of the first translation corresponding to the first central pixel value is equal to the absolute value of the first translation corresponding to the second central pixel value, the absolute value of the second translation corresponding to the first central pixel value is compared with the absolute value of the second translation corresponding to the second central pixel value. If they are equal, the sum of the statistical values ​​of the first sliding window with the first central pixel value as the intermediate value is calculated, that is, the value of the first sliding window with the first central pixel value as the intermediate value is determined, and the proportion of the value to the total number of first pixel values ​​is determined, which is recorded as the second proportion.

[0193] In a similar manner, determine a first proportion of the total number of first pixel values ​​occupied by the sum of the statistical values ​​of the sliding window with the second central pixel value as the intermediate value, and determine a first representative pixel value from the first central pixel value and the second central pixel value based on the size relationship between the first proportion and the second proportion, and determine the translation amount corresponding to the first representative pixel value.

[0194] In this way, by comparing the proportion of the value of the first sliding window whose first central pixel value is the middle value with the proportion of the value of the first sliding window whose first central pixel value is the middle value when the two translation amounts are equal, the first representative pixel value is selected. In this way, the area that most needs to be optimized between the first central pixel value and the second central pixel value can be selected as the first representative pixel value, thereby improving the accuracy of the first representative pixel value.

[0195] In order to determine the first representative pixel value and the translation amount corresponding to the first representative pixel value according to the size relationship between the first proportion and the second proportion, in an optional embodiment, according to the size relationship between the first proportion and the second proportion, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined, including:

[0196] When the second proportion is greater than the first proportion, the first central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the first central pixel value is determined as the translation amount corresponding to the first representative pixel value;

[0197] When the second proportion is smaller than the first proportion, the second central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the second central pixel value is determined as the translation amount corresponding to the first representative pixel value.

[0198] It can be understood that when the first proportion is compared with the second proportion, when the second proportion is greater than the first proportion, it means that the value of the first sliding window with the first central pixel value as the middle value is greater than the value of the first sliding window with the second central pixel value as the middle value, that is, the proportion of pixel values ​​in the first sliding window with the first central pixel value as the middle value is larger, which is the skin color area that most needs to be optimized, so the first central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the first central pixel value is determined as the translation amount corresponding to the first representative pixel value.

[0199] When the second proportion is smaller than the first proportion, it means that the value of the first sliding window with the first central pixel value as the middle value is smaller than the value of the first sliding window with the second central pixel value as the middle value, that is, the proportion of pixel values ​​in the second sliding window with the second central pixel value as the middle value is larger, which is the skin color area that most needs to be optimized. Therefore, the second central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the second central pixel value is determined as the translation amount corresponding to the first representative pixel value.

[0200] For the equal situation, the first central pixel value can be determined as the first representative pixel value, and the translation amount corresponding to the first central pixel value can be determined as the translation amount corresponding to the first representative pixel value. The second central pixel value can also be determined as the first representative pixel value, and the translation amount corresponding to the second central pixel value can be determined as the translation amount corresponding to the first representative pixel value. Here, the embodiment of the present application does not make specific limitations on this.

[0201] In this way, by further comparing the proportion of pixel values ​​in the sliding window with the center pixel value as the middle value, the first representative pixel value and the translation value corresponding to the first representative pixel value are determined, thereby improving the accuracy of the first representative pixel value and the translation value corresponding to the first representative pixel value, which helps to improve the skin color optimization effect.

[0202] In addition, in order to determine the translation amount corresponding to the first representative pixel value according to the first translation amount and the second translation amount, in an optional embodiment, determining the translation amount corresponding to the first representative pixel value according to the first translation amount and the second translation amount includes:

[0203] Obtaining hierarchical control coefficients;

[0204] Determining a weight of the first translation amount and a weight of the second translation amount according to the hierarchical control coefficient;

[0205] Based on the weight of the first translation amount and the weight of the second translation amount, a weighted sum is performed on the first translation amount and the second translation amount to obtain a translation amount corresponding to the first representative pixel value.

[0206] It is understandable that since the first representative pixel value only corresponds to an area of ​​the human face, if the entire skin color area uses the sum of the first translation amount and the second translation amount to process the second pixel value, the optimization of certain skin color areas will be ineffective. In order to further improve the skin color optimization effect, a hierarchical control method of the first translation amount and the second translation amount is adopted here to determine the translation amount corresponding to the first representative pixel value.

[0207] Among them, the hierarchical control coefficient is first obtained. The hierarchical control coefficient here can be a value pre-set by the user, or a value determined according to the image parameters of the image, or a value determined according to the image parameters of different areas in the image, or a value determined according to the region to which the portrait belongs, wherein the hierarchical control coefficient is a value between 0 and 1; here, the embodiment of the present application does not make any specific limitation on this.

[0208] After determining the hierarchical control coefficients, the weight of the first translation amount and the weight of the second translation amount may be determined using the hierarchical control coefficients, thereby performing a weighted summation of the first translation amount and the second translation amount to determine the translation amount corresponding to the first representative pixel value.

[0209] In this way, the weights of the first translation amount and the second translation amount are determined by the graded control coefficient, so that the translation amounts corresponding to the first representative pixel values ​​determined based on different graded control coefficients are different, which helps to achieve flexible processing of different skin color areas of the portrait and improves the effect of skin color optimization.

[0210] In order to determine the weight of the first translation and the weight of the second translation, in an optional embodiment, the weight of the first translation and the weight of the second translation are determined according to the hierarchical control coefficient, including:

[0211] When the hierarchical control coefficient is greater than or equal to zero and less than or equal to the preset first coefficient, the weight of the first translation is determined to be the ratio of the hierarchical control coefficient to the preset first coefficient, and the weight of the second translation is determined to be zero;

[0212] When the hierarchical control coefficient of the second translation is greater than the preset first coefficient and less than or equal to one, the weight of the first translation is determined to be one, and the weight of the second translation is determined to be the ratio of the hierarchical control coefficient to the preset second coefficient.

[0213] The preset second coefficient is equal to a difference between one and the preset first coefficient.

[0214] It can be understood that it is checked whether the hierarchical control coefficient is greater than or equal to 0 and less than or equal to the preset first coefficient. If it is, it means that the deflection correction and translation processing is required at this time, and the preference translation processing is not required. Therefore, the weight of the first translation amount is determined to be the ratio of the hierarchical control coefficient to the preset first coefficient, and the weight of the second translation amount is 0. Otherwise, it is checked whether the hierarchical control coefficient of the second translation amount is greater than the preset first coefficient and less than or equal to 1. If it is, it means that both the deflection correction and translation processing are required at this time, so the weight of the first translation amount is determined to be 1, and the weight of the second translation amount is the ratio of the hierarchical control coefficient to the preset second coefficient.

[0215] In this way, the translation amount corresponding to the obtained first representative pixel value is controlled in a hierarchical manner according to different hierarchical control coefficients, so that the skin color of the portrait can be optimized in a hierarchical manner, thereby improving the effect of skin color optimization.

[0216] Since the first representative pixel value of each portrait can be understood as only accurately representing the skin color area that needs to be optimized in the skin area of ​​the portrait, and cannot accurately represent other areas, such as the extremely dark area in the skin color area, makeup such as eye shadow and blush, and the area illuminated by another light source under dual light sources, etc., then, if the pixel value of the skin color area of ​​the portrait is processed according to the translation amount corresponding to the above first representative pixel value, there may be a problem of local negative optimization. In order to further improve the effect of skin color optimization, in an optional embodiment, the method further includes:

[0217] determining an adjustment weight of a third pixel value;

[0218] According to the adjustment weight, the translation amount corresponding to the second representative pixel value is adjusted to obtain the translation amount corresponding to the first representative pixel value again.

[0219] It can be understood that the adjustment weight of the third pixel value is determined, wherein the third pixel value is: the pixel value of the first area in the second color space. In other words, the adjustment weight is determined for the pixel value of the first area in the second color space, and the adjustment weight is used to adjust the translation amount corresponding to the first representative pixel value.

[0220] However, since the translation amount corresponding to the first representative pixel value is a vector in the first color space, it needs to be converted into a vector in the second color space. Therefore, the translation amount corresponding to the second representative pixel value needs to be adjusted according to the adjustment weight to regain the translation amount corresponding to the first representative pixel value.

[0221] The second representative pixel value is: the pixel value of the first representative pixel value in the second color space; the translation amount corresponding to the second representative pixel value is: the translation amount corresponding to the first representative pixel value converted to the translation amount in the second color space. That is to say, after determining the adjustment weight in the second color space, the translation amount corresponding to the second representative pixel value in the second color space needs to be adjusted to obtain the translation amount corresponding to the first representative pixel value again.

[0222] The first color space and the second color space are different color spaces.

[0223] In this way, by determining the adjustment weight of the pixel value in another color space, the translation amount corresponding to the first representative pixel value can be adjusted, thereby preventing negative optimization from occurring in skin color optimization and improving the effect of skin color optimization.

[0224] In order to determine the adjustment weight of the third pixel value, in an optional embodiment, determining the adjustment weight of the third pixel value includes:

[0225] Determine a translation amount corresponding to the second representative pixel value;

[0226] The adjustment weight is determined according to the degree of deviation, the preset fixed translation interval and the translation amount corresponding to the second representative pixel value.

[0227] It can be understood that in order to determine the adjustment weight of the third pixel value, it is necessary to convert the color space of the first representative pixel value and the translation amount corresponding to the first representative pixel value to the second color space, which corresponds to the second representative pixel value and the translation amount corresponding to the second representative pixel value.

[0228] In addition, a preset fixed translation interval is pre-stored locally, and its value is a vector in the second color space. Knowing the translation amount corresponding to the above preset fixed translation interval and the second representative pixel value, the adjustment weight can be determined together with the degree of deviation, wherein the degree of deviation is: the difference between the third pixel value and the second representative pixel value.

[0229] In practical applications, a clip function can be used to set the adjustment weight between 0 and 1 to determine the adjustment weight.

[0230] Among them, the first representative pixel value is the second component and the third component in the first color space, and the third pixel value mainly selects the second component and the third component in the second color space. For example, the first representative pixel value includes the ab component, and the third pixel value includes the CH component.

[0231] In this way, the adjustment weight is determined by the degree of deviation, the preset fixed translation interval and the translation amount corresponding to the second representative pixel value, so that the determined adjustment weight is related to the difference between the third pixel value and the second representative pixel value, and is related to the preset fixed translation interval. The appropriate adjustment weight can be determined to regain the translation amount corresponding to the first representative pixel value, thereby preventing negative optimization of skin color optimization.

[0232] In order to re-determine the translation amount corresponding to the first representative pixel value, in an optional embodiment, adjusting the translation amount corresponding to the second representative pixel value according to the adjustment weight to re-obtain the translation amount corresponding to the first representative pixel value includes:

[0233] According to the adjustment weight, adjusting the translation amount corresponding to the second representative pixel value to obtain an adjusted translation amount;

[0234] Adjusting the second representative pixel value according to the adjusted translation amount to obtain an adjusted value of the second representative pixel value;

[0235] The translation amount corresponding to the first representative pixel value is re-determined according to the adjustment value of the second representative pixel value and the first representative pixel value.

[0236] Understandably, the translation amount corresponding to the second representative pixel value is first adjusted according to the adjustment weight to obtain the adjusted translation amount. Here, the translation amount corresponding to the second representative pixel value is adjusted in the second color space, and then the second representative pixel value is adjusted according to the adjusted offset, so as to obtain the adjusted value of the second representative pixel value. Finally, the translation amount corresponding to the first representative pixel value is re-determined according to the adjusted value of the second representative pixel value and the first representative pixel value.

[0237] That is, when the adjusted value of the second representative pixel value in the second color space is known, the translation amount corresponding to the first representative pixel value can be re-determined based on the adjusted value of the second representative pixel value and the first representative pixel value.

[0238] In this way, by adjusting the second representative pixel value in the second color space, the adjusted value of the second representative pixel value is obtained, based on which the translation amount corresponding to the first representative pixel value can be adjusted, thereby preventing the occurrence of negative optimization of skin color.

[0239] In order to re-determine the translation amount corresponding to the first representative pixel value according to the adjustment value of the second representative pixel value and the first representative pixel value, in an optional embodiment, re-determining the translation amount corresponding to the first representative pixel value according to the adjustment value of the second representative pixel value and the first representative pixel value includes:

[0240] Converting the adjustment value of the second representative pixel value from the second color space to the first color space to obtain the adjustment value of the first representative pixel value;

[0241] The vector from the adjustment value of the first representative pixel value in the coordinate system formed by the components included in the central pixel value to the first representative pixel value is re-determined as the translation amount corresponding to the first representative pixel value.

[0242] It can be understood that the adjustment value of the second representative pixel value is first converted from the second color space to the first color space, and then converted to a value in the first color space, which is the adjustment value of the first representative pixel value. After knowing the adjustment value of the first representative pixel value, the vector from the adjustment value of the first representative pixel value to the first representative pixel value in the coordinate system composed of the components included in the central pixel value is used as the translation amount corresponding to the first representative pixel value, and the second pixel value is adjusted using this translation amount.

[0243] In this way, the translation amount corresponding to the first representative pixel value is adjusted, which can prevent the problem of negative optimization when optimizing the skin color area, and further improve the optimization effect of portrait skin color optimization.

[0244] The following is an example to describe the image processing method described in one or more of the above embodiments.

[0245] Figure 2 A schematic diagram of an example of an optional image processing system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the image processing system 200 may include: an imaging unit 21 , a processor 22 , a memory 23 and an input / output unit 24 .

[0246] Among them, the imaging unit 21 can output the image being captured to the processor 22. The memory 23 stores the captured images, which can be acquired and processed by the processor 22. The processor 22 usually includes one or more processors of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), and a neural network processor (NPU). The deep neural network model usually runs on a GPU or a DSP or an NPU processor. The input and output unit 24 includes but is not limited to a display, a touch screen, etc., and the user interacts through the input and output unit 24, including displaying images, etc.

[0247] Figure 3 A flowchart of an example of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the image processing method may include:

[0248] S301: Acquire the original image;

[0249] The portrait image to be processed for skin color optimization is obtained from the imaging unit 21 or the storage unit 23. The portrait image obtained from the storage unit 23 comes from, including but not limited to, image file storage from applications such as files and photo albums.

[0250] Here, a portrait image refers to an image with a person as the main subject. When acquired from the imaging unit 21, this instance belongs to the post-processing stage of the ISP link, and the output information of the ISP link pre-module can be obtained. Usually, relevant information can be obtained, including but not limited to face detection frame, face key points, face attributes, portrait skin mask (mask), portrait instance segmentation (mask), and other information output by the ISP pre-module. According to conditions such as whether there is a face and the proportion of the face area, it can be judged whether it is a portrait image. Acquired from the storage unit 23, it can usually read the above-mentioned stored relevant information to determine whether it is a portrait image.

[0251] If there is only an image but the relevant information is not sufficient to determine whether it is a portrait image, it is necessary to use algorithms such as face detection, face key points, portrait skin segmentation, and portrait instance segmentation to obtain relevant information and determine whether it is a portrait image.

[0252] S302: Calculate the skin (mask) of each person in the portrait image and the corresponding face area skin (mask);

[0253] Figure 4 A schematic diagram of an optional example of an image containing a portrait provided in an embodiment of the present application, such as Figure 4 As shown, the image includes two portraits, namely the first person and the second person. For each portrait, based on the facial key points corresponding to each person, the facial area skin (mask) corresponding to each person can be created to obtain the facial area skin mask of the first person and the facial area skin mask of the second person, that is, non-skin areas such as eyebrows, eyes, and lips are not included.

[0254] Based on the intersection of the face skin segmentation (mask) and the instance segmentation (mask) corresponding to each person, the first skin (mask) corresponding to each person can be calculated. The face area in the first skin (mask) corresponding to each person is replaced with the face area skin (mask) (equivalent to the second area) to obtain the skin (mask) corresponding to each person, which is Figure 4 The combination of the first person's skin mask and the second person's skin mask (equivalent to the first area) is used to improve the accuracy of the facial area on the first skin mask.

[0255] S303: Convert the pixel values ​​of each person's skin (mask) area to obtain Lab values;

[0256] The Lab value is equivalent to the first pixel value. The portrait image format in this example is not specifically limited, and is usually in RGB format or YUV format. In this example, the portrait image format is RGB format.

[0257] It is well known in the industry that Lab space has good linearity that matches the subjective perception of the human eye, which is conducive to high-precision skin color adjustment. The Lab value of the Lab space, where L represents brightness, the L value range is [0, 100], a represents the position between green and red, the a value range is [-128, 127], and b represents the position from blue to yellow, the b value range is [-128, 127]. This example does not limit the data type and number of bits of the Lab value, which can be an integer type (such as INT16, INT8, etc.) or a floating point type. This example can use a 32-bit floating point type.

[0258] In this example, the pixel values ​​of each person's skin (mask) area are converted into Lab values ​​in the Lab space according to the conversion formula.

[0259] S304: Performing histogram statistics and processing on the Lab value area of ​​the skin (mask) in the face area of ​​each person, and calculating the first skin color center and the second skin color center;

[0260] The first skin color center corresponds to the first center pixel value, and the second skin color center corresponds to the second center pixel value.

[0261] The histogram of L values ​​is calculated using a histogram. The number of groups (bins) is 101. The boundary of the first group is [0, 1), the boundary of the second group is [1, 2), and so on. The boundary of the 100th group is set to [99, 100), and the boundary of the 101st group is [100, 101).

[0262] The sliding window summation is performed for the L value histogram. The size of the sliding window for summing the L histogram is one of the hyperparameters of this example. It should be able to better represent the range of the regional skin, including the brightness range of the subject skin area in the non-mixed light scene and the brightness range of each sub-subject area corresponding to the mixed light scene. This example does not impose specific restrictions on this.

[0263] In this example, the sliding window size is set to 21, and based on the sliding window summation result, the L maximum sliding window summation value Max(filter(L_hist)) and its middle value center are obtained. Max(filter(L_hist)) , and get satisfaction and center Max(filter(L_hist)) Distance greater than The maximum value under the condition is taken as the second largest sliding window sum of L sec_Max(filter(L_hist)) and its middle value center sec_Max(filter(L_hist)) .

[0264] Max(filter(L_hist)), center sec_Max(filter(L_hist)) The summed value represents the area corresponding to the brightness range. Normalization (i.e. multiply by ) to obtain the proportion of the area of ​​the corresponding sliding window to the overall face area, that is, the proportion of the sum of the maximum sliding windows of L L The proportion of the second largest sliding window sum value

[0265] For the skin (mask) area of ​​the face area and the brightness meets the threshold ultra_low <L<threshold ultra_high The area, where threshold ultra_low is the lower limit of the brightness range, threshold ultra_high The upper limit of the brightness range is calculated by using the histogram calculation method to calculate the two-dimensional histogram ab_hist of the ab value, and its shape is 256*256. ab [0,0] represents the a value [-128, -127) and the b value [-128, -127). ab [0,1] represents the a value [-128, -127) and the b value [-127, -126), and so on. ab [255,255] represents the a value [127,128) and the b value [127,128).

[0266] Where threshold ultra_low , is one of the hyperparameters of this example, indicating that the corresponding ab value is unstable when the brightness is lower than this value. This example does not make any specific restrictions, and this value is set to 10. ultra_high , is one of the hyperparameters of this example, indicating that the corresponding ab value is unstable when the brightness is higher than this value. This example does not make any specific limitation. This value is set to 9 in the present invention.

[0267] Slide window sum for ab_hist, sum window size filter_size ab_hist It is one of the hyperparameters of this example, and should be able to better represent the range of regional skin, including the ab range of the main skin area in the non-mixed light scene and the ab range of each sub-subject area corresponding to the mixed light scene. ab_hist Without specific limitation, this example is set to (5, 5).

[0268] According to the sliding window summation result, obtain the maximum sliding window sum value of a*b* Max(filter(ab_hist)) and its center value center Max(filter(ab_hist)) And the corresponding face skin (mask) area face_mask Max(filter(ab_hist)) . And get satisfaction and center Max(filter(ab_hist)) The distance is greater than The maximum value under the conditions is taken as the second largest sliding window sum of a*b*sec_Max(filter(ab_hist)) and its center value center sec_Max(filter(ab_hist)) And the corresponding facial skin (mask) area.

[0269] The sum of a*b*Max(filter(ab_hist)) and a*b*Sec_Max(filter(ab_hist)) represents the area corresponding to the color range. Normalization (i.e. multiply by ) to obtain the ratio of the area of ​​the corresponding sliding window to the overall face area, that is, the ratio of a*b*the sum of the maximum sliding window values a*b*The second largest sliding window sum value ratio

[0270] S305: Setting skin color standard ab plane ellipses of different regions / different color temperatures / different brightness;

[0271] Among them, there is a "Da Vinci Skin Color Indicator" for skin color. This indicator is expressed in the HSV space, indicating that normal or preferred skin color can be defined within a standard range.

[0272] In this example, different ellipses are set in the ab plane of the Lab space according to different regions / different color temperatures / different brightness. The points inside the ellipse represent normal skin color, the range of the ellipse represents the normal skin color range, and the points outside the ellipse represent skin color cast. The favorite center is set inside the ellipse to represent the most preferred skin color.

[0273] Targeting different regions, that is, users in different regions / countries around the world have different aesthetic tastes, which leads to different definitions of normal skin color ranges and preference centers.

[0274] Different color temperatures refer to the color temperature of the face area, that is, due to people's color constancy and pursuit of atmosphere under different color temperatures, the normal skin color range and preference center are defined differently.

[0275] For different brightness, it refers to the average brightness of the face, that is, due to different requirements of people on skin color saturation under different brightness, the normal skin color range and preference center are defined differently.

[0276] It can be further expanded to different genders, ages, etc., and this example does not limit it.

[0277] In this example, an ellipse and a preferred center are set, and based on the skin color rating of a large number of portrait images in the region / color temperature / region (for example, China, color temperature 2700k-3500k, L brightness 50), an ellipse and a preferred center within the ellipse can be obtained according to the evaluation results. The specific statistical scoring method is not described in detail in this example.

[0278] Figure 5 A schematic diagram of an example of an optional standard area provided in an embodiment of the present application, such as Figure 5 As shown, the X-axis of the two-dimensional coordinate system is the a value, the Y-axis is the b value, the ellipse is equivalent to the above-mentioned standard area, and the preference center is the preference center of the standard area.

[0279] S306: Determine the number of light sources of each person's face based on the histogram processing information of the Lab value of the skin (mask) area of ​​the face area of ​​each person;

[0280] Based on the sum of a*b*the second largest sliding window Calculate the weight of the dual light source scene. This example does not limit the specific weight calculation method. This example can use the following methods:

[0281] if That is, the weight of the dual-light source scene is 1, and the weight of the non-dual-light source scene is 0.

[0282] If the dual light source scene is 1, the first skin color center needs to be calculated separately Max(filter(ab_hist)) and the second skin color center sec_Max(filter(ab_hist)) The desired translation adjustment.

[0283] If processed as a single light source, the color temperature of the face can usually be obtained from the imaging link or storage unit to obtain the face temperature (face CCT) output by the ISP AWB module. Max(filter(ab_hist)) The corresponding face skin (mask) area face_mask Max(filter(ab_hist)) According to the region, the face temperature and the face brightness, select the corresponding ellipse for subsequent processing.

[0284] If the dual light source is used for processing, the color temperature values ​​of the first skin color center and the second skin color center must be calculated separately. This example does not impose any specific restrictions on the calculation of the color temperature values ​​of the two centers. This example can use the following method:

[0285] First, the face temperature (face CCT) of the human face output by the AWB module is obtained. According to the distance between the first skin color center and the second skin color center in the ab plane, the color temperature difference ΔCCT between the first skin color center and the second skin color center is estimated. Max_vs_secMax This example does not apply to ΔCCT Max_vs_secMax To make specific restrictions, a table lookup method based on the characteristics of each device can be used. This example uses a unified empirical value ΔCCT Max_vs_secMax =2000K.

[0286] In the case of two light sources, the skin color center corresponding to the low color temperature light source is more "warm and yellow" than the skin color center corresponding to the high color temperature. In this example, the skin color center with a larger value (usually a*>0) is calculated on the ab plane, and the color temperature of the skin color center with a larger hue angle (i.e., more yellowish) is higher in the first quadrant of the ab plane. If the color temperature of the first largest skin color center is greater than that of the second largest skin color center, the color temperature value corresponding to the first largest skin color center is calculated as follows:

[0287]

[0288] The color temperature value corresponding to the second largest skin color center is calculated as follows:

[0289]

[0290] The corresponding face brightness can be calculated separately with the single light source processing. Then the first skin color center and the second skin color center can select the corresponding ellipse and the preferred center in the ellipse according to their respective color temperature and brightness.

[0291] S307: For the first skin color center or the second skin color center, according to the set preference tendency classification control strategy, calculate the corresponding a*b* translation amount, and determine the representative skin color and its a*b* translation amount;

[0292] Among them, for the first skin color center or the second skin color center, the first skin color center deviation correction a*b* translation amount is calculated respectively according to the corresponding ellipse and the preferred center in the ellipse (equivalent to the first translation corresponding to the first central pixel value), the second skin color center correction a*b* translation (equivalent to the second translation amount corresponding to the first central pixel value) and the first skin color center preference a*b* translation amount (equivalent to the first translation amount corresponding to the second central pixel value) Second skin color center preference a*b* shift (equivalent to the second translation amount corresponding to the above-mentioned second central pixel value).

[0293] This example does not impose any specific restrictions on the above translation amount. This example adopts the following method:

[0294] If the skin color center is already within the ellipse, the corresponding or If the center of the skin color is outside the ellipse, then or It is equal to the translation from the skin color center to the intersection point, where the intersection point is the intersection of the line from the skin color center to the preference center in the ellipse and the ellipse. or It is the amount of translation from the center of the skin color to the preferred center within the ellipse after the correction translation.

[0295] Figure 6 A schematic diagram of an example of an optional standard area translation amount provided in an embodiment of the present application, such as Figure 6 As shown, is the correction translation of the first skin color center, is the correction translation of the second skin color center, is the preferred translation amount for the center of the first skin color, The preferred translation amount for the center of the second skin color.

[0296] In order to determine the representative skin color (equivalent to the first representative pixel value), determine the representative skin color center represent , if a single light source scene is processed, the representative skin color is the first skin color center.

[0297] Determine the representative skin color center represent , if it is processed as a dual light source scene, by comparing and The absolute value of is determined, and the skin color center with the largest absolute value is selected as the representative skin color. By comparing and The skin color center with the largest absolute value is selected as the representative skin color.

[0298] Determine the representative skin color center represent Corresponding correction translation and favorite shift amount / favorite tendency to adjust the shift amount That is, the corresponding translation amount of the skin color.

[0299] That is, the basis for determining the representative skin color can be understood as the importance or urgency of the corresponding skin color center to be enhanced and optimized. In the common dual-light source mixed light environment, the skin color area to be optimized can be accurately selected, and the selection of the skin color area that is already good can be avoided, resulting in inadequate optimization.

[0300] Since there are multiple options for implementing the translation amount and translation path based on the ellipse in the ab plane, this example does not impose specific restrictions on the hierarchical control strategy based on the preference tendency of the ellipse.

[0301] This example can be used as follows: and Set the control strategy, that is, set the preference coefficient λ like (equivalent to the above-mentioned preset first coefficient), whose range is [0, 1], when λ like ≥0.6 means entering the preference adjustment from the correction. The a*b* shift amount corresponding to the final processing of the skin color is calculated as follows:

[0302]

[0303] Hierarchical control is reflected in First, the deviation correction is ensured, and then the degree of adjustment of preference is controlled. A clear grading logic is achieved. At the same time, based on different ellipses for different regions / color temperatures / brightness, as well as a representative skin color mechanism for dual light source (expandable multi-light source) environments, coverage of various light source environments is achieved.

[0304] S308: According to each person's representative skin color and its corresponding final processed a*b* translation amount, the pixel-by-pixel adjustment weight calculation of each person's skin (mask) area is calculated to complete the skin color optimization of each person.

[0305] Each person's representative skin color center represent , which can be understood as only accurately representing the skin (mask) area of ​​the person that needs to be optimized, and cannot accurately represent other areas, including but not limited to extremely dark areas, eye shadow and blush makeup, areas illuminated by another light source under dual light sources, and other areas that represent the center of the skin color. represent Corresponding face_mask represent Other skin colors.

[0306] Therefore, if all pixels in the skin (mask) area are shifted by a*b* Performing uniform translation will lead to local negative optimization. Calculate the pixel-by-pixel adjustment weights of the skin (mask) area to optimize each person's skin color.

[0307] Since the calculation method of adjusting the weight of each pixel can be various, the present invention does not make any specific limitation. The present invention requires that based on the above representative skin color center represent The translation corresponding to the skin color The design is based on the principle of avoiding local negative optimization while ensuring full and accurate optimization of the optimization area.

[0308] This example uses the following method. represent , The representative skin color (C, H) in the LCH color space can be calculated represent (equivalent to the second representative pixel value mentioned above), the translation amount corresponding to the representative skin color in the LCH color space (equivalent to the translation amount corresponding to the second representative pixel value mentioned above), where H and C are the hue (Hue) and saturation (Chroma) of the LCH space, That is, the corresponding changes in C and H. For any pixel in the skin (mask) area of ​​each person (corresponding to the person) i,j , calculate the LCH space

[0309] Calculate pixels independently i,j Hue adjustment weight and saturation adjustment weight Set the hyperparameter fixed translation interval (ΔC, ΔH) whole_equal_shift (equivalent to the above preset fixed translation interval) to ensure the center represent All pixels in the subject skin color area are shifted by the same amount to ensure that the portrait is not distorted. and Guarantee center represent Pixels outside the skin color area of ​​the subject are properly processed to avoid negative optimization of the skin color, avoid causing skin color discontinuities, and make the skin color more uniform. and The calculation of is as follows:

[0310]

[0311] The above weight formula, if the pixel In center represent and (ΔC, ΔH) whole_equal_shift Defined equal translation interval, then and It is 1.0, and gradually decreases to 0.0 after exceeding the equal translation range.

[0312] The final a*b* translation of the pixel As shown in formula (6), all translations are based on center represent Perform the calculation:

[0313]

[0314] based on Complete pixel-by-pixel weighted adjustment of each person's skin mask area to optimize each person's skin color.

[0315] For the above example, you can also make the following extensions:

[0316] Extension 1: The calculation of skin color and translation is not limited to Lab space

[0317] This example adopts a specific implementation method based on the Lab color space. The "determining the number of facial light sources, selecting the representative skin color to be optimized", the pixel-by-pixel weight adjustment strategy based on the representative skin color and translation amount, and the preference classification strategy described in this example can also be applied to HSV, LCH, sRGB and other spaces, and this example does not limit them.

[0318] Extension 2: Calculation of skin color and translation amount can be accelerated by image downsampling and other processing

[0319] In this example, the representative skin color and translation amount can be calculated by first downsampling the facial skin area image to accelerate the calculation.

[0320] Extension 3: In this example, for each face, you can also set a strategy to selectively optimize the faces in the image, including but not limited to the first and third faces in the face area ranking.

[0321] In summary, this example proposes a portrait skin color optimization method based on zoning adjustment of representative skin colors and hierarchical control of preference tendency adjustment. It is proposed to comprehensively calculate weights based on histogram area ratio, color temperature, brightness, required a*b* shift, etc., to judge whether a single light source is dual light sources or multiple light sources, select the representative skin color to be optimized, and accurately locate the skin color area that needs the most optimization. The pixel-by-pixel adjustment weight is calculated based on the representative skin color value and the a*b* shift, maintaining sufficient integrity of the skin color adjustment, while avoiding negative optimization of local skin color areas. Based on the skin color ellipse and preference center in the a*b* plane under different regions / color temperatures / brightness conditions, a preference tendency adjustment hierarchical control algorithm is set, which can not only adapt to various light source environments for accurate skin color correction control, but also perform clear skin color preference tendency adjustment control.

[0322] This example proposes a portrait skin color optimization method based on zoning adjustment of representative skin colors and hierarchical control of preference tendency adjustment. The present invention uses portrait skin mask information to adjust only the skin area of ​​the portrait. This method does not limit the color space, and this method specifically adopts the adjustment after switching to the Lab space known in the industry. This method calculates the histogram of the Lab value of the facial skin area, proposes a comprehensive calculation weight based on the histogram area ratio, color temperature, brightness, required a*b* translation, etc., judges a single light source or a dual light source or multiple light sources, selects the representative skin color to be optimized, and accurately locates the skin color area that needs to be optimized the most. The pixel-by-pixel adjustment weight is calculated based on the representative skin color value and the a*b* translation, maintaining sufficient skin color adjustment integrity, while avoiding negative optimization of local skin color areas. Based on the skin color ellipse and preference center in the a*b* plane under different regions / color temperatures / brightness conditions, a preference tendency adjustment hierarchical control algorithm is set, which can not only adapt to various light source environments for accurate skin color correction control, but also can perform clear skin color preference tendency adjustment control.

[0323] The embodiment of the present application provides an image processing method, obtaining a first pixel value of the image, the first pixel value is: the pixel value of the second area in the first color space, the first area is the skin color area of ​​the portrait in the image, and the second area is the skin color area of ​​the face of the portrait in the first area, based on the distribution of the first pixel value, determining the central pixel value from the first pixel value, determining the standard area of ​​the central pixel value, based on the central pixel value and the standard area, the standard area is an area in a coordinate system composed of components included in the central pixel value, determining the first representative pixel value of the second area from the central pixel value, and determining the translation amount corresponding to the first representative pixel value, adjusting the second pixel value according to the translation amount corresponding to the first representative pixel value, obtaining and outputting a processed image, the second pixel value is: the first area in the first Pixel values ​​in color space; that is, in the embodiment of the present application, the distribution of pixel values ​​in the above-mentioned facial skin color area is used to determine the central pixel value, the standard area of ​​the central pixel value is determined, and then the first representative pixel value can be determined based on the central pixel value and its standard area, so that the determined first representative pixel value can better represent the area in the image where the facial skin color needs to be optimized, and the translation amount corresponding to the first representative pixel value is determined, and an adjustment amount can be determined for the first representative pixel value, and then the portrait skin color area is adjusted based on the translation amount corresponding to the first representative pixel value. Since the determined first representative central pixel value can represent the area in the image where the facial skin color needs to be optimized, the translation amount corresponding to the first representative pixel value makes the adjustment of the portrait skin color area more accurate, thereby improving the effect of portrait skin color optimization.

[0324] Based on the same inventive concept as the above-mentioned embodiment, the embodiment of the present application provides an image processing device, Figure 7 A schematic diagram of the structure of an optional image processing device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the image processing device includes: an acquisition module 71, a first determination module 72, a second determination module 73, a third determination module 74 and an output module 75; wherein,

[0325] The acquisition module 71 is used to acquire a first pixel value of the image; wherein the first pixel value is: a pixel value of the second area in the first area in the first color space; the first area is a skin color area of ​​the portrait in the image; the second area is a skin color area of ​​the face of the portrait in the first area;

[0326] A first determination module 72, configured to determine a central pixel value from the first pixel values ​​based on a distribution of the first pixel values;

[0327] A second determination module 73 is used to determine a standard area of ​​the central pixel value; wherein the standard area is an area in a coordinate system formed by components included in the central pixel value;

[0328] A third determination module 74 is used to determine a first representative pixel value of the second area from the central pixel value based on the central pixel value and the standard area, and determine a translation amount corresponding to the first representative pixel value;

[0329] The output module 75 is used to adjust the second pixel value according to the translation amount corresponding to the first representative pixel value, and obtain and output the processed image; wherein the second pixel value is: the pixel value of the first area in the first color space.

[0330] In an optional embodiment, the first determination module 72 is specifically used to: perform statistics on the distribution of the first pixel value to obtain a statistical result; slide on the pixel value of the statistical result according to a preset sliding window and a preset sliding step to obtain a first sliding window; select a sliding window from the first sliding window based on the value of the first sliding window, and determine the central pixel value from the pixel values ​​of the selected sliding window; wherein the value of the first sliding window is the sum of the statistical values ​​of the first sliding window.

[0331] In an optional embodiment, the first determination module 72 performs statistics on the distribution of the first pixel value to obtain the statistical result, including: selecting an area in the second area where the first component of the first pixel value falls within a preset component interval;

[0332] Statistics are performed on the distribution of the second component of the first pixel value of the selected area and the third component of the first pixel value of the selected area to obtain statistical results.

[0333] In an optional embodiment, the first determination module 72 selects a sliding window from the first sliding window based on the value of the first sliding window, and determines the central pixel value from the pixel values ​​of the selected sliding window, including: selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining the first central pixel value from the pixel values ​​of the selected sliding window; determining the statistical results except the preset pixel value interval in the statistical results as partial statistical results; wherein the preset pixel value interval is: an interval of the preset pixel value floating up and down based on the first central pixel value; sliding on the pixel values ​​of the partial statistical results according to the preset sliding window and the preset sliding step to obtain a second sliding window; selecting a sliding window from the second sliding window based on the value of the second sliding window, and determining the second central pixel value from the pixel values ​​of the selected sliding window; wherein the value of the second sliding window is the sum of the statistical values ​​of the second sliding window; and determining the first central pixel value and the second central pixel value as the central pixel value.

[0334] In an optional embodiment, the first determination module 72 selects a sliding window from the first sliding window based on the value of the first sliding window, and determines the central pixel value from the pixel values ​​of the selected sliding window, including: determining the middle pixel value of the sliding window corresponding to the maximum value in the value of the first sliding window as the central pixel value.

[0335] In an optional embodiment, the first determination module 72 selects a sliding window from the first sliding window based on the value of the first sliding window, and determines the first center pixel value from the pixel values ​​of the selected sliding window, including: determining the middle pixel value of the sliding window corresponding to the maximum value in the value of the first sliding window as the first center pixel value; the first determination module 72 selects a sliding window from the second sliding window based on the value of the second sliding window, and determines the second center pixel value from the pixel values ​​of the selected sliding window, including: determining the middle pixel value of the sliding window corresponding to the maximum value in the value of the second sliding window as the second center pixel value.

[0336] In an optional embodiment, the first pixel value includes one or more of the following: a first component of the first pixel value, a second component of the first pixel value, and a third component of the first pixel value.

[0337] In an optional embodiment, the second determination module 73 is specifically configured to determine a standard area of ​​the central pixel value according to the region to which the portrait in the second area belongs and / or the image parameters corresponding to the central pixel value.

[0338] In an optional embodiment, the second determination module 73 determines the standard area of ​​the central pixel value according to the area to which the portrait of the second area belongs and / or the image parameters corresponding to the central pixel value, including: determining the number of light sources in the second area; based on the number of light sources in the second area, determining the standard area of ​​the central pixel value according to the area to which the portrait of the second area belongs and / or the image parameters corresponding to the central pixel value.

[0339] In an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value, and the second determination module 73 determines the number of light sources in the second area, including: determining a first proportion of the total number of first pixel values ​​occupied by the sum of statistical values ​​of a sliding window with the second central pixel value as the intermediate value; when the first proportion is less than a preset proportion, determining the number of light sources to be one; when the first proportion is greater than or equal to the preset proportion, determining the number of light sources to be two.

[0340] In an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value; the second determination module 73 determines the standard area of ​​the central pixel value based on the number of light sources in the second area, according to the area to which the portrait of the second area belongs and / or the image parameters corresponding to the central pixel value, including: when the number of light sources in the second area is one, determining the standard area of ​​the central pixel value according to the area to which the portrait of the second area belongs, the color temperature value of the second area and the brightness value corresponding to the first central pixel value; wherein the brightness value corresponding to the first central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the first central pixel value as the middle value.

[0341] In an optional embodiment, the central pixel value includes: a first central pixel value and a second central pixel value; the second determination module 73 determines the standard area of ​​the central pixel value based on the number of light sources in the second area, according to the area to which the portrait of the second area belongs and / or the image parameters corresponding to the central pixel value, including: when the number of light sources in the second area is two, the first standard area of ​​the first central pixel value is determined according to the area to which the portrait of the second area belongs, the color temperature value corresponding to the first central pixel value and the brightness value corresponding to the first central pixel value; wherein the brightness value corresponding to the first central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the first central pixel value as the middle value; the second standard area of ​​the second central pixel value is determined according to the area to which the portrait of the second area belongs, the color temperature value corresponding to the first central pixel value and the brightness value corresponding to the second central pixel value; wherein the brightness value corresponding to the second central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the second central pixel value as the middle value; the first standard area of ​​the first central pixel value and the second standard area of ​​the second central pixel value are determined as the standard areas of the central pixel value.

[0342] In an optional embodiment, the device is also used to: determine the sum of the color temperature value of the second area and the preset color temperature value as the color temperature value corresponding to the first central pixel value; and determine the difference between the color temperature value of the second area and the preset color temperature value as the color temperature value corresponding to the second central pixel value.

[0343] In an optional embodiment, the third determination module 74 is specifically used to: when the number of light sources in the second area is one, determine the first central pixel value as the first representative pixel value; when the first representative pixel value is within the range of the standard area, determine the translation amount corresponding to the first representative pixel value to be zero; when the first representative pixel value is outside the range of the standard area, determine the translation amount corresponding to the first representative pixel value based on the first vector in the coordinate system formed by the components included in the central pixel value; wherein the first vector is: a vector from the first representative pixel value to the preferred center point of the standard area.

[0344] In an optional embodiment, the third determination module 74 determines the translation amount corresponding to the first representative pixel value based on the first vector in the coordinate system formed by the components included in the center pixel value, including: determining the second vector in the coordinate system as the translation amount corresponding to the first representative pixel value; wherein the second vector is: the vector from the first representative pixel value to the intersection point; the intersection point is: the intersection point of the line connecting the first representative pixel value and the preferred center point of the standard area and the boundary of the standard area.

[0345] In an optional embodiment, the third determination module 74 determines the translation amount of the first representative pixel value based on the first vector in the coordinate system formed by the components included in the center pixel value, including: determining the second vector in the coordinate system as the first translation amount corresponding to the first representative pixel value; wherein the second vector is: the vector from the first representative pixel value to the intersection point; the intersection point is: the intersection point of the line connecting the first representative pixel value and the preference center point of the standard area and the boundary of the standard area; determining the third vector in the coordinate system as the second translation amount corresponding to the first representative pixel value; wherein the third vector is: the vector from the intersection point to the preference center point of the standard area; determining the translation amount corresponding to the first representative pixel value based on the first translation amount and the second translation amount.

[0346] In an optional embodiment, the third determination module 74 determines the translation amount corresponding to the first representative pixel value based on the first translation amount and the second translation amount, including: determining the sum of the first translation amount and the second translation amount as the translation amount corresponding to the first representative pixel value.

[0347] In an optional embodiment, the third determination module 74 is specifically used to: when the number of light sources in the second area is two, determine the translation amount corresponding to the first central pixel value according to the first central pixel value and the first standard area of ​​the first central pixel value; determine the translation amount corresponding to the second central pixel value according to the second central pixel value and the second standard area of ​​the second central pixel value; determine the first representative pixel value from the first central pixel value and the second central pixel value based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, and determine the translation amount corresponding to the first representative pixel value.

[0348] In an optional embodiment, the third determination module 74 determines the first representative pixel value from the first central pixel value and the second central pixel value based on the translation corresponding to the first central pixel value and the translation corresponding to the second central pixel value, and determines the translation corresponding to the first representative pixel value, including: selecting the maximum value of the absolute value of the translation corresponding to the first central pixel value and the absolute value of the translation corresponding to the second central pixel value; determining the central pixel value corresponding to the maximum value as the first representative pixel value; and determining the translation corresponding to the maximum value as the translation corresponding to the first representative pixel value.

[0349] In an optional embodiment, the translation amount corresponding to the first central pixel value includes: a first translation amount corresponding to the first central pixel value and a second translation amount corresponding to the first central pixel value; the translation amount corresponding to the second central pixel value includes: a first translation amount corresponding to the second central pixel value and a second translation amount corresponding to the second central pixel value.

[0350] In an optional embodiment, the third determination module 74 determines the first representative pixel value from the first central pixel value and the second central pixel value based on the translation corresponding to the first central pixel value and the translation corresponding to the second central pixel value, and determines the translation corresponding to the first representative pixel value, including: when the absolute value of the first translation corresponding to the first central pixel value is equal to the absolute value of the first translation corresponding to the second central pixel value, selecting the maximum value between the absolute value of the second translation corresponding to the first central pixel value and the absolute value of the second translation corresponding to the second central pixel value; determining the central pixel value corresponding to the maximum value as the first representative pixel value; and determining the translation corresponding to the maximum value as the translation corresponding to the first representative pixel value.

[0351] In an optional embodiment, the third determination module 74 determines the first representative pixel value from the first central pixel value and the second central pixel value based on the translation corresponding to the first central pixel value and the translation corresponding to the second central pixel value, and determines the translation corresponding to the first representative pixel value, including: when the absolute value of the first translation corresponding to the first central pixel value is equal to the absolute value of the first translation corresponding to the second central pixel value, and the absolute value of the second translation corresponding to the first central pixel value is equal to the absolute value of the second translation corresponding to the second central pixel value, determining a second proportion of the total number of first pixel values ​​occupied by the sum of the statistical values ​​of the sliding window with the first central pixel value as the intermediate value; determining a first proportion of the total number of first pixel values ​​occupied by the sum of the statistical values ​​of the sliding window with the second central pixel value as the intermediate value; and determining the first representative pixel value from the first central pixel value and the second central pixel value according to the relationship between the first proportion and the second proportion, and determining the translation corresponding to the first representative pixel value.

[0352] In an optional embodiment, the third determination module 74 determines the first representative pixel value from the first central pixel value and the second central pixel value according to the size relationship between the first proportion and the second proportion, and determines the translation amount corresponding to the first representative pixel value, including: when the second proportion is greater than the first proportion, the first central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the first central pixel value is determined as the translation amount corresponding to the first representative pixel value; when the second proportion is less than the first proportion, the second central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the second central pixel value is determined as the translation amount corresponding to the first representative pixel value.

[0353] In an optional embodiment, the third determination module 74 determines the translation amount corresponding to the first representative pixel value based on the first translation amount and the second translation amount, including: obtaining a hierarchical control coefficient; determining the weight of the first translation amount and the weight of the second translation amount based on the hierarchical control coefficient; and performing weighted summation of the first translation amount and the second translation amount based on the weight of the first translation amount and the weight of the second translation amount to obtain the translation amount corresponding to the first representative pixel value.

[0354] In an optional embodiment, the third determination module 74 determines the weight of the first translation and the weight of the second translation according to the hierarchical control coefficient, including: when the hierarchical control coefficient is greater than or equal to zero and less than or equal to the preset first coefficient, determining the weight of the first translation as the ratio of the hierarchical control coefficient to the preset first coefficient, and determining the weight of the second translation as zero; when the hierarchical control coefficient of the second translation is greater than the preset first coefficient and less than or equal to one, determining the weight of the first translation as one, and determining the weight of the second translation as the ratio of the hierarchical control coefficient to the preset second coefficient; wherein the preset second coefficient is equal to the difference between one and the preset first coefficient.

[0355] In an optional embodiment, the device is also used to: determine an adjustment weight of a third pixel value; wherein the third pixel value is: the pixel value of the first area in the second color space; according to the adjustment weight, adjust the translation amount corresponding to the second representative pixel value to regain the translation amount corresponding to the first representative pixel value; wherein the second representative pixel value is: the pixel value of the first representative pixel value in the second color space; the translation amount corresponding to the second representative pixel value is: converting the translation amount corresponding to the first representative pixel value to the translation amount in the second color space.

[0356] In an optional embodiment, the device determines the adjustment weight of the third pixel value, including: determining the translation amount corresponding to the second representative pixel value; determining the adjustment weight according to the degree of deviation, a preset fixed translation interval and the translation amount corresponding to the second representative pixel value; wherein the degree of deviation is: the difference between the third pixel value and the second representative pixel value.

[0357] In an optional embodiment, the device adjusts the translation amount corresponding to the second representative pixel value according to the adjustment weight to regain the translation amount corresponding to the first representative pixel value, including: adjusting the translation amount corresponding to the second representative pixel value according to the adjustment weight to regain the adjusted translation amount; adjusting the second representative pixel value according to the adjusted translation amount to regain the adjusted value of the second representative pixel value; and re-determining the translation amount corresponding to the first representative pixel value according to the adjusted value of the second representative pixel value and the first representative pixel value.

[0358] In an optional embodiment, the device redetermines the translation amount corresponding to the first representative pixel value based on the adjustment value of the second representative pixel value and the first representative pixel value, including: converting the adjustment value of the second representative pixel value from the second color space to the first color space to obtain the adjustment value of the first representative pixel value; and redetermining the translation amount corresponding to the first representative pixel value from the adjustment value of the first representative pixel value to the vector of the first representative pixel value in the coordinate system formed by the components included in the central pixel value.

[0359] In practical applications, the acquisition module 71, the first determination module 72, the second determination module 73, the third determination module 74 and the output module 75 can be implemented by a processor located on the image processing device, specifically a CPU, a microprocessor (Microprocessor Unit, MPU), a digital signal processor (Digital Signal Processing, DSP) or a field programmable gate array (Field Programmable Gate Array, FPGA) and the like.

[0360] Figure 8 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application, such as Figure 8 As shown, an embodiment of the present application provides an electronic device 800, including:

[0361] A processor 81 and a storage medium 82 storing instructions executable by the processor; the storage medium 82 relies on the processor 81 to perform operations through a communication bus 83, and when the instructions are executed by the processor, the image processing method executed by the processor side in one or more of the above embodiments is executed.

[0362] It should be noted that in actual application, the various components in the computer device are coupled together through the communication bus 83. It is understandable that the communication bus 83 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 83 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as communication buses 83.

[0363] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0364] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories.

[0365] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0366] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0367] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0368] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0369] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A method for processing an image, characterized in that: include: Obtain a first pixel value of the image; wherein the first pixel value is: a pixel value of a second area in the first area in a first color space; the first area is a skin color area of ​​a portrait in the image; the second area is a skin color area of ​​a face of the portrait in the first area; determining a central pixel value from the first pixel values ​​based on a distribution of the first pixel values; Determine a standard area of ​​the central pixel value; wherein the standard area is an area in a coordinate system formed by components included in the central pixel value; Based on the central pixel value and the standard area, determining a first representative pixel value of the second area from the central pixel value, and determining a translation amount corresponding to the first representative pixel value; According to the translation amount corresponding to the first representative pixel value, the second pixel value is adjusted to obtain and output a processed image; wherein the second pixel value is: the pixel value of the first area in the first color space.

2. The method according to claim 1, characterized in that The determining a central pixel value from the first pixel values ​​based on the distribution of the first pixel values ​​comprises: Performing statistics on the distribution of the first pixel values ​​to obtain statistical results; Sliding on the pixel value of the statistical result according to a preset sliding window and a preset sliding step size to obtain a first sliding window; A sliding window is selected from the first sliding window based on the value of the first sliding window, and the central pixel value is determined from the pixel values ​​of the selected sliding window; wherein the value of the first sliding window is the sum of the statistical values ​​of the first sliding window.

3. The method according to claim 2, characterized in that Performing statistics on the distribution of the first pixel value to obtain statistical results includes: Selecting from the second area an area where the first component of the first pixel value falls within a preset component interval; Statistics are performed on the distribution of the second component of the first pixel value of the selected area and the third component of the first pixel value of the selected area to obtain the statistical result.

4. The method according to claim 2, characterized in that: The selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining the central pixel value from the pixel values ​​of the selected sliding window, comprises: Selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining a first central pixel value from pixel values ​​of the selected sliding window; Determine the statistical results other than the preset pixel value interval in the statistical results as partial statistical results; wherein the preset pixel value interval is: an interval of preset pixel values ​​floating up and down based on the first central pixel value; Sliding on the pixel values ​​of the partial statistical results according to a preset sliding window and a preset sliding step size to obtain a second sliding window; Selecting a sliding window from the second sliding window based on the value of the second sliding window, and determining a second central pixel value from the pixel values ​​of the selected sliding window; wherein the value of the second sliding window is the sum of the statistical values ​​of the second sliding window; The first central pixel value and the second central pixel value are determined as the central pixel value.

5. The method according to claim 2, characterized in that: The selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining the central pixel value from the pixel values ​​of the selected sliding window, comprises: The middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the first sliding window is determined as the central pixel value.

6. The method according to claim 4, characterized in that The selecting a sliding window from the first sliding window based on the value of the first sliding window, and determining a first central pixel value from pixel values ​​of the selected sliding window, comprises: Determine the middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the first sliding window as the first central pixel value; The selecting a sliding window from the second sliding window based on the value of the second sliding window, and determining a second central pixel value from pixel values ​​of the selected sliding window, comprises: The middle pixel value of the sliding window corresponding to the maximum value among the values ​​of the second sliding window is determined as the second central pixel value.

7. The method according to any one of claims 1 to 6, characterized in that: The first pixel value includes one or more of the following: A first component of the first pixel value, a second component of the first pixel value, and a third component of the first pixel value.

8. The method according to any one of claims 1 to 6, characterized in that: The standard area for determining the central pixel value includes: The standard area of ​​the central pixel value is determined according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value.

9. The method according to claim 8, characterized in that The step of determining the standard area of ​​the central pixel value according to the region to which the portrait of the second area belongs and the image parameter corresponding to the central pixel value comprises: determining the number of light sources in the second area; Based on the number of light sources in the second area, according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value, a standard area for the central pixel value is determined.

10. The method according to claim 9, characterized in that The central pixel value includes: a first central pixel value and a second central pixel value, and determining the number of light sources in the second area includes: Determine a first proportion of the sum of statistical values ​​of a sliding window with the second central pixel value as an intermediate value to the total number of the first pixel values; When the first proportion is less than a preset proportion, determining the number of the light source to be one; When the first proportion is greater than or equal to the preset proportion, the number of the light sources is determined to be two.

11. The method according to claim 9, characterized in that The central pixel value includes: a first central pixel value and a second central pixel value; the determining of the standard area of ​​the central pixel value based on the number of light sources in the second area, according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value, includes: When the number of light sources in the second area is one, determining a standard area for the central pixel value according to the region to which the portrait in the second area belongs, the color temperature value of the second area, and the brightness value corresponding to the first central pixel value; The brightness value corresponding to the first central pixel value is: the average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the first central pixel value as the middle value.

12. The method according to claim 9, characterized in that The central pixel value includes: a first central pixel value and a second central pixel value; the determining of the standard area of ​​the central pixel value based on the number of light sources in the second area, according to the region to which the portrait in the second area belongs and / or the image parameter corresponding to the central pixel value, includes: In the case where the number of light sources in the second area is two, determining a first standard area for the first central pixel value according to the area to which the portrait in the second area belongs, the color temperature value corresponding to the first central pixel value, and the brightness value corresponding to the first central pixel value; wherein the brightness value corresponding to the first central pixel value is: an average brightness value of an image area corresponding to pixel values ​​of a sliding window with the first central pixel value as an intermediate value; Determine a second standard area for the second central pixel value according to the area to which the portrait of the second area belongs, the color temperature value corresponding to the first central pixel value, and the brightness value corresponding to the second central pixel value; wherein the brightness value corresponding to the second central pixel value is: an average brightness value of the image area corresponding to the pixel values ​​of the sliding window with the second central pixel value as the middle value; A first standard area for the first central pixel value and a second standard area for the second central pixel value are determined as the standard areas for the central pixel values.

13. The method according to claim 12, characterized in that The method further comprises: Determine the sum of the color temperature value of the second area and the preset color temperature value as the color temperature value corresponding to the first central pixel value; The difference between the color temperature value of the second area and the preset color temperature value is determined as the color temperature value corresponding to the second central pixel value.

14. The method according to claim 4, characterized in that The determining, based on the central pixel value and the standard area, a first representative pixel value of the second area from the central pixel value, and determining a translation amount corresponding to the first representative pixel value, comprises: When the number of the light source in the second area is one, determining the first central pixel value as the first representative pixel value; When the first representative pixel value is within the range of the standard area, determining that the translation amount corresponding to the first representative pixel value is zero; When the first representative pixel value is outside the range of the standard area, the translation amount corresponding to the first representative pixel value is determined according to the first vector in the coordinate system formed by the components included in the center pixel value; wherein the first vector is: the vector from the first representative pixel value to the preferred center point of the standard area.

15. The method according to claim 14, characterized in that The determining, according to a first vector in a coordinate system formed by components included in the central pixel value, a translation amount corresponding to the first representative pixel value, comprises: The second vector in the coordinate system is determined as the translation amount corresponding to the first representative pixel value; wherein the second vector is: the vector from the first representative pixel value to the intersection point; the intersection point is: the intersection point of the line connecting the first representative pixel value and the preferred center point of the standard area and the boundary of the standard area.

16. The method according to claim 14, characterized in that The determining the translation amount of the first representative pixel value according to a first vector in a coordinate system formed by components included in the central pixel value includes: Determine the second vector in the coordinate system as the first translation amount corresponding to the first representative pixel value; wherein the second vector is: a vector from the first representative pixel value to an intersection point; the intersection point is: an intersection point of a line connecting the first representative pixel value and the preferred center point of the standard area and a boundary of the standard area; Determine the third vector in the coordinate system as the second translation amount corresponding to the first representative pixel value; wherein the third vector is: a vector from the intersection point to the preferred center point of the standard area; The translation amount corresponding to the first representative pixel value is determined according to the first translation amount and the second translation amount.

17. The method according to claim 16, characterized in that The determining, according to the first translation amount and the second translation amount, the translation amount corresponding to the first representative pixel value includes: The sum of the first translation amount and the second translation amount is determined as the translation amount corresponding to the first representative pixel value.

18. The method according to claim 12, characterized in that The determining, based on the central pixel value and the standard area, a first representative pixel value of the second area from the central pixel value, and determining a translation amount corresponding to the first representative pixel value, comprises: When the number of light sources in the second area is two, determining a translation amount corresponding to the first central pixel value according to the first central pixel value and a first standard area of ​​the first central pixel value; determining a translation amount corresponding to the second central pixel value according to the second central pixel value and a second standard area of ​​the second central pixel value; Based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined.

19. The method according to claim 18, characterized in that The determining the first representative pixel value from the first central pixel value and the second central pixel value based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, and determining the translation amount corresponding to the first representative pixel value, comprises: Selecting the maximum value of the absolute value of the translation amount corresponding to the first central pixel value and the absolute value of the translation amount corresponding to the second central pixel value; Determine the central pixel value corresponding to the maximum value as the first representative pixel value; The translation amount corresponding to the maximum value is determined as the translation amount corresponding to the first representative pixel value.

20. The method according to claim 18, characterized in that The translation amount corresponding to the first central pixel value includes: the first translation amount corresponding to the first central pixel value and the second translation amount corresponding to the first central pixel value; the translation amount corresponding to the second central pixel value includes: the first translation amount corresponding to the second central pixel value and the second translation amount corresponding to the second central pixel value.

21. The method according to claim 20, characterized in that The determining the first representative pixel value from the first central pixel value and the second central pixel value based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, and determining the translation amount corresponding to the first representative pixel value, comprises: When the absolute value of the first translation amount corresponding to the first central pixel value is equal to the absolute value of the first translation amount corresponding to the second central pixel value, selecting the maximum value of the absolute value of the second translation amount corresponding to the first central pixel value and the absolute value of the second translation amount corresponding to the second central pixel value; Determine the central pixel value corresponding to the maximum value as the first representative pixel value; The translation amount corresponding to the maximum value is determined as the translation amount corresponding to the first representative pixel value.

22. The method according to claim 20, characterized in that The determining the first representative pixel value from the first central pixel value and the second central pixel value based on the translation amount corresponding to the first central pixel value and the translation amount corresponding to the second central pixel value, and determining the translation amount corresponding to the first representative pixel value, comprises: When the absolute value of a first translation amount corresponding to the first central pixel value is equal to the absolute value of a first translation amount corresponding to the second central pixel value, and the absolute value of a second translation amount corresponding to the first central pixel value is equal to the absolute value of a second translation amount corresponding to the second central pixel value, determining a second proportion of the total number of the first pixel values ​​occupied by the sum of statistical values ​​of a sliding window with the first central pixel value as an intermediate value; Determine a first proportion of the sum of statistical values ​​of a sliding window with the second central pixel value as an intermediate value to the total number of the first pixel values; According to the size relationship between the first proportion and the second proportion, the first representative pixel value is determined from the first central pixel value and the second central pixel value, and the translation amount corresponding to the first representative pixel value is determined.

23. The method according to claim 22, characterized in that The determining the first representative pixel value from the first central pixel value and the second central pixel value according to the magnitude relationship between the first proportion and the second proportion, and determining the translation amount corresponding to the first representative pixel value, comprises: When the second proportion is greater than the first proportion, the first central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the first central pixel value is determined as the translation amount corresponding to the first representative pixel value; When the second proportion is smaller than the first proportion, the second central pixel value is determined as the first representative pixel value, and the translation amount corresponding to the second central pixel value is determined as the translation amount corresponding to the first representative pixel value.

24. The method according to claim 16, characterized in that The determining, according to the first translation amount and the second translation amount, the translation amount corresponding to the first representative pixel value includes: Obtaining hierarchical control coefficients; Determining a weight of the first translation amount and a weight of the second translation amount according to the hierarchical control coefficient; Based on the weight of the first translation amount and the weight of the second translation amount, a weighted sum is performed on the first translation amount and the second translation amount to obtain the translation amount corresponding to the first representative pixel value.

25. The method according to claim 24, characterized in that The determining, according to the hierarchical control coefficient, the weight of the first translation amount and the weight of the second translation amount comprises: In the case where the hierarchical control coefficient is greater than or equal to zero and less than or equal to a preset first coefficient, determining the weight of the first translation amount to be a ratio of the hierarchical control coefficient to the preset first coefficient, and determining the weight of the second translation amount to be zero; When the graded control coefficient of the second translation is greater than the preset first coefficient and less than or equal to one, the weight of the first translation is determined to be one, and the weight of the second translation is determined to be the ratio of the graded control coefficient to the preset second coefficient; wherein the preset second coefficient is equal to the difference between one and the preset first coefficient.

26. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Determine an adjustment weight of a third pixel value; wherein the third pixel value is: a pixel value of the first area in a second color space; According to the adjustment weight, the translation amount corresponding to the second representative pixel value is adjusted to obtain the translation amount corresponding to the first representative pixel value again; wherein, the second representative pixel value is: the pixel value of the first representative pixel value in the second color space; the translation amount corresponding to the second representative pixel value is: the translation amount corresponding to the first representative pixel value converted to the translation amount in the second color space.

27. The method according to claim 26, characterized in that The determining the adjustment weight of the third pixel value includes: determining a translation amount corresponding to the second representative pixel value; The adjustment weight is determined according to the degree of deviation, a preset fixed translation interval and the translation amount corresponding to the second representative pixel value; wherein the degree of deviation is: the difference between the third pixel value and the second representative pixel value.

28. The method according to claim 26, characterized in that The adjusting the translation amount corresponding to the second representative pixel value according to the adjustment weight to obtain the translation amount corresponding to the first representative pixel value again includes: According to the adjustment weight, adjusting the translation amount corresponding to the second representative pixel value to obtain an adjusted translation amount; adjusting the second representative pixel value according to the adjusted translation amount to obtain an adjusted value of the second representative pixel value; The translation amount corresponding to the first representative pixel value is re-determined according to the adjustment value of the second representative pixel value and the first representative pixel value.

29. The method according to claim 28, characterized in that The re-determining the translation amount corresponding to the first representative pixel value according to the adjustment value of the second representative pixel value and the first representative pixel value includes: Converting the adjustment value of the second representative pixel value from the second color space to the first color space to obtain the adjustment value of the first representative pixel value; The vector from the adjustment value of the first representative pixel value to the first representative pixel value in the coordinate system formed by the components included in the central pixel value is re-determined as the translation amount corresponding to the first representative pixel value.

30. An image processing device, characterized in that: include: An acquisition module is used to acquire a first pixel value of an image; wherein the first pixel value is: a pixel value of a second area in a first area in a first color space; the first area is a skin color area of ​​a portrait in the image; the second area is a skin color area of ​​a face of the portrait in the first area; A first determining module, configured to determine a central pixel value from the first pixel values ​​based on the distribution of the first pixel values; A second determination module is used to determine a standard area of ​​the central pixel value; wherein the standard area is an area in a coordinate system formed by components included in the central pixel value; A third determination module, configured to determine, based on the central pixel value and the standard area, a first representative pixel value of the second area from the central pixel value, and determine a translation amount corresponding to the first representative pixel value; An output module is used to adjust the second pixel value according to the translation amount corresponding to the first representative pixel value to obtain and output a processed image; wherein the second pixel value is: the pixel value of the first area in the first color space.

31. An electronic device, characterized in that: include: A processor and a storage medium storing instructions executable by the processor; The storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in any one of claims 1 to 29 is executed.

32. A computer storage medium, characterized in that Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the image processing method described in any one of claims 1 to 29.

Images