Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image processing method, device, electronic equipment and storage medium, belonging to the technical field of computer. The method comprises: determining saturation enhancement values of a plurality of pixel points in a first image based on the saturation of the plurality of pixel points, the saturation enhancement value of a pixel point and the current saturation of the pixel point being in a nonlinear relationship; performing masking on the first image based on frequency information of the first image to obtain a texture mask image; performing binary segmentation on the texture mask image to obtain a binary mask image; adjusting the saturation of each pixel point in the first image based on the saturation enhancement value and a saturation adjustment coefficient to obtain a second image, the saturation of the second image being greater than that of the first image. The above method can avoid the saturation enhancement value of a pixel point originally having a high saturation in the image being too high, while improving the saturation of a pixel point representing texture, thereby improving the display effect of the image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of Internet technology, more and more users like to share the video works taken on the Internet. In the process of shooting, the images or video pictures taken by some shooting devices have the problem of low saturation, such as dull and gray, and not bright enough color. Therefore, how to enhance the saturation of the image or video picture is a technical problem to be solved.

[0003] In the related art, the image is usually converted from RGB space to HSV (Hue, Saturation, Value) space, and the chroma and brightness of the image are separated. Then, on the basis of not changing the brightness of the image, the saturation of all colors in the image is enhanced in the chroma dimension by using gamma transformation, exponential transformation and the like.

[0004] However, using the above method, the originally high-saturation area in the image will be oversaturated, which reduces the display effect of the image. SUMMARY

[0005] The present disclosure provides an image processing method and device, electronic equipment and storage medium, which can improve the display effect of the image. The technical scheme of the present disclosure is as follows:

[0006] According to an aspect of an embodiment of the present disclosure, an image processing method is provided, comprising:

[0007] Based on the saturation of a plurality of pixel points in the first image, a saturation enhancement value of the plurality of pixel points is determined, the saturation enhancement value of the pixel point and the current saturation of the pixel point are in a nonlinear relationship;

[0008] Based on the frequency information of the first image, the first image is masked to obtain a texture mask image, the frequency information is used to indicate the change frequency of the gray value of a plurality of pixel points in the first image, the frequency information includes low frequency information and high frequency information, the change frequency of the gray value of the plurality of pixel points indicated by the high frequency information is higher than the change frequency of the gray value of the plurality of pixel points indicated by the low frequency information, the high frequency information is used to indicate the area where the pixel points representing the texture in the first image are located, and the texture mask image is used to indicate the pixel points representing the texture in the first image.

[0009] The texture mask image is binarized and segmented to obtain a binarized mask image, and the pixel value of the pixel point in the binarized mask image is used to represent the saturation adjustment coefficient of the pixel point.

[0010] adjust, based on the saturation enhancement value and the saturation adjustment coefficient of each pixel in the first image, the saturation of the each pixel to obtain a second image, and the saturation of the second image is greater than that of the first image.

[0011] According to another aspect of the embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0012] A first determination unit configured to determine, based on the saturation of a plurality of pixels in a first image, a saturation enhancement value of the plurality of pixels, the saturation enhancement value of the pixel and the current saturation of the pixel being in a nonlinear relationship;

[0013] A first mask unit configured to mask the first image based on frequency information of the first image to obtain a texture mask image, the frequency information being used to indicate the change frequency of the gray value of the plurality of pixels in the first image, the frequency information comprising low frequency information and high frequency information, the change frequency of the gray value of the plurality of pixels indicated by the high frequency information being higher than that indicated by the low frequency information, the high frequency information being used to indicate the area in the first image where the pixels representing the texture are located, and the texture mask image being used to indicate the pixels representing the texture in the first image.

[0014] A binaryzation and segmentation unit configured to perform binaryzation and segmentation on the texture mask image to obtain a binaryzation mask image, the pixel value of the pixel in the binaryzation mask image being used to represent the saturation adjustment coefficient of the pixel.

[0015] An adjustment unit configured to adjust, based on the saturation enhancement value and the saturation adjustment coefficient of each pixel in the first image, the saturation of the each pixel to obtain a second image, and the saturation of the second image is greater than that of the first image.

[0016] In some embodiments, the first mask unit is configured to mask the first image based on the high frequency information of the first image to obtain an intermediate mask image, and perform erosion and expansion on the intermediate mask image to obtain the texture mask image.

[0017] In some embodiments, the apparatus further comprises:

[0018] A conversion unit configured to convert the first image to a Lab color space;

[0019] An extraction unit configured to extract the luminance component of the first image in the Lab color space;

[0020] a wavelet decomposition unit configured to perform wavelet decomposition on a luminance component of the first image to obtain frequency information of the first image.

[0021] In some embodiments, the apparatus further includes:

[0022] a mean filter unit configured to perform mean filtering on a pixel value of each pixel point in the binarization mask image to obtain a saturation adjustment coefficient of the each pixel point.

[0023] In some embodiments, the apparatus further includes:

[0024] a second mask unit configured to mask the first image based on skin pixel information and a chroma image and a luminance image of the first image to obtain a skin color mask image, the skin pixel information being used to indicate a chroma range and a luminance range of pixel points representing skin, the chroma image being used to indicate chroma of each pixel point in the first image, the luminance image being used to indicate luminance of each pixel point in the first image, the skin color mask image being used to indicate the pixel points representing skin in the first image.

[0025] a second determination unit configured to determine, for any pixel point in the skin color mask image, a saturation adjustment coefficient of the pixel point based on chroma and luminance of the pixel point, the saturation adjustment coefficient being used to indicate a saturation adjustment proportion of the pixel point, the saturation adjustment coefficient being a positive number not greater than 1.

[0026] In some embodiments, the second mask unit is configured to convert the first image to HSV space to obtain a chroma image of the first image; convert the first image to YUV space to obtain a luminance image of the first image; determine chroma and luminance of each pixel point in the first image based on the chroma image and the luminance image; determine at least one pixel point in the first image based on the skin pixel information and the chroma and luminance of each pixel point in the first image, chroma of the at least one pixel point being within the chroma range indicated by the skin pixel information, luminance of the at least one pixel point being within the luminance range indicated by the skin pixel information; mask the first image based on the at least one pixel point to obtain the skin color mask image.

[0027] In some embodiments, the second determining unit is configured to, for any pixel point, determine a first adjustment coefficient based on a chroma of the pixel point and a chroma range indicated by the skin pixel information, the first adjustment coefficient being used to represent a saturation adjustment coefficient of the pixel point in a chroma dimension; determine a second adjustment coefficient based on a luminance of the pixel point and a luminance range indicated by the skin pixel information, the second adjustment coefficient being used to represent a saturation adjustment coefficient of the pixel point in a luminance dimension; and determine a saturation adjustment coefficient of the pixel point based on the first adjustment coefficient and the second adjustment coefficient.

[0028] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, which includes:

[0029] one or more processors;

[0030] a memory for storing program code executable by the processor;

[0031] wherein the processor is configured to execute the program code to implement the image processing method described above.

[0032] According to another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which, when program code in the computer readable storage medium is executed by a processor of an electronic device, enables the electronic device to perform the image processing method described above.

[0033] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, which includes computer programs / instructions that, when executed by a processor, implement the image processing method described above.

[0034] The embodiments of the present disclosure provide an image processing scheme, which determines a saturation enhancement value that is in a non-linear relationship with the current saturation of a pixel point, by the current saturation of the pixel point. Thus, the saturation enhancement value of the pixel point can be low when the current saturation of the pixel point is low or high, and the saturation enhancement value of the pixel point can be high when the current saturation of the pixel point is moderate. This avoids over-saturation in areas of the image that originally have high saturation. Masking the first image by the frequency information of the first image can determine a plurality of pixel points representing textures in the texture mask image. Further, the saturation adjustment coefficients of the pixel points in the texture mask image are determined by binarization segmentation. By determining the saturation adjustment coefficients of the pixel points representing textures, the saturation enhancement effect on the area where the textures are located can be improved, thereby improving the display effect of the image.

[0035] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure and, do not limit the present disclosure.

[0037] Figure 1 is a schematic diagram of an implementation environment according to an example embodiment.

[0038] Figure 2 is a flowchart of an image processing method according to an example embodiment.

[0039] Figure 3 is a flowchart of another image processing method according to an example embodiment.

[0040] Figure 4 is a schematic diagram of a saturation adjustment function according to an example embodiment.

[0041] Figure 5 is a schematic diagram of a luminance-chrominance two-dimensional coordinate diagram according to an example embodiment.

[0042] Figure 6 is a schematic diagram of a second adjustment coefficient according to an example embodiment.

[0043] Figure 7 is a flowchart of generating a texture mask image according to an example embodiment.

[0044] Figure 8 is a flowchart of image processing according to an example embodiment.

[0045] Figure 9 is a block diagram of an apparatus for image processing according to an example embodiment.

[0046] Figure 10 is another block diagram of an apparatus for image processing according to an example embodiment.

[0047] Figure 11 is a block diagram of an electronic device according to an example embodiment. DETAILED DESCRIPTION

[0048] In order to make the ordinary person in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings.

[0049] It should be noted that the terms "first", "second", and the like in the description and claims of the present disclosure and the foregoing drawings are used to distinguish between similar objects and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present disclosure described herein can be carried out in other than the order shown or described herein. The implementations described in the following example embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0050] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the first image involved in the present application is obtained under sufficient authorization.

[0051] The electronic device can be provided as a terminal or a server. When the electronic device is provided as a terminal, the method of image processing can be implemented by the terminal; when the electronic device is provided as a server, the method of image processing can be implemented by the server; the method of image processing can also be implemented by interaction between the server and the terminal; the method of image processing can also be implemented by the terminal sending an image processing request to the server and the server processing the image.

[0052] Figure 1 is a schematic diagram of an implementation environment of a method of image processing according to an example embodiment. Referring to Figure 1 , the implementation environment specifically includes a terminal 101 and a server 102.

[0053] The terminal 101 can be at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), and a laptop computer. The terminal 101 can have an application program installed and running thereon, which is used to process images, such as enhancing the saturation, sharpness and contrast of images, etc. The terminal 101 can be connected to the server 102 through a wireless network or a wired network. The server 102 is used to provide background services for the application program.

[0054] The terminal 101 can be one of a plurality of terminals, and the embodiment is exemplified by the terminal 101. A person skilled in the art can know that the number of the terminals can be more or less. For example, the terminals can be several, or the terminals can be tens or hundreds, or more, and the number of the terminals and the type of the equipment are not limited in the embodiment.

[0055] The server 102 is at least one of a server, a plurality of servers, a cloud computing platform, and a virtualization center. Optionally, the number of the servers can be more or less, and the embodiment is not limited thereto. Of course, the server 102 can also include other functional servers to provide more comprehensive and diversified services. In some embodiments, the server 102 undertakes the main computing work, and the terminal 101 undertakes the secondary computing work; or the server 102 undertakes the secondary computing work, and the terminal 101 undertakes the main computing work; or the server 102 and the terminal 101 cooperatively compute in a distributed computing architecture. The server 102 can be connected with the terminal 101 and other terminals through a wireless network or a wired network, and the number of the servers can be more or less, and the embodiment is not limited thereto.

[0056] Figure 2 is a flowchart of a method of image processing according to an exemplary embodiment, as shown in Figure 2 The method is performed by an electronic device and includes the following steps:

[0057] In step S201, the electronic device determines saturation enhancement values of a plurality of pixel points in a first image based on saturations of the plurality of pixel points in the first image. The saturation enhancement value of a pixel point and the current saturation of the pixel point are in a non-linear relationship.

[0058] In the embodiment, the first image is an image whose saturation is to be enhanced. The electronic device can determine at least one first image from a video or an image uploaded by a user. The electronic device can also directly obtain a first image uploaded by a user. The electronic device determines the current saturation of a plurality of pixel points in the first image. The electronic device determines the saturation enhancement values of the plurality of pixel points in the first image based on the relationship between the current saturation of the pixel point and the saturation enhancement value indicated by the saturation adjustment function. The saturation enhancement value is the numerical value of the saturation of the pixel point enhanced on the basis of the current saturation. The saturation adjustment function is a non-linear convex function, and when the current saturation of the pixel point is low or high, the saturation enhancement value is low; when the current saturation of the pixel point is moderate, the saturation enhancement value is high. The saturation adjustment function of the non-linear convex function determines the saturation enhancement value of the pixel point, which can avoid the saturation enhancement value of the pixel point originally having a high saturation in the first image also being high, thereby avoiding over-saturation in a partial region of the first image.

[0059] In some embodiments, in a case where the first image is an image in RGB (Red, Green, Blue) format, the electronic device converts the first image from RGB space to HSL (Hue, Saturation, Lightness) space. The electronic device determines, based on an S (Saturation) component of the first image in the HSL space, the current saturation of the plurality of pixel points in the first image.

[0060] In step S202, the electronic device performs masking on the first image based on frequency information of the first image to obtain a texture mask image, the frequency information being used to indicate variation frequencies of the gray values of the plurality of pixel points in the first image, the frequency information including low-frequency information and high-frequency information, the variation frequency of the gray values of the plurality of pixel points indicated by the high-frequency information being higher than the variation frequency of the gray values of the plurality of pixel points indicated by the low-frequency information, the high-frequency information being used to indicate a region in which the pixel points representing the texture are located in the first image, and the texture mask image being used to indicate the pixel points representing the texture in the first image.

[0061] In the embodiments of the present disclosure, the electronic device obtains frequency information of the first image. The frequency information is used to indicate variation frequencies of the gray values of the plurality of pixel points in the first image. The frequency information includes low-frequency information and high-frequency information. The low-frequency information is used to represent a region in which the gray values of the pixel points in the image vary slowly, and the low-frequency information can indicate a large flat region in the image, such as a sky region and a background region in the image. The high-frequency information is used to represent a region in which the gray values of the pixel points in the image vary rapidly, and the high-frequency information can indicate a region in which the texture is located in the image. Therefore, the variation frequency of the gray values of the plurality of pixel points indicated by the high-frequency information is higher than the variation frequency of the gray values of the plurality of pixel points indicated by the low-frequency information. The electronic device determines, based on the high-frequency information of the first image, the plurality of pixel points in the region in which the texture is located in the image. The pixel points are the pixel points representing the texture. The electronic device sets the mask value of the pixel points representing the texture in the first image to 1 and sets the mask value of the remaining pixel points to 0 according to the plurality of pixel points representing the texture. The electronic device performs masking on the first image according to the mask values of the respective pixel points to obtain a texture mask image. The texture mask image is used to indicate the pixel points representing the texture in the first image.

[0062] In step S203, the electronic device performs binaryzation segmentation on the texture mask image to obtain a binaryzation mask image, and the pixel value of a pixel point in the binaryzation mask image is used to represent a saturation adjustment coefficient of the pixel point.

[0063] In the embodiment of the present disclosure, the electronic device performs binary segmentation on the texture mask image to obtain a binary mask image. In the binary mask image, the pixel value of a pixel point representing texture is 1, and the pixel value of the other pixel points is 0. The pixel value of the pixel point in the binary mask image is used to represent the saturation adjustment coefficient of the pixel point. By masking the first image with high-frequency information, the plurality of pixel points representing texture in the texture mask image can be determined. Then, the saturation adjustment coefficient of each pixel point in the texture mask image is determined by binary segmentation. By setting the saturation adjustment coefficient of the pixel point representing texture to the maximum value 1, the saturation enhancement effect of the region where the texture is located can be improved.

[0064] In step S204, the electronic device adjusts the saturation of each pixel point in the first image based on the saturation enhancement value and the saturation adjustment coefficient of the pixel point to obtain a second image, and the saturation of the second image is greater than that of the first image.

[0065] In the embodiment of the present disclosure, the electronic device adjusts the saturation of each pixel point in the first image according to the saturation enhancement value and the saturation adjustment coefficient of the pixel point. For a pixel point in the first image, the electronic device determines the actual saturation enhancement value of the pixel point according to the product of the saturation enhancement value and the saturation adjustment coefficient of the pixel point. For example, in the case that the saturation enhancement value of a certain pixel point is 0.4 and the saturation adjustment coefficient is 0.5, the electronic device determines that the actual saturation enhancement value of the pixel point is 0.4*0.5=0.2. For a pixel point representing texture in the first image, in the case that the saturation enhancement value of the pixel point is 0.4 and the saturation adjustment coefficient is 1, the electronic device determines that the actual saturation enhancement value of the pixel point is 0.4*1=0.4. Therefore, by setting a higher saturation adjustment coefficient for the pixel point representing texture in the case of the same saturation enhancement value, the saturation enhancement effect of the pixel point representing texture can be improved. The electronic device increases the saturation of each pixel point by the actual saturation enhancement value of the pixel point based on the current saturation of each pixel point in the first image to obtain a second image.

[0066] In some embodiments, the electronic device can determine the actual adjustment coefficient of a pixel point according to the saturation enhancement value and the saturation adjustment coefficient of the pixel point. In the case that the first image is an image in RGB format, the electronic device adjusts the R value, the G value and the B value of the pixel point in the R, G and B channels respectively according to the actual adjustment coefficient of the pixel point to obtain a second image. For example, in the case that the saturation enhancement value of a certain pixel point is 0.2 and the saturation adjustment coefficient is 0.5, the electronic device determines that the actual adjustment coefficient of the pixel point is 0.2*0.5=0.1. The electronic device adjusts the R value, the G value and the B value of the pixel point in the R, G and B channels respectively according to the actual adjustment coefficient 0.1 to obtain a second image.

[0067] This disclosure provides an image processing method that determines a saturation enhancement value that has a non-linear relationship with the current saturation of a pixel. Therefore, it ensures that the saturation enhancement value is low when the current saturation of a pixel is low or high, and high when the current saturation of a pixel is moderate. This avoids oversaturation in areas of the image that were originally highly saturated. By masking the first image using its frequency information, multiple pixels representing the texture in the texture mask image can be determined. Then, a saturation adjustment coefficient for each pixel in the texture mask image is determined through binarization segmentation. By determining the saturation adjustment coefficients of the pixels representing the texture, the saturation enhancement effect on the texture area can be improved, thereby enhancing the image display effect.

[0068] In some embodiments, the first image is masked based on the high-frequency information of the first image to obtain a texture mask image, including masking the first image based on the high-frequency information of the first image to obtain an intermediate mask image; and eroding and dilating the intermediate mask image to obtain a texture mask image.

[0069] In this embodiment of the disclosure, by eroding and dilating the intermediate mask image to obtain the texture mask image, the burrs and noise in the intermediate mask image can be removed, thereby improving the accuracy of the electronic device in determining the texture mask image.

[0070] In some embodiments, the method further includes converting the first image to the Lab color space; extracting the luminance component of the first image in the Lab color space; and performing wavelet decomposition on the luminance component of the first image to obtain the frequency information of the first image.

[0071] In this embodiment of the disclosure, the high-frequency and mid-frequency information obtained by wavelet decomposition can indicate the region where the texture is located in the first image, thereby enabling the determination of multiple pixels representing the texture in the first image.

[0072] In some embodiments, the method further includes performing mean filtering on the pixel values ​​of each pixel in the binarized mask image to obtain the saturation adjustment coefficient of each pixel.

[0073] In this embodiment of the disclosure, by applying mean filtering to the binarized mask image, the changes in the saturation adjustment coefficients of each pixel in the binarized mask image can be made smoother, preventing discontinuous saturation enhancement effects of pixels located at the edges.

[0074] In some embodiments, the method further includes masking the first image based on skin pixel information and the chroma and luminance images of the first image to obtain a skin color mask image. The skin pixel information is used to indicate the chroma range and luminance range of pixels representing skin, the chroma image is used to indicate the chroma of each pixel in the first image, the luminance image is used to indicate the luminance of each pixel in the first image, and the skin color mask image is used to indicate the pixels representing skin in the first image. For any pixel in the skin color mask image, a saturation adjustment coefficient is determined based on the chroma and luminance of the pixel. The saturation adjustment coefficient is used to indicate the saturation adjustment ratio of the pixel, and the saturation adjustment coefficient is a positive number not greater than 1.

[0075] In this embodiment, multiple pixels representing skin in the first image can be determined using skin pixel information, thereby enabling masking of the first image to obtain a skin color mask image. By determining a saturation adjustment coefficient of no more than 1 for each pixel representing skin, the saturation enhancement value of the pixels representing skin can be reduced. This avoids excessively high saturation enhancement values ​​for pixels representing skin in the image, which could cause the skin of the target object in the image to appear yellowish, thus improving the image display effect.

[0076] In some embodiments, a skin color mask image is obtained by masking the first image based on skin pixel information and the chroma and luminance images of the first image, including: converting the first image to HSV space to obtain the chroma image of the first image; converting the first image to YUV space to obtain the luminance image of the first image; determining the chroma and luminance of each pixel in the first image based on the chroma image and the luminance image; determining at least one pixel in the first image based on the skin pixel information and the chroma and luminance of each pixel in the first image, wherein the chroma of the at least one pixel is within the chroma range indicated by the skin pixel information and the luminance of the at least one pixel is within the luminance range indicated by the skin pixel information; and masking the first image based on the at least one pixel to obtain the skin color mask image.

[0077] In this embodiment of the disclosure, by determining at least one pixel representing skin in the first image and masking the first image based on the aforementioned pixel, the pixel representing skin and the remaining pixels in the first image can be separated. This facilitates processing one type of pixel during image processing without affecting another type of pixel.

[0078] In some embodiments, for any pixel in a skin color mask image, determining a saturation adjustment coefficient for the pixel based on its chroma and luminance includes: for any pixel, determining a first adjustment coefficient based on the pixel's chroma and the chroma range indicated by skin pixel information, wherein the first adjustment coefficient represents the pixel's saturation adjustment coefficient in the chroma dimension; determining a second adjustment coefficient based on the pixel's luminance and the luminance range indicated by skin pixel information, wherein the second adjustment coefficient represents the pixel's saturation adjustment coefficient in the luminance dimension; and determining a saturation adjustment coefficient for the pixel based on the first and second adjustment coefficients.

[0079] In this embodiment of the disclosure, by determining a first adjustment coefficient and a second adjustment coefficient, the saturation adjustment coefficient of pixels located at the edges of the chroma range can be made higher, while the saturation adjustment coefficient of pixels located in the middle of the chroma range can be made lower. Therefore, the edges of the skin color mask image are smoothed, preventing discontinuous saturation enhancement effects for pixels located at the edges.

[0080] The above Figure 2 The diagram illustrates an image processing flow according to this disclosure. The image processing scheme provided by this disclosure will be further described below. Figure 3 This is a flowchart illustrating another image processing method according to an exemplary embodiment, the method being performed by an electronic device, see [link to flowchart]. Figure 3 The method includes:

[0081] In step S301, the electronic device acquires the first image and the saturation of multiple pixels in the first image.

[0082] In this embodiment, the first image is an image whose saturation needs to be enhanced. The electronic device can determine at least one first image from a video or image uploaded by the user. The electronic device can also directly acquire the first image uploaded by the user. When the first image is an RGB format image, the electronic device can determine the saturation of the pixel based on the R, B, and G values ​​of the pixel. For example, the electronic device determines the saturation of the pixel using the saturation calculation formula shown in formula (1) below.

[0083] S=max(R,G,B)-min (R,G,B) (1)

[0084] Where S is the saturation of a pixel. R, G, and B represent the R, G, and B values ​​of a pixel in the RGB space, respectively. According to formula (1), electronic devices can determine the saturation of a pixel by the difference between the maximum and minimum values ​​of its R, B, and G values.

[0085] In some embodiments, the electronic device can determine the saturation of each pixel in the first image through the saturation component of the first image. If the first image is an RGB format image, the electronic device converts the first image from RGB space to HSL space. The electronic device determines the current saturation of multiple pixels in the first image based on the S (saturation) component of the first image in the HSL space.

[0086] In step S302, the electronic device determines the saturation enhancement value of multiple pixels based on the saturation of multiple pixels in the first image. The saturation enhancement value of a pixel and the current saturation of the pixel have a non-linear relationship.

[0087] In this embodiment, the electronic device determines the saturation enhancement value of multiple pixels in a first image based on the relationship between the current saturation of a pixel and the saturation enhancement value indicated by the saturation adjustment function. The saturation enhancement value is the numerical value by which the saturation of a pixel is increased based on its current saturation. The saturation adjustment function is a non-linear convex function; when the current saturation of a pixel is low or high, the saturation enhancement value is low; when the current saturation of a pixel is moderate, the saturation enhancement value is high. By determining the saturation enhancement value of a pixel using a non-linear convex saturation adjustment function, it is possible to avoid pixels with already high saturation in the first image also having high saturation values, thereby preventing oversaturation in certain areas of the first image.

[0088] For example, the electronic device determines the saturation enhancement value of each pixel in the first image using the saturation adjustment function shown in the following formula (2).

[0089] S_level = ratio * (sin(pi * S)) 2 (2)

[0090] Where S_level is the saturation enhancement value of the pixel. ratio is the maximum value of the saturation enhancement value. S is the current saturation of the pixel. The sin function indicates that the saturation adjustment function is a non-linear sine function, and pi is pi. Figure 4 This is a schematic diagram of a saturation adjustment function. (Example) Figure 4 As shown, the horizontal axis represents the current saturation S of a pixel, and the vertical axis represents the saturation enhancement value S_level of the pixel. The ratio is 0.4, indicating that the maximum output value of the saturation adjustment function is 0.4, which means that the maximum saturation enhancement value of each pixel in the first image is 0.4.

[0091] In step S303, the electronic device converts the first image to HSV and YUV spaces to obtain the chroma image and luminance image of the first image, respectively. The chroma image is used to indicate the chroma of each pixel in the first image, and the luminance image is used to indicate the luminance of each pixel in the first image.

[0092] In this embodiment of the disclosure, the electronic device obtains the chroma image and luminance image of the first image through color gamut space conversion. The electronic device converts the first image to the HSV (Hue, Saturation, Value) space. The electronic device obtains the chroma image of the first image based on the chroma component H of the first image in the HSV space. Here, the chroma component H represents the chroma of each pixel in the first image, and the pixel value of each pixel in the chroma image is its chroma.

[0093] The electronic device converts the first image to the YUV color space. Based on the luminance component Y of the first image in the YUV color space, the electronic device obtains a luminance image of the first image. Here, the luminance component Y represents the luminance of each pixel in the first image, and the pixel value of each pixel in the luminance image is its luminance.

[0094] In step S304, the electronic device determines the chromaticity and luminance of each pixel in the first image based on the chromaticity image and the luminance image, respectively.

[0095] In this embodiment of the disclosure, the electronic device determines the chromaticity of each pixel in the first image based on the pixel values ​​of each pixel in the chromaticity image of the first image. The electronic device determines the luminance of each pixel in the first image based on the pixel values ​​of each pixel in the luminance image of the first image.

[0096] It should be noted that the above steps S303-S304 are illustrated using the example of an electronic device determining the chromaticity and luminance of a pixel through the chromaticity image and luminance image of the first image. In some embodiments, after obtaining the chromaticity component H and luminance component Y of the first image, the electronic device may not generate chromaticity and luminance images. The electronic device directly determines the chromaticity and luminance of each pixel in the first image based on the chromaticity component H and luminance component Y of the first image.

[0097] In some embodiments, the electronic device can determine the brightness of a pixel by using the R, B, and G values ​​of the pixel in the RGB space. Accordingly, the electronic device does not need to convert the first image to the YUV space; the electronic device can determine the brightness of the pixel using the brightness conversion formula shown in formula (3) below.

[0098] Y=R*0.299+G*0.587+B*0.114 (3)

[0099] Where Y represents the brightness of the pixel. R, G, and B represent the R, G, and B values ​​of the pixel in the RGB space, respectively. 0.299, 0.587, and 0.114 represent the conversion coefficients for the R, G, and B values, respectively.

[0100] In some embodiments, after determining the chromaticity and luminance of each pixel in a first image, the electronic device can draw such a... Figure 5 The diagram shows a two-dimensional luminance-chrominance coordinate system. In this system, the horizontal axis represents the chrominance component, and the vertical axis represents the luminance component. The chrominance component values ​​range from 0 to 360, each representing a different chromaticity. The range of 0-25 corresponds to red and orange, 25-50 to orange and yellow, 50-150 to green and blue, 150-250 to blue and purple, and 250-360 to purple and red. The luminance component values ​​range from 0.0 to 1.0, with 0.0 corresponding to the lowest luminance and 1.0 to the highest. By plotting the luminance-chrominance coordinate system of the first image, the chrominance and luminance distribution of each pixel in the image can be displayed relatively intuitively.

[0101] In step S305, the electronic device determines at least one pixel in the first image based on the skin pixel information and the chromaticity and luminance of each pixel in the first image. The chromaticity of the at least one pixel is within the chromaticity range indicated by the skin pixel information, and the luminance of the at least one pixel is within the luminance range indicated by the skin pixel information. The skin pixel information is used to indicate the chromaticity range and luminance range of the pixels representing the skin.

[0102] In this embodiment of the disclosure, the electronic device acquires skin pixel information. The skin pixel information indicates the chromaticity range and luminance range of the pixels representing the skin. The chromaticity range and luminance range can be obtained by statistically analyzing the chromaticity and luminance of a large number of pixels representing the skin. The electronic device compares the chromaticity and luminance of each pixel in the first image with the chromaticity range and luminance range, respectively, to determine at least one pixel whose chromaticity falls within the chromaticity range and whose luminance falls within the luminance range. The at least one pixel is a pixel representing the skin in the first image.

[0103] In some embodiments, skin pixel information is also used to indicate the chromaticity and luminance range of pixels representing lips. Accordingly, the electronic device is able to determine at least one pixel representing lips in the first image based on the skin pixel information.

[0104] For example, the chromaticity and luminance ranges of the pixels representing skin and lips, as indicated by the skin pixel information, are (H<20||H>320)&&(Y>0.2&&Y<0.85). Here, || represents an OR operation, and && represents an AND operation. That is, pixels with a chromaticity less than 20 or a chromaticity greater than 320 and a luminance greater than 0.2 and less than 0.85 are considered to represent either skin or lips.

[0105] In some embodiments, the electronic device, based on the chromaticity range and luminance range indicated by the skin pixel information, in the above... Figure 5 In the luminance-chrominance two-dimensional coordinate graph shown, at least one pixel representing skin or one pixel representing lips is identified.

[0106] In step S306, the electronic device performs masking on the first image based on at least one pixel to obtain a skin color mask image, which is used to indicate the pixels in the first image that represent skin.

[0107] In this embodiment of the disclosure, the electronic device sets the mask value of the pixel representing skin in the first image to 1, and sets the mask value of the remaining pixels to 0, based on at least one pixel representing skin. The electronic device then performs a masking operation on the first image based on the mask values ​​of each pixel to obtain a skin color mask image. Alternatively, the electronic device may set the mask value of the pixel representing skin to 0 and set the mask value of the remaining pixels to 1; this embodiment of the disclosure does not impose this limitation. By determining at least one pixel representing skin in the first image and performing a masking operation on the first image based on this pixel, the pixel representing skin and the remaining pixels in the first image can be separated. This facilitates processing one type of pixel during image processing without affecting the other type of pixel.

[0108] In step S307, for any pixel in the skin color mask image, the electronic device determines the saturation adjustment coefficient of the pixel based on the pixel's chroma and luminance. The saturation adjustment coefficient is used to indicate the saturation adjustment ratio of the pixel, and the saturation adjustment coefficient is a positive number not greater than 1.

[0109] In this embodiment of the disclosure, for any pixel representing skin in a skin color mask image, the electronic device determines the chroma and luminance of the pixel. The electronic device can determine the chroma and luminance of the pixel from the chroma image and luminance image of the first image, or it can determine the chroma and luminance of the pixel from the chroma component H and luminance component Y of the first image. The electronic device can also determine the chroma and luminance of the pixel from the aforementioned... Figure 5The chromaticity and luminance of a pixel are determined in the luminance-chromaticity two-dimensional coordinate graph shown, and this embodiment of the present disclosure does not limit this. The electronic device determines a first adjustment coefficient based on the chromaticity of the pixel and the chromaticity range indicated by the skin pixel information. The first adjustment coefficient is used to represent the saturation adjustment coefficient of the pixel in the chromaticity dimension. The electronic device determines a second adjustment coefficient based on the luminance of the pixel and the luminance range indicated by the skin pixel information, and the second adjustment coefficient is used to represent the saturation adjustment coefficient of the pixel in the luminance dimension. The electronic device determines the saturation adjustment coefficient of the pixel based on the first adjustment coefficient and the second adjustment coefficient. For example, the electronic device determines the above-mentioned first adjustment coefficient, second adjustment coefficient and saturation adjustment coefficient by the following formulas (4), (5) and (6), respectively.

[0110]

[0111]

[0112] Skin_level=H_level*Y_level (6)

[0113] Where H_level and Y_level are the first and second adjustment coefficients of the pixel, respectively. The product of the first and second adjustment coefficients is the saturation adjustment coefficient Skin_level of the pixel. H is the chroma of the pixel, Y is the luminance of the pixel, and pi is pi. 20 and 320 indicate that the chroma range indicated by the skin pixel information is (H<20||H>320). 360 is the maximum value of the chroma of the pixel. 0.2 and 0.85 indicate that the luminance range indicated by the skin pixel is (Y>0.2&&Y<0.85). It can be seen from the above formulas (4) and (5) that the first and second adjustment coefficients are non-linear cosine functions. Determining the first adjustment coefficient of the pixel by using the cosine function can make the adjustment coefficient of each pixel in the skin mask image non-linear. Determining the first and second adjustment coefficients by the above cosine function can make the saturation adjustment coefficient of the pixels located at the edge of the chroma range higher and the saturation adjustment coefficient of the pixels located in the middle of the chroma range lower. Therefore, the edges of the skin color mask image are smoothed to prevent discontinuous saturation enhancement of pixels located at the edges.

[0114] For example, Figure 6 This is a schematic diagram of a second adjustment factor, Y_level. For example... Figure 6 As shown, the horizontal axis represents brightness Y, and the vertical axis represents the second adjustment factor Y_level. (This is achieved through...) Figure 6It can be seen that the relationship between the second adjustment coefficient of a pixel and the pixel's brightness is non-linear. Pixels at the edges of the brightness range, such as pixels with brightness of 0.2 and 0.85, have a second adjustment coefficient of the maximum value of 1. Pixels in the middle of the brightness range, such as pixels with brightness of 0.4 and 0.6, have a lower second adjustment coefficient.

[0115] It should be noted that the electronic device can also determine the saturation adjustment coefficients of pixels in high-texture regions of the first image. High-texture regions are areas in the image with a high density of textures, such as outdoor scenes and landscapes. The process of determining the saturation adjustment coefficients of pixels representing textures is explained below.

[0116] The electronic device acquires frequency information from a first image. This frequency information indicates the frequency of grayscale value changes in multiple pixels within the first image. The frequency information includes low-frequency and high-frequency information. Low-frequency information represents areas where the grayscale values ​​of pixels change slowly, indicating large, flat areas such as the sky and background. High-frequency information represents areas where the grayscale values ​​of pixels change drastically, indicating areas containing texture. Therefore, the frequency of grayscale value changes indicated by high-frequency information is higher than that indicated by low-frequency information. Based on the high-frequency information of the first image, the electronic device determines multiple pixels within the texture region. These pixels represent the texture. The electronic device sets the mask value of the texture-representing pixels in the first image to 1 and sets the mask value of the remaining pixels to 0. The electronic device then performs a masking operation on the first image based on the mask values ​​of each pixel, obtaining a texture mask image. This texture mask image indicates the pixels representing the texture in the first image. An electronic device performs binarization segmentation on a texture mask image to obtain a binarized mask image. In this binarized mask image, pixels representing texture have a value of 1, while all other pixels have a value of 0. The pixel values ​​in the binarized mask image are used to represent the saturation adjustment coefficients of those pixels. By masking the first image using high-frequency information, multiple pixels representing the texture in the texture mask image can be identified. Then, the saturation adjustment coefficients of each pixel in the texture mask image are determined through binarization segmentation. By setting the saturation adjustment coefficients of the pixels representing the texture to the maximum value of 1, the saturation enhancement effect on the texture region can be improved.

[0117] In some embodiments, after obtaining the texture mask image, the electronic device applies Gaussian blur to the texture mask image to obtain a texture mask image where the mask values ​​of the pixels smoothly transition from 0 to 1. The electronic device segments the texture mask image using a segmentation threshold. The electronic device sets the pixel values ​​of pixels with mask values ​​less than the segmentation threshold to 0, and sets the pixel values ​​of pixels with mask values ​​greater than the segmentation threshold to 1, thus obtaining a binarized mask image. The segmentation threshold can be a preset value, such as 0.2, 0.3, or 0.4, etc., and this embodiment does not impose any limitation on it.

[0118] In some embodiments, the electronic device obtains the frequency information of the image by performing wavelet decomposition on the luminance component of the first image. The electronic device converts the first image to the Lab color space. The electronic device extracts the luminance component L of the first image in the Lab color space. The electronic device performs wavelet decomposition on the luminance component L of the first image to obtain low-frequency, mid-frequency, and high-frequency information in the vertical and horizontal directions of the first image. The high-frequency and mid-frequency information obtained through wavelet decomposition can indicate the region where the texture is located in the first image, thereby enabling the determination of multiple pixels representing the texture in the first image.

[0119] In some embodiments, during the generation of a texture mask image, the electronic device can remove burrs from the texture mask image through erosion and dilation. The high-frequency information obtained through wavelet decomposition contains a lot of noise, such as burrs and small dots. Noise affects the accuracy of the texture mask image generated by the electronic device. Therefore, the electronic device first masks the first image based on the high-frequency information of the first image to obtain an intermediate mask image. The electronic device then erodes and dilates the intermediate mask image to obtain the texture mask image. Erosion removes burrs from the intermediate mask image, but it also removes some pixels representing texture. Therefore, the electronic device then dilates the eroded intermediate mask image to obtain the texture mask image. In some embodiments, the electronic device expands the edges of the eroded intermediate mask image by applying Gaussian blur to achieve the dilation operation. By obtaining the texture mask image through erosion and dilation of the intermediate mask image, burrs and noise in the intermediate mask image can be removed, improving the accuracy of the texture mask image determined by the electronic device.

[0120] In some embodiments, after obtaining a binarized mask image, the electronic device performs mean filtering on the pixel values ​​of each pixel in the binarized mask image to obtain a texture mask image with smoother edges. Specifically, the pixel values ​​in the texture mask image obtained after mean filtering are the pixel saturation adjustment coefficients. By performing mean filtering on the binarized mask image, the changes in the saturation adjustment coefficients of each pixel in the binarized mask image become smoother, preventing discontinuous saturation enhancement effects at edge pixels.

[0121] To more clearly explain the process by which electronic devices determine the saturation adjustment coefficients of pixels representing textures, the following will combine... Figure 7 The flowchart shown illustrates the process of generating a texture mask image.

[0122] like Figure 7 As shown, firstly, the electronic device converts the first image in RGB format to the Lab color space. Then, it extracts the luminance component L of the first image in the Lab color space. Next, it performs wavelet decomposition on the luminance component L to obtain the frequency information of the first image. Based on the high-frequency information, it masks the first image to obtain an intermediate mask image. Then, it performs erosion and dilation on the intermediate mask image to obtain a texture mask image. Next, it performs binarization segmentation on the texture mask image to obtain a binary mask image. Finally, it performs mean filtering on the pixel values ​​of each pixel in the binary mask image to obtain a texture mask image with smoother edges.

[0123] In step S308, the electronic device adjusts the saturation of each pixel based on the saturation enhancement value and saturation adjustment coefficient of each pixel in the first image to obtain a second image, the second image having a higher saturation than the first image.

[0124] In this embodiment of the disclosure, the electronic device determines the saturation adjustment coefficient of each pixel in the first image based on the saturation adjustment coefficients of each pixel in the skin color mask image and the texture mask image determined in the above steps. For example, the electronic device determines the saturation adjustment coefficient of each pixel in the first image using the following formula (7).

[0125] Masks=skin_mask+(1-skin_mask)*freq_mask (7)

[0126] Where skin_mask is the saturation adjustment coefficient for pixels in the skin color mask image. freq_mask is the saturation adjustment coefficient for pixels in the texture mask image. Masks is the saturation adjustment coefficient for pixels in the final determined first image.

[0127] The electronic device adjusts the saturation of each pixel in the first image based on the pixel's saturation enhancement value and saturation adjustment coefficient to obtain the second image. First, the electronic device determines the actual adjustment coefficient of the pixel based on the pixel's saturation enhancement value and saturation adjustment coefficient. For example, the electronic device determines the actual adjustment coefficient of the pixel using the following formula (8).

[0128] P = S_level * Masks (8)

[0129] Where P is the actual adjustment coefficient of the pixel. S_level is the saturation enhancement value of the pixel. Masks is the saturation adjustment coefficient of the pixel in the first image. As can be seen from formula (8), the electronic device uses the product between the saturation enhancement value S_level and the saturation adjustment coefficient Masks as the actual adjustment coefficient P of the pixel. For pixels outside the skin color mask image and texture mask image in the first image, the electronic device does not need to determine the saturation adjustment coefficient of the above pixels. The electronic device directly determines the actual adjustment coefficient of the pixel based on the saturation enhancement value and the current saturation of the pixel. For example, the electronic device determines the actual adjustment coefficient of pixels outside the skin color mask image and texture mask image using the following formula (9).

[0130]

[0131] Where S_level is the saturation enhancement value of the pixel. S is the current saturation of the pixel. P is the actual adjustment coefficient of the pixel.

[0132] Then, the electronic device adjusts the R, G and B values ​​of the pixel in the R, G and B channels respectively according to the actual adjustment coefficient of the pixel using the adjustment formulas shown in formulas (10), (11) and (12) to obtain the second image.

[0133] R ′ =RY*P+R*P (10)

[0134] G ′ =GY*P+G*P (11)

[0135] B ′ =BY*P+B*P (12)

[0136] Where Y represents the pixel's brightness. P represents the actual adjustment coefficient for the pixel. R, G, and B represent the pixel's R, G, and B values ​​in the RGB space, respectively. ′ G ′ and B ′These represent the R, G, and B values ​​in the second image after pixel adjustment.

[0137] For example, if the saturation enhancement value of a certain pixel is 0.4 and the saturation adjustment coefficient is 0.5, the electronic device determines the actual adjustment coefficient P of the pixel to be 0.4 * 0.5 = 0.2. Based on the actual adjustment coefficient of 0.2, the electronic device adjusts the R, G, and B values ​​of the pixel in the R, G, and B channels respectively to obtain the second image.

[0138] To more clearly illustrate the process of obtaining the second image by enhancing the saturation of the first image, the following will combine... Figure 8 The image processing flowchart shown illustrates the above process.

[0139] like Figure 8 As shown, the electronic device determines the saturation enhancement values ​​of multiple pixels in the first image based on the saturation adjustment function of a nonlinear convex function, thus avoiding oversaturation in certain areas of the first image. The electronic device masks the first image to obtain a skin tone mask image and a texture mask image. The electronic device determines the saturation adjustment coefficients of pixels in different mask images, achieving skin tone protection and texture enhancement in the first image. Based on the saturation enhancement values ​​and saturation adjustment coefficients of the pixels in the first image, the electronic device determines the actual adjustment coefficients for each pixel. The electronic device then adjusts the saturation of the pixels using these actual adjustment coefficients to obtain the second image.

[0140] This disclosure provides an image processing method that determines a saturation enhancement value that has a non-linear relationship with the current saturation of a pixel. Therefore, it ensures that the saturation enhancement value is low when the current saturation of a pixel is low or high, and high when the current saturation of a pixel is moderate. This avoids oversaturation in areas of the image that were originally highly saturated. By masking the first image using its frequency information, multiple pixels representing textures in the texture mask image can be determined. By determining the saturation adjustment coefficients of the pixels representing textures, the saturation enhancement effect on the texture area can be improved. Using skin pixel information, multiple pixels representing skin in the first image and their saturation adjustment coefficients can be determined. The saturation adjustment coefficients of the pixels representing skin are positive numbers not greater than 1, which reduces the saturation enhancement value of the pixels representing skin. This avoids excessively high saturation enhancement values ​​for the pixels representing skin, which could cause the skin of the target object in the image to appear yellowish, thus improving the image display effect.

[0141] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0142] Figure 9 This is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 9 As shown, the device includes: a first determining unit 901, a first masking unit 902, a binarization segmentation unit 903, and an adjustment unit 904.

[0143] The first determining unit 901 is configured to determine a saturation enhancement value for multiple pixels based on the saturation of multiple pixels in the first image, wherein the saturation enhancement value of a pixel and the current saturation of the pixel have a non-linear relationship.

[0144] The first masking unit 902 is configured to mask the first image based on the frequency information of the first image to obtain a texture mask image. The frequency information is used to indicate the frequency of gray value changes of multiple pixels in the first image. The frequency information includes low-frequency information and high-frequency information. The gray value change frequency of multiple pixels indicated by the high-frequency information is higher than the gray value change frequency of multiple pixels indicated by the low-frequency information. The high-frequency information is used to indicate the region where the pixels representing the texture are located in the first image. The texture mask image is used to indicate the pixels representing the texture in the first image.

[0145] Binarization segmentation unit 903 is configured to perform binarization segmentation on the texture mask image to obtain a binarized mask image, wherein the pixel value of the pixel in the binarized mask image is used to represent the saturation adjustment coefficient of the pixel.

[0146] The adjustment unit 904 is configured to adjust the saturation of each pixel based on the saturation enhancement value and saturation adjustment coefficient of each pixel in the first image to obtain a second image, wherein the saturation of the second image is greater than that of the first image.

[0147] In some embodiments, the first masking unit 902 is configured to mask the first image based on the high-frequency information of the first image to obtain an intermediate mask image; and to erode and dilate the intermediate mask image to obtain a texture mask image.

[0148] In some embodiments, Figure 10 This is a block diagram of another image processing apparatus according to an exemplary embodiment, such as Figure 9 As shown, the device also includes:

[0149] The conversion unit 905 is configured to convert the first image to the Lab color space;

[0150] Extraction unit 906 is configured to extract the luminance component of the first image in the Lab color space;

[0151] Wavelet decomposition unit 907 is configured to perform wavelet decomposition on the brightness component of the first image to obtain the frequency information of the first image.

[0152] In some embodiments, the apparatus further includes:

[0153] The mean filtering unit 908 is configured to perform mean filtering on the pixel values ​​of each pixel in the binarized mask image to obtain the saturation adjustment coefficient of each pixel.

[0154] In some embodiments, the apparatus further includes:

[0155] The second masking unit 909 is configured to mask the first image based on skin pixel information and the chroma image and luminance image of the first image to obtain a skin color mask image. The skin pixel information is used to indicate the chroma range and luminance range of the pixels representing the skin. The chroma image is used to indicate the chroma of each pixel in the first image. The luminance image is used to indicate the luminance of each pixel in the first image. The skin color mask image is used to indicate the pixels representing the skin in the first image.

[0156] The second determining unit 910 is configured to determine a saturation adjustment coefficient for any pixel in the skin color mask image based on the pixel's chroma and luminance. The saturation adjustment coefficient is used to indicate the saturation adjustment ratio of the pixel and is a positive number not greater than 1.

[0157] In some embodiments, the second masking unit 909 is configured to convert the first image to HSV space to obtain a chroma image of the first image; convert the first image to YUV space to obtain a luminance image of the first image; determine the chroma and luminance of each pixel in the first image based on the chroma image and the luminance image, respectively; determine at least one pixel in the first image based on skin pixel information and the chroma and luminance of each pixel in the first image, wherein the chroma of the at least one pixel is within the chroma range indicated by the skin pixel information and the luminance of the at least one pixel is within the luminance range indicated by the skin pixel information; and mask the first image based on the at least one pixel to obtain a skin color mask image.

[0158] In some embodiments, the second determining unit 910 is configured to, for any pixel, determine a first adjustment coefficient based on the chroma range indicated by the pixel's chroma and skin pixel information, the first adjustment coefficient representing the pixel's saturation adjustment coefficient in the chroma dimension; determine a second adjustment coefficient based on the pixel's brightness and the brightness range indicated by the skin pixel information, the second adjustment coefficient representing the pixel's saturation adjustment coefficient in the brightness dimension; and determine the pixel's saturation adjustment coefficient based on the first and second adjustment coefficients.

[0159] This disclosure provides an image processing apparatus that determines a saturation enhancement value that has a non-linear relationship with the current saturation of a pixel. Therefore, it ensures that the saturation enhancement value is low when the current saturation of a pixel is low or high, and high when the current saturation of a pixel is moderate. This avoids oversaturation in areas of the image that were originally highly saturated. By masking the first image using the frequency information of the first image, multiple pixels representing the texture in the texture mask image can be determined. Then, a saturation adjustment coefficient for each pixel in the texture mask image is determined through binarization segmentation. By determining the saturation adjustment coefficients of the pixels representing the texture, the saturation enhancement effect on the texture area can be improved, thereby improving the image display effect.

[0160] It should be noted that the image processing apparatus provided in the above embodiments is only illustrated by the division of the above functional units during image processing. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the electronic device can be divided into different functional units to complete all or part of the functions described above. In addition, the image processing apparatus and image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0161] Regarding the image processing apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0162] Figure 11 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Typically, the electronic device 1100 includes a processor 1101 and a memory 1102.

[0163] Processor 1101 may include one or more processing cores, such as a quad-core processor, an eleven-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0164] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one program code, which is executed by the processor 1101 to implement the image processing method provided in the method embodiments of this disclosure.

[0165] In some embodiments, the electronic device 1100 may optionally include a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: radio frequency circuitry 1104, display screen 1105, camera assembly 1106, audio circuitry 1107, and power supply 11011.

[0166] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0167] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other electronic devices through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this disclosure.

[0168] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, which serves as the front panel of electronic device 1100; in other embodiments, there may be at least two display screens, respectively disposed on different surfaces of electronic device 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of electronic device 1100. Furthermore, display screen 1105 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0169] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device, and the rear-facing camera is located on the back of the electronic device. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0170] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the electronic device 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0171] Power supply 11011 is used to supply power to the various components in electronic device 1100. Power supply 11011 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 11011 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0172] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the electronic device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0173] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1102 including instructions, which can be executed by the processor 1101 of the terminal 1100 to complete the image processing method described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0174] A computer program product includes a computer program that, when executed by a processor, implements the above-described image processing method.

[0175] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0176] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Based on the saturation of multiple pixels in the first image, a saturation enhancement value for the multiple pixels is determined, and the saturation enhancement value of the pixel and the current saturation of the pixel have a non-linear relationship. Based on the frequency information of the first image, the first image is masked to obtain a texture mask image. The frequency information is used to indicate the frequency of grayscale value changes of multiple pixels in the first image. The frequency information includes low-frequency information and high-frequency information. The grayscale value change frequency of multiple pixels indicated by the high-frequency information is higher than that of multiple pixels indicated by the low-frequency information. The high-frequency information is used to indicate the region where the pixels representing the texture are located in the first image. The texture mask image is used to indicate the pixels representing the texture in the first image. The texture mask image is binarized to obtain a binary mask image, and the pixel values ​​of the pixels in the binary mask image are used to represent the saturation adjustment coefficients of the pixels. Based on the saturation enhancement value and saturation adjustment coefficient of each pixel in the first image, the saturation of each pixel is adjusted to obtain a second image, the saturation of the second image being greater than that of the first image.

2. The image processing method according to claim 1, characterized in that, The step of masking the first image based on its frequency information to obtain a texture mask image includes: Based on the high-frequency information of the first image, the first image is masked to obtain an intermediate mask image; The intermediate mask image is eroded and dilated to obtain the texture mask image.

3. The image processing method according to claim 1, characterized in that, The method further includes: Convert the first image to the Lab color space; Extract the luminance component of the first image from the Lab color space; Wavelet decomposition is performed on the brightness component of the first image to obtain the frequency information of the first image.

4. The image processing method according to claim 1, characterized in that, The method further includes: The pixel values ​​of each pixel in the binarized mask image are subjected to mean filtering to obtain the saturation adjustment coefficient of each pixel.

5. The image processing method according to claim 1, characterized in that, The method further includes: Based on skin pixel information and the chroma and luminance images of the first image, the first image is masked to obtain a skin color mask image. The skin pixel information is used to indicate the chroma and luminance range of the pixels representing the skin. The chroma image is used to indicate the chroma of each pixel in the first image. The luminance image is used to indicate the luminance of each pixel in the first image. The skin color mask image is used to indicate the pixels representing the skin in the first image. For any pixel in the skin color mask image, a saturation adjustment coefficient is determined based on the pixel's chroma and luminance. The saturation adjustment coefficient is used to indicate the saturation adjustment ratio of the pixel and is a positive number not greater than 1.

6. The image processing method according to claim 5, characterized in that, The step of masking the first image based on skin pixel information and the chroma and luminance images of the first image to obtain a skin color mask image includes: The first image is converted to the HSV space to obtain the chroma image of the first image; The first image is converted to YUV space to obtain the brightness image of the first image; Based on the chroma image and the luminance image, the chroma and luminance of each pixel in the first image are determined respectively; Based on the skin pixel information and the chromaticity and brightness of each pixel in the first image, at least one pixel is determined in the first image, wherein the chromaticity of the at least one pixel is within the chromaticity range indicated by the skin pixel information, and the brightness of the at least one pixel is within the brightness range indicated by the skin pixel information. Based on the at least one pixel, the first image is masked to obtain the skin color mask image.

7. The image processing method according to claim 5, characterized in that, The step of determining the saturation adjustment coefficient of any pixel in the skin color mask image based on the pixel's chroma and luminance includes: For any pixel, a first adjustment coefficient is determined based on the pixel's chroma and the chroma range indicated by the skin pixel information. The first adjustment coefficient is used to represent the saturation adjustment coefficient of the pixel in the chroma dimension. Based on the brightness of the pixel and the brightness range indicated by the skin pixel information, a second adjustment coefficient is determined. The second adjustment coefficient is used to represent the saturation adjustment coefficient of the pixel in the brightness dimension. The saturation adjustment coefficient of the pixel is determined based on the first adjustment coefficient and the second adjustment coefficient.

8. An image processing apparatus, characterized in that, The device includes: The first determining unit is configured to determine a saturation enhancement value for a plurality of pixels based on the saturation of a plurality of pixels in a first image, wherein the saturation enhancement value of the pixel and the current saturation of the pixel have a non-linear relationship. The first masking unit is configured to mask the first image based on the frequency information of the first image to obtain a texture mask image. The frequency information is used to indicate the frequency of grayscale value changes of multiple pixels in the first image. The frequency information includes low-frequency information and high-frequency information. The grayscale value change frequency of multiple pixels indicated by the high-frequency information is higher than the grayscale value change frequency of multiple pixels indicated by the low-frequency information. The high-frequency information is used to indicate the region where the pixels representing texture are located in the first image. The texture mask image is used to indicate the pixels representing texture in the first image. The binarization segmentation unit is configured to perform binarization segmentation on the texture mask image to obtain a binarized mask image, wherein the pixel values ​​of the pixels in the binarized mask image are used to represent the saturation adjustment coefficients of the pixels. The adjustment unit is configured to adjust the saturation of each pixel in the first image based on the saturation enhancement value and saturation adjustment coefficient, so as to obtain a second image, wherein the saturation of the second image is greater than that of the first image.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the image processing method as described in any one of claims 1 to 7.

Images