Image processing method, device, electronic device and computer-readable storage medium

Abstract

Embodiments of the present application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, relating to the technical field of white balance adjustment. The method includes: determining at least one gray point in an original image; a gray point is a pixel representing a neutral color in the original image; classifying each gray point based on its color information in a first color space to obtain multiple gray point clusters; calculating a white balance gain value for the original image based on the gray point clusters and the color temperature information of the gray points; and performing white balance adjustment on the original image based on the white balance gain value to obtain a target image. Embodiments of the present application calculate the white balance gain value of an image based on the gray points in the original image, thereby improving the robustness of the white balance correction effect.

Description

Technical Field

[0001] The present application relates to the technical field of white balance adjustment. Specifically, the present application relates to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the advancement of image acquisition and processing technologies, the application of image sensors and image capture devices has become increasingly widespread, and people's requirements for image quality are also increasing. During the image acquisition or capture process, the resulting image may have color casts due to the influence of lighting or angle. For example, when taking a snapshot, interference from reflected light, improper development or scanning methods, etc., can all cause color casts in the image.

[0003] White balance describes the accuracy of the color white produced by mixing the three primary colors of red, green, and blue in a display. Image white balance is characterized by its ability to reproduce white objects under any light source. Adjusting white balance can compensate for color casts that occur when shooting under specific light sources by enhancing the corresponding complementary colors.

[0004] When adjusting the white balance of an image in a mixed color temperature scene, existing technologies typically first divide the color temperature regions and then use an existing single color temperature balance algorithm to adjust the white balance of each color temperature region. This method is relatively rough in its division of color temperature regions and is easily affected by large color blocks and highlight areas, making it difficult to find an accurate white balance gain value. This results in poor image color cast correction. Summary of the Invention

[0005] The present invention provides an imaging method, apparatus, electronic device, and computer-readable storage medium that can address the problem of poor image color cast correction due to inaccurate white balance gain values ​​in the prior art. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, an image processing method is provided, the method comprising:

[0007] Determine at least one gray point in the original image; the gray point is a pixel point representing a neutral color in the original image;

[0008] Classifying each gray point according to color information of the first color space of each gray point to obtain multiple gray point clusters;

[0009] The white balance gain value of the original image is calculated according to the gray point cluster and the color temperature information of the gray points, and the white balance of the original image is adjusted according to the white balance gain value to obtain the target image.

[0010] Optionally, the determining of at least one gray point in the original image includes:

[0011] Performing color gamut conversion on the color information of the first color space of each pixel of the original image to obtain the color information of the second color space of each pixel;

[0012] At least one gray point is determined according to the color information of each pixel in the second color space.

[0013] Optionally, the original image includes a plurality of sub-images without overlapping areas;

[0014] The above-mentioned determining at least one gray point based on the color information of the second color space of each pixel point includes:

[0015] Screening candidate gray points from each pixel according to color information of the second color space of each pixel;

[0016] According to the candidate gray points included in each sub-image, the gray point corresponding to each sub-image is determined.

[0017] Optionally, the above-mentioned determining the gray point corresponding to each sub-image based on the candidate gray points included in each sub-image includes:

[0018] For each sub-image, determining an average pixel point of the sub-image according to an average value of color information of each candidate gray point in the sub-image in the first color space;

[0019] The average pixel point of each sub-image is used as the gray point corresponding to each sub-image.

[0020] Optionally, the above-mentioned calculation of the white balance gain value of the original image based on the gray point clusters and the color temperature information of the gray points includes:

[0021] For each gray point cluster, the local gain value of the gray point cluster is calculated according to the color temperature information of each gray point in the gray point cluster;

[0022] The white balance gain value of the original image is calculated based on the local gain value of each gray point cluster.

[0023] Optionally, the above-mentioned calculation of the white balance gain value of the original image based on the local gain value of each gray point cluster includes:

[0024] Determine the weight of each gray point cluster according to the number of gray points included in each gray point cluster;

[0025] The local gain value of each gray point cluster is weighted according to the weight to obtain the white balance gain value of the original image.

[0026] Optionally, for each gray point cluster, calculating the local gain value of the gray point cluster according to the color temperature information of each gray point in the gray point cluster includes:

[0027] For each gray point cluster, a histogram is constructed based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point;

[0028] Calculate the local gain value of the gray point cluster based on the histogram.

[0029] Optionally, the above-mentioned calculation of the local gain value of the gray point cluster based on the histogram includes:

[0030] Determine the largest column vector in the histogram;

[0031] The local gain value of the gray point cluster is calculated according to the color information of the first color space of each gray point included in the maximum column vector.

[0032] Optionally, the above-mentioned calculating the local gain value of the gray point cluster according to the color information of the first color space of each gray point included in the maximum column vector includes:

[0033] The average value of the color information of the first color space of each gray point included in the maximum column vector is used as the light source information of the gray point cluster;

[0034] The local gain value of the gray point cluster is determined according to the proportion of each color channel in the light source information.

[0035] According to another aspect of an embodiment of the present application, there is provided an image processing apparatus, the apparatus comprising:

[0036] A determination module, configured to determine at least one gray point in the original image; a gray point is a pixel representing a neutral color in the original image;

[0037] a classification module, configured to classify each gray point according to color information of the first color space of each gray point to obtain a plurality of gray point clusters;

[0038] The calculation module is used to calculate the white balance gain value of the original image according to the gray point cluster and the color temperature information of the gray point, and perform white balance adjustment on the original image according to the white balance gain value to obtain the target image.

[0039] Optionally, when determining at least one gray point in the original image, the determination module is configured to:

[0040] Performing color gamut conversion on the color information of the first color space of each pixel of the original image to obtain the color information of the second color space of each pixel;

[0041] At least one gray point is determined according to the color information of each pixel in the second color space.

[0042] Optionally, the original image includes a plurality of sub-images without overlapping areas;

[0043] When determining at least one gray point based on the color information of each pixel in the second color space, the determination module is configured to:

[0044] Screening candidate gray points from each pixel according to color information of the second color space of each pixel;

[0045] According to the candidate gray points included in each sub-image, the gray point corresponding to each sub-image is determined.

[0046] Optionally, when determining the gray point corresponding to each sub-image based on the candidate gray points included in each sub-image, the determination module is configured to:

[0047] For each sub-image, determining an average pixel point of the sub-image according to an average value of color information of each candidate gray point in the sub-image in the first color space;

[0048] The average pixel point of each sub-image is used as the gray point corresponding to each sub-image.

[0049] Optionally, when calculating the white balance gain value of the original image based on the gray point clusters and the color temperature information of the gray points, the calculation module is used to:

[0050] For each gray point cluster, the local gain value of the gray point cluster is calculated according to the color temperature information of each gray point in the gray point cluster;

[0051] The white balance gain value of the original image is calculated based on the local gain value of each gray point cluster.

[0052] Optionally, when calculating the white balance gain value of the original image based on the local gain value of each gray point cluster, the calculation module is used to:

[0053] Determine the weight of each gray point cluster according to the number of gray points included in each gray point cluster;

[0054] The local gain value of each gray point cluster is weighted according to the weight to obtain the white balance gain value of the original image.

[0055] Optionally, when calculating the local gain value of each gray point cluster according to the color temperature information of each gray point in the gray point cluster, the calculation module is used to:

[0056] For each gray point cluster, a histogram is constructed based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point;

[0057] Calculate the local gain value of the gray point cluster based on the histogram.

[0058] Optionally, when calculating the local gain value of the gray point cluster according to the histogram, the calculation module is used to:

[0059] Determine the largest column vector in the histogram;

[0060] The local gain value of the gray point cluster is calculated according to the color information of the first color space of each gray point included in the maximum column vector.

[0061] Optionally, when calculating the local gain value of the gray point cluster according to the color information of the first color space of each gray point included in the maximum column vector, the calculation module is used to:

[0062] The average value of the color information of the first color space of each gray point included in the maximum column vector is used as the light source information of the gray point cluster;

[0063] The local gain value of the gray point cluster is determined according to the proportion of each color channel in the light source information.

[0064] According to another aspect of an embodiment of the present application, an electronic device is provided, which includes: a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the method shown in the first aspect of the embodiment of the present application.

[0065] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiments of the present application are implemented.

[0066] According to one aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiment of the present application are implemented.

[0067] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0068] The embodiment of the present application classifies each gray point based on the color information of at least one gray point in the original image to obtain multiple gray point clusters corresponding to the original image; wherein the gray point is a pixel point representing a neutral color in the original image, which can indicate the brightness information of the original image. Then, the white balance gain value of the original image is calculated based on the color temperature information of each gray point cluster and each gray point, so that the white balance of the original image is adjusted according to the white balance gain value to obtain the target image. The embodiment of the present application realizes the calculation of the white balance gain value of the image based on the gray point. Compared with the prior art method of obtaining the white balance gain value based on the coarse-grained color temperature area of ​​the image, the color information and color temperature information of each gray point are fully considered in the present application. The calculated white balance gain is more accurate and the reliability of the white balance adjustment is improved. At the same time, the present application can identify the color temperature type of each gray point based on the clustering of gray point color information, that is, the gray point cluster, to achieve white balance adjustment based on mixed color temperature scenes, thereby improving the robustness of the white balance correction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0070] Figure 1 A schematic diagram of an application scenario of an image processing method provided in an embodiment of the present application;

[0071] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0072] Figure 3 A schematic diagram of a process for obtaining gray points in an image processing method provided in an embodiment of the present application;

[0073] Figure 4 A schematic diagram of a process for calculating a white balance gain value in an image processing method provided in an embodiment of the present application;

[0074] Figure 5 A flowchart of an exemplary image processing method provided in an embodiment of the present application;

[0075] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0076] Figure 7 A schematic structural diagram of an image processing electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0078] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0079] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0080] The human eye's perception of color remains essentially constant under varying lighting conditions. For example, a white object appears white whether viewed in the early morning sun or under dim nightlight. This is because the human brain adapts to the color of objects under varying lighting conditions over time. This unique adaptability allows the human brain to detect and correct for color shifts under varying lighting conditions. Therefore, white objects appear white regardless of sunlight, indoor lighting, shadows, or fluorescent lighting. However, image sensors lack this adaptability. Due to the imbalance in their output under varying lighting conditions, color distortion can occur in the sensor output, resulting in undesirable reddish or bluish tints. To ensure that sensor output more closely matches human visual standards, it is necessary to mimic the human brain's behavior and adjust color according to changes in lighting to achieve satisfactory color. Automatic white balance automatically adjusts image color for different color temperatures using statistical information and algorithms to eliminate color casts and ensure images are more consistent with human visual perception.

[0081] The inventors have discovered that when adjusting white balance, the currently more common practice is to assume the scene as a single color temperature scene and use the calculated gain as the gain for each pixel. However, in actual scenes, mixed color temperature scenes are often encountered, that is, the image contains two or more color temperatures. If the current single color temperature gain method is directly used for white balance processing, the white balance gain result is likely to be biased towards the color temperature area with a large proportion of pixels, which will cause serious color cast of pixels at other color temperatures in the image.

[0082] With technological advancements, various white balance techniques have been developed to identify color temperature scenes and improve color cast issues. However, the identification and processing of mixed color temperature scenes remains incomplete. Current methods fall into three main categories: algorithms that use the grayscale world; algorithms that favor high or low global corrections; and algorithms that segment the grayscale area into multiple segments based on color temperature to identify and distinguish mixed color temperature scenes and then perform white balance gain correction. However, these methods fail to meet the high-quality image quality requirements for mixed color temperature scenes.

[0083] The image processing method, device, electronic device and computer-readable storage medium provided in this application are intended to solve the above technical problems in the prior art.

[0084] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0085] like Figure 1 As shown, the image processing method of the present application can be applied to Figure 1 In the scenario shown, specifically, the server can obtain the original image from the client, and then perform gray point detection on the original image to obtain at least one gray point, where the gray point is a pixel point representing a neutral color in the original image; the server classifies each gray point according to the color information of the first color space of each gray point to obtain multiple gray point clusters; and calculates the white balance gain value of the original image based on the gray point clusters and the color temperature information of the gray points. After obtaining the white balance gain value, the server can send the white balance gain value to the client, so that the client can adjust the white balance of the original image according to the white balance gain value to obtain the target image.

[0086] Figure 1 In the scenario shown, the above image processing method can be performed in a server, and in other scenarios, it can also be performed in a terminal.

[0087] Those skilled in the art will understand that the “terminal” used here may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), an MID (Mobile Internet Device), etc.; the “server” may be implemented as an independent server or a server cluster consisting of multiple servers.

[0088] An image processing method is provided in an embodiment of the present application. Figure 2 As shown, it can be applied to a server or terminal for image processing, and the method includes:

[0089] S201: Determine at least one gray point in an original image.

[0090] The gray points are pixels representing neutral colors in the original image. The pixel data of the original image can be obtained by format conversion based on the original image encoding data.

[0091] Neutral colors, also known as achromatic colors, refer to a series of shades of gray, consisting of black, white, and blends of black and white. Neutral colors are neither cold nor warm. Black, white, and gray are the three most commonly used neutral colors. In this application, neutral colors can be used to indicate image brightness information. Detecting the original image based on gray points avoids the effects of overexposed areas or excessive noise in the original image, improving the accuracy of subsequent white balance gain calculations.

[0092] Specifically, the server or terminal for performing image processing may detect the color information of each pixel in the original image to determine at least one gray point in the original image.

[0093] In some embodiments, a server or terminal used for image processing can determine the color information of each pixel in the original image point by point based on preset detection conditions. If the color information of a pixel satisfies the detection conditions, the pixel is treated as a gray point. The detection conditions are used to determine whether the color information of the pixel is a neutral color.

[0094] In other embodiments, the server or terminal used for image processing can divide the original image into blocks to obtain multiple non-overlapping sub-images, and then perform gray point detection on each sub-image separately, and determine the gray point corresponding to each sub-image; wherein the gray point corresponding to each sub-image can represent the brightness information of the sub-image.

[0095] S202 , classifying each gray point according to color information of the first color space of each gray point to obtain a plurality of gray point clusters.

[0096] The first color space may be an RGB (red, green, and blue) primary color space.

[0097] Specifically, a server or terminal for image processing may cluster each gray point according to the three primary color data of each gray point to obtain a plurality of gray point clusters.

[0098] In an embodiment of the present application, all gray points may be classified based on a clustering algorithm, wherein the clustering algorithm may be a mean-shift algorithm, and the classification steps may include:

[0099] A center point is randomly determined among all the gray points, and multiple candidate gray points whose distance from the center point does not exceed a preset distance are determined; then the offset vectors between the center point and each candidate gray point are calculated, and the moving distance of the center point is determined according to the offset vectors corresponding to all the candidate gray points. The center point is iteratively updated based on the above moving distance until the offset vector meets the convergence condition. At this time, all gray points visited during the iteration belong to the same gray point cluster.

[0100] The embodiment of the present application classifies gray points to obtain multiple gray point clusters, where each gray point cluster can represent a color temperature type in the original image. When the original image is a mixed color temperature image, the white balance gain value of the original image can be accurately calculated based on each color temperature type, thereby improving the adaptability and robustness of white balance adjustment.

[0101] S203 , calculating a white balance gain value of the original image according to the gray point clusters and the color temperature information of the gray points, and performing white balance adjustment on the original image according to the white balance gain value to obtain a target image.

[0102] Specifically, the server or terminal used for image processing can perform statistics on the color temperature information of the gray points in each gray point cluster to obtain statistical information, and calculate the white balance gain value based on the statistical information, and then adjust the white balance of each pixel point of the original image according to the white balance gain value to obtain the target image.

[0103] The embodiment of the present application classifies each gray point based on the color information of at least one gray point in the original image to obtain multiple gray point clusters corresponding to the original image; wherein the gray point is a pixel point representing a neutral color in the original image, which can indicate the brightness information of the original image. Then, the white balance gain value of the original image is calculated based on the color temperature information of each gray point cluster and each gray point, so that the white balance of the original image is adjusted according to the white balance gain value to obtain the target image. The embodiment of the present application realizes the calculation of the white balance gain value of the image based on the gray point. Compared with the prior art method of obtaining the white balance gain value based on the coarse-grained color temperature area of ​​the image, the color information and color temperature information of each gray point are fully considered in the present application. The calculated white balance gain is more accurate and the reliability of the white balance adjustment is improved. At the same time, the present application can identify the color temperature type of each gray point based on the clustering of gray point color information, that is, the gray point cluster, to achieve white balance adjustment based on mixed color temperature scenes, thereby improving the robustness of the white balance correction effect.

[0104] An embodiment of the present application provides a possible implementation method, which includes: before determining at least one gray point in the original image in step S201, the following steps are performed:

[0105] (1) A server or terminal for image processing can obtain encoded data of an original image through an image acquisition device such as a camera. The original image can be a frame of a plurality of images continuously acquired by the image acquisition device.

[0106] The encoded data of the original image can be Bayer format data. Bayer format data consists of 1 / 2 G pixels, 1 / 4 R pixels, and 1 / 4 B pixels. Each pixel in Bayer format data only includes a portion of the spectrum. Common Bayer format images have four different distributions: RGGB, BGGR, GRBG, and GBRG.

[0107] (2) The server or terminal used for image processing can use the linear interpolation method to obtain the three primary color information of each pixel, i.e., RGB data, based on the Bayer format data of the original image. The linear interpolation method processes the pixels as follows:

[0108] When a green (G) pixel lacks blue (B) and red (R) components, the red (R) and blue (B) components of the pixel are determined by averaging the two red (R) and blue (B) pixels closest to the green pixel.

[0109] When calculating the blue (B) component of a red (R) pixel, it can be determined by averaging the four nearest blue pixels.

[0110] When calculating the red (R) component of a blue (B) pixel, the red component can be determined by averaging the four nearest red pixels.

[0111] When calculating the green (G) component of a red (R) and blue (B) pixel, it can be determined by averaging the four nearest green pixels.

[0112] When the original pixel is an edge pixel, it is padded according to the mirroring principle.

[0113] In an embodiment of the present application, a possible implementation is provided. In step S201, determining at least one gray point in the original image includes:

[0114] (1) Performing color gamut conversion on the color information of the first color space of each pixel of the original image to obtain the color information of the second color space of each pixel.

[0115] Specifically, the server or terminal for image processing can convert the color information of each pixel of the original image in the first color space into the color information of each pixel in the second color space according to a preset color space conversion formula.

[0116] The first color space may be an RGB primary color space, and the second color space may be a YCbCr color space (Y indicates brightness, Cb and Cr indicate concentration offsets of blue and red, respectively). Specifically, the color space conversion formula is as follows:

[0117]

[0118] In the above formula (1), R, G, and B are the color information of each channel in the RGB primary color space; Y, Cb, and Cr are the color information of each channel in the YCbCr color space; and the above space conversion matrix can be determined by experimental values ​​in actual applications.

[0119] (2) Determine at least one gray point based on the color information of each pixel in the second color space.

[0120] Specifically, the server or terminal for image processing may set gray point screening conditions according to the color information of each pixel in the second color space, and screen each pixel according to the screening conditions to obtain at least one gray point.

[0121] The gray point screening conditions can be as follows:

[0122]

[0123] The above Y min 、Y max and is the threshold value, which can be obtained based on actual application tests.

[0124] In some implementations of the embodiments of the present application, each pixel of the original image may be detected point by point based on the above method, and subsequent processing may be performed based on all gray points obtained by the detection;

[0125] In other implementations of the present application, the original image can be segmented to obtain multiple sub-images, and the gray points corresponding to each sub-image can be determined; the sub-images and gray points have a one-to-one correspondence. Image segmentation can reduce pixel computational redundancy and improve image processing efficiency. The specific image segmentation processing method will be described in detail below.

[0126] An embodiment of the present application provides a possible implementation method, when the original image includes multiple sub-images without overlapping areas;

[0127] The above-mentioned determining at least one gray point based on the color information of the second color space of each pixel point includes:

[0128] a. Screening out candidate gray points from each pixel according to the color information of the second color space of each pixel.

[0129] Specifically, the server or terminal for image processing can use a preset gray point screening condition to screen each pixel according to the color information of the second color space of each pixel to obtain a candidate gray point. The specific screening condition is shown in formula (2) above and will not be repeated here.

[0130] b. Determine the gray point corresponding to each sub-image based on the candidate gray points included in each sub-image.

[0131] Specifically, such as Figure 3 As shown, the server or terminal for image processing can average the RGB color data of each candidate gray point included in each sub-image to obtain a gray point corresponding to the sub-image. The detailed steps will be explained in detail below:

[0132] In an embodiment of the present application, a possible implementation method is provided. The above-mentioned determination of the gray points corresponding to each sub-image based on the candidate gray points included in each sub-image includes:

[0133] b1. For each sub-image, determine the average pixel of the sub-image based on the average value of the color information of each candidate gray point in the sub-image in the first color space;

[0134] b2. The average pixel point of each sub-image is used as the gray point corresponding to each sub-image.

[0135] In an embodiment of the present application, a terminal or server used for image processing can obtain the RGB three-channel color data of all candidate gray points included in each sub-image, and then calculate the average value of the above RGB three-channel color data based on the number of candidate gray points in the sub-image, and use the above average value as the color information of the gray point corresponding to the sub-image.

[0136] In an embodiment of the present application, a possible implementation method is provided. In step S203, the white balance gain value of the original image is calculated based on the gray point clusters and the color temperature information of the gray points, including:

[0137] (1) For each gray point cluster, the local gain value of the gray point cluster is calculated according to the color temperature information of each gray point in the gray point cluster.

[0138] Specifically, a server or terminal used for image processing can calculate the color temperature of each gray point based on the color information of the first color space of each gray point, namely the RGB value; and then calculate the local gain value of the gray point cluster based on the color temperature information of each gray point in the gray point cluster.

[0139] In some embodiments, a server or terminal used for image processing may convert the color information of each gray point in the first color space into the CIE1931xyY color space. Since many natural and artificial light sources have chromaticities that coincide with or are very close to specific chromaticities on the Planckian curve, and in the CIE1931xyY color space, each color temperature has a unique pair of x and y chromaticity coordinates on the Planckian curve, the light source information, i.e., the color temperature information, of each pixel can be determined by referring to the Planckian color temperature.

[0140] First, the chromaticity coordinates x and y in the CIE1931xyY chromaticity space can be obtained by the following chromaticity space conversion formula, where the chromaticity space conversion formula is as follows:

[0141]

[0142]

[0143] Y is the brightness information of the corresponding gray point; R, G, and B are the three primary color data of the corresponding gray point.

[0144] Secondly, the color temperature of each gray point can be calculated using the color temperature calculation formula (5).

[0145]

[0146] The calculation parameters in formula (5) can be measured by color temperature conversion experiments and can have the following values:

[0147] n=(xx e ) / (yye )

[0148] x e =0.3366

[0149] y e =0.1735

[0150] A0=-949.86315

[0151] A1=6253.80338

[0152] A2=28.70599

[0153] A3=0.00004

[0154] t1=0.92159

[0155] t2=0.20039

[0156] t3=0.07125

[0157] An embodiment of the present application provides a possible implementation method, in which, for each gray point cluster, the local gain value of the gray point cluster is calculated based on the color temperature information of each gray point in the gray point cluster, including:

[0158] a. For each gray point cluster, a histogram is constructed based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point.

[0159] In the embodiment of the present application, in order to generate an accurate and easily identifiable color temperature sampling curve based on the histogram, the color temperature T can be converted to the MIRED (micro-inverted) scale, that is, the conversion of the following formula (6) is achieved:

[0160]

[0161] At the same time, the above T Mired As the horizontal coordinate of the histogram, the contribution value Y of each gray point is calculated according to the brightness value Y of the gray point. a , change Y a As the vertical coordinate of the histogram. Wherein, a is a positive integer, and in this embodiment, the value can be a=3 according to the actual application. Based on the above values, the pixel points with high brightness in the original image can be focused on based on the histogram, and the influence of the pixel points with low brightness can be limited. For example, based on a gray point cluster, it can be determined that there are 40 groups of gray points in a histogram, that is, bins=40, the horizontal axis range of the histogram is 2000K-10000K, and the group interval is 200K; the vertical axis of the histogram is the total contribution value of each color temperature range, which is equal to the weighted cumulative sum of the contribution of each gray point falling into the color temperature range.

[0162] b. Calculate the local gain value of the gray point cluster based on the histogram.

[0163] An embodiment of the present application provides a possible implementation method, wherein the above-mentioned calculation of the local gain value of the gray point cluster based on the histogram includes:

[0164] b1. Determine the largest column vector in the histogram.

[0165] Specifically, the gray point group with the highest cumulative sum of contribution values ​​of each gray point, that is, the maximum column vector, can be obtained from each group of bins in the histogram.

[0166] b2. Calculate the local gain value of the gray point cluster according to the color information of the first color space of each gray point included in the maximum column vector.

[0167] An embodiment of the present application provides a possible implementation method, wherein the above-mentioned calculation of the local gain value of the gray point cluster based on the color information of the first color space of each gray point included in the maximum column vector includes:

[0168] b21. Taking the average value of the color information of the first color space of each gray point included in the maximum column vector as the light source information of the gray point cluster.

[0169] The light source information of the gray point cluster can represent the color temperature type corresponding to the gray point cluster.

[0170] In the embodiment of the present application, the light source information of the gray point cluster may include the color values ​​of the three channels R_weight, G_weight, and B_weight, which can be calculated according to the following formula (7):

[0171]

[0172] Among them, R i , G i 、B i are the R, G, and B values ​​of the i-th gray point in the maximum column vector of the histogram, M is the number of gray points in the corresponding gray point cluster, and M is a positive integer.

[0173] When the original image is divided into blocks, count i is the number of candidate gray points obtained by the sub-image block corresponding to the i-th gray point in the gray point detection stage; when the original image is not divided into blocks, count i =1.

[0174] b22. Determine the local gain value of the gray point cluster based on the proportion of each color channel in the light source information.

[0175] In the embodiment of the present application, the local gain value of each gray point cluster includes the white balance gain of the red channel and the white balance gain of the blue channel, which can be calculated by the following formula:

[0176] R_gain j =G_weight / R_weight; (8)

[0177] B_gain j =G_weight / B_weight (9)

[0178] Where j is a positive integer indicating the number of gray point clusters. j B_gain is the white balance gain of the red channel corresponding to the jth gray point cluster. j is the white balance gain of the blue channel corresponding to the jth gray point cluster.

[0179] (2) Calculate the white balance gain value of the original image based on the local gain value of each gray point cluster.

[0180] In an embodiment of the present application, a possible implementation method is provided, wherein the white balance gain value of the original image is calculated based on the local gain value of each gray point cluster, including:

[0181] a. Determine the weight of each gray point cluster according to the number of gray points included in each gray point cluster.

[0182] In the embodiment of the present application, the weight of each gray point cluster can be calculated based on the ratio of the number of gray points included in each gray point cluster to the total number of all gray points in the original image;

[0183] b. Weight the local gain value of each gray point cluster according to the weight to obtain the white balance gain value of the original image.

[0184] In the embodiment of the present application, the white balance gain value of the original image may include a red channel gain value R_gain and a blue channel gain value B_gain, which can be calculated by the following formula:

[0185]

[0186]

[0187]

[0188] Among them, w j is the weight of the jth gray point cluster, R is the total number of gray point clusters, R_gain j B_gain is the white balance gain of the red channel corresponding to the jth gray point cluster. jis the white balance gain of the blue channel corresponding to the jth gray point cluster.

[0189] In the embodiment of the present application, after the white balance gain value of the original image is calculated, the white balance gain value can be used to adjust the white balance of the original image pixel by pixel. The adjustment method is as follows:

[0190] When the white balance gain values ​​of the original image include the red channel gain value R_gain and the blue channel gain value B_gain, the RGrGbB color data of the original image can be corrected using the above gain values. Because the R, G, and B color channels are independent of each other, the white balance of the original image can be adjusted based on the following correction formula:

[0191]

[0192] RGrGbB is the original color data of each pixel in the original image, and R′Gr′Gb′B′ is the color data of each pixel after white balance adjustment.

[0193] In order to better understand the above image processing method, Figure 4 and Figure 5 An example of an image processing method of the present application is described in detail, and the method includes the following steps:

[0194] S501: Obtain coded data of an original image through an image acquisition device.

[0195] The encoding data of the original image can be Bayer format data.

[0196] S502 , using a linear interpolation method to obtain the three primary color information of each pixel, namely, RGB data, based on the Bayer format data of the original image.

[0197] S503 , performing color gamut conversion on the RGB color information of each pixel of the original image to obtain YCbCr color information of each pixel.

[0198] S504 , screening candidate gray points from each pixel point according to the YCbCr color information of each pixel point.

[0199] S505: Divide the original image into blocks to obtain multiple sub-images without overlapping areas.

[0200] S506 , for each sub-image, determine the average pixel point of the sub-image according to the average value of the RGB color information of each candidate gray point in the sub-image; and use the average pixel point of each sub-image as the gray point corresponding to each sub-image.

[0201] S507 , classifying each gray point according to the RGB color information of each gray point to obtain a plurality of gray point clusters.

[0202] S508 : For each gray point cluster, construct a histogram based on each gray point in the gray point cluster.

[0203] The abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point.

[0204] S509, calculating the local gain value of the gray point cluster according to the RGB color information of each gray point included in the maximum column vector in the histogram;

[0205] S510 , determining the weight of each gray point cluster according to the number of gray points included in each gray point cluster, and weighting the local gain value of each gray point cluster according to the weight to obtain a white balance gain value of the original image.

[0206] The embodiment of the present application classifies each gray point based on the color information of at least one gray point in the original image to obtain multiple gray point clusters corresponding to the original image; wherein the gray point is a pixel point representing a neutral color in the original image, which can indicate the brightness information of the original image. Then, the white balance gain value of the original image is calculated based on the color temperature information of each gray point cluster and each gray point, so that the white balance of the original image is adjusted according to the white balance gain value to obtain the target image. The embodiment of the present application realizes the calculation of the white balance gain value of the image based on the gray point. Compared with the prior art method of obtaining the white balance gain value based on the coarse-grained color temperature area of ​​the image, the color information and color temperature information of each gray point are fully considered in the present application. The calculated white balance gain is more accurate and the reliability of the white balance adjustment is improved. At the same time, the present application can identify the color temperature type of each gray point based on the clustering of gray point color information, that is, the gray point cluster, to achieve white balance adjustment based on mixed color temperature scenes, thereby improving the robustness of the white balance correction effect.

[0207] The present application provides an image processing device, such as Figure 6 As shown, the image processing device 60 may include: a determination module 601, a classification module 602, and a calculation module 603;

[0208] The determining module 601 is configured to determine at least one gray point in the original image; a gray point is a pixel representing a neutral color in the original image;

[0209] A classification module 602 is configured to classify each gray point according to color information of the first color space of each gray point to obtain a plurality of gray point clusters;

[0210] The calculation module 603 is used to calculate the white balance gain value of the original image according to the gray point clusters and the color temperature information of the gray points, and perform white balance adjustment on the original image according to the white balance gain value to obtain a target image.

[0211] An embodiment of the present application provides a possible implementation method, in which the determination module 601, when determining at least one gray point in the original image, is configured to:

[0212] Performing color gamut conversion on the color information of the first color space of each pixel of the original image to obtain the color information of the second color space of each pixel;

[0213] At least one gray point is determined according to the color information of each pixel in the second color space.

[0214] An embodiment of the present application provides a possible implementation method, wherein the original image includes multiple sub-images without overlapping areas;

[0215] When determining at least one gray point based on the color information of each pixel in the second color space, the determination module 601 is configured to:

[0216] Screening candidate gray points from each pixel according to color information of the second color space of each pixel;

[0217] According to the candidate gray points included in each sub-image, the gray point corresponding to each sub-image is determined.

[0218] In an embodiment of the present application, a possible implementation is provided. When determining the gray point corresponding to each sub-image based on the candidate gray points included in each sub-image, the determination module 601 is configured to:

[0219] For each sub-image, determining an average pixel point of the sub-image according to an average value of color information of each candidate gray point in the sub-image in the first color space;

[0220] The average pixel point of each sub-image is used as the gray point corresponding to each sub-image.

[0221] In an embodiment of the present application, a possible implementation is provided. When the calculation module 603 calculates the white balance gain value of the original image based on the gray point clusters and the color temperature information of the gray points, it is configured to:

[0222] For each gray point cluster, the local gain value of the gray point cluster is calculated according to the color temperature information of each gray point in the gray point cluster;

[0223] The white balance gain value of the original image is calculated based on the local gain value of each gray point cluster.

[0224] In an embodiment of the present application, a possible implementation is provided. When the calculation module 603 calculates the white balance gain value of the original image based on the local gain value of each gray point cluster, it is configured to:

[0225] Determine the weight of each gray point cluster according to the number of gray points included in each gray point cluster;

[0226] The local gain value of each gray point cluster is weighted according to the weight to obtain the white balance gain value of the original image.

[0227] In an embodiment of the present application, a possible implementation is provided. When the calculation module 603 calculates the local gain value of each gray point cluster according to the color temperature information of each gray point in the gray point cluster, the calculation module 603 is configured to:

[0228] For each gray point cluster, a histogram is constructed based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point;

[0229] Calculate the local gain value of the gray point cluster based on the histogram.

[0230] In an embodiment of the present application, a possible implementation is provided. When the calculation module 603 calculates the local gain value of the gray point cluster according to the histogram, it is used to:

[0231] Determine the largest column vector in the histogram;

[0232] The local gain value of the gray point cluster is calculated according to the color information of the first color space of each gray point included in the maximum column vector.

[0233] In an embodiment of the present application, a possible implementation is provided. When the calculation module 603 calculates the local gain value of the gray point cluster based on the color information of the first color space of each gray point included in the maximum column vector, it is configured to:

[0234] The average value of the color information of the first color space of each gray point included in the maximum column vector is used as the light source information of the gray point cluster;

[0235] The local gain value of the gray point cluster is determined according to the proportion of each color channel in the light source information.

[0236] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.

[0237] The embodiment of the present application classifies each gray point based on the color information of at least one gray point in the original image to obtain multiple gray point clusters corresponding to the original image; wherein the gray point is a pixel point representing a neutral color in the original image, which can indicate the brightness information of the original image. Then, the white balance gain value of the original image is calculated based on the color temperature information of each gray point cluster and each gray point, so that the white balance of the original image is adjusted according to the white balance gain value to obtain the target image. The embodiment of the present application realizes the calculation of the white balance gain value of the image based on the gray point. Compared with the prior art method of obtaining the white balance gain value based on the coarse-grained color temperature area of ​​the image, the color information and color temperature information of each gray point are fully considered in the present application. The calculated white balance gain is more accurate and the reliability of the white balance adjustment is improved. At the same time, the present application can identify the color temperature type of each gray point based on the clustering of gray point color information, that is, the gray point cluster, to achieve white balance adjustment based on mixed color temperature scenes, thereby improving the robustness of the white balance correction effect.

[0238] In an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of an image processing method. Compared to related art methods, the present embodiment can achieve the following: Based on the color information of at least one gray point in an original image, the present embodiment classifies each gray point to obtain multiple gray point clusters corresponding to the original image; wherein the gray point is a pixel representing a neutral color in the original image, which can indicate the brightness information of the original image. Then, based on the color temperature information of each gray point cluster and each gray point, a white balance gain value of the original image is calculated, so that the original image is white-balanced according to the white balance gain value to obtain a target image. The embodiment of the present application realizes the calculation of the white balance gain value of the image based on the gray points. Compared with the existing technology of obtaining the white balance gain value based on the coarse-grained color temperature area of ​​the image, the present application fully considers the color information and color temperature information of each gray point, and the calculated white balance gain is more accurate, thereby improving the reliability of the white balance adjustment. At the same time, the present application can identify the color temperature type of each gray point based on the clustering of gray point color information, that is, the gray point cluster, to achieve white balance adjustment based on the mixed color temperature scene, thereby improving the robustness of the white balance correction effect.

[0239] In an alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7The electronic device 700 shown includes: a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the electronic device 700 may further include a transceiver 704, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 704 is not limited to one, and the structure of the electronic device 700 does not constitute a limitation on the embodiments of the present application.

[0240] The processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 701 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0241] The bus 702 may include a path for transmitting information between the above components. The bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 702 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0242] The memory 703 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0243] The memory 703 is used to store the computer program for executing the embodiments of the present application, and the execution is controlled by the processor 701. The processor 701 is used to execute the computer program stored in the memory 703 to implement the steps shown in the above method embodiments.

[0244] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, notebook computers, PADs, etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0245] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0246] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that when the computer device executes the computer instructions, the following conditions are achieved:

[0247] Determine at least one gray point in the original image; the gray point is a pixel point representing a neutral color in the original image;

[0248] Classifying each gray point according to color information of the first color space of each gray point to obtain multiple gray point clusters;

[0249] The white balance gain value of the original image is calculated according to the gray point cluster and the color temperature information of the gray points, and the white balance of the original image is adjusted according to the white balance gain value to obtain the target image.

[0250] The terms "first," "second," "third," "fourth," "1," "2," and the like (if any) in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than that shown or described in the drawings.

[0251] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the order of implementation of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times respectively. Under different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0252] The above description is only an optional implementation method for some implementation scenarios of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of this application, the use of other similar implementation methods based on the technical ideas of this application also falls within the protection scope of the embodiments of this application.

Claims

1. An image processing method, characterized in that: include: Determine at least one gray point in the original image; The gray points are pixels representing neutral colors in the original image; classifying each gray point according to color information of the first color space of each gray point to obtain a plurality of gray point clusters; For each gray point cluster, a histogram is constructed based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point; Determining a maximum column vector in the histogram; using an average value of color information in the first color space of each gray point included in the maximum column vector as light source information of the gray point cluster; and determining a local gain value of the gray point cluster based on a proportion of each color channel in the light source information; The white balance gain value of the original image is calculated according to the local gain value of each gray point cluster, and the white balance adjustment is performed on the original image according to the white balance gain value to obtain a target image.

2. The method according to claim 1, characterized in that The determining of at least one gray point in the original image includes: Performing color gamut conversion on the color information of the first color space of each pixel of the original image to obtain color information of the second color space of each pixel; At least one gray point is determined according to the color information of the second color space of each pixel.

3. The method according to claim 2, characterized in that The original image includes a plurality of sub-images without overlapping areas; and determining at least one gray point based on color information of the second color space of each pixel point includes: Screening candidate gray points from each pixel according to color information of the second color space of each pixel; According to the candidate gray points included in each sub-image, the gray points corresponding to the sub-images are determined.

4. The method according to claim 3, characterized in that The step of determining the gray points corresponding to the sub-images according to the candidate gray points included in the sub-images includes: For each sub-image, determining an average pixel point of the sub-image according to an average value of color information of each candidate gray point in the sub-image in the first color space; The average pixel point of each sub-image is used as the gray point corresponding to each sub-image.

5. The method according to claim 1, wherein Calculating the white balance gain value of the original image according to the local gain value of each gray point cluster includes: determining a weight of each gray point cluster according to the number of gray points included in each gray point cluster; The local gain value of each gray point cluster is weighted according to the weight to obtain a white balance gain value of the original image.

6. An image processing device, characterized in that include: A determination module, configured to determine at least one gray point in the original image; The gray points are pixels representing neutral colors in the original image; a classification module, configured to classify each gray point according to color information of the first color space of each gray point to obtain a plurality of gray point clusters; a calculation module, configured to construct a histogram for each gray point cluster based on each gray point in the gray point cluster; wherein the abscissa of the histogram represents the color temperature information of each gray point, and the ordinate of the histogram represents the brightness information of each gray point; Determining a maximum column vector in the histogram; using an average value of color information in the first color space of each gray point included in the maximum column vector as light source information of the gray point cluster; and determining a local gain value of the gray point cluster based on a proportion of each color channel in the light source information; The white balance gain value of the original image is calculated according to the local gain value of each gray point cluster, and the white balance adjustment is performed on the original image according to the white balance gain value to obtain a target image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images