Image white balance correction method, apparatus, storage medium, and electronic device

Abstract

This disclosure relates to a method, apparatus, storage medium, and electronic device for correcting the white balance of an image. [Solution] The first quantum efficiency response curve of a reference image sensor is determined, the second quantum efficiency response curve of a target image sensor whose color filter array differs from that of the reference image sensor is determined, the target matrix used for converting between the second quantum efficiency response curve and the first quantum efficiency response curve is determined, the first white balance plank curve of the target image sensor is determined, white balance correction parameters suitable for the target image sensor are determined based on the target matrix, the first white balance plank curve and the first image captured by the target image sensor, and white balance correction processing is performed on the second image captured by the target image sensor according to the white balance correction parameters.

Description

Technical Field

[0001] The present disclosure relates to machine vision technology, and particularly to a method, apparatus, storage medium, and electronic device for white balance correction of an image.

Background Art

[0002] Currently, image sensors are widely used in various fields. In some cases, it is necessary to perform white balance correction processing on the images captured by the image sensors. In the conventional technology, an image sensor using a color filter array cannot achieve a good processing effect of white balance correction. For example, an image sensor using an RCCG array may cause color deviation in the image after white balance correction processing. Here, R refers to red, C refers to transmissive transmission, and G refers to green.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, apparatus, storage medium, and electronic device for white balance correction of an image.

Means for Solving the Problems

[0004] The method for white balance correction of an image according to the present disclosure includes: determining a first quantum efficiency response curve of a reference image sensor; determining a second quantum efficiency response curve of a target image sensor, wherein the color filter array of the target image sensor is different from that of the reference image sensor; determining a target matrix used for conversion between the second quantum efficiency response curve and the first quantum efficiency response curve; The steps include determining the first white balance plank curve of the target image sensor, The steps include determining white balance correction parameters suitable for the target image sensor based on the target matrix, the first white balance plank curve, and the first image captured by the target image sensor, The process includes the step of performing a white balance correction process on a second image captured by the target image sensor according to the white balance correction parameters.

[0005] The white balance correction device for the images related to this disclosure is: A first determinative module for determining the first quantum efficiency response curve of a reference image sensor, A second deterministic module for determining the second quantum efficiency response curve of a target image sensor that is different from the reference image sensor, the color filter array, A third deterministic module for determining the target matrix used in the conversion between the second quantum efficiency response curve determined by the second deterministic module and the first quantum efficiency response curve determined by the first deterministic module, A fourth determination module for determining the first white balance plank curve of the target image sensor, A fifth determinator module for determining white balance correction parameters suitable for the target image sensor, based on the target matrix determined by the third determinator module, the first white balance plank curve determined by the fourth determinator module, and the first image captured by the target image sensor. The system includes a processing module for performing white balance correction processing on a second image captured by the target image sensor according to the white balance correction parameters determined by the fifth determination module.

[0006] In this disclosure, a computer-readable storage medium stores a computer program that performs the white balance correction method for the image.

[0007] The electronic device relating to this disclosure comprises a processor and a memory that stores instructions that the processor can execute, and the processor performs the image white balance correction method by reading and executing instructions that it can execute from the memory.

[0008] In the computer program product relating to this disclosure, when an instruction in the computer program product is executed by a processor, the image white balance correction method described above is performed. [Effects of the Invention]

[0009] According to the image white balance correction method, apparatus, storage medium, and electronic device described herein, an image sensor employing a second type of color filter array is used as a reference image sensor, and an image sensor employing a first type of color filter array is used as a target image sensor. A target matrix used for the conversion between the respective quantum efficiency response curves of the reference image sensor and the target image sensor is determined. Next, a white balance correction parameter suitable for the target image sensor is determined by combining the first white balance Planck curve of the target image sensor with the first image captured by the target image sensor. As a result, white balance correction processing can be performed on the second image captured by the target image sensor according to the white balance correction parameter suitable for the target image sensor. Even if the white balance correction method for the reference image sensor cannot be applied to the target image sensor due to a significant difference between the imaging characteristics of the target image sensor and the imaging characteristics of the reference image sensor, a reasonably dedicated white balance correction parameter can be determined for the target image sensor, and this parameter can be used for white balance correction processing of the second image captured by the target image sensor. In this way, by employing a new, suitable white balance correction method for the image captured by the image sensor of the first type of color filter array, a good white balance correction processing effect can be achieved. [Brief explanation of the drawing]

[0010] [Figure 1] This is a flowchart illustrating the image white balance correction method according to the embodiment of this disclosure. [Figure 2] This is a flowchart illustrating a method for determining white balance correction parameters suitable for a target image sensor based on a target matrix, a first white balance plank curve, and a first image captured by a target image sensor according to an embodiment of the present disclosure. [Figure 3] This is a flowchart illustrating a method for determining white balance correction parameters suitable for a target image sensor based on the target matrix, second white balance plank curve, and target gray zone according to the embodiments of this disclosure. [Figure 4] This is a flowchart illustrating a method for determining the weights corresponding to each of the multiple subblocks based on the second white balance plank curve according to the embodiment of this disclosure and the positions corresponding to each of the multiple subblocks. [Figure 5] This is a flowchart illustrating a method for determining the weights corresponding to multiple subblocks based on a second white balance plank curve according to another embodiment of the present disclosure and the positions corresponding to each of the multiple subblocks. [Figure 6] This is a flowchart illustrating a method for determining the positions corresponding to each of the multiple subblocks in the coordinate system where the second white balance Planck curve is located, based on the target matrix and the color information of each of the multiple subblocks according to the embodiment of this disclosure. [Figure 7] This is a flowchart illustrating a method for determining white balance correction parameters suitable for a target image sensor based on the weighted position and the inverse matrix of the target matrix according to an embodiment of the present disclosure. [Figure 8-1] This diagram limits the distribution of multiple points that make up the white balance Planck curve of an RCCG image sensor in conventional technology. [Figure 8-2] This figure limits the distribution of multiple points that constitute the white balance Planck curve of an RCCG image sensor, which is similar to the RGGB characteristics in the embodiments of this disclosure. [Figure 9] This figure shows the structure of an image white balance correction device according to an embodiment of the present disclosure. [Figure 10] This figure shows the structure of the fifth definitive module of the image white balance correction device according to the embodiment of the present disclosure. [Figure 11] This figure shows the structure of the second definitive submodule of the image white balance correction device according to an embodiment of the present disclosure. [Figure 12] It is a diagram showing the structure of the second determination unit of the white balance correction device for an image according to an embodiment of the present disclosure. [Figure 13] It is a diagram showing the structure of the fourth determination unit of the white balance correction device for an image according to an embodiment of the present disclosure. [Figure 14] It is a diagram showing the structure of the third determination unit of the white balance correction device for an image according to an embodiment of the present disclosure. [Figure 15] It is a diagram showing the structure of the third determination unit of the white balance correction device for an image according to another embodiment of the present disclosure. [Figure 16] It is a diagram showing the structure of an electronic device according to an embodiment of the present disclosure. Aspects for implementing the invention

[0011] Hereinafter, for the purpose of interpreting the present disclosure, exemplary embodiments of the present disclosure will be described in detail while referring to the drawings. The following embodiments are only some of the embodiments of the present disclosure, not all of the embodiments. In other words, the present disclosure is not limited only to the following exemplary embodiments.

[0012] It should be noted that unless otherwise specifically explained, the arrangement, numerical expressions, and numerical values of the components and steps described in the following embodiments are not limited only to the disclosure of the present disclosure.

[0013] [Summary of the application] Image sensors are currently used in a wide range of fields. Images captured by an image sensor (hereinafter also referred to as "images") can have color by using the image sensor's color filter array (abbreviated as CFA). The color filter array can include, but is not limited to, RCCG arrays, RCCB arrays, RGGB arrays, etc. Similarly, image sensors can include, but are not limited to, RCCG image sensors, RCCB image sensors, RGGB image sensors, etc. R refers to red, C refers to transparent transmission, G refers to green, and B refers to blue.

[0014] In the process of processing images captured by a target image sensor using an Image Signal Processor (ISP) system, it is common practice to perform white balance correction processing on the captured image. White balance can be understood as the balance of white; that is, by adjusting the image so that white target objects in the real world are reproduced as white in the image, the image more accurately reflects the color of the target object. Algorithms for performing white balance correction processing may include, but are not limited to, the gray world assumption method, the perfect reflector assumption method, and the dynamic thresholding method.

[0015] Conventional technologies have a challenge in achieving a good white balance correction effect on images captured by image sensors employing specific color filter arrays. For example, images captured by image sensors employing an RCCG array may exhibit red or purple color bias after white balance correction processing. Therefore, determining how to perform white balance correction processing on images captured by image sensors employing color filter arrays to achieve a better white balance correction effect is a crucial issue for those skilled in the art.

[0016] [Exemplary System] In conventional technology, when white balance correction processing is performed on an image sensor employing a specific color filter array (hereinafter referred to as "Type 1 color filter array"), a good white balance correction effect cannot be achieved. However, it has been confirmed that when white balance correction processing is performed on an image sensor employing a different specific color filter array (hereinafter referred to as "Type 2 color filter array"), a good white balance correction effect can be achieved. For example, when white balance correction processing is performed on an image sensor employing an RCCG array, a good white balance correction effect cannot be achieved. However, when white balance correction processing is performed on an image sensor employing an RGGB array, a good white balance correction effect can be achieved. An important finding revealed by the inventor's research is that the reason why a good white balance correction effect cannot be achieved by white balance correction processing an image sensor employing an RCCG array is that the image sensor employing an RCCG array uses the same white balance correction method as the image sensor employing an RGGB array. In other words, because there are significant differences in imaging properties between image sensors employing an RCCG array and those employing an RGGB array, the white balance correction method corresponding to an image sensor employing an RGGB array cannot be directly applied to images captured by an image sensor employing an RCCG array.

[0017] In view of this, in the embodiments of this disclosure, an image sensor using a second type of color filter array is used as a reference image sensor, and an image sensor using a first type of color filter array is used as a target image sensor, and a target matrix used for the conversion between the respective quantum efficiency response curves of the reference image sensor and the target image sensor is determined. Next, a white balance compensation parameter suitable for the target image sensor is determined by combining the first white balance Planck curve of the target image sensor and the first image captured by the target image sensor, and this parameter is used for white balance correction processing of the second image captured by the target image sensor. In this way, by employing a new suitable white balance correction method for an image captured by an image sensor employing a first type of color filter array, a good white balance correction processing effect can be achieved.

[0018] [Exemplary method] Figure 1 is a flowchart of an image white balance correction method according to an embodiment of the present disclosure. The method shown in Figure 1 includes steps 110, 120, 130, 140, 150, and 160.

[0019] In step 110, the first quantum efficiency (QE) response curve of the reference image sensor is determined.

[0020] In the selectable examples of this disclosure, a reference image sensor can be used as a reference image sensor to help obtain white balance correction parameters that are compatible with other image sensors. The color filter array of the reference image sensor may belong to a second type of color filter array. For example, the color filter array of the reference image sensor may be an RGGB array, in which case the reference image sensor may be an RGGB image sensor. The white balance correction parameters may be parameters used in the white balance correction process.

[0021] In the optional examples of this disclosure, the first quantum efficiency response curve of the reference image sensor may be a curve with wavelength as the x-axis and response probability as the y-axis. In the examples of this disclosure, the value range of the wavelength may be 400 nm to 700 nm. The value range of the response probability may be 0% to 100%. The first quantum efficiency response curve of the reference image sensor may also be called the QE data of the reference image sensor.

[0022] In step 120, the second quantum efficiency response curve of the target image sensor is determined. The color filter array of the target image sensor is different from the color filter array of the reference image sensor.

[0023] In the selectable examples of this disclosure, the target image sensor may be an image sensor for which a suitable white balance correction parameter needs to be determined. The color filter array of the target image sensor may belong to a first type of color filter array. The color filter array of the target image sensor is different from an RGGB array. That is, the color filter array of the target image sensor is not an RGGB array. In this case, the target image sensor is also not an RGGB image sensor. In the examples, the color filter array of the target image sensor may be an RCCG array or an RCCB array. In this case, the target image sensor may also be an RCCG image sensor or an RCCB image sensor.

[0024] In the optional examples of this disclosure, the second quantum efficiency response curve of the target image sensor may be a curve with wavelength as the horizontal coordinate and response probability as the vertical coordinate. The second quantum efficiency response curve of the target image sensor may also be called the QE data of the target image sensor.

[0025] In step 130, the target matrix used for the conversion between the second quantum efficiency response curve and the first quantum efficiency response curve is determined.

[0026] In the optional examples of this disclosure, the second quantum efficiency response curve and the first quantum efficiency response curve may be two curves plotted in the same coordinate system. In this case, by using least squares or other regression algorithms, a matrix used for transforming various curves can be determined, and this determined matrix can be designated as the target matrix. The target matrix may also be called the QG correction matrix. In the examples of this disclosure, the size of the target matrix may be 3*3; that is, the number of rows and columns of the target matrix are both 3.

[0027] In step 140, the first white balance plank curve of the target image sensor is determined.

[0028] In the selectable examples of this disclosure, the first white balance Planck curve of the target image sensor may be a curve with the x-coordinate being a ratio of type 1 and the y-coordinate being a ratio of type 2. Both the ratio of type 1 and the ratio of type 2 are ratios representing the color difference between channels.

[0029] For example, if the color filter array of the target image sensor is an RCCG array, the first type ratio value is the ratio of the pixel value of the R channel to the pixel value of the C channel, and the second type ratio value is the ratio of the pixel value of the G channel to the pixel value of the C channel.

[0030] For example, if the color filter array of the target image sensor is an RCCB array, the ratio of the first kind is the ratio of the pixel value of the R channel to the pixel value of the C channel, and the ratio of the second kind is the ratio of the pixel value of the B channel to the pixel value of the C channel.

[0031] In step 150, white balance correction parameters suitable for the target image sensor are determined based on the target matrix, the first white balance Planck curve, and the first image captured by the target image sensor.

[0032] In the optional examples of this disclosure, the first image may be an original image obtained by capturing an image of a pre-defined object. The pre-defined object may include, but is not limited to, a 24-color color card, a gray card, etc. This ensures that the first image includes a gray area. The gray area can be understood as an area that appears white, black, or gray. The original image may also be called raw data.

[0033] In the selectable examples of this disclosure, a white balance correction parameter adapted to a target image sensor means a parameter adapted to the white balance correction processing of an image captured by the target image sensor. The parameter guides inter-channel color correction so that white target objects in the real world (physical world) appear white in the image.

[0034] In step 160, white balance correction is performed on the second image captured by the target image sensor according to the white balance correction parameters.

[0035] In the optional examples of this disclosure, the second image may be a source image obtained by capturing an image of a pre-defined object, or a source image obtained by capturing an image of an object other than a pre-defined object. The second image may be the same as the first image, or it may be a different image.

[0036] Here, the second image can be raw data obtained by the target image sensor capturing an image. In that case, white balance correction processing can be directly performed on the raw data of the second image according to the white balance correction parameters. Alternatively, the raw data as the second image can be converted into an image that is easier for the user to observe (non-raw data), and then white balance correction processing can be performed on the converted image.

[0037] In the embodiments of this disclosure, an image sensor employing a second type of color filter array is used as a reference image sensor, and an image sensor employing a first type of color filter array is used as a target image sensor. A target matrix used for the conversion between the respective quantum efficiency response curves of the reference image sensor and the target image sensor is determined. Next, a white balance correction parameter suitable for the target image sensor is determined by combining the first white balance Planck curve of the target image sensor with the first image captured by the target image sensor. Therefore, white balance correction processing can be performed on the second image captured by the target image sensor according to the white balance correction parameter suitable for the target image sensor. Even if the white balance correction method of the reference image sensor cannot be applied to the target image sensor due to a significant difference between the imaging characteristics of the target image sensor and the imaging characteristics of the reference image sensor, a reasonably dedicated white balance correction parameter can be determined for the target image sensor, and this parameter can be used for white balance correction processing on the second image captured by the target image sensor. In this way, by employing a new suitable white balance correction method for the image captured by the image sensor employing a first type of color filter array, a good white balance correction processing effect can be achieved.

[0038] Figure 2 is a flowchart illustrating a method for determining white balance correction parameters suitable for a target image sensor, based on a target matrix, a first white balance Planck curve, and a first image captured by the target image sensor according to an embodiment of the present disclosure. The method in Figure 2 may include steps 210, 220, and 230.

[0039] In step 210, the target matrix is ​​used to convert the first white balance plank curve to the second white balance plank curve of the reference image sensor.

[0040] As described above, the target matrix can be used to convert between the second quantum efficiency response curve and the first quantum efficiency response curve. This allows for two possible scenarios for the target matrix: Scenario 1 is when the target matrix is ​​used to map the second quantum efficiency response curve to the first quantum efficiency response curve. Scenario 2 is when the target matrix is ​​used to map the first quantum efficiency response curve to the second quantum efficiency response curve. For ease of understanding, Scenario 1 will be explained below as an example.

[0041] In the selectable examples of this disclosure, if the first white balance Planck curve is M1, the second white balance Planck curve of the reference image sensor is M2, and the target matrix is ​​MQe, then M2 can be obtained by the following equation. [Mathematical formula] M2 = M1 * MQe

[0042] It should be noted that the second white balance plank curve of the reference image sensor is not the actual white balance plank curve of the reference image sensor, but rather the white balance plank curve of the reference image sensor that was inferred from the target matrix and the first white balance plank curve.

[0043] In step 220, the target gray area in the first image captured by the target image sensor is determined.

[0044] In the optional examples of this disclosure, user input can be received, and the gray zone selected by the user input in the first image can be designated as the target gray zone. User input may include, but is not limited to, touch input, keyboard input, mouse input, etc.

[0045] In the examples of this disclosure, the first image may be a source image obtained by capturing an image on a 24-color card. The first image may contain a total of 24 color blocks arranged in 4 rows and 6 columns. The gray zone selected by the user's input operation in the first image may include the 6 color blocks located in the last row of the 24 color blocks. That is, the target gray zone may include the 6 color blocks located in the last row of the 24 color blocks.

[0046] In one embodiment, the target gray zone may not be determined by user input but by an algorithm. For example, a neural network model for identifying gray zones from images can be pre-trained, and the target gray zone can be identified from a first image using this pre-trained neural network model.

[0047] In step 230, the white balance correction parameters suitable for the target image sensor are determined based on the target matrix, the second white balance Planck curve, and the target gray zone.

[0048] In the optional examples of this disclosure, the color information of the target gray zone is determined. Assuming that the target gray zone includes the six color blocks located in the last row of 24 color blocks, the color information of the target gray zone may include the color information of each of the six color blocks located in the last row of 24 color blocks. Next, based on the target matrix, the second white balance plank curve, and the color information of the target gray zone, white balance correction parameters suitable for the target image sensor are determined.

[0049] It should be noted that the first quantum efficiency response curve can influence the imaging characteristics of the reference image sensor, and the second quantum efficiency response curve can influence the imaging characteristics of the target image sensor. The target matrix used in the conversion between the second and first quantum efficiency response curves can represent the relationship between the imaging characteristics of the target image sensor and the reference image sensor. Since the first white balance Planck curve belongs to the imaging characteristics of the target image sensor, the relationship between imaging characteristics represented by the target matrix can be used to effectively infer the second white balance Planck curve for the reference image sensor. Based on the target matrix, the second white balance Planck curve, and the target gray zone of the first image captured by the target image sensor, the white balance correction parameters suitable for the target image sensor are determined. In the process of determining the white balance correction parameters, by effectively utilizing information regarding the imaging characteristics of the image sensor and ensuring the degree of fit between the white balance correction parameters and the target image sensor, the rationality and reliability of the determined white balance correction parameters can be ensured.

[0050] Figure 3 is a flowchart of a method for determining white balance correction parameters suitable for a target image sensor based on a target matrix, a second white balance plank curve, and a target gray zone according to an embodiment of the present disclosure. The method in Figure 3 may include steps 310, 320, 330, 340, and 350.

[0051] In step 310, the color information of each of the multiple sub-blocks included in the target gray zone is determined.

[0052] In the selectable examples of this disclosure, the multiple subblocks contained within the target gray zone can be N subblocks, where N can be an integer greater than or equal to 2. Referring to the example in the preamble, if the target gray zone contains six color blocks located in the last row of 24 color blocks, each of the six color blocks can be a subblock. In this case, N can be 6.

[0053] In the examples of this disclosure, the target image sensor is an RCCG image sensor, and the color information of each subblock in N subblocks may include pixel values ​​in the R channel, C channel, and G channel of the subblock. Assuming that the subblock contains U pixel dots, U1 pixel dots correspond to the R channel, U2 pixel dots correspond to the C channel, and U3 pixel dots correspond to the G channel. The average value of U1 pixel value that corresponds one-to-one with U1 pixel dot can be taken as the pixel value in the R channel of the subblock, the average value of U2 pixel values ​​that correspond one-to-one with U2 pixel dots can be taken as the pixel value in the C channel of the subblock, and the average value of U3 pixel values ​​that correspond one-to-one with U3 pixel dots can be taken as the pixel value in the G channel of the subblock.

[0054] The method for determining the pixel values ​​of the R channel, G channel, and B channel of the subblock is not limited to the method illustrated above. For example, interpolation can be performed on the first image to obtain a first full-resolution image corresponding to the R channel of the first image, a second full-resolution image corresponding to the C channel of the first image, and a third full-resolution image corresponding to the G channel of the first image. The first full-resolution image includes the pixel value in the R channel of each pixel dot in the first image, the second full-resolution image includes the pixel value in the C channel of each pixel dot in the first image, and the third full-resolution image includes the pixel value in the G channel of each pixel dot in the first image. U pixel values ​​corresponding one-to-one with U pixel dots in the subblock can be obtained from the first full-resolution image, and the obtained U pixel values ​​can be used as the pixel values ​​in the R channel of the subblock. U pixel values ​​corresponding one-to-one with U pixel dots in the subblock can be obtained from the second full-resolution image, and the obtained U pixel values ​​can be used as the pixel values ​​in the C channel of the subblock. From the third full-resolution image, U pixel values ​​corresponding one-to-one with U pixel dots in the subblock can be obtained, and the obtained U pixel values ​​can be used as the pixel values ​​in the G channel of the subblock.

[0055] In other examples, the target image sensor is an RCCB image sensor, and the color information of each of the N subblocks may include the pixel values ​​in the R channel, G channel, and B channel of the subblock. The method for determining the pixel values ​​in the R channel, G channel, and B channel of the subblock is described in the examples above and will not be explained in detail here.

[0056] In step 320, based on the target matrix and the color information of each of the multiple subblocks, the corresponding positions of each of the multiple subblocks are determined in the coordinate system in which the second white balance Planck curve is located.

[0057] In the optional examples of this disclosure, a second white balance Planck curve is drawn in a coordinate system in which the value of the first kind is the x-coordinate and the value of the second kind is the y-coordinate. This coordinate system is the coordinate system in which the second white balance Planck curve is located.

[0058] In the selectable examples of the present disclosure, for each of the N subblocks, the position corresponding to the subblock in the coordinate system in which the second white balance Planck curve is located is determined based on the target matrix and the color information of the subblock. The position corresponding to the subblock may be the position of a single point in the coordinate system in which the second white balance Planck curve is located.

[0059] In step 330, the weights corresponding to each of the subblocks are determined based on the second white balance plank curve and the positions corresponding to each of the subblocks.

[0060] In some embodiments of this disclosure, as shown in Figure 4, step 330 may include steps 410 and 420.

[0061] In step 410, distance information is determined between the position corresponding to each of the multiple subblocks and the second white balance Planck curve.

[0062] The position corresponding to each of the N subblocks represents the position of a single point in the coordinate system where the second white balance Planck curve is located. The distance between a single point in the coordinate system and the second white balance Planck curve can be determined according to a conventional method for calculating the distance from a point to a line, and this determined distance can be used as distance information between the position corresponding to the subblock and the second white balance Planck curve.

[0063] In step 420, the weights corresponding to each of the multiple subblocks are determined based on the distance information corresponding to each of the multiple subblocks.

[0064] In the optional examples of this disclosure, the independent variable can be set to distance information, the dependent variable to weights, and the objective function to have a negative correlation between the independent and dependent variables. For example, the objective function can be a linear function with a slope less than 0, an exponential function with a base number less than 1, and so on.

[0065] In the selectable examples of this disclosure, for each of the N subblocks, the distance information corresponding to the subblock can be substituted into the objective function as the range of the independent variable to obtain the corresponding range of the dependent variable, and the obtained range of the dependent variable can be used as the weights corresponding to the subblocks.

[0066] By relying on distance information between the positions corresponding to each subblock and the second white balance Planck curve, the weights corresponding to each subblock can be rationally determined. For example, the weights corresponding to each subblock can be determined according to the following rule: the closer the position corresponding to a subblock is to the second white balance Planck curve, the larger the weight corresponding to the subblock; and the further the position corresponding to a subblock is from the second white balance Planck curve, the smaller the weight corresponding to the subblock. By using the confirmed weights to determine the white balance correction parameters, the rationality and reliability of the white balance correction parameters can be ensured.

[0067] In other selectable embodiments of the present disclosure, as shown in Figure 5, step 330 may include steps 510, 520, and 530.

[0068] In step 510, a dilation process is performed on the second white balance Planck curve in the coordinate system where the second white balance Planck curve is located to obtain the dilated region.

[0069] In the selectable examples of this disclosure, in the coordinate system in which the second white balance Planck curve is located, the second white balance Planck curve can be used as the center line, and both the left and right sides can be expanded by half of a predetermined width to form an expanded region having a predetermined width. The range of values ​​for the predetermined width can be set appropriately depending on the actual situation, but is not limited to this disclosure.

[0070] In step 520, distribution information for the expansion region at the positions corresponding to each of the multiple subblocks is determined.

[0071] In the selectable examples of the present disclosure, the distribution information of the position corresponding to each of the N subblocks relative to the expansion region can represent whether or not the position corresponding to the subblock is located within the expansion region.

[0072] In step 530, the weights corresponding to each of the multiple subblocks are determined based on the distribution information corresponding to each of the multiple subblocks.

[0073] In the optional examples of this disclosure, if the distribution information corresponding to each of the N subblocks indicates that the position corresponding to this subblock is located within the expansion region, then the first pre-set weight can be the weight corresponding to the subblock. If the distribution information corresponding to this subblock indicates that the position corresponding to this subblock is not located within the expansion region, then the second pre-set weight can be the weight corresponding to the subblock. In the examples of this disclosure, the first pre-set weight can be 1 and the second pre-set weight can be 0.

[0074] The expansion region can be efficiently and quickly obtained by applying an expansion process to the second white balance Planck curve. Based on the distribution information of the expansion region for the positions corresponding to each of the multiple subblocks, the weights corresponding to each of the multiple subblocks can be rationally determined. For example, the weights corresponding to multiple subblocks can be determined according to the following rule: when the position corresponding to a subblock is located within the expansion region, the weight corresponding to the subblock is set to 1, and when the position corresponding to a subblock is located outside the expansion region, the weight corresponding to the subblock is set to 0. By using the determined weights to determine the white balance correction parameters, the rationality and reliability of the white balance correction parameters can be ensured.

[0075] In the optional examples of this disclosure, the embodiment of Figure 4 and the embodiment of Figure 5 can also be used in combination. For example, weights corresponding to multiple subblocks can be determined based on distance information and distribution information corresponding to multiple subblocks.

[0076] Even if one embodiment is adopted in step 330 to determine the weights corresponding to each of the multiple subblocks, step 340 can be carried out based on each of these weights corresponding to each of the multiple subblocks.

[0077] In step 340, weighting is performed on the positions corresponding to each of the subblocks using the weights assigned to each subblock to obtain the weighted position.

[0078] In the selectable examples of this disclosure, if the number of subblocks is N subblocks, and there is a one-to-one correspondence between the N subblocks and N positions, and also a one-to-one correspondence between the N subblocks and N weights, then the weighted positions can be obtained by using the N weights to perform a weighted average on the N positions.

[0079] In the examples of this disclosure, N weights are K1, K2, K3, ..., K N Let the N positions be P1, P2, P3, ..., P N Let P be the weighted position. These satisfy the following equations. [Mathematical formula] P=(K1*P1+K2*P2+K3*P3+…+K N *P N ) / (K1+K2+K3+…+K N )

[0080] In step 350, the white balance correction parameters suitable for the target image sensor are determined based on the weighted position and the inverse matrix of the target matrix.

[0081] In the optional examples of this disclosure, the weighted position may be the position of a point in the coordinate system in which the second white balance Planck curve is located. Based on the horizontal and vertical coordinates of this point and the inverse matrix of the target matrix, white balance correction parameters suitable for the target image sensor can be determined. In the examples of this disclosure, the target matrix is ​​MQe, and the inverse matrix of the target matrix is ​​MQe -1 It can be done this way.

[0082] In the embodiments of this disclosure, the positions corresponding to each of the multiple subblocks can be rationally determined in the coordinate system in which the second white balance Planck curve is located, based on the color information of each of the multiple subblocks included in the target matrix and the target gray zone. Then, the weights corresponding to the multiple subblocks can be rationally determined, and weighting can be performed on the determined positions based on the determined weights. The weighted positions obtained by weighting and the inverse matrix of the target matrix can be used to determine the white balance correction parameters. By introducing weighted average operation, the rationality and reliability of the determined white balance correction parameters can be ensured by avoiding the unfavorable effects that result from considering only the information of a single subblock.

[0083] In the embodiments of this disclosure, the number of first images can be multiple images (for example, M images). For each of the M first images, a corresponding target gray zone can be determined. By performing the method of the embodiment in Figure 3 on the target gray zone corresponding to each first image, the corresponding white balance correction parameters can be determined. Subsequently, by performing an average operation or a weighted average operation on the white balance correction parameters corresponding to each of the M first images, an average operation result or a weighted average operation result can be obtained, and this average operation result or weighted average operation result can be used as a white balance correction parameter that is suitable for the target image sensor.

[0084] Figure 6 is a flowchart of a method for determining the positions corresponding to each of the subblocks in the coordinate system in which the second white balance Planck curve is located, based on the target matrix and the color information of each of the subblocks according to an embodiment of the present disclosure. The method in Figure 6 may include steps 610, 620, 630, and 640. In the method in Figure 6, the color filter array of the target image sensor is an RCCG array.

[0085] In step 610, for the target subblock in multiple subblocks, the first ratio of the pixel value of the R channel to the pixel value of the C channel, and the second ratio of the pixel value of the G channel to the pixel value of the C channel are determined based on the color information of the target subblock.

[0086] In the selectable examples of this disclosure, multiple subblocks may be N subblocks, and the target subblock may be any one of the N subblocks. The term "target" in "target subblock" is not limited to the target subblock.

[0087] In the optional examples of this disclosure, the color information of a target subblock may include a first pixel value in the R channel of the target subblock, a second pixel value in the C channel of the target subblock, and a third pixel value in the G channel of the target subblock. By dividing the first pixel value by the second pixel value, a first ratio value between the pixel value in the R channel and the pixel value in the C channel can be obtained. By dividing the third pixel value by the second pixel value, a second ratio value between the pixel value in the G channel and the pixel value in the C channel can be obtained.

[0088] In step 620, the target matrix is ​​used to convert the values ​​of the first ratio to the values ​​of the third ratio.

[0089] In the selectable examples of this disclosure, the value of the first ratio can be multiplied by the target matrix to obtain the multiplication result, and the obtained multiplication result can be taken as the value of the third ratio.

[0090] In step 630, the target matrix is ​​used to convert the second ratio value to the fourth ratio value.

[0091] In the selectable examples of this disclosure, the value of the second ratio can be multiplied by the target matrix to obtain the multiplication result, and the obtained multiplication result can be taken as the value of the fourth ratio.

[0092] In step 640, the position corresponding to the target subblock in the coordinate system where the second white balance Planck curve is located is determined based on the values ​​of the third ratio and the fourth ratio.

[0093] In the optional examples of this disclosure, it can be understood that the value of the third ratio belongs to the value of the first type of ratio, and the value of the fourth ratio belongs to the value of the second type of ratio. In that case, a position can be determined in the coordinate system in which the second white balance Planck curve is located, where the horizontal coordinate is the value of the third ratio and the vertical coordinate is the value of the fourth ratio, and this position can be designated as the position corresponding to the target subblock.

[0094] In the embodiments of this disclosure, the values ​​of the first ratio and the second ratio can be effectively and quickly determined based on the color information of the target subblock, and the values ​​of the first ratio and the second ratio can be effectively and quickly converted to the values ​​of the third ratio and the fourth ratio using the target matrix. This provides an effective criterion for determining the position corresponding to the target subblock and ensures the accuracy of the determined position.

[0095] The position corresponding to the target subblock is not limited to the position where the horizontal coordinate is the third ratio value and the vertical coordinate is the fourth ratio value in the coordinate system where the second white balance Planck curve is located. For example, there can be a slight deviation between the position corresponding to the target subblock and the position where the horizontal coordinate is the third ratio value and the vertical coordinate is the fourth ratio value.

[0096] Figure 7 is a flowchart of a method for determining white balance correction parameters suitable for a target image sensor based on the weighted position and the inverse matrix of the target matrix according to an embodiment of the present disclosure. The method in Figure 7 may include steps 710, 720, 730, 740, 750, and 760. In the method in Figure 7, the color filter array of the target image sensor is an RCCG array.

[0097] In step 710, the fifth ratio of the pixel values ​​of the R channel to the pixel values ​​of the C channel, and the sixth ratio of the pixel values ​​of the G channel to the pixel values ​​of the C channel are determined based on the weighted position.

[0098] In the optional examples of this disclosure, the horizontal coordinate of the weighted position may be the value of the fifth ratio between the pixel value of the R channel and the pixel value of the C channel, and the vertical coordinate of the weighted position may be the value of the sixth ratio between the pixel value of the G channel and the pixel value of the C channel.

[0099] In step 720, the inverse matrix of the target matrix is ​​used to convert the value of the 5th ratio to the value of the 7th ratio.

[0100] In the selectable examples of this disclosure, the value of the fifth ratio can be multiplied by the inverse matrix of the target matrix to obtain the multiplication result, and the obtained multiplication result can be taken as the value of the seventh ratio.

[0101] In step 730, the first color compensation coefficient associated with the R channel is determined based on the value of the seventh ratio.

[0102] In the optional examples of this disclosure, the reciprocal of the value of the seventh ratio may be the first color correction coefficient associated with the R channel. The first color correction coefficient may be a coefficient used for color correction between the R channel and the C channel. The first color correction coefficient may be denoted as Rgain. If the value of the seventh ratio is denoted as R / C, the following equation can be satisfied. [Mathematical formula] Rgain = 1 / (R / C)

[0103] In step 740, the inverse matrix of the target matrix is ​​used to convert the value of the 6th ratio to the value of the 8th ratio.

[0104] In the optional examples of this disclosure, the value of the sixth ratio can be multiplied by the inverse matrix of the target matrix to obtain the multiplication result, and the obtained multiplication result can be taken as the value of the eighth ratio.

[0105] In step 750, the second color correction coefficient associated with the G channel is determined based on the value of the 8th ratio.

[0106] In the optional examples of this disclosure, the reciprocal of the value of the eighth ratio may be the second color correction coefficient associated with the G channel. The second color correction coefficient may be a coefficient used for color correction between the G channel and the C channel. The second color correction coefficient may be denoted as Ggain. If the value of the eighth ratio is denoted as G / C, the following equation can be satisfied. [Mathematical formula] Ggain = 1 / (G / C)

[0107] In step 760, the white balance correction parameters suitable for the target image sensor are determined based on the first color correction coefficient and the second color correction coefficient.

[0108] In the selectable examples of this disclosure, the white balance correction parameters that are suitable for the target image sensor may include a first color correction coefficient and a second color correction coefficient.

[0109] In the examples of this disclosure, the target image sensor is an RCCG image sensor, and the second image is raw data captured by the RCCG image sensor, the raw data may contain V pixel dots. V1 pixel dots correspond to the R channel, V2 pixel dots correspond to the C channel, and V3 pixel dots correspond to the G channel. Raw data with white balance correction can be obtained by multiplying the individual pixel values ​​of V1 pixel dots by a first color correction coefficient, and multiplying the individual pixel values ​​of V3 pixel dots by a second color correction coefficient.

[0110] Interpolation processing is performed on the raw data as a second image to obtain interpolation results. These interpolation results may include full-resolution images corresponding to the R channel, C channel, and G channel, respectively. These interpolation results can be considered as images suitable for the user's observation. Subsequently, white balance correction processing can be performed on the interpolation results using the first and second color correction coefficients.

[0111] In the embodiments of this disclosure, the fifth ratio value between the pixel values ​​of the R channel and the C channel, and the sixth ratio value between the pixel values ​​of the G channel and the C channel can be effectively and quickly determined based on the weighted position. The fifth ratio value can then be effectively and quickly converted to the seventh ratio value, and the sixth ratio value to the eighth ratio value, using the inverse matrix of the target matrix. In this way, by providing an effective criterion for determining the first and second color correction coefficients, it is possible to guide the color correction between channels. Furthermore, reasonable and reliable white balance correction parameters can be determined.

[0112] In the optional examples of this disclosure, the optical specifications of the target image sensor and the optical specifications of the reference image sensor may be the same.

[0113] It is important to note that when the optical specifications of the target image sensor and the reference image sensor are the same, it means that only the color filter array of the target image sensor and the color filter array of the reference image sensor differ, and all other parameters are the same. For example, the resolution, pixel size, shutter type, etc., of the target image sensor and the reference image sensor can all be the same. This prevents differences in the optical specifications of the target image sensor and the reference image sensor from affecting the determination of the white balance correction parameters, and ensures the rationality and reliability of the determined white balance correction parameters.

[0114] In the optional examples of this disclosure, the time at which the second image is captured may be later than (after) the time at which the first image is captured.

[0115] In the selectable examples of this disclosure, the first image is an image captured by the target image sensor at time t0, and the second image is an image captured by the target image sensor at time t1, where time t1 may be later than time t0. In the examples of this disclosure, time t1 may be the time immediately following time t0, or time t1 may be a time after a preset time has elapsed since time t0. By performing white balance correction on the image captured by the target image sensor after time t1, a good white balance correction effect can be achieved.

[0116] In the selectable examples of this disclosure, the reference image sensor may be an RGGB image sensor, and the target image sensor may be an RCCG.

[0117] By performing calibration on the RCCG image sensor, a white balance Planck curve (corresponding to the first white balance Planck curve in the preceding paragraph) is obtained. The specific calibration process may include (a1) to (a2) below.

[0118] (a1) Raw data is acquired by capturing images of 24 color cards using an RCCG image sensor at a certain color temperature. The acquired raw data is divided into multiple subdata parts, and in response to user input, a portion of the subdata obtained by the division is selected to form a white balance gray zone (similar to the target gray zone in the preceding paragraph). The white balance gray zone can contain 6 subblocks. For each of the 6 subblocks, the ratio value Z1 of the pixel value of the R channel to the pixel value of the C channel can be statistically calculated, and the ratio value Z2 of the pixel value of the G channel to the pixel value of the C channel can be statistically calculated.

[0119] (a2) The R / C ratio is obtained by calculating the average value of the six Z1 values ​​that correspond one-to-one with the six subblocks, and the G / C ratio is obtained by calculating the average value of the six G / C values ​​that correspond one-to-one with the six subblocks. This allows us to obtain the R / C ratio and G / C ratio at a certain color temperature. Using this method, we can obtain the R / C ratio and G / C ratio at various color temperatures. Then, the first white balance Planck curve is constructed using the R / C ratio (corresponding to the value of the first kind of ratio in the preceding paragraph) as the x-coordinate and the G / C ratio (corresponding to the value of the second kind of ratio in the preceding paragraph) as the y-coordinate.

[0120] Furthermore, the quantum efficiency response curves for the RCCG image sensor and the RGGB image sensor can be determined. Using the least squares method or other regression algorithms, a QE correction matrix (corresponding to the target matrix in the preceding paragraph) can be determined to map the quantum efficiency response curve of the RCCG image sensor to the quantum efficiency response curve of the RGGB image sensor. Using the QE correction matrix, the first white balance Planck curve can be transformed into the second white balance Planck curve.

[0121] For each of the six subblocks related to (a1), the Z1 and Z2 corresponding to this subblock are multiplied by the QE correction matrix, respectively, to determine the position coordinates of this subblock in the coordinate system where the second white balance Planck curve is located (corresponding to the position corresponding to this subblock). The weights are determined for each of the six subblocks according to the rule that the closer the position is to the second white balance Planck curve, the larger the weight corresponding to the subblock. Using the determined weights, a weighted average is performed on the positions corresponding to each of the six subblocks to obtain the weighted position. The horizontal and vertical coordinates of the weighted position are multiplied by the inverse matrix of the QE correction matrix, respectively, to obtain the values ​​of the 7th ratio and the 8th ratio in the preceding paragraph. Subsequently, the reciprocals of the values ​​of the 7th ratio and the 8th ratio are calculated to obtain Rgain and Ggain in the preceding paragraph, and Rgain and Ggain can be used to construct white balance correction parameters suitable for the RCCG image sensor. By applying white balance correction processing to the Raw data later captured by the RCCG image sensor using white balance correction parameters, inter-channel color correction can be achieved, ensuring that white objects in the real world still appear white in the image.

[0122] Through experiments, the inventors discovered that in the prior art, the white balance Planck curve of an RCCG image sensor is limited by the seven points shown in Figure 8-1, and in the embodiments of this disclosure, the second white balance Planck curve is limited by the seven points shown in Figure 8-2. In Figures 8-1 and 8-2, D75, D65, D50, TL84, CWF, A, and H each represent one light source, and different light sources correspond to different color temperatures. From Figures 8-1 and 8-2, it can be seen that the points corresponding to different color temperatures in Figure 8-2 are more dispersed than the points in Figure 8-1. By improving the positional discrimination of the points corresponding to different color temperatures, it is possible to avoid color temperature interference affecting the white balance correction processing effect and improve the white balance correction processing effect.

[0123] It should be noted that the target image sensor is not limited to RCCG image sensors, and theoretically could be any image sensor that is not an RGGB image sensor, but this disclosure does not limit the specific type of target image sensor.

[0124] Based on the above, in the embodiments of this disclosure, images captured using the second type of color filter array can achieve a good white balance correction effect by a newly adapted white balance correction method.

[0125] [Exemplary device] Figure 9 shows the structure of an image white balance correction device according to an exemplary embodiment of the present disclosure. The white balance correction device shown in Figure 9, A first determination module 910 for determining the first quantum efficiency response curve of a reference image sensor, A second deterministic module 920 for determining the second quantum efficiency response curve of a target image sensor that is different from the reference image sensor, and A third deterministic module 930 is used to determine the target matrix used in the conversion between the second quantum efficiency response curve determined by the second deterministic module 920 and the first quantum efficiency response curve determined by the first deterministic module 910, A fourth determination module 940 for determining the first white balance plank curve of the target image sensor, A fifth determination module 950 determines white balance correction parameters suitable for the target image sensor based on the target matrix determined by the third determination module 930, the first white balance plank curve determined by the fourth determination module 940, and the first image captured by the target image sensor. The system includes a processing module 960 for performing white balance correction processing on a second image captured by a target image sensor, according to the white balance correction parameters determined by the fifth determination module 950.

[0126] In an optional example of the present disclosure, as shown in Figure 10, the fifth determinative module 950 includes a conversion submodule 1010 for converting a first white balance plank curve determined by the fourth determinative module 940 to a second white balance plank curve of a reference image sensor using a target matrix determined by the third determinative module 930; a first determinative submodule 1020 for determining a target gray zone in a first image captured by the target image sensor; and a second determinative submodule 1030 for determining white balance correction parameters that fit the target image sensor based on the target matrix determined by the third determinative module 930, the second white balance plank curve obtained by the conversion submodule 1010, and the target gray zone obtained by the first determinative submodule 1020.

[0127] In an optional example of the present disclosure, as shown in Figure 11, the second determinative submodule 1030 includes a first determinative unit 1110 for determining the color information of each of the multiple subblocks included in the target gray zone determined by the first determinative submodule 1020, a second determinative unit 1120 for determining the positions corresponding to each of the multiple subblocks in the coordinate system where the second white balance plank curve is located, based on the target matrix determined by the third determinative module 930 and the color information of each of the multiple subblocks determined by the first determinative unit 1110, and the second white balance plank curve obtained by the transformation submodule 1010 and the second determinative unit 1120 The system includes: a third determination unit 1130 for determining the weights corresponding to each of the acquired subblocks based on the corresponding positions of each of the acquired subblocks; a weighting unit 1140 for obtaining weighted positions by weighting the positions corresponding to each of the acquired subblocks determined by the second determination unit 1120 using the weights corresponding to each of the acquired subblocks determined by the third determination unit 1130; and a fourth determination unit 1150 for determining white balance correction parameters that are suitable for the target image sensor based on the weighted positions obtained by the weighting unit 1140 and the inverse matrix of the target matrix.

[0128] In the selectable examples of this disclosure, the color filter array of the target image sensor is an RCCG array, and as shown in Figure 12, the second determination unit 1120 includes a first determination subunit 1210 for determining, for target subblocks in a plurality of subblocks, a first ratio value of the R channel pixel value to the C channel pixel value and a second ratio value of the G channel pixel value to the C channel pixel value based on the color information of the target subblock, and the third determination module 930 for determining, the first determination subunit 1210 The system includes a first transformation subunit 1220 for converting a determined first ratio value to a third ratio value, a second transformation subunit 1230 for converting a second ratio value determined by the first determination subunit 1210 to a fourth ratio value using a target matrix determined by the third determination module 930, and a second determination subunit 1240 for determining the position corresponding to the target subblock in the coordinate system where the second white balance Planck curve is located, based on the third ratio value obtained by the transformation of the first transformation subunit 1220 and the fourth ratio value obtained by the transformation of the second transformation subunit 1230.

[0129] In the selectable examples of this disclosure, the color filter array of the target image sensor is an RCCG array, and as shown in Figure 13, the fourth determinative unit 1150 includes a third determinative subunit 1310 for determining the fifth ratio value between the pixel values ​​of the R channel and the C channel, and the sixth ratio value between the pixel values ​​of the G channel and the C channel, based on the weighted position obtained by the weighting unit 1140; a third transform subunit 1320 for converting the fifth ratio value determined by the third determinative subunit 1310 to the seventh ratio value using the inverse matrix of the target matrix; and based on the seventh ratio value obtained by the third transform subunit 1320, the R channel The system includes a fourth determining subunit 1330 for determining the first color correction coefficient related to the G channel, a fourth conversion subunit 1340 for converting the value of the sixth ratio determined by the third determining subunit 1310 to the value of the eighth ratio using the inverse matrix of the target matrix, a fifth determining subunit 1350 for determining the second color correction coefficient related to the G channel based on the value of the eighth ratio obtained by the fourth conversion subunit 1340, and a sixth determining subunit 1360 for determining white balance correction parameters that are suitable for the target image sensor based on the first color correction coefficient determined by the fourth determining subunit 1330 and the second color correction coefficient determined by the fifth determining subunit 1350.

[0130] In an optional example of the present disclosure, as shown in Figure 14, the third determinative unit 1130 includes a seventh determinative subunit 1410 for determining distance information between the positions corresponding to each of the subblocks and the second white balance Planck curve, and an eighth determinative subunit 1420 for determining the weights corresponding to each of the subblocks based on the distance information corresponding to each of the subblocks determined by the seventh determinative subunit 1410.

[0131] In an optional example of the present disclosure, as shown in Figure 15, the third determinative unit 1130 includes an expansion subunit 1510 for performing an expansion process on the second white balance Planck curve in the coordinate system in which the second white balance Planck curve is located to obtain an expanded region, a ninth determinative subunit 1520 for determining distribution information for the expanded region obtained by the expansion subunit 1510 at positions corresponding to a plurality of subblocks, and a tenth determinative subunit 1530 for determining weights corresponding to a plurality of subblocks based on the distribution information corresponding to a plurality of subblocks determined by the ninth determinative subunit 1520.

[0132] In the selectable examples of this disclosure, the color filter array of the reference image sensor is an RGGB array and / or The optical specifications of the target image sensor and the optical specifications of the reference image sensor are the same, and / or, The time at which the second image is captured is later than (after) the time at which the first image is captured.

[0133] In the apparatus of this disclosure, the functions and effects of the invention of this disclosure can be achieved by appropriately selecting or flexibly combining the selectable embodiments, selectable models, and selectable examples, but these will not be described in detail in this disclosure.

[0134] [Example electronic devices] Figure 16 is a block diagram showing an electronic device according to an embodiment of the present disclosure, the electronic device 1600 including one or more processors 1610 and memory 1620.

[0135] The processor 1610 may be a central processing unit (CPU) or another form of processing unit having data processing capability and / or instruction execution capability. The processor 1610 can control other components in the electronic device 1600 to perform desired functions.

[0136] The memory 1620 may include one or more computer program products, which may include computer-readable storage media of various types, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. The computer-readable storage media may store one or more computer program instructions, and the processor 1110 can perform the methods of each embodiment of the present disclosure and / or other desired functions by executing one or more computer program instructions.

[0137] In the examples of this disclosure, the electronic device 1600 may further include an input device 1630 and an output device 1640 connected to each other via a bus system and / or other forms of connection means (not shown).

[0138] The input device 1630 may include, for example, a keyboard, a mouse, or the like.

[0139] The output device 1640 can output various types of information to the outside. The output device 1640 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0140] For simplicity, Figure 16 shows only some of the components of the electronic device 1600 relevant to this disclosure, omitting components such as buses and input / output interfaces. The electronic device 1600 may further include any other appropriate components depending on the specific application.

[0141] [Examples of computer program products and computer-readable storage media] Embodiments of this disclosure further provide computer program products, including computer program instructions, in addition to the methods and apparatus described herein. When computer program instructions are executed by a processor, the processor can be caused to perform the steps of the methods of each embodiment of this disclosure described in the “Exemplary Methods” of this specification.

[0142] A computer program product can be created using any combination of one or more programming languages ​​to produce program code for performing the operations of the embodiments of this disclosure, and such programming languages ​​may include object-oriented programming languages ​​such as Java and C++, and may also include general procedural programming languages ​​such as the C language or similar programming languages. The program code may be executed as follows: it may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0143] Furthermore, embodiments of this disclosure further provide computer-readable storage media in which computer program instructions are stored. When computer program instructions are executed by a processor, the processor can be caused to perform the steps of the methods of each embodiment of this disclosure described in the “Exemplary Methods” of this specification.

[0144] Any combination of one or more types of readable media can be used as a computer-readable storage medium. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (non-exclusive list) of readable storage media include electrical connections with one or more wires, portable disks, hard drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0145] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, merits, and effects mentioned in this disclosure are illustrative and not limiting, and various examples of this disclosure do not necessarily possess these advantages, merits, and effects. Furthermore, the specific details of the above disclosure are for illustrative and easy-to-understand purposes only and are not limiting, and the above details do not necessarily restrict this disclosure to being realized by the above specific details.

[0146] The above description is provided for illustrative and illustrative purposes only. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. While several exemplary embodiments and examples have been described above, those skilled in the art will be able to recognize certain variations, modifications, changes, additions, and subcombinations thereof.

Claims

1. A method for correcting the white balance of an image, wherein each step is performed by an image white balance correction device, The steps include determining the first quantum efficiency response curve of the reference image sensor, A step of determining the second quantum efficiency response curve of a target image sensor, wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor, A step of determining the target matrix used for the conversion between the second quantum efficiency response curve and the first quantum efficiency response curve, The steps include determining the first white balance plank curve of the target image sensor, The steps include determining white balance correction parameters suitable for the target image sensor based on the target matrix, the first white balance Planck curve, and the first image captured by the target image sensor, A method for correcting the white balance of an image, characterized by comprising the step of performing a white balance correction process on a second image captured by the target image sensor according to the white balance correction parameters.

2. The step of determining white balance correction parameters suitable for the target image sensor based on the target matrix, the first white balance plank curve, and the first image captured by the target image sensor is: The steps include using the target matrix to convert the first white balance plank curve to the second white balance plank curve of the reference image sensor, The steps include determining the target gray zone in the first image captured by the target image sensor, The method for correcting the white balance of an image according to claim 1, comprising the step of determining white balance correction parameters that are suitable for the target image sensor based on the target matrix, the second white balance plank curve, and the target gray zone.

3. The step of determining white balance correction parameters suitable for the target image sensor based on the target matrix, the second white balance plank curve, and the target gray zone is as follows: The steps include determining the color information of each of the multiple subblocks included in the target gray zone, The steps include determining the positions corresponding to each of the multiple subblocks in the coordinate system where the second white balance Planck curve is located, based on the target matrix and the color information of each of the multiple subblocks, A step of determining the weights corresponding to each of the multiple subblocks based on the second white balance plank curve and the positions corresponding to each of the multiple subblocks, A step of obtaining a weighted position by using the weights corresponding to each of the multiple subblocks and performing weighting at the positions corresponding to each of the multiple subblocks, The image white balance correction method according to claim 2, comprising the step of determining white balance correction parameters suitable for the target image sensor based on the weighted position and the inverse matrix of the target matrix.

4. The color filter array of the target image sensor is an RCCG array. The step of determining the position corresponding to each of the multiple sub-blocks in the coordinate system where the second white balance Planck curve is located, based on the target matrix and the color information of each of the multiple sub-blocks, is as follows: For target subblocks in a plurality of subblocks, the steps include determining a first ratio value between the pixel value of the R channel and the pixel value of the C channel, and a second ratio value between the pixel value of the G channel and the pixel value of the C channel, based on the color information of the target subblocks. The steps include: using the target matrix to convert the value of the first ratio to the value of the third ratio; The steps include: using the target matrix to convert the value of the second ratio to the value of the fourth ratio; The image white balance correction method according to claim 3, comprising the step of determining the position corresponding to the target subblock in the coordinate system in which the second white balance Planck curve is located, based on the value of the third ratio and the value of the fourth ratio.

5. The color filter array of the target image sensor is an RCCG array. The step of determining white balance correction parameters suitable for the target image sensor based on the weighted position and the inverse matrix of the target matrix is: Based on the weighted position, the steps include determining the value of the fifth ratio between the pixel value of the R channel and the pixel value of the C channel, and the value of the sixth ratio between the pixel value of the G channel and the pixel value of the C channel, The steps include converting the value of the fifth ratio to the value of the seventh ratio using the inverse matrix of the target matrix, The steps include determining the first color correction coefficient related to the R channel based on the value of the seventh ratio, The steps include converting the value of the sixth ratio to the value of the eighth ratio using the inverse matrix of the target matrix, The steps include determining the second color correction coefficient related to the G channel based on the value of the eighth ratio, The image white balance correction method according to claim 3, comprising the step of determining white balance correction parameters suitable for the target image sensor based on the first color correction coefficient and the second color correction coefficient.

6. The step of determining the weights corresponding to each of the multiple subblocks based on the second white balance plank curve and the positions corresponding to each of the multiple subblocks is as follows: A step of determining distance information between the position corresponding to each of the multiple subblocks and the second white balance plank curve, The image white balance correction method according to claim 3, comprising the step of determining a weight corresponding to each of the multiple subblocks based on the distance information corresponding to each of the multiple subblocks.

7. The step of determining the weights corresponding to each of the multiple subblocks based on the second white balance plank curve and the positions corresponding to each of the multiple subblocks is as follows: The steps include: performing an expansion process on the second white balance Planck curve in the aforementioned coordinate system to obtain an expanded region; A step of determining distribution information for the expansion region at positions corresponding to each of the multiple subblocks, The image white balance correction method according to claim 3, comprising the step of determining a weight corresponding to each of the multiple subblocks based on the distribution information corresponding to each of the multiple subblocks.

8. The color filter array of the aforementioned reference image sensor is an RGB array, and / or The optical specifications of the target image sensor and the optical specifications of the reference image sensor are the same, and / or The image white balance correction method according to any one of claims 1 to 7, characterized in that the time at which the second image is captured is after the time at which the first image is captured.

9. A first determination module for determining the first quantum efficiency response curve of a reference image sensor, A second deterministic module for determining the second quantum efficiency response curve of a target image sensor that is different from the reference image sensor, the color filter array, A third deterministic module for determining the target matrix used in the conversion between the second quantum efficiency response curve determined by the second deterministic module and the first quantum efficiency response curve determined by the first deterministic module, A fourth determination module for determining the first white balance plank curve of the target image sensor, A fifth determination module for determining white balance correction parameters suitable for the target image sensor, based on the target matrix determined by the third determination module, the first white balance plank curve determined by the fourth determination module, and the first image captured by the target image sensor. An image white balance correction device, characterized by including a processing module for performing white balance correction processing on a second image captured by the target image sensor according to the white balance correction parameters determined by the fifth determination module.

10. A computer-readable storage medium, A computer-readable storage medium characterized in that it stores a computer program that performs the image white balance correction method described in any one of claims 1 to 7.

11. An electronic device comprising a processor and memory for storing instructions that the processor can execute, An electronic device characterized in that the processor reads and executes the executable instructions from the memory to perform the image white balance correction method described in any one of claims 1 to 7.

Images