Image white balance correction method and device, storage medium and electronic equipment

Abstract

The invention discloses an image white balance correction method and device, a storage medium and electronic equipment. The method comprises the following steps: determining a first quantum efficiency response curve of a reference image sensor; a second quantum efficiency response curve of the target image sensor is determined, and the color filtering array of the target image sensor is different from the color filtering array of the reference image sensor; determining a target matrix for a transition between the second quantum efficiency response curve and the first quantum efficiency response curve; determining a first white balance Planck curve of the target image sensor; determining a white balance compensation parameter matched with the target image sensor based on the target matrix, the first white balance Planck curve and a first image acquired by the target image sensor; and performing white balance correction processing on a second image acquired by the target image sensor according to the white balance compensation parameter. In the embodiment of the invention, aiming at the image acquired by the target image sensor, a good white balance correction processing effect can be achieved.

Description

Technical Field

[0001] The present disclosure relates to machine vision technology, and in particular to a method, device, storage medium and electronic device for white balance correction of an image. Background Art

[0002] At present, image sensors are widely used. In some scenarios, it is necessary to perform white balance correction on the images captured by the image sensor. In the related art, for image sensors using some color filter arrays, it is impossible to achieve a good white balance correction effect. For example, for image sensors using RCCG arrays, the image after white balance correction may have color cast; where R represents red, C represents transparent transmission, and G represents green. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, device, storage medium and electronic device for white balance correction of an image.

[0004] According to one aspect of an embodiment of the present disclosure, a method for correcting white balance of an image is provided, comprising:

[0005] determining a first quantum efficiency response curve of a reference image sensor;

[0006] Determining a second quantum efficiency response curve of a target image sensor, wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor;

[0007] determining a target matrix for conversion between the second quantum efficiency response curve and the first quantum efficiency response curve;

[0008] Determining a first white balance Planckian curve of the target image sensor;

[0009] Determining white balance compensation parameters adapted to the target image sensor based on the target matrix, the first white balance Planck curve, and a first image captured by the target image sensor;

[0010] According to the white balance compensation parameters, white balance correction processing is performed on the second image captured by the target image sensor.

[0011] According to another aspect of an embodiment of the present disclosure, there is provided a device for correcting white balance of an image, comprising:

[0012] A first determination module, used to determine a first quantum efficiency response curve of a reference image sensor;

[0013] A second determination module, configured to determine a second quantum efficiency response curve of a target image sensor whose color filter array is different from the reference image sensor;

[0014] a third determination module, configured to determine a target matrix for conversion between the second quantum efficiency response curve determined by the second determination module and the first quantum efficiency response curve determined by the first determination module;

[0015] A fourth determination module, configured to determine a first white balance Planck curve of the target image sensor;

[0016] a fifth determining module, configured to determine white balance compensation parameters adapted to the target image sensor based on the target matrix determined by the third determining module, the first white balance Planck curve determined by the fourth determining module, and the first image captured by the target image sensor;

[0017] A processing module is used to perform white balance correction processing on the second image captured by the target image sensor according to the white balance compensation parameter determined by the fifth determination module.

[0018] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned white balance correction method for an image.

[0019] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0020] processor;

[0021] a memory for storing instructions executable by the processor;

[0022] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned image white balance correction method.

[0023] According to another aspect of the embodiments of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, the above-mentioned image white balance correction method is performed.

[0024] Based on the image white balance correction method, device, storage medium, electronic device and program product provided by the above embodiments of the present disclosure, the image sensor using the second type of color filter array can be used as the reference image sensor, and the image sensor using the first type of color filter array can be used as the target image sensor, and the target matrix for conversion between the quantum efficiency response curves of the reference image sensor and the target image sensor can be determined, and then the white balance compensation parameters adapted to the target image sensor can be determined in combination with the first white balance Planck curve of the target image sensor and the first image captured by the target image sensor, so as to perform white balance correction processing on the second image captured by the target image sensor according to the white balance compensation parameters adapted to the target image sensor. In this way, even if the imaging characteristics of the target image sensor and the reference image sensor are greatly different, resulting in the white balance correction processing method for the reference image sensor not being applicable to the target image sensor, the dedicated white balance compensation parameters can be reasonably determined for the target image sensor, so as to be used for the white balance correction processing of the second image captured by the target image sensor. In this way, for the image captured by the image sensor using the second type of color filter array, a new adapted white balance correction method can be used to achieve a good white balance correction processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of a method for correcting white balance of an image provided by some exemplary embodiments of the present disclosure.

[0026] Figure 2 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on a target matrix, a first white balance Planck curve, and a first image captured by a target image sensor, provided by some exemplary embodiments of the present disclosure.

[0027] Figure 3 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on a target matrix, a second white balance Planck curve, and a target gray area provided by some exemplary embodiments of the present disclosure.

[0028] Figure 4 It is a flowchart of a method for determining weights corresponding to a plurality of sub-blocks respectively based on a second white balance Planck curve and positions corresponding to the plurality of sub-blocks respectively, provided by some exemplary embodiments of the present disclosure.

[0029] Figure 5 It is a flowchart diagram of a method for determining weights corresponding to a plurality of sub-blocks respectively based on a second white balance Planck curve and positions corresponding to the plurality of sub-blocks respectively, provided by some other exemplary embodiments of the present disclosure.

[0030] Figure 6It is a flowchart of a method for determining positions corresponding to a plurality of sub-blocks respectively in a coordinate system where a second white balance Planck curve is located, based on a target matrix and respective color information of the plurality of sub-blocks, provided by some exemplary embodiments of the present disclosure.

[0031] Figure 7 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on weighted positions and an inverse matrix of a target matrix according to some exemplary embodiments of the present disclosure.

[0032] Figure 8-1 It is a schematic diagram of the distribution of several points of the white balance Planck curve used to define the RCCG image sensor in the related art.

[0033] Figure 8-2 Schematic diagram of the distribution of several points of the RCCG white balance Planck curve used to define RGGB-like characteristics in some exemplary embodiments of the present disclosure.

[0034] Fig. 9 It is a schematic diagram of the structure of a white balance correction device for an image provided by some exemplary embodiments of the present disclosure.

[0035] Fig.10 is a structural diagram of the fifth determination module in some exemplary embodiments of the present disclosure.

[0036] Fig.11 is a structural diagram of a second determination submodule in some exemplary embodiments of the present disclosure.

[0037] Fig.12 is a schematic structural diagram of a second determining unit in some exemplary embodiments of the present disclosure.

[0038] Fig.13 is a structural schematic diagram of a fourth determining unit in some exemplary embodiments of the present disclosure.

[0039] Fig.14 is a schematic structural diagram of a third determining unit in some exemplary embodiments of the present disclosure.

[0040] Fig.15 It is a structural schematic diagram of the third determination unit in some other exemplary embodiments of the present disclosure.

[0041] Fig.16 It is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0043] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0044] Application Overview

[0045] At present, image sensors are widely used. Based on the color filter array (CFA) of the image sensor, the image captured by the image sensor can be colored. The color filter array can include but is not limited to the RCCG array, RCCB array, RGGB array, etc., and accordingly, the image sensor can include but is not limited to the RCCG image sensor, RCCB image sensor, RGGB image sensor, etc.; wherein R represents red, C represents transparent transmission, G represents green, and B represents blue.

[0046] When using an image signal processing (ISP) system to process images captured by an image sensor, it is often necessary to perform white balance correction on the images captured by the image sensor. White balance can be understood as the balance of white color, that is, the target object that is white in the real physical world still appears white in the image, so that the image can more accurately reflect the color of the target object. The algorithms used to implement white balance correction processing may include but are not limited to the gray world method, the perfect reflection method, the dynamic threshold method, etc.

[0047] In the related art, it is not possible to achieve a good white balance correction processing effect for images captured by image sensors using some color filter arrays. For example, for images captured by image sensors using RCCG arrays, the images after white balance correction processing may appear reddish or purple. How to achieve a good white balance correction processing effect for images captured by image sensors using these color filter arrays is a problem worthy of attention for those skilled in the art.

[0048] Exemplary Systems

[0049] In the related art, although a good white balance correction processing effect cannot be achieved for images collected by image sensors using some color filter arrays (hereinafter referred to as the first type of color filter arrays), a good white balance correction processing effect can be achieved for images collected by image sensors using other color filter arrays (hereinafter referred to as the second type of color filter arrays). For example, a good white balance correction processing effect cannot be achieved for images collected by image sensors using the RCCG array, while a good white balance correction processing effect can be achieved for images collected by image sensors using the RGGB array. The inventors have found through research that the reason why a good white balance correction processing effect cannot be achieved for images collected by image sensors using the RCCG array is that the white balance correction method for image sensors using the RGGB array is directly reused for images collected by image sensors using the RCCG array, and there are large differences in the imaging characteristics of image sensors using the RCCG array and image sensors using the RGGB array, resulting in the white balance correction method not being applicable to images collected by image sensors using the RCCG array.

[0050] In view of this, in the embodiments of the present disclosure, an image sensor using the second type of color filter array can be used as a reference image sensor, and an image sensor using the first type of color filter array can be used as a target image sensor, and a target matrix for conversion between the quantum efficiency response curves of the reference image sensor and the target image sensor can be determined, and then combined with the first white balance Planck curve of the target image sensor and the first image captured by the target image sensor, a white balance compensation parameter adapted to the target image sensor can be determined for white balance correction processing of the second image captured by the target image sensor. In this way, for images captured by the image sensor using the first type of color filter array, a new and adapted white balance correction method can be used to achieve a good white balance correction processing effect.

[0051] Exemplary Methods

[0052] Figure 1 It is a flowchart of a method for correcting white balance of an image provided by some exemplary embodiments of the present disclosure. Figure 1 The method shown may include step 110 , step 120 , step 130 , step 140 , step 150 , and step 160 .

[0053] Step 110 , determining a first quantum efficiency (QE) response curve of a reference image sensor.

[0054] Optionally, the reference image sensor may be an image sensor used as a reference benchmark to assist in obtaining white balance compensation parameters adapted to other image sensors. The color filter array of the reference image sensor may belong to the second type of color filter array. As an example, the color filter array of the reference image sensor may be an RGGB array, and accordingly, the reference image sensor may be an RGGB image sensor. The white balance compensation parameters may be parameters used for white balance correction processing of an image.

[0055] Optionally, the first quantum efficiency response curve of the reference image sensor may be a curve with wavelength as the horizontal coordinate and response rate as the vertical coordinate. As an example, the wavelength may range from 400 nanometers to 700 nanometers. The response rate may range from 0% to 100%. The first quantum efficiency response curve of the reference image sensor may also be referred to as QE data of the reference image sensor.

[0056] Step 120 , determining a second quantum efficiency response curve of a target image sensor, wherein a color filter array of the target image sensor is different from a color filter array of a reference image sensor.

[0057] Optionally, the target image sensor may be an image sensor for which the corresponding white balance compensation parameters need to be determined. The color filter array of the target image sensor may belong to the first type of color filter array. The color filter array of the target image sensor may be different from the RGGB array, that is, the color filter array of the target image sensor may be a non-RGGB array, and accordingly, the target image sensor may be a non-RGGB image sensor. As an example, the color filter array of the target image sensor may be an RCCG array or an RCCB array, and accordingly, the target image sensor may be an RCCG image sensor or an RCCB image sensor.

[0058] Optionally, the second quantum efficiency response curve of the target image sensor may be a curve with wavelength as the horizontal coordinate and response rate as the vertical coordinate. The second quantum efficiency response curve of the target image sensor may also be referred to as QE data of the target image sensor.

[0059] Step 130 , determining a target matrix for conversion between the second quantum efficiency response curve and the first quantum efficiency response curve.

[0060] Optionally, the second quantum efficiency response curve and the first quantum efficiency response curve may be different curves drawn in the same coordinate system. The matrix used for conversion between different curves may be determined using the least square method or other regression algorithms, and the determined matrix may be used as the target matrix. The target matrix may also be referred to as a QE correction matrix. As an example, the size of the target matrix may be 3*3, that is, the number of rows and the number of columns of the target matrix may both be 3.

[0061] Step 140: determine a first white balance Planckian curve of the target image sensor.

[0062] Optionally, the first white balance Planck curve of the target image sensor may be a curve with the first analog ratio as the horizontal coordinate and the second analog ratio as the vertical coordinate; wherein the first analog ratio and the second analog ratio may both be ratios for reflecting the color difference between channels.

[0063] For example, if the color filter array of the target image sensor is an RCCG array, the first type of ratio may be the ratio of the pixel value of the R channel to the pixel value of the C channel, and the second type of ratio may be the ratio of the pixel value of the G channel to the pixel value of the C channel.

[0064] For another example, if the color filter array of the target image sensor is an RCCB array, the first type of ratio may be the ratio of the pixel value of the R channel to the pixel value of the C channel, and the second type of ratio may be the ratio of the pixel value of the B channel to the pixel value of the C channel.

[0065] Step 150: Determine white balance compensation parameters adapted to the target image sensor based on the target matrix, the first white balance Planck curve, and the first image captured by the target image sensor.

[0066] Optionally, the first image may be a native image obtained by capturing an image of a predetermined object. The predetermined object may include, but is not limited to, a 24-color color card, a gray card, etc., as long as a gray area exists in the first image; wherein the gray area may be understood as an area that is visually white, black, or gray. The native image may also be referred to as Raw data.

[0067] Optionally, the white balance compensation parameters adapted to the target image sensor may refer to: parameters suitable for performing white balance correction processing on the image captured by the target image sensor, which are used to guide how to perform color compensation between channels so that target objects that are white in the real physical world appear white in the image.

[0068] Step 160 , performing white balance correction processing on the second image captured by the target image sensor according to the white balance compensation parameter.

[0069] Optionally, the second image may be a native image obtained by capturing images of a predetermined object, or may be a native image obtained by capturing images of objects other than the predetermined object. The second image may be the same image as the first image, or may be different images.

[0070] Here, the second image is the Raw data obtained by the target image sensor through image acquisition. According to the white balance compensation parameter, the Raw data as the second image can be directly subjected to white balance correction processing. Alternatively, the Raw data as the second image can be first converted into an image suitable for user observation (non-Raw data), and then the converted image can be subjected to white balance correction processing.

[0071] In the embodiment of the present disclosure, an image sensor using the second type of color filter array can be used as a reference image sensor, and an image sensor using the first type of color filter array can be used as a target image sensor. The target matrix for conversion between the quantum efficiency response curves of the reference image sensor and the target image sensor can be determined. Then, in combination with the first white balance Planck curve of the target image sensor and the first image captured by the target image sensor, a white balance compensation parameter adapted to the target image sensor can be determined, so that a white balance correction process can be performed on the second image captured by the target image sensor according to the white balance compensation parameter adapted to the target image sensor. In this way, even if the imaging characteristics of the target image sensor and the reference image sensor are greatly different, resulting in the white balance correction process method for the reference image sensor not being applicable to the target image sensor, a dedicated white balance compensation parameter can be reasonably determined for the target image sensor, so as to be used for the white balance correction process of the second image captured by the target image sensor. In this way, for the image captured by the image sensor using the second type of color filter array, a new adapted white balance correction process method can be used to achieve a good white balance correction process effect.

[0072] Figure 2 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on a target matrix, a first white balance Planck curve, and a first image captured by a target image sensor, provided by some exemplary embodiments of the present disclosure. Figure 2 The illustrated method may include step 210 , step 220 , and step 230 .

[0073] Step 210: using the target matrix, convert the first white balance Planckian curve into a second white balance Planckian curve of the reference image sensor.

[0074] As described above, the target matrix can be used for conversion between the second quantum efficiency response curve and the first quantum efficiency response curve. Then, the target matrix can have the following two possibilities: Case 1: The target matrix is ​​used to map the second quantum efficiency response curve to the first quantum efficiency response curve; Case 2: The target matrix is ​​used to map the first quantum efficiency response curve to the second quantum efficiency response curve. For ease of understanding, the following description will take the above case 1 as an example.

[0075] Optionally, the first white balance Planck curve can be expressed as M1, the second white balance Planck curve of the reference image sensor can be expressed as M2, and the target matrix can be expressed as Mqe. Then, M2 can be obtained by the following formula:

[0076] M2=M1*Mqe

[0077] It should be noted that the second white balance Planck curve of the reference image sensor is not the real white balance Planck curve of the reference image sensor, but a white balance Planck curve of the reference image sensor inferred based on the target matrix and the first white balance Planck curve.

[0078] Step 220: determine a target gray area in the first image captured by the target image sensor.

[0079] Optionally, a user input operation may be received, and a gray area selected by the user input operation in the first image may be used as a target gray area. The user input operation may include but is not limited to a touch input operation, a keyboard input operation, a mouse input operation, and the like.

[0080] As an example, the first image may be a native image obtained by capturing an image of a 24-color color card. There may be 24 color blocks in 4 rows and 6 columns in the first image. The gray area selected by the user input operation in the first image may include the 6 color blocks located in the last row of the 24 color blocks, that is, the target gray area includes the 6 color blocks located in the last row of the 24 color blocks.

[0081] In some embodiments, the target gray area may also be determined not by user input operations but by algorithms. For example, a neural network model for identifying gray areas from images may be pre-trained, and the target gray area may be identified from the first image using the pre-trained neural network model.

[0082] Step 230 : determining white balance compensation parameters adapted to the target image sensor based on the target matrix, the second white balance Planck curve and the target gray area.

[0083] Optionally, the color information of the target gray area may be determined. Assuming that the target gray area includes the 6 color blocks located in the last row of the 24 color blocks, the color information of the target gray area may include the color information of each of the 6 color blocks located in the last row of the 24 color blocks. Next, the white balance compensation parameters adapted to the target image sensor may be determined based on the target matrix, the second white balance Planck curve, and the color information of the target gray area.

[0084] It should be noted that the first quantum efficiency response curve can affect the imaging characteristics of the reference image sensor, and the second quantum efficiency response curve can affect the imaging characteristics of the target image sensor. Therefore, the target matrix used for the conversion between the second quantum efficiency response curve and the first quantum efficiency response curve can be used to characterize the relationship between the imaging characteristics of the target image sensor and the reference image sensor. Since the first white balance Planck curve belongs to the imaging characteristics of the target image sensor, the relationship between the imaging characteristics represented by the target matrix can be used to efficiently and reliably infer the second white balance Planck curve for the reference image sensor. According to the target matrix, the second white balance Planck curve, and the target gray area of ​​the first image captured by the target image sensor, the white balance compensation parameters adapted to the target image sensor are determined. In the process of determining the white balance compensation parameters, the information related to the imaging characteristics of the image sensor can be effectively used, which is conducive to ensuring the adaptability of the white balance compensation parameters to the target image sensor, thereby ensuring the rationality and reliability of the determined white balance compensation parameters.

[0085] Figure 3 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on a target matrix, a second white balance Planck curve, and a target gray area provided by some exemplary embodiments of the present disclosure. Figure 3 The method shown may include step 310 , step 320 , step 330 , step 340 , and step 350 .

[0086] Step 310: Determine the color information of each of the multiple sub-blocks included in the target gray area.

[0087] Optionally, the multiple sub-blocks included in the target gray area can be represented as N sub-blocks, wherein N can be an integer greater than or equal to 2. Referring to the above example, the target gray area can include 6 color blocks located in the last row of 24 color blocks, then each of the 6 color blocks can be used as a sub-block, and accordingly, N can be 6.

[0088] In one example, the target image sensor may be an RCCG image sensor, and the color information of each of the N sub-blocks may include the pixel value of the sub-block in the R channel, the pixel value of the C channel, and the pixel value of the G channel. Assuming that there are a total of U pixels in the sub-block, of which U1 pixels correspond to the R channel, U2 pixels correspond to the C channel, and U3 pixels correspond to the G channel, the average of the U1 pixel values ​​corresponding to the U1 pixels one by one may be used as the pixel value of the sub-block in the R channel, the average of the U2 pixel values ​​corresponding to the U2 pixels one by one may be used as the pixel value of the sub-block in the C channel, and the average of the U3 pixel values ​​corresponding to the U3 pixels one by one may be used as the pixel value of the sub-block in the G channel.

[0089] Of course, the determination method of the pixel value of the sub-block in the R channel, the pixel value of the C channel, and the pixel value of the G channel is not limited to this. For example, the first image can be interpolated to obtain a first full-resolution image corresponding to the R channel of the first image, a second full-resolution image corresponding to the C channel of the first image, and a third full-resolution image corresponding to the G channel of the first image; wherein the first full-resolution image may include the pixel value of each pixel point in the first image in the R channel; the second full-resolution image may include the pixel value of each pixel point in the first image in the C channel; and the third full-resolution image may include the pixel value of each pixel point in the first image in the G channel. From the first full-resolution image, U pixel values ​​corresponding to the U pixel points in the sub-block can be obtained, and the average of the obtained U pixel values ​​can be used as the pixel value of the sub-block in the R channel. From the second full-resolution image, U pixel values ​​corresponding to the U pixel points in the sub-block can be obtained, and the average of the obtained U pixel values ​​can be used as the pixel value of the sub-block in the C channel. From the third full-resolution image, U pixel values ​​corresponding one-to-one to the U pixel points in the sub-block can be obtained, and the average value of the obtained U pixel values ​​can be used as the pixel value of the sub-block in the G channel.

[0090] In another example, the target image sensor may be an RCCB image sensor, and the color information of each sub-block in the N sub-blocks may include the pixel value of the sub-block in the R channel, the pixel value of the G channel, and the pixel value of the B channel. The determination method of the pixel value of the sub-block in the R channel, the pixel value of the G channel, and the pixel value of the B channel can refer to the relevant introduction in the previous example, and will not be repeated here.

[0091] Step 320 , based on the target matrix and the color information of each of the plurality of sub-blocks, in the coordinate system where the second white balance Planck curve is located, determine the positions corresponding to the plurality of sub-blocks respectively.

[0092] Optionally, the second white balance Planck curve may be drawn in a coordinate system with the first analog value as the abscissa and the second analog value as the ordinate. The coordinate system is the coordinate system where the second white balance Planck curve resides.

[0093] Optionally, for each of the N sub-blocks, the position corresponding to the sub-block can be determined in the coordinate system where the second white balance Planck curve is located based on the target matrix and the color information of the sub-block. The position corresponding to the sub-block can be the position of a point in the coordinate system where the second white balance Planck curve is located.

[0094] Step 330 : Determine the weights corresponding to the plurality of sub-blocks respectively based on the second white balance Planck curve and the positions corresponding to the plurality of sub-blocks respectively.

[0095] In some optional embodiments of the present disclosure, Figure 4 As shown, step 330 may include step 410 and step 420 .

[0096] Step 410: Determine distance information between positions corresponding to a plurality of sub-blocks and a second white balance Planck curve.

[0097] Since the position corresponding to each of the N sub-blocks can be the position of a point in the coordinate system where the second white balance Planck curve is located, the distance value between this point in the coordinate system and the second white balance Planck curve can be determined according to the method used to calculate the distance value from a point to a line in the relevant technology, and the determined distance value can be used as the distance information between the position corresponding to the sub-block and the second white balance Planck curve.

[0098] Step 420: Determine the weights corresponding to the plurality of sub-blocks based on the distance information corresponding to the plurality of sub-blocks.

[0099] Optionally, an objective function may be pre-set in which the independent variable is distance information, the dependent variable is weight, and the dependent variable and the independent variable are negatively correlated. As an example, the objective function may be a linear function with a slope less than 0, an exponential function with a base less than 1, and the like.

[0100] Optionally, for each of the N sub-blocks, the distance information corresponding to the sub-block can be taken as the value of the independent variable and substituted into the above objective function for calculation to obtain the corresponding value of the dependent variable, and the obtained value can be used as the weight corresponding to the sub-block.

[0101] In this way, the weights corresponding to the multiple sub-blocks can be reasonably determined based on the distance information between the positions corresponding to the multiple sub-blocks and the second white balance Planck curve. For example, the weights corresponding to the multiple sub-blocks can be determined according to the following rules: the closer the position corresponding to the sub-block is to the second white balance Planck curve, the greater the weight corresponding to the sub-block; the farther the position corresponding to the sub-block is from the second white balance Planck curve, the smaller the weight corresponding to the sub-block. The determined weights are used to determine the white balance compensation parameters, which is conducive to ensuring the rationality and reliability of the white balance compensation parameters.

[0102] In some other optional embodiments of the present disclosure, Figure 5 As shown, step 330 may include step 510 , step 520 and step 530 .

[0103] Step 510: In the coordinate system where the second white balance Planck curve is located, the second white balance Planck curve is expanded to obtain an expanded area.

[0104] Optionally, in the coordinate system where the second white balance Planck curve is located, the second white balance Planck curve can be used as the center line and expanded to both sides by 1 / 2 of the preset width to obtain an expansion area with a width of the preset width. The value of the preset width can be set according to actual conditions, and the present disclosure does not limit this.

[0105] Step 520: Determine distribution information of positions corresponding to the plurality of sub-blocks relative to the dilated area.

[0106] Optionally, for each sub-block among the N sub-blocks, distribution information of a position corresponding to the sub-block relative to the expansion region may be used to indicate whether the position corresponding to the sub-block is located within the expansion region.

[0107] Step 530: Determine the weights corresponding to the plurality of sub-blocks based on the distribution information corresponding to the plurality of sub-blocks.

[0108] Optionally, for each of the N sub-blocks, if the distribution information corresponding to the sub-block is used to characterize that the position corresponding to the sub-block is located within the expansion region, the first preset weight may be used as the weight corresponding to the sub-block. If the distribution information corresponding to the sub-block is used to characterize that the position corresponding to the sub-block is not located within the expansion region, the second preset weight may be used as the weight corresponding to the sub-block. As an example, the first preset weight may be 1, and the second preset weight may be 0.

[0109] In this way, by performing expansion processing on the second white balance Planck curve, the expansion area can be determined efficiently and quickly, and the weights corresponding to the multiple sub-blocks can be reasonably determined according to the distribution information of the positions corresponding to the multiple sub-blocks relative to the expansion area. For example, the weights corresponding to the multiple sub-blocks can be determined according to the following rules: if the position corresponding to the sub-block is within the expansion area, the weight corresponding to the sub-block is 1; if the position corresponding to the sub-block is outside the expansion area, the weight corresponding to the sub-block is 0. Using the determined weights to determine the white balance compensation parameters is conducive to ensuring the rationality and reliability of the white balance compensation parameters.

[0110] Optionally, Figure 4 The embodiment shown and Figure 5 The illustrated embodiments may also be used in combination. For example, the weights corresponding to the multiple sub-blocks may be determined based on the distance information corresponding to the multiple sub-blocks and the distribution information corresponding to the multiple sub-blocks.

[0111] Regardless of the implementation method of step 330, after the weights corresponding to the multiple sub-blocks are determined, step 340 can be performed based on the weights corresponding to the multiple sub-blocks.

[0112] Step 340 , weighting the positions corresponding to the multiple sub-blocks respectively using the weights corresponding to the multiple sub-blocks respectively, to obtain weighted positions.

[0113] Optionally, the multiple sub-blocks may be N sub-blocks, the N sub-blocks may correspond one-to-one to N positions, and the N sub-blocks may also correspond one-to-one to N weights. Using the N weights, the N positions may be weighted averaged to obtain a weighted position.

[0114] In one example, the N weights are represented as K1, K2, K3, ..., K N , the N positions are represented as P1, P2, P3, ..., P N , the weighted position is expressed as P, then we can have:

[0115] P=(K1*P1+K2*P2+K3*P3+……+K N *P N ) / (K1+K2+K3+……+K N )

[0116] Step 350 , determining white balance compensation parameters adapted to the target image sensor based on the weighted positions and the inverse matrix of the target matrix.

[0117] Optionally, the weighted position may be the position of a point in the coordinate system where the second white balance Planck curve is located. Then, the white balance compensation parameters adapted to the target image sensor may be determined based on the abscissa and ordinate of the point and the inverse matrix of the target matrix. As an example, the target matrix may be expressed as Mqe, and the inverse matrix of the target matrix may be expressed as Mqe -1 .

[0118] In the embodiments of the present disclosure, the positions corresponding to the multiple sub-blocks can be reasonably determined in the coordinate system where the second white balance Planck curve is located based on the target matrix and the color information of each of the multiple sub-blocks included in the target gray area. On this basis, the weights corresponding to the multiple sub-blocks can be reasonably determined, and the determined positions can be weighted according to the determined weights. The weighted positions obtained by weighting and the inverse matrix of the target matrix can be used to determine the white balance compensation parameters. In this way, the adverse effects that may be caused by considering only the relevant information of a single sub-block can be avoided by introducing the weighted average operation, which is conducive to ensuring the rationality and reliability of the determined white balance compensation parameters.

[0119] In some embodiments, the number of first images may be multiple, for example, M. For each of the M first images, a corresponding target gray area may be determined. For the target gray area corresponding to each first image, Figure 3In the illustrated embodiment, the corresponding white balance compensation parameters are determined, and then a mean operation or a weighted average operation can be performed on the white balance compensation parameters corresponding to the M first images, and the obtained mean operation result or weighted average operation result is used as the white balance compensation parameter adapted to the target image sensor.

[0120] Figure 6 It is a flowchart of a method for determining positions corresponding to a plurality of sub-blocks respectively in a coordinate system where a second white balance Planck curve is located, based on a target matrix and respective color information of the plurality of sub-blocks, provided by some exemplary embodiments of the present disclosure. Figure 6 The method shown may include step 610 , step 620 , step 630 , and step 640 . Figure 6 In the method shown, the color filter array of the target image sensor may be an RCCG array.

[0121] Step 610 , for a target subblock among the multiple subblocks, based on the color information of the target subblock, determine a first ratio of the pixel value of the R channel to the pixel value of the C channel, and a second ratio of the pixel value of the G channel to the pixel value of the C channel.

[0122] Optionally, the multiple sub-blocks may be N sub-blocks, the target sub-block may be any sub-block among the N sub-blocks, and the “target” in the target sub-block does not constitute a limitation on the target sub-block.

[0123] Optionally, the color information of the target sub-block may include a first pixel value of the target sub-block in the R channel, a second pixel value in the C channel, and a third pixel value in the G channel. Then, the first pixel value may be divided by the second pixel value to obtain a first ratio of the pixel value of the R channel to the pixel value of the C channel. In addition, the third pixel value may be divided by the second pixel value to obtain a second ratio of the pixel value of the G channel to the pixel value of the C channel.

[0124] Step 620: Convert the first ratio into a third ratio using the target matrix.

[0125] Optionally, the first ratio may be multiplied by the target matrix, and the multiplication result may be used as the third ratio.

[0126] Step 630: using the target matrix, convert the second ratio into a fourth ratio.

[0127] Optionally, the second ratio may be multiplied by the target matrix, and the multiplication result may be used as the fourth ratio.

[0128] Step 640: Determine the position corresponding to the target sub-block in the coordinate system where the second white balance Planck curve is located based on the third ratio and the fourth ratio.

[0129] Optionally, the third ratio can be considered to belong to the first type of ratio, and the fourth ratio can be considered to belong to the second type of ratio. Then, in the coordinate system where the second white balance Planck curve is located, the position with the horizontal coordinate being the third ratio and the vertical coordinate being the fourth ratio can be determined, and this position can be used as the position corresponding to the target sub-block.

[0130] In the embodiments of the present disclosure, the first ratio and the second ratio can be determined efficiently and quickly based on the color information of the target sub-block. The first ratio and the second ratio can be converted into the third ratio and the fourth ratio by using the target matrix. In this way, an effective reference can be provided for determining the position corresponding to the target sub-block, which is conducive to ensuring the rationality of the determined position.

[0131] Of course, the position corresponding to the target sub-block is not limited to the position where the horizontal axis is the third ratio and the vertical axis is the fourth ratio in the coordinate system of the second white balance Planck curve. For example, the position corresponding to the target sub-block may also have a slight deviation from the position where the horizontal axis is the third ratio and the vertical axis is the fourth ratio.

[0132] Figure 7 It is a flowchart of a method for determining white balance compensation parameters adapted to a target image sensor based on weighted positions and an inverse matrix of a target matrix according to some exemplary embodiments of the present disclosure. Figure 7 The method shown may include step 710 , step 720 , step 730 , step 740 , step 750 , and step 760 . Figure 7 In the method shown, the color filter array of the target image sensor may be an RCCG array.

[0133] Step 710 , based on the weighted positions, determine a fifth ratio of the pixel value of the R channel to the pixel value of the C channel, and a sixth ratio of the pixel value of the G channel to the pixel value of the C channel.

[0134] Optionally, the abscissa of the weighted position may be taken as the fifth ratio of the pixel value of the R channel to the pixel value of the C channel, and the ordinate of the weighted position may be taken as the sixth ratio of the pixel value of the G channel to the pixel value of the C channel.

[0135] Step 720: Convert the fifth ratio into a seventh ratio using the inverse matrix of the target matrix.

[0136] Optionally, the fifth ratio may be multiplied by an inverse matrix of the target matrix, and the multiplication result may be used as the seventh ratio.

[0137] Step 730, determining a first color compensation coefficient associated with the R channel based on the seventh ratio.

[0138] Optionally, the reciprocal of the seventh ratio may be used as a first color compensation coefficient associated with the R channel. The first color compensation coefficient may be a coefficient for color compensation between the R channel and the C channel. The first color compensation coefficient may also be expressed as Rgain. Assuming that the seventh ratio is expressed as R / C, then:

[0139] Rgain=1 / (R / C)

[0140] Step 740: Convert the sixth ratio into an eighth ratio using the inverse matrix of the target matrix.

[0141] Optionally, the sixth ratio may be multiplied by an inverse matrix of the target matrix, and the multiplication result may be used as the eighth ratio.

[0142] Step 750, determining a second color compensation coefficient associated with the G channel based on the eighth ratio.

[0143] Optionally, the reciprocal of the eighth ratio may be used as a second color compensation coefficient associated with the G channel. The second color compensation coefficient may be a coefficient for color compensation between the G channel and the C channel. The second color compensation coefficient may also be expressed as Ggain. Assuming that the eighth ratio is expressed as G / C, then:

[0144] Ggain=1 / (G / C)

[0145] Step 760: Determine a white balance compensation parameter adapted to the target image sensor based on the first color compensation coefficient and the second color compensation coefficient.

[0146] Optionally, the white balance compensation parameters adapted to the target image sensor may include a first color compensation coefficient and a second color compensation coefficient.

[0147] In one example, the target image sensor is an RCCG image sensor, the second image is Raw data collected by the RCCG image sensor, and there are V pixels in the Raw data, of which V1 pixels correspond to the R channel, V2 pixels correspond to the C channel, and V3 pixels correspond to the G channel. Then, the pixel values ​​of the V1 pixels can be multiplied by the first color compensation coefficient, and the pixel values ​​of the V3 pixels can be multiplied by the second color compensation coefficient, so that the Raw data after white balance correction can be obtained.

[0148] Of course, the Raw data serving as the second image may also be interpolated, and the interpolation result may include full-resolution images corresponding to the R channel, the C channel, and the G channel, respectively. The interpolation result may be considered as an image suitable for user observation, and then the first color compensation coefficient and the second color compensation coefficient may be used to perform white balance correction on the interpolation result.

[0149] In the embodiments of the present disclosure, based on the weighted positions, the fifth ratio of the pixel value of the R channel to the pixel value of the C channel, and the sixth ratio of the pixel value of the G channel to the pixel value of the C channel can be efficiently and quickly determined. By utilizing the inverse matrix of the target matrix, the fifth ratio can be efficiently and quickly converted into the seventh ratio, and the sixth ratio can be converted into the eighth ratio. In this way, an effective reference can be provided for determining the first color compensation coefficient and the second color compensation coefficient, so as to provide guidance on how to perform color compensation between channels. On this basis, more reasonable and reliable white balance compensation parameters can be determined.

[0150] In some optional examples, the optical specifications of the target image sensor may be the same as the optical specifications of the reference image sensor.

[0151] It should be noted that the same optical specifications of the target image sensor as the reference image sensor can be understood as: except for the different color filter arrays, the other parameters of the target image sensor and the reference image sensor are the same. For example, the resolution, pixel size, shutter type, etc. of the target image sensor and the reference image sensor can be the same. In this way, it is helpful to avoid the influence of the difference in optical specifications of the target image sensor and the reference image sensor on the determination of the white balance compensation parameters, thereby ensuring the rationality and reliability of the determined white balance compensation parameters.

[0152] In some optional examples, the second image may be captured at a later time than the first image.

[0153] Optionally, the first image may be an image captured by the target image sensor at time t0, and the second image may be an image captured by the target image sensor at time t1, and time t1 may be later than time t0. As an example, time t1 may be the next time after time t0, or time t1 may be separated from time t0 by a preset time interval. In this way, a good white balance correction processing effect can be achieved for the image captured by the target image sensor after time t1.

[0154] In some optional examples, the reference image sensor may be an RGGB image sensor, and the target image sensor may be an RCCG image sensor.

[0155] For the RCCG image sensor, the corresponding white balance Planck curve (equivalent to the first white balance Planck curve mentioned above) can be obtained by calibration. The specific calibration process may include:

[0156] (a1) At a certain color temperature, the 24-color color chart is imaged by the RCCG image sensor to obtain the corresponding Raw data. The obtained Raw data is divided into multiple sub-data, and based on the user input operation, a number of sub-data are selected from the divided multiple sub-data to form a white balance gray area (similar to the target gray area mentioned above). The white balance gray area can include 6 sub-blocks. For each of the 6 sub-blocks, the ratio Z1 of the pixel value of the R channel to the pixel value of the C channel can be counted, and the ratio Z2 of the pixel value of the G channel to the pixel value of the C channel can be counted.

[0157] (a2) Calculate the average values ​​of the 6 Z1s corresponding to the 6 sub-blocks to obtain the R / C ratio, and calculate the average values ​​of the 6 G / Cs corresponding to the 6 sub-blocks to obtain the G / C ratio, so as to obtain the R / Cratio and G / C ratio at a certain color temperature. In a similar manner, the R / C ratio and G / C ratio at multiple color temperatures can be obtained. Afterwards, the first white balance Planck curve can be constructed with the R / C ratio (equivalent to the first type of ratio value mentioned above) as the horizontal coordinate and the G / C ratio (equivalent to the second type of ratio value mentioned above) as the vertical coordinate.

[0158] In addition, the quantum efficiency response curves of the RCCG image sensor and the RGGB image sensor can be determined respectively. The QE correction matrix (equivalent to the target matrix mentioned above) used to map the quantum efficiency response curve of the RCCG image sensor to the quantum efficiency response curve of the RGGB image sensor can be determined by the least square method or other regression algorithms. The first white balance Planck curve can be converted into the second white balance Planck curve using the QE correction matrix.

[0159] For each of the 6 sub-blocks involved in (a1), the Z1 and Z2 corresponding to the sub-block can be multiplied by the QE correction matrix to determine the landing point of the sub-block in the coordinate system where the second white balance Planck curve is located (equivalent to the position corresponding to the sub-block). According to the rule that the closer the landing point is to the second white balance Planck curve, the greater the weight, a weight can be determined for each of the 6 sub-blocks. Using the determined weights, the positions corresponding to the 6 sub-blocks can be weighted averaged to obtain a weighted position. The abscissa and ordinate of the weighted position are multiplied by the inverse matrix of the QE correction matrix, respectively, to obtain the seventh ratio and the eighth ratio mentioned above. Afterwards, the reciprocals of the seventh ratio and the eighth ratio can be calculated respectively to obtain the Rgain and Ggain mentioned above, and Rgain and Ggain can constitute white balance compensation parameters adapted to the RCCG image sensor. For the Raw data subsequently collected by the RCCG image sensor, the white balance compensation parameters can be used to perform white balance correction processing on the Raw data to achieve color compensation between channels, so as to make white objects in the real physical world appear white in the image as much as possible.

[0160] The inventors have found through experiments that in the related art, the white balance Planck curve of the RCCG image sensor can be obtained by Figure 8-1 In the embodiment of the present disclosure, the second white balance Planck curve can be defined by Figure 8-2 The 7 points shown are defined. Figure 8-1 and Figure 8-2 In the table, D75, D65, D50, TL84, CWF, A, and H represent different light sources, and different light sources correspond to different color temperatures. Figure 8-1 and Figure 8-2 It can be seen that Figure 8-2 The points corresponding to different color temperatures are relative to Figure 8-1 By increasing the positional distinction of points corresponding to different color temperatures, it is helpful to avoid the influence of color temperature interference on the white balance correction processing effect, thereby improving the white balance correction processing effect.

[0161] It should be noted that the target image sensor is not limited to the RCCG image sensor, and theoretically can be any non-RGGB image sensor. The present disclosure does not limit the specific type of the target image sensor.

[0162] In summary, in the embodiments of the present disclosure, for images captured by an image sensor using the second type of color filter array, a new and adapted white balance correction method can be used to achieve a good white balance correction processing effect.

[0163] Exemplary Devices

[0164] Fig. 9 It is a schematic diagram of the structure of a white balance correction device for an image provided by some exemplary embodiments of the present disclosure. Fig. 9 The apparatus shown may include:

[0165] A first determination module 910, configured to determine a first quantum efficiency response curve of a reference image sensor;

[0166] A second determination module 920 is used to determine a second quantum efficiency response curve of a target image sensor whose color filter array is different from that of a reference image sensor;

[0167] A third determination module 930 is used to determine a target matrix for conversion between the second quantum efficiency response curve determined by the second determination module 920 and the first quantum efficiency response curve determined by the first determination module 910;

[0168] A fourth determination module 940 is used to determine a first white balance Planck curve of the target image sensor;

[0169] a fifth determination module 950, configured to determine white balance compensation parameters adapted to the target image sensor based on the target matrix determined by the third determination module 930, the first white balance Planck curve determined by the fourth determination module 940, and the first image captured by the target image sensor;

[0170] The processing module 960 is used to perform white balance correction processing on the second image captured by the target image sensor according to the white balance compensation parameter determined by the fifth determining module 950.

[0171] In some optional examples, such as Fig.10 As shown, the fifth determination module 950 includes:

[0172] A conversion submodule 1010, configured to convert the first white balance Planck curve determined by the fourth determination module 940 into a second white balance Planck curve of the reference image sensor by using the target matrix determined by the third determination module 930;

[0173] A first determination submodule 1020 is used to determine a target gray area in a first image captured by a target image sensor;

[0174] The second determination submodule 1030 is used to determine white balance compensation parameters adapted to the target image sensor based on the target matrix determined by the third determination module 930, the second white balance Planck curve obtained by the conversion submodule 1010, and the target gray area determined by the first determination submodule 1020.

[0175] In some optional examples, such as Fig.11 As shown, the second determining submodule 1030 includes:

[0176] A first determining unit 1110 is used to determine color information of each of the multiple sub-blocks included in the target gray area determined by the first determining submodule 1020;

[0177] The second determining unit 1120 is used to determine the positions corresponding to the plurality of sub-blocks in the coordinate system where the second white balance Planck curve is located based on the target matrix determined by the third determining module 930 and the color information of the plurality of sub-blocks determined by the first determining unit 1110;

[0178] The third determination unit 1130 is used to determine the weights corresponding to the plurality of sub-blocks respectively based on the second white balance Planck curve obtained by the conversion sub-module 1010 and the positions corresponding to the plurality of sub-blocks respectively determined by the second determination unit 1120;

[0179] The weighting unit 1140 is used to weight the positions corresponding to the multiple sub-blocks determined by the second determining unit 1120 by using the weights corresponding to the multiple sub-blocks determined by the third determining unit 1130 to obtain weighted positions;

[0180] The fourth determining unit 1150 is used to determine the white balance compensation parameters adapted to the target image sensor based on the weighted positions obtained by the weighting unit 1140 and the inverse matrix of the target matrix.

[0181] In some optional examples, the color filter array of the target image sensor is an RCCG array;

[0182] like Fig.12 As shown, the second determining unit 1120 includes:

[0183] A first determining subunit 1210 is used to determine, for a target subblock among the multiple subblocks, a first ratio of a pixel value of an R channel to a pixel value of a C channel, and a second ratio of a pixel value of a G channel to a pixel value of a C channel based on color information of the target subblock;

[0184] A first conversion subunit 1220 is configured to convert the first ratio determined by the first determination subunit 1210 into a third ratio by using the target matrix determined by the third determination module 930;

[0185] A second conversion subunit 1230 is configured to convert the second ratio determined by the first determination subunit 1210 into a fourth ratio by using the target matrix determined by the third determination module 930;

[0186] The second determining subunit 1240 is used to determine the position corresponding to the target subblock in the coordinate system where the second white balance Planck curve is located based on the third ratio converted by the first converting subunit 1220 and the fourth ratio converted by the second converting subunit 1230.

[0187] In some optional examples, the color filter array of the target image sensor is an RCCG array;

[0188] like Fig.13 As shown, the fourth determining unit 1150 includes:

[0189] The third determining subunit 1310 is used to determine a fifth ratio of the pixel value of the R channel to the pixel value of the C channel, and a sixth ratio of the pixel value of the G channel to the pixel value of the C channel based on the weighted position obtained by the weighting unit 1140;

[0190] A third conversion subunit 1320 is used to convert the fifth ratio determined by the third determination subunit 1310 into a seventh ratio by using an inverse matrix of the target matrix;

[0191] The fourth determining subunit 1330 is used to determine a first color compensation coefficient associated with the R channel based on the seventh ratio obtained by the third converting subunit 1320;

[0192] The fourth conversion subunit 1340 is used to convert the sixth ratio determined by the third determination subunit 1310 into an eighth ratio by using an inverse matrix of the target matrix;

[0193] a fifth determining subunit 1350, configured to determine a second color compensation coefficient associated with the G channel based on the eighth ratio obtained by the fourth converting subunit 1340;

[0194] The sixth determining subunit 1360 is used to determine a white balance compensation parameter adapted to the target image sensor based on the first color compensation coefficient determined by the fourth determining subunit 1330 and the second color compensation coefficient determined by the fifth determining subunit 1350 .

[0195] In some optional examples, such as Fig.14 As shown, the third determining unit 1130 includes:

[0196] A seventh determining subunit 1410 is used to determine distance information between positions corresponding to the plurality of sub-blocks and a second white balance Planck curve;

[0197] The eighth determining subunit 1420 is configured to determine weights corresponding to the plurality of subblocks respectively based on the distance information corresponding to the plurality of subblocks respectively determined by the seventh determining subunit 1410 .

[0198] In some optional examples, such as Fig.15 As shown, the third determining unit 1130 includes:

[0199] The expansion subunit 1510 is used to expand the second white balance Planck curve in the coordinate system where the second white balance Planck curve is located to obtain an expansion area;

[0200] A ninth determining subunit 1520, configured to determine distribution information of positions corresponding to the plurality of sub-blocks relative to the dilated area obtained by the dilation subunit 1510;

[0201] The tenth determining subunit 1530 is used to determine the weights corresponding to the multiple sub-blocks respectively based on the distribution information corresponding to the multiple sub-blocks respectively determined by the ninth determining subunit 1520.

[0202] In some optional examples,

[0203] The color filter array of the reference image sensor is an RGGB array;

[0204] and / or,

[0205] The optical specifications of the target image sensor are the same as the optical specifications of the reference image sensor;

[0206] and / or,

[0207] The acquisition time of the second image is later than the acquisition time of the first image.

[0208] In the device of the present disclosure, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.

[0209] Exemplary Electronic Devices

[0210] Fig.16 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 1600 includes one or more processors 1610 and a memory 1620 .

[0211] The processor 1610 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions.

[0212] The memory 1620 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1610 may execute one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0213] In one example, the electronic device 1600 may further include: an input device 1630 and an output device 1640 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0214] The input device 1630 may also include, for example, a keyboard, a mouse, etc.

[0215] The output device 1640 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0216] Of course, to simplify, Fig.16 Only some of the components related to the present disclosure in the electronic device 1600 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 1600 may also include any other appropriate components.

[0217] Exemplary computer program products and computer-readable storage media

[0218] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0219] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0220] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0221] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0222] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. The specific details disclosed above are only for the purpose of illustration and ease of understanding, not limitation. The above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0223] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for correcting white balance of an image, comprising: determining a first quantum efficiency response curve of a reference image sensor; Determining a second quantum efficiency response curve of a target image sensor, wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor; determining a target matrix for conversion between the second quantum efficiency response curve and the first quantum efficiency response curve; Determining a first white balance Planckian curve of the target image sensor; Determining white balance compensation parameters adapted to the target image sensor based on the target matrix, the first white balance Planck curve, and a first image captured by the target image sensor; According to the white balance compensation parameters, white balance correction processing is performed on the second image captured by the target image sensor.

2. The method according to claim 1, wherein: The step of determining white balance compensation parameters adapted to the target image sensor based on the target matrix, the first white balance Planck curve, and the first image captured by the target image sensor includes: Using the target matrix, converting the first white balance Planckian curve into a second white balance Planckian curve of the reference image sensor; Determine a target gray area in a first image captured by the target image sensor; Based on the target matrix, the second white balance Planckian curve and the target gray area, a white balance compensation parameter adapted to the target image sensor is determined.

3. The method according to claim 2, wherein: The step of determining a white balance compensation parameter adapted to the target image sensor based on the target matrix, the second white balance Planck curve, and the target gray area includes: Determining color information of each of the plurality of sub-blocks included in the target gray area; Based on the target matrix and the color information of each of the plurality of sub-blocks, determining positions corresponding to the plurality of sub-blocks in a coordinate system where the second white balance Planck curve is located; Determining weights corresponding to the plurality of sub-blocks respectively based on the second white balance Planck curve and the positions corresponding to the plurality of sub-blocks respectively; Using the weights respectively corresponding to the plurality of sub-blocks, weighting the positions respectively corresponding to the plurality of sub-blocks to obtain weighted positions; Based on the weighted positions and the inverse matrix of the target matrix, white balance compensation parameters adapted to the target image sensor are determined.

4. The method according to claim 3, wherein: The color filter array of the target image sensor is an RCCG array; The determining, based on the target matrix and the color information of each of the plurality of sub-blocks, in a coordinate system where the second white balance Planck curve is located, positions corresponding to the plurality of sub-blocks respectively include: For a target sub-block among the plurality of sub-blocks, determining a first ratio of a pixel value of an R channel to a pixel value of a C channel, and a second ratio of a pixel value of a G channel to a pixel value of a C channel based on color information of the target sub-block; Using the target matrix, converting the first ratio into a third ratio; using the target matrix, converting the second ratio into a fourth ratio; Based on the third ratio and the fourth ratio, a position corresponding to the target sub-block is determined in a coordinate system where the second white balance Planck curve is located.

5. The method according to claim 3, wherein: The color filter array of the target image sensor is an RCCG array; The determining, based on the weighted position and the inverse matrix of the target matrix, a white balance compensation parameter adapted to the target image sensor comprises: Based on the weighted positions, determining a fifth ratio of the pixel value of the R channel to the pixel value of the C channel, and a sixth ratio of the pixel value of the G channel to the pixel value of the C channel; Using an inverse matrix of the target matrix, converting the fifth ratio into a seventh ratio; determining a first color compensation coefficient associated with the R channel based on the seventh ratio; Using an inverse matrix of the target matrix, converting the sixth ratio into an eighth ratio; determining a second color compensation coefficient associated with the G channel based on the eighth ratio; Based on the first color compensation coefficient and the second color compensation coefficient, a white balance compensation parameter adapted to the target image sensor is determined.

6. The method according to claim 3, wherein: The determining weights corresponding to the plurality of sub-blocks respectively based on the second white balance Planck curve and the positions corresponding to the plurality of sub-blocks respectively includes: Determine distance information between the positions respectively corresponding to the plurality of sub-blocks and the second white balance Planck curve; Based on the distance information respectively corresponding to the plurality of sub-blocks, weights respectively corresponding to the plurality of sub-blocks are determined.

7. The method according to claim 3, wherein: The determining weights corresponding to the plurality of sub-blocks respectively based on the second white balance Planck curve and the positions corresponding to the plurality of sub-blocks respectively includes: In the coordinate system, dilating the second white balance Planck curve to obtain a dilated area; Determine distribution information of the positions respectively corresponding to the plurality of sub-blocks relative to the expansion area; Based on the distribution information respectively corresponding to the plurality of sub-blocks, weights respectively corresponding to the plurality of sub-blocks are determined.

8. The method according to any one of claims 1 to 7, wherein: The color filter array of the reference image sensor is an RGGB array; and / or, The optical specifications of the target image sensor are the same as the optical specifications of the reference image sensor; and / or, The acquisition time of the second image is later than the acquisition time of the first image.

9. A white balance correction device for an image, comprising: A first determination module, used to determine a first quantum efficiency response curve of a reference image sensor; A second determination module, configured to determine a second quantum efficiency response curve of a target image sensor whose color filter array is different from the reference image sensor; a third determination module, configured to determine a target matrix for conversion between the second quantum efficiency response curve determined by the second determination module and the first quantum efficiency response curve determined by the first determination module; A fourth determination module, configured to determine a first white balance Planck curve of the target image sensor; a fifth determining module, configured to determine white balance compensation parameters adapted to the target image sensor based on the target matrix determined by the third determining module, the first white balance Planck curve determined by the fourth determining module, and the first image captured by the target image sensor; A processing module is used to perform white balance correction processing on the second image captured by the target image sensor according to the white balance compensation parameter determined by the fifth determination module.

10. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image white balance correction method according to any one of claims 1 to 8.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image white balance correction method described in any one of claims 1-8.

Images