Calibration Method, Device, Equipment, Readable Storage Medium and Program Product

Abstract

Embodiments of the present application provide a calibration method, device, equipment, readable storage medium, and program product. The method includes: obtaining an image to be calibrated; determining the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be calibrated; determining color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information; and performing color calibration on the image to be calibrated based on the color correction matrices corresponding to at least two color temperatures to determine the calibrated image. In this way, the problem of color cast in a mixed color temperature scene is effectively solved, the gap between the color of the calibrated image and the true color is reduced, and the efficiency of color calibration is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology. Specifically, this application relates to a calibration method, apparatus, device, readable storage medium, and program product. Background Art

[0002] The color perception of humans is mainly determined by the reflectivity of objects. The color of an object observed under different lighting conditions is constant, that is, it has color constancy. However, an image acquisition device (such as a digital camera, etc.) does not have this characteristic. The image color obtained by the image acquisition device is jointly determined by the light source, the reflectivity of the imaging object, and the spectral response function of the imaging system. Therefore, the obtained image will have a color difference due to changes in lighting conditions and the response deviation of the sensor to the RGB spectrum, that is, the color cast problem in the color temperature scene. In order to correct the color difference and make the image closer to the color seen by the human eye, it is necessary to calibrate the color of the image acquired by the image acquisition device to restore the true color of the image, so as to obtain the intrinsic color of the object in the image. The prior art cannot solve the color cast problem in the color temperature scene (such as a mixed color temperature scene), resulting in low efficiency of color calibration. Summary of the Invention

[0003] Aiming at the shortcomings of the existing methods, this application proposes a calibration method, apparatus, device, computer-readable storage medium, and computer program product to solve the problem of how to improve the efficiency of color calibration.

[0004] In a first aspect, this application provides a calibration method, including:

[0005] Obtain an image to be calibrated;

[0006] Determine the color temperature information and gain information of the color temperature scene in the image to be calibrated;

[0007] Based on the color temperature information and gain information, determine color correction matrices corresponding to at least two color temperatures in the color temperature scene;

[0008] Based on the color correction matrices corresponding to at least two color temperatures, perform color calibration on the image to be calibrated to determine the calibrated image.

[0009] In one embodiment, obtaining the image to be calibrated includes:

[0010] Obtain an original image;

[0011] Perform preprocessing on the original image to obtain the image to be calibrated, and the preprocessing includes black level calibration.

[0012] In one embodiment, determining the color temperature information and gain information of the color temperature scene in the image to be calibrated includes:

[0013] If the color temperature scene in the image to be corrected is a mixed color temperature scene, the image to be corrected is processed by a white balance algorithm to obtain second color temperature estimation values of at least two color temperatures in the mixed color temperature scene, a set of white balance gains for each of the at least two color temperatures, and a set of mixed color temperature white balance gains for the at least two color temperatures;

[0014] Among them, the color temperature information of the color temperature scene includes second color temperature estimation values of at least two color temperatures, and the gain information of the color temperature scene includes a set of white balance gains for each of the at least two color temperatures and a set of mixed color temperature white balance gains for the at least two color temperatures.

[0015] In one embodiment, determining color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information includes:

[0016] Determining color correction matrices corresponding to at least two color temperatures in the color temperature scene based on a plurality of preset reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information.

[0017] In one embodiment, determining color correction matrices corresponding to at least two color temperatures in the color temperature scene based on a plurality of preset reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information includes:

[0018] If the color temperature scene in the image to be corrected is a mixed color temperature scene, then based on a set of white balance gains respectively corresponding to at least two color temperatures in the mixed color temperature scene and a set of mixed color temperature white balance gains for the at least two color temperatures, determine the angular error between each set of white balance gains and the set of mixed color temperature white balance gains;

[0019] Based on each angular error, determine the fusion weight for each of the at least two color temperatures;

[0020] Based on a plurality of preset reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, and second color temperature estimation values of at least two color temperatures, determine an initial color correction matrix for each color temperature;

[0021] Based on the fusion weight for each color temperature and the initial color correction matrix for each color temperature, perform a fusion process to determine color correction matrices corresponding to at least two color temperatures;

[0022] Among them, the color temperature information includes second color temperature estimation values of at least two color temperatures, and the gain information includes a set of white balance gains respectively corresponding to at least two color temperatures and a set of mixed color temperature white balance gains for the at least two color temperatures.

[0023] In a second aspect, the present application provides a correction device, including:

[0024] A first processing module, configured to obtain an image to be corrected;

[0025] A second processing module, configured to determine the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be corrected;

[0026] A third processing module, configured to determine a color correction matrix corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information;

[0027] A fourth processing module, configured to perform color correction on the image to be corrected based on the color correction matrix corresponding to at least two color temperatures, and determine the corrected image.

[0028] In one embodiment, the first processing module is specifically configured to:

[0029] Obtain the original image;

[0030] Perform preprocessing on the original image to obtain the image to be corrected, where the preprocessing includes black level correction.

[0031] In one embodiment, the second processing module is specifically configured to:

[0032] If the color temperature scene in the image to be corrected is a mixed color temperature scene, process the image to be corrected through a white balance algorithm to obtain second color temperature estimates of at least two color temperatures in the mixed color temperature scene, a set of white balance gains for each of the at least two color temperatures, and a set of mixed color temperature white balance gains for the at least two color temperatures;

[0033] Wherein, the color temperature information of the color temperature scene includes second color temperature estimates of at least two color temperatures, and the gain information of the color temperature scene includes a set of white balance gains for each color temperature and a set of mixed color temperature white balance gains for the at least two color temperatures.

[0034] In one embodiment, the third processing module is specifically configured to:

[0035] Based on a plurality of preset reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information, determine a color correction matrix corresponding to at least two color temperatures in the color temperature scene.

[0036] In one embodiment, the third processing module is specifically configured to:

[0037] If the color temperature scene in the image to be corrected is a mixed color temperature scene, determine the angular error between each set of white balance gains and the set of mixed color temperature white balance gains based on the set of white balance gains respectively corresponding to at least two color temperatures in the mixed color temperature scene and the set of mixed color temperature white balance gains for the at least two color temperatures;

[0038] Based on each angular error, determine the fusion weight for each of the at least two color temperatures;

[0039] Determine the initial color correction matrix for each color temperature based on a plurality of preset reference color temperatures, the color correction matrices respectively corresponding to the plurality of reference color temperatures, and the second color temperature estimation values of at least two color temperatures;

[0040] Perform fusion processing based on the fusion weight of each color temperature and the initial color correction matrix of each color temperature to determine the color correction matrix corresponding to at least two color temperatures;

[0041] Among them, the color temperature information includes the second color temperature estimation values of at least two color temperatures, and the gain information includes a set of white balance gains respectively corresponding to at least two color temperatures and a set of mixed color temperature white balance gains of at least two color temperatures.

[0042] In a third aspect, the present application further provides an imaging device, including an image acquisition device and the correction device provided in the first aspect of the present application, and the image acquisition device is used to acquire an original image.

[0043] In a fourth aspect, the present application provides an electronic device, including: a processor, a memory, and a bus;

[0044] The bus is used to connect the processor and the memory;

[0045] The memory is used to store operation instructions;

[0046] The processor is used to execute the correction method in the first aspect of the present application by calling the operation instructions.

[0047] In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program, and the computer program is used to execute the correction method described in the first aspect of the present application.

[0048] In a sixth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the correction method in the first aspect of the present application.

[0049] The technical solution provided by the embodiments of the present application has at least the following beneficial effects:

[0050] Obtain an image to be corrected; determine the color temperature information of the color temperature scene in the image to be corrected and the gain information of the color temperature scene; based on the color temperature information and the gain information, determine color correction matrices corresponding to at least two color temperatures in the color temperature scene; based on the color correction matrices corresponding to the at least two color temperatures, perform color correction on the image to be corrected to determine the corrected image. In this way, the fusion weights of each color temperature in the at least two color temperatures can be calculated based on the color temperature information and the gain information, so as to weight the initial color correction matrices of each color temperature in the at least two color temperatures to obtain a fused color correction matrix, that is, the color correction matrices corresponding to the at least two color temperatures. Applying the color correction matrices corresponding to the at least two color temperatures to the color correction of the image to be corrected effectively solves the color cast problem in the mixed color temperature scene, reduces the gap between the color of the corrected image and the true color, and improves the efficiency of color correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application.

[0052] Figure 1 Schematic flowchart of a correction method provided by an embodiment of the present application;

[0053] Figure 2 Schematic flowchart of another correction method provided by an embodiment of the present application;

[0054] Figure 3 Schematic structural diagram of a correction device provided by an embodiment of the present application;

[0055] Figure 4 Schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following describes the embodiments of the present application with reference to the drawings in the present application. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions of the embodiments of the present application.

[0057] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include plural forms. It should be further understood that the terms "comprising" and "including" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the technical field of the present application, etc. It should be understood that when we say an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates being implemented as "A", or being implemented as "B", or being implemented as "A and B".

[0058] It can be understood that in the specific embodiments of the present application, when it comes to data related to image correction, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0059] To make the objectives, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0060] The correction method provided by the correction system in the embodiments of the present application involves fields such as image processing.

[0061] To better understand and illustrate the solutions of the embodiments of the present application, some technical terms involved in the embodiments of the present application will be briefly described below.

[0062] RGB: The RGB color model is a color standard in the industrial field. It obtains various colors through the changes of the three color channels of red (R), green (G), and blue (B) and their mutual superposition. RGB represents the colors of the three channels of red, green, and blue. This standard almost includes all colors that can be perceived by human vision and is one of the most widely used color systems.

[0063] The solutions provided by the embodiments of the present application involve image correction technology. The technical solutions of the present application will be described in detail below with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application with reference to the accompanying drawings.

[0064] See Figure 1 , Figure 1 which shows a schematic flowchart of a calibration method provided by an embodiment of the present application. Among them, this method can be executed by any electronic device. As an optional implementation, this method can be executed by an imaging device. Specifically, it is executed by a calibration device in the imaging device. For the convenience of description, in the description of some optional embodiments below, the calibration device will be taken as an example of the execution subject of this method. As shown Figure 1 , the calibration method provided by the embodiment of the present application includes the following steps:

[0065] S201, obtain the image to be calibrated.

[0066] Specifically, for example, the original image is collected by an image acquisition device, and the image acquisition device can be any type of image acquisition device, such as a CMOS image sensor, a CCD image sensor, etc. The format of the original image is the Bayer format, and the original image can be the original picture inside the camera, and the suffix name of the original picture can be.raw. Preprocess the original image to obtain the image to be calibrated; among them, the preprocessing can include DPC (defective pixel concealment, bad pixel correction), BLC (black level correction, black level correction), LSC (lens shading correction, lens shading correction), NR RAW (Noise Reduction for RAW, RAW domain noise reduction), etc.

[0067] S202, determine the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be calibrated.

[0068] Specifically, the color temperature scene can be a mixed color temperature scene; if there are at least two light sources in the color temperature scene, the color temperature scene where the images under at least two light sources are located is the mixed color temperature scene; for example, in an image illuminated by a fluorescent lamp and an incandescent lamp together, some parts of the image are mainly illuminated by the fluorescent lamp, and some parts of the image are mainly illuminated by the incandescent lamp, then there will be obvious color differences in the same color in different parts of the image, and color correction needs to be performed on this image.

[0069] S203, based on the color temperature information and the gain information, determine the color correction matrix corresponding to at least two color temperatures in the color temperature scene.

[0070] Specifically, the color temperature information can be a color temperature estimated value, and the gain information can be a set of white balance gains for each color temperature, a set of mixed color temperature white balance gains for at least two color temperatures, etc.

[0071] S204. Perform color correction on the image to be corrected based on color correction matrices corresponding to at least two color temperatures, and determine the corrected image.

[0072] Specifically, through matrix operations, calculate the product of the color correction matrices corresponding to at least two color temperatures and the matrix of the image to be corrected pixel by pixel to obtain the matrix of the corrected image.

[0073] In the embodiments of the present application, an image to be corrected is obtained; the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be corrected are determined; based on the color temperature information and the gain information, color correction matrices corresponding to at least two color temperatures in the color temperature scene are determined; and color correction is performed on the image to be corrected based on the color correction matrices corresponding to at least two color temperatures to determine the corrected image. In this way, the color cast problem in the color temperature scene is effectively solved, the gap between the color of the corrected image and the real color is reduced, and the efficiency of color correction is improved.

[0074] In one embodiment, obtaining the image to be corrected includes:

[0075] Obtain the original image;

[0076] Perform preprocessing on the original image to obtain the image to be corrected, and the preprocessing includes black level correction.

[0077] Specifically, the format of the original image is the Bayer format. The original image can be the original picture inside the camera, and the suffix name of the original picture can be.raw. The preprocessing can include DPC, BLC, LSC, NR RAW, etc.

[0078] In one embodiment, the original image needs to be at least subjected to BLC processing; wherein, the BLC processing includes: subtracting a fixed value (the experimentally measured black level value) from each pixel in the original image to obtain the image after BLC correction, that is, obtaining the image to be corrected.

[0079] It should be noted that when the sensor of the image acquisition device converts the analog signal into a digital signal, due to the conversion accuracy limitation, a very small part of the voltage value cannot be distinguished, so a value needs to be added to ensure the details of the dark part of the image; the black level refers to the minimum value of black. When the image sensor senses all-black (all-zero) data, the minimum signal value output corresponds to the black level value of the image sensor.

[0080] In one embodiment, determining the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be corrected includes:

[0081] If the color temperature scene in the image to be corrected is a single color temperature scene, then process the image to be corrected through the white balance algorithm to obtain the first color temperature estimate value of the single color temperature in the single color temperature scene and a set of white balance gains of the single color temperature;

[0082] Among them, the color temperature information of the color temperature scene includes a first color temperature estimation value, and the gain information of the color temperature scene includes a set of white balance gains for a single color temperature.

[0083] Specifically, the white balance algorithm is, for example, a white balance algorithm based on color temperature estimation, etc. A set of white balance gains for a single color temperature can be a set of white balance gain values, and a set of white balance gain values includes multiple white balance gain values; for example, each color channel in RGB corresponds to a white balance gain value, then a set of white balance gain values includes three white balance gain values.

[0084] In one embodiment, determining the color temperature information and the gain information of the color temperature scene in the image to be corrected includes:

[0085] If the color temperature scene in the image to be corrected is a mixed color temperature scene, then process the image to be corrected through a white balance algorithm to obtain second color temperature estimation values of at least two color temperatures in the mixed color temperature scene, a set of white balance gains for each of the at least two color temperatures, and a set of mixed color temperature white balance gains for the at least two color temperatures;

[0086] Among them, the color temperature information of the color temperature scene includes second color temperature estimation values of at least two color temperatures, and the gain information of the color temperature scene includes a set of white balance gains for each color temperature and a set of mixed color temperature white balance gains for the at least two color temperatures.

[0087] Specifically, the white balance algorithm is, for example, a white balance algorithm based on color temperature estimation, etc. A set of white balance gains for each color temperature in the mixed color temperature scene can be a set of white balance gain values, and a set of white balance gain values includes multiple white balance gain values; for example, each color channel in RGB corresponds to a white balance gain value, then a set of white balance gain values includes three white balance gain values.

[0088] In one embodiment, determining a color correction matrix corresponding to at least one color temperature in the color temperature scene based on at least one of the color temperature information and the gain information includes:

[0089] Based on a preset plurality of reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, and at least one of the color temperature information and the gain information, determine a color correction matrix corresponding to at least one color temperature in the color temperature scene.

[0090] In one embodiment, determining color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information includes:

[0091] Based on a preset plurality of reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information, determine color correction matrices corresponding to at least two color temperatures in the color temperature scene.

[0092] Specifically, the CCM (Color Correction Matrix) corresponding to each reference color temperature value among multiple reference color temperature values can be a 3×3 matrix. To ensure that the white balance is not disrupted, the sum of each row in the CCM is 1. In fact, only 6 parameters need to be obtained for the CCM. The CCM is shown in formula (1):

[0093]

[0094] where x1, x2, x3, x4, x5, and x6 are the 6 parameters in the CCM.

[0095] To calibrate the color to an approximate level of the human eye through the above 3×3 matrix, it is necessary to calibrate the image acquisition device at different color temperatures to calculate the CCM and construct a color temperature value - color correction matrix mapping table. For example, select 2670K (A light source), 3005K (U30 light source), 3541K (U35 light source), 3955K (TL84 light source), 4827K (D50 light source), 6059K (D65 light source), and 7157K (D75 light source) to calibrate and obtain the CCM; among them, the above light source color temperatures are all the color temperature values actually measured by a spectro - illuminometer at an illuminance of about 600 lx. For example, the following is the CCM corresponding to seven color temperatures obtained by calibrating the sensor of the image acquisition device, that is, the color temperature value - color correction matrix mapping table:

[0096] A: [0.24375532, -0.57738802, -0.52545141, -0.15197077, 0.08487664, -1.11628831]

[0097] U30: [0.04225663, -0.34149986, -0.55949693, -0.08417125, 0.0464566, -0.72671586]

[0098] U35: [-0.07854388, -0.3152296, -0.49007842, -0.11669218, 0.03448496, -0.6293961]

[0099] TL84: [-0.44151576, -0.25136999, -0.46006646, -0.05201912, 0.01489581, -0.46047852]

[0100] D50: [0.32105463, -0.52113296, -2.01137616, -0.27112471, -0.06282265, -0.46496113]

[0101] D65: [0.0897554, -0.41826068, -0.35877453, -0.26054942, -0.03223076, -0.45469695]

[0102] D75: [-0.13498796, -0.29285067, -0.47414933, -0.1062478, 0.04239653, -0.65921262]

[0103] By using the interpolation algorithm and taking the CCMs corresponding to the above seven color temperatures as the benchmarks, a color correction matrix for a certain target color temperature M(T X )(where the target color temperature is, for example, a color temperature in the color temperature scenario of the image to be corrected) can be obtained. For example, the interpolation algorithm is a linear interpolation algorithm, and the formula (2) of the linear interpolation algorithm is as follows:

[0104]

[0105] where T A 、T U30 、T U35 、T TL84 、T D50 、T D65 、T D75 represent the color temperature values of A, U30, U35, TL84, D50, D65, and D75 respectively, that is, multiple reference color temperature values; M(T A ), M(T U30 ), M(T U35 ), M(T TL84 ), M(T D50 ), M(T D65 ), M(T D75 ) represent the color correction matrices of A, U30, U35, TL84, D50, D65, and D75 respectively, that is, the color correction matrices corresponding to multiple reference color temperature values respectively. T X is the color temperature estimated value of the target color temperature, and M(T X ) is the color correction matrix of the target color temperature.

[0106] Based on the estimated n color temperature values, the color temperature values and color correction matrices of two adjacent color temperatures in the color temperature - color correction matrix mapping table are found; through formula (2), interpolation calculations are performed element by element at the corresponding positions to obtain the color correction matrix M(T X ) corresponding to TX )。

[0107] When n = 1, the color temperature scene in the image to be corrected is a single color temperature scene, and there is a unique color correction matrix corresponding to the single color temperature. The color correction matrix M(T X ) obtained by formula (2) is the color correction matrix corresponding to the single color temperature.

[0108] When n > 1, the color temperature scene in the image to be corrected is a mixed color temperature scene. According to a group of white balance gains corresponding to n color temperatures (at least two color temperatures in the mixed color temperature scene) and a group of mixed color temperature white balance gains corresponding to n color temperatures, calculate the angular error between each group of white balance gains and the group of mixed color temperature white balance gains, and obtain the fusion weight coefficient of each color temperature in the n color temperatures (the fusion weight of each color temperature in at least two color temperatures). According to the fusion weight coefficient of each color temperature, perform weighted calculation on the color correction matrix of each color temperature (the initial color correction matrix of each color temperature) to obtain the fused color correction matrix (the color correction matrix corresponding to at least two color temperatures).

[0109] In one embodiment, based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and at least one of color temperature information and gain information, determining the color correction matrix corresponding to at least one color temperature in the color temperature scene includes:

[0110] If the color temperature scene in the image to be corrected is a single color temperature scene, then based on the plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and the first color temperature estimated value of the single color temperature in the single color temperature scene, perform interpolation algorithm processing to determine the color correction matrix corresponding to the single color temperature; the color temperature information includes the first color temperature estimated value.

[0111] Specifically, when n = 1, the color temperature scene in the image to be corrected is a single color temperature scene, and there is a unique color correction matrix corresponding to the single color temperature. The color correction matrix M(T X ) obtained by formula (2) is the color correction matrix corresponding to the single color temperature.

[0112] In one embodiment, based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, color temperature information, and gain information, determining the color correction matrix corresponding to at least two color temperatures in the color temperature scene includes:

[0113] If the color temperature scene in the image to be corrected is a mixed color temperature scene, then based on a group of white balance gains respectively corresponding to at least two color temperatures in the mixed color temperature scene and a group of mixed color temperature white balance gains of at least two color temperatures, determine the angular error between each group of white balance gains and the group of mixed color temperature white balance gains;

[0114] Determine the fusion weight of each color temperature among at least two color temperatures based on the angular errors of each angle;

[0115] Based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and the second color temperature estimation values of at least two color temperatures, determine the initial color correction matrix of each color temperature;

[0116] Based on the fusion weight of each color temperature and the initial color correction matrix of each color temperature, perform a fusion process to determine the color correction matrix corresponding to at least two color temperatures;

[0117] Wherein, the color temperature information includes the second color temperature estimation values of at least two color temperatures, and the gain information includes a set of white balance gains respectively corresponding to at least two color temperatures and a set of mixed color temperature white balance gains of at least two color temperatures.

[0118] Specifically, when n > 1, the color temperature scene in the image to be corrected is a mixed color temperature scene. According to a set of white balance gains respectively corresponding to n color temperatures (at least two color temperatures in the mixed color temperature scene) and a set of mixed color temperature white balance gains corresponding to n color temperatures, calculate the angular errors between each set of white balance gains and a set of mixed color temperature white balance gains, obtain the fusion weight coefficients of each color temperature among n color temperatures (the fusion weight of each color temperature among at least two color temperatures), and perform weighted calculation on the color correction matrix of each color temperature (the initial color correction matrix of each color temperature) according to the fusion weight coefficients of each color temperature, to obtain the fused color correction matrix (the color correction matrix corresponding to at least two color temperatures).

[0119] For example, if n = 2, the calculation formula (3) of the fusion weight coefficient α is as follows:

[0120]

[0121] Wherein, gains is a set of mixed color temperature white balance gains (a set of mixed color temperature white balance gain values); gains i and gains j are respectively a set of white balance gains of two color temperatures (a set of white balance gain values). The set of white balance gains of each color temperature among the two color temperatures is a four-dimensional vector, and the format of the four-dimensional vector is [R gain , Gr gain , Gb gain , B gain ; D is the angular error, and the calculation formula (4) of D is as follows:

[0122]

[0123] Wherein, gain1 represents a set of mixed color temperature white balance gains, and gain2 represents a set of white balance gains.

[0124] The calculation formula (5) of the color correction matrix M after fusion (the color correction matrices corresponding to two color temperatures) is as follows:

[0125] M = αM i +(1 - α)M j Formula (5)

[0126] Where, M i and M j respectively represent the color correction matrix of one color temperature among the two color temperatures (the initial color correction matrix of one color temperature), and α and 1 - α respectively represent the fusion weight coefficients of one color temperature among the two color temperatures.

[0127] For another example, if n = 3, the calculation formula (6) of the fusion weight coefficients (α, β) is as follows:

[0128]

[0129] The calculation formula (7) of the color correction matrix M after fusion (the color correction matrix corresponding to three color temperatures) is as follows:

[0130] M = αM i +βM j +(1 - α - β)M k Formula (7)

[0131] Where, M i and M j and M k respectively represent the color correction matrix of one color temperature among the three color temperatures (the initial color correction matrix of one color temperature), and α, β, and (1 - α - β) respectively represent the fusion weight coefficients of one color temperature among the three color temperatures.

[0132] For another example, if n = N, the calculation formula (8) of the fusion weight coefficients (a1, a2, …, a N ) is as follows:

[0133]

[0134] The calculation formula (9) of the color correction matrix M after fusion (the color correction matrix corresponding to N color temperatures) is as follows:

[0135] M = a1M1 + a2M2 + … + (1 - a1 - a2 - … - a N-1 )M N Formula (9)

[0136] Where, M1, M2…M N respectively represent the color correction matrix of one color temperature among the N color temperatures (the initial color correction matrix of one color temperature), and a1, a2…1 - a1 - a2 - … - a N-1respectively represent the fusion weight coefficients of one color temperature among N color temperatures.

[0137] In one embodiment, based on the color correction matrix corresponding to at least one color temperature, color correction is performed on the image to be corrected to determine the corrected image.

[0138] For example, through matrix operations, calculate the product between the color correction matrix M corresponding to at least one color temperature and the matrix of the image to be corrected pixel by pixel to obtain the matrix of the corrected image. The formula (10) for calculating the corrected image is shown as follows:

[0139]

[0140] wherein, represents the matrix of the corrected image, represents the matrix of the image to be corrected.

[0141] Applying the embodiments of the present application has at least the following beneficial effects:

[0142] It is possible to calculate the fusion weight of each color temperature among at least two color temperatures based on the color temperature information and gain information, thereby weighting the initial color correction matrix of each color temperature among at least two color temperatures to obtain a fused color correction matrix, that is, the color correction matrix corresponding to at least two color temperatures. Applying the color correction matrix corresponding to at least two color temperatures to the color correction of the image to be corrected effectively solves the color cast problem in the mixed color temperature scenario, reduces the gap between the color of the corrected image and the true color, and improves the efficiency of color correction.

[0143] To better understand the method provided by the embodiments of the present application, the following further illustrates the solutions of the embodiments of the present application in combination with examples of specific application scenarios.

[0144] In an embodiment of a specific application scenario, for example, in the color correction scenario of an image, referring to Figure 2 , which shows the processing flow of another correction method. As Figure 2 shown, the processing flow of the correction method provided by the embodiments of the present application includes the following steps:

[0145] S301, the camera captures the original image.

[0146] S302, the camera preprocesses the original image to obtain the image to be corrected. The preprocessing includes black level correction.

[0147] S303, the camera determines the color temperature information of the color temperature scenario and the gain information of the color temperature scenario in the image to be corrected.

[0148] Specifically, if the color temperature scene in the image to be corrected is a single color temperature scene, the image to be corrected is processed through a white balance algorithm to obtain a first color temperature estimate of a single color temperature in the single color temperature scene and a set of white balance gains for the single color temperature; wherein, the color temperature information of the color temperature scene includes the first color temperature estimate, and the gain information of the color temperature scene includes a set of white balance gains for the single color temperature.

[0149] If the color temperature scene in the image to be corrected is a mixed color temperature scene, the image to be corrected is processed through a white balance algorithm to obtain a second color temperature estimate of at least two color temperatures in the mixed color temperature scene, a set of white balance gains for each of the at least two color temperatures, and a set of mixed color temperature white balance gains for the at least two color temperatures; wherein, the color temperature information of the color temperature scene includes the second color temperature estimate of the at least two color temperatures, and the gain information of the color temperature scene includes a set of white balance gains for each color temperature and a set of mixed color temperature white balance gains for the at least two color temperatures.

[0150] S304. The camera determines a color correction matrix corresponding to at least one color temperature in the color temperature scene based on a plurality of preset reference color temperature values, color correction matrices respectively corresponding to the plurality of reference color temperature values, and at least one of the color temperature information and the gain information.

[0151] Specifically, when n = 1, the color temperature scene in the image to be corrected is a single color temperature scene, and there is a unique color correction matrix corresponding to the single color temperature. The color correction matrix M(T X ) obtained through formula (2) is the color correction matrix corresponding to the single color temperature (the color correction matrix corresponding to one color temperature in the color temperature scene).

[0152] When n > 1, the color temperature scene in the image to be corrected is a mixed color temperature scene. According to a set of white balance gains respectively corresponding to n color temperatures (at least two color temperatures in the mixed color temperature scene) and a set of mixed color temperature white balance gains corresponding to the n color temperatures, the angular error between each set of white balance gains and the set of mixed color temperature white balance gains is calculated to obtain a fusion weight coefficient for each of the n color temperatures (the fusion weight for each of the at least two color temperatures). According to the fusion weight coefficient of each color temperature, the color correction matrix of each color temperature (the initial color correction matrix of each color temperature) is weighted and calculated to obtain a fused color correction matrix (the color correction matrix corresponding to at least two color temperatures in the color temperature scene).

[0153] S305. The camera performs color correction on the image to be corrected based on the color correction matrix corresponding to at least one color temperature to determine the corrected image.

[0154] Applying the embodiments of the present application has at least the following beneficial effects:

[0155] Effectively solve the color cast problem in color temperature scenarios (such as single color temperature scenarios and mixed color temperature scenarios), reduce the gap between the color of the corrected image and the true color, and improve the efficiency of color correction.

[0156] Based on the same inventive concept, the embodiment of the present application also provides a correction device, and the structural schematic diagram of the correction device is as Figure 3 shown. The correction device 40 includes a first processing module 401, a second processing module 402, a third processing module 403, and a fourth processing module 404.

[0157] The first processing module 401 is used to obtain the image to be corrected;

[0158] The second processing module 402 is used to determine the color temperature information of the color temperature scenario and the gain information of the color temperature scenario in the image to be corrected;

[0159] The third processing module 403 is used to determine the color correction matrices corresponding to at least two color temperatures in the color temperature scenario based on the color temperature information and the gain information;

[0160] The fourth processing module 404 is used to perform color correction on the image to be corrected based on the color correction matrices corresponding to at least two color temperatures, and determine the corrected image.

[0161] In one embodiment, the first processing module 401 is specifically used for:

[0162] Obtain the original image;

[0163] Perform preprocessing on the original image to obtain the image to be corrected, and the preprocessing includes black level correction.

[0164] In one embodiment, the second processing module 402 is specifically used for:

[0165] If the color temperature scenario in the image to be corrected is a mixed color temperature scenario, the image to be corrected is processed through a white balance algorithm to obtain the second color temperature estimates of at least two color temperatures in the mixed color temperature scenario, a set of white balance gains for each of the at least two color temperatures, and a set of mixed color temperature white balance gains for the at least two color temperatures;

[0166] Among them, the color temperature information of the color temperature scenario includes the second color temperature estimates of at least two color temperatures, and the gain information of the color temperature scenario includes a set of white balance gains for each color temperature and a set of mixed color temperature white balance gains for at least two color temperatures.

[0167] In one embodiment, the third processing module 403 is specifically used for:

[0168] Determine the color correction matrices corresponding to at least two color temperatures in the color temperature scene based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information.

[0169] In one embodiment, the third processing module 403 is specifically configured to:

[0170] If the color temperature scene in the image to be corrected is a mixed color temperature scene, determine the angular error between each group of white balance gains and a group of mixed color temperature white balance gains corresponding to at least two color temperatures in the mixed color temperature scene based on a group of white balance gains corresponding to at least two color temperatures respectively in the mixed color temperature scene and a group of mixed color temperature white balance gains of at least two color temperatures;

[0171] Based on each angular error, determine the fusion weight of each color temperature in at least two color temperatures;

[0172] Determine the initial color correction matrix of each color temperature based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and the second color temperature estimation values of at least two color temperatures;

[0173] Perform a fusion process based on the fusion weight of each color temperature and the initial color correction matrix of each color temperature to determine the color correction matrices corresponding to at least two color temperatures;

[0174] Wherein, the color temperature information includes the second color temperature estimation values of at least two color temperatures, and the gain information includes a group of white balance gains corresponding to at least two color temperatures respectively and a group of mixed color temperature white balance gains of at least two color temperatures.

[0175] In one embodiment, the correction device may be an ISP (Image Signal Processor).

[0176] Applying the embodiments of the present application has at least the following beneficial effects:

[0177] Obtain the image to be corrected; determine the color temperature information of the color temperature scene in the image to be corrected and the gain information of the color temperature scene; determine the color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information; perform color correction on the image to be corrected based on the color correction matrices corresponding to at least two color temperatures to determine the corrected image. In this way, the problem of color cast in the color temperature scene is effectively solved, the gap between the color of the corrected image and the real color is reduced, and the efficiency of color correction is improved.

[0178] The embodiment of the present application further provides an imaging device, which includes an image acquisition device and the calibration device provided by the present application. The image acquisition device is used to acquire an original image. The image acquisition device can be any type of image acquisition device, such as a CMOS image sensor, a CCD image sensor, etc. The embodiment of the present application does not specifically limit the type of the image acquisition device.

[0179] In some embodiments, the imaging device can be a smart phone, a camera, a video camera, a tablet computer, a laptop computer, a monitoring device, etc.

[0180] The embodiment of the present application further provides an electronic device. The schematic structural diagram of the electronic device is as Figure 4 shown, Figure 4 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as connected through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004. The transceiver 4004 can be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiment of the present application.

[0181] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of the present application. The processor 4001 can also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0182] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 can be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard structure) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation,Figure 4 It is represented only by a thick line, but it does not mean that there is only one bus or one type of bus.

[0183] The memory 4003 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.

[0184] The memory 4003 is used to store the computer program for implementing the embodiments of the present application and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0185] Among them, the electronic device includes but is not limited to: imaging devices, etc.

[0186] Applying the embodiments of the present application has at least the following beneficial effects:

[0187] Obtain the image to be corrected; determine the color temperature information of the color temperature scene and the gain information of the color temperature scene in the image to be corrected; based on the color temperature information and the gain information, determine the color correction matrix corresponding to at least two color temperatures in the color temperature scene; based on the color correction matrix corresponding to at least two color temperatures, perform color correction on the image to be corrected to determine the corrected image. In this way, the color cast problem in the color temperature scene is effectively solved, the gap between the color of the corrected image and the real color is reduced, and the efficiency of color correction is improved.

[0188] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0189] The embodiments of the present application also provide a computer program product, including a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0190] Based on the same principle as the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any optional embodiment of the present application described above.

[0191] It should be understood that although the flowchart of the embodiments of the present application indicates each operation step by an arrow, the execution order of these steps is not limited to the order indicated by the arrow. Unless otherwise clearly stated in this document, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart may be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage among these sub-steps or stages may also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.

[0192] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art in the technical field of the present application, without departing from the technical concept of the solution of the present application, using other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. A calibration device, characterized in that, Comprising: A first processing module, configured to obtain an image to be corrected; A second processing module, configured to determine the color temperature information of the color temperature scene in the image to be corrected and the gain information of the color temperature scene; A third processing module, configured to determine color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information; A fourth processing module, configured to perform color correction on the image to be corrected based on the color correction matrices corresponding to the at least two color temperatures, and determine the corrected image; The third processing module is specifically configured to: Determine color correction matrices corresponding to at least two color temperatures in the color temperature scene based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information; The third processing module is specifically configured to: If the color temperature scene in the image to be corrected is a mixed color temperature scene, determine the angular error between each group of white balance gains and a group of mixed color temperature white balance gains of the at least two color temperatures based on a group of white balance gains respectively corresponding to at least two color temperatures in the mixed color temperature scene and the group of mixed color temperature white balance gains of the at least two color temperatures; Determine the fusion weight of each color temperature in the at least two color temperatures based on each angular error; Determine an initial color correction matrix for each color temperature based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and the second color temperature estimate of the at least two color temperatures; Perform a fusion process based on the fusion weight of each color temperature and the initial color correction matrix of each color temperature, and determine the color correction matrices corresponding to the at least two color temperatures; Wherein, the color temperature information includes the second color temperature estimate of the at least two color temperatures, and the gain information includes a group of white balance gains respectively corresponding to the at least two color temperatures and a group of mixed color temperature white balance gains of the at least two color temperatures.

2. The device according to claim 1, characterized in that, The first processing module is specifically configured to: Obtain an original image; Perform preprocessing on the original image to obtain an image to be corrected, and the preprocessing includes black level correction.

3. The device according to claim 1, characterized in that, The second processing module is specifically configured to: If the color temperature scene in the image to be corrected is a mixed color temperature scene, process the image to be corrected through a white balance algorithm to obtain the second color temperature estimate of at least two color temperatures in the mixed color temperature scene, a group of white balance gains for each color temperature in the at least two color temperatures, and a group of mixed color temperature white balance gains of the at least two color temperatures; Wherein, the color temperature information of the color temperature scene includes the second color temperature estimate of the at least two color temperatures, and the gain information of the color temperature scene includes a group of white balance gains for each color temperature and a group of mixed color temperature white balance gains of the at least two color temperatures.

4. A calibration method, characterized in that, Comprising: Obtain an image to be corrected; Determine the color temperature information of the color temperature scene in the image to be corrected and the gain information of the color temperature scene; Determine color correction matrices corresponding to at least two color temperatures in the color temperature scene based on the color temperature information and the gain information; Perform color correction on the image to be corrected based on the color correction matrices corresponding to the at least two color temperatures, and determine the corrected image; Determining a color correction matrix corresponding to at least two color temperatures in the color temperature scenario based on the color temperature information and the gain information includes: Determining a color correction matrix corresponding to at least two color temperatures in the color temperature scenario based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information; The determining a color correction matrix corresponding to at least two color temperatures in the color temperature scenario based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, the color temperature information, and the gain information includes: If the color temperature scenario in the image to be corrected is a mixed color temperature scenario, determining an angular error between each group of white balance gains and a set of mixed color temperature white balance gains corresponding to at least two color temperatures in the mixed color temperature scenario based on a set of white balance gains respectively corresponding to at least two color temperatures in the mixed color temperature scenario and the set of mixed color temperature white balance gains of the at least two color temperatures; Determining a fusion weight for each color temperature in the at least two color temperatures based on each angular error; Determining an initial color correction matrix for each color temperature based on a plurality of preset reference color temperature values, the color correction matrices respectively corresponding to the plurality of reference color temperature values, and a second color temperature estimate of the at least two color temperatures; Performing a fusion process based on the fusion weight of each color temperature and the initial color correction matrix of each color temperature to determine a color correction matrix corresponding to the at least two color temperatures; Wherein, the color temperature information includes a second color temperature estimate of the at least two color temperatures, and the gain information includes a set of white balance gains respectively corresponding to the at least two color temperatures and a set of mixed color temperature white balance gains of the at least two color temperatures.

5. An imaging device, characterized in that, Including an image acquisition device and the correction device according to any one of claims 1-3, wherein the image acquisition device is configured to acquire an original image.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to claim 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the steps of the method according to claim 4.

Images