Color restoration method and device for multi-color-temperature mixed light source scene, equipment and medium

Abstract

The invention discloses a color restoration method and device for a multi-color-temperature mixed light source scene, equipment and a medium, and the method comprises the steps: dividing a to-be-restored image into a plurality of image blocks, and calculating the white balance gain offset value of each image block; all the white balance gain offset values are corrected based on the scene change coefficient, and each target white balance gain offset value is obtained based on the proportionality coefficient and the corrected white balance gain offset values; calculating target CCM saturation based on the proportionality coefficient and the color temperature value of the white balance block in the image block, and determining a target CCM based on the target CCM saturation and the calibration CCM; correcting the corresponding image block based on each target white balance gain offset value; and performing color restoration on the corrected image block by using the target CCM. According to the method, the local white balance gain and the dynamic CCM are used for performing color rendition on the image blocks, color temperature areas do not need to be distinguished, meanwhile, the real-time performance and the effect of color rendition are guaranteed, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of image processing technology, and in particular to a method, device, equipment, and medium for color restoration of a multi-color temperature mixed light source scene. Background Art

[0002] In mixed light source scenarios, the traditional method is to use AWB and CCM to perform unified color restoration on the image. This method cannot take into account the color restoration of different color temperature areas, which will inevitably cause excessive color errors in some areas and affect the color restoration effect.

[0003] Based on this, existing technologies use clustering and other algorithms to divide different color temperature areas, assigning different gains to pixels in different color temperature areas, with a transition in gain values ​​between areas. However, this increases the complexity of color restoration. As a result, for high-frame-rate video applications, complex color temperature area calculations cannot guarantee real-time performance, resulting in the color restoration processing speed not keeping up with the video frame rate, affecting the continuity of the video image and reducing the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and medium for color restoration in a multi-color temperature mixed light source scene, which uses local white balance gain and dynamic CCM to perform color restoration on the segmented image, eliminating the need for high-precision color temperature area segmentation processing. While ensuring the real-time performance of color restoration, it achieves better color restoration effects in a multi-color temperature mixed light source scene, thereby improving the user experience.

[0005] In a first aspect, an embodiment of the present application provides a method for color restoration of a multi-color temperature mixed light source scene, comprising:

[0006] Divide the image to be restored into m*n image blocks, and calculate the white balance gain offset value of each image block, where m and n are both positive integers;

[0007] Obtaining a scene change coefficient and a scaling coefficient, correcting all the white balance gain offset values ​​based on the scene change coefficient to obtain a corrected intermediate white balance gain offset value, and obtaining a target white balance gain offset value corresponding to each image block based on the scaling coefficient and each intermediate white balance gain offset value;

[0008] calculating a target CCM saturation based on the scaling factor and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM, wherein the first block is the image block belonging to a white balance area;

[0009] performing local white balance correction on the corresponding image block based on each target white balance gain offset value;

[0010] The target CCM is used to perform global color restoration on the image block after local white balance correction.

[0011] In some embodiments, calculating the white balance gain offset value of each image block includes:

[0012] Dividing all the image blocks into the first blocks and the second blocks based on a white balance algorithm, wherein the second blocks are the image blocks that do not belong to the white balance area;

[0013] Calculating a weighted average white balance gain based on the color temperature values ​​and the white balance gains of all the first blocks;

[0014] Determine a result obtained by dividing each of the white balance gains by the weighted average white balance gain as the white balance gain offset value of each of the first blocks;

[0015] For any second block, an average value of the white balance gain offset values ​​of a plurality of first blocks adjacent to the second block is calculated, and the average value is determined as the white balance gain offset value of the second block.

[0016] In some embodiments, calculating a weighted average white balance gain based on the color temperature values ​​and the white balance gains of all the first blocks includes:

[0017] Determining a color temperature weight of each of the first blocks based on each of the color temperature values;

[0018] An average value of the product of the white balance gain of each first block and the corresponding color temperature weight is determined as the weighted average white balance gain.

[0019] In some embodiments, all of the white balance gain offset values ​​are corrected based on the scene change coefficient to obtain a corrected intermediate white balance gain offset value, including:

[0020] Obtaining resting threshold and movement threshold;

[0021] All the white balance gain offset values ​​are corrected based on a comparison relationship between the scene change coefficient and the still threshold and the motion threshold to obtain the intermediate white balance gain offset value.

[0022] In some embodiments, all the white balance gain offset values ​​are corrected based on the comparison relationship between the scene change coefficient and the still threshold and the motion threshold to obtain the intermediate white balance gain offset value, which is obtained according to the following formula:

[0023]

[0024] Among them, R'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on the R channel, G'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on the G channel, B'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on channel B, R_gain_off i,j G_gain_off is the white balance gain offset value of the image block in row i and column j on the R channel before correction. i,j B_gain_off is the white balance gain offset value of the image block in row i and column j on the G channel before correction. i,j is the white balance gain offset value of the image block at the i-th row and j-th column on the B channel before correction, mov is the scene change coefficient, Thm is the still threshold, and Ths is the motion threshold.

[0025] In some embodiments, calculating a target CCM saturation based on the scale factor and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM includes:

[0026] Calculating the weighted average color temperature value based on the color temperature value and the color temperature weight of each first block;

[0027] Calculating a color temperature standard deviation based on the weighted average color temperature value and each of the color temperature values;

[0028] Calculating an initial CCM saturation based on the color temperature standard deviation, and calculating the target CCM saturation based on the initial CCM saturation and the proportional coefficient;

[0029] The target CCM is calculated based on the calibrated CCM using a linear interpolation algorithm, wherein the color temperature value corresponding to the target CCM is the weighted average color temperature value, and the saturation corresponding to the target CCM is the target CCM saturation.

[0030] In some embodiments, performing local white balance correction on the corresponding image block based on each target white balance gain offset value includes:

[0031] Determining respective reference offset values ​​corresponding to respective target white balance gain offset values, wherein the reference offset values ​​are the target white balance gain offset values ​​of respective image blocks corresponding to a previous frame of the current image to be restored;

[0032] Smoothing each of the target white balance gain offset values ​​based on a preset smoothing algorithm and each of the reference offset values ​​to obtain each new target white balance gain offset value;

[0033] Local white balance correction is performed on the corresponding image block based on each of the new target white balance gain offset values.

[0034] In a second aspect, an embodiment of the present application provides a control device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the color restoration method for a multi-color temperature mixed light source scene as described in the first aspect.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device comprising the control device of the second aspect.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the color restoration method for a multi-color temperature mixed light source scene as described in the first aspect.

[0037] Embodiments of the present application provide a color restoration method, apparatus, device, and medium for a multi-color temperature mixed light source scene, the method comprising: dividing an image to be restored into m*n image blocks, and calculating a white balance gain offset value for each of the image blocks, where m and n are both positive integers; obtaining a scene change coefficient and a proportional coefficient, correcting all of the white balance gain offset values ​​based on the scene change coefficient to obtain a corrected intermediate white balance gain offset value, and obtaining a target white balance gain offset value corresponding to each of the image blocks based on the proportional coefficient and each of the intermediate white balance gain offset values; calculating a target CCM saturation based on the proportional coefficient and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM, where the first block is the image block belonging to a white balance area; performing local white balance correction on the corresponding image block based on each target white balance gain offset value; and performing global color restoration on the image block after the local white balance correction using the target CCM. According to the solution provided in the embodiment of the present application, local white balance gain and dynamic CCM are used to perform color restoration on the blocked image, without the need for high-precision color temperature area blocking processing. While ensuring the real-time color restoration, better color restoration effects are achieved in multi-color temperature mixed light source scenarios, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1This is a flowchart of the steps of a color restoration method for a multi-color temperature mixed light source scene provided by an embodiment of the present application;

[0039] Figure 2 This is a flowchart of the steps of a color restoration method for a multi-color temperature mixed light source scene provided by another embodiment of the present application;

[0040] Figure 3 This is a structural diagram of a control device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] It is understood that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0043] In mixed light source scenarios, the traditional method is to use AWB and CCM to perform unified color restoration on the image. This method cannot take into account the color restoration of different color temperature areas, which will inevitably cause excessive color errors in some areas and affect the color restoration effect.

[0044] Based on this, existing technologies use clustering and other algorithms to divide different color temperature areas, assigning different gains to pixels in different color temperature areas, with a transition in gain values ​​between areas. However, this increases the complexity of color restoration. As a result, for high-frame-rate video applications, complex color temperature area calculations cannot guarantee real-time performance, resulting in the color restoration processing speed not keeping up with the video frame rate, affecting the continuity of the video image and reducing the user experience.

[0045] To address the above-mentioned problems, embodiments of the present application provide a method, apparatus, device, and medium for color restoration of a multi-color temperature mixed light source scene. The method includes: dividing an image to be restored into m*n image blocks, and calculating a white balance gain offset value for each of the image blocks, where m and n are both positive integers; obtaining a scene variation coefficient and a scaling coefficient, correcting all of the white balance gain offset values ​​based on the scene variation coefficient to obtain a corrected intermediate white balance gain offset value, and obtaining a target white balance gain offset value corresponding to each of the image blocks based on the scaling coefficient and each of the intermediate white balance gain offset values; calculating a target CCM saturation based on the scaling coefficient and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM, where the first block is the image block belonging to the white balance area; performing local white balance correction on the corresponding image block based on each target white balance gain offset value; and performing global color restoration on the image block after the local white balance correction using the target CCM. According to the solution provided in the embodiment of the present application, local white balance gain and dynamic CCM are used to perform color restoration on the blocked image, without the need for high-precision color temperature area blocking processing. While ensuring the real-time color restoration, better color restoration effects are achieved in multi-color temperature mixed light source scenarios, thereby improving the user experience.

[0046] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0047] refer to Figure 1 , Figure 1 This is a flowchart of the steps of a color restoration method for a multi-color temperature mixed light source scene provided by an embodiment of the present application. The embodiment of the present application provides a color restoration method for a multi-color temperature mixed light source scene, which includes but is not limited to the following steps:

[0048] Step S10 : Divide the image to be restored into m*n image blocks, and calculate the white balance gain offset value of each image block, where m and n are both positive integers.

[0049] Specifically, the execution subject of the color restoration method of the multi-color temperature mixed light source scene of the present application can be an electronic device such as a security camera, a tablet, and a driving recorder.

[0050] It should be noted that if Figure 2 As shown, before triggering the operation of dividing the image to be restored for subsequent color restoration, the present application first determines whether it is a daytime outdoor scene through the illumination information of automatic exposure. Since the daytime outdoor scene is a common monochromatic temperature light source scene, there is no need to apply the present application scheme. The color restoration method of the present application is aimed at images under multi-color temperature mixed light source scenes.

[0051] Specifically, the specific number of image blocks into which the image to be restored is divided in this application depends on the hardware computing power of the electronic device that executes the color restoration method. The greater the hardware computing power, the larger the corresponding m and n values, so that the divided image blocks are closer to the horizontal and vertical pixel numbers of the image, and the subsequent adaptive local white balance boundaries are more accurate. In addition, in this embodiment, the hardware computing power can also be adjusted according to the number of blocks, so it can also be used in application scenarios with tight computing power.

[0052] It is understandable that calculating the white balance gain offset value of each image block can provide an effective data basis for subsequent white balance correction of each image block, correcting the color cast of the image block, and improving the accuracy of color restoration.

[0053] Specifically, in some embodiments, Figure 1 The calculation of the white balance gain offset value of each image block in step S10 includes but is not limited to the following steps:

[0054] Step S11, dividing all image blocks into first blocks and second blocks based on a white balance algorithm, wherein the second blocks are image blocks that do not belong to the white balance area;

[0055] Step S12, calculating a weighted average white balance gain based on the color temperature values ​​and white balance gains of all first blocks;

[0056] Step S13, dividing each white balance gain by the weighted average white balance gain to obtain a result as the white balance gain offset value of each first block;

[0057] Step S14 : For any second block, calculate the average value of the white balance gain offset values ​​of a plurality of first blocks adjacent to the second block, and determine the average value as the white balance gain offset value of the second block.

[0058] Specifically, in this embodiment, each first block white balance gain corresponds to the gain (R_gain, G_gain, B_gain) acting on the three channels R, G, and B on the pixel.

[0059] Understandably, the reference Figure 2In the present application, all image blocks are first divided into two categories, namely, first blocks and second blocks, using a preset white balance algorithm. The first blocks are image blocks belonging to the white balance area, and the second blocks are image blocks not belonging to the white balance area. Each first block corresponds to a color temperature value and a white balance gain. Then, the color temperature values ​​and white balance gains of all the first blocks are counted. Based on the color temperature values ​​and white balance gains of all the first blocks, a weighted average white balance gain, a weighted average color temperature value, and a color temperature standard deviation are calculated. The weighted average color temperature value and color temperature standard deviation here are used to calculate the subsequent CCM. The result obtained by dividing each white balance gain by the weighted average white balance gain is determined as the white balance gain offset value of each first block. For any second block, the average of the white balance gain offset values ​​of multiple first blocks adjacent to the second block is calculated, and the average value is determined as the white balance gain offset value of the second block. In this way, the white balance gain offset values ​​of all image blocks are calculated. Since the white balance area in this embodiment is a neutral color area, and the neutral color area will exhibit color deviations directly related to the color temperature of the light source under different light sources, thereby providing reliable information for estimating the color temperature of the light source, based on this, this embodiment calculates the white balance gain offset value of all image blocks based on the color temperature value and white balance gain of the first block in the white balance area, and uses this white balance gain offset value as the basic white balance gain offset value, providing an effective data basis for subsequent correction of the basic white balance gain offset value to obtain a more accurate offset value, thereby providing effective support for the color reproduction effect.

[0060] In addition, in this embodiment, the white balance gain offset value of the second block can also be calculated using a surface interpolation method.

[0061] Specifically, in some embodiments, step S12 includes but is not limited to the following steps:

[0062] Step S121, determining the color temperature weight of each first block based on each color temperature value;

[0063] Step S122 : determining an average value of the product of the white balance gain of each first block and the corresponding color temperature weight as a weighted average white balance gain.

[0064] It can be understood that after determining the color temperature value and white balance gain of each first block, the color temperature weight of each first block is determined based on the each color temperature value, and the average value of the product of the white balance gain of each first block and the corresponding color temperature weight is determined as the weighted average white balance gain. In this way, the color temperature information of different first blocks can be integrated to generate the globally optimal white balance correction parameter (i.e., the weighted average white balance gain), providing an effective data basis for obtaining a more accurate white balance gain offset value, and achieving more robust and natural color reproduction under complex lighting conditions.

[0065] Step S20 , obtaining a scene change coefficient and a scaling coefficient, correcting all white balance gain offset values ​​based on the scene change coefficient to obtain corrected intermediate white balance gain offset values, and obtaining a target white balance gain offset value corresponding to each image block based on the scaling coefficient and each intermediate white balance gain offset value.

[0066] Specifically, the scene change coefficient of this embodiment is obtained by hardware devices such as a gyroscope or by calculation based on the difference between two previous and next frames of images.

[0067] Specifically, in this embodiment, the target white balance gain offset value corresponding to each image block is obtained based on the proportional coefficient and each intermediate white balance gain offset value, which is obtained according to the following formula:

[0068] R”_gain_off i,j =(R'_gain_off i,j -1)×coeff+1,0 <coeff<1;

[0069] G”_gain_off i,j =(G'_gain_off i,j -1)×coeff+1,0 <coeff<1;

[0070] B”_gain_off i,j =(B'_gain_off i,j -1)×coeff+1,0 <coeff<1;

[0071] Where R'_gain_off i,j The target white balance gain offset value of the image block in row i and column j on the R channel, G'_gain_off i,j The target white balance gain offset value of the image block in the i-th row and j-th column on the G channel, B'_gain_off i,j The target white balance gain offset value of the image block in row i and column j on the B channel, R'_gain_off i,j G'_gain_off is the intermediate white balance gain offset value of the image block in row i and column j on the R channel. i,j B'_gain_off is the intermediate white balance gain offset value of the image block in the i-th row and j-th column on the G channel. i,j is the middle white balance gain offset value of the image block in the i-th row and j-th column on the B channel, coeff is the proportional coefficient, and the value of the proportional coefficient can be determined according to actual needs.

[0072] It's understandable that in actual color restoration applications, lighting conditions and shooting scenes can change rapidly, such as switching from warm indoor lighting to cool outdoor lighting, jitter or rapid panning during handheld shooting, or mixed light sources like neon lights, natural light, and artificial light. Using static white balance gain offset values ​​for image blocks is inadequate for dynamic scenes, resulting in poor color restoration. Based on this, this embodiment uses scene variation coefficients to correct all white balance gain offset values, effectively addressing the limitations of static white balance algorithms in dynamic scenes and ensuring the real-time and adaptable nature of color restoration. Furthermore, this embodiment uses the proportional coefficient and the intermediate white balance gain offset values ​​after correction by the scene variation coefficient to derive target white balance gain offset values ​​corresponding to each image block. This means that the intermediate white balance gain offset values ​​are corrected using the proportional coefficient. This is done to take into account that performing white balance correction may affect the subsequent CCM correction effect. As the scene changes, the scene variation coefficient will dynamically change. Referring to the above formula, by multiplying the intermediate white balance gain offset values ​​by the proportional coefficient, the white balance gain offset values ​​can be dynamically corrected while indirectly adjusting the CCM correction effect. This allows the CCM to better adapt to scene changes and achieve dynamic CCM color reproduction.

[0073] In some embodiments, the step S20 of correcting all white balance gain offset values ​​based on the scene variation coefficient to obtain a corrected intermediate white balance gain offset value includes but is not limited to the following steps:

[0074] Step S21, obtaining a stillness threshold and a motion threshold;

[0075] Step S22 , correcting all white balance gain offset values ​​based on the comparison relationship between the scene change coefficient and the static threshold and the motion threshold to obtain an intermediate white balance gain offset value.

[0076] Specifically, the scene change coefficient quantifies the degree of dynamic change of the scene. Combined with the stillness threshold and the motion threshold, it can determine whether the current scene is in a still, moving or transitional state. Different states correspond to different correction strategies for the white balance gain offset value.

[0077] It should be noted that, in this embodiment, all white balance gain offset values ​​are corrected based on the comparison relationship between the scene change coefficient and the stationary threshold and the motion threshold to obtain the intermediate white balance gain offset value, which is obtained according to the following formula:

[0078]

[0079] Among them, R'_gain_off i,j$R'\_gain\_off$ is the intermediate white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $R$ channel, and $G'\_gain\_off$ i,j $G'\_gain\_off$ is the intermediate white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $G$ channel, and $B'\_gain\_off$ i,j $B'\_gain\_off$ is the intermediate white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $B$ channel, and $R\_gain\_off$ i,j $R\_gain\_off$ is the white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $R$ channel before correction, and $G\_gain\_off$ i,j $G\_gain\_off$ is the white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $G$ channel before correction, and $B\_gain\_off$ i,j $B\_gain\_off$ is the white balance gain offset value of the image block at the $i$-th row and $j$-th column on the $B$ channel before correction. $mov$ is the scene change coefficient, $Ths$ is the static threshold, and $Thm$ is the motion threshold.

[0080] It can be understood that from the above formula for correcting the white balance gain offset value, when $mov\geq Thm$, it indicates that the current shooting scene is in a motion state, and the amplitude of adjusting the white balance gain offset value needs to be increased. When $Ths\lt mov\lt Thm$, it indicates that the current shooting scene is in a slow change state, and it is more appropriate to use progressive correction for the white balance gain offset value. When $mov\leq Ths$, it indicates that the current shooting scene is in a static state, and the white balance gain offset value is not corrected to maintain the stability of the white balance gain and avoid unnecessary fluctuations.

[0081] Step S30: Calculate the target CCM saturation based on the proportionality coefficient and the color temperature value corresponding to the first sub-block in the image block, and determine the target CCM based on the target CCM saturation and the calibrated CCM, where the first sub-block is the image block belonging to the white balance area.

[0082] In some embodiments, Figure 1 Calculating the target CCM saturation based on the proportionality coefficient and the color temperature value corresponding to the first sub-block in the image block in step S30 includes, but is not limited to, the following steps:

[0083] Step S31: Calculate the weighted average color temperature value based on the color temperature value and the color temperature weight of each first sub-block;

[0084] Step S32: Calculate the color temperature standard deviation based on the weighted average color temperature value and each color temperature value;

[0085] Step S33: Calculate the initial CCM saturation based on the color temperature standard deviation, and calculate the target CCM saturation based on the initial CCM saturation and the proportionality coefficient;

[0086] Step S34 : Calculate the target CCM based on the calibration CCM using a linear interpolation algorithm, wherein the color temperature value corresponding to the target CCM is the weighted average color temperature value, and the saturation corresponding to the target CCM is the target CCM saturation.

[0087] Specifically, in this embodiment, the method of allocating color temperature weights may be determined based on factors such as the importance of the region where the image block is located or the neutral color confidence.

[0088] Understandably, the reference Figure 2 This embodiment calculates a weighted average color temperature value based on the color temperature values ​​of the first block in combination with the color temperature weights. The color temperature standard deviation is calculated based on the weighted average color temperature value and each color temperature value. The color temperature standard deviation is used to indicate the degree of dispersion of the color temperature distribution. The initial CCM saturation is calculated based on the color temperature standard deviation. The target CCM saturation is calculated based on the initial CCM saturation and the proportional coefficient, thereby providing an effective data basis for determining the target CCM.

[0089] It should be noted that in this embodiment, the target CCM saturation is calculated based on the initial CCM saturation and the proportional coefficient, and is obtained according to the following formula: target CCM saturation = (initial CCM saturation - 100%) × (1-coeff) + 100%. Thus,

[0090] It can be understood that after determining the initial CCM saturation, this embodiment multiplies the initial CCM saturation by the proportional coefficient, which can control the sensitivity of saturation changes and prevent overcorrection, thereby providing effective support for obtaining a target CCM that adapts to dynamic changes in complex lighting environments.

[0091] In addition, this embodiment may also calculate the color temperature extreme difference value or the color temperature average difference value instead of the color temperature standard deviation.

[0092] In addition, this embodiment can also dynamically reduce the saturation by combining the initial saturation through the ISP module, replacing the color restoration effect of the dynamic CCM in this embodiment.

[0093] Step S40 : performing local white balance correction on the corresponding image block based on each target white balance gain offset value.

[0094] In some embodiments, Figure 1 Step S40 includes but is not limited to the following steps:

[0095] Step S41, determining respective reference offset values ​​corresponding to respective target white balance gain offset values, wherein the reference offset values ​​are target white balance gain offset values ​​of respective image blocks corresponding to a previous frame of the image to be restored;

[0096] Step S42, smoothing each target white balance gain offset value based on a preset smoothing algorithm and each reference offset value to obtain each new target white balance gain offset value;

[0097] Step S43 : performing local white balance correction on the corresponding image block based on each new target white balance gain offset value.

[0098] Understandably, the reference Figure 2 In this embodiment, the target white balance gain offset value, which has been adjusted by the scaling coefficient, is smoothed in the temporal dimension in combination with the target white balance gain offset value of the image block corresponding to the previous frame of the image to be restored, to obtain new target white balance gain offset values. This is primarily intended to address visual discomfort caused by sudden changes in white balance between frames in video or continuous shooting, strike a balance between color accuracy and visual comfort, better ensure color restoration effects, and enhance user experience.

[0099] Step S50 , performing global color restoration on the image block after local white balance correction using the target CCM.

[0100] It is understood that, with reference to the description of the above embodiments, and Figure 2 , after determining that the current shooting environment is a multi-color temperature mixed light source scene, the image to be restored is divided into m*n image blocks, and it is determined whether each image block is located in the white balance area. If so, the image block located in the white balance area is determined as the first block, and the white balance gain and color temperature value of the first block are determined. According to the white balance gain and color temperature of the first block, a weighted average white balance gain and a white balance gain offset value of the first block are weighted and calculated; for an image block that is not located in the white balance area, it is determined as the second block, and the average of the white balance gain offset values ​​of multiple first blocks adjacent to the second block is determined as the white balance gain offset value corresponding to the second block; in this way, the white balance gain offset value of all image blocks is obtained. value; the white balance gain offset value of each image block is corrected by smoothing according to the scene change coefficient, the proportional coefficient, and the result of the previous frame (target white balance gain offset value) to obtain the final target white balance gain offset value; the target CCM is calculated according to the discrete degree and proportional coefficient of the color temperature distribution corresponding to the first block; after local white balance correction is performed on each corresponding image block using the target white balance gain offset value, color restoration is performed on all image blocks using the target CCM. In this way, there is no need for high-precision color temperature area block processing, which ensures the real-time color restoration while achieving better color restoration effect in multi-color temperature mixed light source scenes, thereby improving user experience.

[0101] like Figure 3 As shown, Figure 3: is a structural diagram of a control device provided in one embodiment of the present application. The present invention also provides a control device 300, comprising:

[0102] The processor 310 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0103] The memory 320 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 320 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 320 and is called by the processor 310 to execute the color restoration method for the multi-color temperature mixed light source scene of the embodiments of this application;

[0104] Input / output interface 330, used to implement information input and output;

[0105] Communication interface 340, used to implement communication interaction between the apparatus and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0106] bus 350 , which transmits information between the various components of the device (e.g., processor 310 , memory 320 , input / output interface 330 , and communication interface 340 );

[0107] The processor 310 , the memory 320 , the input / output interface 330 and the communication interface 340 are connected to each other in communication within the device via the bus 350 .

[0108] In addition, an embodiment of the present application further provides an electronic device, including the control device 300 of the above embodiment.

[0109] In addition, an embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the color restoration method of the multi-color temperature mixed light source scene is implemented.

[0110] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory optionally includes a memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned networks include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0111] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0112] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation. Those skilled in the art can also make various equivalent modifications or substitutions under the shared conditions that do not violate the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A color restoration method for a multi-color temperature mixed light source scene, characterized in that: include: Divide the image to be restored into m*n image blocks, and calculate the white balance gain offset value of each image block, where m and n are both positive integers; Obtaining a scene change coefficient and a scaling coefficient, correcting all the white balance gain offset values ​​based on the scene change coefficient to obtain a corrected intermediate white balance gain offset value, and obtaining a target white balance gain offset value corresponding to each image block based on the scaling coefficient and each intermediate white balance gain offset value; calculating a target CCM saturation based on the scaling factor and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM, wherein the first block is the image block belonging to a white balance area; performing local white balance correction on the corresponding image block based on each target white balance gain offset value; The target CCM is used to perform global color restoration on the image block after local white balance correction.

2. The color restoration method for a multi-color temperature mixed light source scene according to claim 1, characterized in that: Calculating the white balance gain offset value of each image block includes: Dividing all the image blocks into the first blocks and the second blocks based on a white balance algorithm, wherein the second blocks are the image blocks that do not belong to the white balance area; Calculating a weighted average white balance gain based on the color temperature values ​​and the white balance gains of all the first blocks; Determine a result obtained by dividing each of the white balance gains by the weighted average white balance gain as the white balance gain offset value of each of the first blocks; For any second block, an average value of the white balance gain offset values ​​of a plurality of first blocks adjacent to the second block is calculated, and the average value is determined as the white balance gain offset value of the second block.

3. The color restoration method for a multi-color temperature mixed light source scene according to claim 2, characterized in that: Calculating a weighted average white balance gain based on the color temperature values ​​and the white balance gains of all the first blocks includes: Determining a color temperature weight of each of the first blocks based on each of the color temperature values; An average value of the product of the white balance gain of each first block and the corresponding color temperature weight is determined as the weighted average white balance gain.

4. The color restoration method for a multi-color temperature mixed light source scene according to claim 1, characterized in that: Correcting all the white balance gain offset values ​​based on the scene change coefficient to obtain a corrected intermediate white balance gain offset value includes: Obtaining resting threshold and movement threshold; All the white balance gain offset values ​​are corrected based on a comparison relationship between the scene change coefficient and the still threshold and the motion threshold to obtain the intermediate white balance gain offset value.

5. The color restoration method for a multi-color temperature mixed light source scene according to claim 4, characterized in that: Based on the comparison relationship between the scene change coefficient and the static threshold and the motion threshold, all the white balance gain offset values ​​are corrected to obtain the intermediate white balance gain offset value, which is obtained according to the following formula: Among them, R'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on the R channel, G'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on the G channel, B'_gain_off i,j is the intermediate white balance gain offset value of the image block in row i and column j on channel B, R_gain_off i,j G_gain_off is the white balance gain offset value of the image block in row i and column j on the R channel before correction. i,j B_gain_off is the white balance gain offset value of the image block in row i and column j on the G channel before correction. i,j is the white balance gain offset value of the image block at the i-th row and j-th column on the B channel before correction, mov is the scene change coefficient, Thm is the still threshold, and Ths is the motion threshold.

6. The color restoration method for a multi-color temperature mixed light source scene according to claim 3, characterized in that: Calculating a target CCM saturation based on the scale factor and a color temperature value corresponding to a first block in the image block, and determining a target CCM based on the target CCM saturation and a calibrated CCM, including: Calculating the weighted average color temperature value based on the color temperature value and the color temperature weight of each first block; Calculating a color temperature standard deviation based on the weighted average color temperature value and each of the color temperature values; Calculating an initial CCM saturation based on the color temperature standard deviation, and calculating the target CCM saturation based on the initial CCM saturation and the proportional coefficient; The target CCM is calculated based on the calibrated CCM using a linear interpolation algorithm, wherein the color temperature value corresponding to the target CCM is the weighted average color temperature value, and the saturation corresponding to the target CCM is the target CCM saturation.

7. The color restoration method for a multi-color temperature mixed light source scene according to claim 1, characterized in that: Performing local white balance correction on the corresponding image block based on each target white balance gain offset value, including: Determining respective reference offset values ​​corresponding to respective target white balance gain offset values, wherein the reference offset values ​​are the target white balance gain offset values ​​of respective image blocks corresponding to a previous frame of the current image to be restored; Smoothing each of the target white balance gain offset values ​​based on a preset smoothing algorithm and each of the reference offset values ​​to obtain each new target white balance gain offset value; Local white balance correction is performed on the corresponding image block based on each of the new target white balance gain offset values.

8. A control device, characterized in that: The method comprises at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the color restoration method for a multi-color temperature mixed light source scene as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: Comprising the control device according to claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the color restoration method for a multi-color temperature mixed light source scene according to any one of claims 1 to 7.

Images