Image color restoration method and device, storage medium and electronic equipment

Abstract

A color restoration method and device of an image, a storage medium and an electronic device are disclosed. The method comprises: obtaining first color statistical information of a region including a predetermined object in a first color restoration correction image; wherein the first color restoration correction image is determined based on a first native image collected by a reference image sensor for the predetermined object; determining an uncorrected image based on a second native image collected by a target image sensor for the predetermined object; determining second color statistical information of a region including the predetermined object in the uncorrected image; adjusting the first color statistical information to obtain multiple adjustment results of the first color statistical information; and determining a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image, the second color statistical information and the multiple adjustment results. The embodiments of the present disclosure can achieve good color restoration effect for the target image sensor.

Description

Technical Field

[0001] This disclosure relates to machine vision technology, and in particular to a method, apparatus, storage medium, and electronic device for color reproduction of an image. Background Technology

[0002] Currently, image sensors are widely used. In the process of processing images acquired by image sensors using an Image Signal Processor (ISP) system, white balance and color restoration are often required. However, some related technologies fail to achieve satisfactory color restoration for image sensors employing color filter arrays. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a method, apparatus, storage medium, and electronic device for color restoration of images.

[0004] According to one aspect of the present disclosure, a method for color restoration of an image is provided, comprising:

[0005] Obtain first color statistics information of the region including the predetermined object in the first color restoration and correction image; wherein, the first color restoration and correction image is determined based on a first native image acquired by a reference image sensor for the predetermined object;

[0006] An uncorrected image is determined based on a second native image acquired by the target image sensor for the predetermined object; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor.

[0007] Determine second color statistics for the region in the uncorrected image that includes the predetermined object;

[0008] Adjusting the first color statistics information yields various adjustment results for the first color statistics information;

[0009] Based on the uncorrected image, the second color statistics, and various adjustment results, a target color restoration correction matrix adapted to the target image sensor is determined, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0010] According to another aspect of the present disclosure, an image color restoration apparatus is provided, comprising:

[0011] The acquisition module is used to acquire first color statistics of the region including the predetermined object in the first color restoration and correction image; wherein the first color restoration and correction image is determined based on a first native image acquired by a reference image sensor for the predetermined object;

[0012] The first determining module is used to determine an uncorrected image based on a second native image acquired by the target image sensor for the predetermined object; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor.

[0013] The second determining module is used to determine second color statistics information of the region including the predetermined object in the uncorrected image determined by the first determining module;

[0014] An adjustment module is used to adjust the first color statistics information acquired by the acquisition module to obtain multiple adjustment results of the first color statistics information;

[0015] The third determining module is used to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determining module, the second color statistical information determined by the second determining module, and various adjustment results obtained by the adjustment module, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the color restoration method of the image described above.

[0017] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0018] processor;

[0019] Memory used to store the processor's executable instructions;

[0020] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-described image color restoration method.

[0021] According to another aspect of the present disclosure, a computer program product is provided that, when instructions in the computer program product are executed by a processor, performs the above-described image color restoration method.

[0022] Based on the image color restoration method, apparatus, storage medium, electronic device, and program product provided in the above embodiments of this disclosure,

[0023] An image sensor employing a second-type color filter array can be used as a reference image sensor, and an image sensor employing a first-type color filter array can be used as the target image sensor. Since the first color restoration correction image is determined based on the first native image acquired by the reference image sensor for the predetermined object, the first color statistics of the region including the predetermined object in the first color restoration correction image can reflect the good color performance of the reference image sensor under the premise of color restoration of the image. Based on the uncorrected image, the second color statistics of the region including the predetermined object in the uncorrected image, and various adjustment results of the first color statistics, a target color restoration correction matrix adapted to the target image sensor is determined. This allows the good color performance of the reference image sensor to be effectively applied in the determination process of the target color restoration correction matrix. Thus, it is beneficial to determine a target color restoration correction matrix that can better guarantee color performance. Using the target color restoration correction matrix for the color restoration of the target image to be corrected associated with the target image sensor can achieve a good color restoration effect for the target image sensor. Attached Figure Description

[0024] Figure 1 This is a schematic flowchart of an image color restoration method provided by some exemplary embodiments of this disclosure.

[0025] Figure 2 This is a flowchart illustrating a method for obtaining first color statistics of a region including a predetermined object in a first color restoration and correction image, provided by some exemplary embodiments of this disclosure.

[0026] Figure 3-1 This is a schematic diagram of a predetermined object in some exemplary embodiments of this disclosure.

[0027] Figure 3-2 This is a schematic diagram of a predetermined object in some other exemplary embodiments of this disclosure.

[0028] Figure 4 This is a flowchart illustrating a method for adjusting first color statistics to obtain various adjustment results of the first color statistics, provided by some exemplary embodiments of this disclosure.

[0029] Figure 5 This is a flowchart illustrating a method for determining a target color restoration correction matrix adapted to a target image sensor based on an uncorrected image, second color statistics, and various adjustment results, provided by some exemplary embodiments of this disclosure.

[0030] Figure 6 This is a flowchart illustrating a method for determining multiple reference color restoration correction matrices for converting second color statistics into multiple adjustment results, provided by some exemplary embodiments of this disclosure.

[0031] Figure 7 This is a flowchart illustrating a method for determining a target color restoration correction matrix adapted to a target image sensor from multiple reference color restoration correction matrices based on multiple second color restoration correction images, according to some exemplary embodiments of this disclosure.

[0032] Figure 8 This disclosure provides a flowchart illustrating a method for determining a target color restoration correction matrix adapted to a target image sensor from multiple reference color restoration correction matrices based on multiple second color restoration correction images, according to other exemplary embodiments.

[0033] Figure 9 This is a schematic diagram of a parameter setting page in some exemplary embodiments of this disclosure.

[0034] Figure 10 This is a schematic diagram of the structure of an image color restoration apparatus provided by some exemplary embodiments of this disclosure.

[0035] Figure 11 This is a schematic diagram of a third determining module in some exemplary embodiments of this disclosure.

[0036] Figure 12 This is a schematic diagram of a first determined submodule in some exemplary embodiments of this disclosure.

[0037] Figure 13 This is a schematic diagram of a second determining submodule in some exemplary embodiments of this disclosure.

[0038] Figure 14 This is a schematic diagram of a second determining submodule in some other exemplary embodiments of this disclosure.

[0039] Figure 15 This is a schematic diagram of the adjustment module in some exemplary embodiments of this disclosure.

[0040] Figure 16 This is a schematic diagram of the acquisition module in some exemplary embodiments of this disclosure.

[0041] Figure 17 This is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of this disclosure. Detailed Implementation

[0042] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0043] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0044] Application Overview

[0045] Currently, image sensors are widely used. Based on the color filter array (CFA) of the image sensor, the image acquired by the image sensor can contain color. The color filter array can include, but is not limited to, RCCG array, RCCB array, RGGB array, etc., and correspondingly, the image sensor can include, but is not limited to, RCCG image sensor, RCCB image sensor, RGGB image sensor, etc.; where R represents red, C represents transparent transmission, G represents green, and B represents blue.

[0046] In the process of processing images acquired by image sensors using an ISP system, white balance and color restoration are often performed. By restoring the color of the image, color deviations caused by the different responses of the human eye and the image sensor to the spectrum can be corrected to conform to human visual perception and meet perceptual needs.

[0047] In related technologies, image sensors employing certain color filter arrays often fail to achieve satisfactory color reproduction. Therefore, how to achieve good color reproduction for image sensors using these color filter arrays is a problem worthy of attention for those skilled in the art.

[0048] Exemplary System

[0049] In related technologies, while image sensors employing certain color filter arrays (hereinafter referred to as first-type color filter arrays) cannot achieve good color reproduction, image sensors employing other color filter arrays (hereinafter referred to as second-type color filter arrays) can achieve good color reproduction. For example, image sensors using RCCG arrays cannot achieve good color reproduction, while image sensors using RGGB arrays can achieve good color reproduction.

[0050] In view of this, in the embodiments of this disclosure, an image sensor employing a second type of color filter array can be used as a reference image sensor, and an image sensor employing a first type of color filter array can be used as a target image sensor. Based on a first primary color image acquired by the reference image sensor for a predetermined object, a first color restoration correction image can be determined. By adjusting the first color statistics of the region including the predetermined object in the first color restoration correction image, various adjustment results of the first color statistics can be obtained. Based on a second native image acquired by the target image sensor for the predetermined object, an uncorrected image can be determined. Based on the uncorrected image, the second color statistics of the region including the predetermined object in the uncorrected image, and the various adjustment results, a target color restoration correction matrix adapted to the target image sensor can be determined. Using the target color restoration correction matrix for color restoration of the target image to be corrected associated with the target image sensor can achieve a good color restoration effect for the target image sensor.

[0051] Exemplary methods

[0052] Figure 1 This is a schematic flowchart of an image color restoration method provided by some exemplary embodiments of this disclosure. Figure 1 The method shown may include steps 110, 120, 130, 140 and 150.

[0053] Step 110: Obtain first color statistics information of the region including the predetermined object in the first color restoration and correction image; wherein, the first color restoration and correction image is determined based on the first native image of the predetermined object acquired by the reference image sensor.

[0054] Optionally, the predetermined object can be an object with multiple colors. For example, the predetermined object can be a 24-color swatch. Another example is a rainbow umbrella.

[0055] Optionally, the reference image sensor can be an image sensor serving as a reference benchmark, used to assist in obtaining a color restoration correction matrix adapted to other image sensors; wherein, the color restoration correction matrix is ​​a matrix used for color restoration of the image, and the color restoration correction matrix can also be called a CCM correction matrix. The color filtering array of the reference image sensor can belong to the second type of color filtering array. As an example, the color filtering array of the reference image sensor can be an RGGB array, and correspondingly, the reference image sensor can be an RGGB image sensor, and the first native image acquired by the reference image sensor for a predetermined object can be an RGGB image. According to the color restoration scheme in the related art, the RGGB image, which serves as the first native image, can be color restored to obtain a first color restoration corrected image; or, the RGGB image, which serves as the first native image, can first be converted into an image suitable for user observation, and then the converted image can be color restored to obtain a first color restoration corrected image. It should be noted that the native image involved in the embodiments of this disclosure can also be called Raw data; the image suitable for user observation involved in the embodiments of this disclosure can be, for example, an RGB image.

[0056] Optionally, the first color restoration and correction image can be pre-stored in a predetermined storage area. Accordingly, in step 110, the first color restoration and correction image in the predetermined storage area can be read, and color statistics can be performed on the region including the predetermined object in the read first color restoration and correction image to obtain first color statistical information. The region including the predetermined object in the first color restoration and correction image may include multiple color blocks, and the first color statistical information may include color statistical information corresponding to each of the multiple color blocks; wherein, the color statistical information corresponding to any color block is used to characterize the color of that color block.

[0057] Of course, the method of obtaining the first color statistical information in step 110 is not limited to this. For example, color statistics can be performed on the region including the predetermined object in the first color restoration and correction image in advance to obtain the first color statistical information, and the first color statistical information can be stored in a predetermined storage area in advance. Accordingly, in step 110, the first color statistical information in the predetermined storage area can be read to obtain the first color statistical information.

[0058] Step 120: Based on the second native image acquired by the target image sensor for the predetermined object, determine the uncorrected image; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor.

[0059] Optionally, the target image sensor can be an image sensor for which a color restoration correction matrix needs to be determined. The color filter array of the target image sensor can belong to the first type of color filter array. The color filter array of the target image sensor can be different from the RGGB array; that is, the color filter array of the target image sensor can be a non-RGGB array, and correspondingly, the target image sensor can be a non-RGGB image sensor. As an example, the color filter array of the target image sensor can be an RCCG array or an RCCB array, and correspondingly, the target image sensor can be an RCCG image sensor or an RCCB image sensor. The second native image acquired by the target image sensor for the predetermined object can be an RCCG image or an RCCB image acquired for the predetermined object. In step 120, the second native image can be used as an uncorrected image, or the second native image can be converted into an image suitable for user observation, and then the converted image can be used as an uncorrected image; or, white balance can be applied to the second native image, and the white-balanced second native image can be used as an uncorrected image.

[0060] Step 130: Determine the second color statistics of the region in the uncorrected image that includes the predetermined object.

[0061] Optionally, color statistics can be performed on the region containing the predetermined object in the uncorrected image to obtain second color statistics information. The region containing the predetermined object in the uncorrected image may include multiple color patches, and the second color statistics information may include color statistics information corresponding to each of the multiple color patches; wherein, the color statistics information corresponding to any color patch is used to characterize the color of that color patch.

[0062] Step 140: Adjust the first color statistics information to obtain various adjustment results for the first color statistics information.

[0063] Optionally, multiple adjustment results of the first color statistics can be obtained by applying random perturbation to the first color statistics; wherein different adjustment results can correspond to different random perturbations.

[0064] Step 150: Based on the uncorrected image, second color statistics, and various adjustment results, determine the target color restoration correction matrix adapted to the target image sensor, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0065] Optionally, the transformation relationships between the second color statistics and various adjustment results can be determined, and a target color restoration correction matrix adapted to the target image sensor can be determined based on the uncorrected image and the determined transformation relationships. As an example, the target color restoration correction matrix can be a 3x3 matrix, meaning that both the number of rows and columns of the target color restoration correction matrix can be 3. The target color restoration correction matrix can be used for color restoration of the target image to be corrected associated with the target image sensor. The association between the target image to be corrected and the target image sensor can be understood as the target image to be corrected being determined based on the original image acquired by the target image sensor (this original image may be different from the second original image). For example, the original image acquired by the target image sensor can be used as the target image to be corrected. Another example is that the original image acquired by the target image sensor can be converted into an image suitable for user observation, and the converted image can be used as the target image to be corrected. Yet another example is that white balance can be applied to the original image acquired by the target image sensor, and the white-balanced original image can be used as the target image to be corrected.

[0066] In the embodiments of this disclosure, an image sensor employing a second type of color filter array can be used as a reference image sensor, and an image sensor employing a first type of color filter array can be used as a target image sensor. Since the first color restoration correction image is determined based on a first native image acquired by the reference image sensor for a predetermined object, the first color statistics of the region including the predetermined object in the first color restoration correction image can reflect the good color performance of the reference image sensor under the premise of color restoration of the image. Based on the uncorrected image, the second color statistics of the region including the predetermined object in the uncorrected image, and various adjustment results of the first color statistics, a target color restoration correction matrix adapted to the target image sensor is determined. This allows the good color performance of the reference image sensor to be effectively applied in the determination process of the target color restoration correction matrix. Thus, it is beneficial to determine a target color restoration correction matrix that can better guarantee color performance. Using the target color restoration correction matrix for the color restoration of the target image to be corrected associated with the target image sensor can achieve a good color restoration effect for the target image sensor.

[0067] Figure 2 This is a flowchart illustrating a method for obtaining first color statistics of a region including a predetermined object in a first color restoration and correction image, provided by some exemplary embodiments of this disclosure. Figure 2 The method shown may include steps 210, 220 and 230.

[0068] Step 210: Determine the image type of the first color restoration and correction image.

[0069] Optionally, the image type of the first color restoration and correction image can be either raw or non-raw. For example, if the first color restoration and correction image is an RGB image, then its image type can be non-raw. As another example, if the first color restoration and correction image is an RGGB image, then its image type can be raw.

[0070] Step 220: Determine the color statistics method that matches the image type of the first color restoration and correction image.

[0071] Step 230: According to the determined color statistics method, perform color statistics on the region including the predetermined object in the first color restoration and correction image to obtain the first color statistics information.

[0072] Optionally, the region including the predetermined object in the first color restoration and correction image may include multiple color patches, for example, K color patches, each of the K color patches may include multiple pixels; where K may be an integer greater than or equal to 2. Different color patches may include the same or different numbers of pixels. As an example, such as... Figure 3-1 , Figure 3-2 As shown, the predetermined object can be a 24-color swatch. Therefore, K color blocks can be 24 color blocks, which can be arranged in a 4x6 grid. Furthermore, each of the 24 color blocks can include p rows and q columns of pixels; where p and q are both integers greater than or equal to 2.

[0073] If the image type of the first color restoration and correction image is not raw, the color statistics method adapted to the image type of the first color restoration and correction image can be the first color statistics method. Then, according to the first color statistics method, color statistics can be performed on K color patches in the first color restoration and correction image to obtain the color statistics information corresponding to each of the K color patches. The color statistics information corresponding to each of the K color patches can constitute the first color statistics information. The following section uses the case where the first color restoration and correction image is an RGB image as an example to elaborate on the process of obtaining the first color statistics information based on the first color statistics method.

[0074] Since the first color restoration and correction image is an RGB image, each pixel in the first color restoration and correction image has a pixel value corresponding to the R channel, a pixel value corresponding to the G channel, and a pixel value corresponding to the B channel. Therefore, for each of the K color patches in the first color restoration and correction image, we can calculate the first average of the pixel values ​​corresponding to the R channel for all pixels in that color patch, the second average of the pixel values ​​corresponding to the G channel for all pixels in that color patch, and the third average of the pixel values ​​corresponding to the B channel for all pixels in that color patch. The calculated first, second, and third averages can constitute the color statistics information corresponding to that color patch. In this way, we can obtain K color statistics information corresponding one-to-one with the K color patches in the first color restoration and correction image, and these K color statistics information can constitute the first color statistics information.

[0075] If the first color-corrected image is of RAW type, the color statistics method adapted to this image type can be the second color statistics method. Then, according to the second color statistics method, color statistics can be performed on K color patches in the first color-corrected image to obtain the color statistics information corresponding to each of the K color patches. The color statistics information corresponding to each of the K color patches can constitute the first color statistics information. The following section uses the case where the first color-corrected image is an RGGB image as an example to elaborate on the process of obtaining the first color statistics information based on the first color statistics method.

[0076] Since the first color restoration and correction image is an RGGB image, each pixel in the first color restoration and correction image has only one pixel value, which corresponds to either the R channel, the G channel, or the B channel. Therefore, for each of the K color patches in the first color restoration and correction image, we can calculate the fourth mean of the pixel values ​​corresponding to the R channel, the fifth mean of the pixel values ​​corresponding to the G channel, and the sixth mean of the pixel values ​​corresponding to the B channel. The calculated fourth, fifth, and sixth mean values ​​constitute the color statistics information corresponding to that color patch. In this way, we can obtain K color statistics information corresponding one-to-one with the K color patches in the first color restoration and correction image, and these K color statistics information can constitute the first color statistics information.

[0077] In the embodiments of this disclosure, based on the image type of the first color restoration and correction image, an appropriate color statistics method can be adopted to perform color statistics on the region including the predetermined object in the first color restoration and correction image, which helps to ensure the rationality and reliability of the obtained first color statistics information.

[0078] In some embodiments, the uncorrected image can be an RGB image, that is, the image type of the uncorrected image can be a non-raw type. Additionally, the uncorrected image can include K color patches. Therefore, in step 130, color statistics can be performed on the K color patches in the uncorrected image according to the second color statistics method to obtain the color statistics information corresponding to each of the K color patches. The color statistics information corresponding to each of the K color patches can constitute the second color statistics information.

[0079] In other embodiments, the uncorrected image can be an RCCG image, that is, the image type of the uncorrected image can be raw. Additionally, the uncorrected image can include K color patches. Therefore, in step 130, color statistics can be performed on the K color patches in the uncorrected image according to the first color statistics method to obtain the color statistics information corresponding to each of the K color patches. The color statistics information corresponding to each of the K color patches can constitute the second color statistics information. It should be noted that in this case, compared with... Figure 2 The difference in the process of obtaining the first color statistical information based on the first color statistical method in the illustrated embodiment is mainly that: for each of the K color blocks in the uncorrected image, it is necessary to calculate the average value of the pixel values ​​of all pixels in the color block corresponding to the R channel, the average value of the pixel values ​​of all pixels in the color block corresponding to the C channel, and the average value of the pixel values ​​of all pixels in the color block corresponding to the G channel.

[0080] Figure 4 This is a flowchart illustrating a method for adjusting first color statistics to obtain various adjustment results of the first color statistics, provided by some exemplary embodiments of this disclosure. Figure 4 The method shown may include steps 410, 420, 430 and 440.

[0081] Step 410: Based on the first color statistics information, determine that each color patch in the region including the predetermined object corresponds to N color statistics values ​​of the N channels included in the first color restoration and correction image.

[0082] Optionally, multiple color patches can be represented as K color patches, and the first color statistics information can include N color statistics values ​​for each of the K color patches corresponding to the N channels included in the first color restoration and correction image. Then, the N color statistics values ​​for each of the K color patches corresponding to the N channels included in the first color restoration and correction image can be extracted from the first color statistics information.

[0083] In one example, N channels can be 3 channels: R channel, G channel, and B channel. If the first color restoration and correction image is an RGB image, that is, the image type of the first color restoration and correction image is not raw, then for each of the K color patches in the first color restoration and correction image, the corresponding first mean, the corresponding second mean, and the corresponding third mean can be extracted from the first color statistics information. The extracted first mean can be used as the color statistics value of the color patch corresponding to the R channel, the extracted second mean can be used as the color statistics value of the color patch corresponding to the G channel, and the extracted third mean can be used as the color statistics value of the color patch corresponding to the B channel. If the first color restoration and correction image is an RGGB image, that is, the image type of the first color restoration and correction image is raw, then for each of the K color patches in the first color restoration and correction image, the corresponding fourth mean, the corresponding fifth mean, and the corresponding sixth mean can be extracted from the first color statistics information. The extracted fourth mean can be used as the color statistics value of the color patch corresponding to the R channel, the extracted fifth mean can be used as the color statistics value of the color patch corresponding to the G channel, and the extracted sixth mean can be used as the color statistics value of the color patch corresponding to the B channel.

[0084] Step 420: Within the preset random deviation value range, determine that each color patch in the multiple color patches corresponds to N random deviation values ​​of N channels.

[0085] Optionally, the upper limit of the preset random deviation value range can be represented as r2, and the lower limit can be represented as r1, then the preset random deviation value range can be represented as [r1, r2]. For each of the K color blocks, random deviation values ​​corresponding to N channels can be randomly selected within [r1, r2] to obtain N random deviation values. For example, for each of the K color blocks, random deviation values ​​corresponding to the R channel, G channel, and B channel can be randomly selected within [r1, r2] to obtain 3 random deviation values ​​corresponding to the 3 channels for that color block.

[0086] Step 430: The N color statistics corresponding to each color block in the multiple color blocks are superimposed with the corresponding random deviation values ​​to obtain the N color superimposed values ​​corresponding to each color block in the multiple color blocks.

[0087] Optionally, the summation of each color statistical value and its corresponding random deviation value can be achieved through a summation process. For example, for each of the K color patches, the color statistical value corresponding to the R channel can be summed with the random deviation value corresponding to the R channel, and the sum can be used as the color overlay value corresponding to the R channel. Similarly, the color overlay value corresponding to the G channel and the color overlay value corresponding to the B channel can also be obtained in this way.

[0088] Step 440: Based on the N color superposition values ​​corresponding to each color block in the multiple color blocks, determine one of the multiple adjustment results of the first color statistics information.

[0089] Optionally, for each of the K color patches, the N superimposed color values ​​corresponding to that patch can constitute the adjusted color statistics for that patch. In this way, the adjusted color statistics corresponding to each of the K color patches can be obtained, and these adjusted color statistics can constitute one adjustment result of the first color statistics. Other adjustment results of the first color statistics are obtained in a similar manner and will not be elaborated upon here.

[0090] In the embodiments of this disclosure, color statistical values ​​can be extracted from the first color statistical information, a random deviation value can be determined within a preset random deviation value range, and the color statistical values ​​and the random deviation values ​​can be superimposed. This is equivalent to randomly changing the color statistical values ​​in the first color statistical information, thereby efficiently and quickly adjusting the first color statistical information by applying random perturbation.

[0091] Figure 5 This is a flowchart illustrating a method for determining a target color restoration correction matrix adapted to a target image sensor based on an uncorrected image, second color statistics, and various adjustment results, provided by some exemplary embodiments of this disclosure. Figure 5 The method shown may include steps 510, 520 and 530.

[0092] Step 510: Determine multiple reference color restoration correction matrices for converting the second color statistics into various adjustment results.

[0093] Optionally, the multiple adjustment results can be represented as D adjustment results; where D can be an integer greater than or equal to 2. For each of the D adjustment results, a reference color reproduction correction matrix can be determined to convert the second color statistics to that adjustment result. This reference color reproduction correction matrix can be used to characterize the conversion relationship between the second color statistics and the adjustment result. In this way, D reference color reproduction correction matrices corresponding one-to-one with the D adjustment results can be obtained. Each of the D reference color reproduction correction matrices can be a 3*3 matrix.

[0094] In some alternative embodiments of this disclosure, such as Figure 6 As shown, step 510 may include steps 610 and 620.

[0095] Step 610: Determine the weights of the multiple color blocks in the region including the predetermined object.

[0096] Optionally, multiple color blocks can be represented as K color blocks. In step 610, the weights corresponding to the K color blocks can be determined according to preset rules. As an example, the predetermined object can be a 24-color swatch, and the preset rules can include, but are not limited to, the following rules: (1) make the weight corresponding to the main color block higher than the weight corresponding to other color blocks. The main color block can include, for example, red, green, yellow, white, blue, etc.; (2) according to the actual application scenario, make the weight corresponding to the color block with the color to be focused higher than the weight corresponding to other color blocks. In the scene of shooting people, the color to be focused can be, for example, skin color. In the scene of shooting scenery, the color to be focused can be, for example, green (the color of grass), blue (the color of the sky), etc.

[0097] Step 620: Based on the weights corresponding to the multiple color blocks and the regression algorithm, determine multiple reference color restoration correction matrices for converting the second color statistical information into multiple adjustment results.

[0098] Optionally, based on the weights corresponding to the K color patches and the regression algorithm, D reference color restoration correction matrices corresponding one-to-one with the D adjustment results can be determined. The regression algorithm involved in the embodiments of this disclosure can be either a linear regression algorithm or a nonlinear regression algorithm. As an example, the regression algorithm involved in the embodiments of this disclosure can be the least squares method. Accordingly, the reference color restoration correction matrix corresponding to any adjustment result can be obtained by minimizing the objective function value. The objective function value can be expressed by the following formula:

[0099]

[0100] Where, k i Let x represent the weight of the i-th color block out of the K color blocks. i This represents the adjusted color statistics corresponding to the i-th color patch in the adjustment result, Mat represents the reference color reproduction correction matrix corresponding to the adjustment result, and y i This represents the color statistics information corresponding to the i-th color block in the second color statistics information.

[0101] Figure 6 In the embodiment shown, the second color statistics can be used as the original training value for the regression algorithm, and the various adjustment results can be used as the training target values ​​for the regression algorithm. By combining the weights corresponding to the multiple color blocks, the reference color restoration correction matrix corresponding to the various adjustment results can be obtained efficiently and quickly through the operation of the regression algorithm.

[0102] Of course, the implementation of step 510 is not limited to this. For example, for each of the multiple adjustment results, a matrix for converting the adjustment result to the second color statistics information can be determined first, and then the inverse matrix of the matrix can be calculated. The obtained inverse matrix can be used as the reference color restoration correction matrix corresponding to the adjustment result.

[0103] Step 520: Using multiple reference color restoration and correction matrices, color restoration is performed on the uncorrected image to obtain multiple second color restoration and correction images.

[0104] Optionally, the multiple reference color restoration correction matrices can be represented as D reference color restoration correction matrices. For each of the D reference color restoration correction matrices, the uncorrected image can be color restored using that reference color restoration correction matrix to obtain a corresponding second color restoration corrected image. In this way, D second color restoration corrected images corresponding one-to-one with the D reference color restoration correction matrices can be obtained.

[0105] In one example, the uncorrected image can be an RGB image. Any pixel in the uncorrected image corresponds to the pixel value of the R channel, the pixel value of the G channel, and the pixel value of the B channel, denoted as u1, u2, u3, respectively. A reference color restoration correction matrix is ​​denoted as Mat. The corresponding pixels in the second color restoration corrected image corresponding to this reference color restoration correction matrix are denoted as v1, v2, v3, respectively, denoted as v1, v2, v3, respectively. Then:

[0106]

[0107] Step 530: Based on multiple second color restoration correction images, determine the target color restoration correction matrix adapted to the target image sensor from multiple reference color restoration correction matrices.

[0108] Optionally, based on multiple second color restoration correction images, the merits of multiple reference color restoration correction matrices can be evaluated. On this basis, a target color restoration correction matrix adapted to the target image sensor can be determined from the multiple reference color restoration correction matrices.

[0109] In some alternative embodiments of this disclosure, such as Figure 7 As shown, step 530 may include steps 710, 720, 730 and 740.

[0110] Step 710: Determine the third color statistics corresponding to the regions including the predetermined objects in each second color restoration and correction image, and determine the color deviation between each third color statistics and the first color statistics.

[0111] Optionally, all D second color restoration and correction images can be of non-raw type. In this case, for each of the D second color restoration and correction images, color statistics can be performed on the region containing the predetermined object within that second color restoration and correction image according to the second color statistical method to obtain corresponding third color statistical information. This results in D third color statistical information pieces corresponding one-to-one with the D second color restoration and correction images. Alternatively, all D second color restoration and correction images can be of raw type. In this case, D third color statistical information pieces can be obtained based on the first color statistical method. For each of the D third color statistical information pieces, the color deviation between the third color statistical information piece and the first color statistical information piece can be determined by subtraction.

[0112] In one example, each of the D third color statistics can include color statistics corresponding to K color patches. The color statistics for each color patch can include color statistics for the R, G, and B channels, respectively; that is, the third color statistics can include 3K color statistics. Similarly, the first color statistics also include 3K color statistics. Then, the 3K color statistics in the third color statistics can be subtracted from the 3K color statistics in the first color statistics to obtain 3K color differences. Here, K color differences correspond to the R channel, K color differences correspond to the G channel, and K color differences correspond to the B channel. These 3K color statistics can be used to represent the color deviation between the third and first color statistics. If the intended object is a 24-color swatch, the 3K color differences can be represented as a tensor with a width of 6, a height of 4, and 3 channels. This tensor can be used to represent the color deviation between the third and first color statistics.

[0113] Step 720: Determine the signal-to-noise ratio corresponding to each second color restoration correction image.

[0114] Optionally, for each of the D second color restoration correction images, the signal-to-noise ratio (SNR) of that second color restoration correction image can be determined, thereby obtaining D SNRs corresponding one-to-one with the D second color restoration correction images. Generally speaking, the signal-to-noise ratio refers to the ratio of the intensity of the useful signal to the intensity of the interference signal.

[0115] Step 730: Based on the color deviation and / or signal-to-noise ratio corresponding to each second color restoration correction image, select a portion of the reference color restoration correction matrices from the multiple reference color restoration correction matrices.

[0116] Step 740: Based on the second color restoration correction image corresponding to the selected partial reference color restoration correction matrix, determine the target color restoration correction matrix adapted to the target image sensor from the selected partial reference color restoration correction matrix.

[0117] It should be noted that by executing steps 710 and 720, D color deviations corresponding one-to-one with the D reference color reproduction correction matrices, and D signal-to-noise ratios corresponding one-to-one with the D reference color reproduction correction matrices, can be obtained. In step 730, based on the D color deviations and / or the D signal-to-noise ratios, a portion of the reference color reproduction correction matrices can be selected from the D reference color reproduction correction matrices.

[0118] For example, from D reference color correction matrices, reference color correction matrices with color deviations less than a preset deviation threshold can be selected. Here, the preset deviation threshold may include a first deviation threshold for the R channel, a second deviation threshold for the G channel, and a third deviation threshold for the B channel. For any reference color correction matrix, the color deviation may include 3K color differences. If the maximum value among the K color differences corresponding to the R channel is less than the first deviation threshold, the maximum value among the K color differences corresponding to the G channel is less than the second deviation threshold, and the maximum value among the K color differences corresponding to the B channel is less than the third deviation threshold, then the color deviation can be determined to be less than the preset deviation threshold.

[0119] For example, one can select reference color reproduction correction matrices from D reference color reproduction correction matrices whose corresponding signal-to-noise ratio is greater than a preset signal-to-noise ratio threshold.

[0120] Alternatively, from the D reference color reproduction correction matrices, the reference color reproduction correction matrices with color deviations less than a preset deviation threshold and corresponding signal-to-noise ratios greater than a preset signal-to-noise ratio threshold can be selected.

[0121] If in step 730 only one reference color reproduction correction matrix is ​​selected from D reference color reproduction correction matrices, then in step 740, this reference color reproduction correction matrix can be used as the target color reproduction correction matrix.

[0122] If in step 730, more than one reference color reproduction correction matrix is ​​selected from D reference color reproduction correction matrices, then in step 740, a further reference color reproduction correction matrix can be selected from the more than one reference color reproduction correction matrix, and this reference color reproduction correction matrix can be used as the target color reproduction correction matrix.

[0123] Figure 7 In the illustrated embodiment, for each of the multiple second color restoration correction images, the quality of the reference color restoration correction matrix corresponding to that second color restoration correction image can be determined based on the color deviation and / or the signal-to-noise ratio corresponding to that second color restoration correction image. For example, the smaller the color deviation corresponding to the second color restoration correction image, and / or the larger the signal-to-noise ratio corresponding to the second color restoration correction image, the better the reference color restoration correction matrix corresponding to that second color restoration correction image can be determined. This provides an effective reference for determining the target color restoration correction matrix, which helps ensure the rationality and reliability of the finally determined target color restoration correction matrix.

[0124] In other alternative embodiments of this disclosure, such as Figure 8 As shown, step 530 may include steps 810, 820 and 830.

[0125] Step 810: Perform a predetermined machine vision task on multiple second color restoration and correction images respectively, and obtain multiple task execution results.

[0126] Optionally, the predetermined machine vision task can be an object detection task, which can be a task for detecting predetermined objects in an image. In step 810, a pre-trained object detection model can be used to perform object detection on D second color restoration and correction images respectively, so as to obtain D detection results corresponding one-to-one with the D second color restoration and correction images; wherein, each of the D detection results can be a task execution result, and each task execution result can include a two-dimensional detection box and the confidence level corresponding to the two-dimensional detection box.

[0127] Step 820: Evaluate the execution results of multiple tasks to obtain multiple evaluation results.

[0128] It should be noted that by executing step 810, D task execution results can be obtained. For each of the D task execution results, a confidence score can be extracted from the task execution result, and the extracted confidence score can be used as the evaluation result corresponding to the task execution result. Alternatively, the extracted confidence score can be converted to a set evaluation score range, and the resulting evaluation score can be used as the evaluation result corresponding to the task execution result.

[0129] In some embodiments, for each of the D task execution results, the task execution result can also be displayed on the corresponding second color restoration and correction image, so as to facilitate the user to observe the task execution result and score it, and use the score value as the evaluation result corresponding to the task execution result.

[0130] Step 830: Based on multiple evaluation results, determine the target color reproduction correction matrix adapted to the target image sensor from multiple reference color reproduction correction matrices.

[0131] It should be noted that by executing step 820, D evaluation results corresponding one-to-one with the D reference color reproduction correction matrices can be obtained. Optionally, all D evaluation results can be in numerical form, and the larger the value, the better the corresponding reference color reproduction correction matrix. Then, in step 830, the evaluation result with the largest value can be selected from the D evaluation results, and the reference color reproduction correction matrix corresponding to the selected evaluation result can be used as the target color reproduction correction matrix.

[0132] In some embodiments, all D evaluation results can be in numerical form, and the smaller the value, the better the corresponding reference color reproduction correction matrix. Therefore, in step 830, the evaluation result with the smallest value can be selected from the D evaluation results, and the reference color reproduction correction matrix corresponding to the selected evaluation result can be used as the target color reproduction correction matrix.

[0133] Figure 8 In the illustrated embodiment, for each of the multiple second color restoration correction images, the quality of the reference color restoration correction matrix corresponding to that second color restoration correction image can be determined based on the task execution result corresponding to that second color restoration correction image. This provides a valid reference for determining the target color restoration correction matrix, helping to ensure the rationality and reliability of the finally determined target color restoration correction matrix.

[0134] Optionally, Figure 7 The illustrated implementation and Figure 8 The embodiments shown can be used in combination. For example, if in step 730, one or more reference color restoration correction matrices are selected from D reference color restoration correction matrices, since one or more reference color restoration correction matrices correspond to one or more second color restoration correction images, then a predetermined machine vision task can be performed on each of the one or more second color restoration correction images to obtain one or more task execution results. Afterwards, the one or more task execution results can be evaluated, and based on the obtained evaluation results, a target color restoration correction matrix can be determined from the one or more reference color restoration correction matrices.

[0135] In the embodiments of this disclosure, by determining multiple reference color restoration correction matrices for converting second color statistical information into various adjustment results, the color restoration of the uncorrected image can be performed using these multiple reference color restoration correction matrices to obtain multiple second color restoration correction images. Based on these multiple second color restoration correction images, the quality of the multiple reference color restoration correction matrices can be effectively evaluated. This provides a valid reference for determining the target color restoration correction matrix, helping to ensure the rationality and reliability of the finally determined target color restoration correction matrix. For example, based on the target color restoration correction matrix, the color performance of the target image sensor can be made as consistent as possible with the color performance of the reference image sensor, thereby ensuring the color restoration effect for the target image sensor and also ensuring good signal-to-noise ratio performance of the target image to be corrected after color restoration. Thus, the embodiments of this disclosure not only conform to human visual perception and meet perceptual needs but also take into account both color performance and noise performance.

[0136] In some optional examples, the reference image sensor may be an RGGB image sensor, and the target image sensor may be an RCCG image sensor.

[0137] Therefore, at a preset color temperature, the RGGB image sensor can be used to acquire a native image of a 24-color swatch (e.g., the first native image mentioned above). Based on the first native image, color restoration can be performed to obtain a first color restoration correction image. By performing color statistics on the regions of the 24-color swatch included in the first color restoration correction image, first color statistical information can be obtained. By applying random perturbations to the first color statistical information, various adjustment results for the first color statistical information can be obtained.

[0138] Alternatively, at a preset color temperature, an RCCG image sensor can be used to acquire an image of a 24-color chart (equivalent to the second native image mentioned above). Based on the second native image, an uncorrected image can be identified. By performing color statistics on the regions in the uncorrected image that include predetermined objects, second color statistical information can be obtained.

[0139] Optionally, the weights corresponding to multiple color blocks can also be determined. For example, in... Figure 9 On the parameter settings page shown, enter 24 weights that correspond one-to-one with the 24 color blocks (see [reference]). Figure 9 (The large rectangle in the upper left part).

[0140] Alternatively, the second color statistics can be used as the raw training values ​​for the regression algorithm, and the various adjustment results can be used as the target training values ​​for the regression algorithm. Combining these with the weights corresponding to the multiple color patches, multiple reference color restoration and correction matrices can be obtained through the regression algorithm. Using these multiple reference color restoration and correction matrices, color restoration can be performed on the uncorrected image, resulting in multiple second color restoration and correction images.

[0141] Subsequently, the color deviation and signal-to-noise ratio (SNR) corresponding to multiple second color restoration and correction images can be determined. Based on the determined color deviation and SNR, the target color restoration and correction matrix is ​​determined from multiple reference color restoration and correction matrices. Alternatively, the target color restoration and correction matrix can be determined from multiple reference color restoration and correction matrices by combining human visual and machine visual evaluations. The target color restoration and correction matrix can be used for color restoration of the target image to be corrected associated with the target image sensor.

[0142] In summary, the embodiments of this disclosure can achieve good color reproduction for image sensors using the first color filter array, which can not only meet the perception of the human eye and satisfy the perception needs, but also take into account both color performance and noise performance.

[0143] Exemplary device

[0144] Figure 10 This is a schematic diagram of the structure of an image color restoration apparatus provided by some exemplary embodiments of this disclosure. Figure 10 The apparatus shown may include:

[0145] The acquisition module 1010 is used to acquire first color statistics information of the region including the predetermined object in the first color restoration and correction image; wherein, the first color restoration and correction image is determined based on the first native image acquired by the reference image sensor for the predetermined object;

[0146] The first determining module 1020 is used to determine an uncorrected image based on a second native image acquired by the target image sensor for a predetermined object; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor.

[0147] The second determining module 1030 is used to determine the second color statistics of the region including the predetermined object in the uncorrected image determined by the first determining module 1020;

[0148] The adjustment module 1040 is used to adjust the first color statistics information acquired by the acquisition module 1010 to obtain various adjustment results of the first color statistics information;

[0149] The third determining module 1050 is used to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determining module 1020, the second color statistical information determined by the second determining module 1030, and the various adjustment results obtained by the adjustment module 1040, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0150] In some optional examples, such as Figure 11 As shown, the third determining module 1050 includes:

[0151] The first determining submodule 1110 is used to determine multiple reference color restoration correction matrices for converting the second color statistical information determined by the second determining module 1030 to the multiple adjustment results obtained by the adjustment module 1040 respectively.

[0152] The color restoration submodule 1120 is used to restore the color of the uncorrected image determined by the first determining submodule 1110 using multiple reference color restoration correction matrices determined by the first determining submodule 1110, so as to obtain multiple second color restoration correction images.

[0153] The second determining submodule 1130 is used to determine, based on the multiple second color restoration correction images obtained by the color restoration submodule 1120, a target color restoration correction matrix adapted to the target image sensor from the multiple reference color restoration correction matrices determined by the first determining submodule 1110.

[0154] In some optional examples, such as Figure 12 As shown, the first determining submodule 1110 includes:

[0155] The first determining unit 1210 is used to determine the weights corresponding to multiple color blocks in the region including the predetermined object;

[0156] The second determining unit 1220 is used to determine, based on the weights corresponding to the multiple color blocks determined by the first determining unit 1210 and the regression algorithm, multiple reference color restoration correction matrices for converting the second color statistical information determined by the second determining module 1030 to the multiple adjustment results obtained by the adjustment module 1040.

[0157] In some optional examples, such as Figure 13 As shown, the second determining submodule 1130 includes:

[0158] The third determining unit 1310 is used to determine the third color statistical information corresponding to the regions including the predetermined objects in each of the second color restoration correction images obtained by the color restoration submodule 1120, and to determine the color deviation between each third color statistical information and the first color statistical information.

[0159] The fourth determining unit 1320 is used to determine the signal-to-noise ratio of each second color restoration correction image obtained by the color restoration submodule 1120;

[0160] The filtering unit 1330 is used to filter a portion of the reference color restoration correction matrices from the multiple reference color restoration correction matrices obtained by the color restoration submodule 1120 based on the color deviations determined by the third determining unit 1310 and / or the signal-to-noise ratios determined by the fourth determining unit 1320.

[0161] The fifth determining unit 1340 is used to determine the target color restoration correction matrix adapted to the target image sensor from the partial reference color restoration correction matrix corresponding to the second color restoration correction image filtered by the filtering unit 1330.

[0162] In some optional examples, such as Figure 14 As shown, the second determining submodule 1130 includes:

[0163] Execution unit 1410 is used to perform predetermined machine vision tasks on multiple second color restoration and correction images obtained by color restoration submodule 1120 respectively, and obtain multiple task execution results;

[0164] The evaluation unit 1420 is used to evaluate the multiple task execution results obtained by the execution unit 1410 respectively, and obtain multiple evaluation results;

[0165] The sixth determining unit 1430 is used to determine, based on the multiple evaluation results obtained by the execution unit 1410, a target color restoration correction matrix adapted to the target image sensor from the multiple reference color restoration correction matrices determined by the first determining submodule 1110.

[0166] In some optional examples, such as Figure 15 As shown, the adjustment module 1040 includes:

[0167] The third determining submodule 1510 is used to determine, based on the first color statistical information obtained by the acquisition module 1010, the N color statistical values ​​of each color block in the region including the predetermined object corresponding to the N channels included in the first color restoration correction image.

[0168] The fourth determining submodule 1520 is used to determine, within a preset random deviation value range, N random deviation values ​​corresponding to each color block in the multiple color blocks for N channels.

[0169] The superposition submodule 1530 is used to superimpose the N color statistical values ​​corresponding to each color block in the multiple color blocks determined by the third determining submodule 1510 with the corresponding random deviation values ​​determined by the fourth determining submodule 1520 to obtain the N color superposition values ​​corresponding to each color block in the multiple color blocks.

[0170] The fifth determining submodule 1540 is used to determine one of the multiple adjustment results of the first color statistics information obtained by the acquisition module 1010 based on the N color superposition values ​​corresponding to each of the multiple color blocks obtained by the superposition submodule 1530.

[0171] In some optional examples, such as Figure 16 As shown, the acquisition module includes:

[0172] The sixth determining submodule 1610 is used to determine the image type of the first color restoration and correction image;

[0173] The seventh determining submodule 1620 is used to determine the color statistics method that is compatible with the image type determined by the sixth determining submodule 1610;

[0174] Color statistics submodule 1630 is used to perform color statistics on the region including the predetermined object in the first color restoration and correction image according to the color statistics method determined by the seventh determination submodule 1620, so as to obtain the first color statistics information.

[0175] In the apparatus disclosed herein, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve the corresponding functions and effects, and this disclosure does not list them all.

[0176] Exemplary electronic devices

[0177] Figure 17 The illustration shows a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 1700 includes one or more processors 1710 and a memory 1720.

[0178] The processor 1710 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1700 to perform desired functions.

[0179] The memory 1720 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1710 may execute one or more computer program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions.

[0180] In one example, the electronic device 1700 may also include an input device 1730 and an output device 1740, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0181] The input device 1730 may also include, for example, a keyboard, a mouse, etc.

[0182] The output device 1740 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0183] Of course, for the sake of simplicity, Figure 17 Only some of the components of the electronic device 1700 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1700 may include any other suitable components depending on the specific application.

[0184] Exemplary computer program products and computer-readable storage media

[0185] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0186] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0187] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0188] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0189] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. The specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the specific details described above.

[0190] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for color restoration of an image, comprising: Obtain first color statistics information of the region including the predetermined object in the first color restoration and correction image; wherein, the first color restoration and correction image is determined based on a first native image acquired by a reference image sensor for the predetermined object; An uncorrected image is determined based on a second native image acquired by the target image sensor for the predetermined object; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor. Determine second color statistics for the region in the uncorrected image that includes the predetermined object; Adjusting the first color statistics information yields various adjustment results for the first color statistics information; Based on the uncorrected image, the second color statistics, and various adjustment results, a target color restoration correction matrix adapted to the target image sensor is determined, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix; wherein, the target color restoration correction matrix is ​​determined based on the transformation relationship between the uncorrected image and the second color statistics and various adjustment results respectively.

2. The method according to claim 1, wherein, The step of determining the target color restoration correction matrix adapted to the target image sensor based on the uncorrected image, the second color statistics, and various adjustment results includes: Determine multiple reference color restoration correction matrices for converting the second color statistics into various adjustment results, respectively; The uncorrected image is color restored using multiple reference color restoration correction matrices to obtain multiple second color restoration correction images. Based on multiple second color restoration correction images, a target color restoration correction matrix adapted to the target image sensor is determined from multiple reference color restoration correction matrices.

3. The method according to claim 2, wherein, The determination of multiple reference color restoration correction matrices for converting the second color statistics information into various adjustment results includes: Determine the weights corresponding to multiple color blocks in the region including the predetermined object; Based on the weights corresponding to the multiple color patches and the regression algorithm, multiple reference color restoration correction matrices are determined for converting the second color statistics information into multiple adjustment results.

4. The method according to claim 2, wherein, The step of determining a target color restoration correction matrix adapted to the target image sensor from multiple reference color restoration correction matrices based on multiple second color restoration correction images includes: The third color statistics information corresponding to the regions including the predetermined object in each of the second color restoration and correction images is determined, and the color deviation between each of the third color statistics information and the first color statistics information is determined; Determine the signal-to-noise ratio corresponding to each of the second color restoration and correction images; Based on the color deviation and / or the signal-to-noise ratio corresponding to each of the second color restoration and correction images, a portion of the reference color restoration and correction matrices are selected from the plurality of reference color restoration and correction matrices; Based on the second color restoration correction image corresponding to the selected portion of the reference color restoration correction matrix, a target color restoration correction matrix adapted to the target image sensor is determined from the selected portion of the reference color restoration correction matrix.

5. The method according to claim 2, wherein, The step of determining a target color restoration correction matrix adapted to the target image sensor from multiple reference color restoration correction matrices based on multiple second color restoration correction images includes: A predetermined machine vision task is performed on multiple second color restoration and correction images to obtain multiple task execution results; The execution results of the multiple tasks are evaluated separately to obtain multiple evaluation results; Based on the multiple evaluation results, a target color reproduction correction matrix adapted to the target image sensor is determined from the multiple reference color reproduction correction matrices.

6. The method according to any one of claims 1-5, wherein, The adjustment of the first color statistics information yields various adjustment results, including: Based on the first color statistics information, each of the multiple color patches in the region including the predetermined object is determined to correspond to N color statistics values ​​of the N channels included in the first color restoration and correction image; Within a preset random deviation value range, each of the multiple color patches is determined to correspond to N random deviation values ​​of the N channels; The N color statistical values ​​corresponding to each of the multiple color blocks are superimposed with the corresponding random deviation values ​​to obtain the N color superimposed values ​​corresponding to each of the multiple color blocks; Based on the N color superposition values ​​corresponding to each of the multiple color blocks, one of the various adjustment results of the first color statistics is determined.

7. The method according to any one of claims 1-5, wherein, The step of obtaining the first color statistics information of the region including the predetermined object in the first color restoration and correction image includes: Determine the image type of the first color restoration and correction image; Determine a color statistics method that is compatible with the image type; According to the color statistics method, color statistics are performed on the region including the predetermined object in the first color restoration and correction image to obtain the first color statistics information.

8. A color reproduction device for an image, comprising: The acquisition module is used to acquire first color statistics of the region including the predetermined object in the first color restoration and correction image; wherein the first color restoration and correction image is determined based on a first native image acquired by a reference image sensor for the predetermined object; The first determining module is used to determine an uncorrected image based on a second native image acquired by the target image sensor for the predetermined object; wherein the color filter array of the target image sensor is different from the color filter array of the reference image sensor. The second determining module is used to determine second color statistics information of the region including the predetermined object in the uncorrected image determined by the first determining module; An adjustment module is used to adjust the first color statistics information acquired by the acquisition module to obtain multiple adjustment results of the first color statistics information; The third determining module is used to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determining module, the second color statistical information determined by the second determining module, and the various adjustment results obtained by the adjustment module, so as to restore the color of the target image to be corrected associated with the target image sensor through the target color restoration correction matrix; wherein, the target color restoration correction matrix is ​​determined based on the conversion relationship between the uncorrected image and the second color statistical information and the various adjustment results.

9. A computer-readable storage medium storing a computer program that is executed by a processor to implement the color restoration method for an image according to any one of claims 1-7.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image color restoration method according to any one of claims 1-7.

Images