Image color restoration method and device, storage medium and electronic equipment

Abstract

The invention discloses an image color restoration method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring first color statistical information of an area including a predetermined object in a first color rendition correction image; wherein the first color rendition correction image is determined based on a first original image collected by the reference image sensor for the predetermined object; determining an uncorrected image based on a second native image collected by the target image sensor for the predetermined object; determining second color statistical information of a region including the predetermined object in the uncorrected image; adjusting the first color statistical information to obtain multiple adjustment results of the first color statistical information; and determining a target color rendition correction matrix matched with the target image sensor based on the uncorrected image, the second color statistical information and the plurality of adjustment results. According to the embodiment of the invention, a good color restoration effect can be achieved for the target image sensor.

Description

Technical Field

[0001] The present disclosure relates to machine vision technology, and in particular to a method, device, storage medium and electronic device for color restoration of an image. Background Art

[0002] At present, image sensors are widely used. When using an image signal processing (ISP) system to process images collected by image sensors, it is often necessary to perform image white balance, color restoration, etc. In the related art, image sensors using some color filter arrays cannot achieve good color restoration effects. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a method, device, storage medium and electronic device for color restoration of an image.

[0004] According to one aspect of an embodiment of the present disclosure, a method for color restoration of an image is provided, comprising:

[0005] Acquire first color statistical information of a region including a predetermined object in a first color restoration corrected image; wherein the first color restoration corrected image is determined based on a first native image acquired by a reference image sensor for the predetermined object;

[0006] Determining an uncorrected image based on a second native image captured by a target image sensor for the predetermined object; wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor;

[0007] determining second color statistics of a region of the uncorrected image including the predetermined object;

[0008] Adjusting the first color statistical information to obtain multiple adjustment results of the first color statistical information;

[0009] Based on the uncorrected image, the second color statistical information and the multiple adjustment results, a target color restoration correction matrix adapted to the target image sensor is determined, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0010] According to another aspect of the embodiments of the present disclosure, there is provided a color restoration device for an image, comprising:

[0011] An acquisition module, configured to acquire first color statistical information of a region including a predetermined object in a first color restoration-corrected image; wherein the first color restoration-corrected image is determined based on a first native image acquired by a reference image sensor for the predetermined object;

[0012] A first determination module is configured to determine an uncorrected image based on a second native image acquired by a target image sensor for the predetermined object; wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor;

[0013] a second determination module, configured to determine second color statistical information of the area including the predetermined object in the uncorrected image determined by the first determination module;

[0014] an adjusting module, configured to adjust the first color statistical information acquired by the acquiring module to obtain a plurality of adjustment results of the first color statistical information;

[0015] a third determination module, configured to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determination module, the second color statistical information determined by the second determination module, and the multiple adjustment results obtained by the adjustment module, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0016] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned color restoration method of the image.

[0017] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0018] processor;

[0019] a memory for storing instructions executable by the processor;

[0020] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned image color restoration method.

[0021] According to another aspect of the embodiments of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, the above-mentioned color restoration method of an image is performed.

[0022] Based on the image color restoration method, device, storage medium, electronic device and program product provided by the above embodiments of the present disclosure,

[0023] An image sensor using the second type of color filter array can be used as a reference image sensor, and an image sensor using the first type of color filter array can be used as a target image sensor. Since the first color restoration correction image is determined based on the first native image collected by the reference image sensor for the predetermined object, the first color statistical information of the area including the predetermined object in the first color restoration correction image can reflect the good color performance of the reference image sensor under the premise of color restoration of the image. Based on the uncorrected image, the second color statistical information of the area including the predetermined object in the uncorrected image, and the various adjustment results of the first color statistical information, a target color restoration correction matrix adapted to the target image sensor is determined, and the good color performance of the reference image sensor can be effectively applied to the determination process of the target color restoration correction matrix. In this way, it is conducive to determining a target color restoration correction matrix that can better guarantee the color performance, and the target color restoration correction matrix is ​​used for the color restoration of the target image to be corrected associated with the target image sensor, which can achieve a good color restoration effect for the target image sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of a method for color restoration of an image provided by some exemplary embodiments of the present disclosure.

[0025] Figure 2 It is a flowchart of a method for obtaining first color statistical information of a region including a predetermined object in a first color restoration corrected image provided by some exemplary embodiments of the present disclosure.

[0026] Figure 3-1 is a schematic diagram of a predetermined object in some exemplary embodiments of the present disclosure.

[0027] Figure 3-2 It is a schematic diagram of a predetermined object in some other exemplary embodiments of the present disclosure.

[0028] Figure 4 It is a flowchart of a method for adjusting first color statistical information to obtain multiple adjustment results of the first color statistical information provided by some exemplary embodiments of the present disclosure.

[0029] Figure 5 It is a flowchart of a method for determining a target color restoration correction matrix adapted to a target image sensor based on an uncorrected image, second color statistical information, and multiple adjustment results, provided by some exemplary embodiments of the present disclosure.

[0030] Figure 6 It is a flowchart of a method for determining a plurality of reference color restoration correction matrices for converting second color statistical information into a plurality of adjustment results respectively, provided by some exemplary embodiments of the present disclosure.

[0031] Figure 7 It is a flowchart of a method for determining a target color restoration correction matrix adapted to a target image sensor from multiple reference color restoration correction matrices based on multiple second color restoration correction images provided by some exemplary embodiments of the present disclosure.

[0032] Figure 8 Some other exemplary embodiments of the present disclosure provide a flowchart of a method for determining a target color restoration correction matrix adapted to a target image sensor from a plurality of reference color restoration correction matrices based on a plurality of second color restoration correction images.

[0033] Fig. 9 is a schematic diagram of a parameter setting page in some exemplary embodiments of the present disclosure.

[0034] Fig.10 It is a schematic diagram of the structure of a color restoration device for an image provided by some exemplary embodiments of the present disclosure.

[0035] Fig.11 is a schematic diagram of a third determination module in some exemplary embodiments of the present disclosure.

[0036] Fig.12 is a schematic diagram of a first determination submodule in some exemplary embodiments of the present disclosure.

[0037] Fig.13 is a schematic diagram of a second determination submodule in some exemplary embodiments of the present disclosure.

[0038] Fig.14 is a schematic diagram of a second determination submodule in some other exemplary embodiments of the present disclosure.

[0039] Fig.15 is a schematic diagram of an adjustment module in some exemplary embodiments of the present disclosure.

[0040] Fig.16 is a schematic diagram of an acquisition module in some exemplary embodiments of the present disclosure.

[0041] Fig.17 It is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0043] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0044] Application Overview

[0045] At present, image sensors are widely used. Based on the color filter array (CFA) of the image sensor, the image captured by the image sensor can be colored. The color filter array can include but is not limited to the RCCG array, RCCB array, RGGB array, etc., and accordingly, the image sensor can include but is not limited to the RCCG image sensor, RCCB image sensor, RGGB image sensor, etc.; wherein R represents red, C represents transparent transmission, G represents green, and B represents blue.

[0046] When using the ISP system to process images collected by the image sensor, it is often necessary to perform image white balance, color restoration, etc. Through image color restoration, the color deviation caused by the different responses of the human eye and the image sensor to the spectrum can be corrected to meet the human eye's perception and meet perception needs.

[0047] In the related art, image sensors using some color filter arrays cannot achieve good color reproduction effects. How to achieve good color reproduction effects for image sensors using these color filter arrays is a problem worthy of attention for those skilled in the art.

[0048] Exemplary Systems

[0049] In the related art, although a good color reproduction effect cannot be achieved for image sensors using some color filter arrays (hereinafter referred to as the first type of color filter arrays), a good color reproduction effect can be achieved for image sensors using other color filter arrays (hereinafter referred to as the second type of color filter arrays). For example, a good color reproduction effect cannot be achieved for image sensors using the RCCG array, but a good color reproduction effect can be achieved for image sensors using the RGGB array.

[0050] In view of this, in an embodiment of the present disclosure, an image sensor using a second type of color filter array can be used as a reference image sensor, and an image sensor using a first type of color filter array can be used as a target image sensor. Based on a first primary color image captured by the reference image sensor for a predetermined object, a first color restoration correction image can be determined. By adjusting the first color statistical information of an area including the predetermined object in the first color restoration correction image, a variety of adjustment results of the first color statistical information can be obtained. Based on a second native image captured by the target image sensor for the predetermined object, an uncorrected image can be determined. Based on the uncorrected image, the second color statistical information of an area including the predetermined object in the uncorrected image, and a variety of adjustment results, a target color restoration correction matrix adapted to the target image sensor can be determined. Using the target color restoration correction matrix for color restoration of a target image to be corrected associated with the target image sensor can achieve a good color restoration effect for the target image sensor.

[0051] Exemplary Methods

[0052] Figure 1 It is a flowchart of a method for color restoration of an image provided by some exemplary embodiments of the present disclosure. Figure 1 The method shown may include step 110 , step 120 , step 130 , step 140 , and step 150 .

[0053] Step 110, obtaining first color statistical information of a region including a predetermined object in a first color restoration corrected image; wherein the first color restoration corrected image is determined based on a first native image captured by a reference image sensor for the predetermined object.

[0054] Optionally, the predetermined object may be an object having multiple colors. For example, the predetermined object may be a color card with 24 colors. For another example, the predetermined object may be a rainbow umbrella.

[0055] Optionally, the reference image sensor may be an image sensor used as a reference benchmark to assist in obtaining a color restoration correction matrix adapted to other image sensors; wherein the color restoration correction matrix is ​​a matrix used for color restoration of an image, and the color restoration correction matrix may also be referred to as a CCM correction matrix. The color filter array of the reference image sensor may belong to the second type of color filter array. As an example, the color filter array of the reference image sensor may be an RGGB array, and accordingly, the reference image sensor may be an RGGB image sensor, and the first native image acquired by the reference image sensor for a predetermined object may be an RGGB image. According to the color restoration scheme in the related art, the RGGB image as the first native image may be color restored to obtain a first color restoration corrected image, or the RGGB image as the first native image may be first converted into an image suitable for user observation, and then the converted image may be color restored to obtain a first color restoration corrected image. It should be noted that the native image involved in the embodiments of the present disclosure may also be referred to as Raw data; the image suitable for user observation involved in the embodiments of the present disclosure may be, for example, an RGB image.

[0056] Optionally, the first color restoration corrected image may be stored in a predetermined storage area in advance. Accordingly, in step 110, the first color restoration corrected image in the predetermined storage area may be read, and color statistics may be performed on the area including the predetermined object in the read first color restoration corrected image to obtain first color statistical information. The area including the predetermined object in the first color restoration corrected image may include multiple color blocks, and the first color statistical information may include color statistical information corresponding to the multiple color blocks respectively; wherein the color statistical information corresponding to any color block is used to characterize the color of the color block.

[0057] Of course, the method of obtaining the first color statistical information in step 110 is not limited to this. For example, color statistics can be performed in advance on the area including the predetermined object in the first color restoration and correction image to obtain the first color statistical information, and the first color statistical information can be stored in a predetermined storage area in advance. Accordingly, in step 110, the first color statistical information in the predetermined storage area can be read to achieve the acquisition of the first color statistical information.

[0058] Step 120 , determining an uncorrected image based on a second native image captured by a target image sensor for a predetermined object; wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor.

[0059] Optionally, the target image sensor may be an image sensor for which a color restoration correction matrix that is adapted needs to be determined. The color filter array of the target image sensor may belong to the first type of color filter array. The color filter array of the target image sensor may be different from the RGGB array, that is, the color filter array of the target image sensor may be a non-RGGB array, and accordingly, the target image sensor may be a non-RGGB image sensor. As an example, the color filter array of the target image sensor may be an RCCG array or an RCCB array, and accordingly, the target image sensor may be an RCCG image sensor or an RCCB image sensor, and the second native image captured by the target image sensor for a predetermined object may be an RCCG image or an RCCB image captured for a predetermined object. In step 120, the second native image may be used as an uncorrected image, or the second native image may be converted into an image suitable for user observation, and then the converted image may be used as an uncorrected image, or the second native image may be white balanced, and the second native image after white balance may be used as an uncorrected image.

[0060] Step 130: determining second color statistical information of a region including a predetermined object in the uncorrected image.

[0061] Optionally, color statistics may be performed on the region including the predetermined object in the uncorrected image to obtain second color statistical information. The region including the predetermined object in the uncorrected image may include multiple color blocks, and the second color statistical information may include color statistical information corresponding to the multiple color blocks respectively; wherein the color statistical information corresponding to any color block is used to characterize the color of the color block.

[0062] Step 140: adjust the first color statistical information to obtain a plurality of adjustment results of the first color statistical information.

[0063] Optionally, a plurality of adjustment results of the first color statistical information may be obtained by applying random perturbations to the first color statistical information; wherein different adjustment results may correspond to different random perturbations.

[0064] Step 150, based on the uncorrected image, the second color statistical information and the multiple adjustment results, determine a target color restoration correction matrix adapted to the target image sensor, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0065] Optionally, the conversion relationship between the second color statistical information and the multiple adjustment results can be determined, and based on the uncorrected image and the determined conversion relationship, the target color restoration correction matrix adapted to the target image sensor is determined. As an example, the target color restoration correction matrix can be a 3*3 matrix, that is, the number of rows and the number of columns of the target color restoration correction matrix can both be 3. The target color restoration correction matrix can be used for color restoration of the target image to be corrected associated with the target image sensor. The association of the target image to be corrected with the target image sensor can be understood as the target image to be corrected being determined based on the native image captured by the target image sensor (the native image may be different from the second native image). For example, the native image captured by the target image sensor can be used as the target image to be corrected. For another example, the native image captured by the target image sensor can be converted into an image suitable for user observation, and the converted image can be used as the target image to be corrected. For another example, the native image captured by the target image sensor can be white balanced, and the native image after white balance can be used as the target image to be corrected.

[0066] In the embodiments of the present disclosure, an image sensor using the second type of color filter array can be used as a reference image sensor, and an image sensor using the first type of color filter array can be used as a target image sensor. Since the first color restoration correction image is determined based on the first native image collected by the reference image sensor for the predetermined object, the first color statistical information of the area including the predetermined object in the first color restoration correction image can be reflected in the good color performance of the reference image sensor under the premise of color restoration of the image. Based on the uncorrected image, the second color statistical information of the area including the predetermined object in the uncorrected image, and the various adjustment results of the first color statistical information, a target color restoration correction matrix adapted to the target image sensor is determined, and the good color performance of the reference image sensor can be effectively applied to the determination process of the target color restoration correction matrix. In this way, it is conducive to determining a target color restoration correction matrix that can better guarantee the color performance, and the target color restoration correction matrix is ​​used for the color restoration of the target image to be corrected associated with the target image sensor, which can achieve a good color restoration effect for the target image sensor.

[0067] Figure 2 It is a flowchart of a method for obtaining first color statistical information of a region including a predetermined object in a first color restoration corrected image provided by some exemplary embodiments of the present disclosure. Figure 2 The illustrated method may include step 210 , step 220 , and step 230 .

[0068] Step 210: Determine the image type of the first color restoration corrected image.

[0069] Optionally, the image type of the first color restoration corrected image may have two possible situations, namely, a raw type and a non-raw type. For example, if the first color restoration corrected image is an RGB image, the image type of the first color restoration corrected image may be a non-raw type. For another example, if the first color restoration corrected image is an RGGB image, the image type of the first color restoration corrected image may be a raw type.

[0070] Step 220: Determine a color statistics method that is compatible with the image type of the first color restoration corrected image.

[0071] Step 230: Perform color statistics on a region including a predetermined object in the first color restoration and correction image according to the determined color statistics method to obtain first color statistics information.

[0072] Optionally, the area including the predetermined object in the first color restoration corrected image may include multiple color blocks, for example, K color blocks, each of the K color blocks may include multiple pixel points; wherein K may be an integer greater than or equal to 2. The number of pixel points included in different color blocks may be the same or different. As an example, Figure 3-1 , Figure 3-2 As shown, the predetermined object may be a 24-color color card, then the K color blocks may be 24 color blocks, and the 24 color blocks may be arranged in 4 rows and 6 columns. In addition, each of the 24 color blocks may include p rows and q columns of pixels; wherein p and q may be integers greater than or equal to 2.

[0073] If the image type of the first color restoration and correction image is a non-raw type, the color statistics method adapted to the image type of the first color restoration and correction image may be the first color statistics method. Then, color statistics may be performed on the K color blocks in the first color restoration and correction image according to the first color statistics method to obtain color statistics information corresponding to the K color blocks, and the color statistics information corresponding to the K color blocks may constitute the first color statistics information. The following takes the case where the first color restoration and correction image is an RGB image as an example to introduce the process of obtaining the first color statistics information based on the first color statistics method.

[0074] Since the first color restoration and correction image is an RGB image, each pixel point in the first color restoration and correction image has a pixel value corresponding to the R channel, a pixel value corresponding to the G channel, and a pixel value corresponding to the B channel. Then, for each of the K color blocks in the first color restoration and correction image, the first mean of the pixel values ​​of all the pixels in the color block corresponding to the R channel, the second mean of the pixel values ​​of all the pixels in the color block corresponding to the G channel, and the third mean of the pixel values ​​of all the pixels in the color block corresponding to the B channel can be calculated, and the calculated first mean, second mean, and third mean can constitute the color statistical information corresponding to the color block. In this way, K color statistical information corresponding to the K color blocks in the first color restoration and correction image can be obtained, and the K color statistical information can constitute the first color statistical information.

[0075] If the image type of the first color restoration corrected image is a raw type, the color statistics method adapted to the image type of the first color restoration corrected image may be a second color statistics method. Then, color statistics may be performed on the K color blocks in the first color restoration corrected image according to the second color statistics method to obtain color statistics information corresponding to the K color blocks, and the color statistics information corresponding to the K color blocks may constitute the first color statistics information. The following takes the case where the first color restoration corrected image is an RGGB image as an example to introduce the process of obtaining the first color statistics information based on the first color statistics method.

[0076] Since the first color restoration corrected image is an RGGB image, each pixel in the first color restoration corrected image has only one pixel value, and the pixel value either corresponds to the R channel, the G channel, or the B channel. Then, for each of the K color blocks in the first color restoration corrected image, the fourth mean of the pixel values ​​of all the pixels corresponding to the R channel in the color block, the fifth mean of the pixel values ​​of all the pixels corresponding to the G channel in the color block, and the sixth mean of the pixel values ​​of all the pixels corresponding to the B channel in the color block can be calculated, and the calculated fourth mean, fifth mean, and sixth mean can constitute the color statistical information corresponding to the color block. In this way, K color statistical information corresponding to the K color blocks in the first color restoration corrected image can be obtained, and the K color statistical information can constitute the first color statistical information.

[0077] In the embodiments of the present disclosure, according to the image type of the first color restoration and corrected image, an adaptive color statistics method can be used to perform color statistics on the area including the predetermined object in the first color restoration and corrected image, which is beneficial to ensure the rationality and reliability of the obtained first color statistical information.

[0078] In some embodiments, the uncorrected image may be an RGB image, that is, the image type of the uncorrected image may be a non-raw type. In addition, the uncorrected image may include K color blocks. Then, in step 130, color statistics may be performed on the K color blocks in the uncorrected image according to the second color statistics method to obtain color statistical information corresponding to the K color blocks, and the color statistical information corresponding to the K color blocks may constitute the second color statistical information.

[0079] In some other embodiments, the uncorrected image may be an RCCG image, that is, the image type of the uncorrected image may be a raw type. In addition, the uncorrected image may include K color blocks. Then, in step 130, color statistics may be performed on the K color blocks in the uncorrected image according to the first color statistics method to obtain color statistical information corresponding to the K color blocks, and the color statistical information corresponding to the K color blocks may constitute the second color statistical information. It should be noted that in this case, Figure 2 The difference in the process of obtaining the first color statistical information based on the first color statistical method in the illustrated embodiment is mainly that here, for each of the K color blocks in the uncorrected image, it is necessary to calculate the mean of the pixel values ​​of the R channel corresponding to all the pixels in the color block, the mean of the pixel values ​​of the C channel corresponding to all the pixels in the color block, and the pixel values ​​of the G channel corresponding to all the pixels in the color block.

[0080] Figure 4 It is a flowchart of a method for adjusting first color statistical information to obtain multiple adjustment results of the first color statistical information provided by some exemplary embodiments of the present disclosure. Figure 4 The method shown may include step 410 , step 420 , step 430 , and step 440 .

[0081] Step 410: Based on the first color statistical information, determine that each of the plurality of color blocks in the region including the predetermined object corresponds to N color statistical values ​​of the N channels included in the first color restoration corrected image.

[0082] Optionally, the multiple color blocks may be represented as K color blocks, and the first color statistical information may include N color statistical values ​​corresponding to N channels included in the first color restoration corrected image for each color block in the K color blocks. Then, the N color statistical values ​​corresponding to N channels included in the first color restoration corrected image for each color block in the K color blocks may be extracted from the first color statistical information.

[0083] In one example, the N channels may be 3 channels, namely, an R channel, a G channel, and a B channel. If the first color restoration and correction image is an RGB image, that is, the image type of the first color restoration and correction image is a non-raw type, then corresponding to each of the K color blocks in the first color restoration and correction image, a corresponding first mean, a corresponding second mean, and a corresponding third mean may be extracted from the first color statistical information, and the extracted first mean may be used as the color statistical value of the color block corresponding to the R channel, the extracted second mean may be used as the color statistical value of the color block corresponding to the G channel, and the extracted third mean may be used as the color statistical value of the color block corresponding to the B channel. If the first color restoration and corrected image is an RGGB image, that is, the image type of the first color restoration and corrected image is a raw type, then corresponding to each of the K color blocks in the first color restoration and corrected image, a corresponding fourth mean, a corresponding fifth mean and a corresponding sixth mean can be extracted from the first color statistical information, and the extracted fourth mean can be used as the color statistical value of the color block corresponding to the R channel, the extracted fifth mean can be used as the color statistical value of the color block corresponding to the G channel, and the extracted sixth mean can be used as the color statistical value of the color block corresponding to the B channel.

[0084] Step 420 : Determine N random deviation values ​​corresponding to N channels for each color block in the plurality of color blocks within a preset random deviation value range.

[0085] Optionally, the upper limit of the preset random deviation value range can be expressed as r2, and the lower limit can be expressed as r1, and the preset random deviation value range can be expressed as [r1, r2]. For each of the K color blocks, the random deviation values ​​corresponding to the N channels can be randomly screened in [r1, r2] to obtain N random deviation values. For example, for each of the K color blocks, the random deviation values ​​corresponding to the R channel, the G channel, and the B channel can be randomly screened in [r1, r2] to obtain 3 random deviation values ​​corresponding to the 3 channels of the color block.

[0086] Step 430 : Superimpose the N color statistics corresponding to each color block in the plurality of color blocks with the corresponding random deviation value to obtain N color superposition values ​​corresponding to each color block in the plurality of color blocks.

[0087] Optionally, each color statistic value and the corresponding random deviation value may be superimposed by summing. For example, for each color block among the K color blocks, the color statistic value corresponding to the R channel of the color block and the random deviation value corresponding to the R channel of the color block may be summed, and the summed result may be used as the color superposition value corresponding to the R channel of the color block. In this way, the color superposition value corresponding to the G channel of the color block and the color superposition value corresponding to the B channel of the color block may also be obtained.

[0088] Step 440 : determining one of a plurality of adjustment results of the first color statistical information based on the N color superposition values ​​corresponding to each color block in the plurality of color blocks.

[0089] Optionally, for each color block among the K color blocks, the N color superposition values ​​corresponding to the color block can constitute the adjusted color statistical information corresponding to the color block. In this way, the adjusted color statistical information corresponding to the K color blocks can be obtained, and the adjusted color statistical information corresponding to the K color blocks can constitute an adjustment result of the first color statistical information. The method for obtaining other adjustment results of the first color statistical information is similar to this, and will not be repeated here.

[0090] In the embodiments of the present disclosure, color statistical values ​​can be extracted from the first color statistical information, random deviation values ​​can be determined within a preset random deviation value range, and the color statistical values ​​and the random deviation values ​​can be superimposed. This is equivalent to randomly changing the color statistical values ​​in the first color statistical information, thereby efficiently and quickly adjusting the first color statistical information by applying random disturbances.

[0091] Figure 5 It is a flowchart of a method for determining a target color restoration correction matrix adapted to a target image sensor based on an uncorrected image, second color statistical information, and multiple adjustment results, provided by some exemplary embodiments of the present disclosure. Figure 5 The method shown may include step 510 , step 520 , and step 530 .

[0092] Step 510: Determine a plurality of reference color restoration correction matrices for converting the second color statistical information into a plurality of adjustment results respectively.

[0093] Optionally, the multiple adjustment results may be represented as D adjustment results, wherein D may be an integer greater than or equal to 2. For each of the D adjustment results, a reference color restoration correction matrix for converting the second color statistical information to the adjustment result may be determined, and the reference color restoration correction matrix may be used to characterize the conversion relationship between the second color statistical information and the adjustment result. In this way, D reference color restoration correction matrices corresponding to the D adjustment results may be obtained. The D reference color restoration correction matrices may all be 3*3 matrices.

[0094] In some optional embodiments of the present disclosure, Figure 6 As shown, step 510 may include step 610 and step 620 .

[0095] Step 610: Determine weights corresponding to a plurality of color blocks in a region including a predetermined object.

[0096] Optionally, multiple color blocks can be represented as K color blocks. In step 610, the weights corresponding to the K color blocks can be determined according to preset rules. As an example, the predetermined object can be a 24-color color card, and the preset rules can include but are not limited to the following rules: (1) The weight corresponding to the main color block is higher than the weights corresponding to other color blocks. The main color block can include, for example, red, green, yellow, white, blue and other color blocks; (2) According to the actual application scenario, the weight corresponding to the color block with the color to be paid attention to is higher than the weights corresponding to other color blocks. In the scene of character shooting, the color to be paid attention to can be, for example, skin color. In the scene of scenery shooting, the color to be paid attention to can be, for example, green (the color of the grass), blue (the color of the sky), etc.

[0097] Step 620 : Based on the weights corresponding to the plurality of color blocks and the regression algorithm, a plurality of reference color restoration correction matrices for converting the second color statistical information into a plurality of adjustment results are determined.

[0098] Optionally, based on the weights corresponding to the K color blocks and the regression algorithm, D reference color restoration correction matrices corresponding to the D adjustment results can be determined. The regression algorithm involved in the embodiments of the present disclosure can be either a linear regression algorithm or a nonlinear regression algorithm. As an example, the regression algorithm involved in the embodiments of the present disclosure can be a least squares method. Accordingly, the reference color restoration correction matrix corresponding to any adjustment result can be obtained by minimizing the objective function value. The objective function value can be expressed by the following formula:

[0099]

[0100] Among them, k i represents the weight corresponding to the i-th color block among the K color blocks, x i represents the adjusted color statistics corresponding to the i-th color block in the adjustment result, Mat represents the reference color restoration correction matrix corresponding to the adjustment result, y i Represents the color statistical information corresponding to the i-th color block in the second color statistical information.

[0101] Figure 6 In the illustrated embodiment, the second color statistical information can be used as the original training value of the regression algorithm, and the various adjustment results can be used as the training target values ​​of the regression algorithm. Combined with the weights corresponding to the multiple color blocks, the reference color restoration correction matrices corresponding to the various adjustment results can be obtained efficiently and quickly through the operation of the regression algorithm.

[0102] Of course, the implementation method of step 510 is not limited to this. For example, for each of the multiple adjustment results, a matrix for converting the adjustment result to the second color statistical information can be determined first, and then the inverse matrix of the matrix can be calculated, and the obtained inverse matrix can be used as a reference color restoration correction matrix corresponding to the adjustment result.

[0103] Step 520 , performing color restoration on the uncorrected image using a plurality of reference color restoration correction matrices to obtain a plurality of second color restoration corrected images.

[0104] Optionally, the plurality of reference color restoration correction matrices may be represented as D reference color restoration correction matrices. For each of the D reference color restoration correction matrices, the reference color restoration correction matrix may be used to perform color restoration on the uncorrected image to obtain a corresponding second color restoration corrected image. In this way, D second color restoration corrected images corresponding one-to-one to the D reference color restoration correction matrices may be obtained.

[0105] In one example, the uncorrected image may be an RGB image, and the pixel value corresponding to the R channel, the pixel value corresponding to the G channel, and the pixel value corresponding to the B channel of any pixel point in the uncorrected image are represented as u1, u2, and u3, respectively. A reference color restoration correction matrix is ​​represented as Mat, and the pixel value corresponding to the R channel, the pixel value corresponding to the G channel, and the pixel value corresponding to the B channel of the corresponding pixel point in the second color restoration correction image corresponding to the reference color restoration correction matrix are represented as v1, v2, and v3, respectively. Then:

[0106]

[0107] Step 530 : Based on the plurality of second color restoration and correction images, determine a target color restoration and correction matrix adapted to the target image sensor from a plurality of reference color restoration and correction matrices.

[0108] Optionally, based on the multiple second color restoration correction images, the advantages and disadvantages of multiple reference color restoration correction matrices can be evaluated, and on this basis, a target color restoration correction matrix adapted to the target image sensor can be determined from the multiple reference color restoration correction matrices.

[0109] In some optional embodiments of the present disclosure, Figure 7 As shown, step 530 may include step 710 , step 720 , step 730 and step 740 .

[0110] Step 710: determine third color statistical information corresponding to the regions including the predetermined object in each second color restoration-corrected image, and determine a color deviation between each third color statistical information and the first color statistical information.

[0111] Optionally, the image types of the D second color restoration and correction images may all be non-raw types, then, for each of the D second color restoration and correction images, color statistics may be performed on the area including the predetermined object in the second color restoration and correction image according to the second color statistics method to obtain corresponding third color statistical information, thereby obtaining D third color statistical information corresponding to the D second color restoration and correction images. Of course, the image types of the D second color restoration and correction images may all be raw types, then, based on the first color statistics method, D third color statistical information may be obtained. For each of the D third color statistical information, the color deviation between the third color statistical information and the first color statistical information may be determined by subtraction.

[0112] In one example, each of the D third color statistics may include color statistics corresponding to K color blocks, and the color statistics corresponding to each color block may include color statistics corresponding to the R channel, the G channel, and the B channel, that is, the third color statistics may include 3K color statistics. Similarly, the first color statistics also include 3K color statistics. Then, the 3K color statistics in the third color statistics may be subtracted from the 3K color statistics in the first color statistics to obtain 3K color difference values, wherein the number of color difference values ​​corresponding to the R channel is K, the number of color difference values ​​corresponding to the G channel is K, and the number of color difference values ​​corresponding to the B channel is also K. The 3K color statistics may be used to represent the color deviation between the third color statistics and the first color statistics. If the predetermined object is a 24-color color card, the 3K color differences may be a tensor with a width of 6, a height of 4, and a channel number of 3, which may be used to represent the color deviation between the third color statistics and the first color statistics.

[0113] Step 720: Determine the signal-to-noise ratios corresponding to the second color restoration-corrected images.

[0114] Optionally, for each of the D second color restoration corrected images, a signal-to-noise ratio (SNR) of the second color restoration corrected image may be determined, thereby obtaining D signal-to-noise ratios corresponding to the D second color restoration corrected images. Generally speaking, the signal-to-noise ratio refers to the ratio of the strength of a useful signal to the strength of an interference signal.

[0115] Step 730 : based on the color deviations and / or the signal-to-noise ratios respectively corresponding to the second color restoration and correction images, select part of the reference color restoration and correction matrices from a plurality of reference color restoration and correction matrices.

[0116] Step 740 : Based on the second color restoration correction image corresponding to the screened partial reference color restoration correction matrix, determine a target color restoration correction matrix adapted to the target image sensor from the screened partial reference color restoration correction matrix.

[0117] It should be noted that, by executing step 710 and step 720, D color deviations corresponding to the D reference color restoration correction matrices and D signal-to-noise ratios corresponding to the D reference color restoration correction matrices can be obtained. In step 730, part of the reference color restoration correction matrices can be screened from the D reference color restoration correction matrices based on the D color deviations and / or the D signal-to-noise ratios.

[0118] For example, a reference color restoration correction matrix whose corresponding color deviation is less than a preset deviation threshold can be screened from D reference color restoration correction matrices. Here, the preset deviation threshold may include a first deviation threshold corresponding to the R channel, a second deviation threshold corresponding to the G channel, and a third deviation threshold corresponding to the B channel. For the color deviation corresponding to any reference color restoration correction matrix, the color deviation may include 3K color difference values. If the maximum value among the K color difference values ​​corresponding to the R channel is less than the first deviation threshold, the maximum value among the K color difference values ​​corresponding to the G channel is less than the second deviation threshold, and the maximum value among the K color difference values ​​corresponding to the B channel is less than the third deviation threshold, it can be determined that the color deviation is less than the preset deviation threshold.

[0119] For another example, a reference color restoration correction matrix having a corresponding signal-to-noise ratio greater than a preset signal-to-noise ratio threshold may be selected from D reference color restoration correction matrices.

[0120] Alternatively, a reference color restoration correction matrix whose corresponding color deviation is smaller than a preset deviation threshold and whose corresponding signal-to-noise ratio is larger than a preset signal-to-noise ratio threshold may be selected from the D reference color restoration correction matrices.

[0121] Assuming that in step 730, only one reference color restoration correction matrix is ​​screened out from the D reference color restoration correction matrices, then in step 740, the reference color restoration correction matrix may be used as the target color restoration correction matrix.

[0122] Assuming that in step 730, more than one reference color restoration correction matrix is ​​screened out from D reference color restoration correction matrices, then in step 740, a reference color restoration correction matrix can be further screened out from the more than one reference color restoration correction matrix, and the reference color restoration correction matrix can be used as the target color restoration correction matrix.

[0123] Figure 7 In the illustrated embodiment, for each second color restoration correction image among the plurality of second color restoration correction images, the quality of the reference color restoration correction matrix corresponding to the second color restoration correction image can be determined based on the color deviation and / or the corresponding signal-to-noise ratio corresponding to the second color restoration correction image. For example, the smaller the color deviation corresponding to the second color restoration correction image, and / or the larger the signal-to-noise ratio corresponding to the second color restoration correction image, the better the reference color restoration correction matrix corresponding to the second color restoration correction image can be determined. In this way, an effective reference can be provided for determining the target color restoration correction matrix, which is conducive to ensuring the rationality and reliability of the target color restoration correction matrix finally determined.

[0124] In some other optional embodiments of the present disclosure, Figure 8 As shown, step 530 may include step 810 , step 820 , and step 830 .

[0125] Step 810 , performing predetermined machine vision tasks on the plurality of second color restoration and correction images respectively to obtain a plurality of task execution results.

[0126] Optionally, the predetermined machine vision task may be a target detection task, and the target detection task may be a task for detecting a predetermined object in an image. In step 810, a pre-trained target detection model may be used to perform target detection on the D second color restoration and correction images, respectively, to obtain D detection results corresponding to the D second color restoration and correction images; wherein each of the D detection results may be used as a task execution result, and each task execution result may include a two-dimensional detection frame and a confidence level corresponding to the two-dimensional detection frame.

[0127] Step 820, respectively evaluate the execution results of the multiple tasks to obtain multiple evaluation results.

[0128] It should be noted that D task execution results can be obtained by executing step 810. For each of the D task execution results, a confidence level can be extracted from the task execution result, and the extracted confidence level can be used as the evaluation result corresponding to the task execution result, or the extracted confidence level can be converted to a set evaluation score range, and the obtained evaluation score can be used as the evaluation result corresponding to the task execution result.

[0129] In some embodiments, for each of the D task execution results, the task execution result can also be displayed on the corresponding second color restoration and corrected image to facilitate the user to observe and score the task execution result, and the score value can be used as the evaluation result corresponding to the task execution result.

[0130] Step 830 , based on the multiple evaluation results, determine a target color restoration correction matrix adapted to the target image sensor from a plurality of reference color restoration correction matrices.

[0131] It should be noted that by executing step 820, D evaluation results corresponding to the D reference color restoration correction matrices can be obtained. Optionally, the D evaluation results can all be in numerical form, and the larger the numerical value, the better the corresponding reference color restoration correction matrix. Then, in step 830, the evaluation result with the largest numerical value can be selected from the D evaluation results, and the reference color restoration correction matrix corresponding to the selected evaluation result can be used as the target color restoration correction matrix.

[0132] In some embodiments, the D evaluation results may all be in numerical form, and the smaller the numerical value, the better the corresponding reference color restoration correction matrix. Then, in step 830, the evaluation result with the smallest numerical value may be selected from the D evaluation results, and the reference color restoration correction matrix corresponding to the selected evaluation result may be used as the target color restoration correction matrix.

[0133] Figure 8 In the illustrated embodiment, for each of the plurality of second color restoration correction images, the quality of the reference color restoration correction matrix corresponding to the second color restoration correction image can be determined based on the task execution result corresponding to the second color restoration correction image. In this way, an effective reference can be provided for determining the target color restoration correction matrix, which is conducive to ensuring the rationality and reliability of the target color restoration correction matrix finally determined.

[0134] Optionally, Figure 7 The embodiment shown and Figure 8 The illustrated embodiments may be used in combination. For example, if in step 730, more than one reference color restoration correction matrix is ​​selected from the D reference color restoration correction matrices, since the more than one reference color restoration correction matrix corresponds to more than one second color restoration correction image, then, the predetermined machine vision task may be performed on the more than one second color restoration correction image, and more than one task execution result may be obtained. After that, the more than one task execution result may be evaluated, and based on each evaluation result obtained, the target color restoration correction matrix may be determined from the more than one reference color restoration correction matrix.

[0135] In the embodiments of the present disclosure, by determining a plurality of reference color restoration correction matrices for converting the second color statistical information into a plurality of adjustment results, the plurality of reference color restoration correction matrices can be used to perform color restoration of the uncorrected image, respectively, to obtain a plurality of second color restoration corrected images. Based on the plurality of second color restoration corrected images, the advantages and disadvantages of the plurality of reference color restoration correction matrices can be effectively evaluated. In this way, an effective reference can be provided for the determination of the target color restoration correction matrix, which is conducive to ensuring the rationality and reliability of the target color restoration correction matrix finally determined. For example, based on the target color restoration correction matrix, the color performance of the target image sensor can be made as consistent as possible with the color performance of the reference image sensor, thereby ensuring the color restoration effect for the target image sensor, and also making the image signal-to-noise ratio of the target image to be corrected after color restoration perform well. In this way, the embodiments of the present disclosure can not only conform to the perception of the human eye and meet the perception requirements, but also take into account both the color performance and the noise performance.

[0136] In some optional examples, the reference image sensor may be an RGGB image sensor, and the target image sensor may be an RCCG image sensor.

[0137] Then, the RGGB image sensor can be called to collect a native image of a 24-color color card (such as the first native image mentioned above) at a preset color temperature. Based on the first native image, color restoration can be performed to obtain a first color restoration corrected image. By performing color statistics on the area of ​​the 24-color color card in the first color restoration corrected image, first color statistical information can be obtained. By applying random perturbations to the first color statistical information, various adjustment results of the first color statistical information can be obtained.

[0138] In addition, the RCCG image sensor can be called to collect an image of a 24-color color card (equivalent to the second native image mentioned above) at a preset color temperature. Based on the second native image, an uncorrected image can be determined, and second color statistical information can be obtained by performing color statistics on an area including a predetermined object in the uncorrected image.

[0139] Optionally, weights corresponding to multiple color blocks may also be determined. Fig. 9 In the parameter setting page shown, enter 24 weights corresponding to the 24 color blocks (see Fig. 9 (large rectangular box in the upper left center).

[0140] In addition, the second color statistical information can be used as the original training value of the regression algorithm, and the multiple adjustment results can be used as the training target values ​​of the regression algorithm, and then combined with the weights corresponding to the multiple color blocks, through the operation of the regression algorithm, multiple reference color restoration correction matrices can be obtained. Using multiple reference color restoration correction matrices, the color of the uncorrected image is restored to obtain multiple second color restoration corrected images.

[0141] Afterwards, the color deviations and signal-to-noise ratios respectively corresponding to the plurality of second color restoration correction images may be determined, and based on the determined color deviations and the determined signal-to-noise ratios, a target color restoration correction matrix may be determined from a plurality of reference color restoration correction matrices. Alternatively, the target color restoration correction matrix may be determined from a plurality of reference color restoration correction matrices by combining human vision and machine vision evaluation. The target color restoration correction matrix may be used for color restoration of a target image to be corrected associated with a target image sensor.

[0142] In summary, by adopting the embodiments of the present disclosure, it is possible to achieve a good color reproduction effect for an image sensor using the first color filter array, which can not only conform to the perception of the human eye and meet the perception requirements, but also take into account both color performance and noise performance.

[0143] Exemplary Devices

[0144] Fig.10 It is a schematic diagram of the structure of a color restoration device for an image provided by some exemplary embodiments of the present disclosure. Fig.10 The apparatus shown may include:

[0145] The acquisition module 1010 is used to acquire first color statistical information of a region including a predetermined object in a first color restoration corrected image; wherein the first color restoration corrected image is determined based on a first native image captured by a reference image sensor for the predetermined object;

[0146] A first determination module 1020 is configured to determine an uncorrected image based on a second native image acquired by a target image sensor for a predetermined object, wherein a color filter array of the target image sensor is different from a color filter array of a reference image sensor;

[0147] A second determination module 1030, configured to determine second color statistical information of a region including a predetermined object in the uncorrected image determined by the first determination module 1020;

[0148] An adjustment module 1040 is used to adjust the first color statistical information obtained by the obtaining module 1010 to obtain multiple adjustment results of the first color statistical information;

[0149] The third determination module 1050 is used to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determination module 1020, the second color statistical information determined by the second determination module 1030, and the multiple adjustment results obtained by the adjustment module 1040, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

[0150] In some optional examples, such as Fig.11 As shown, the third determination module 1050 includes:

[0151] A first determination submodule 1110 is used to determine a plurality of reference color restoration correction matrices for converting the second color statistical information determined by the second determination module 1030 to a plurality of adjustment results obtained by the adjustment module 1040 respectively;

[0152] The color restoration submodule 1120 is used to perform color restoration on the uncorrected image determined by the first determination submodule 1110 using the multiple reference color restoration correction matrices determined by the first determination submodule 1110 to obtain multiple second color restoration corrected images;

[0153] The second determination submodule 1130 is used to determine a target color restoration correction matrix adapted to the target image sensor from the multiple reference color restoration correction matrices determined by the first determination submodule 1110 based on the multiple second color restoration corrected images obtained by the color restoration submodule 1120.

[0154] In some optional examples, such as Fig.12 As shown, the first determination submodule 1110 includes:

[0155] A first determining unit 1210 is used to determine weights corresponding to a plurality of color blocks in a region including a predetermined object;

[0156] The second determination unit 1220 is used to determine multiple reference color restoration correction matrices for converting the second color statistical information determined by the second determination module 1030 to multiple adjustment results obtained by the adjustment module 1040 based on the weights corresponding to the multiple color blocks determined by the first determination unit 1210 and the regression algorithm.

[0157] In some optional examples, such as Fig.13 As shown, the second determination submodule 1130 includes:

[0158] The third determination unit 1310 is used to determine the third color statistical information corresponding to the regions including the predetermined object in each second color restoration corrected image obtained by the color restoration submodule 1120, and determine the color deviation between each third color statistical information and the first color statistical information;

[0159] The fourth determining unit 1320 is used to determine the signal-to-noise ratios corresponding to the second color restoration corrected images obtained by the color restoration submodule 1120;

[0160] A screening unit 1330 is used to screen part of the reference color restoration correction matrices from the multiple reference color restoration correction matrices obtained by the color restoration submodule 1120 based on the color deviations determined by the third determination unit 1310 and / or the signal-to-noise ratios determined by the fourth determination unit 1320;

[0161] The fifth determination unit 1340 is used to determine a target color restoration correction matrix adapted to the target image sensor from the partial reference color restoration correction matrix filtered by the screening unit 1330 based on the second color restoration correction image corresponding to the partial reference color restoration correction matrix filtered by the screening unit 1330.

[0162] In some optional examples, such as Fig.14 As shown, the second determination submodule 1130 includes:

[0163] The execution unit 1410 is used to execute a predetermined machine vision task on each of the plurality of second color restoration and correction images obtained by the color restoration submodule 1120 to obtain a plurality of task execution results;

[0164] The evaluation unit 1420 is used to evaluate the multiple task execution results obtained by the execution unit 1410 respectively to obtain multiple evaluation results;

[0165] The sixth determination unit 1430 is configured to determine a target color restoration correction matrix adapted to the target image sensor from the multiple reference color restoration correction matrices determined by the first determination submodule 1110 based on the multiple evaluation results obtained by the execution unit 1410 .

[0166] In some optional examples, such as Fig.15 As shown, the adjustment module 1040 includes:

[0167] A third determining submodule 1510 is used to determine, based on the first color statistical information acquired by the acquiring module 1010, N color statistical values ​​corresponding to N channels included in the first color restoration corrected image for each of the multiple color blocks in the area including the predetermined object;

[0168] The fourth determination submodule 1520 is used to determine N random deviation values ​​corresponding to N channels of each color block of the plurality of color blocks within a preset random deviation value range;

[0169] The superposition submodule 1530 is used to superimpose the N color statistical values ​​corresponding to each color block in the multiple color blocks determined by the third determination submodule 1510 with the corresponding random deviation values ​​determined by the fourth determination submodule 1520, respectively, to obtain N color superposition values ​​corresponding to each color block in the multiple color blocks;

[0170] The fifth determination submodule 1540 is used to determine one of the multiple adjustment results of the first color statistical information obtained by the acquisition module 1010 based on the N color superposition values ​​corresponding to each color block in the multiple color blocks obtained by the superposition submodule 1530 .

[0171] In some optional examples, such as Fig.16 As shown, the acquisition module includes:

[0172] A sixth determination submodule 1610 is used to determine the image type of the first color restoration corrected image;

[0173] A seventh determination submodule 1620 is used to determine a color statistics method adapted to the image type determined by the sixth determination submodule 1610;

[0174] The color statistics submodule 1630 is used to perform color statistics on the area including the predetermined object in the first color restoration and correction image according to the color statistics method determined by the seventh determination submodule 1620 to obtain first color statistics information.

[0175] In the device of the present disclosure, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.

[0176] Exemplary Electronic Devices

[0177] Fig.17 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 1700 includes one or more processors 1710 and a memory 1720 .

[0178] The processor 1710 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1700 to perform desired functions.

[0179] The memory 1720 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1710 may execute one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0180] In one example, the electronic device 1700 may further include: an input device 1730 and an output device 1740 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0181] The input device 1730 may also include, for example, a keyboard, a mouse, etc.

[0182] The output device 1740 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0183] Of course, to simplify, Fig.17 Only some of the components related to the present disclosure in the electronic device 1700 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application scenarios, the electronic device 1700 may also include any other appropriate components.

[0184] Exemplary computer program products and computer-readable storage media

[0185] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0186] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0187] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0188] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0189] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. The specific details disclosed above are only for the purpose of illustration and ease of understanding, not limitation. The above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0190] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for color restoration of an image, comprising: Acquire first color statistical information of a region including a predetermined object in a first color restoration corrected image; wherein the first color restoration corrected image is determined based on a first native image acquired by a reference image sensor for the predetermined object; Determining an uncorrected image based on a second native image captured by a target image sensor for the predetermined object; wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor; determining second color statistics of a region of the uncorrected image including the predetermined object; Adjusting the first color statistical information to obtain multiple adjustment results of the first color statistical information; Based on the uncorrected image, the second color statistical information and the multiple adjustment results, a target color restoration correction matrix adapted to the target image sensor is determined, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

2. The method according to claim 1, wherein: The step of determining a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image, the second color statistical information, and the plurality of adjustment results comprises: Determining a plurality of reference color restoration correction matrices for converting the second color statistical information into a plurality of adjustment results respectively; Using the plurality of reference color restoration correction matrices respectively, color restoration is performed on the uncorrected image to obtain a plurality of second color restoration corrected images; Based on the plurality of the second color restoration correction images, a target color restoration correction matrix adapted to the target image sensor is determined from the plurality of the reference color restoration correction matrices.

3. The method according to claim 2, wherein: The determining of a plurality of reference color restoration correction matrices for converting the second color statistical information into a plurality of adjustment results respectively comprises: Determine weights corresponding to a plurality of color blocks in a region including the predetermined object; Based on the weights and regression algorithm respectively corresponding to the plurality of color blocks, a plurality of reference color restoration correction matrices for converting the second color statistical information into the plurality of adjustment results are determined.

4. The method according to claim 2, wherein: The step of determining a target color restoration correction matrix adapted to the target image sensor from a plurality of reference color restoration correction matrices based on the plurality of second color restoration correction images comprises: Determine third color statistical information corresponding to the regions including the predetermined object in each of the second color restoration-corrected images, and determine a color deviation between each of the third color statistical information and the first color statistical information; Determine the signal-to-noise ratios corresponding to the second color restoration-corrected images respectively; Based on the color deviations and / or the signal-to-noise ratios respectively corresponding to the second color restoration correction images, selecting part of the reference color restoration correction matrices from a plurality of the reference color restoration correction matrices; Based on the second color restoration correction image corresponding to the screened part of the reference color restoration correction matrix, a target color restoration correction matrix adapted to the target image sensor is determined from the screened part of the reference color restoration correction matrix.

5. The method according to claim 2, wherein: The step of determining a target color restoration correction matrix adapted to the target image sensor from a plurality of reference color restoration correction matrices based on the plurality of second color restoration correction images comprises: respectively executing a predetermined machine vision task on the plurality of the second color restoration and correction images to obtain a plurality of task execution results; Evaluate the execution results of the plurality of tasks respectively to obtain a plurality of evaluation results; Based on the plurality of evaluation results, a target color restoration correction matrix adapted to the target image sensor is determined from the plurality of reference color restoration correction matrices.

6. The method according to any one of claims 1 to 5, wherein: The adjusting the first color statistical information to obtain multiple adjustment results of the first color statistical information includes: Based on the first color statistical information, determining that each of the plurality of color blocks in the area including the predetermined object corresponds to N color statistical values ​​of the N channels included in the first color restoration corrected image; Determine, within a preset random deviation value range, N random deviation values ​​corresponding to the N channels for each of the plurality of color blocks; Superimposing the N color statistics corresponding to each of the plurality of color blocks with the corresponding random deviation value to obtain N color superposition values ​​corresponding to each of the plurality of color blocks; Based on the N color superposition values ​​corresponding to each of the plurality of color blocks, one adjustment result among the plurality of adjustment results of the first color statistical information is determined.

7. The method according to any one of claims 1 to 5, wherein: The step of obtaining first color statistical information of a region including a predetermined object in the first color restoration-corrected image comprises: determining an image type of the first color restoration corrected image; Determine a color statistics method suitable for the image type; According to the color statistics method, color statistics are performed on the area including the predetermined object in the first color restoration and correction image to obtain the first color statistics information.

8. A color restoration device for an image, comprising: An acquisition module, configured to acquire first color statistical information of a region including a predetermined object in a first color restoration-corrected image; wherein the first color restoration-corrected image is determined based on a first native image acquired by a reference image sensor for the predetermined object; A first determination module is configured to determine an uncorrected image based on a second native image acquired by a target image sensor for the predetermined object; wherein a color filter array of the target image sensor is different from a color filter array of the reference image sensor; a second determination module, configured to determine second color statistical information of the area including the predetermined object in the uncorrected image determined by the first determination module; an adjusting module, configured to adjust the first color statistical information acquired by the acquiring module to obtain a plurality of adjustment results of the first color statistical information; a third determination module, configured to determine a target color restoration correction matrix adapted to the target image sensor based on the uncorrected image determined by the first determination module, the second color statistical information determined by the second determination module, and the multiple adjustment results obtained by the adjustment module, so as to perform color restoration on the target image to be corrected associated with the target image sensor through the target color restoration correction matrix.

9. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image color restoration method according to any one of claims 1 to 7.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the color restoration method of the image described in any one of claims 1-7.

Images