Multispectral awb method, system and apparatus based on gaussian smoothing weighted voting

Abstract

This invention discloses a multispectral AWB method, system, and apparatus based on Gaussian smoothing weighted voting, comprising: using a preprocessed image as the input image... I Input image I The system divides the image into multiple patches of equal size with no overlap. The average spectrum of each patch is calculated, and the nearest candidate light source is found based on the distance between the average spectrum and candidate light sources in the light source database to determine the patch's color temperature. Multi-scale patch weights are calculated to obtain the total patch weight. All patches are weighted and voted on, and the voting results are Gaussian smoothed to obtain the predicted color temperature. The predicted color temperature is then queried from the light source database to obtain the corresponding light source illumination value. This invention uses statistical methods to establish a light source database combined with a Gaussian smoothing weighted voting algorithm, effectively achieving automatic white balance for color-biased images in complex scenes, and providing a more accurate white balance effect.

Description

Technical Field

[0001] This invention relates to the field of Automatic White Balance (AWB) in computer vision, and particularly to a multispectral automatic white balance method based on Gaussian smoothing weighted voting using multispectral images in complex scenes. Specifically, it is a multispectral AWB method, system, and apparatus based on Gaussian smoothing weighted voting. Background Technology

[0002] Automatic white balance (AWB) is a crucial component of the image processing workflow. It simulates the color constancy of the human visual system (the ability to perceive the color of an object as constant even as scene lighting changes). This process, also known as color constancy calculation, is the first step in camera image processing. In other words, the purpose of AWB is to endow photographic equipment with color constancy, eliminating color casts caused by ambient light sources. To achieve the functionality of AWB and eliminate color casts in images, researchers have proposed numerous AWB methods.

[0003] In the current field of automatic white balance (AWB), most methods only use RGB images for illumination estimation. Since RGB images only have three channels and can only capture the visible light range, they suffer from insufficient illumination information. Currently, only a few methods utilize multispectral images with rich illumination information. However, since the final color correction still requires RGB images, existing methods using multispectral images for illumination estimation typically require acquiring a large number of image samples from different scenes and camera domains for training to learn the nonlinear mapping relationship between the illumination spectrum and the RGB response. This results in high image acquisition costs and a complex process. This research area represents both a current research hotspot and a significant challenge.

[0004] Secondly, current research on automatic white balance (AWB) algorithms using multispectral images cannot address the impact of abnormal areas such as specular reflection, shadow occlusion, and non-dominant light sources on illumination estimation. Therefore, existing automatic white balance algorithms using multispectral images have many shortcomings and require improvement and refinement. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a multispectral automatic white balance method based on Gaussian smoothing weighted voting, which can effectively achieve automatic white balance for color-shifted images in complex scenes, resulting in a more accurate white balance effect. Based on the designed algorithm, compared with the state-of-the-art existing automatic white balance techniques using multispectral images, the illumination estimation of this invention is more accurate.

[0006] The complete technical solution of this invention includes: The multispectral AWB method based on Gaussian smoothing weighted voting includes the following steps: (1) Use the multispectral image to be analyzed as the input image. I ; (2) Input image I Divide the patch into multiple patches of the same size with no overlap; (3) Calculate the average spectrum of each patch, find the nearest candidate light source based on the average spectrum and the distance of the candidate light sources in the light source database, and determine the color temperature of the patch; (4) Calculate the color temperature weights of the patch at multiple scales, and finally obtain the total color temperature weights of the patch; (5) Weight the total color temperature of all patches by voting, and smooth the voting results using Gaussian to obtain the predicted color temperature; (6) Query the light source database based on the predicted color temperature to obtain the RGB light source illumination value corresponding to the color temperature.

[0007] Furthermore, the light source database is obtained by statistically mapping the spectral distribution characteristics of the illumination with the corresponding RGB response using statistical methods.

[0008] Furthermore, the light source acquisition method for the light source database is as follows: acquiring multispectral images of illumination indicators under different color temperature illumination environments and RGB images of different lenses; detecting and extracting the gray pixel portion in the illumination indicator to obtain the light source for each acquired color temperature. The multispectral values ​​of illumination and the RGB values ​​of illumination from different lenses.

[0009] Furthermore, it also includes step (7): based on the RGB light source illumination value obtained in step (6), perform automatic white balance processing on the image.

[0010] Furthermore, in step (3), an average calculation is performed on the spatial dimension of each patch to obtain the average spectral value. The Euclidean distance with each light source reference vector in the light source database is calculated, and the index of the light source reference vector with the smallest distance is selected as the light source index. Use the color temperature of the light source index as the color temperature of this patch. .

[0011] Furthermore, in step (4), the total weight of each patch is calculated based on the illumination distance weight, brightness weight, and region consistency weight. .

[0012] Furthermore, in step (5), based on the original weighted voting results... The new voting results are calculated by introducing Gaussian kernel smoothing. Considering the impact of similar color temperatures on the number of votes for the current color temperature, the color temperature with the highest number of votes was obtained. .

[0013] Furthermore, the final decision was based on the color temperature with the highest number of votes. Search the light source database to obtain the final illumination estimation result of the target RGB lens. .

[0014] Furthermore, a multispectral AWB system employing the described method.

[0015] Furthermore, a multispectral AWB device incorporating the aforementioned system.

[0016] The advantage of this invention over existing technologies lies in solving the problems of RGB mapping difficulties and the significant impact of anomalous regions on the estimation results in existing automatic white balance algorithms using multispectral images during illumination estimation. Specifically, these problems include: existing methods for illumination estimation using multispectral images typically require collecting a large number of image samples from different scenes and camera domains for training to learn the nonlinear mapping relationship between the illumination spectrum and the RGB response. Training the mapping for each RGB camera domain requires a large number of images captured by that camera, resulting in high acquisition costs and a complex process; secondly, anomalous regions in the image, such as specular highlights, shadow occlusion, and non-dominant light sources, cannot reflect the true illumination in the scene. The advantages over existing technologies specifically include: (1) Multispectral Light Source Database: To address the difficulty of learning the mapping between illumination spectrum and RGB response, this paper does not use deep learning to learn the mapping. Instead, it uses statistical methods to establish a light source database, statistically mapping the spectral distribution characteristics of illumination with the corresponding RGB response for fast and low-cost illumination estimation. For each new RGB camera, the light source database of this invention only requires a small number of illumination indicator images under the corresponding color temperature illumination, avoiding the problems of large-scale data collection and repeated training.

[0017] (2) To address the problem that abnormal regions in real images, such as high-brightness reflection, shadow occlusion, and non-dominant light sources, cause deviations in illumination statistical features, leading to unstable illumination estimation results based on statistical features, the Gaussian smoothing weighted voting algorithm proposed in this invention considers weight information in multiple dimensions and applies different weights to information in different regions of the image, thus solving the problem that abnormal regions in the image affect the overall illumination estimation results. Through Gaussian smoothing calculation, the similarity between similar color temperatures is considered, and the results are reweighted, enabling the algorithm to effectively utilize illumination information in regions of the multispectral image that are more favorable for illumination estimation, thus solving the problem that small voting biases cause serious deviations in color temperature estimation. Attached Figure Description

[0018] Figure 1This is a flowchart illustrating the multispectral automatic white balance algorithm based on Gaussian smoothing weighted voting, as presented in this invention.

[0019] Figure 2 This is a schematic diagram of light source acquisition in a light source database provided in an embodiment of the present invention.

[0020] Figure 3 Example image for detecting and extracting gray pixels in a light indicator. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0022] The purpose of this invention is to provide a multispectral automatic white balance method based on Gaussian smoothing weighted voting, so as to achieve effective automatic white balance of multispectral images in complex scenes. The algorithm flow of this invention is as follows: Figure 1 As shown.

[0023] Figure 2 This is a schematic diagram illustrating a light source database and light source acquisition method provided in an embodiment of this application. Figure 2 As shown, firstly, multispectral and RGB images of the illuminated indicator under different color temperature illumination environments were acquired, for example, color temperature ranges from 2700K to 8000K were acquired in 54 increments. To ensure that the illumination is as uniform as possible on the illuminated indicator, four light bulbs were assumed in the scene, and the color temperature of all bulbs was kept consistent during a single acquisition, ensuring that there were no other light sources affecting the process.

[0024] like Figure 3 As shown, multispectral images and RGB images from different lenses were acquired, and the gray pixels in the illumination indicator were detected and extracted to obtain the color temperature of each acquisition. Multispectral values ​​of illumination RGB values ​​of light from different lenses , wait.

[0025] For the acquired multispectral images and the RGB images of different lenses extracted from each image, statistical methods are used to statistically correlate the spectral distribution characteristics of the illumination with the corresponding RGB responses, and a light source database is established. This statistical method can be completed using existing conventional statistical techniques.

[0026] Includes the following steps: (1) The image to be analyzed is used as the input image. I Optionally, input a multispectral image. The size can be expressed as ,in, This indicates the number of channels in the image. The height of the image can be represented by the number of pixels in the vertical dimension. The width of the image can be represented by the number of pixels in the horizontal dimension.

[0027] Optionally, input a multispectral image. The size can be This indicates that it has 9 different spectral response bands, with a height of 512 pixels and a width of 512 pixels.

[0028] (2) Input image I The patch is divided into multiple regions of the same size with no overlap. For multispectral images Divide it into The patch, assuming the input multispectral image is The specific implementation is shown in the following formula.

[0029]

[0030] in, Indicates the location in the input multispectral image The image of the patch at that location. These represent the height and width of each patch, respectively.

[0031] (3) Calculate the average spectrum of each patch, find the nearest candidate light source based on the average spectrum and the distance of the candidate light sources in the light source database, and determine the color temperature of the patch; The average spectral value is obtained by averaging across the spatial dimensions of each patch. The specific calculation process is shown in the following formula.

[0032]

[0033] Location in multispectral image The average spectral value at that location.

[0034] The aforementioned established light source database contains Each light source reference vector Each vector corresponds to a color temperature. The average spectral value for each multispectral patch. Calculate the Euclidean distance to each light source reference vector, and select the light source reference vector with the smallest distance as the light source index. To obtain the color temperature of the patch. The specific process is shown in the following formula.

[0035]

[0036] ,

[0037] For Euclidean distance, This represents the total number of channels in the image.

[0038] (4) Calculate the patch weights at multiple scales to obtain the total weight of the patch with respect to color temperature; To improve the reliability of matching, a comprehensive weighting function was designed. Specifically, the weights are shown in the following three weights.

[0039] Illumination distance weight: Defined according to the exponential decay law of matching distance: The smaller the distance between the average spectrum of the patch and the nearest reference light source, the more reliable the match and the greater the weight.

[0040] Index the patch to the light source with the smallest distance. The Euclidean distance between them.

[0041] Brightness weighting: Correctly exposed areas in an image better reflect scene lighting information; underexposed and overexposed areas lose information and are unreliable for lighting estimation. Define patch brightness. To normalize the average value of the channels, and define the brightness weighting function. The specific definition is shown in the following formula.

[0042]

[0043]

[0044] Regional consistency weight: In order to reduce the impact of non-dominant light sources on the illumination estimation results and enhance the spatial consistency of the estimation results, the weights are adjusted according to the color temperature differences of the neighboring regions of the patch.

[0045] In illumination estimation tasks, the relationship between color temperature (CCT) and spectral distribution is not linear. Specifically, low color temperatures (e.g., 2700K–3500K) correspond to warm light spectra, and their spectral change rate is much greater than that of high color temperature regions (6500K–8000K). Therefore, the color temperature difference between two patches... It cannot truly reflect the spectral differences, and directly using linear differences as a consistency measure will lead to a distortion in the judgment of regional consistency.

[0046] Therefore, let patch The neighborhood is The color temperature of each patch is converted to its reciprocal, and then the average color temperature of the neighborhood and the color temperature difference are calculated. Finally, the color temperature weights are adjusted based on the color temperature differences. The specific calculation process is shown in the following formula.

[0047]

[0048]

[0049] For patch All neighborhoods The average of the reciprocals of the color temperature.

[0050] The regional consistency weight can be obtained. The formula is:

[0051] Finally, the total weight of the multi-scale patch with respect to color temperature is obtained. :

[0052] (5) Total color temperature weight of all patches Weighted voting is performed, and the voting results are Gaussian smoothed to obtain the predicted color temperature; In weighted voting, the traditional discrete voting mechanism treats similar but different color temperatures as mutually exclusive categories, causing the spectral similarity of similar color temperatures to be ignored. The algorithm tends to select abnormal color temperatures that receive a single high vote, and even a small voting bias can lead to a serious deviation in color temperature estimation.

[0053] This invention employs a Gaussian smoothing weighted voting mechanism, based on the original weighted voting results. The new voting results are calculated by introducing Gaussian kernel smoothing. Considering the impact of similar color temperatures on the number of votes for the current color temperature, the color temperature with the highest number of votes is obtained. The specific implementation is shown in the following formula.

[0054]

[0055]

[0056]

[0057] in The attenuation coefficient is... Control the smoothing range.

[0058] (6) Based on the predicted color temperature The color temperature was obtained by querying the light source database. The corresponding light source illumination value.

[0059] Ultimately, the color temperature with the highest number of votes was used. Search the light source database to obtain the final illumination estimation result of the target RGB lens. .

[0060] Based on the final illumination estimation results of the target RGB lens Automatic white balance processing is performed on the image.

[0061] This invention offers more precise calculations, effectively improving the accuracy of illumination estimation in multi-source scenarios. Specifically, in publicly available datasets, compared to the most relevant prior art, this invention achieves the best results on a self-collected test set, as shown in Table 1.

[0062] Table 1. Comparison of the prediction results of this invention with existing technologies

[0063] By combining the above-mentioned technical features, this invention can effectively achieve automatic white balance for color-shifted images in complex scenes, resulting in a more accurate white balance effect. Thanks to the designed algorithm, the illumination estimation of this invention is more accurate compared to the most advanced existing automatic white balance techniques using multispectral images.

[0064] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multispectral AWB method based on Gaussian smoothing weighted voting, characterized in that, Includes the following steps: (1) Use the multispectral image to be analyzed as the input image. I ; (2) Input image I Divide the patch into multiple patches of the same size with no overlap; (3) Calculate the average spectrum of each patch, find the nearest candidate light source based on the average spectrum and the distance of the candidate light sources in the light source database, and determine the color temperature of the patch; (4) Calculate the color temperature weights of the patch at multiple scales, and finally obtain the total color temperature weights of the patch; (5) Weight the total color temperature of all patches by voting, and smooth the voting results using Gaussian to obtain the predicted color temperature; (6) Query the light source database based on the predicted color temperature to obtain the RGB light source illumination value corresponding to the color temperature.

2. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 1, characterized in that, The light source database is obtained by statistically mapping the spectral distribution characteristics of illumination with the corresponding RGB responses using statistical methods.

3. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 2, characterized in that, The light source database acquisition method is as follows: acquire multispectral images of illumination indicators under different color temperature illumination environments and RGB images of different lenses; detect and extract the gray pixel portion of the illumination indicator to obtain the light source for each acquired color temperature. The multispectral values ​​of illumination and the RGB values ​​of illumination from different lenses.

4. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 3, characterized in that, It also includes step (7): based on the RGB light source illumination value obtained in step (6), perform automatic white balance processing on the image.

5. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 4, characterized in that, In step (3), the average spectral value is calculated on the spatial dimension of each patch. The Euclidean distance with each light source reference vector in the light source database is calculated, and the index of the light source reference vector with the smallest distance is selected as the light source index. Use the color temperature of the light source index as the color temperature of this patch. .

6. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 5, characterized in that, In step (4), the total weight of each patch is calculated based on the illumination distance weight, brightness weight, and region consistency weight. .

7. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 6, characterized in that, In step (5), based on the original weighted voting results The new voting results are calculated by introducing Gaussian kernel smoothing. Considering the impact of similar color temperatures on the number of votes for the current color temperature, the color temperature with the highest number of votes was obtained. .

8. The multispectral AWB method based on Gaussian smoothing weighted voting according to claim 7, characterized in that, Ultimately, the color temperature with the highest number of votes was used. Search the light source database to obtain the final illumination estimation result of the target RGB lens. .

9. A multispectral AWB system employing the method described in any one of claims 1-8.

10. A multispectral AWB device incorporating the system of claim 9.

Images