Color constancy method based on wavelet attention and VMama network

Abstract

The invention discloses a color constancy method based on wavelet attention and a VMama network, and belongs to the field of computer vision and image enhancement. According to the method, illumination parameter estimation is carried out by adopting a designed WA-VMama network, and then color correction is carried out by adopting the estimated illumination parameters, so that a corrected image under standard illumination is obtained. According to the method, a designed wavelet attention mechanism is embedded into VMama, and a deep network model (WA-VMama for short) is constructed, so that global information can be effectively extracted by using the network, local information can be considered, the method has higher feature extraction and expression ability, and the accuracy of illumination parameter estimation is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and image enhancement, and specifically relates to technologies such as deep learning and image processing. Background Art

[0002] Color constancy refers to the perceptual characteristic that color perception tends to remain relatively unchanged when the color of an object changes due to changes in conditions such as lighting. It is a type of perceptual constancy. The color constancy method is used to correct the color of the captured image and restore the image color to the standard light source so that the image color is consistent with the color seen by the human eye.

[0003] The color constancy calculation process is as follows Figure 1 As shown in the figure. First, the illumination parameters of the input color-biased image are estimated, and then the image is corrected to the image under standard white light using the estimated illumination parameters according to the Von Kries coefficient law. Generally speaking, the higher the accuracy of the estimated illumination parameters, the better the correction effect of the image.

[0004] Traditional methods use mathematical models based on certain assumptions to estimate illumination parameters. However, these assumptions limit the scope of application of color constancy, and only images that meet the assumptions can be accurately estimated. In recent years, illumination parameter estimation based on deep learning has made important progress. This method breaks away from the traditional method's dependence on assumptions and conditions, has high stability, and a wider range of applications. It can accurately estimate illumination parameters for different light source scenes, thus achieving good correction effects.

[0005] The color constancy calculation block diagram based on deep learning is as follows Figure 2 The whole block diagram consists of two parts:

[0006] 1) Illumination parameter estimation based on deep learning: First, a large number of labeled (real illumination parameter) color-biased images are used as training samples to train the deep neural network, and then the trained model is used to predict the color-biased images of unknown light sources to obtain the predicted illumination parameters.

[0007] It can be seen that, unlike the traditional idea of ​​feature extraction + fitting, the illumination parameter estimation based on deep learning automatically learns from a large number of samples in an end-to-end framework, establishes a mapping model between color-biased images and illumination parameters, and realizes the estimation of illumination parameters.

[0008] 2) Based on these illumination parameters, the color-shifted image is corrected using the Von Kries coefficient rule to obtain an image under standard illumination.

[0009] At present, color constancy methods generally use convolutional neural networks (CNNs) to estimate illumination parameters. CNNs have excellent local feature extraction capabilities, but due to excessive focus on local areas, they are unable to capture global context information, resulting in low estimation accuracy of illumination parameters. Summary of the invention

[0010] The present invention proposes the following solutions to the above problems:

[0011] (1) Constructed a lighting parameter estimation network based on VMamba structure

[0012] At present, VMamba network structure is increasingly applied to the field of computer vision, and has achieved excellent performance in various tasks such as image classification and image segmentation. VMamba introduces the Structured Selective Scan Space State Sequential Model (S6) applied in the field of natural language processing, which enables each element in a one-dimensional array (such as a text sequence) to interact with any previously scanned sample through a compressed hidden state, while retaining the global receptive field with linear complexity. In addition, VMamba adopts a cross scan module (CSM) for spatial domain traversal. CSM adopts a four-way scanning strategy to bridge the gap between one-dimensional sequences and two-dimensional visual data, ensuring that each element in the feature map can fuse elements in other positions and directions, giving VMamba excellent global context modeling capabilities.

[0013] To this end, this paper uses VMamba as the backbone network and constructs an illumination parameter estimation network, with the aim of achieving accurate estimation of illumination parameters, thereby improving the color cast image correction effect.

[0014] (2) Designed a wavelet attention mechanism

[0015] Although VMamba has excellent global context modeling capabilities, it does not pay enough attention to detail information such as edges. Wavelet transform can decompose an image into low-frequency and high-frequency components, where the low-frequency component represents the general appearance of the image, while the high-frequency component contains rich detail information such as edges. Considering that images with severe color deviation often have blurred edges, the present invention designs a wavelet attention mechanism (WA) to enhance detail information such as edges of images while removing noise interference.

[0016] The designed wavelet attention mechanism is embedded into VMamba in the present invention, and a deep network model (WA-VMamba for short) is constructed, so that the network can not only effectively extract global information, but also take local information into consideration, and has stronger feature extraction and expression capabilities, which helps to improve the accuracy of illumination parameter estimation.

[0017] A color constancy method based on wavelet attention and VMamba network is designed. This method uses the designed WA-VMamba network to estimate illumination parameters, and then uses the estimated illumination parameters for color correction to obtain a corrected image under standard illumination.

[0018] The method consists of two key steps, namely, illumination parameter estimation and color cast image correction. Each step is described in detail below.

[0019] Step 1: Lighting parameter estimation based on WA-VMamba

[0020] This step specifically includes three parts: WA-VMamba network structure design, network training and reasoning.

[0021] (1) WA-VMamba network structure design

[0022] The network uses VMamba as the backbone network and introduces the designed wavelet attention mechanism into it, so that the network can not only extract global context information but also focus on edge detail information and has good feature expression ability.

[0023] Specifically, the features output by the first stage of the VMamba network are fed into the wavelet attention mechanism, and the features enhanced by the wavelet attention mechanism are fused with the high-level features output by the fourth stage, input into two convolutional layers, and then pass through the confidence weighted pooling layer and the Softmax activation layer in sequence to obtain the final output result.

[0024] (2) WA-VMamba network training

[0025] The WA-VMamba network is trained using labeled data samples (including color-biased images and illumination parameters). The training process of the model is optimized and supervised by the angle error loss function to obtain a trained network model for estimating illumination parameters.

[0026] (3) WA-VMamba network reasoning

[0027] Use the trained WA-VMamba network to infer the color-biased image to be corrected and obtain the illumination parameters.

[0028] Step 2: Color cast image correction

[0029] According to the illumination parameters obtained in step 1, the color-shifted image is color corrected according to the Von Kries coefficient rule to obtain a corrected image without obvious color shift.

[0030] Compared with the prior art, the present invention has the following obvious advantages:

[0031] (1) Aiming at the problem that the accuracy of the illumination parameter estimation method based on CNN is low, which leads to poor color cast image correction results, the present invention proposes an illumination parameter estimation network structure WA-VMamba, which uses VMamba as the backbone network to solve the problem that the illumination parameter estimation accuracy of the existing CNN-based method is low;

[0032] (2) A wavelet attention mechanism is designed to enhance the edge detail information of the image, solving the problem of insufficient attention to local detail information based on the VMamba network structure.

[0033] Compared with the existing color constancy method, the method proposed in the present invention can estimate the illumination parameters more accurately, can obtain better correction effect, and has good practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the color constancy calculation process

[0035] Figure 2 Color constancy calculation block diagram based on deep learning

[0036] Figure 3 WA-VMamba network architecture diagram

[0037] Figure 4 VSS block structure diagram

[0038] Figure 52 Schematic diagram of D-DWT decomposition, where (a) original image and (b) wavelet image

[0039] Figure 6 WA module structure diagram DETAILED DESCRIPTION

[0040] Step 1: Lighting parameter estimation based on WA-VMamba

[0041] This step specifically includes three parts: WA-VMamba network structure design, network training and reasoning.

[0042] (1) WA-VMamba network structure design

[0043] The WA-VMamba network uses VMamba as the backbone network and introduces the designed wavelet attention mechanism into it. The network structure is as follows: Figure 3 As shown in Figure 1. In general, the VMamba network consists of four stages. First, the features output by the first stage are sent to the wavelet attention mechanism. The features enhanced by the wavelet attention mechanism are concatenated with the high-level features output by the fourth stage and then input into two convolutional layers. Then, they are sequentially passed through the confidence weighted pooling layer and the Softmax layer to obtain the final output result. The network input is a color-biased image and the output is the estimated illumination parameters.

[0044] The VMamba network used in the present invention consists of four stages, the core of which is a visual state space (VSS) block composed of SS2D modules integrated by S6 and CSM. The number of VSS blocks stacked in each stage is different. Through comparative experiments and analysis of the existing hardware equipment level (including video memory capacity, etc.), it is concluded that the performance is optimal when the number of VSS blocks in each stage is 2, 2, 9, and 2 respectively. The VMamba network first uses three convolution kernels of size 4×4 and stride 4 to convolution the input image. Cut into N non-overlapping blocks in represents a set of real numbers, H and W represent the height and width of the input image respectively. The first stage consists of two VSS blocks. The output of the first stage is Indicates that the height and width of F1 remain unchanged, and the number of channels C1 is 96. The second stage consists of 1 downsampling module and 2 VSS blocks. The output of the second stage is Indicates that the height and width of F2 are reduced to the height and width of F1 The number of channels C2 is 192. The third stage consists of 1 downsampling module and 9 VSS blocks. The output of the third stage is used Indicates that the height and width of F3 are reduced to the height and width of F2 The number of channels C3 is 384. The fourth stage consists of 1 downsampling module and 2 VSS blocks. The output of the fourth stage is used Indicates that the height and width of F4 are reduced to the height and width of F3 The number of channels C4 is 768.

[0045] The VSS block structure is as follows Figure 4 As shown, input feature F in After the normalization layer (NL), it is divided into two branches: one branch passes through a deep convolution layer consisting of a convolution kernel of size 3×3 and a step size of 1 after the linear embedding layer, and then passes through the SiLu activation function and is sent to the SS2D module. Its output passes through the normalization layer to obtain the output F out_1 ; The other branch passes through the linear embedding layer and is activated by the SiLu activation function to obtain the output Fout_2 Finally, the outputs of the two branches are multiplied point by point to obtain the output F out_3 , and then sent to the linear embedding layer to get the output F out_4 , and compare it with the input F in Add point by point to get the output feature F out .

[0046] The WA module designed in the present invention first uses a two-dimensional discrete wavelet transform (2D-DWT) to decompose the input feature map into four sub-bands. The 2D-DWT decomposition diagram is shown in Figure 5 As shown, (a) is the original image, (b) is the wavelet image, including four sub-bands, which are the low-frequency components X LL , vertical component X HL , horizontal component X LH and high frequency component X HH .

[0047] Feature Map The 2D-DWT is shown in formula (1):

[0048] F X_LL =LXL T , F X_LH =HXL T , F X_HL =LXH T , F X_HH =HXL T (1)

[0049] Among them, X represents the feature map F1, L represents the wavelet low-pass filter {l k} k∈Z The coefficient matrix of the wavelet high-pass filter {h k} k∈Z The coefficient matrix, L T represents the transposed matrix of the L coefficient matrix, H T Represents the transposed matrix of the H coefficient matrix. L and H can be expanded to the following form:

[0050]

[0051] Among them, l0 is the coefficient of the wavelet low-pass filter {l0}, l1 is the coefficient of the wavelet low-pass filter {l1}, h0 is the coefficient of the wavelet high-pass filter {h0}, and h1 is the coefficient of the wavelet high-pass filter {h1}.

[0052] The present invention uses the Harr wavelet transform (Haar wavelet transform), whose low-pass filter coefficient matrix is The high-pass filter coefficient matrix is In addition, F X_LL is the low frequency component, F X_LH is the horizontal component, F X_HL is the vertical component, F X_HH is the high frequency component.

[0053] The WA module structure is as follows Figure 6 As shown. F1 is the output of the first stage, and after discrete wavelet transform, four wavelet subband components F are obtained. X_LL 、F X_LH 、F X_HL and F X_HH . Discard the F containing noise X_HH , and for F X_LH 、F X_HL Perform the point-by-point maximum operation to obtain the output F Max , F Max F X_LH 、F X_HL The output obtained by taking the maximum value point by point is then normalized by Softmax in the channel dimension to obtain the attention weight map Wei. X_LL Multiply point by point to get the output F att , F att F X_LL The feature map weighted by the attention weight map Wei is compared with F X_LL Add point by point to get the feature F enhanced by the wavelet attention mechanism WA The process is expressed using formulas (3) to (7):

[0054] {F X_LL , F X_LH , F X_HL , F X_HH = DWT(F1) (3)

[0055] F Max =Max(F X_LH , F X_HL ) (4)

[0056] Wei==Softmax(F Max ) (5)

[0057]

[0058] Among them, DWT(·) represents the two-dimensional discrete wavelet transform of the input feature map, Max(·,·) represents the operation of taking the maximum value point by point, and Softmax(·) represents the Softmax normalization operation in the channel dimension. represents a point-by-point multiplication operation, and ⊕ represents a point-by-point addition operation.

[0059] Then to F WA Downsample to get the downsampled output F′ WA , so that its scale is consistent with the scale of the high-level feature F4 output by the fourth stage of the VMamba network. The two vectors are concatenated in the channel dimension to obtain the concatenated feature F5, whose number of channels is 864. Then, 768 convolution kernels with a size of 1×1 and a stride of 1 are used to reduce the dimension of F5 to obtain the fused feature F V , the number of channels is 768. This process is expressed using formulas (8) to (10):

[0060] F W ' A = Downsampling(F WA ) (8)

[0061] F5=Conca(F' WA ,F4) (9)

[0062] F V =Conv1×1(F5) (10)

[0063] Among them, Downsampling(·) represents the downsampling operation, Conca(·,·) represents the concatenation operation, and Conv1×1(·) represents the dimensionality reduction operation using 1×1 convolution.

[0064] Finally, the fused feature F V Through two convolutional layers (containing 64 6×6 convolution kernels with a stride of 1 and 4 1×1 convolution kernels with a stride of 1), F V The dimension is reduced to four feature channels, where the first three feature channels represent the i Color triplets for lighting parameter estimation The last channel represents the confidence c of the local area illumination parameter estimation i , where i represents the index of different local regions, 1≤i≤N. Using c i right After weighting, the weighted RGB values ​​are summed in the height and width dimensions to achieve aggregation from local to global. Finally, the estimated lighting parameters are obtained through the Softmax activation function. The above process is expressed using formula (11):

[0065]

[0066] (2) WA-VMamba network training

[0067] This method uses a pre-training + fine-tuning strategy to train the WA-VMamba network. First, the VMamba network parameters pre-trained on the ImageNet dataset are used as initial weights; then, the WA-VMamba network is fine-tuned using labeled data (color-biased images and real lighting parameters), and the model training process is optimized and supervised by the angle error loss function to obtain the trained WA-VMamba network.

[0068] Combined with the level of existing hardware equipment, the parameters for training are set as follows: batch size batch_size is set to 2, initial learning rate learning_rate is set to 0.00001. Generally speaking, the size of learning_rate is positively correlated with batch_size. Too large learning_rate will cause oscillation and lead to failure to converge, while too small will slow down the convergence speed but will not affect the model performance. The training round epochs setting range is 4000 to 6000 to ensure that the model can fully converge under the above parameter settings. Too small will lead to incomplete convergence of the model, and too large will lead to serious overfitting.

[0069] The angle error loss function is defined as the angle error (in degrees) between its estimated value and the true value, expressed using formula (12):

[0070]

[0071] in is the actual lighting parameter, is the true value of the R channel, is the true value of the G channel, is the true value of the B channel. is the estimated illumination parameter, is the estimated value of the R channel, is the estimated value of the G channel, is the estimated value of channel B. arccos(· represents the inverse cosine function, E(e r ,e c ) means e r With e c The angle difference between them.

[0072] (3) WA-VMamba network reasoning

[0073] The color-biased image to be corrected is input into the trained WA-VMamba network model, and the network output is the estimated illumination parameters of the color-biased image. Use [e r , e g , e b ] indicates that, among them, represents the set of real numbers, Represents the estimated illumination parameters of the color-biased image The set of real numbers belonging to three dimensions, e r is the value of the R channel of the color-biased image illumination parameter, e g is the value of the G channel of the color-biased image illumination parameter, e b It is the value of the B channel of the color-biased image illumination parameter.

[0074] Step 2: Color cast image correction

[0075] The VonKries coefficient rule states that, with standard white light as a reference, the RGB values ​​of the color-biased image [R1, G1, B1] and the RGB values ​​of the corrected image [R2, G2, B2] have the following relationship:

[0076]

[0077] Among them, R1 is the value of the R channel of the color-shifted image, G1 is the value of the G channel of the color-shifted image, B1 is the value of the B channel of the color-shifted image, R2 is the value of the R channel of the corrected image, G2 is the value of the G channel of the corrected image, B2 is the value of the B channel of the corrected image, and k r is the correction coefficient of the R channel, k g is the correction coefficient of the G channel, k b is the correction coefficient of channel B. r , k g , k b The relationship between it and the illumination parameters is expressed by formulas (14) to (16):

[0078]

[0079] Among them, e r is the value of the R channel of the color-biased image illumination parameter, e g is the value of the G channel of the color-biased image illumination parameter, e b It is the value of the B channel of the color-biased image illumination parameter.

[0080] Therefore, the illumination parameter estimation result obtained in step 1 can be Color shift image based on Von Kries coefficient law Perform color correction, that is, multiply the RGB value of each pixel of the color-biased image by the corresponding correction coefficient to obtain the corrected image The specific process is expressed by formula (17):

[0081]

Claims

1. A color constancy method based on wavelet attention and VMamba network, characterized in that: The following steps are involved: Step 1: Lighting parameter estimation based on WA-VMamba Specifically, it includes three parts: WA-VMamba network structure design, network training and reasoning; (1) WA-VMamba network structure design The WA-VMamba network uses VMamba as the backbone network and introduces the designed wavelet attention mechanism into it. The VMamba network consists of four stages. First, the features output by the first stage are sent to the wavelet attention mechanism. The features enhanced by the wavelet attention mechanism are concatenated with the high-level features output by the fourth stage and then input into two convolutional layers. Then, the confidence-weighted pooling layer and the Softmax layer are sequentially passed to obtain the final output result. The network input is the color-biased image and the output is the estimated illumination parameters. The VMamba network used consists of four stages, the core of which is the visual state space VSS block composed of SS2D modules integrated by S6 and CSM. The number of VSS blocks stacked in each stage is different, and the number of VSS blocks in each stage is 2, 2, 9, and 2 respectively; The VMamba network first uses three convolution kernels of size 4×4 and stride 4 to convolve the input image Cut into N non-overlapping blocks in represents a real number set, H and W represent the height and width of the input image respectively; the first stage consists of 2 VSS blocks, and the output of the first stage is Indicates that the height and width of F1 remain unchanged, and the number of channels C1 is 96; the second stage consists of 1 downsampling module and 2 VSS blocks, and the output of the second stage is Indicates that the height and width of F2 are reduced to the height and width of F1 The number of channels C2 is 192; the third stage consists of 1 downsampling module and 9 VSS blocks. The output of the third stage is used Indicates that the height and width of F3 are reduced to the height and width of F2 The number of channels C3 is 384; the fourth stage consists of 1 downsampling module and 2 VSS blocks. The output of the fourth stage is used Indicates that the height and width of F4 are reduced to the height and width of F3 The number of channels C4 is 768; Input feature F in After the normalization layer NL, it is divided into two branches: one branch passes through a deep convolution layer consisting of a convolution kernel of size 3×3 and a step size of 1 after the linear embedding layer, and then passes through the SiLu activation function and is sent to the SS2D module. Its output passes through the normalization layer to obtain the output F out_1 ; The other branch passes through the linear embedding layer and is activated by the SiLu activation function to obtain the output F out_2 ; Finally, multiply the outputs of the two branches point by point to get the output F out_3 , and then sent to the linear embedding layer to get the output F out_4 , and compare it with the input F in Add point by point to get the output feature F out ; The designed WA module first uses two-dimensional discrete wavelet transform 2D-DWT to decompose the input feature map into four sub-bands, namely the low-frequency component X LL , vertical component X HL , horizontal component X LH and high frequency component X HH ; Feature Map The 2D-DWT is shown in formula (1): F X_LL =LXL T ,F X_LH =HXL T ,F X_HL =LXH T ,F X_HH =HXH T (1) Among them, X represents the feature map F1, L represents the wavelet low-pass filter {l k } k∈Z The coefficient matrix of the wavelet high-pass filter {h k } k∈z The coefficient matrix, L T represents the transposed matrix of the L coefficient matrix, H T Represents the transposed matrix of the H coefficient matrix; L and H can be expanded to the following form: Among them, l0 is the coefficient of the wavelet low-pass filter {l0}, l1 is the coefficient of the wavelet low-pass filter {l1}, h0 is the coefficient of the wavelet high-pass filter {h0}, and h1 is the coefficient of the wavelet high-pass filter {h1}; Using the Harr wavelet transform, its low-pass filter coefficient matrix is The high-pass filter coefficient matrix is F X_LL is the low frequency component, F X_LH is the horizontal component, F X_HL is the vertical component, F X_HH is the high frequency component; The specific structure of the WA module is as follows: F1 is the output of the first stage, and after discrete wavelet transformation, four wavelet subband components F are obtained. X_LL 、F X_LH 、F X_HL and F X_HH ; Discard the F containing noise X_HH , and for F X_LH 、F X_HL Perform the point-by-point maximum operation to obtain the output F Max , F Max F X_LH 、F X_HL The output is obtained by taking the maximum value point by point, and then the Softmax normalization operation is performed in the channel dimension to obtain the attention weight map Wei; Wei and F X_LL Multiply point by point to get the output F att , F att F X_LL The feature map weighted by the attention weight map Wei is X_LL Add point by point to get the feature F enhanced by the wavelet attention mechanism WA ; This process is expressed using formulas (3) to (7): {F X_LL ,F X_LH ,F X_HL ,F X_HH }=DWT(F1) (3) F Max =Max(F X_LH ,F X_HL ) (4) Wei=Softmax(F Max ) (5) Among them, DWT(·) represents the two-dimensional discrete wavelet transform of the input feature map, Max(·,·) represents the operation of taking the maximum value point by point, and Softmax(·) represents the Softmax normalization operation in the channel dimension. represents a point-by-point multiplication operation, Represents a point-by-point addition operation; Then to F WA Downsample to get the downsampled output F′ WA , so that its scale is consistent with the scale of the high-level feature F4 output by the fourth stage of the VMamba network. The two vectors are concatenated in the channel dimension to obtain the concatenated feature F5, whose number of channels is 864. Then, 768 convolution kernels with a size of 1×1 and a stride of 1 are used to reduce the dimension of F5 to obtain the fused feature F V , the number of channels is 768; the process is expressed using formulas (8) to (10): F′ WA =Downsampling(F WA ) (8) F5=Conca(F′ WA ,F4) (9) F V =Conv1×1(F5) (10) Among them, Downsampling(·) represents the downsampling operation, Conca(·,·) represents the concatenation operation, and Conv1×1(·) represents the dimensionality reduction operation using 1×1 convolution; Finally, the fused feature F V Through two convolutional layers, F V The dimension is reduced to four feature channels. The convolution layer contains 64 convolution kernels of size 6×6 and stride 1 and 4 convolution kernels of size 1×1 and stride 1. The first three feature channels represent the convolution kernels from each corresponding local area R. i Color triplets for lighting parameter estimation The last channel represents the confidence c of the local area illumination parameter estimation i , where i represents the index of different local regions, 1≤i≤N; using c i right After weighting, the weighted RGB values ​​are summed in the height and width dimensions to achieve aggregation from local to global. Finally, the estimated illumination parameters are obtained through the Softmax activation function. The above process is expressed by formula (11): (2) WA-VMamba network training Firstly, the parameters of VMamba network pre-trained on ImageNet dataset are used as initial weights. Then, the WA-VMamba network is fine-tuned using labeled data, i.e. color-biased images and real illumination parameters. The model training process is optimized and supervised by the angle error loss function to obtain the trained WA-VMamba network. The parameters during training are set as follows: batch size batch_size is set to 2, initial learning rate learning_rate is set to 0.00001, and training rounds epochs are set in the range of 4000 to 6000; The angle error loss function is defined as the angle error between its estimated value and the true value, in degrees, and is expressed using formula (12): in is the actual lighting parameter, is the true value of the R channel, is the true value of the G channel, is the true value of channel B; is the estimated illumination parameter, is the estimated value of the R channel, is the estimated value of the G channel, is the estimated value of the B channel; arccos(·) represents the inverse cosine function, E(e r , e c ) means e r With e c The angle between them; (3) WA-VMamba network reasoning The color-biased image to be corrected is input into the trained WA-VMamba network model, and the network output is the estimated illumination parameters of the color-biased image. Use [e r , e g , e b ] indicates that, among them, represents the set of real numbers, Represents the estimated illumination parameters of the color-biased image The set of real numbers belonging to three dimensions, e r is the value of the R channel of the color-biased image illumination parameter, e g is the value of the G channel of the color-biased image illumination parameter, e b is the value of the B channel of the color-biased image illumination parameter; Step 2: Color cast image correction The Von Kries coefficient rule states that, with standard white light as a reference, the RGB values ​​of the color-biased image [R1, G1, B1] and the RGB values ​​of the corrected image [R2, G2, B2] have the following relationship: Among them, R1 is the value of the R channel of the color-shifted image, G1 is the value of the G channel of the color-shifted image, B1 is the value of the B channel of the color-shifted image, R2 is the value of the R channel of the corrected image, G2 is the value of the G channel of the corrected image, B2 is the value of the B channel of the corrected image, and k r is the correction coefficient of the R channel, k g is the correction coefficient of the G channel, k b is the correction coefficient of channel B; k r , k g , k b The relationship between it and the illumination parameters is expressed by formulas (14) to (16): Among them, e r is the value of the R channel of the color-biased image illumination parameter, e g is the value of the G channel of the color-biased image illumination parameter, e b is the value of the B channel of the color-biased image illumination parameter; Therefore, according to the illumination parameter estimation result obtained in step 1 Color shift image based on Von Kries coefficient law Perform color correction, that is, multiply the RGB value of each pixel of the color-biased image by the corresponding correction coefficient to obtain the corrected image The specific process is expressed by formula (17):

Images