Image color correction method, system and electronic device based on dual discriminator network

Abstract

This invention relates to an image color correction method, system, and electronic device based on a dual-discriminator network. To address the limitations imposed on the learning capabilities of GANs in color constancy tasks by issues such as color feature misleading and the low sensitivity of the discriminator to color information, we propose a Dual-Discriminator Generative Adversarial Network (DDGAN). It comprises a basic generative adversarial network and a Feature Fusion Discriminator (FFD) module. The FFD is a discriminator module with two feature extraction branches: one extracts color features, and the other extracts globally relevant features. The FFD fuses these features together to weaken structural features and enhance the discriminator's sensitivity to color features. Finally, to obtain a more uniform color and content image, a global consistency constraint is applied to the structural features weakened by the FFD, unifying the structural and color features, ultimately resulting in a high-precision image color correction method.

Description

Technical Field

[0001] This invention belongs to the field of computer and information service technology, and specifically relates to a method for color constancy in removing light sources from color-distorted images. Background Technology

[0002] Color constancy (CC) refers to the ability of the human visual system to maintain its color perception of objects regardless of changes in external lighting conditions. Color is a crucial underlying feature for tasks such as object detection and image semantic segmentation, thus the task of color constancy has received widespread attention in the field of computer vision.

[0003] Researchers have proposed many algorithms to simulate the human eye's perceptual characteristics of color constancy. Existing color constancy algorithms fall into two main categories: one is based on statistical methods, which uses reasonable assumptions about the lighting world to estimate the light source; the other is based on learning methods, which trains networks to learn and extract the lighting features of images.

[0004] Statistical methods typically rely on certain assumptions or preconditions. For example, the White Patch method, based on the assumption that the brightest point in an image is a standard white reflective surface, corrects the image using the standard white portion. The "Grey World" method approximates the entire imaging scene as a diffuse reflective plane, assuming that the average reflectance of the scene approximates a certain known reflectance (the so-called "gray"). Alternatively, a reference object, such as a "Grey ball," may be set in the image. These studies have achieved good results under their respective assumptions. However, in practical machine vision applications, scenarios that do not meet these assumptions or preconditions are commonplace. Therefore, it is necessary to design more adaptable algorithms for these complex imaging conditions.

[0005] With the continuous development of computers and deep learning, researchers have introduced learning-based methods into the color constancy task. Initially, researchers built special mathematical models, used machine learning strategies, and finally obtained the image light source through classification or regression operations. These methods include Bayesian approaches, support vector machines, and decision forests. Bayesian approaches estimate the light source by analyzing the later distribution conditions of image intensity data. Later, the emergence of a large amount of color constancy data enabled methods for estimating image light sources based on neural networks. B. Funt et al. proposed using binary histograms to train neural networks, marking the beginning of CNNs in the color constancy field. Barron also used CNN networks for light source estimation, but the difference is that the CCC (Convolutional Color Constancy) method transforms the image into a chromaticity space and uses chromaticity histograms to transform light source estimation from a generation problem into a discrimination problem.

[0006] Researchers found that the methods mentioned above all estimate the light source of the image through various means. However, the ultimate goal of color constancy is to generate a corrected image. Therefore, they attempted to ignore the light source and directly achieve end-to-end pixel-level image generation. Partha et al. introduced GAN (Generative Adversarial Networks) into the CC field, using three existing GAN network frameworks to achieve end-to-end image generation without predicting the light source. This work demonstrates the significant research importance of GANs in the CC field. Subsequently, they proposed a GAN network method based on physically initial seed points, achieving even better results than before. However, the strong correlation of multiple image features may lead to misguidance and low discriminator sensitivity when GAN networks learn CC tasks. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a dual discriminator generative adversarial network (DDGAN) based on learning key color information.

[0008] The technical solution of this invention is an image color correction method based on a dual discriminator network, comprising the following steps:

[0009] Step 1: Select the color constancy dataset and perform data cleaning on the dataset;

[0010] Step 2: Construct a dual discriminator network model. The dual discriminator network model includes a basic generative adversarial network and a feature fusion discriminator module (FFD). The basic generative adversarial network includes a generator and a discriminator for generating and discriminating images. The feature fusion discriminator module is used to enhance the discriminator's ability to distinguish color features in images.

[0011] First, the color-biased image (Input) and its corresponding mask image are input into the generator in the Generative Adversarial Network (GAN) structure to generate a fake image (Fake). The generated fake image and the real image (GT) are then compared in the discriminator. Therefore, the generated fake image and the real image are respectively input into the discriminator and feature fusion discriminator modules in the GAN structure for discrimination. If both discriminators output "fake," the generator parameters are updated to generate a fake image with a better result; if both discriminators output "real," the generated fake image is the final output.

[0012] Step 3: Train the dual discriminator network model and use the trained model to perform pixel-to-pixel and end-to-end color cast correction.

[0013] Furthermore, the preprocessing includes removing anomalous images from the dataset and performing data augmentation on the dataset.

[0014] Furthermore, the specific processing procedure of the FFD module is as follows;

[0015] The FFD module consists of two branches: a Gaussian image feature extraction branch and a full-pixel relation feature extraction branch. In the Gaussian image feature extraction branch, the input is first subjected to Gaussian blur through convolution, and then subjected to three layers of downsampling convolution to obtain a feature map containing better distinguishable color information. The convolution in the Gaussian blur uses a two-dimensional normally distributed convolution kernel, and the specific formula is as follows:

[0016]

[0017] In the above formula, x c y c These represent the horizontal and vertical coordinates of the corresponding image pixels, Gaussian represents Gaussian blur, and σ represents the blur radius.

[0018] The full-pixel relation feature extraction branch achieves full-pixel relation extraction through non-local attention. Specifically, it calculates the correlation between a pixel in the image and all other pixels. The non-local operation is defined as follows:

[0019]

[0020] In the above formula, x represents the input feature map, i represents the pixel index, j represents all pixels in the image except i, and y represents the output feature map; function f calculates the correlation between all pixels, and function g represents the input information of pixel j. It is a normalization function, and the above formula achieves the output of the same full-pixel relationship as the input size;

[0021] Features extracted from global pixel correlation are fused with features extracted from Gaussian blur blocks to jointly aid in the judgment. Feature fusion is achieved through non-local residual connections.

[0022] Z i =W z y i +Gauss(x i )

[0023] W z It is a weight matrix to be learned for feature transformation, +Gauss(x) i Feature fusion is achieved by adding the features extracted by residual connection and Gaussian blur.

[0024] The fused features are input into the final convolutional layer to determine whether a value is true or false.

[0025] Furthermore, function f directly uses the dot product method, while function g uses simple linear embedding and normalization function. This is achieved using softmax, with the specific formula as follows:

[0026] f(x i x j )=θ(x i ) T θ(x j )

[0027] g(x j ) = W g x j

[0028] In the above formula, θ represents a 1×1×1 convolution. The feature map is multiplied by its transpose after passing through the convolutional layer to obtain the full pixel relationship. W g This is the weight matrix to be learned.

[0029] Furthermore, in step 3, the dual-discriminator network model in the case of dual discriminators is expressed as:

[0030]

[0031] Where E represents the expectation, the generator G and two discriminators (D, D) color The goal is to generate images that are as close as possible to real images through a game-like process, where D is the discriminator in the generative adversarial network (GAN) structure. color is the feature fusion discriminator module, where x represents the input.

[0032] Furthermore, during training, the loss function used by the generator is:

[0033]

[0034] In the above formula, This represents the loss function in the generator. It is a constraint on the overall angular error between the generated fake image and the ground plane, where λ1, λ2, and λ3 are weighting coefficients;

[0035] This is the inherent adversarial loss of a GAN network that satisfies the requirements of generative adversarial communication:

[0036]

[0037] The calculation formula is as follows:

[0038]

[0039]

[0040] Among them, I iimask This means setting the RGB values ​​of the standard color chart area to 255 and the RGB values ​​of other areas to 0. The dot (·) indicates pixel multiplication. ii F and i ij ST These are the light source values ​​for generating the fake image and the ground truth (GT), respectively; ||.|| represents the L2 norm, and MEAN represents averaging, calculating the average of the angle errors of all pixels to obtain the final result.

[0041] L structure The structural loss is calculated using the following formula:

[0042]

[0043] In the above formula, i and j represent the indices of the pixel's horizontal and vertical coordinates, I F For the generated fake image, I GT Here, f is the ground truth image, f is the brightness value after converting the image from RGB space to Lab space, and L is the lightness value. structure For structural loss;

[0044] L global The overall loss is calculated using the following formula:

[0045]

[0046] The cosine similarity between two images is calculated to determine the overall similarity of the images, thereby achieving consistency constraints on generated fake images under ground truth supervision.

[0047] Furthermore, the discriminator's loss consists of two parts:

[0048]

[0049]

[0050] During training, E represents the expectation, and D... color is the feature fusion discriminator module, D is the discriminator in the generative adversarial network structure, and x represents the input.

[0051] On the other hand, the present invention also provides an image color correction system based on a dual discriminator network, comprising the following modules:

[0052] The dataset selection module is used to select color constancy datasets and perform data cleaning on the datasets;

[0053] The model building module is used to build a dual discriminator network model, which includes a basic generative adversarial network and a feature fusion discriminator module (FFD). The basic generative adversarial network includes a generator and a discriminator for generating and discriminating images. The feature fusion discriminator module is used to enhance the discriminator's ability to distinguish color features in images.

[0054] First, the color-biased image (Input) and its corresponding mask image are input into the generator in the Generative Adversarial Network (GAN) structure to generate a fake image (Fake). The generated fake image and the real image (GT) are then compared in the discriminator. Therefore, the generated fake image and the real image are respectively input into the discriminator and feature fusion discriminator modules in the GAN structure for discrimination. If both discriminators output "fake," the generator parameters are updated to generate a fake image with a better result; if both discriminators output "real," the generated fake image is the final output.

[0055] The image correction module is used to train the dual discriminator network model and use the trained model to achieve pixel-to-pixel and end-to-end color distortion correction.

[0056] Thirdly, the present invention also provides an electronic device, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the above-described image color correction method based on a dual discriminator network.

[0057] Compared with existing technologies, the advantages and beneficial effects of this invention are as follows: This invention proposes a Dual Discriminator Generative Adversarial Network (DDGAN), which includes a Color Feature Learning Module (CFL), a Feature Fusion Discriminator Module (FFD), and a Global Consistency Constraint Module (GCC). First, the CFL focuses on regions with uniform color, enabling the generator to learn distinguishable color information. Second, the FFD is a discriminator module containing two feature extraction branches: one branch extracts color features, and the other extracts globally relevant features. The FFD fuses these features together to weaken structural features and enhance the discriminator's sensitivity to color features. Finally, to obtain a more uniform color and content image, the GCC imposes a global consistency constraint on the structural features weakened by the FFD, unifying structural and color features, ultimately resulting in a high-precision image decolorization method. Attached Figure Description

[0058] Figure 1 This invention uses a dual discriminator network structure for uniform color information extraction.

[0059] Figure 2 This is the non-local attention module structure used in this invention;

[0060] Figure 3These are the experimental results of the Color Check RECommended dataset in this embodiment of the invention, including the original image, the generated image, and the ground truth image.

[0061] Figure 4 These are the experimental results of the Cube dataset in this embodiment of the invention, including the original image, the generated image, and the ground truth image.

[0062] Figure 5 These are experimental results of the Nus dataset in this embodiment of the invention, including the original image, the generated image, and the ground truth image. Detailed Implementation

[0063] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0064] This invention provides an image color correction method based on a dual-discriminator network. It proposes a dual-discriminator generative adversarial network (GAN) based on uniform color information, enabling the GAN network to generate images with quality close to the original image. This achieves pixel-to-pixel and end-to-end color cast correction. The network structure is as follows: Figure 1 As shown. Specifically, it includes the following steps:

[0065] Step 1: Set up a basic generative adversarial network, including a generator and a discriminator, for generating and judging images;

[0066] a. The generator adopts a U-net encoder-decoder structure. On the one hand, it can obtain the low-level features of the image through downsampling of the encoder; on the other hand, it can achieve feature preservation during the upsampling process of the decoder through skip connections.

[0067] b. The discriminator adopts the PatchGAN structure, which divides the image into multiple patches during image discrimination, ensuring the discrimination result of each patch while also focusing on the information of each pixel.

[0068] c. Introducing Angular Loss to impose overall constraints on image color information makes the network pay more attention to color information during image generation and discrimination, thus enhancing its sensitivity to color information.

[0069] Step 2 involves setting up a Color Feature Learning Module (CFL) to enable the network to focus on learning more uniform color regions. This distinguishable color information guides the network in identifying and learning key color features.

[0070] a) Use the standard color chart portion of the image that contains more uniform color information, and extract that portion using the mask corresponding to each image.

[0071] b. Add a mask loss function to narrow the gap between the light source values ​​of the masked part of the predicted image and the masked part of the ground truth image, and impose color information constraints.

[0072] Step 3: Set up a Feature Fusion Discriminator Module (FFD) to enhance the discriminator's ability to distinguish color features in the image;

[0073] a) Set a Gaussian blur branch for the image to weaken its structural features and enhance its color features.

[0074] b. Set up a feature fusion branch and use a non-local self-attention mechanism to extract global structural features of the image to ensure the internal correlation of pixels.

[0075] Step 4: Set up a Global Consistency Constraint Module (GCC) to improve the overall consistency of image structure and content.

[0076] a. Set a loss function that constrains the image structure. Ensure high consistency in image structure and texture.

[0077] b, Set a loss that constrains the overall consistency of the image. Ensure consistency in visual style.

[0078] The network consists of two parts: a generator and a discriminator. Image generation is achieved through a game-like competition between these two parts. First, the color-biased image (Input) and its corresponding mask image are input into the generator, which is an Encoder-Decoder structure. The CFL (Color Flicker Function) module acts as an auxiliary module to help the network learn color information, and the generator generates fake images (Fake). These fake images then compete against the ground truth (GT) image in the discriminator. Therefore, the generated fake image and the GT image are input together into the FFD (Fair Decoder) module for discrimination. If the discriminator outputs False, the generator needs to update its parameters to generate better fake images; if the discriminator outputs True, the generated fake image can be output as the final result.

[0079] This invention utilizes a computer for network training and inference, implemented using the PyTorch deep learning framework on the Ubuntu operating system. During the experimental training process, the Adam optimizer was employed, and considering factors such as dataset size and gradient descent, the number of epochs was set to 200. The learning rate was kept constant at 0.0002 for the first 100 epochs, and then gradually decreased to 0 in the subsequent 100 epochs.

[0080] This example uses the constructed model to conduct experiments and evaluations on three standard single-light source color constancy datasets. The datasets include Color Check, Nus, and Cube+. The Color Check dataset contains 568 real-world images. Each image includes a standard color chart to provide the true ground truth (GT) illumination value for each image. The Nus dataset contains eight subsets, each taken with a different camera model, capturing images of real-world scenes. Each image in the Nus dataset also includes a standard color chart. The Cube+ dataset contains 1707 indoor and outdoor photographs from various scenes. Each photograph has a grey ball in the lower right corner, which functions similarly to the standard color chart to determine the GT illumination value.

[0081] In the preprocessing step, the dataset is first cleaned, primarily by removing outlier images from the Cube+ dataset. For all images across all datasets, the standard color chart or Grey Ball portion, containing more uniform color information, is extracted and saved as a mask image. Since some datasets have insufficient training data, data augmentation is performed to increase the training data, and this augmentation is applied to all datasets used for training. The dataset is randomly divided into an 80% training set and a 20% test set. The standard color chart or Grey Ball portion is removed from the images in the test set to avoid relying on this uniform information during testing, which could negatively impact the overall test results.

[0082] (1) In the CFL module of this example, additional weights are added to the standard color chart portion of the image. The standard color chart is a tool for achieving a unified standard for color within a certain range, containing more uniform and distinguishable information. The standard color chart region in the image is extracted using a mask, the illumination value of each pixel in this region is calculated, and the similarity of illumination values ​​is estimated by calculating the average angular error of the pixels.

[0083] The specific calculation formula is as follows:

[0084]

[0085] I ij maskThis sets the RGB values ​​of the standard color chart area to 255, and the RGB values ​​of other areas to 0. · indicates pixel multiplication, extracting the standard color chart area by multiplying pixels one by one. ij F and i ij GT These are the light source values ​​for generating the fake image and the ground plane, respectively.

[0086]

[0087] ||.|| represents the L2 norm, and the final value is obtained by calculating the average of the angle errors of all pixels.

[0088] (2) In the FFD module of this example, the fake images G(x) and GT generated by the generator are directly input into the network, and the two images enter the two branches simultaneously. In the Gaussian image feature extraction branch, the image is Gaussian blurred through convolution. The blurred image becomes smooth, and the main structured features such as texture and content are weakened, retaining only relatively complete color features. The blurred image is processed through three layers of downsampling convolution to obtain a feature map containing better distinguishable color information. The Gaussian blur convolution uses a two-dimensional normally distributed convolution kernel, and the specific formula is as follows:

[0089]

[0090] In the above formula, x c y c These represent the horizontal and vertical coordinates of the corresponding image pixels, respectively. Gaussian represents Gaussian blur, and σ represents the blur radius.

[0091] However, Gaussian blurring may destroy the potential connections between pixels, thus affecting the discriminator's judgment. Therefore, a branch for extracting full pixel relationship features is constructed to assist the network's judgment. This branch extracts full pixel relationships through a non-local attention module, specifically by calculating the correlation between a pixel in the image and all other pixels.

[0092] Depend on Figure 2 As shown in the diagram, a feature map of size x is input into a non-local attention module and linearly mapped (using a 1×1×1 convolution) to features of different dimensions, denoted as θ, Φ, and g from left to right. Then, a dot product is performed on θ and Φ to obtain the similarity values ​​between pixels. A softmax operation is then performed to obtain weights of 0 to 1, representing the correlation values ​​between pixels. Finally, these weights are multiplied back into the feature matrix g to obtain the output feature map y.

[0093] Nonlocal operations are defined as follows:

[0094]

[0095] In the above formula, x represents the input feature map, i represents the pixel index, j represents all pixels in the image except i, and y represents the output feature map. Function f calculates the correlation between all pixels, and function g represents the input information for pixel j. It is a normalization function. The above formula achieves the output of a full-pixel relationship with the same input size.

[0096] In this example, function f directly uses the dot product method, while function g uses simple linear embedding and a normalization function. This is achieved using softmax. The specific formula is:

[0097] f(x i x j )=θ(x i ) T θ(x j )

[0098] g(x j ) = W g x j

[0099] In the above formula, θ represents a 1×1×1 convolution, and the feature map is multiplied by its transpose after passing through the convolutional layer to obtain the full pixel relationship. g This is the weight matrix to be learned.

[0100] Features extracted from global pixel correlation are fused with features extracted from Gaussian blur blocks to aid in the judgment. Feature fusion is achieved through non-local residual connections.

[0101] Z i =W z y i +Gauss(x i )

[0102] W z This is a weight matrix to be learned for feature transformation. +Gauss(x) i Feature fusion is achieved by adding the features extracted by Gaussian blur with residual connections.

[0103] The fused features are input into the final convolutional layer to determine whether a feature is real or fake. A color-focused feature fusion discriminator (FFD) improves sensitivity to color information while maintaining pixel correlation, meeting the network requirements of the color-sensitive (CC) task.

[0104] (3) The GCC module in this example consists of two parts: one is the structural constraints on the image to ensure a high degree of consistency in the image's structure and texture; the other is the overall consistency constraints on the image to ensure a unified image style. The specific implementation is as follows:

[0105] Structural constraints on images are achieved by calculating the similarity of illumination. The image is then converted from RGB space to Lab space. After conversion to Lab space, the a* and b* values ​​represent the color component values, and L... * This represents the brightness value. To constrain the structural consistency of the image, the color component information is ignored, and the brightness value is used to calculate the error. The calculation formula is as follows:

[0106]

[0107] In the above formula, i and j represent the indices of the pixel's horizontal and vertical coordinates, I F For the generated fake image, I GT Here is the ground truth image, and l is the brightness value L after conversion to Lab color space.

[0108] Consistency constraints on the overall image are achieved by calculating the overall similarity of the images.

[0109]

[0110] The cosine similarity between two images is calculated to determine their overall similarity, thus achieving consistency constraints on generated fake images under ground truth (GT) supervision. This consistency constraint ensures that the overall style of the generated images remains unchanged and is close to the GT.

[0111] (4) In this example, the loss function of the dual-discriminator network model can be expressed as follows:

[0112]

[0113] E represents the expectation, and the generator G consists of two discriminators (D, D). color The algorithm employs a game-like process of interaction and competition to generate images that most closely resemble real images. Content constraints are imposed on the generated fake images, specifically implemented as follows:

[0114]

[0115] In the above formula, This represents the loss function in the generator. As mentioned above. This is a constraint on the overall angular error between the generated fake image and the ground truth.

[0116] This is the inherent adversarial loss of a GAN network that satisfies the requirements of generative adversarial communication:

[0117]

[0118] The discriminator's loss mainly includes D, D color Two parts:

[0119]

[0120]

[0121] During training, the weight coefficients λ1, λ2, and λ3 in the loss function are all set to 2. The ratio of the color discriminator's (ColorD) loss to that of the original discriminator is 2:1, ensuring the overall image generation quality while using the color discriminator to assist in the judgment of color information.

[0122] Figure 3-5 This is a comparison of the original image, the generated image, and the ground truth image in the experiment.

[0123] In another embodiment, the present invention also provides an image color correction system based on a dual discriminator network, comprising the following modules:

[0124] The dataset selection module is used to select color constancy datasets and perform data cleaning on the datasets;

[0125] The model building module is used to build a dual discriminator network model, which includes a basic generative adversarial network and a feature fusion discriminator module (FFD). The basic generative adversarial network includes a generator and a discriminator for generating and discriminating images. The feature fusion discriminator module is used to enhance the discriminator's ability to distinguish color features in images.

[0126] First, the color-biased image (Input) and its corresponding mask image are input into the generator in the Generative Adversarial Network (GAN) structure to generate a fake image (Fake). The generated fake image and the real image (GT) are then compared in the discriminator. Therefore, the generated fake image and the real image are respectively input into the discriminator and feature fusion discriminator modules in the GAN structure for discrimination. If both discriminators output "fake," the generator parameters are updated to generate a fake image with a better result; if both discriminators output "real," the generated fake image is the final output.

[0127] The image correction module is used to train the dual discriminator network model and use the trained model to achieve pixel-to-pixel and end-to-end color distortion correction.

[0128] In another embodiment, the present invention also provides an electronic device, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the above-described image color correction method based on a dual discriminator network.

[0129] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. An image color correction method based on a dual discriminator network, characterized in that, Includes the following steps: Step 1: Select the color constancy dataset and preprocess the dataset; Step 2: Construct a dual discriminator network model. The dual discriminator network model includes a basic generative adversarial network and a feature fusion discriminator module (FFD). The basic generative adversarial network includes a generator and a discriminator for generating and discriminating images. The feature fusion discriminator module is used to enhance the discriminator's ability to distinguish color features in images. First, the color-biased image (Input) and its corresponding mask image are input into the generator in the Generative Adversarial Network (GAN) structure to generate a fake image (Fake). The generated fake image and the real image (GT) are then compared in the discriminator. Therefore, the generated fake image and the real image are respectively input into the discriminator and feature fusion discriminator modules in the GAN structure for discrimination. If both discriminators output "fake," the generator parameters are updated to generate a fake image with a better result; if both discriminators output "real," the generated fake image is the final output. The specific processing procedure of the FFD module is as follows; The FFD module consists of two branches: a Gaussian image feature extraction branch and a full-pixel relation feature extraction branch. In the Gaussian image feature extraction branch, the input is first subjected to Gaussian blur through convolution, and then subjected to three layers of downsampling convolution to obtain a feature map containing better distinguishable color information. The convolution in the Gaussian blur uses a two-dimensional normally distributed convolution kernel, and the specific formula is as follows: In the above formula, x c y c These represent the horizontal and vertical coordinates of the corresponding image pixels, Gaussian represents Gaussian blur, and σ represents the blur radius. The full-pixel relation feature extraction branch achieves full-pixel relation extraction through non-local attention. Specifically, it calculates the correlation between a pixel in the image and all other pixels. The non-local operation is defined as follows: In the above formula, x represents the input feature map, i represents the pixel index, j represents all pixels in the image except i, and y represents the output feature map; function f calculates the correlation between all pixels, and function g represents the input information of pixel j. It is a normalization function, and the above formula achieves the output of the same full-pixel relationship as the input size; Features extracted from global pixel correlation are fused with features extracted from Gaussian blur blocks to jointly aid in the judgment. Feature fusion is achieved through non-local residual connections. Z i =W z y i +Gauss(x i ) W z It is a weight matrix to be learned for feature transformation, +Gauss(x) i Feature fusion is achieved by adding the features extracted by residual connection and Gaussian blur. The fused features are input into the final convolutional layer to determine whether a value is true or false. Step 3: Train the dual discriminator network model and use the trained model to perform pixel-to-pixel and end-to-end color cast correction.

2. The image color correction method based on a dual discriminator network according to claim 1, characterized in that: The preprocessing includes removing anomalous images from the dataset and performing data augmentation on the dataset.

3. The image color correction method based on a dual discriminator network according to claim 1, characterized in that: Function f directly uses the dot product method, while function g uses simple linear embedding and normalization function. This is achieved using softmax, with the specific formula as follows: f(x i ,x j )=θ(x i ) T θ(x j ) g(x j )=W g x j In the above formula, θ represents a 1×1×1 convolution. The feature map is multiplied by its transpose after passing through the convolutional layer to obtain the full pixel relationship. W g This is the weight matrix to be learned.

4. The image color correction method based on a dual discriminator network according to claim 1, characterized in that: In step 3, the dual-discriminator network model is expressed as follows in the case of dual discriminators: Where E represents the expectation, the generator G and two discriminators (D, D) color The goal is to generate images that are as close as possible to real images through a game-like process, where D is the discriminator in the generative adversarial network (GAN) structure. color is the feature fusion discriminator module, where x represents the input.

5. The image color correction method based on a dual discriminator network according to claim 1, characterized in that: During training, the generator uses the following loss function: In the above formula, This represents the loss function in the generator. It is a constraint on the overall angular error between the generated fake image and the ground plane, where λ1, λ2, and λ3 are weighting coefficients; This is the inherent adversarial loss of a GAN network that satisfies the requirements of generative adversarial communication: The calculation formula is as follows: Among them, I ij mask This means setting the RGB values ​​of the standard color chart area to 255 and the RGB values ​​of other areas to 0. The dot (·) indicates pixel multiplication. ij F and i ij GT These are the light source values ​​for generating the fake image and the ground truth (GT), respectively; ||.|| represents the L2 norm, and MEAN represents averaging, calculating the average of the angle errors of all pixels to obtain the final result. L structure The structural loss is calculated using the following formula: In the above formula, i and j represent the indices of the pixel's horizontal and vertical coordinates, I F For the generated fake image, I GT Here, L represents the ground truth image, and l is the luminance value after converting the image from RGB space to Lab space. structure For structural loss; L global The overall loss is calculated using the following formula: The cosine similarity between two images is calculated to determine the overall similarity of the images, thereby achieving consistency constraints on generated fake images under ground truth supervision.

6. The image color correction method based on a dual discriminator network according to claim 1, characterized in that: The discriminator's loss consists of two parts: During training, E represents the expectation, and D... color is the feature fusion discriminator module, D is the discriminator in the generative adversarial network structure, and x represents the input.

7. An image color correction system based on a dual discriminator network, characterized in that, Includes the following modules: The dataset selection module is used to select color constancy datasets and perform data cleaning on the datasets; The model building module is used to build a dual discriminator network model, which includes a basic generative adversarial network and a feature fusion discriminator module (FFD). The basic generative adversarial network includes a generator and a discriminator for generating and discriminating images. The feature fusion discriminator module is used to enhance the discriminator's ability to distinguish color features in images. First, the color-biased image (Input) and its corresponding mask image are input into the generator in the Generative Adversarial Network (GAN) structure to generate a fake image (Fake). The generated fake image and the real image (GT) are then compared in the discriminator. Therefore, the generated fake image and the real image are respectively input into the discriminator and feature fusion discriminator modules in the GAN structure for discrimination. If both discriminators output "fake," the generator parameters are updated to generate a fake image with a better result; if both discriminators output "real," the generated fake image is the final output. The specific processing procedure of the FFD module is as follows; The FFD module consists of two branches: a Gaussian image feature extraction branch and a full-pixel relation feature extraction branch. In the Gaussian image feature extraction branch, the input is first subjected to Gaussian blur through convolution, and then subjected to three layers of downsampling convolution to obtain a feature map containing better distinguishable color information. The convolution in the Gaussian blur uses a two-dimensional normally distributed convolution kernel, and the specific formula is as follows: In the above formula, x c y c These represent the horizontal and vertical coordinates of the corresponding image pixels, Gaussian represents Gaussian blur, and σ represents the blur radius. The full-pixel relation feature extraction branch achieves full-pixel relation extraction through non-local attention. Specifically, it calculates the correlation between a pixel in the image and all other pixels. The non-local operation is defined as follows: In the above formula, x represents the input feature map, i represents the pixel index, j represents all pixels in the image except i, and y represents the output feature map; function f calculates the correlation between all pixels, and function g represents the input information of pixel j. It is a normalization function, and the above formula achieves the output of the same full-pixel relationship as the input size; Features extracted from global pixel correlation are fused with features extracted from Gaussian blur blocks to jointly aid in the judgment. Feature fusion is achieved through non-local residual connections. Z i =W z y i +Gauss(x i ) W z It is a weight matrix to be learned for feature transformation, +Gauss(x) i Feature fusion is achieved by adding the features extracted by residual connection and Gaussian blur. The fused features are input into the final convolutional layer to determine whether a value is true or false. The image correction module is used to train the dual discriminator network model and use the trained model to achieve pixel-to-pixel and end-to-end color distortion correction.

8. An electronic device, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the program instructions in the memory to execute the image color correction method based on a dual discriminator network as described in any one of claims 1-6.

Images