Multi-light source auto white balance method and system

Abstract

The application discloses a multi-light source automatic white balance method, and specifically comprises the following steps: acquiring an input image to be processed; performing feature extraction on the input image, inputting the input image into an encoder-decoder network, extracting multi-scale image features containing global semantics and local textures, and obtaining bottleneck features; performing global chroma base estimation, performing global pooling on the bottleneck features, and outputting K global chroma bases through a fully connected layer; performing spatial weight estimation, outputting K spatial weight response maps corresponding to the K global chroma bases from a decoding end of the encoder-decoder network, and performing normalization on the K spatial weight response maps in the base dimension; generating an illumination chroma map, multiplying each global chroma base and the corresponding spatial weight map pixel by pixel, and summing to generate a two-dimensional spatial illumination chroma map; performing white balance correction, performing pixel-by-pixel color correction on the input image, and outputting a white balance corrected image.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing and computational photography technology, specifically relating to an automatic white balance method for spatial changes applicable to scenarios such as mixed indoor and outdoor light sources, and the coexistence of window sunlight and artificial light sources, as well as an image processing system for implementing this method. Background Technology

[0002] The automatic white balance (AWB) in existing camera image signal processing (ISP) workflows typically assumes that the entire image is illuminated by a single light source and estimates a global gain for the red, green, and blue channels. This assumption works under uniform lighting conditions, but in multi-light source scenes, different areas of the same object may be affected by cool sunlight, warm artificial light, or a mixture of these. A single global gain can cause some areas to be over-corrected and others under-corrected, resulting in blue cast, yellow cast, or local color discontinuities.

[0003] Existing technical solutions include pixel-level fusion of preset white balance results, direct regression of pixel-by-pixel illumination maps, and implicit multi-light source modeling based on attention or iterative slot decomposition. For example, CN115988335A proposes an image white balance correction method for multiple light sources, which corrects the image by calculating the influence weight of ambient light sources at different locations on each pixel position in the captured image. CN102892010A achieves separate adjustment of different color temperature regions by dividing the image to be processed into multiple sub-blocks. In addition, there are also solutions that process the image based on preset white balance settings and then input it into a deep neural network to generate a weight map for fusion.

[0004] However, each of the above-mentioned approaches has its shortcomings: preset fusion is limited by the discrete color temperature range; pixel-by-pixel regression is prone to noise and spatial instability; complex attention or iterative structures have a large number of parameters and computational costs, making the training process difficult to stabilize. In addition, the high cost of collecting real multi-light source labeled data restricts the generalization ability of deep learning-based methods under real complex lighting conditions.

[0005] Therefore, it is indeed necessary to provide a multi-source automatic white balance method and system to solve the above problems. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a multi-source automatic white balance method based on the joint estimation of global chromaticity base and spatial weight, which addresses the shortcomings of the prior art.

[0007] To achieve the above objectives, the present invention provides a multi-source automatic white balance method, specifically including the following steps:

[0008] Acquire the input image to be processed, which is an sRGB image, a linear RGB image, or an image obtained from camera RAW through basic ISP processing;

[0009] Feature extraction is performed on the input image by inputting the input image into an encoder-decoder network to extract multi-scale image features containing global semantics and local texture, thereby obtaining bottleneck features;

[0010] Global chromaticity base estimation is performed, the bottleneck features are globally pooled, and K global chromaticity bases are output through a fully connected layer, where K is an integer greater than 1. Each global chromaticity base is shared throughout the entire input image and is used to characterize the chromaticity of the dominant light source in the scene.

[0011] Spatial weight estimation is performed by outputting K spatial weight response maps corresponding one-to-one with the K global chromaticity bases from the decoder end of the encoder-decoder network. The K spatial weight response maps are normalized in the base dimension so that the weights at each pixel position are non-negative and the sum is 1.

[0012] A two-dimensional spatial illumination chromaticity map is generated by multiplying each global chromaticity base with its corresponding spatial weight map pixel by pixel and summing the results.

[0013] White balance correction involves determining the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity map, performing pixel-by-pixel color correction on the input image, and outputting a white balance-corrected image.

[0014] As a further improvement of the present invention, the global chromaticity base is a chromaticity coordinate normalized to the green channel. This indicates that the two-dimensional spatial illumination chromaticity diagram Calculate according to the following formula:

[0015] ,

[0016] in, For pixel coordinates, Let k be the global chromaticity base. For the first Each spatial weight map at pixels The weight value at that point, and .

[0017] As a further improvement of the present invention, the encoder-decoder network comprises multiple resolution levels, each level consisting of lightweight residual feature blocks and downsampling units; the decoder employs upsampling units and makes skip connections with the corresponding coding layers; global pooling is performed on the bottleneck features, specifically: for features of size... The bottleneck features described above are subjected to global average pooling in the spatial dimension to obtain dimensional vector, and then the above A 3D vector is input to at least one fully connected network, and the output is... Dimensional parameters as The global chromaticity base is described above.

[0018] As a further improvement of the present invention, the white balance correction specifically includes: adjusting the two-dimensional spatial illumination chromaticity diagram. Expanded to three-channel relative illumination vectors Calculate the output pixels using the following formula: ,in, The input image contains the red, green, and blue channel values. The output image contains red, green, and blue channel values, with ε being a small constant to prevent division by zero. Exposure preservation, range cropping, and tone mapping are then applied to each channel after pixel-by-pixel correction.

[0019] As a further improvement of the present invention, a training data generation step is also included:

[0020] Obtain a properly colored sRGB image as the ground truth image;

[0021] Perform inverse ISP processing on the ground truth image to approximately restore it to the camera's RAW domain representation;

[0022] Apply a global channel gain perturbation to the three channels in the RAW domain;

[0023] Generate N virtual light sources, N≥1, each of which includes at least color temperature or RGB gain, incident direction, intensity, anchor point position, influence range and attenuation function type;

[0024] Based on the incident direction and anchor point position of each virtual light source, calculate the spatial mask of each virtual light source at each pixel of the image;

[0025] The RGB gain of each virtual light source at the pixel is normalized and weighted according to the spatial mask, and then multiplied with the RAW domain image channel by channel to obtain the RAW image after multi-light source perturbation.

[0026] The RAW image after multi-source perturbation is subjected to forward ISP processing, and the composite input image is rendered.

[0027] Based on the chromaticity base of each virtual light source and the spatial mask, an illumination chromaticity map corresponding to the synthesized input image is generated;

[0028] The synthesized input image is paired with the ground truth image and the illumination chromaticity map to serve as training sample pairs.

[0029] As a further improvement of the present invention, the reverse ISP processing specifically includes: performing inverse gamma correction, inverse tone mapping and linear color space conversion on the true sRGB values ​​in sequence; after converting the linear sRGB to the XYZ color space, mapping it to the camera RGB domain using the inverse matrix or pseudo-inverse matrix of the camera color correction matrix.

[0030] As a further improvement of the present invention, the forward ISP processing includes: sequentially performing demosaicing, color matrix transformation, white point and exposure control, noise processing, tone mapping and gamma encoding.

[0031] As a further improvement of the present invention, the global channel gain perturbation includes: setting D g =1,D r and D β Random sampling is performed within the range of 0.4 to 2.2; the color temperature is sampled between 1000K and 12000K and converted into normalized RGB gain through a blackbody radiation approximation model; the spatial mask is determined by both directional attenuation and distance attenuation, and the distance attenuation adopts any one of Gaussian attenuation, linear attenuation, quadratic attenuation or inverse square attenuation.

[0032] Another object of the present invention is to provide a multi-source automatic white balance system.

[0033] To achieve the above objectives, the present invention provides a multi-source automatic white balance system, comprising:

[0034] The image acquisition module is used to acquire the input image to be processed;

[0035] A multi-scale feature extraction module, comprising an encoder-decoder network, is used to extract multi-scale image features from the input image;

[0036] The global chromaticity base estimation module is connected to the bottleneck of the encoder-decoder network. It is used to perform global pooling on the bottleneck features and output K global chromaticity bases through a fully connected mapping network, where K is an integer greater than 1.

[0037] The spatial weight estimation module is connected to the decoding end of the encoder-decoder network. It is used to output K spatial weight response maps that correspond one-to-one with the K global chromaticity bases, and to perform normalization on the K spatial weight response maps in the base dimension.

[0038] The illumination map fusion module is used to multiply each global chromaticity base with its corresponding spatial weight map pixel by pixel and then sum them to generate a two-dimensional spatial illumination chromaticity map;

[0039] The white balance correction module is used to determine the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity diagram, perform pixel-by-pixel color correction on the input image, and output the white balance corrected image.

[0040] As a further improvement of the present invention, a training data generation module is also included for generating training data pairs.

[0041] In summary, the multi-source automatic white balance method and system provided by this invention jointly estimates the global chromaticity base and spatial weight map in a single forward inference through an encoder-decoder network, and generates a spatial illumination chromaticity map using convex combinations, achieving continuous, smooth, and interpretable spatial variation white balance correction. This invention does not rely on a preset white balance template, does not require displaying light source counts, has a smaller number of parameters and computational load, and is easy to deploy in real time. Furthermore, the multi-source training data synthesis method based on inverse / forward ISP effectively alleviates the problem of insufficient real training data, improves the model's generalization ability in complex mixed lighting scenarios, and has good prospects for industrial applications. Attached Figure Description

[0042] The above-described technical content of the present invention and the following detailed embodiments will be better understood when read in conjunction with the accompanying drawings. It should be noted that the drawings are merely examples of the claimed technical solutions. In the drawings, the same reference numerals represent the same or similar elements.

[0043] Figure 1 This is a flowchart of the multi-source automatic white balance method of the present invention.

[0044] Figure 2 This is a flowchart of the data generation step in the multi-source automatic white balance method of the present invention.

[0045] Figure 3 This is a structural diagram of the encoder-decoder network in the multi-source automatic white balance method of the present invention. Detailed Implementation

[0046] The following detailed description of the features and advantages of the present invention is sufficient to enable those skilled in the art to understand the technical content of the present invention and to implement it accordingly. Furthermore, based on this specification, claims, and drawings, those skilled in the art can easily understand the related objectives and advantages of the present invention.

[0047] The terminology and expressions used herein are for descriptive purposes only, and the invention should not be limited to these terms and expressions. The use of these terms and expressions does not mean the exclusion of any illustrative and descriptive equivalent features (or parts thereof), and it should be recognized that various modifications that may exist should also be included within the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be considered to cover all such equivalents.

[0048] Please refer to Figures 1 to 3 This invention provides a multi-light source automatic white balance method, which specifically includes the following steps:

[0049] Image Acquisition: Acquire the input image to be processed. The input image is an sRGB image, a linear RGB image, or an image obtained from camera RAW through basic ISP processing, and normalize it to a predetermined numerical range.

[0050] Feature extraction: The input image is input into an encoder-decoder network, and the encoder extracts multi-scale image features containing global semantics and local texture to obtain bottleneck features.

[0051] Global chromaticity basis estimation: The bottleneck features are globally pooled, and K global chromaticity bases are output through a fully connected layer, where K is an integer greater than 1. Each global chromaticity base is shared across the entire input image and is used to characterize the chromaticity of the dominant light source in the scene. Preferably, the global chromaticity bases are chromaticity coordinates normalized to the green channel. express.

[0052] Spatial weight estimation: The encoder-decoder network outputs K spatial weight response maps, each corresponding one-to-one with one of the K global chromaticity bases. These K spatial weight response maps are normalized along their base dimensions, ensuring that the weights at each pixel position are non-negative and sum to 1. Preferably, the normalization is implemented using the Softmax function.

[0053] Illumination map generation: Each global chromaticity base is multiplied pixel by pixel with its corresponding spatial weight map and then summed to generate a two-dimensional spatial illumination chromaticity map. The two-dimensional spatial illumination chromaticity diagram Calculate according to the following formula:

[0054] ,

[0055] in, For pixel coordinates, Let k be the global chromaticity base. For the k-th spatial weight map at pixel The weight value at that point, and .

[0056] White balance correction: Determine the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity map, perform pixel-by-pixel color correction on the input image, and output the white balance corrected image.

[0057] Preferably, the white balance correction specifically includes: expanding the two-dimensional spatial illumination chromaticity map m(i,j) into a three-channel relative illumination vector. Calculate the output pixels using the following formula: ,

[0058] in, The input image contains the red, green, and blue channel values. To output the red, green, and blue channel values ​​of the image, To prevent small constants from being divided by zero, exposure holding, range cropping, and tone mapping are then performed on each channel after pixel-by-pixel correction.

[0059] When the input image is a non-linear sRGB image, it can be linearized first, then corrected, and then the color matrix, gamma curve, and tone curve can be reapplied; alternatively, an equivalent sRGB domain mapping can be learned directly during network training. All of the above methods are equivalent implementations of local color adaptation using the estimated spatial illumination map.

[0060] Furthermore, the encoder-decoder network comprises multiple resolution levels, each consisting of lightweight residual feature blocks and downsampling units. Preferably, the lightweight residual feature blocks employ NAFBlock; depthwise separable convolutional blocks, inverted residual blocks, or other structures with equivalent feature extraction capabilities may also be used. The decoder employs upsampling units and skips connections with the corresponding coding layers to recover spatial details of edges, object regions, and illumination transition zones.

[0061] The global chromaticity basis estimation branch is connected to the bottleneck of the encoder-decoder network. Let the bottleneck feature size be C. β ×H β ×W β First, perform global average pooling in the spatial dimension to obtain C. β dimensional vector, and then the C β The input vector is at least one fully connected layer, and the output K×2 dimensional parameters serve as K global chromaticity bases. The output can be directly represented. Alternatively, positive parameters can be output first, and their physical range can be limited by exponentiation, Softplus, or interval mapping.

[0062] The spatial weight estimation branch is connected to the decoding end of the encoder-decoder network. K-channel response maps are output from the convolutional layer, and Softmax is performed along the K-channel directions to obtain the spatial weight maps. To further improve spatial continuity, total variation regularization, edge-aware smoothing constraints, or multi-scale consistency loss can optionally be added, but these additional constraints are not necessary conditions for achieving the basic functions of this invention.

[0063] Preferably, the present invention also provides a method for synthesizing multi-source training data based on imaging physics and camera ISP, for preparing training sample pairs. The method includes the following steps:

[0064] Obtain the ground truth image: Obtain a normally colored sRGB image as the ground truth image.

[0065] Inverse ISP and RAW Approximate Restoration: Inverse ISP processing is performed on the ground-value image to approximately restore it to the camera's RAW domain representation. Specifically, inverse gamma correction, inverse tone mapping, and linear color space conversion are sequentially performed on the ground-value sRGB; after converting the linear sRGB to the XYZ color space, it is mapped to the camera's RGB domain using the inverse or pseudo-inverse matrix of the camera's color correction matrix. Optionally, mosaic sampling, demosaicing, readout noise, and shot noise are simulated to make the data conform to the sensor's imaging characteristics.

[0066] Global color cast perturbation: Apply a global channel gain perturbation to the three channels in the RAW domain. Preferably, set D... g =1,D r and D β Random sampling was performed within the range of 0.4 to 2.2.

[0067] Virtual light source generation: Generate N virtual light sources, N≥1, preferably N∈{1,2,3}. Each virtual light source includes at least color temperature or RGB gain, incident direction, intensity, anchor point position, influence range, and attenuation function type. The color temperature is sampled between 1000K and 12000K and converted to normalized RGB gain using a blackbody radiation approximation model.

[0068] Spatial mask calculation: Based on the incident direction and anchor point position of each virtual light source, calculate the spatial mask of each virtual light source at each pixel of the image. The spatial mask is determined by both directional attenuation and distance attenuation, wherein the distance attenuation adopts any one of Gaussian attenuation, linear attenuation, quadratic attenuation, or inverse square attenuation.

[0069] Multi-light source fusion: The RGB gain of each virtual light source at the pixel is normalized and weighted according to the spatial mask, and then multiplied with the RAW domain image channel by channel to obtain the RAW image after multi-light source perturbation.

[0070] Forward ISP rendering: The RAW image after multi-source perturbation is processed by forward ISP to render a composite input image. The forward ISP processing includes: sequentially performing desacrifice, color matrix transformation, white point and exposure control, noise reduction, tone mapping, and gamma encoding.

[0071] Supervision label generation: Based on the chromaticity base of each virtual light source and the spatial mask, generate an illumination chromaticity map corresponding to the synthesized input image.

[0072] Training sample construction: The synthetic input image is paired with the ground truth image and the illumination chromaticity map to form training sample pairs.

[0073] During training, pre-training can be performed using large-scale synthetic data, followed by fine-tuning on a small amount of real multi-source data. The loss function should include at least the L1 / L2 loss of the illumination map or output image, and may also include perceptual loss, color difference loss, adversarial loss, and weight map smoothing loss.

[0074] The present invention also provides a multi-source automatic white balance system, comprising: an image acquisition module, a multi-scale feature extraction module, a global chromaticity basis estimation module, a spatial weight estimation module, an illumination map fusion module, a white balance correction module, and an optional training data generation module.

[0075] The image acquisition module acquires the input image to be processed. The multi-scale feature extraction module, containing an encoder-decoder network, extracts multi-scale image features from the input image. The global chromaticity basis estimation module, connected to the bottleneck of the encoder-decoder network, performs global pooling on the bottleneck features and outputs K global chromaticity bases (K being an integer greater than 1) through a fully connected mapping network. The spatial weight estimation module, connected to the decoder of the encoder-decoder network, outputs K spatial weight response maps corresponding one-to-one with the K global chromaticity bases and normalizes these K spatial weight response maps in the base dimension. The illumination map fusion module multiplies each global chromaticity base pixel-by-pixel with its corresponding spatial weight map and sums the results to generate a two-dimensional spatial illumination chromaticity map. The white balance correction module determines the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity map, performs pixel-by-pixel color correction on the input image, and outputs a white balance-corrected image. The training data generation module generates training data pairs.

[0076] Each module can be implemented by software instructions executed by the same processor, or it can be deployed on a camera ISP, mobile NPU, GPU, edge computing chip, or cloud image processing service.

[0077] The following are specific embodiments of the multi-source automatic white balance method of the present invention.

[0078] Example 1: Reasoning Example

[0079] An indoor mixed lighting image with a resolution of H×W is input into the network. The number of encoder channels can be set to C, 2C, 4C, 8C, and 16C, with K preferably being 7. The bottleneck branch outputs 7 global chromaticity bases, and the decoding branch outputs 7 H×W weight maps. Weighted fusion is performed on each pixel to obtain the lighting chromaticity, and then white balance is completed according to the local channel gain. Since the number of chromaticity bases is fixed, the number of light sources in the actual scene does not need to be predetermined; when a certain base is not needed, its weight can be close to zero in the entire image. For example, in a uniform scene illuminated only by a single light source, the weight of only one chromaticity base is close to 1 in the entire image, and the weights of the other 6 chromaticity bases are close to 0. The system degenerates into global white balance, but can still work normally.

[0080] In a specific example, the input image is an office scene with mixed lighting from window sunlight (color temperature 5500K) and indoor warm-colored LED lights (color temperature 2800K). The left side of the image is dominated by sunlight, the right side by warm light, and a mixed transition zone is formed in the middle. After forward inference by the encoder-decoder network, among the K=7 chromaticity bases, two chromaticity bases correspond to the daylight color temperature and the warm light color temperature, respectively. In the left region, the weight of the daylight chromaticity base is close to 1, and the weight of the warm light chromaticity base is close to 0; the opposite is true for the right region; the weights of the two chromaticity bases in the middle transition zone show a smooth gradient. The lighting chromaticity map generated by weighted fusion changes continuously in space. After pixel-by-pixel correction, both the window area and the light area are restored to neutral colors, and no color jumps or artifacts appear in the transition zone.

[0081] Example 2: System Example

[0082] The system for implementing the method of the present invention includes: an image acquisition module, a multi-scale feature extraction module, a global chromaticity basis estimation module, a spatial weight estimation module, an illumination map fusion module, a white balance correction module, and an optional training data generation module.

[0083] In one specific embodiment, the system is deployed in a camera ISP chip, which integrates a white balance processing circuit configured to perform the aforementioned method steps. The image acquisition module receives RAW data or RGB images processed by the basic ISP from the camera sensor; each module executes in a pipelined sequence; the white balance correction module outputs the corrected image for subsequent ISP modules (such as color enhancement, sharpening, compression, etc.) to process.

[0084] In another embodiment, the system is deployed on a mobile NPU or GPU, and performs multi-source white balance correction on real-time captured images or locally stored images by loading pre-trained encoder-decoder network weights. The system runs in real-time on the mobile device at a frame rate of over 30fps, meeting the needs of video shooting and previewing scenarios.

[0085] In another embodiment, the system is deployed on a cloud-based image processing server, receives images uploaded by users via an application programming interface (API), performs white balance correction, and returns the correction result. This cloud-based deployment is suitable for terminal devices with limited computing resources or for backend service scenarios requiring batch processing of large numbers of images.

[0086] The training data generation module runs offline and can generate synthetic training data in batches during the development or model update phases. Specifically, the module acquires a library of normally colored sRGB images, performs inverse ISP on each image to recover the RAW domain representation, applies randomized global gain perturbation and spatially varied multi-source perturbation (1-3 sources, randomized color temperature sampling from 1000K to 12000K, randomized incident direction and anchor point position, and randomized attenuation type) to the RAW domain, and then performs forward ISP rendering to obtain the synthetic input image. Simultaneously, it generates corresponding illumination chromaticity maps as supervision labels based on the virtual light source parameters. The generated training sample pairs are used to pre-train the encoder-decoder network, which can then be fine-tuned on a small amount of real multi-source data to further improve the model's generalization performance in real-world scenes.

[0087] In summary, the multi-source automatic white balance method and system provided by this invention jointly estimates the global chromaticity base and spatial weight map in a single forward inference through an encoder-decoder network, and generates a spatial illumination chromaticity map using convex combinations, achieving continuous, smooth, and interpretable spatial variation white balance correction. This invention does not rely on a preset white balance template, does not require displaying light source counts, has a smaller number of parameters and computational load, and is easy to deploy in real time. Furthermore, the multi-source training data synthesis method based on inverse / forward ISP effectively alleviates the problem of insufficient real training data, improves the model's generalization ability in complex mixed lighting scenarios, and has good prospects for industrial applications.

[0088] Similarly, it should be noted that although the present invention has been described with reference to the specific embodiments described above, those skilled in the art should recognize that the above embodiments are only used to illustrate the present invention, and various equivalent changes or substitutions can be made without departing from the spirit of the present invention. Therefore, any changes or modifications made to the above embodiments within the essential spirit of the present invention will fall within the scope of the claims of the present invention.

Claims

1. A multi-source automatic white balance method, characterized in that, Specifically, the following steps are included: Acquire the input image to be processed, which is an sRGB image, a linear RGB image, or an image obtained from camera RAW through basic ISP processing; Feature extraction is performed on the input image by inputting the input image into an encoder-decoder network to extract multi-scale image features containing global semantics and local texture, thereby obtaining bottleneck features; Global chromaticity base estimation is performed, the bottleneck features are globally pooled, and K global chromaticity bases are output through a fully connected layer, where K is an integer greater than 1. Each global chromaticity base is shared throughout the entire input image and is used to characterize the chromaticity of the dominant light source in the scene. Spatial weight estimation is performed by outputting K spatial weight response maps corresponding one-to-one with the K global chromaticity bases from the decoder end of the encoder-decoder network. The K spatial weight response maps are normalized in the base dimension so that the weights at each pixel position are non-negative and the sum is 1. A two-dimensional spatial illumination chromaticity map is generated by multiplying each global chromaticity base with its corresponding spatial weight map pixel by pixel and summing the results. White balance correction involves determining the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity map, performing pixel-by-pixel color correction on the input image, and outputting a white balance-corrected image.

2. The multi-source automatic white balance method according to claim 1, characterized in that, The global chromaticity base uses chromaticity coordinates normalized to the green channel. This indicates that the two-dimensional spatial illumination chromaticity diagram Calculate according to the following formula: ， in, For pixel coordinates, Let k be the global chromaticity base. For the first Each spatial weight map at pixels The weight value at that point, and .

3. The multi-source automatic white balance method according to claim 1, characterized in that, The encoder-decoder network comprises multiple resolution levels, each consisting of lightweight residual feature blocks and downsampling units; the decoder employs upsampling units and skips connections with the corresponding coding layers; global pooling is performed on the bottleneck features, specifically: for features of size... The bottleneck features described above are subjected to global average pooling in the spatial dimension to obtain dimensional vector, and then the above A 3D vector is input to at least one fully connected network, and the output is... Dimensional parameters as The global chromaticity base is described above.

4. The multi-source automatic white balance method according to claim 1, characterized in that, The white balance correction specifically includes: adjusting the two-dimensional spatial illumination chromaticity map. Expanded to three-channel relative illumination vectors Calculate the output pixels using the following formula: ,in, The input image contains the red, green, and blue channel values. The output image contains red, green, and blue channel values, with ε being a small constant to prevent division by zero. Exposure preservation, range cropping, and tone mapping are then applied to each channel after pixel-by-pixel correction.

5. The multi-source automatic white balance method according to claim 1, characterized in that, It also includes the training data generation step: Obtain a properly colored sRGB image as the ground truth image; Perform inverse ISP processing on the ground truth image to approximately restore it to the camera's RAW domain representation; Apply a global channel gain perturbation to the three channels in the RAW domain; Generate N virtual light sources, N≥1, each of which includes at least color temperature or RGB gain, incident direction, intensity, anchor point position, influence range and attenuation function type; Based on the incident direction and anchor point position of each virtual light source, calculate the spatial mask of each virtual light source at each pixel of the image; The RGB gain of each virtual light source at the pixel is normalized and weighted according to the spatial mask, and then multiplied with the RAW domain image channel by channel to obtain the RAW image after multi-light source perturbation. The RAW image after multi-source perturbation is subjected to forward ISP processing, and the composite input image is rendered. Based on the chromaticity base of each virtual light source and the spatial mask, an illumination chromaticity map corresponding to the synthesized input image is generated; The synthesized input image is paired with the ground truth image and the illumination chromaticity map to serve as training sample pairs.

6. The multi-source automatic white balance method according to claim 5, characterized in that, The reverse ISP processing specifically includes: performing inverse gamma correction, inverse tone mapping, and linear color space conversion sequentially on the true sRGB value; after converting the linear sRGB to the XYZ color space, mapping it to the camera RGB domain using the inverse matrix or pseudo-inverse matrix of the camera color correction matrix.

7. The multi-source automatic white balance method according to claim 5, characterized in that, The forward ISP processing includes: sequentially performing demosaicing, color matrix transformation, white point and exposure control, noise reduction, tone mapping, and gamma encoding.

8. The multi-source automatic white balance method according to claim 5, characterized in that, The global channel gain perturbation includes: setting D g =1,D r and D β Random sampling is performed within the range of 0.4 to 2.2; the color temperature is sampled between 1000K and 12000K and converted into normalized RGB gain through a blackbody radiation approximation model; the spatial mask is determined by both directional attenuation and distance attenuation, and the distance attenuation adopts any one of Gaussian attenuation, linear attenuation, quadratic attenuation or inverse square attenuation.

9. A multi-source automatic white balance system, characterized in that, include: The image acquisition module is used to acquire the input image to be processed; A multi-scale feature extraction module, comprising an encoder-decoder network, is used to extract multi-scale image features from the input image; The global chromaticity base estimation module is connected to the bottleneck of the encoder-decoder network. It is used to perform global pooling on the bottleneck features and output K global chromaticity bases through a fully connected mapping network, where K is an integer greater than 1. The spatial weight estimation module is connected to the decoding end of the encoder-decoder network. It is used to output K spatial weight response maps that correspond one-to-one with the K global chromaticity bases, and to perform normalization on the K spatial weight response maps in the base dimension. The illumination map fusion module is used to multiply each global chromaticity base with its corresponding spatial weight map pixel by pixel and then sum them to generate a two-dimensional spatial illumination chromaticity map. The white balance correction module is used to determine the pixel-by-pixel correction gain based on the two-dimensional spatial illumination chromaticity diagram, perform pixel-by-pixel color correction on the input image, and output the white balance corrected image.

10. The multi-source automatic white balance system according to claim 9, characterized in that, It also includes a training data generation module for generating training data pairs.

Images