A multi-light source oriented image white balance method and system

Abstract

The application discloses a multi-light-source-oriented image white balance method and system, wherein the method comprises the following steps: detecting a plurality of light source regions in an input image and obtaining energy information of the light source regions based on an offline constructed light source brightness reference model; determining an influence radius of a target light source according to the energy information, and dividing all image pixels into corresponding light source influence regions based on signed distances from each pixel in the input image to a circular boundary of each light source region, wherein the light source influence regions are not overlapped; statistically determining a local white balance parameter in each light source influence region based on pixels in the region; performing pixel-level weighted fusion on the local white balance parameter according to distances from each pixel to a center of a corresponding light source region and the energy information, generating a white balance gain of each pixel, and correcting the input image by using the white balance gain.

Description

Technical Field

[0001] This application relates to the field of image white balance technology, and more specifically, to an image white balance method and system for multiple light sources. Background Technology

[0002] In the field of image processing, Automatic White Balance (AWB) is a key module in Image Signal Processors (ISPs). Its goal is to correct image colors under different light source color temperatures, making white objects appear true white, thereby ensuring the accuracy of overall color reproduction. Traditional AWB methods typically assume that there is sufficient color information in the image, or that the scene is illuminated by a single light source, and generate white balance gain through global statistics (such as the gray-world method or perfect reflection method) or color temperature estimation based on Planck locus.

[0003] However, in scenarios involving multiple light sources with different color temperatures, such as nighttime streets with LED streetlights, vehicle lights, neon lights, and display screens simultaneously, or stage lighting with mixed illumination from multiple color temperature sources, traditional global white balance methods cannot simultaneously meet the color correction needs of areas illuminated by different light sources. Forcing the use of a single white balance gain will result in some areas having correct colors while other areas suffer from severe color casts. Summary of the Invention

[0004] To address the aforementioned technical problems, this application discloses an image white balance method and system for multi-light source scenes. By first detecting each light source and dividing it into non-overlapping influence regions based on its energy, then independently calculating white balance parameters within each region and performing pixel-level weighted fusion on the entire image accordingly, spatial adaptation and smooth transition of color correction in multi-light source mixed scenes are achieved. Specifically, the technical solution of this application is as follows: In a first aspect, this application discloses an image white balance method for multiple light sources, comprising: Based on an offline constructed light source brightness reference model, several light source regions in the input image are detected, and the energy information of the light source regions is obtained. The influence radius of the target light source is determined based on the energy information, and the entire image is divided into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, with each light source influence region not overlapping. Within each area affected by the light source, local white balance parameters are statistically analyzed based on the pixels within the area. Based on the distance of each pixel to the center of the corresponding light source area and the energy information, the local white balance parameters are weighted and fused at the pixel level to generate the white balance gain of each pixel, and the input image is corrected using the white balance gain.

[0005] In some implementations, the offline-constructed light source brightness reference model detects several light source regions in the input image and obtains the energy information of the light source regions; including: The current ambient illuminance is obtained, and the G-channel detection threshold corresponding to the current ambient illuminance is obtained by querying the light source brightness reference model; wherein, the light source brightness reference model is configured to record a set of G-channel thresholds for multiple typical light sources under multiple ambient illuminance levels; The G channel pixels in the Bayer format data of the input image are filtered. Based on the G-channel detection threshold, threshold segmentation is performed on the filtered G-channel pixels to generate a binary mask for the candidate light source region. The binary mask is subjected to morphological processing and connected component labeling to obtain the pixel set, centroid position, and total energy value of the G channel for each light source region.

[0006] In some embodiments, the image white balance method for multiple light sources further includes: pre-constructing the light source brightness reference model, specifically including: Multiple ambient illuminance levels were set in a darkroom, and Bayer images were captured for various typical light sources at each ambient illuminance level. Extract the luminous pixel region of each of the light sources in the corresponding image, and calculate the average brightness of the G channel pixels within the luminous pixel region; Under the same ambient illuminance level, the minimum value among the average brightness of the G channel pixels of all tested light sources is taken as the G channel detection threshold corresponding to the illuminance level; Each ambient illuminance level is associated with its corresponding G-channel detection threshold and stored to form a lookup table. A linear interpolation method is then used to obtain the G-channel detection threshold corresponding to the unmeasured intermediate illuminance value.

[0007] In some implementations, the filtering of the G channel pixels in the Bayer format data of the input image specifically includes: The G channel pixels in the Bayer format data are subjected to bilateral filtering to obtain the first filtered image; Gaussian filtering is applied to the G channel pixels in the first filtered image to obtain the second filtered image; The convolution calculations of the bilateral filter and the Gaussian filter only apply to the two G-channel pixel positions within each 2×2 unit of the Bayer arrangement, while the R-channel and B-channel pixels remain unchanged.

[0008] In some implementations, the step of determining the influence radius of the target light source based on the energy information, and dividing the entire image pixels into corresponding light source influence regions based on the signed distance of each pixel in the input image to the circular boundary of each light source region; specifically includes: The influence radius scaling factor for each light source region is determined based on the ratio of the total energy value of the G channel in each light source region to the minimum total energy value of the G channel in all light source regions, and the influence radius of each light source region is calculated based on a preset base radius unit. For each pixel in the input image, the Euclidean distance from the target pixel to the center of each light source region is calculated. The influence radius of the corresponding light source region is subtracted from the Euclidean distance to obtain the signed distance from the target pixel to the circular boundary of the light source region. The signed distance is negative when the target pixel is inside the circular boundary and positive when the target pixel is outside the circular boundary. For each target pixel, the light source region with the smallest signed distance is selected as the region to which the target pixel belongs.

[0009] In some implementations, the step of statistically analyzing local white balance parameters based on pixels within each light source-affected area specifically includes: The input image is divided into sliding blocks of fixed size; Based on the region to which each pixel belongs within the statistical block, the light source influence region to which each statistical block belongs is determined; Within each statistical block, the average pixel value of the R channel, the average pixel value of the G channel, and the average pixel value of the B channel are calculated respectively, and the R / G ratio and B / G ratio of the statistical block are calculated. For each region affected by the light source, the R / G ratio and B / G ratio of all statistical blocks therein are weighted and averaged according to confidence level to obtain the local white balance parameters of the region affected by the light source, including the region R / G ratio and the region B / G ratio.

[0010] In some implementations, the confidence weights are Planck trajectory weights; It also includes determining the Planck trajectory weights in the following ways: A Planck locus lookup table is pre-established to record the ideal R / G ratio and ideal B / G ratio for different color temperatures; Calculate the Euclidean distance from the R / G ratio and B / G ratio of each statistical block to the nearest point in the Planck trajectory lookup table; The Planck trajectory weights of the statistical block are calculated based on the Euclidean distance, and the Planck trajectory weights are negatively correlated with the Euclidean distance.

[0011] In some implementations, the step of performing pixel-level weighted fusion of the local white balance parameters based on the distance of each pixel to the center of the corresponding light source region and the energy information to generate the white balance gain of each pixel specifically includes: Calculate the Euclidean distance from each target pixel in the input image to the center of each light source region, and determine the maximum Euclidean distance. The distance weight of the target pixel relative to each of the light sources is calculated based on the square of the difference between the maximum Euclidean distance and the Euclidean distance from the target pixel to the center of each of the light sources. Based on the distance weight, the local white balance parameters of all light source regions corresponding to the target pixel are weighted and averaged to obtain the pixel R / G ratio and pixel B / G ratio of the target pixel; Based on the pixel R / G ratio and the pixel B / G ratio, calculate the R channel gain and B channel gain of the target, and set the G channel gain to 1.0.

[0012] In some embodiments, the correction of the input image using the white balance gain specifically includes: For each pixel in the input image, multiplication correction is performed on the R-channel pixel value, G-channel pixel value, and B-channel pixel value according to the corresponding R-channel gain, G-channel gain, and B-channel gain, respectively, and the white balance corrected image is output.

[0013] Secondly, this application also discloses an image white balance system for multiple light sources, the system being used to implement the image white balance method for multiple light sources described in any of the above embodiments, comprising: The light source detection module is used to detect several light source regions in the input image based on an offline constructed light source brightness reference model, and to obtain the energy information of the light source regions. The influence region division module is used to determine the influence radius of the target light source based on the energy information, and divide the entire image pixels into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, wherein the influence regions of each light source do not overlap. The local statistics module is used to statistically analyze local white balance parameters based on pixels within each of the light source's affected areas. The spatial fusion correction module is used to perform pixel-level weighted fusion of the local white balance parameters based on the distance of each pixel to the center of the corresponding light source area and the energy information, generate the white balance gain of each pixel, and use the white balance gain to correct the input image.

[0014] Compared with the prior art, this application has at least one of the following beneficial effects: 1. This application determines the circular influence radius of each light source based on the total energy value of its G channel, and classifies all pixels in the image based on the signed distance from each pixel to the circular boundary of each light source, ensuring that the influence areas of each light source do not overlap and cover the entire image. This avoids the problem of overcorrection in strong light source areas or the submersion of weak light sources, making the correction range of light sources of different intensities more consistent with the visual perception of multi-light source lighting scenes by the human eye.

[0015] 2. This application independently calculates the distance from each pixel to the center of each light source, giving greater weight to closer pixels, and combines this with the intensity values ​​of each light source to perform a pixel-level weighted average of the local white balance parameters, generating continuously varying R-channel and B-channel gain maps. This ensures that at the boundary between the areas affected by different light sources, the white balance gain smoothly transitions from the parameters of light source A to those of light source B, completely eliminating blockiness and color abruptness, resulting in a natural and consistent visual effect.

[0016] 3. This application calculates the deviation of the rg and bg values ​​of each statistical block from the Planck trajectory, assigning higher fusion weights to statistical blocks with smaller deviations. This allows statistical blocks whose colors are closer to the standard light source to contribute more to the weighted average of regional parameters, effectively suppressing the interference of noise blocks or non-neutral colored objects in the region on white balance parameters. This significantly improves the accuracy and anti-interference capability of white balance correction in multi-light source scenes. Attached Figure Description

[0017] The preferred embodiments will now be described in a clear and easy-to-understand manner, in conjunction with the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of this application.

[0018] Figure 1 This is a flowchart illustrating the steps of an embodiment of an image white balance method for multiple light sources according to this application. Figure 2 This is a schematic diagram illustrating the effect of dividing the input image into a light source region and a light source influence region in the embodiments provided in this application; Figure 3 A schematic diagram illustrating the effect of sliding division of the input image using statistical blocks of fixed size in the embodiments provided in this application; Figure 4 A schematic diagram illustrating the effect of determining the light source influence area to which each statistical block belongs in the embodiments provided in this application; Figure 5 This is a structural block diagram of an embodiment of an image white balance system for multiple light sources according to this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0021] To keep the drawings concise, each figure only schematically shows the parts relevant to the invention, and these do not represent the actual structure of the product. Furthermore, to facilitate understanding, in some figures, only one of components with the same structure or function is schematically depicted, or only one is labeled. In this document, "one" not only means "only one," but can also mean "more than one."

[0022] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] In this document, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0024] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific implementation methods of this application will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without creative effort.

[0026] Current automatic white balance (AWB) technologies mainly fall into two categories: The first category is based on global color statistics, such as the gray-world method, white point method, or correction methods based on the RGB channel mean. These methods assume the scene is illuminated by a single light source and estimate the light source's color temperature through the overall color ratio. The second category is based on color temperature models or scene recognition, combining Planck trajectories, color temperature lookup tables, or machine learning classification results to estimate white balance parameters. However, all of these methods have significant limitations in multi-light source mixed scenes.

[0027] First, existing AWB technology lacks the ability to simultaneously process multiple light sources with different color temperatures. When warm-toned light sources (such as incandescent lamps) and cool-toned light sources (such as cool white LED light) appear in an image simultaneously, the global statistical results will be biased towards a certain intermediate color temperature between the two, causing warm areas to appear bluish and cool areas to appear yellowish. Even if some methods introduce local white balance (such as block AWB), they often lack accurate detection of light source areas and division of influence ranges, resulting in color abrupt changes or unnatural transition zones at the boundaries between different light sources.

[0028] Secondly, existing methods typically use statistical blocks of fixed size or fixed grid divisions, which cannot adaptively adjust the influence range according to the actual energy intensity of the light source. Strong light sources (such as car headlights) should have a larger influence radius, while weak light sources (such as small indicator lights) should have a smaller influence range. Fixed blocks cannot reflect this difference, easily leading to overcorrection around strong light sources or the suppression of weak light sources.

[0029] Furthermore, existing methods are prone to color flickering issues in multi-light source scenarios. Due to the lack of continuous weighting of light source energy and spatial distance, the white balance parameters between frames may change drastically. Especially in video sequences, when the camera or light source position moves slightly, the corrected tone will exhibit unnatural jitter.

[0030] To address the aforementioned issues, this invention proposes an image white balance method for multi-light sources. By accurately detecting each light source, dividing the affected region based on energy, independently calculating local white balance parameters, and employing pixel-level distance-weighted fusion, smooth, stable, and accurate color correction is achieved in multi-light source scenarios.

[0031] Reference manual attached Figure 1As shown, an embodiment of the image white balance method for multiple light sources according to this application specifically includes the following steps: S1, based on the offline constructed light source brightness reference model, detect several light source regions in the input image and obtain the energy information of the light source regions.

[0032] The input image is preferably a raw Bayer format image directly output by the image sensor. Bayer format refers to a typical arrangement of the color filter array (CFA) covering the surface of the image sensor, where each pixel location records only the intensity value of a single channel from the three color channels: red (R), green (G), or blue (B). In the RGGB type Bayer arrangement, each 2×2 basic repeating unit contains two diagonally distributed G channel sampling points and one each of the R and B channel sampling points, with the sampling density of the G channel being twice that of the R or B channels.

[0033] S2, determine the influence radius of the target light source based on the energy information, and divide the entire image pixels into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, wherein the influence regions of each light source do not overlap.

[0034] For details, please refer to the attached instruction manual. Figure 2 As shown, the light source region refers to the set of pixels occupied by the light-emitting subject itself in the input image, such as the imaging range of an actual light-emitting object like a lamp bead or lamp tube. The energy information refers to the sum of the brightness values ​​of all G-channel pixels within the light source region, denoted as sum_val_i, which characterizes the luminous intensity of the light source.

[0035] The area affected by the light source refers to the spatial range dominated by the white balance correction parameters of a certain light source, that is, the surrounding area of ​​the image affected by the color temperature of the light source. Its range is much larger than the light source area itself and can cover the illuminated scene area around the light source.

[0036] The relative size of the circular influence radius of each light source is determined based on its total energy G (the larger the energy, the larger the radius). Then, for each pixel in the entire image, the signed distance from it to the circular boundary of each light source region is calculated. The light source that minimizes this distance is selected as the assigned light source for that pixel, thus dividing all pixels in the image into the corresponding light source influence regions. Refer to the attached instruction manual. Figure 2 As shown, Figure 2This diagram illustrates the effect of dividing the input image into light source regions and light source influence regions in this embodiment. The solid black circle "●" represents the position of the corresponding light source in the input image. "Light Source 0," "Light Source 1," and "Light Source 2" are the numbers of the three light sources, used to distinguish them. Each boundary curve divides the input image into light source influence regions belonging to the aforementioned light sources, ensuring that each pixel has a unique light source influence region affiliation. This division method allows strong light sources to obtain a wider correction influence region, while weak light sources only affect their surrounding areas, consistent with the actual physical influence range of light sources of different intensities.

[0037] S3, within each area affected by the light source, statistical local white balance parameters are calculated based on the pixels within the area.

[0038] Specifically, Automatic White Balance (AWB) is a key module in the Image Signal Processor (ISP). Its goal is to correct the color of an image under different light source color temperatures so that white objects appear true white.

[0039] White balance parameters refer to the channel ratios used to describe the color characteristics of a light source, typically including the red-to-green ratio (R / G) and the blue-to-green ratio (B / G). These reflect the intensity ratio of the red and blue channels relative to the green channel in an image, thus providing a basis for subsequent calculations of the correction gain for each channel. This application calculates local white balance parameters independently within each light source's influence area, avoiding interference between light sources of different color temperatures.

[0040] S4. Based on the distance of each pixel to the center of the corresponding light source area and the energy information, the local white balance parameters are weighted and fused at the pixel level to generate the white balance gain of each pixel, and the input image is corrected using the white balance gain.

[0041] Specifically, for each pixel in the image, a distance weight is calculated relative to each light source region based on the distance of each pixel to the center of the corresponding light source region. This distance weight ensures that pixels closer to the center of the light source are more significantly affected by the white balance parameters of that light source. Based on this distance weight, a weighted average of the local white balance parameters of all light source regions corresponding to that pixel is calculated to obtain the pixel-level red-to-green ratio (R / G) and pixel-level blue-to-green ratio (B / G). Then, the R-channel gain and B-channel gain of the pixel are calculated based on these pixel-level R / G and B / G ratios, respectively.

[0042] Finally, for each pixel in the input image, its original R channel pixel value is multiplied by the corresponding R channel gain, the B channel pixel value is multiplied by the corresponding B channel gain, and the G channel pixel value remains unchanged, thus completing the white balance correction of the entire image.

[0043] This application's pixel-level distance-weighted fusion mechanism independently calculates the continuous distance weights from each pixel to the center of each light source, allowing the white balance gain to change continuously across the entire image. This results in a smooth transition of the white balance gain from the parameters of light source A to those of light source B at the boundary between areas affected by different light sources, eliminating block effects and color jumps, and outputting a multi-light source white balance image with natural colors and smooth transitions.

[0044] Based on the above embodiments, this application discloses another embodiment of an image white balance method for multiple light sources, wherein step S1 specifically includes the following sub-steps: S11, Obtain the current ambient illuminance, and obtain the G-channel detection threshold corresponding to the current ambient illuminance by querying the light source brightness reference model. The light source brightness reference model is configured to record a set of G-channel thresholds for various typical light sources under multiple ambient illuminance levels.

[0045] Specifically, ambient illuminance refers to the brightness of ambient light in a scene, measured in lux, and can be provided by an Auto Exposure (AE) module or a separate illuminance sensor.

[0046] In some implementations, the light source brightness reference model is generated by setting multiple ambient illuminance levels in a dark room, acquiring Bayer images for various typical light sources, statistically analyzing the average brightness of the G channel for each typical light source, and taking the minimum value among all light sources under the same illuminance level as the G channel detection threshold for that illuminance level. This generates a look-up table (LUT) between the ambient illuminance and the G channel threshold. Optionally, the various typical light sources include, but are not limited to, light-emitting diode (LED) lamps, incandescent lamps, fluorescent lamps, sodium lamps, display screens, mobile phone flashlights, vehicle taillights, and candle flames. In practical applications, by querying this LUT based on the current ambient illuminance, and combining this with filtering and threshold segmentation of the G channel of the Bayer image, all light source regions in the image can be detected.

[0047] S12, filter the G channel pixels in the Bayer format data of the input image.

[0048] Specifically, this embodiment uses a common RGGB type Bayer array as an example. Each 2×2 basic repeating unit contains two diagonally distributed G channel sampling points, as well as one R channel and one B channel sampling point. For input images that are not in Bayer format, such as RGB images, they can also be converted into Bayer images through inverse processing.

[0049] In some embodiments, step S12 specifically includes: S121, perform bilateral filtering on the G channel pixels in the Bayer format data to obtain the first filtered image.

[0050] S122, Gaussian filtering is performed on the G channel pixels in the first filtered image to obtain the second filtered image.

[0051] In some optional implementations, the bilateral filter kernel size is 7×7 pixels, the spatial domain standard deviation (sigma) is 1.5, and the value domain standard deviation is 0.05. Bilateral filtering can effectively preserve the edge structure of the light source area while suppressing sensor noise.

[0052] In some alternative implementations, the Gaussian filter kernel size is 9×9 pixels, and the standard deviation is 1.0. Gaussian filtering can make the brightness distribution of the G channel within the light source region more uniform, avoiding breaks in the light source region during subsequent thresholding due to the presence of a single dark pixel.

[0053] It should be noted that the convolution calculations of the bilateral filtering and Gaussian filtering mentioned above are only applied to the same channel spatial position in the Bayer domain. That is, the convolution operation is only performed on the two G channel pixels that are diagonally distributed in each 2×2 basic repeating unit in the RGGB type Bayer array. The R channel pixels and B channel pixels do not participate in the cumulative calculation of the filtering convolution and their pixel values ​​remain unchanged. The filtered G channel pixel values ​​are written back to the original spatial position, and the channel spatial distribution structure of the Bayer array remains unchanged.

[0054] S13, based on the G channel detection threshold, threshold segmentation is performed on the filtered G channel pixels to generate a binary mask for the candidate light source region.

[0055] The threshold segmentation refers to the process of comparing the filtered G channel pixel values ​​with the G channel detection threshold obtained in step S11, and classifying the pixels as candidate light-emitting points or non-candidate light-emitting points based on the comparison result.

[0056] Specifically, each 2×2 basic repeating unit of the filtered Bayer image is traversed, and the arithmetic mean of the two G-channel pixels within that unit is calculated, denoted as G_ave. This mean is then compared with the G-channel detection threshold Gth_cur: if G_ave is greater than or equal to Gth_cur, all four pixel positions within that 2×2 unit are marked as candidate emission points, and the corresponding binary mask value is set to 1. If G_ave is less than Gth_cur, all four pixel positions within that 2×2 unit are marked as non-candidate emission points, and the corresponding binary mask value is set to 0.

[0057] By labeling in 2×2 units, the spatial integrity of the Bayer array can be maintained, avoiding the disruption of structural consistency in subsequent morphological processing and connected component analysis caused by labeling only a single G pixel position. After traversal, a binary mask image with the same resolution as the input image is generated, where the pixel positions with a value of 1 constitute an initial set of candidate light source regions.

[0058] S14, perform morphological processing and connected component labeling on the binary mask to obtain the pixel set, centroid position, and total energy value of the G channel for each light source region.

[0059] Specifically, morphological processing includes closing and opening operations. Closing involves first dilating the binary mask and then eroding it, using circular structuring elements with a radius of 3 to 5 pixels to fill small holes and gaps within the candidate light source region, connecting scattered areas of the same light source into a complete connected body. Opening involves first eroding the binary mask and then dilating it, using the same circular structuring elements to eliminate isolated single-point noise or tiny false regions smaller than 10 pixels, preventing non-light source noise from being mistakenly detected as light sources. After morphological correction, the corrected light source mask is obtained.

[0060] The connected component labeling refers to using an 8-connectivity labeling algorithm on the corrected light source mask to aggregate interconnected pixels into independent connected regions, with each connected region corresponding to a real light source region. 8-connectivity means that for a central pixel, the pixels in its eight adjacent positions (up, down, left, right, and diagonally) are considered connected to that central pixel.

[0061] In other implementations, for each marked connected component, the following information is output: the set of pixel coordinates of the light source region, used to determine the spatial occupancy of the light source; the centroid position, i.e., the average value of all pixel coordinates within the region, used to represent the spatial center position of the light source; and the total energy value of the G channels, which is the sum of the brightness values ​​of all G channel pixels at the original filtered Bayer image position corresponding to the light source region, obtained as sum_val_i, used to characterize the luminous intensity of the light source.

[0062] Optionally, the following formula can be used to calculate: for each connected region (light source region) Region_i, sum_val_i of the brightness values ​​of all G channel pixels within it on the Bayer_gauss image.

[0063] .

[0064] Calculate the location of the centroid of this region: .

[0065] .

[0066] Where n_i is the number of pixels in Region_i; x_i and y_i are the x and y coordinates of the i-th pixel, respectively, and their values ​​are limited to Region_i.

[0067] This embodiment detects all light-emitting light source regions in the input image and obtains the position and energy information of each light source, providing a data foundation for subsequent influence area division and white balance parameter calculation.

[0068] This application provides another embodiment of an image white balance method for multiple light sources. Based on any of the above embodiments, before executing step S1, it further includes: S01, pre-constructing the light source brightness reference model, specifically including the following sub-steps: S011, set multiple ambient illuminance levels in a darkroom, and under each ambient illuminance level, capture Bayer images of various typical light sources.

[0069] S012, extract the luminous pixel region of each of the luminous light sources in the corresponding image, and calculate the average brightness of the G channel pixels in the luminous pixel region.

[0070] S013, Under the same ambient illuminance level, take the minimum value among the average brightness of the G channel pixels of all tested light sources as the G channel detection threshold corresponding to the illuminance level.

[0071] S014, associate and store each of the environmental illuminance levels with their corresponding G-channel detection thresholds to form a lookup table, and use a linear interpolation method to obtain the G-channel detection thresholds corresponding to the unmeasured intermediate illuminance values.

[0072] Specifically, multiple ambient illuminance levels are set in a darkroom, and various typical light sources are lit at each level. Bayer images are acquired for each light source, and the pixel regions where the light sources are located in the images are marked manually or using an automatic segmentation tool. The average brightness of the G-channel pixels in that region is calculated, and then the minimum value among the average brightness of the G-channel of all tested light sources at the same illuminance level is taken as the G-channel detection threshold corresponding to that illuminance level.

[0073] The G-channel detection threshold is less than or equal to the average brightness of all types of light sources under the given ambient illuminance, ensuring that each real light source can be detected in subsequent segmentation without being missed. All ambient illuminance levels are associated with and stored with their corresponding G-channel detection thresholds, forming a look-up table (LUT).

[0074] In some alternative implementations, for unmeasured intermediate illuminance values, a linear interpolation method is used to calculate the corresponding G-channel detection threshold. In practical applications, the appropriate G-channel detection threshold for the current scene can be quickly obtained by querying the LUT based on the current ambient illuminance.

[0075] Based on the above embodiments, this application discloses another embodiment of an image white balance method for multiple light sources, wherein step S2 specifically includes the following sub-steps: S21, determine the influence radius scaling factor of each light source region based on the ratio of the total energy value of the G channel in each light source region to the minimum total energy value of the G channel in all light source regions, and calculate the influence radius of each light source region based on a preset base radius unit.

[0076] Specifically, the radius of influence is determined as follows: First, find the minimum value of the total energy of the G channel in all detected light source regions, min_sum, min_sum = MIN(sum_val_i).

[0077] Divide the total energy of each light source region by the minimum value to obtain the radius scaling factor r_i = sum_val_i / min_sum for each light source region. The higher the energy of the light source, the larger its radius scaling factor.

[0078] Then, multiply this scaling factor by a preset base radius unit R_unit to obtain the actual influence radius of the light source R_i = r_i × R_unit. Optionally, R_unit is 50 pixels, which can be adjusted according to the image size.

[0079] S22, for each pixel in the input image, calculate the Euclidean distance from the target pixel to the center of each light source region, and subtract the influence radius of the corresponding light source region from the Euclidean distance to obtain the signed distance from the target pixel to the circular boundary of the light source region. The signed distance is negative when the target pixel is inside the circular boundary, and positive when the target pixel is outside the circular boundary.

[0080] Specifically, the signed distance refers to calculating the Euclidean distance dist_i from any pixel in the image to the center of a certain light source region, and then subtracting the influence radius R_i of the light source from this distance to obtain d_i = dist_i - R_i, where a negative value of d_i indicates that the pixel is within the circular influence range of the light source, and a positive value of d_i indicates that it is outside the range.

[0081] S23, for each target pixel, select the light source region with the smallest signed distance as the region to which the target pixel belongs.

[0082] Specifically, the physical meaning of the minimum signed distance is: among all the circular influence ranges of light sources, the pixel is closest to the circular boundary of which light source. The smaller the negative value, the greater the degree to which the pixel is inside the influence circle of that light source, and the smaller the positive value, the closer the pixel is to the boundary of the influence circle of that light source.

[0083] Repeat the above operation for each pixel in the image. After traversing the entire image, each pixel has a unique light source region label, forming a division of the light source influence region covering the entire image. Each pixel belongs to only one light source influence region, and these influence regions do not overlap.

[0084] Based on the above embodiments, this application discloses another embodiment of an image white balance method for multiple light sources, wherein step S3 specifically includes the following sub-steps: S31, the input image is divided into sliding blocks of fixed size.

[0085] S32, determine the light source influence area to which each statistical block belongs based on the region to which each pixel belongs within the statistical block.

[0086] For details, please refer to the attached instruction manual. Figure 3 As shown, the input image is divided into non-overlapping statistical blocks of fixed size.

[0087] Subsequently, based on the pixel-level light source influence area obtained in step S2, the light source influence area to which each statistical block belongs is determined. (See attached instruction manual.) Figure 4 As shown, Figure 4 The dashed grid represents the statistical blocks obtained by dividing the input image, the solid curves represent the boundaries between the light source influence areas, and the black solid circle "●" indicates the position of the corresponding light source in the input image. The numbers "0", "1", and "2" within each statistical block are the light source influence area affiliation identifiers, indicating that the statistical block belongs to the light source influence area corresponding to light source 0, light source 1, and light source 2, respectively.

[0088] Optionally, for each statistical block, the label map checks the light source region labels corresponding to all pixel positions covered by the statistical block, and determines whether these labels are all the same, i.e., whether the statistical block completely falls within the influence area of ​​a single light source. If so, the statistical block is assigned to the influence area of ​​that light source.

[0089] If the statistical block spans the boundaries of multiple light source influence areas, i.e. contains two or more different area labels, then the number of pixels belonging to each light source area within the statistical block is counted, and the light source area with the most pixels is taken as the area to which the statistical block belongs.

[0090] If two or more light source regions have the same number of pixels, then one of them is arbitrarily selected as the region to which the light source belongs.

[0091] S33, within each statistical block, calculate the average pixel value of the R channel, the average pixel value of the G channel, and the average pixel value of the B channel, and calculate the R / G ratio and B / G ratio of the statistical block.

[0092] Specifically, within each statistical block, the mean value R_avg of all R channel pixels, the mean value G_avg of all G channel pixels, and the mean value B_avg of all B channel pixels within that block are calculated, and then the red-green ratio (R / G) and blue-green ratio (B / G) of that statistical block are calculated.

[0093] In some implementations, the red-green ratio and blue-green ratio are calculated using the following formulas.

[0094] Statistical block red-to-green ratio: rg_block = R_avg / G_avg.

[0095] Statistical block blue-green ratio: bg_block = B_avg / G_avg.

[0096] The red-green ratio reflects the intensity ratio of red to green within the statistical block, while the blue-green ratio reflects the intensity ratio of blue to green. Together, they constitute a two-dimensional chromaticity coordinate system describing the color characteristics of a light source.

[0097] S34, for each of the light source-affected areas, the R / G ratio and B / G ratio of all the statistical blocks therein are weighted and averaged according to the confidence level to obtain the local white balance parameters of the light source-affected area, including the area R / G ratio and the area B / G ratio.

[0098] Specifically, the confidence weight is used to measure the reliability of the color information of each statistical block in the fusion of regional parameters. The higher the weight, the greater the contribution of the statistical block to the final white balance parameter of the region. For each light source-affected region, the R / G ratio and B / G ratio of all statistical blocks to which it belongs are weighted and averaged to obtain the local white balance parameters rg_i and bg_i of the region.

[0099] In some implementations, the local white balance parameter is calculated using the following formula.

[0100] rg_i = ( Σ w_k × rg_block_k ) / Σ w_k.

[0101] bg_i = ( Σ w_k × bg_block_k ) / Σ w_k.

[0102] Where w_k is the confidence weight of the kth statistical block, rg_block is the red-green ratio, and bg_block is the blue-green ratio.

[0103] This embodiment effectively suppresses the interference of noise blocks or non-neutral color objects in the region on white balance statistics, making the local correction parameters of each light source's affected area closer to the true color temperature of that light source.

[0104] In some implementations, the confidence weights in step S34 are Planck trajectory weights.

[0105] Specifically, the Planck trajectory is a curve on a chromaticity diagram that describes the color coordinates corresponding to the radiation from an ideal blackbody at different color temperatures. The color coordinates of the standard light source all fall near this trajectory. The Planck trajectory weight reflects how closely the color of a statistical block is close to that of the standard light source. The closer the color of a statistical block is to the standard light source, the more reliable it is considered and given a higher fusion weight.

[0106] In other embodiments, the image white balance method for multiple light sources further includes determining the Planck trajectory weights by: S301, a Planck locus lookup table is pre-established to record the ideal R / G ratio and ideal B / G ratio corresponding to different color temperatures.

[0107] S302, calculate the Euclidean distance from the R / G ratio and B / G ratio of each statistical block to the nearest point in the Planck trajectory lookup table.

[0108] S303, calculate the Planck trajectory weight of the statistical block based on the Euclidean distance, wherein the Planck trajectory weight is negatively correlated with the Euclidean distance.

[0109] Specifically, the color temperature range of the Planck locus lookup table typically covers the common light source color temperature range from 2000K to 10000K. The Euclidean distance *dist* from the nearest point in the Planck locus lookup table to the R / G ratio and B / G ratio of each statistical block is calculated. This distance measures the degree to which the color of the statistical block deviates from the standard light source locus; the smaller the distance, the closer the color of the statistical block is to the standard light source at a certain color temperature.

[0110] The Planck trajectory weights of the statistical blocks are calculated based on the Euclidean distance. The Planck trajectory weights are negatively correlated with the Euclidean distance, meaning that the smaller the distance, the higher the weight, and the larger the distance, the lower the weight.

[0111] In the following optional implementation, the weights are calculated using a Gaussian function: w = exp(-dist² / (2σ²)).

[0112] Where σ is a preset scale parameter, which is taken as 0.1~0.2.

[0113] This weighting mechanism allows statistical blocks whose colors are closer to those of the standard light source to dominate the fusion of regional white balance parameters, effectively suppressing the interference of noise blocks or non-neutral colored objects in the region on the statistical results.

[0114] Based on the above embodiments, this application discloses another embodiment of an image white balance method for multiple light sources, wherein step S4 specifically includes the following sub-steps: S41, calculate the Euclidean distance from each target pixel in the input image to the center of each light source region, and determine the maximum Euclidean distance.

[0115] Specifically, Euclidean distance represents the straight-line distance on the image plane from a target pixel to the center of a certain light source region. It is used to quantify the spatial distance between the pixel and various light sources. For a target pixel, let the coordinates of the target pixel to be processed be (x, y). Traverse all N detected light source regions to calculate N Euclidean distance values: dist_i = sqrt((x-cx_i)²+(y-cy_i)²), and take the maximum value as dist_max. This maximum distance reflects the spatial distance of the pixel to the farthest light source.

[0116] S42, calculate the distance weight of the target pixel relative to each of the light sources based on the square of the difference between the maximum Euclidean distance and the Euclidean distance from the target pixel to the center of each of the light sources.

[0117] Specifically, for the i-th light source region, its distance weight is: dwgt_i = (dist_max - dist_i)².

[0118] The closer a target pixel is to the center of a light source, the smaller dist_i becomes, the larger the difference between dist_max and dist_i becomes, and the larger the corresponding distance weight dwgt_i becomes. Conversely, the farther a target pixel is from the center of a light source, the smaller dwgt_i becomes.

[0119] This embodiment further amplifies the difference in weights between near and far pixels through squaring operations, so that pixels closer to the center of the light source have a significantly higher response weight to the white balance parameters of the light source, while the weight of pixels farther away decreases sharply, thereby realizing the spatial localization of the white balance parameters.

[0120] S43, based on the distance weight, the local white balance parameters of all light source regions corresponding to the target pixel are weighted and averaged to obtain the pixel R / G ratio and pixel B / G ratio of the target pixel.

[0121] Specifically, this embodiment considers both the spatial distance from the target pixel to the light source and the intensity of the light source itself. The local white balance parameters rg_i and bg_i of all N light source regions are weighted and averaged based on distance weights to obtain the pixel-level red-green ratio and pixel-level blue-green ratio of the target pixel.

[0122] In some implementations, the pixel-level red-green ratio and pixel-level blue-green ratio are calculated using the following formulas.

[0123] Pixel-level red-green ratio: .

[0124] Pixel-level blue-green ratio: .

[0125] Wherein, dwgt_i is the distance weight. sum_val_i is the total energy value of the G channel calculated in step S14, which is used to characterize the luminous intensity of the light source. The higher the energy of the light source, the greater its influence on the color of the surrounding area.

[0126] The weighted averaging method in this embodiment allows the correction parameters to change continuously in space. Pixels closer to a certain light source are mainly controlled by the white balance parameters of that light source, while pixels farther away from the light source are affected by multiple light sources.

[0127] S44, calculate the R channel gain and B channel gain of the target based on the pixel R / G ratio and the pixel B / G ratio, and set the G channel gain to 1.0.

[0128] Specifically, for a 2×2 Bayer cell, calculate the gain: R-channel gain: gain_R(x,y) = 1.0 / ;in, It represents the red-green ratio at the pixel level.

[0129] B-channel gain: gain_B(x,y) = 1.0 / ;in, It is the blue-green ratio at the pixel level.

[0130] G-channel gain: gain_G(x,y) = 1.0.

[0131] Based on the above embodiments, this application discloses another embodiment of an image white balance method for multiple light sources, wherein step S4 further includes: S45, for each pixel in the input image, perform multiplication correction on the R channel pixel value, G channel pixel value and B channel pixel value according to the corresponding R channel gain, G channel gain and B channel gain, and output the white balance corrected image.

[0132] Specifically, after calculating the white balance gain of the target pixel, it is multiplied by the R-channel and B-channel pixel values ​​of the corresponding pixel in the original input image to complete the white balance correction for that pixel. The corrected pixel values ​​for each channel are as follows: Corrected R channel pixel value: R_corrected = R_raw × gain_R(x,y).

[0133] Corrected B-channel pixel values: B_corrected = B_raw × gain_B(x,y)

[0134] Corrected G channel pixel value: G_corrected = G_raw × 1.0.

[0135] Where R_raw, B_raw, and G_raw represent the original pixel values ​​of the R, B, and G channels in the current 2×2 Bayer cell before correction, respectively. gain_R(x,y) and gain_B(x,y) represent the R channel gain and B channel gain at position (x,y), respectively.

[0136] After traversing all pixels in the image, the complete image after white balance correction is output. This application ultimately adjusts the gain of each channel so that the corrected white object satisfies R=G=B. While keeping the G channel constant, the R and B channels need to be multiplied by their respective gain values ​​to achieve balance.

[0137] Based on the same concept, this application also discloses an image white balance system for multiple light sources. The system is used to implement the steps described in any of the above method embodiments. Specifically, an embodiment of an image white balance system for multiple light sources is provided in the appendix to the specification. Figure 5 As shown, it specifically includes: The light source detection module is used to detect several light source regions in the input image based on an offline constructed light source brightness reference model, and to obtain the energy information of the light source regions.

[0138] The influence region division module is used to determine the influence radius of the target light source based on the energy information, and divide the entire image pixels into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, wherein the influence regions of each light source do not overlap.

[0139] The local statistics module is used to statistically analyze local white balance parameters based on pixels within each area affected by the light source.

[0140] The spatial fusion correction module is used to perform pixel-level weighted fusion of the local white balance parameters based on the distance of each pixel to the center of the corresponding light source area and the energy information, generate the white balance gain of each pixel, and use the white balance gain to correct the input image.

[0141] The image white balance method and system for multiple light sources in this application have the same technical concept, and the technical details of the embodiments of the two are applicable to each other. To reduce repetition, they will not be described again here.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of program modules is merely an example. In practical applications, the above functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software program unit. Furthermore, the specific names of the program modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0143] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A white balance method for images with multiple light sources, characterized in that, include: Based on an offline constructed light source brightness reference model, several light source regions in the input image are detected, and the energy information of the light source regions is obtained. The influence radius of the target light source is determined based on the energy information, and the entire image is divided into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, with each light source influence region not overlapping. Within each area affected by the light source, local white balance parameters are statistically analyzed based on the pixels within the area. Based on the distance of each pixel to the center of the corresponding light source area and the energy information, the local white balance parameters are weighted and fused at the pixel level to generate the white balance gain of each pixel, and the input image is corrected using the white balance gain.

2. The image white balance method for multiple light sources as described in claim 1, characterized in that, The aforementioned offline-constructed light source brightness reference model detects several light source regions in the input image and obtains the energy information of the light source regions; including: The current ambient illuminance is obtained, and the G-channel detection threshold corresponding to the current ambient illuminance is obtained by querying the light source brightness reference model; wherein, the light source brightness reference model is configured to record a set of G-channel thresholds for multiple typical light sources under multiple ambient illuminance levels; The G channel pixels in the Bayer format data of the input image are filtered. Based on the G-channel detection threshold, threshold segmentation is performed on the filtered G-channel pixels to generate a binary mask for the candidate light source region. The binary mask is subjected to morphological processing and connected component labeling to obtain the pixel set, centroid position, and total energy value of the G channel for each light source region.

3. The image white balance method for multiple light sources as described in claim 2, characterized in that, Also includes: The light source brightness reference model is pre-constructed, specifically including: Multiple ambient illuminance levels were set in a darkroom, and Bayer images were captured for various typical light sources at each ambient illuminance level. Extract the luminous pixel region of each of the light sources in the corresponding image, and calculate the average brightness of the G channel pixels within the luminous pixel region; Under the same ambient illuminance level, the minimum value among the average brightness of the G channel pixels of all tested light sources is taken as the G channel detection threshold corresponding to the illuminance level; Each ambient illuminance level is associated with its corresponding G-channel detection threshold and stored to form a lookup table. A linear interpolation method is then used to obtain the G-channel detection threshold corresponding to the unmeasured intermediate illuminance value.

4. The image white balance method for multiple light sources as described in claim 2, characterized in that, The filtering process performed on the G channel pixels in the Bayer format data of the input image specifically includes: The G channel pixels in the Bayer format data are subjected to bilateral filtering to obtain the first filtered image; Gaussian filtering is applied to the G channel pixels in the first filtered image to obtain the second filtered image; The convolution calculations of the bilateral filter and the Gaussian filter only apply to the two G-channel pixel positions within each 2×2 unit of the Bayer arrangement, while the R-channel and B-channel pixels remain unchanged.

5. The image white balance method for multiple light sources as described in claim 2, characterized in that, The process of determining the influence radius of the target light source based on the energy information, and dividing the entire image pixels into corresponding light source influence regions based on the signed distance of each pixel in the input image to the circular boundary of each light source region; specifically includes: The influence radius scaling factor for each light source region is determined based on the ratio of the total energy value of the G channel in each light source region to the minimum total energy value of the G channel in all light source regions, and the influence radius of each light source region is calculated based on a preset base radius unit. For each pixel in the input image, the Euclidean distance from the target pixel to the center of each light source region is calculated. The influence radius of the corresponding light source region is subtracted from the Euclidean distance to obtain the signed distance from the target pixel to the circular boundary of the light source region. The signed distance is negative when the target pixel is inside the circular boundary and positive when the target pixel is outside the circular boundary. For each target pixel, the light source region with the smallest signed distance is selected as the region to which the target pixel belongs.

6. The image white balance method for multiple light sources as described in claim 1, characterized in that, The method described above, within each region affected by the light source, statistically analyzes local white balance parameters based on pixels within that region; specifically, it includes: The input image is divided into sliding blocks of fixed size; Based on the region to which each pixel belongs within the statistical block, the light source influence region to which each statistical block belongs is determined; Within each statistical block, the average pixel value of the R channel, the average pixel value of the G channel, and the average pixel value of the B channel are calculated respectively, and the R / G ratio and B / G ratio of the statistical block are calculated. For each region affected by the light source, the R / G ratio and B / G ratio of all statistical blocks therein are weighted and averaged according to confidence level to obtain the local white balance parameters of the region affected by the light source, including the region R / G ratio and the region B / G ratio.

7. The image white balance method for multiple light sources as described in claim 6, characterized in that, The confidence weights are Planck trajectory weights; It also includes determining the Planck trajectory weights in the following ways: A Planck locus lookup table is pre-established to record the ideal R / G ratio and ideal B / G ratio for different color temperatures; Calculate the Euclidean distance from the R / G ratio and B / G ratio of each statistical block to the nearest point in the Planck trajectory lookup table; The Planck trajectory weights of the statistical block are calculated based on the Euclidean distance, and the Planck trajectory weights are negatively correlated with the Euclidean distance.

8. The image white balance method for multiple light sources as described in claim 1, characterized in that, The step of performing pixel-level weighted fusion of the local white balance parameters based on the distance of each pixel to the center of the corresponding light source region and the energy information to generate the white balance gain of each pixel specifically includes: Calculate the Euclidean distance from each target pixel in the input image to the center of each light source region, and determine the maximum Euclidean distance. The distance weight of the target pixel relative to each of the light sources is calculated based on the square of the difference between the maximum Euclidean distance and the Euclidean distance from the target pixel to the center of each of the light sources. Based on the distance weight, the local white balance parameters of all light source regions corresponding to the target pixel are weighted and averaged to obtain the pixel R / G ratio and pixel B / G ratio of the target pixel; Based on the pixel R / G ratio and the pixel B / G ratio, calculate the R channel gain and B channel gain of the target, and set the G channel gain to 1.

0.

9. The image white balance method for multiple light sources as described in claim 8, characterized in that, The method of correcting the input image using the white balance gain specifically includes: For each pixel in the input image, multiplication correction is performed on the R-channel pixel value, G-channel pixel value, and B-channel pixel value according to the corresponding R-channel gain, G-channel gain, and B-channel gain, respectively, and the white balance corrected image is output.

10. An image white balance system for multiple light sources, characterized in that, The system is used to implement the image white balance method for multiple light sources as described in any one of claims 1-9, comprising: The light source detection module is used to detect several light source regions in the input image based on an offline constructed light source brightness reference model, and to obtain the energy information of the light source regions. The influence region division module is used to determine the influence radius of the target light source based on the energy information, and divide the entire image pixels into corresponding light source influence regions based on the signed distance from each pixel in the input image to the circular boundary of each light source region, wherein the influence regions of each light source do not overlap. The local statistics module is used to statistically analyze local white balance parameters based on pixels within each of the light source's affected areas. The spatial fusion correction module is used to perform pixel-level weighted fusion of the local white balance parameters based on the distance of each pixel to the center of the corresponding light source area and the energy information, generate the white balance gain of each pixel, and use the white balance gain to correct the input image.

Images