White balance enhancement method, system, electronic device, and storage medium

Abstract

The application discloses a white balance enhancement method, system, electronic equipment and storage medium, and relates to the technical field of image processing. The method obtains a first red-blue gain pair of a first channel based on color image statistical characteristics and a second red-blue gain pair of a second channel based on multispectral characteristics; determines the stability states of the two channels based on respective historical gain sequences, and determines the physical deviation degrees of the two channels according to a preset calibration white point track; and arbitrates the two gain pairs to obtain a target red-blue gain pair and outputs the target red-blue gain pair to an image processing module. The application performs parameter-level arbitration based on the stability states and the physical deviation degrees of the algorithms, breaks away from the dependence on pixel-level registration and single algorithm self-reporting, gives an interpretable and reliable fusion conclusion when the two channels are different, can identify real scene switching and noise disturbance, and improves the reliability and anti-flicker capability of white balance enhancement.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a white balance enhancement method, system, electronic device, and storage medium. Background Technology

[0002] With the development of digital imaging technology, cameras, video conferencing terminals, mobile terminals, and other devices with image acquisition capabilities typically require white balance processing of acquired images under different color temperatures and lighting conditions to ensure that neutral color areas in the image present more natural colors. Automatic white balance technology usually analyzes the color information in the image to determine the white balance gain parameters used for image signal processing, thereby correcting the red and blue channels of the image. As imaging applications continue to expand, white balance processing needs to adapt to various complex imaging environments, such as changes in indoor and outdoor lighting, mixed light sources, low illumination, and large areas of monochrome.

[0003] Existing automatic white balance schemes fall into two categories: one estimates white balance gain based on statistical features such as pixel distribution, grayscale statistics, or white point detection in color images; the other incorporates multispectral image information to determine white balance gain based on spectral features, enhancing adaptability to scenarios with insufficient white light reference and complex spectral composition. In some imaging devices, different white balance estimation channels can generate corresponding red-blue gain pairs, and the white balance gain used for subsequent image processing can be determined based on preset rules, channel priorities, or differences between candidate results.

[0004] However, under complex lighting conditions, low illumination conditions, or uneven color distribution in the image content, different white balance estimation channels may produce candidate red-blue gain pairs that differ. Existing schemes often lack effective arbitration criteria that reflect the reliability of candidate results when selecting from multiple candidate red-blue gain pairs. Therefore, it is difficult to stably determine the white balance gain suitable for the current image processing when candidate results are inconsistent, which may lead to color deviations in the output image or unstable color performance between adjacent image frames. Summary of the Invention

[0005] The purpose of this application is to provide a white balance enhancement method, system, electronic device and storage medium to solve the problem that existing multi-channel white balance estimation result fusion lacks an objective arbitration benchmark independent of each estimation algorithm itself, and to objectively arbitrate the dual-channel white balance estimation results in a parameter-level fusion manner, thereby improving the reliability and adaptability of white balance enhancement.

[0006] To achieve the above objectives, a first aspect of this application provides a white balance enhancement method, the white balance enhancement method comprising the following steps: Obtain the first red-blue gain pair for the first white balance estimation channel, determined based on the statistical characteristics of the color image, and the second red-blue gain pair for the second white balance estimation channel, determined based on the spectral characteristics of the multispectral image; Based on the first historical gain sequence corresponding to the first red-blue gain pair and the second historical gain sequence corresponding to the second red-blue gain pair, the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel are determined respectively; wherein, the first stability state and the second stability state are used to characterize whether the gain output of the corresponding white balance estimation channel is stable over time, including a stable state or an unstable state.

[0007] Based on the preset calibration white dot trajectory located in the red-blue gain parameter space, the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair are determined respectively. Based on the first stability state, the second stability state, the first physical deviation, and the second physical deviation, the first red-blue gain pair and the second red-blue gain pair are arbitrated to obtain the target red-blue gain pair; The target red-blue gain pair is output to the image signal processing module.

[0008] Optionally, determining the first stability state and the second stability state based on the first historical gain sequence and the second historical gain sequence respectively includes: For each white balance estimation channel, based on the red gain and blue gain of the most recent N frames, calculate the historical standard deviation of red gain, the historical standard deviation of blue gain, the average inter-frame difference of red gain, and the average inter-frame difference of blue gain. The jitter score of the corresponding white balance estimation channel is determined based on the historical standard deviation of the red gain, the historical standard deviation of the blue gain, the average inter-frame difference of the red gain, and the average inter-frame difference of the blue gain. Based on the comparison between the jitter score and the preset jitter threshold, it is determined whether the corresponding white balance estimation channel is in a stable or unstable state.

[0009] Optionally, the jitter score includes a standard deviation component and an inter-frame difference component; The standard deviation component is obtained by normalizing the sum of the historical standard deviations of the red gain and the historical standard deviations of the blue gain relative to a preset standard deviation sensitivity constant; The inter-frame difference component is obtained by normalizing the sum of the average inter-frame difference of the red gain and the average inter-frame difference of the blue gain relative to a preset inter-frame difference sensitivity constant. The jitter score is a weighted sum of the standard deviation component and the inter-frame difference component.

[0010] Optionally, the preset calibration white point trajectory is formed by sequentially connecting multiple standard white points obtained by the imaging device under multiple standard light sources during factory calibration in the red-blue gain parameter space; The steps of determining the first physical deviation and the second physical deviation respectively include: Use the red-blue gain pair whose physical deviation is to be determined as the current gain point; Calculate the distances from the current gain point to the line segments formed by each adjacent standard white point in the preset calibration white point trajectory; The minimum value among the distances is determined as the physical deviation of the current gain point.

[0011] Optionally, the step of arbitrating the first red-blue gain pair and the second red-blue gain pair based on the first stability state, the second stability state, the first physical deviation, and the second physical deviation to obtain the target red-blue gain pair includes: Calculate the divergence degree between the first red-blue gain pair and the second red-blue gain pair, the divergence degree being determined by the sum of the absolute difference in red gain and the absolute difference in blue gain between the first red-blue gain pair and the second red-blue gain pair; When both the first white balance estimation channel and the second white balance estimation channel are in a stable state and the degree of divergence is less than a preset divergence threshold, the first red-blue gain pair is determined as the target red-blue gain pair. When only one of the first white balance estimation channels and the second white balance estimation channel is in a stable state, the red-blue gain pair corresponding to the white balance estimation channel in the stable state is determined as the target red-blue gain pair. When both the first white balance estimation channel and the second white balance estimation channel are in a stable state and the divergence degree is not less than the preset divergence threshold, the red-blue gain pair with the smaller physical deviation is determined as the target red-blue gain pair.

[0012] Optionally, when both the first white balance estimation channel and the second white balance estimation channel are in an unstable state, the target red-blue gain pair of the previous frame is maintained, and the observation window is started. When the first white balance estimation channel and the second white balance estimation channel are detected to have recovered to a stable state within the observation window, the historical stable red-blue gain pairs before entering the unstable state and the new stable red-blue gain pairs after recovery of stability are obtained for each white balance estimation channel. Based on the historical stable red-blue gain pairs and the new stable red-blue gain pairs of each white balance estimation channel, determine the gain change direction vector of the corresponding white balance estimation channel. Based on the consistency of the gain change direction vectors of the two white balance estimation channels, it can be determined whether a real scene switch has occurred.

[0013] Optionally, determining whether a real scene switch has occurred based on the consistency of the gain change direction vectors of the two white balance estimation channels includes: When the signs of the non-zero corresponding components of the two gain change direction vectors are consistent, it is determined that a real scene switch has occurred, and the target red-blue gain pair after recovery and stabilization is taken as the gain pair to be smoothed. When there are corresponding components with opposite signs in the two gain change direction vectors, an abnormal disturbance is determined, and the historical stable red-blue gain pair is taken as the gain pair to be smoothed. The target red-blue gain pair is obtained by performing first-order infinite impulse response filtering on the gain pair to be smoothed and the red-blue gain pair output from the previous frame, and applying preset upper and lower limit constraints on the filtering result. The smoothing coefficient used when a real scene switch occurs is greater than the smoothing coefficient used when no real scene switch occurs.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a white balance enhancement system, the white balance enhancement system comprising: The dual-channel gain acquisition module is used to acquire the first red-blue gain pair determined by the statistical characteristics of the color image in the first white balance estimation channel, and the second red-blue gain pair determined by the spectral characteristics of the multispectral image in the second white balance estimation channel. The temporal stability calculation module is used to determine the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel based on the first historical gain sequence and the second historical gain sequence, respectively; wherein the first stability state and the second stability state are used to characterize whether the gain output of the corresponding white balance estimation channel is stable over time, including a stable state or an unstable state.

[0015] The physical deviation calculation module is used to determine the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair based on the preset calibration white point trajectory located in the red-blue gain parameter space. The arbitration output module is used to arbitrate the first red-blue gain pair and the second red-blue gain pair according to the first stability state, the second stability state, the first physical deviation, and the second physical deviation, to obtain a target red-blue gain pair, and output the target red-blue gain pair to the image signal processing module.

[0016] In addition, to achieve the above objectives, the present invention also provides an electronic device, the electronic device including a processor and a memory, the memory storing a computer program, the computer program being executed by the processor to implement the steps of the method described above.

[0017] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0018] Compared with existing technologies, this invention introduces dual objective benchmarks of stability state and physical deviation, independent of each estimation algorithm itself, to perform parameter-level arbitration on the dual-path white balance estimation results. This eliminates the dependence on the self-reported results of a single algorithm and eliminates the need for pixel-level spatial registration. Therefore, it can provide interpretable and reliable fusion conclusions when there are discrepancies in the results of multiple paths. Furthermore, it can combine temporal observation and direction vector consistency to identify real scene switching and noise disturbances, effectively improving the reliability, adaptability and anti-flicker capability of white balance enhancement. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a schematic flowchart of Embodiment 1 of the white balance enhancement method of this application; Figure 3 This is a schematic diagram of the system architecture of the white balance enhancement system of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0022] like Figure 1 As shown, Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of this application.

[0023] The electronic devices in the embodiments of this application can be PCs, servers, smart terminals, etc.

[0024] like Figure 1As shown, the electronic device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0025] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the electronic device of this application, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0026] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an executable program.

[0027] The operating system is a program that manages and controls electronic devices and software resources, and supports the operation of the network communication module, user interface module, executable program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0028] exist Figure 1 In the illustrated electronic device, the electronic device calls the executable program stored in the memory 1005 through the processor 1001 and performs the operations in the various embodiments of the white balance enhancement method described below.

[0029] Example 1: The white balance enhancement method provided in this invention is applied to scenarios such as automatic white balance enhancement processing or R&D verification of camera modules. For ease of description, the path for white balance estimation based on the statistical features of color images is hereinafter referred to as the first white balance estimation channel, the path for white balance estimation based on the spectral features of multispectral images is hereinafter referred to as the second white balance estimation channel, and the processing unit inside the imaging device that receives white balance enhancement processing and is used for final color correction is hereinafter referred to as the image signal processing module. The first white balance estimation channel and the second white balance estimation channel independently output their respective red-blue gain estimation results, and the two are fused and arbitrated at the upper-level processing side using the method of this application.

[0030] In this embodiment, the enhancement process first obtains the first red-blue gain pair determined by the first white balance estimation channel and the second red-blue gain pair determined by the second white balance estimation channel. Then, based on the historical gain sequence corresponding to each channel, the stability state of the two channels is determined respectively. Based on the preset calibration white point trajectory in the red-blue gain parameter space, the physical deviation of the two gain pairs is determined respectively. Finally, the stability state and physical deviation are combined to arbitrate the two gain pairs to obtain the target red-blue gain pair. The target red-blue gain pair is then output to the image signal processing module, thereby objectively fusing the dual white balance estimation results without relying on pixel-level spatial registration or simply trusting the self-reported result of one channel.

[0031] The specific steps of the white balance enhancement method according to the embodiments of the present invention will be described in detail below.

[0032] Reference Figure 2 Step S100: Obtain the first red-blue gain pair determined by the statistical features of the color image for the first white balance estimation channel, and the second red-blue gain pair determined by the spectral features of the multispectral image for the second white balance estimation channel. In this embodiment, the first white balance estimation channel extracts pixel statistical features (e.g., pixel mean under the gray-world assumption, white point statistics, etc.) from the input color image (e.g., RGB image), and estimates a set of red-blue gains for white balance correction, denoted as the first red-blue gain pair. Simultaneously, the second white balance estimation channel extracts spectral features (e.g., energy distribution and spectral response of each spectral band) from the input multispectral image, and estimates another set of red-blue gains, denoted as the second red-blue gain pair. The processing side reads these two sets of red-blue gain pairs from the two channels respectively, as input for subsequent arbitration.

[0033] Specifically, the color image upon which the first white balance estimation channel is based can be a regular RGB color image in one implementation; in another, it can be an image of any color domain (such as a feature image usable for statistics in the YUV, HSV, etc. domains) output by an imaging sensor after preliminary color interpolation. The multispectral image upon which the second white balance estimation channel is based can be the output of a multispectral sensor containing more than three channels of spectral response in one implementation; in another, it can be equivalent multispectral data obtained by combining a regular RGB sensor with narrowband filtering or by inversion through a spectral reconstruction algorithm. It should be noted that the gain estimation results of both channels are expressed in the unified parameter form of red-blue gain pairs, so that the subsequent fusion arbitration is performed entirely in the red-blue gain parameter space, without requiring strict pixel-level alignment of the two images. The further combination of the two gain pairs with historical sequences for stability evaluation is detailed in Example 2; the calculation of their physical deviation is detailed in Example 3.

[0034] The purpose of this step is to obtain the red-blue gain pairs output by each of the two white balance estimation channels as the objects to be fused and arbitrated, so as to uniformly represent the two estimation results in a parametric form.

[0035] Step S200: Based on the first historical gain sequence corresponding to the first red-blue gain pair and the second historical gain sequence corresponding to the second red-blue gain pair, determine the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel, respectively. In this embodiment, after acquiring the two red-blue gain pairs of the current frame, the processing side also maintains a time series composed of the red-blue gains output by each channel in several consecutive past frames. That is, the first historical gain sequence and the second historical gain sequence correspond to the first and second white balance estimation channels, respectively. Based on these historical gain sequences, the processing side evaluates whether the gain output of each channel is stable over time, thereby obtaining a first stability state and a second stability state (e.g., stable or unstable). The first stability state and the second stability state are used to characterize whether the gain output of the corresponding white balance estimation channel is stable over time, including a stable state or an unstable state.

[0036] Specifically, the historical gain sequence can be constructed in several ways: In one implementation, the sequence consists of the red and blue gains of each channel arranged sequentially from the most recent N consecutive frames (N is a configurable positive integer, such as 8, 16, etc.); in another implementation, the sequence can also use a sliding window with time decay to gradually reduce the influence of more distant historical frames; in yet another implementation, the sequence can also be a set of effective gain samples after removing abnormal frames within a recent period. The stability state can be determined in one implementation based on a comprehensive assessment of the dispersion of the historical sequence and the degree of inter-frame variation (specific calculations and threshold comparisons are detailed in Example 2); in another implementation, stability can also be characterized by combining other temporal features (such as trend slope and number of abrupt changes). The purpose of this step is to obtain a stability state that reflects whether the estimation results of each channel reliably fluctuate, providing a temporal basis for the reliability of the result in subsequent arbitration.

[0037] Step S300: Based on the preset calibration white dot trajectory located in the red-blue gain parameter space, determine the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair respectively. In this embodiment, the processing side has a pre-set calibration white point trajectory obtained by the factory calibration of the imaging device in the red-blue gain parameter space (i.e., a two-dimensional parameter plane with red gain and blue gain as coordinate axes); the first red-blue gain pair and the second red-blue gain pair are respectively taken as points in the parameter space, and the degree of deviation of each of them from the calibration white point trajectory is calculated to obtain the first physical deviation and the second physical deviation, which are used to characterize whether the gain pair is within the physically reasonable white point distribution range.

[0038] Specifically, there are several ways to obtain the preset calibration white point trajectory: In one implementation, the imaging device measures the corresponding standard white points under multiple standard light sources (such as D65, A light sources, CWF fluorescent lamps, etc. with different color temperatures) under a standard integrating sphere light source before leaving the factory, and connects these standard white points sequentially in the red-blue gain parameter space to form a broken line trajectory; in another implementation, the trajectory can also be generated by interpolation from a standard white point lookup table provided by the equipment manufacturer, or obtained by sampling and fitting under different illumination gradients. The calculation of physical deviation can be done in one implementation by calculating the shortest geometric distance from the current gain point to each line segment of the trajectory and taking the minimum value (detailed in Example 3); in another implementation, the normalized distance from the current gain point to the nearest point on the trajectory, or a physical reasonableness confidence function established based on the trajectory can also be used to represent the degree of deviation. The purpose of this step is to obtain an objective benchmark based on the physical calibration of the equipment, independent of the estimation algorithm of each channel, so as to determine whether a certain gain pair is physically reasonable, providing a third-party verification basis for subsequent arbitration.

[0039] Step S400: Based on the first stability state, the second stability state, the first physical deviation, and the second physical deviation, arbitrate the first red-blue gain pair and the second red-blue gain pair to obtain the target red-blue gain pair. In this embodiment, the processing side integrates the stability status of the two channels and the physical deviation of the two gain pairs, and selects or discards the first red-blue gain pair and the second red-blue gain pair according to a predetermined arbitration rule to obtain the final target red-blue gain pair for output.

[0040] Specifically, the information combination used for arbitration can be organized in several ways: In one implementation, the divergence degree between the two gain pairs (i.e., the degree of difference between them in red and blue gains) can be calculated first, and then graded arbitration can be performed based on conditions such as "whether both paths are stable", "the magnitude of the divergence", and "which path has a smaller physical deviation". In another implementation, the stability state and physical deviation can be uniformly mapped to the confidence score of each path, and then selected or weighted and fused according to the confidence score. It should be noted that, regardless of the organization method, the input of arbitration is strictly limited to the four types of information listed in this step (two types of stability states and two types of physical deviations), so as to ensure that the fusion decision is always based on independent and objective dual benchmarks. When both channels are in an unstable state, arbitration can further introduce a time-series observation and scene switching identification mechanism, and the relevant processing is detailed in Example 5. The purpose of this step is to provide a unique and interpretable target red-blue gain pair so that the fusion result still has a reliable basis in the divergence scenario.

[0041] Step S500: Output the target red-blue gain pair to the image signal processing module. In this embodiment, the processing side outputs the target red-blue gain pair obtained from the arbitration to the image signal processing module (ISP), which then performs white balance color correction on the image based on the red-blue gain pair.

[0042] Specifically, the output can take several forms: In one implementation, the red and blue gain are directly written into the configuration register or parameter interface of the image signal processing module as two scalar parameters; in another implementation, the target red-blue gain pair, along with the status information (such as stability of each channel and physical deviation) on which the arbitration is based, can be output together for logging, visualization, or further processing by the image signal processing module or upper-level system; in yet another implementation, a first-order time smoothing and gain upper and lower limit constraints can be applied to the target red-blue gain pair before output to suppress sudden changes in the image caused by decision switching (the relevant smoothing processing is detailed in Example 5). The purpose of this step is to implement the arbitration result into the actual image color correction link, completing the closed loop of white balance enhancement.

[0043] This embodiment acquires the red-blue gain pairs output by each of the two white balance estimation channels on the upper-level processing side, determines the stability state based on the historical gain sequence of each channel, and determines the physical deviation based on the preset calibration white point trajectory. Based on this, it arbitrates the two gain pairs to obtain the target red-blue gain pair and outputs it. This achieves objective white balance enhancement without relying on pixel-level spatial registration or simply trusting the self-reported results of one channel. Since the fusion arbitration is performed entirely in the red-blue gain parameter space, it completely avoids the sensor parallax and pixel-level registration problems between multispectral and color images. Because the arbitration simultaneously introduces temporal stability and factory physical calibration—objective benchmarks independent of each algorithm—it can provide interpretable and reliable fusion conclusions when the two results diverge. Furthermore, the arbitration rules rely only on the above four types of objective information, making the entire decision-making process highly interpretable and secure, effectively improving the reliability and adaptability of white balance enhancement.

[0044] Example 2: Based on the above embodiments, another embodiment of the white balance enhancement method of the present invention is proposed. The difference between this embodiment and the above embodiments lies in that, in the step of determining the first stability state and the second stability state based on the first historical gain sequence and the second historical gain sequence, a specific method for determining the stability state is further provided, including the calculation of the dispersion and inter-frame variation of the most recent N frames, the composition of the jitter score, and the comparison of the jitter score with a threshold. The steps of obtaining the dual-path gain pair, calculating the physical deviation, and arbitration in Embodiment 1 are also applicable in this embodiment. The following focuses on a detailed explanation of the determination of the stability state.

[0045] Step a: For each white balance estimation channel, based on the red and blue gains of the most recent N frames, calculate the historical standard deviation of the red gain, the historical standard deviation of the blue gain, the average inter-frame difference of the red gain, and the average inter-frame difference of the blue gain. In this embodiment, the processing side maintains a recent gain buffer of length N for each white balance estimation channel. The buffer stores the red gain sequence and blue gain sequence output by the channel in the most recent N frames. Based on this, the historical standard deviation of red gain in N frames, the historical standard deviation of blue gain in N frames, the average inter-frame difference of red gain between adjacent frames, and the average inter-frame difference of blue gain between adjacent frames are calculated respectively.

[0046] Specifically, the number N of the most recent N frames can have several values: In one implementation, N is a fixed integer (e.g., 8, 16, etc.), determined by the system configuration; in another implementation, N can be dynamically and adaptively adjusted according to the frame rate or scene, for example, increasing N in low frame rate scenes to accumulate more sufficient statistical samples; in yet another implementation, an exponential weighting method can be used to assign different weights to historical frames, in which case the historical standard deviation and the average inter-frame difference are correspondingly replaced by weighted statistics. The calculation of the historical standard deviation of red and blue gains and the average inter-frame difference is, in one implementation, obtained using conventional statistical formulas (the square root of the variance gives the standard deviation, and the mean of the differences between adjacent frames gives the inter-frame difference); in another implementation, statistics more robust to outliers (e.g., dispersion estimation based on the absolute deviation of the median) can be used instead of the ordinary standard deviation to improve resistance to sudden noise. The purpose of this step is to quantify the fluctuation of single-path gain output over time from multiple perspectives, providing a raw metric for constructing the jitter score.

[0047] Step b: Determine the jitter score of the corresponding white balance estimation channel based on the historical standard deviation of the red gain, the historical standard deviation of the blue gain, the average inter-frame difference of the red gain, and the average inter-frame difference of the blue gain. In this embodiment, the processing side combines the four statistics obtained in step a into a scalar—the jitter score—to characterize the overall jitter severity of the channel gain output.

[0048] The mathematical expression for the above four statistics and jitter score can be formalized as follows: Taking the red channel σR as an example Similarly, σB (blue channel); Similarly, ΔB (blue channel);

[0049] Where Thσ and ThΔ are preset sensitivity constants; when Jitter < 0.15, the channel is considered stable, otherwise it is considered unstable.

[0050] Specifically, there are several ways to construct the jitter score: In one implementation, the sum of the historical standard deviations of the red and blue channels, and the sum of the average inter-frame differences of the red and blue channels are normalized and then added together (the specific normalization and weighting methods are detailed in step c); in another implementation, maximum value fusion (taking the more significant jitter dimension of the two channels) or a more complex statistical learning model can be used to map to the jitter score; in yet another implementation, a correlation term between the red and blue channels can be further introduced to characterize whether there are inconsistent abnormal fluctuations between the channels. It should be noted that the jitter score is a comprehensive scalar, and a larger value usually indicates more severe jitter and less reliable estimation. The purpose of this step is to compress the multi-dimensional fluctuation features into a single comparable jitter score, which is convenient for subsequent direct comparison with the threshold.

[0051] Step c: Based on the comparison between the jitter score and the preset jitter threshold, determine whether the corresponding white balance estimation channel is in a stable or unstable state. In this embodiment, the processing side compares the jitter score obtained in step b with a preset jitter threshold: when the jitter score is lower than the preset jitter threshold, the channel is determined to be in a stable state; otherwise, the channel is determined to be in an unstable state.

[0052] Specifically, the preset jitter threshold can be determined in several ways: In one implementation, a fixed threshold calibrated at the factory is used (e.g., a schematic value of 0.15 determined based on the actual sample distribution, with the actual value determined by the calibration). In another implementation, the threshold can be adaptively adjusted according to the scene or device model; for example, the threshold can be appropriately relaxed in high-dynamic scenes to avoid frequent judgments of instability. In yet another implementation, the determination of stability and instability can be extended to multiple levels (e.g., stable, critical, jitter levels, or more), rather than just binary, thus providing a finer-grained state input for identifying genuine and fake scene switching and selecting smoothing coefficients. The purpose of this step is to binarize (or classify) the continuous jitter score into a clear stability state, serving as the direct basis for arbitration in step S400 of Example 1 and the instability processing in Example 5.

[0053] This embodiment introduces a jitter score mechanism based on the dispersion of the most recent N frames and the degree of inter-frame variation, building upon Embodiment 1, to achieve a quantitative judgment of the temporal stability of each white balance estimation channel. Since the jitter score integrates two dimensions—historical standard deviation (reflecting long-term dispersion) and average inter-frame difference (reflecting short-term abrupt changes)—it can more comprehensively characterize channel jitter than a single index. Because the jitter score outputs a clear stable / unstable state after comparison with a preset jitter threshold, it provides a unified and comparable temporal benchmark for the reliability of the result in dual-path arbitration. This allows subsequent fusion to prioritize the more stable path in divergent scenarios, effectively improving the robustness of white balance enhancement against time-varying noise and transient disturbances.

[0054] Example 3: Based on the above embodiments, another embodiment of the white balance enhancement method of the present invention is proposed. The difference between this embodiment and the above embodiments lies in that, in the step of determining the first physical deviation and the second physical deviation according to the preset calibration white point trajectory located in the red-blue gain parameter space, the origin of the preset calibration white point trajectory and the specific calculation method of the physical deviation are further explained. The steps of obtaining the dual-path gain pair, determining the stability state, and arbitration in Embodiment 1 are also applicable in this embodiment. The determination of the physical deviation will be explained in detail below.

[0055] Step d, the preset calibration white point trajectory is formed by sequentially connecting multiple standard white points obtained by the imaging device under multiple standard light sources during factory calibration in the red-blue gain parameter space: In this embodiment, during the pre-shipment calibration process, the imaging device measures the corresponding standard white points (i.e., the red-blue gain that should be obtained under ideal white balance) under multiple standard light sources (such as reference light sources with different color temperatures). These standard white points are placed in a red-blue gain parameter space with red gain and blue gain as coordinate axes, and connected sequentially according to the color temperature of the light source from low to high (or from high to low) to form a preset calibration white point trajectory with a broken line. This trajectory represents the physically reasonable white point distribution path of the device under different standard illuminations.

[0056] Specifically, there are several ways to select the standard light source and construct the trajectory: In one implementation, standard white points are measured using several international or industry standard reference light sources (such as D65, A, CWF, TL84, etc.) and connected sequentially; in another implementation, several calibration points under typical mixed light sources can be added in addition to the standard light sources to make the trajectory closer to the actual spectral distribution used; in yet another implementation, the trajectory can also be formed by spline interpolation of discrete standard white points to form a smooth curve, rather than a simple broken line. It should be noted that this preset calibration white point trajectory is determined at the factory and provided to the processing side with the equipment firmware or configuration, and has absolute physical reference significance. The purpose of this step is to provide an objective and fixed third-party benchmark for calculating physical deviation.

[0057] Step e: Take the red-blue gain pair whose physical deviation is to be determined as the current gain point; calculate the distance from the current gain point to the line segment formed by each adjacent standard white point in the preset calibration white point trajectory; determine the minimum value of each distance as the physical deviation of the current gain point: In this embodiment, the processing side regards a certain red-blue gain pair to be evaluated as a current gain point in the red-blue gain parameter space, calculates the vertical (or shortest) distance from the point to each adjacent standard white point line on the preset calibration white point trajectory, and takes the minimum value among these distances as the physical deviation of the gain point (i.e. the degree to which the gain pair deviates from the physical reasonable white point trajectory).

[0058] The mathematical expression for the deviation between the calibration curve and the physical deviation can be formalized as follows: Calibration curve: Curve = {P_1, P_2, …, P_m}, P_i = (R_i, B_i), i=1…m Physical deviation: , where X=(R,B) is the current gain point.

[0059] When D < 0.15, the gain point is considered to conform to physical laws (Valid); in the case of dual-path splitting, the one with the smaller D value (closer to the calibration curve) is accepted.

[0060] Specifically, there are several ways to calculate the distance: In one implementation, the Euclidean shortest distance from the current gain point to each line segment (i.e., the distance from the point to the foot of the perpendicular from the line segment; if the foot of the perpendicular falls outside the line segment, the distance to the endpoint is taken) is used, and the global minimum is selected. In another implementation, other metrics such as Manhattan distance and Chebyshev distance can be used, or the red-blue gain can be normalized before calculating the distance, as long as the monotonicity of smaller deviation and greater physical reasonableness is maintained. In yet another implementation, the distance can be further normalized to a physical reasonableness confidence level between 0 and 1 (e.g., using a certain calibration upper limit as the denominator) so that it can be used in arbitration in a unified manner with other dimensions such as jitter score. It should be noted that when the physical deviation is less than a certain preset value, the gain point can be determined to conform to physical laws (Valid). The specific value of this preset value is based on the measured sample distribution calibration (e.g., 0.15). The purpose of this step is to transform whether the gain point is physically reasonable into a dimensionless (or unit-based) relative quantity that can be directly used for comparison, thereby characterizing the physical reasonableness of the gain point in a device-independent manner.

[0061] This embodiment introduces a physical deviation of the geometric distance based on the factory-calibrated white point trajectory, building upon Embodiment 1, to achieve independent verification of the physical rationality of the dual-path gain. Since the physical deviation is measured by the shortest geometric distance from the midpoint of the red-blue gain parameter space to the calibration trajectory, and the calibration trajectory originates from actual measurements using a standard light source from the device's factory, it constitutes a completely independent third-party objective benchmark, separate from traditional statistical algorithms and multispectral models. Because this deviation is a dimensionless or normalizable relative quantity, it can stably characterize the physical rationality of the gain across devices and resolutions. When the dual-path results differ, the one closer to the calibration curve (i.e., with smaller physical deviation) can be accepted, effectively improving the interpretability and safety of white balance enhancement decisions.

[0062] Example 4: Based on the above embodiments, another embodiment of the white balance enhancement method of the present invention is proposed. The difference between this embodiment and the above embodiments lies in that, in the step of arbitrating the first red-blue gain pair and the second red-blue gain pair according to the first stability state, the second stability state, the first physical deviation, and the second physical deviation to obtain the target red-blue gain pair, specific arbitration rules are further provided, including the calculation of the divergence degree of the two-way gain pair and a three-branch arbitration based on the dual-path stability state and the magnitude of the divergence degree. The steps of obtaining the dual-path gain pair, determining the stability state, and calculating the physical deviation degree in Embodiment 1 are also applicable in this embodiment. The arbitration process will be described in detail below.

[0063] Step f: Calculate the divergence between the first red-blue gain pair and the second red-blue gain pair, wherein the divergence is determined by the sum of the absolute differences in red gain and blue gain between the first and second red-blue gain pairs. In this embodiment, the processing side denotes the first red-blue gain pair as (first red gain, first blue gain) and the second red-blue gain pair as (second red gain, second blue gain). It calculates the absolute difference between the two in the red gain dimension and the absolute difference in the blue gain dimension, and sums the two to obtain the divergence degree, which is used to characterize the overall separation degree of the two estimation results in the parameter space.

[0064] The mathematical expression for the above degree of divergence can be formalized as follows: Diff = |R_trad R_nn| + |B_trad B_nn| Where (R_trad, B_trad) and (R_nn, B_nn) are the first and second red-blue gain pairs, respectively; when Diff < 0.05, it is considered an undisputed pass-through, and when Diff ≥ 0.05, it is considered stable but with discrepancies.

[0065] Specifically, there are several ways to calculate the degree of divergence: In one implementation, the sum of the absolute differences in red and blue gains is directly taken (i.e., the Manhattan distance form); in another implementation, other metrics such as Euclidean distance (the square root of the sum of the squares of the two absolute differences) or Chebyshev distance (taking the larger of the two absolute differences) can also be used, as long as the monotonicity of the greater the divergence, the more significant the difference between the two paths is maintained; in yet another implementation, the red and blue gains can be normalized before calculating the degree of divergence to eliminate the influence of different gain dimensions or value ranges. The purpose of this step is to quantify the degree of inconsistency between the two estimation results into a single comparable degree of divergence, providing a triggering basis for subsequent branch arbitration.

[0066] Step g: Based on the dual-path stability state and divergence degree, obtain the target red-blue gain pair according to the following branch arbitration: In this embodiment, the processing side combines the stability state (stable or unstable) of the first white balance estimation channel and the second white balance estimation channel with the divergence degree obtained in the previous step, and determines the target red-blue gain pair according to the following three cases.

[0067] Firstly, when both the first and second white balance estimation channels are in a stable state and the divergence is less than a preset divergence threshold, the first red-blue gain pair is determined as the target red-blue gain pair. At this point, both paths are reliable and close to each other, and one of them can be trusted (in one implementation, the first white balance estimation channel is trusted; in another implementation, the second channel can be trusted, or the average of the two can be taken). The divergence is less than the threshold, indicating that the conclusions of the two paths are consistent, and there is no need to further select based on physical deviation.

[0068] Secondly, when only one of the first and second white balance estimation channels is in a stable state, the red-blue gain pair corresponding to the stable white balance estimation channel is determined as the target red-blue gain pair. Since only one path is reliable, this stable path is naturally adopted; the other path is unstable, and its estimation result is unreliable, so it is excluded.

[0069] Third, when both the first and second white balance estimation channels are in a stable state and the divergence is not less than the preset divergence threshold, the red-blue gain pair with the smaller physical deviation is determined as the target red-blue gain pair. At this time, both paths are stable but the conclusions diverge significantly. Stability alone is insufficient to determine which path to choose. Therefore, the physical deviation described in Example 3 is introduced as a third-party benchmark, and the path that is closer to the calibrated white point trajectory (i.e., with smaller physical deviation and more in line with physical laws) is accepted.

[0070] It should be noted that when both the first and second white balance estimation channels are in an unstable state, none of the above three branches are applicable. In this case, the time-series observation and scene switching identification process should be entered, and the relevant processing is detailed in Example 5. In addition, the preset divergence threshold is a calibrable empirical parameter, and the specific value is based on the actual measured sample distribution (e.g., 0.05).

[0071] The purpose of this step is to provide a hierarchical arbitration conclusion that integrates three objective information: stability state, divergence degree, and physical deviation degree, so that a unique and interpretable target red-blue gain pair can be output under different combinations of credibility.

[0072] This embodiment achieves refined fusion of the two-path white balance estimation results by introducing a divergence degree and a three-branch arbitration rule based on Embodiment 1. Since the arbitration first filters out the unreliable path based on the stability state, then determines whether a physical benchmark needs to be introduced based on the divergence degree, and finally adopts the more reasonable side when there is a bistable divergence based on the physical deviation degree, a hierarchical decision matrix without decision-making blind spots is formed. Because all judgments are based on the two independent objective benchmarks of the stability state in Embodiment 2 and the physical deviation degree in Embodiment 3, interpretable and reliable fusion conclusions can be given when the two-path results diverge, avoiding the decision-making blind spots caused by simple weighting or single-path acceptance, and effectively improving the reliability and adaptability of white balance enhancement.

[0073] Example 5: Based on the above embodiments, another embodiment of the white balance enhancement method of the present invention is proposed. The difference between this embodiment and the above embodiments lies in the supplementary processing for the case in Embodiment 4 where both channels are in an unstable state, excluding the arbitration branch. This includes maintaining the result of the previous frame and starting the observation window, extracting historical and newly stable gain pairs after stabilization to determine the gain change direction vector, identifying real scene switching and abnormal disturbances based on the consistency of the two direction vectors, and applying first-order infinite impulse response filtering and gain upper and lower limit constraints to the conclusions. The steps of obtaining dual-channel gain pairs in Embodiment 1, determining the stability state in Embodiment 2, calculating the physical deviation in Embodiment 3, and arbitration in Embodiment 4 are also applicable in this embodiment. The following focuses on a detailed explanation of the unstable state processing and scene switching identification.

[0074] Step h: When both the first white balance estimation channel and the second white balance estimation channel are in an unstable state, maintain the target red-blue gain pair from the previous frame and start the observation window. In this embodiment, when Embodiment 2 determines that both the first and second white balance estimation channels are in an unstable state (i.e., both estimations are unreliable), the processing side does not immediately adopt the current gain pair of either channel, but maintains the target red-blue gain pair already output in the previous frame unchanged, and opens an observation window to continuously monitor whether the two channels have recovered to stability over the next period of time.

[0075] Specifically, there are several ways to preserve the results of the previous frame: in one implementation, the target red-blue gain pair from the previous frame is directly used as the output of the current frame, achieving a freeze; in another implementation, a smoother gradient can be applied to the output based on the freeze to avoid extreme abrupt changes. The duration of the observation window can be determined in several ways: in one implementation, a window with a fixed number of frames or a fixed duration (such as tens of frames or hundreds of milliseconds) is used; in another implementation, the window can be dynamically expanded or contracted, for example, automatically extending the observation period when both channels are continuously jittering, and ending the wait early when either channel first stabilizes. The purpose of this step is to avoid using incorrect gain that could lead to image abnormalities when both channels are unreliable, and to reserve an observation period for subsequent identification of whether the change is a real scene transition or a brief noise disturbance.

[0076] Step i: When the first white balance estimation channel and the second white balance estimation channel are detected to have recovered to a stable state within the observation window, the historical stable red-blue gain pairs before entering the unstable state and the new stable red-blue gain pairs after recovery are obtained for each white balance estimation channel: In this embodiment, the processing side continuously judges the stability status of the two channels during the observation window. Once both channels have recovered to a stable state, the historical stable red-blue gain pairs (referred to as historical stable values) of each channel during the most recent stable period before entering the unstable state, as well as the current new stable red-blue gain pairs (referred to as new stable values) after the channel has recovered to a stable state, are extracted respectively.

[0077] Specifically, there are several ways to extract historical stable red-blue gain pairs: in one implementation, the average gain of the most recent stable frames before entering the unstable state is taken as the historical stable value; in another implementation, the gain of the last stable frame before entering the unstable state, or the median during the stable period, can also be taken as the historical stable value. The extraction of new stable red-blue gain pairs is similar; in one implementation, the average gain of the most recent stable frames after stabilization is taken. The purpose of this step is to prepare two anchor points—before and after the jump—for subsequent calculations of the direction of change of each gain path.

[0078] Step j: Based on the historical stable red-blue gain pairs and the new stable red-blue gain pairs of each white balance estimation channel, determine the gain change direction vector of the corresponding white balance estimation channel; based on the consistency of the gain change direction vectors of the two white balance estimation channels, determine whether a real scene switch has occurred. In this embodiment, the processing side subtracts the new stable value from the historical stable value of each channel to obtain the change sign of the red gain dimension and blue gain dimension of that channel, forming a gain change direction vector (for example, using positive and negative signs to indicate whether each dimension is increasing or decreasing); then compares whether the signs of the non-zero corresponding components of the two direction vectors of the first and second white balance estimation channels are consistent, so as to identify the nature of this jump.

[0079] The mathematical expression for the gain change direction vector mentioned above can be formalized as follows: Dir_trad=(sgn(R_new^trad R_old^trad),sgn(B_new^trad B_old^trad)) Dir_nn=(sgn(R_new^nn R_old^nn), sgn(B_new^nn B_old^nn) ) When the signs of the corresponding non-zero components of two vectors are completely identical, it is determined to be a jump in the same direction (real scene switching); when there are corresponding components with opposite signs, it is determined to be an abnormal disturbance.

[0080] Specifically, consistency criteria can be varied: In one implementation, when the signs of all non-zero corresponding components of the two direction vectors are completely consistent, a real scene switch is determined to have occurred (both independent algorithms consistently identify that the illumination has changed in the same direction, indicating that the real scene has indeed switched); when the two direction vectors have corresponding components with opposite signs, it is determined to be an abnormal disturbance (the two change directions contradict each other, which is more likely caused by noise or a single-path misjudgment). In another implementation, a threshold for the proportion of consistent component signs can be introduced (such as requiring the number of consistent components to be no less than a certain proportion) to relax the judgment, or a preset conservative strategy can be applied when one of the two direction vectors is a zero vector. The purpose of this step is to distinguish between real illumination scene switches and meaningless noise / misjudgment jumps by whether the change directions of the two independent algorithms corroborate each other, thereby avoiding the latter-induced erroneous color temperature flicker.

[0081] Step k: Based on the consistency identification result, select the gain pair to be smoothed, and perform first-order infinite impulse response filtering on the gain pair to be smoothed and the output red-blue gain pair of the previous frame. Apply preset upper and lower limit constraints on the gain to obtain the target red-blue gain pair. In this embodiment, the processing side selects the gain pair to be smoothed based on the identification conclusion of the previous step: if it is determined to be a real scene switch, the newly stable red-blue gain pair after recovery is used as the gain pair to be smoothed; if it is determined to be an abnormal disturbance, the historical stable red-blue gain pair is used as the gain pair to be smoothed (i.e., rejecting the current abnormal jump and reverting to the historical stable value before the jitter). Subsequently, the processing side performs first-order infinite impulse response (IIR) filtering on the gain pair to be smoothed and the red-blue gain pair output from the previous frame, and applies preset upper and lower gain limits to the filtering result to obtain the target red-blue gain pair finally output in the current frame.

[0082] The mathematical expression for the first-order IIR time smoothing described above can be formalized as follows: R_final = Clamp(α·R_target + (1 α)·R_{t 1}, R_min, R_max ), and the blue gain B_final is similar.

[0083] Where α is the smoothing coefficient: a larger value (such as 0.7) is used for rapid response during unidirectional jumps (real scene switching), and a smaller value (such as 0.3) is used for normal steady state to filter out small noise.

[0084] Specifically, there are several forms of first-order IIR filtering: In one implementation, R_final = α·R_target + (1 α)·R_{t The form of 1} (the same applies to blue gain), where α is the smoothing coefficient, R_target is the gain to be smoothed, and R_{t} is the target gain. 1) represents the output gain of the previous frame. In another implementation, other first-order recursive filtering or limiting low-pass filtering methods can also be used, as long as they can smooth out abrupt changes. The preset gain upper and lower limits (i.e., the Clamp operation) in one implementation limit the filtering result to between the minimum and maximum red-blue gain allowed by the device, preventing out-of-bounds errors. The value of the smoothing coefficient α varies depending on the scenario in one implementation: a larger value (e.g., 0.7 in the example) is used when a real scene switch occurs to quickly respond to the new scene; a smaller value (e.g., 0.3 in the example) is used when no real scene switch occurs (abnormal disturbance fallback) to filter out minor noise. All the above values ​​are based on actual measurement calibration. The purpose of this step is to smoothly and restrictively implement the arbitration / identification conclusion into the final output, enabling both rapid following during real switches and stable suppression of flicker during abnormal disturbances.

[0085] This embodiment, by supplementing the observation window and direction vector consistency identification when both paths are unstable, based on embodiments 1, 2, and 4, achieves accurate differentiation between real scene switching and noise / false judgment disturbances. Since the identification criterion is whether the gain change directions of the two independent algorithms corroborate each other, it can effectively filter out erroneous color temperature jumps caused by single-path misjudgments or occasional noise. By applying first-order IIR filtering and gain upper and lower limit constraints to the identified conclusions, and using different smoothing coefficients depending on whether a scene is switching, it can quickly follow new lighting during real scene switching and stably back off to avoid flickering during abnormal disturbances. This gives white balance enhancement the ability to distinguish between true and false jumps without flickering, further improving image stability and user experience in complex dynamic scenes.

[0086] This application also provides a white balance enhancement system. (See reference...) Figure 3 The white balance enhancement system includes a dual-channel gain acquisition module 10, a timing stability calculation module 20, a physical deviation calculation module 30, and an arbitration output module 40. The four functional modules respectively undertake the actions defined in steps S100, S200, S300, S400 to S500 of the method in Embodiment 1.

[0087] Specifically, the dual-channel gain acquisition module is used to acquire the first red-blue gain pair determined by the first white balance estimation channel and the second red-blue gain pair determined by the second white balance estimation channel, corresponding to step S100 in Embodiment 1. In one implementation, this module can be composed of a dual-channel reading unit that is respectively connected to the color image statistical path and the multispectral image spectral path, or it can be composed of a single-channel alternating reading or interrupt-triggered reading unit. The temporal stability calculation module is used to determine the first stability state and the second stability state based on the first historical gain sequence and the second historical gain sequence, respectively, corresponding to step two in Embodiment 1 and Embodiment 2. In one implementation, this module can further include a recent N-frame gain buffer subunit and a jitter score calculation subunit. The physical deviation calculation module is used to determine the first physical deviation and the second physical deviation based on the preset calibration white point trajectory, respectively, corresponding to step S300 in Embodiment 1 and Embodiment 3. In one implementation, this module can include a calibration trajectory storage subunit and a point-to-polyline distance calculation subunit. The arbitration output module is used to arbitrate the two gain pairs based on the overall stability state and physical deviation to obtain the target red-blue gain pair and output it to the image signal processing module, corresponding to steps S400 and S500 in Embodiment 1 and Embodiments 4 and 5. In one implementation, the module may include a divergence calculation subunit, a three-branch arbitration subunit, and an unstable state observation and smoothing subunit.

[0088] It should be noted that the above division of the four functional modules is only a logical division. In actual deployment, they can be deployed within the same processing unit process or distributed. In one implementation, the system itself is a general-purpose computer or image processing unit running the method of the embodiments of this invention. In another implementation, the system can also be a hardware acceleration box or test fixture dedicated to white balance enhancement, with the above four modules implemented internally in software or firmware. In yet another implementation, each module can also be implemented by hardware logic circuits such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). The purpose of this system is to solidify the method flow of parameter-level fusion arbitration dual-channel white balance estimation on the upper-level processing side into a reusable processing device.

[0089] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the white balance enhancement method as described in any of the above embodiments.

[0090] Specifically, the processor can be implemented in various ways: in one implementation, the processor is a general-purpose central processing unit (CPU); in another, the processor can be a microcontroller (MCU), digital signal processor (DSP), system-on-a-chip (SoC), graphics processing unit (GPU), neural network processor (NPU), image signal processor (ISP), or application-specific integrated circuit (ASIC), etc.; the memory can be random access memory (RAM), read-only memory (ROM), flash memory, or any combination thereof. In yet another implementation, the electronic device itself is the processing unit described in the above embodiments, and the dual-channel acquisition, stability calculation, physical deviation calculation, and arbitration functions required for white balance enhancement are all implemented by the processor in the electronic device executing the program in memory. The purpose of this electronic device is to carry out the method of this solution on a standardized hardware carrier, so that white balance enhancement can run stably on terminals, industrial control computers, testing instruments, or embedded terminals.

[0091] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the white balance enhancement method as described in any of the above embodiments.

[0092] Specifically, the computer-readable storage medium can be implemented in various forms: in one implementation, it is a removable medium such as an optical disc, USB flash drive, or secure digital card (SD card); in another implementation, it can be a fixed storage medium such as a solid-state drive (SSD), an embedded multimedia card (eMMC), or non-volatile memory (such as EEPROM, flash memory chips); in yet another implementation, the computer program can also be stored on the network side or in cloud storage (such as network attached storage, NAS), and remotely invoked and executed by a processor. It should be noted that the above storage media are merely examples; any medium capable of storing a computer program and being read and executed by a processor to implement the method of this solution is within the scope of protection of this application. The purpose of this medium is to enable the method of this solution to be widely deployed and reused in a distributable and reusable form.

[0093] In this embodiment, the various thresholds involved in the aforementioned embodiments (such as the preset jitter threshold, preset standard deviation sensitivity constant, preset inter-frame difference sensitivity constant in Embodiment 2, the preset divergence threshold in Embodiment 4, the physical deviation reasonable threshold in Embodiment 3, the smoothing coefficient and gain upper and lower limits in Embodiment 5, etc.) and various quantitative parameters (such as the most recent N frames in Embodiment 2) are all calibrable empirical parameters, and their specific values ​​are determined based on the actual sample distribution of the device under test.

[0094] Specifically, there are several calibration methods: In one implementation, a sample set is first constructed using sample devices with known lighting scenarios, and the numerical distribution of each relative quantity under each scenario and abnormal situation is statistically analyzed. The boundary points of each threshold are determined based on the overlapping areas of the distributions. In another implementation, testers can iteratively fine-tune the thresholds based on experience and the results of on-site trial runs. It should be noted that since this embodiment extensively uses relative quantities such as physical deviation and divergence as the criteria for judgment, and these relative quantities are dimensionless or normalized ratios, the calibration results are transferable between different resolutions, different photosensitive devices, and even different camera models. There is no need to redesign the absolute threshold for each model—this is precisely the implementation convenience brought about by the unit independence of this solution. The purpose of this explanation is to enable those skilled in the art to reproduce the threshold setting based on the methodology disclosed in this solution, without relying on a fixed value.

[0095] In this embodiment, for the technical features disclosed in the foregoing embodiments, any equivalent substitutions that use substantially the same means, achieve substantially the same function, and reach substantially the same effect, and that could be conceived by a person skilled in the art without inventive effort before the application date, are all within the scope of protection of this solution. For example: replacing the geometric distance physical deviation in the red-blue gain parameter space with monotonic transformations such as logarithmic transformation and normalization; replacing the jitter score from a weighted average of standard deviation components and inter-frame difference components with other equivalent multidimensional fluctuation fusion forms; replacing the three-branch arbitration with equivalent confidence score ranking or weighted fusion; replacing the first-order IIR filter with other first-order recursive smoothing or limiting low-pass filtering; swapping the roles of color image statistics / multispectral analysis between the first white balance estimation channel and the second white balance estimation channel, or extending it to more than two parametric-level fusions—all of the above equivalent transformations do not depart from the inventive concept of this solution.

[0096] The white balance enhancement method, system, device, and computer-readable storage medium provided by this invention can be practically applied in at least one of the following industrial scenarios: First, automatic white balance testing and enhancement on camera module mass production lines, quickly determining and fusing dual-channel white balance estimation results when each module comes off the production line; Second, R&D verification and performance sampling of camera modules, evaluating the stability and fusion effect of the white balance algorithm under different lighting and mixed light source scenarios; Third, factory quality inspection of terminal devices (such as video conferencing terminals, mobile phones, and computer cameras), verifying whether the white balance enhancement function of the whole device meets the standards; Fourth, automated regression testing and quality traceability, using the stability status, physical deviation, and arbitration conclusions output by this solution in a structured manner to continuously compare the white balance performance after firmware version iteration. Because this solution replaces pixel-level registration with parameter-level fusion and is easy to deploy in software or embedded form, it can be directly embedded into existing production lines and quality inspection processes, and has clear industrial applicability.

[0097] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0098] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

Claims

1. A white balance enhancement method, characterized in that, The white balance enhancement method includes: Obtain the first red-blue gain pair for the first white balance estimation channel, determined based on the statistical characteristics of the color image, and the second red-blue gain pair for the second white balance estimation channel, determined based on the spectral characteristics of the multispectral image; Based on the first historical gain sequence corresponding to the first red-blue gain pair and the second historical gain sequence corresponding to the second red-blue gain pair, the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel are determined respectively; wherein, the first stability state and the second stability state are used to characterize whether the gain output of the corresponding white balance estimation channel is stable over time, including a stable state or an unstable state. Based on the preset calibration white dot trajectory located in the red-blue gain parameter space, the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair are determined respectively. Based on the first stability state, the second stability state, the first physical deviation, and the second physical deviation, the first red-blue gain pair and the second red-blue gain pair are arbitrated to obtain the target red-blue gain pair; The target red-blue gain pair is output to the image signal processing module.

2. The white balance enhancement method according to claim 1, characterized in that, The step of determining the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel based on the first historical gain sequence and the second historical gain sequence, respectively, includes: For each white balance estimation channel, based on the red gain and blue gain of the most recent N frames, calculate the historical standard deviation of red gain, the historical standard deviation of blue gain, the average inter-frame difference of red gain, and the average inter-frame difference of blue gain. The jitter score of the corresponding white balance estimation channel is determined based on the historical standard deviation of the red gain, the historical standard deviation of the blue gain, the average inter-frame difference of the red gain, and the average inter-frame difference of the blue gain. Based on the comparison between the jitter score and the preset jitter threshold, it is determined whether the corresponding white balance estimation channel is in a stable or unstable state.

3. The white balance enhancement method according to claim 2, characterized in that, The jitter score includes the standard deviation component and the inter-frame difference component; The standard deviation component is obtained by normalizing the sum of the historical standard deviations of the red gain and the historical standard deviations of the blue gain relative to a preset standard deviation sensitivity constant; The inter-frame difference component is obtained by normalizing the sum of the average inter-frame difference of the red gain and the average inter-frame difference of the blue gain relative to a preset inter-frame difference sensitivity constant. The jitter score is a weighted sum of the standard deviation component and the inter-frame difference component.

4. The white balance enhancement method according to claim 1, characterized in that, The preset calibration white point trajectory is formed by sequentially connecting multiple standard white points obtained by the imaging device under multiple standard light sources during factory calibration in the red-blue gain parameter space; The step of determining the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair includes: Use the red-blue gain pair whose physical deviation is to be determined as the current gain point; Calculate the distances from the current gain point to the line segments formed by each adjacent standard white point in the preset calibration white point trajectory; The minimum value among the distances is determined as the physical deviation of the current gain point.

5. The white balance enhancement method according to claim 1, characterized in that, The step of arbitrating the first red-blue gain pair and the second red-blue gain pair based on the first stability state, the second stability state, the first physical deviation, and the second physical deviation to obtain the target red-blue gain pair includes: Calculate the divergence degree between the first red-blue gain pair and the second red-blue gain pair, the divergence degree being determined by the sum of the absolute difference in red gain and the absolute difference in blue gain between the first red-blue gain pair and the second red-blue gain pair; When both the first white balance estimation channel and the second white balance estimation channel are in a stable state and the degree of divergence is less than a preset divergence threshold, the first red-blue gain pair is determined as the target red-blue gain pair. When both the first white balance estimation channel and the second white balance estimation channel are in a stable state and the divergence degree is not less than the preset divergence threshold, the red-blue gain pair with the smaller physical deviation is determined as the target red-blue gain pair.

6. The white balance enhancement method according to claim 5, characterized in that, When both the first white balance estimation channel and the second white balance estimation channel are in an unstable state, maintain the target red-blue gain pair of the previous frame and start the observation window; When the first white balance estimation channel and the second white balance estimation channel are detected to have recovered to a stable state within the observation window, the historical stable red-blue gain pairs before entering the unstable state and the new stable red-blue gain pairs after recovery of stability are obtained for each white balance estimation channel. Based on the historical stable red-blue gain pairs and the new stable red-blue gain pairs of each white balance estimation channel, determine the gain change direction vector of the corresponding white balance estimation channel. Based on the consistency of the gain change direction vectors of the two white balance estimation channels, it can be determined whether a real scene switch has occurred.

7. The white balance enhancement method according to claim 6, characterized in that, The step of determining whether a real scene switch has occurred based on the consistency of the gain change direction vectors of the two white balance estimation channels includes: When the signs of the non-zero corresponding components of the two gain change direction vectors are consistent, it is determined that a real scene switch has occurred, and the target red-blue gain pair after recovery and stabilization is taken as the gain pair to be smoothed. When there are corresponding components with opposite signs in the two gain change direction vectors, an abnormal disturbance is determined, and the historical stable red-blue gain pair is taken as the gain pair to be smoothed. The target red-blue gain pair is obtained by performing first-order infinite impulse response filtering on the gain pair to be smoothed and the red-blue gain pair output from the previous frame, and applying preset upper and lower limit constraints on the filtering result. The smoothing coefficient used when a real scene switch occurs is greater than the smoothing coefficient used when no real scene switch occurs.

8. A white balance enhancement system, characterized in that, The white balance enhancement system includes: The dual-channel gain acquisition module is used to acquire the first red-blue gain pair determined by the statistical characteristics of the color image in the first white balance estimation channel, and the second red-blue gain pair determined by the spectral characteristics of the multispectral image in the second white balance estimation channel. The temporal stability calculation module is used to determine the first stability state of the first white balance estimation channel and the second stability state of the second white balance estimation channel based on the first historical gain sequence and the second historical gain sequence, respectively; wherein, the first stability state and the second stability state are used to characterize whether the gain output of the corresponding white balance estimation channel is stable over time, including a stable state or an unstable state. The physical deviation calculation module is used to determine the first physical deviation of the first red-blue gain pair and the second physical deviation of the second red-blue gain pair based on the preset calibration white point trajectory located in the red-blue gain parameter space. The arbitration output module is used to arbitrate the first red-blue gain pair and the second red-blue gain pair according to the first stability state, the second stability state, the first physical deviation, and the second physical deviation, to obtain a target red-blue gain pair, and output the target red-blue gain pair to the image signal processing module.

9. An electronic device, characterized in that, It includes a processor and a memory; the memory stores a computer program; when the computer program is executed by the processor, the electronic device performs the white balance enhancement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the white balance enhancement method according to any one of claims 1 to 7.

Images