White balance parameter determination methods, devices, equipment, storage media, and software products.

Abstract

This application discloses a method, apparatus, device, storage medium, and program product for determining white balance parameters, relating to the field of image data processing technology. The white balance parameter determination method includes: acquiring original color image data and multispectral image data from the same scene; acquiring weight scores for each band of the multispectral image data, and adaptively weighting and compressing the multispectral image data based on the weight scores to obtain multispectral compressed features; extracting features from the original color image data to obtain depth features and statistical prior features; and predicting the multispectral compressed features, depth features, and statistical prior features to obtain the white balance parameters of the original color image data. This application, by introducing adaptive weighted compression, reduces the computational load required for subsequent feature extraction and parameter prediction while preserving multispectral information gain, enabling the entire white balance parameter determination method to be deployed on resource-constrained edge devices.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, and in particular to methods, apparatus, devices, storage media and program products for determining white balance parameters. Background Technology

[0002] Automatic white balance (AWB) is a critical processing step in imaging systems. Its main function is to determine the gain parameters of the image's color channels under different lighting conditions, so that the overall color of the output image is as close as possible to the real scene. Existing AWB technologies typically predict white balance control parameters such as Rgain and Bgain based on RGB image statistical rules, empirical models, or neural network models, and perform corrections through ISP, AIQ, or AWB modules.

[0003] While existing multispectral-assisted white balance technologies have proposed schemes to incorporate multispectral information into color correction or white balance control, significant limitations still exist in different approaches. Due to the large number of multispectral input channels and significant inter-spectral redundancy, directly feeding the original multispectral channels into the backbone network would result in a large number of model parameters and computational load, which is not conducive to real-time deployment in video conferencing terminals, UVC devices, or edge chips. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, device, storage medium and program product for determining white balance parameters, which aims to solve the technical problem that the existing solutions have a large number of multispectral input channels, which is not conducive to real-time deployment.

[0005] To achieve the above objectives, this application proposes a method for determining white balance parameters, the method comprising: Acquire raw color image data and multispectral image data of the same scene; The weight scores of each band of the multispectral image data are obtained, and the multispectral image data is adaptively weighted and compressed based on the weight scores to obtain multispectral compression features. Feature extraction is performed based on the original color image data to obtain depth features and statistical prior features; The white balance parameters of the original color image data are obtained by predicting the multispectral compression features, the depth features, and the statistical prior features.

[0006] In some implementations, the step of adaptively weighting and compressing the multispectral image data based on the weighted scores to obtain multispectral compressed features includes: The weighted multispectral image data is obtained by weighting each band of the multispectral image data based on the weighted score. The weighted multispectral image data is subjected to correlation enhancement and fixed-dimensional channel compression to obtain multispectral compressed features.

[0007] In some implementations, the step of obtaining the weighted scores for each band of the multispectral image data includes: Global pooling is performed on each band of the multispectral image data to obtain statistical data for each band. The weighted scores of each band of the multispectral image data are determined based on the statistical data.

[0008] In some implementations, the step of extracting features from the original color image data to obtain depth features and statistical prior features includes: The original color image data is input into the color coding branch for depth feature extraction to obtain the depth features of the original color image data; The statistical prior features of the original color image data are obtained by performing feature statistics on the original color image data.

[0009] In some embodiments, the step of predicting the white balance parameters of the original color image data by analyzing the multispectral compressed features, the depth features, and the statistical prior features includes: Statistical confidence assessment is performed based on the aforementioned statistical prior features to obtain statistical confidence weights; The multispectral compressed features, the depth features, and the statistical prior features with the statistical confidence weights are fused to obtain joint characterization features; The joint characterization features are input into a dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data.

[0010] In some embodiments, the dual-head prediction structure includes a first prediction head and a second prediction head; the step of inputting the joint representation features into the dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data includes: The basic reciprocal gain parameter is obtained by predicting the joint representation features using the first prediction head. The joint characterization features are predicted using the second prediction head to obtain residual correction parameters; The white balance parameters of the original color image data are obtained by combining the basic reciprocal gain parameter and the residual correction parameter.

[0011] Furthermore, to achieve the above objectives, this application also proposes a white balance parameter determination device, which includes: The data acquisition module is used to acquire raw color image data and multispectral image data of the same scene; The weighted compression module is used to obtain the weight scores of each band of the multispectral image data, and to perform adaptive weighted compression on the multispectral image data based on the weight scores to obtain multispectral compression features. The feature extraction module is used to extract features based on the original color image data to obtain depth features and statistical prior features; The prediction module is used to predict the multispectral compressed features, the depth features, and the statistical prior features to obtain the white balance parameters of the original color image data.

[0012] In addition, to achieve the above objectives, this application also proposes a white balance parameter determination device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the white balance parameter determination method as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the white balance parameter determination method described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the white balance parameter determination method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This application acquires raw color image data and multispectral image data from the same scene; obtains weight scores for each band of the multispectral image data, and performs adaptive weighted compression on the multispectral image data based on the weight scores to obtain multispectral compressed features; extracts features from the raw color image data to obtain depth features and statistical prior features; and predicts the white balance parameters of the raw color image data using the multispectral compressed features, depth features, and statistical prior features. Compared to existing technologies that directly feed raw multispectral data containing dozens or even hundreds of bands into a neural network for processing, this application introduces adaptive weighted compression. First, it scores the effectiveness or importance of each band, and then weights and compresses the multispectral data based on the scoring results, ultimately outputting a fixed and relatively small number of multispectral compressed features. This process eliminates redundant bands, suppresses invalid information, and only transmits key spectral information to subsequent networks. This significantly reduces the computational load required for subsequent feature extraction and parameter prediction while preserving multispectral information gain, enabling the entire white balance parameter determination method to be deployed on resource-constrained edge devices. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for determining white balance parameters in Embodiment 1 of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the method for determining white balance parameters in this application. Figure 3 This is a flowchart illustrating Embodiment 3 of the method for determining white balance parameters in this application. Figure 4 This is a schematic diagram of the module structure of the white balance parameter determination device according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware operating environment involved in the white balance parameter determination method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: acquire color original image data and multispectral image data in the same scene; acquire the weight score of each band of the multispectral image data, and perform adaptive weighted compression on the multispectral image data based on the weight score to obtain multispectral compression features; extract features based on the color original image data to obtain depth features and statistical prior features; predict the multispectral compression features, depth features and statistical prior features to obtain the white balance parameters of the color original image data.

[0023] This application provides a solution that, by using adaptive weighted compression based on weighted scoring to output fixed-dimensional features, eliminates redundant bands while retaining key spectral information. Furthermore, it compresses the input dimension to a range easily processed by the backend network, reducing the computational load of the original multispectral scheme while improving the accuracy of white balance parameter calculation and enhancing the real-time deployment capability of white balance parameter calculation. The channel dimension after mapping and compression is fixed and does not change with scene variations; therefore, the structure of the backend neural network can be completely fixed without dynamic adjustment. Simultaneously, the weights of each band change with the scene during compression, achieving a "fixed interface + adaptive content" approach. This ensures both the universality of the model structure and the ability to flexibly adapt to different lighting conditions. This allows the module to be "plug and play" like a regular convolutional layer, greatly reducing the difficulty of engineering integration.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, computer, server, etc., or an electronic device or virtual device capable of performing the above functions. The following description uses a white balance parameter determining device (hereinafter referred to as the determining device) as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a method for determining white balance parameters, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the method for determining white balance parameters in this application, as provided in Embodiment 1.

[0026] In this embodiment, the white balance parameter determination method includes steps S10 to S40: Step S10: Obtain color raw image data and multispectral image data of the same scene.

[0027] It should be noted that the aforementioned raw color image data refers to the raw pixel data directly output by the RGB image sensor, such as RGB RAW data. The aforementioned multispectral image data can be image data acquired through a multispectral sensor. When acquiring data through a multispectral sensor, light intensity information across multiple wavelength ranges can be collected. Each wavelength range can correspond to an independent band channel, meaning that multispectral image data can contain image data from multiple continuous or discrete bands. For example, a multispectral sensor may contain 16 bands, covering different regions from visible light to near-infrared.

[0028] It should be understood that the aforementioned raw color image data and multispectral image data can have a corresponding relationship, that is, data obtained by shooting the same scene.

[0029] In some embodiments of this application, in order to improve computational accuracy, the original color image data and multispectral image data may be preprocessed. Specifically, one or more preprocessing methods such as Black Level Correction (BLC), Lens Shading Correction (LSC), bad pixel correction, normalization, downsampling, or size unification may be performed on the original color image data and multispectral image data.

[0030] In some embodiments of this application, when there is a difference in field of view or installation deviation between the RGB image sensor and the multispectral sensor, the embodiments of this application may further perform preprocessing methods such as local cropping, spatial alignment or scale unification processing to achieve alignment between the original color image data and the multispectral image data.

[0031] In a specific implementation, the determining device in this application embodiment can acquire RGB RAW image data and multispectral RAW image data corresponding to the same scene as model input within the same exposure time or a preset synchronization time window.

[0032] Step S20: Obtain the weight score of each band of the multispectral image data, and perform adaptive weighted compression on the multispectral image data based on the weight score to obtain multispectral compression features.

[0033] It should be noted that for different bands, a weighted score can be used to characterize their importance for white balance estimation in the current scene. The higher the weighted score, the more critical the information of that band. Through adaptive weighted compression, multiple bands can be mapped to a fixed channel dimension output based on the weighted score, resulting in feature data with a fixed number of channels, i.e., multispectral compressed features. These multispectral compressed features retain the information most useful for white balance determination in the original multispectral image data.

[0034] In its implementation, the device determines the weight score of each band in the multispectral image data within the current scene when acquiring the data. Based on the weight score of each band, adaptive weighted compression processing is performed on the multispectral image data. Specifically, the device multiplies the image data of each band by its corresponding weight score, enhancing bands of high importance and suppressing bands of low importance. Subsequently, the device maps the weighted multiband data into multispectral compressed features with fewer channels through a linear transformation with a fixed output dimension (e.g., a convolutional layer or fully connected layer with a fixed number of output channels). This method ensures a fixed number of output feature channels, while the information distribution within each output channel changes with the environment, thus solving the model redundancy and deployment burden problem caused by directly feeding multiple channels into the backbone network in multispectral assisted white balance schemes.

[0035] Step S30: Based on the original color image data, feature extraction is performed to obtain depth features and statistical prior features; Step S40: Predict the multispectral compression features, the depth features, and the statistical prior features to obtain the white balance parameters of the original color image data.

[0036] It should be noted that the aforementioned deep features are high-level semantic features automatically extracted from raw color image data through neural networks or deep learning models. These deep features can characterize abstract information such as texture, edges, local structures, and color distribution patterns in the image. For example, a deep feature can be the feature map output by the last convolutional layer in a convolutional neural network.

[0037] It should be explained that the aforementioned statistical prior features can be features obtained based on the pixel value distribution, color histogram, or other mathematical statistics of the original color image data, used to reflect the color distribution patterns of the original color image data. For example, statistical prior features can be the histogram statistics of the red, green, and blue channels, or the color distribution information obtained by dividing the image into multiple local regions and statistically analyzing each region separately.

[0038] It is understood that the final white balance parameter can be obtained by using multispectral compressed features, depth features, and statistical prior features as inputs, and predicting them through network layers or regression models. This white balance parameter can be a gain coefficient used to correct color deviations in the original color image data, and may include red channel gain values, blue channel gain values, green channel gain values, etc., which are not limited in this embodiment. For example, the white balance parameter can be a vector (Red gain, Blue gain), where Red gain is the red channel gain value, Blue gain is the blue channel gain value, and the green channel gain value can be 1.

[0039] In practical implementation, when the device obtains raw color image data, it can perform different types of feature extraction operations on the raw color image data, namely depth feature extraction and statistical prior feature extraction. For depth feature extraction, the raw color image data can be input into a pre-trained deep neural network. This network abstracts the high-level representation of the image layer by layer through multiple convolutions and nonlinear transformations, ultimately outputting depth features. For statistical prior feature extraction, the device does not rely on a neural network but directly performs mathematical statistical calculations on the raw color image data. For example, it statistically analyzes the pixel value distribution of red, green, and blue channels in the entire image or a local region, calculates the color mean, variance, or histogram, and organizes these calculation results into statistical prior features. Depth features and statistical prior features reflect different attributes of the raw color image data at the semantic and statistical levels, respectively, and complement each other. When determining multispectral compression features, depth features, and statistical prior features, these features can be fused, and predictions can be made based on the fused features to obtain white balance parameters.

[0040] This application's embodiments acquire raw color image data and multispectral image data from the same scene; obtain weight scores for each band of the multispectral image data, and adaptively weighted compress the multispectral image data based on the weight scores to obtain multispectral compressed features; extract features based on the raw color image data to obtain depth features and statistical prior features; and predict the white balance parameters of the raw color image data using the multispectral compressed features, depth features, and statistical prior features. Compared to existing technologies that directly feed raw multispectral data containing dozens or even hundreds of bands into a neural network for processing, this application introduces adaptive weighted compression. First, it scores the effectiveness or importance of each band, and then weights and compresses the multispectral data based on the scoring results, ultimately outputting a fixed and relatively small number of multispectral compressed features. This process eliminates redundant bands, suppresses invalid information, and only transmits key spectral information to subsequent networks, thereby significantly reducing the computational load required for subsequent feature extraction and parameter prediction while preserving multispectral information gain. This allows the entire white balance parameter determination method to be deployed on resource-constrained edge devices.

[0041] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating Embodiment 2 of the method for determining white balance parameters in this application.

[0042] like Figure 2 As shown in the embodiment of this application, the step of adaptively weighting and compressing the multispectral image data based on the weight score to obtain multispectral compressed features includes: Step S210: Based on the weighted score, weight each band of the multispectral image data is applied to obtain the weighted multispectral image data; Step S220: Perform correlation enhancement and fixed-dimensional channel compression on the weighted multispectral image data to obtain multispectral compressed features.

[0043] It should be noted that in this embodiment, each band of the multispectral image data is adaptively enhanced through weighted scoring, thereby enhancing key bands and suppressing redundant bands to obtain weighted multispectral data. Subsequently, 1×1 convolution or cascaded 1×1 convolution can be used to enhance the correlation and compress the channels of the weighted multispectral data to generate a compact multispectral feature representation, i.e., multispectral compressed features.

[0044] It should be explained that the channel compression in this embodiment refers to dynamically adjusting the contribution (i.e., weight scoring) of the output bands according to the current scene, while keeping the number of compressed output channels fixed. Then, a fixed-dimensional compressive convolution maps the multi-band input into a smaller number of compact feature channels. In other words, the number of output feature channels is fixed, but the information distribution carried in each output channel changes with the environment. For example, weighted data of 16 bands can be compressed into feature data for 4 fixed output channels. This design can solve the model redundancy and deployment burden problems caused by directly feeding multiple channels into the backbone network in existing multispectral assisted white balance schemes.

[0045] In one example embodiment of this application, one or more 1×1 convolutional layers can be used to process weighted multispectral image data. The output channels of this 1×1 convolutional layer are fixed at K, and the dimension of these output channels is much smaller than the feature dimension C of the weighted multispectral image data. The kernel weights of this 1×1 convolutional layer can be... For each spatial location in the weighted multispectral image data It can output the corresponding features, such as: ; in, Indicates multispectral compression characteristics, This represents the channel index value. This adaptive weighting method ensures that each output channel k is a linear weighted sum of all input bands, thereby mixing and recombining relevant information originally scattered across different bands, enhancing the correlation between different bands.

[0046] In some embodiments of this application, in order to perform weighted scoring on each band, the step of obtaining the weighted scores of each band of the multispectral image data includes: performing global pooling based on each band in the multispectral image data to obtain statistical data for each band; and determining the weighted scores of each band of the multispectral image data based on the statistical data.

[0047] It should be noted that the embodiments of this application may include a band response scoring module. By inputting multispectral data into the band response scoring module, the effectiveness or importance of each multispectral band in the current scene is evaluated, and a weight score corresponding to each band is generated. The weight score can be constructed based on at least one of the following information: (1) the global mean, variance, contrast or brightness distribution of each band; (2) the degree of difference, correlation or redundancy between each band; (3) the strength and trend of band response in the local area; and (4) the band-by-band attention weight generated by the channel attention mechanism.

[0048] In some embodiments of this application, global pooling can be performed on multispectral images to obtain statistical data for each band, and then weighted scores for each band can be generated through a mapping layer, attention layer, or multilayer perceptron.

[0049] It is understood that global pooling is a data processing method that aggregates multispectral image data along its spatial dimension, compressing the two-dimensional pixel matrix corresponding to each band into a scalar value. This scalar value is the statistical data, which can summarize the overall response intensity or distribution characteristics of the corresponding band in the current image. The obtained statistical data can be directly used as the weight score for the corresponding band, or the statistical data for each band can be normalized and then used as the weight score, or these statistical data can be input into a pre-trained mapping network (e.g., a lightweight network composed of fully connected layers) to output a weight score. This application does not limit this approach. Through these methods, a weight score can be assigned to each band of the multispectral image data. This weight score reflects the response intensity or information richness of the corresponding band in the current scene. The higher the weight score, the greater the contribution of that band to the white balance parameters in the current scene.

[0050] This application embodiment obtains statistical data for each band by performing global pooling on each band of the multispectral image data; determines the weight score of each band of the multispectral image data based on the statistical data; weights each band of the multispectral image data based on the weight scores to obtain weighted multispectral image data; and performs correlation enhancement and fixed-dimensional channel compression on the weighted multispectral image data to obtain multispectral compression features. Compared to directly using the original multispectral image data, this application embodiment achieves scene-driven band enhancement / suppression by adaptively weighting the multispectral image data based on weight scores. By using fixed-dimensional channel compression, it reduces the computational resources required. This design is particularly suitable for devices with limited computing resources that require real-time processing (such as video conferencing terminals, mobile phones, and edge cameras), and can introduce multispectral information into the white balance parameter determination process without significantly increasing hardware costs.

[0051] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and / or second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the method for determining white balance parameters in Embodiment 3 of this application.

[0052] like Figure 3As shown in the embodiment of this application, the step of extracting features based on the original color image data to obtain depth features and statistical prior features includes: Step S310: Input the original color image data into the color coding branch for depth feature extraction to obtain the depth features of the original color image data; Step S320: Perform feature statistics on the original color image data to obtain the statistical prior features of the original color image data.

[0053] It should be noted that the aforementioned color coding branch can be a neural network branch that extracts deep features from raw color image data. This branch can include one or more convolutional layers, pooling layers, activation function layers, and fully connected layers. Through layer-by-layer nonlinear transformations, it maps the input raw pixel data into a high-dimensional semantic feature representation, thereby extracting deep features related to color distribution, brightness structure, and local texture.

[0054] It should be explained that the statistical prior features in this application embodiment can be RGB Hist features, which can be obtained in at least one of the following ways: (1) Statistically calculate the global R, G, and B three-channel histograms for the entire RGB image; (2) Divide the RGB image into several local regions and calculate the histograms for each region respectively; (3) Perform channel normalization, brightness normalization, or logarithmic compression on the histograms to reduce the impact of exposure changes.

[0055] In some embodiments of this application, the RGB Hist features can be obtained by using a partitioned statistical method to preserve local color distribution differences.

[0056] In some embodiments of this application, in order to predict the white balance parameters, the step of predicting the multispectral compressed features, the depth features, and the statistical prior features to obtain the white balance parameters of the original color image data includes: performing a statistical confidence assessment based on the statistical prior features to obtain statistical confidence weights; fusing the multispectral compressed features, the depth features, and the statistical prior features with the statistical confidence weights to obtain joint representation features; and inputting the joint representation features into a dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data.

[0057] It should be noted that the above credibility assessment can be used to evaluate the reliability of statistical prior features. The higher the credibility weight, the more reliable the statistical prior features. For example, areas with a single color, overexposure, or underexposure will be assigned a lower credibility weight, while areas with rich colors and normal exposure will be assigned a higher credibility weight. In this embodiment, the statistical features of the partition histogram generated by partition statistics can be input into the credibility assessment module to output the corresponding statistical credibility weight. The statistical credibility weight is related to at least one or more of the following factors: (1) color concentration; (2) brightness saturation; (3) texture richness; (4) local color cast; (5) the presence of abnormally bright areas, monochromatic dominant areas, or low-information areas.

[0058] It should be explained that the above-mentioned joint representation features can be a comprehensive feature vector obtained by fusing multispectral compressed features, deep features, and statistical prior features with statistical confidence weights. In the embodiments of this application, the fusion adopts at least one of the following methods: (1) direct splicing followed by fusion through a fully connected layer or convolutional mapping; (2) gated weighted fusion; (3) attention fusion; (4) hierarchical multi-scale fusion.

[0059] It should be noted that during the fusion process, statistical prior features can first be dimensionally aligned through a mapping layer, and then combined with depth features and multispectral features to form joint characterization features for white balance parameter prediction.

[0060] It is understood that the aforementioned dual-head prediction result includes a neural network structure with two parallel prediction branches. These two branches can share input features and output different prediction parameters based on those features. Specifically, the dual-head prediction structure includes a first prediction head and a second prediction head. The step of inputting the joint representation features into the dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data includes: predicting the joint representation features using the first prediction head to obtain a basic reciprocal gain parameter; predicting the joint representation features using the second prediction head to obtain a residual correction parameter; and combining the basic reciprocal gain parameter and the residual correction parameter to obtain the white balance parameters of the original color image data.

[0061] It should be noted that the aforementioned basic reciprocal gain parameter refers to the value obtained by taking the reciprocal of the white balance basic gain, which can include Ainvr for the red channel and Ainvb for the blue channel. The aforementioned residual correction parameter can be a correction amount used to compensate for the deviation between the basic gain and the true white balance parameter, and can include the red channel's... And the Blue Channel .

[0062] Understandably, the basic reciprocal gain parameter can be used to reflect the overall trend of lighting color in the scene (e.g., warm or cool tones), providing a coarse baseline for white balance correction. By predicting the basic reciprocal gain, the model can quickly pinpoint the approximate color temperature range of the current scene, ensuring that most areas do not exhibit severe color casts. The residual correction parameter can reflect local deviations that do not conform to the overall lighting trend (e.g., the special color of a large reflective object in the scene, color differences caused by local shadows, or uneven light sources). The residual correction parameter is responsible for providing fine-grained correction capabilities, enabling the model to handle complex situations such as the failure of the grayscale world assumption, insufficient depth feature extraction, or statistical prior anomalies.

[0063] It should be noted that the basic reciprocal parameters and residual correction parameters can be combined through preset combination relationships to obtain the final white balance parameters.

[0064] In some embodiments of this application, the fundamental gain parameters of different channels can be obtained by inverse transformation based on the fundamental reciprocal parameters. he ,in , The final white balance parameters are then generated based on the combination of residual correction parameters. An exemplary combination generation method is shown below: (1) Correction of additive residuals: ,

[0065] (2) Proportion correction: , ; (3) Gating correction: , ,in , is the gating coefficient.

[0066] It should be noted that the above dual-head prediction results can be distinguished from the existing single-head direct regression gain scheme, and can achieve hierarchical modeling of the basic trend of white balance and the correction amount for complex scenes.

[0067] In some embodiments of this application, during the training phase of the model, the white balance parameter prediction model can also be supervised learning based on labeled samples. The training loss during the training phase can include at least: (1) basic parameter loss, used to constrain the error between the basic reciprocal gain parameter or the basic gain parameter and the target parameter; (2) final parameter loss, used to constrain the numerical error between the final Rgain, Bgain and the target gain; (3) orientation loss, used to constrain the angle error between the predicted white balance vector and the target white balance vector; (4) structural consistency loss, used to constrain the basic parameters, residual correction parameters and final parameters to satisfy the preset combination relationship; (5) residual amplitude constraint loss, used to suppress the abnormally large residual correction parameter; (6) confidence modulation loss, used to adjust the influence of the statistical branch on the white balance prediction result according to the RGB statistical confidence weight.

[0068] In some embodiments of this application, the orientation loss can be constructed based on the following white balance vector: , And construct a loss function based on the cosine similarity or the angle error between the two.

[0069] In some embodiments of this application, when using additive residual correction for white balance, the structural consistency loss can be constrained. and When using proportional correction or gating correction methods, relational constraints are constructed based on the corresponding combination formulas.

[0070] It should be noted that the above training mechanism can simultaneously constrain the numerical accuracy, directional consistency, and structural rationality of white balance parameters, unlike traditional training methods that only focus on the final gain error. This application, as an enhanced AWB control parameter generator, provides more accurate and stable white balance control parameters for existing ISP / AWB links.

[0071] This application embodiment extracts deep features from the original color image data by inputting it into a color coding branch. It then performs feature statistics on the original color image data to obtain statistical prior features. Based on these prior features, it evaluates statistical confidence to obtain statistical confidence weights. Finally, it fuses the multispectral compressed features, deep features, and statistical prior features with statistical confidence weights to obtain joint representation features. These joint representation features are then input into a dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data. Because the original color image data is described from different dimensions using features from two different branches, and the features extracted from parallel branches are fused, the model can utilize the powerful fitting ability of deep learning while retaining the robust prior knowledge of traditional algorithms. When training data coverage is incomplete or abnormal scenes occur, the statistical prior features can play a constraining and supplementary role, reducing the prediction bias of white balance parameters. The fused joint representation features integrate multi-angle information, allowing subsequent white balance parameter prediction to not only rely on spectral analysis but also refer to the structural and statistical regularities of image content, thus enabling more reasonable white balance decisions in various complex environments.

[0072] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for determining white balance parameters in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0073] This application also provides a white balance parameter determination device, please refer to... Figure 4 , Figure 4 This is a schematic diagram of the module structure of the white balance parameter determination device according to an embodiment of this application. The white balance parameter determination device includes: Data acquisition module 10 is used to acquire color raw image data and multispectral image data in the same scene; The weighted compression module 20 is used to obtain the weight scores of each band of the multispectral image data, and to perform adaptive weighted compression on the multispectral image data based on the weight scores to obtain multispectral compression features. Feature extraction module 30 is used to extract features based on the original color image data to obtain depth features and statistical prior features; The prediction module 40 is used to predict the multispectral compressed features, the depth features, and the statistical prior features to obtain the white balance parameters of the original color image data.

[0074] The white balance parameter determination device provided in this application, employing the white balance parameter determination method described in the above embodiments, can solve the technical problem that existing solutions have a large number of multispectral input channels, which is not conducive to real-time deployment. Compared with the prior art, the beneficial effects of the white balance parameter determination device provided in this application are the same as those of the white balance parameter determination method provided in the above embodiments, and other technical features in the white balance parameter determination device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0075] This application provides a white balance parameter determination device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the white balance parameter determination method in the above embodiment 1.

[0076] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a white balance parameter determination device suitable for implementing embodiments of this application. The white balance parameter determination device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The white balance parameter determination device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0077] like Figure 5As shown, the white balance parameter determination device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the white balance parameter determination device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the white balance parameter determining device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows white balance parameter determining devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0078] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0079] The white balance parameter determination device provided in this application, employing the white balance parameter determination method described in the above embodiments, can solve the technical problem that existing solutions have a large number of multispectral input channels, which is not conducive to real-time deployment. Compared with the prior art, the beneficial effects of the white balance parameter determination device provided in this application are the same as those of the white balance parameter determination method provided in the above embodiments, and other technical features of this white balance parameter determination device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0080] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0081] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0082] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the white balance parameter determination method in the above embodiments.

[0083] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0084] The aforementioned computer-readable storage medium may be included in the white balance parameter determination device; or it may exist independently and not assembled into the white balance parameter determination device.

[0085] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the white balance parameter determining device, cause the white balance parameter determining device to: Acquire raw color image data and multispectral image data of the same scene; The weight scores of each band of the multispectral image data are obtained, and the multispectral image data is adaptively weighted and compressed based on the weight scores to obtain multispectral compression features. Feature extraction is performed based on the original color image data to obtain depth features and statistical prior features; The white balance parameters of the original color image data are obtained by predicting the multispectral compression features, the depth features, and the statistical prior features.

[0086] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0088] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0089] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described white balance parameter determination method. This solves the technical problem that existing solutions have a large number of multispectral input channels, which is not conducive to real-time deployment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the white balance parameter determination method provided in the above embodiments, and will not be repeated here.

[0090] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the white balance parameter determination method described above.

[0091] The computer program product provided in this application can solve the technical problem that the existing solutions have a large number of multispectral input channels, which is not conducive to real-time deployment. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the white balance parameter determination method provided in the above embodiments, and will not be repeated here.

[0092] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for determining white balance parameters, characterized in that, The method includes: Acquire raw color image data and multispectral image data of the same scene; The weight scores of each band of the multispectral image data are obtained, and the multispectral image data is adaptively weighted and compressed based on the weight scores to obtain multispectral compression features. Based on the original color image data, feature extraction is performed to obtain depth features and statistical prior features; The white balance parameters of the original color image data are obtained by predicting the multispectral compression features, the depth features, and the statistical prior features. The step of predicting the white balance parameters of the original color image data by using the multispectral compressed features, the depth features, and the statistical prior features includes: Statistical confidence assessment is performed based on the aforementioned statistical prior features to obtain statistical confidence weights; The multispectral compressed features, the depth features, and the statistical prior features with the statistical confidence weights are fused to obtain joint characterization features; The joint characterization features are input into a dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data. The dual-head prediction structure includes a first prediction head and a second prediction head; the step of inputting the joint representation features into the dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data includes: The basic reciprocal gain parameter is obtained by predicting the joint representation features using the first prediction head. The joint characterization features are predicted using the second prediction head to obtain residual correction parameters; The white balance parameters of the original color image data are obtained by combining the basic reciprocal gain parameter and the residual correction parameter.

2. The method for determining white balance parameters as described in claim 1, characterized in that, The step of adaptively weighting and compressing the multispectral image data based on the weight score to obtain multispectral compressed features includes: The weighted multispectral image data is obtained by weighting each band of the multispectral image data based on the weighted score. The weighted multispectral image data is subjected to correlation enhancement and fixed-dimensional channel compression to obtain multispectral compressed features.

3. The method for determining white balance parameters as described in claim 1, characterized in that, The step of obtaining the weighted scores of each band of the multispectral image data includes: Global pooling is performed on each band of the multispectral image data to obtain statistical data for each band. The weighted scores of each band of the multispectral image data are determined based on the statistical data.

4. The method for determining white balance parameters as described in claim 1, characterized in that, The step of extracting features based on the original color image data to obtain depth features and statistical prior features includes: The original color image data is input into the color coding branch for depth feature extraction to obtain the depth features of the original color image data; The statistical prior features of the original color image data are obtained by performing feature statistics on the original color image data.

5. A white balance parameter determination device, characterized in that, The white balance parameter determination device includes: The data acquisition module is used to acquire raw color image data and multispectral image data of the same scene; The weighted compression module is used to obtain the weight scores of each band of the multispectral image data, and to perform adaptive weighted compression on the multispectral image data based on the weight scores to obtain multispectral compression features. The feature extraction module is used to extract features based on the original color image data to obtain depth features and statistical prior features; The prediction module is used to predict the multispectral compressed features, the depth features, and the statistical prior features to obtain the white balance parameters of the original color image data. The step of predicting the white balance parameters of the original color image data by using the multispectral compressed features, the depth features, and the statistical prior features includes: Statistical confidence assessment is performed based on the aforementioned statistical prior features to obtain statistical confidence weights; The multispectral compressed features, the depth features, and the statistical prior features with the statistical confidence weights are fused to obtain joint characterization features; The joint characterization features are input into a dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data. The dual-head prediction structure includes a first prediction head and a second prediction head; the step of inputting the joint representation features into the dual-head prediction structure for prediction to obtain the white balance parameters of the original color image data includes: The basic reciprocal gain parameter is obtained by predicting the joint representation features using the first prediction head. The joint characterization features are predicted using the second prediction head to obtain residual correction parameters; The white balance parameters of the original color image data are obtained by combining the basic reciprocal gain parameter and the residual correction parameter.

6. A white balance parameter determination device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the white balance parameter determination method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the white balance parameter determination method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the white balance parameter determination method as described in any one of claims 1 to 4.

Images