Image white point determination method and electronic equipment

Abstract

The embodiment of the invention provides an image white point determination method and electronic equipment. The method comprises the following steps: sequentially carrying out region segmentation and multispectral feature extraction on each frame of image in an obtained image sequence, and determining a multispectral vector corresponding to each region of each frame of image; identifying the type of each region, and determining the structure label of each region; executing a spectral gradient calculation operation, and determining a spectral gradient value and a reliability label of each region; inputting the multispectral vector, the structure label, the spectral gradient value and the reliability label into an area white point determination model, and outputting a three-dimensional white point of each area; determining the confidence coefficient of each region based on the three-dimensional white points, the spectral gradient values and the structure labels; the global white points of each frame of image are determined based on the three-dimensional white points and the confidence of all the areas in each frame of image, so that the color restoration quality of ISP can be remarkably improved, and the requirement of a complex scene for high-precision color restoration can be better met.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and electronic device for determining white spots in an image. Background Technology

[0002] In recent years, with the development of technology, Automatic White Balance (AWB) has been widely used due to its simple calculation process and low hardware deployment cost.

[0003] However, taking multispectral imaging cameras and mobile device cameras as examples, the traditional AWB method relies on the spectral information of a single frame image to construct a light source model and uses a single global white point to achieve full-image color correction. This processing method has the defects of insufficient single-frame spectral information and ignoring the temporal dimension and structural priors. It is prone to color temperature errors and white point drift in mixed light sources and complex reflection scenes, making it difficult to meet the production and application needs of high-end imaging equipment and professional vision acquisition systems that have high requirements for color reproduction accuracy. Summary of the Invention

[0004] The image white point determination method and electronic device provided in this application are intended to solve at least some of the defects existing in the image white point determination methods of existing electronic devices.

[0005] In a first aspect, embodiments of this application provide a method for determining white spots in an image. This method includes: sequentially performing region segmentation and multispectral feature extraction on each frame of an obtained image sequence to determine a multispectral vector corresponding to each region of each frame; identifying the type of each region and determining a structural label for each region; performing a spectral gradient calculation operation to determine the spectral gradient value of each region and outputting a corresponding reliability label; inputting the multispectral vector, the structural label, the spectral gradient value, and the reliability label into a region white spot determination model to output a three-dimensional white spot for each region; determining the confidence level of each region based on the three-dimensional white spot, the spectral gradient value, and the structural label; and determining the global white spot of each frame based on the three-dimensional white spots and the confidence levels of all regions in each frame.

[0006] In some embodiments, the step of sequentially performing region segmentation and multispectral feature extraction on each frame of the obtained image sequence to determine the multispectral vector corresponding to each region of each frame includes: acquiring the image sequence; performing region segmentation on each frame of the image sequence to generate N regions of each frame; and performing multispectral feature extraction on each of the N regions according to a preset number of multispectral channels to generate the multispectral vector corresponding to each region of each frame.

[0007] In some embodiments, performing the spectral gradient calculation operation to determine the spectral gradient value of each region and outputting the corresponding reliability label includes: obtaining two consecutive frames of multispectral vectors for each region; performing the spectral gradient calculation operation based on the two consecutive frames of multispectral vectors to determine the spectral gradient value of each region; and determining the reliability of the spectral gradient value based on a spectral gradient threshold and outputting the corresponding reliability label.

[0008] In some embodiments, the step of performing spectral gradient calculation based on the two consecutive multispectral vectors to determine the spectral gradient value of each region includes: calculating the difference between the two consecutive multispectral vectors channel by channel and taking the absolute value according to a preset number of multispectral channels to generate a multispectral difference vector for each region in each spectral channel; and summing the squares of the multispectral difference vectors for each region across all multispectral channels to generate the spectral gradient value for each region; wherein the spectral gradient value is used to distinguish between brightness changes and spectral changes.

[0009] In some embodiments, the step of determining the reliability of the spectral gradient value based on the spectral gradient threshold and outputting the corresponding reliability label includes: when the spectral gradient value is greater than the spectral gradient threshold, determining the corresponding region as an unstable region and outputting an unreliable label; when the spectral gradient value is less than or equal to the spectral gradient threshold, determining the corresponding region as a stable region and outputting a reliable label.

[0010] In some embodiments, the step of inputting the multispectral vector, the structural label, the spectral gradient value, and the reliability label into a region white point determination model and outputting a three-dimensional white point for each region includes: constructing an input feature vector by using the multispectral vector, the structural label, the spectral gradient value, and the reliability label as input features; inputting the input feature vector into a region white point determination model and outputting a three-dimensional white point for each region; wherein the length of the input feature vector is the sum of the number of spectral channels, the number of structural labels, and a preset constant; the region white point determination model is a multilayer perceptron; and the three dimensions of the three-dimensional white point correspond to the calibration parameters of the red, green, and blue color channels, respectively.

[0011] In some embodiments, determining the confidence level of each region based on the 3D white points, the spectral gradient values, and the structural labels includes: performing a weighted mapping calculation operation based on a first exponential decay function and the spectral gradient values ​​to determine a first weight value for each region; querying a preset mapping table for a second weight value that matches the structural label of each region; performing a weighted mapping calculation operation based on a second exponential decay function and the spectral gradient values ​​to determine a third weight value for each region; performing a deviation calculation operation based on the 3D white points of all regions in each frame of the image and the 3D white points of each region to determine a fourth weight value for each region; and performing a product operation based on the first weight value, the second weight value, the third weight value, and the fourth weight value to determine the confidence level of each region; wherein the first weight value characterizes the spectral stability of the corresponding region; the second weight value characterizes the structural reliability of the corresponding region; the third weight value characterizes the temporal consistency of the corresponding region; and the fourth weight value characterizes the global consistency of the corresponding region.

[0012] In some embodiments, the method further includes: when the spectral gradient value is greater than the spectral gradient threshold, acquiring two consecutive frames of spectral gradient values ​​for the corresponding region; performing a weighted fusion operation based on the two consecutive frames of spectral gradient values ​​and a preset weighting coefficient to determine a target spectral gradient value for each region; updating the spectral gradient value of the corresponding region with the target spectral gradient value; wherein the smoothness of the target spectral gradient value is higher than that of the spectral gradient value before the weighted fusion operation.

[0013] Secondly, embodiments of this application provide an electronic device. The electronic device includes a memory and a processor, the processor being communicatively connected to the memory. The memory stores computer program instructions, which, when invoked by the processor, cause the processor to execute the image white point determination method described above.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium storing processor-executable computer program instructions, which, when executed by the processor, cause the computer to perform the image white point determination method as described above.

[0015] The beneficial effects of the image white point determination provided in this application embodiment are as follows: By sequentially performing the entire process of region segmentation and multispectral feature extraction, region type recognition, spectral gradient calculation and reliability label output, three-dimensional white point determination, confidence determination, and global white point determination on each frame of the image sequence, the technical defects of traditional AWB, such as insufficient single-frame spectral information, white point drift in mixed light fields, neglect of temporal and structural priors, and multispectral reflection interference, can be fundamentally solved. It effectively reduces reflection noise and flickering light source interference, and avoids the influence of local light sources or reflections on global white balance. At the same time, by weakening the interference of bad regions through confidence weighting, it has extremely high robustness and requires no additional hardware. It can be paired with any multispectral sensor, thereby meeting the requirements for color restoration accuracy in complex scenes such as indoor / outdoor mixed light sources, screen lighting, local light fields, high dynamic range (HDR), and multi-light source photography, and significantly improving the color restoration quality of the image signal processor (ISP). Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0017] Figure 1 A schematic diagram of the image white point determination method provided in the embodiments of this application; Figure 2 A flowchart illustrating the method for determining multispectral vectors provided in this application embodiment; Figure 3 A flowchart illustrating the method for determining structural labels provided in this application embodiment; Figure 4 A flowchart illustrating the method for determining spectral gradient values ​​and reliability labels provided in this application embodiment; Figure 5 A flowchart of a method for determining spectral gradient values ​​provided in an embodiment of this application; Figure 6 A flowchart illustrating the method for determining reliability tags provided in this application embodiment; Figure 7 A flowchart illustrating the method for determining three-dimensional white points provided in this application embodiment; Figure 8 A flowchart illustrating the method for determining confidence level provided in embodiments of this application; Figure 9 A flowchart illustrating the method for determining a target spectral gradient value provided in this application embodiment; Figure 10This is a schematic diagram of the architecture of the image white point determination device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the architecture of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To facilitate understanding of this application, a more detailed description of this application will be provided below in conjunction with the accompanying drawings and specific embodiments.

[0019] It should be noted that when a component is said to be "set on" another component, it can be directly on the other component or there may be an intervening component. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be an intervening component, or it can refer to the two components being interconnected via signals. When a component is considered to be "coupled" to another component, it can be directly coupled to the other component or there may be an intervening component, or it can refer to the two components interacting via signals.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0021] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0022] Based on the core requirement of AWB in imaging systems to correct color deviations under different light source conditions, the industry currently generally adopts traditional AWB technology to complete the image color correction process in order to achieve the goal of image color reproduction that conforms to human visual perception.

[0023] The specific implementation process of the traditional AWB technology solution is as follows: First, the image sensor of the imaging device acquires single-frame image data and extracts the spectral information of the single-frame image; then, a unified global light source model is constructed based on the spectral information of the single-frame image, and a single global white point is obtained through algorithm; finally, based on the global white point, a unified color correction operation is performed on the entire image to complete the white balance processing and output the corrected image.

[0024] However, in the process of developing this invention, the applicant noticed that traditional AWB technology suffers from four major technical bottlenecks, which severely restrict the improvement of color reproduction quality in imaging systems. Firstly, insufficient single-frame spectral information leads to unstable light source inference: even with a multispectral sensor, relying solely on the spectral information of a single frame image is easily affected by the reflective properties of various materials in the scene. A significant amount of time is spent analyzing and processing the interfered spectral data, rather than on correction calculations based on real light source information, resulting in poor stability of the light source inference results and an inability to accurately match real light source conditions. Secondly, the globally unified light source model has inherent defects: traditional solutions employ a design approach of using a unified light source model for the entire image. A single global white point cannot accurately represent the complex light field distribution commonly found in real-world scenes, such as mixed light sources and local light source differences (e.g., the coexistence of screen light sources, LED strip light sources, and window light), which easily leads to problems such as incorrect color temperature judgment and color reproduction distortion. Third, neglecting effective temporal information leads to insufficient judgment basis: Mobile devices typically capture multiple "viewfinder frames" before taking a photo. The spectral change trends in these consecutive frames can effectively distinguish reflective objects from stable light sources (e.g., the spectrum of reflective objects changes abruptly with device movement, while the spectrum of stable light sources remains relatively constant). However, traditional AWB methods do not utilize this effective temporal information, missing a key basis for improving the accuracy of light source judgment, resulting in insufficient robustness in complex scenes. Fourth, the lack of structural prior fusion limits accuracy: Different material regions in a scene (such as sky, grass, walls, skin, etc.) possess stable spectral characteristics, and different materials exhibit typical multispectral features. This structural prior information plays an important auxiliary role in white point determination, but traditional solutions do not incorporate it into the algorithm design, resulting in limited accuracy and applicability of white point determination.

[0025] Furthermore, with the diversification of application scenarios for imaging equipment (such as indoor / outdoor mixed light sources, HDR shooting, multi-light source photography, etc.) and the continuous improvement of users' requirements for color reproduction accuracy, the performance defects of traditional AWB technology solutions will be significantly amplified as the complexity of the scene increases. Its core logic of "single frame processing + global unified model" cannot match the core requirements for color reproduction accuracy and algorithm robustness in complex scenes, becoming a key obstacle restricting the development of application scenarios such as high-end imaging equipment and professional visual acquisition systems.

[0026] To address the technical bottlenecks of traditional AWB technology, such as insufficient single-frame spectral information, defects in the global unified model, neglect of temporal dimension information, and lack of structural prior fusion, which prevent it from meeting the high-precision color restoration requirements in complex scenarios, the applicant has developed an image white point determination algorithm. Its core design logic is as follows: for each frame in the image sequence, the entire process of performing region segmentation and multispectral feature extraction, region type recognition, spectral gradient calculation and reliability label output, three-dimensional white point determination, confidence determination, and global white point determination is performed sequentially. The specific implementation process is as follows: First, each frame of the obtained image sequence is sequentially segmented into regions and multispectral features are extracted to determine the multispectral vector corresponding to each region of each frame. Second, the type of each region is identified, and the structural label of each region is determined. Then, based on the multispectral vectors of two consecutive frames of each region, a spectral gradient calculation operation is performed to determine the spectral gradient value of each region and output the corresponding reliability label. Next, the multispectral vector, structural label, spectral gradient value, and reliability label are input into the region white point determination model to output the three-dimensional white point of each region. Then, based on the three-dimensional white point, spectral gradient value, and structural label, a confidence calculation operation is performed to determine the confidence of each region. Finally, based on the three-dimensional white point and confidence of all regions in each frame, a weighted least squares calculation operation is performed to determine the global white point of each frame.

[0027] On the one hand, by segmenting each frame of the image for regional processing and combining spectral gradient calculation to analyze the temporal spectral change trend, reflection noise interference can be effectively suppressed, and reflective and flickering light sources can be accurately avoided. This fundamentally avoids the problem of unstable light source inference caused by insufficient single-frame spectral information in traditional solutions. At the same time, the interference from local light sources or material reflections is limited to the corresponding area, avoiding affecting the accuracy of global white point and solving the defect that the global unified model cannot adapt to complex mixed light fields. On the other hand, by associating region types with structural labels, the typical multispectral features of high-quality regions are fully utilized to improve the accuracy of white point determination. Confidence determination is used to evaluate the credibility of regional white points, and interference regions are given low weights, effectively weakening them in the global white point determination process and significantly improving the robustness of the algorithm. In addition, this algorithm can be used with any multispectral sensor hardware without the need for additional dedicated hardware modules, effectively reducing hardware costs and adaptation difficulties.

[0028] Based on the above-described inventive concept, the image white point determination method provided in this application embodiment can be generally applied to various imaging electronic devices. This fundamentally solves the core problems of traditional AWB technology, such as the instability of multispectral light in complex scenes, white point drift under mixed light fields, inaccuracy caused by the lack of constraints in single-frame AWB, and color temperature errors caused by multispectral reflection interference. This significantly improves the color restoration quality of the ISP, better matching the high-precision color reproduction requirements of complex scenes such as indoor / outdoor mixed light sources, screen lighting, local light fields, HDR, and multi-light source photography. For ease of understanding, the following combines... Figure 1 The specific implementation of the image white point determination method provided in the embodiments of this application is illustrated by example.

[0029] Figure 1 This is a schematic diagram of an image white spot determination method provided in an embodiment of this application. Figure 1 As shown, the method for determining white points in this image includes: S101. Perform region segmentation and multispectral feature extraction on each frame of the obtained image sequence in sequence, and determine the multispectral vector corresponding to each region of each frame image. Among them, "image sequence" refers to an ordered set of multiple frames of images that are temporally correlated and are continuously acquired by a multispectral imaging sensor at a preset frame rate.

[0030] "Region segmentation" refers to the operation of dividing each frame of an image into N independent regions that do not overlap, according to a fixed size rule. For example, a 5×5 grid can be used to generate 25 independent regions.

[0031] "Multispectral feature extraction" refers to performing multispectral feature extraction operations on each of the N regions of each frame of an image, according to a preset number of K multispectral channels, for each segmented region.

[0032] "Multispectral vector" refers to a K-dimensional numerical vector corresponding to the preset K multispectral channels, used to characterize the multispectral features of each region in each frame of the image.

[0033] In some embodiments, firstly, a sequence of images with temporal correlation is continuously acquired by a multispectral imaging sensor at a preset frame rate; secondly, each frame of the image sequence is segmented into N independent regions that do not overlap; finally, multispectral features are extracted sequentially from the N regions of each frame to determine the multispectral vector corresponding to each region of each frame.

[0034] S102. Identify the type of each region and determine the structural label of each region; The “type of each region” refers to the category of scene objects classified by the semantic features, edge features, or color features of each region in each frame of the image. Specifically, these include: sky, grass / vegetation, skin, wall, metal, screen / projection, high-saturation color objects, etc. Different types of regions have different reliability in AWB white point calculation.

[0035] "Structural labels" refer to the identification information generated after identifying the type of each region in each frame of an image through a lightweight semantic model, edge model, or color rules. They are used to characterize the type of each region in each frame of an image.

[0036] In some embodiments, firstly, the type of each region is identified by a lightweight semantic model, edge model, or color rule, and the scene object category to which each region belongs is determined, i.e., the type of each region is determined; then, based on the type identification result of each region, a corresponding structural label is assigned to it.

[0037] S103. Perform spectral gradient calculation to determine the spectral gradient value of each region and output the corresponding reliability label; Among them, the "spectral gradient calculation operation" is the core operation step used to evaluate the temporal stability of the spectrum in the same region. Specifically, it refers to a series of processing steps that sequentially perform channel-by-channel difference operation and channel difference square summation operation on two consecutive frames of multispectral vectors in the same region. The function of this operation is to quantify and distinguish the brightness change and spectral change in the region. The output spectral gradient value can be used as the quantitative basis for subsequent region reliability determination.

[0038] "Spectral gradient value" refers to the quantitative index output after performing spectral gradient calculation on two consecutive frames of multispectral vectors of the same region. Its value is positively correlated with the temporal fluctuation of the spectral characteristics of the region. The more significant the fluctuation, the higher the spectral gradient value. It is the core quantitative basis for determining the reliability of a region.

[0039] The "reliability label" is a binary identifier used to mark the white point of each region to determine its reliability. Its generation logic is as follows: compare the spectral gradient value of each region with the preset spectral gradient threshold, and assign the corresponding region a "reliable" or "unreliable" label attribute based on the comparison result.

[0040] In some embodiments, firstly, the spectral gradient of two consecutive frames of multispectral vectors for each region is calculated to obtain the spectral gradient value of the corresponding region; then, the spectral gradient value of each region is compared with a preset spectral gradient threshold to generate a reliability label for the corresponding region.

[0041] S104. Input the multispectral vector, the structural label, the spectral gradient value, and the reliability label into the region white point determination model, and output the three-dimensional white point of each region. Among them, the "regional white point determination model" refers to the white point determination algorithm model built for each region white point solution task. The model takes multispectral vectors, structural labels, spectral gradient values, and reliability labels as input features, and achieves accurate determination of regional white points by learning the mapping relationship between the spectral characteristics of different types of regions and white points.

[0042] It should be noted that the region white point determination model adopts a 2-3 layer multilayer perceptron (MLP) structure. The specific model parameters are set as follows: Input layer: The number of nodes is consistent with the dimension of the input features, which is used to receive the input features composed of multispectral vectors, structural labels, spectral gradient values ​​and reliability labels; Hidden layer: 1-2 hidden layers are set, and the number of nodes in each layer is configured to be 16-64. The input features are deeply fused and feature extracted through nonlinear activation functions; Output layer: 3 nodes are set, which correspond to the 3 color channel components of the 3D white point, and output the white point determination result of each region.

[0043] Example: A 2-3 layer MLP: Input:X i Hidden layer: 16-64 nodes Output: 3-D white dots Among them, "three-dimensional white point" refers to the three-dimensional color space value used to characterize the white balance reference of each region. It usually corresponds to the red, green and blue color space (RGB) value. It is the output result of the region white point determination model and can be used to correct the color deviation of the region to achieve automatic white balance.

[0044] In this embodiment, the multispectral vector, structural label, spectral gradient value, and reliability label of each region are input into a multilayer perceptron. The multilayer perceptron performs the following operations in sequence: the input layer receives the multispectral vector, structural label, spectral gradient value, and reliability label, and passes them to the hidden layer; the hidden layer performs weighted summation and nonlinear transformation on the multispectral vector, structural label, spectral gradient value, and reliability label, fusing the spectral information of the multispectral vector, the category prior of the structural label, the stability information of the spectral gradient value, and the credibility information of the reliability label; the output layer performs linear mapping on the fused features of the hidden layer and outputs the three-dimensional white point of the region.

[0045] S105. Based on the three-dimensional white points, the spectral gradient values, and the structural labels, determine the confidence level of each region; Among them, the confidence level of each region can be determined by the confidence level calculation operation. The "confidence level calculation operation" is the core operation used to quantify and evaluate the credibility of the white point determination results of each region. Specifically, it refers to a series of processing steps to obtain the reliability of the region's white points by comprehensively considering three types of indicators: the rationality of the three-dimensional white points, the regional stability represented by the spectral gradient value, and the prior credibility carried by the structural label.

[0046] "Confidence level" refers to the normalized quantitative index obtained through confidence level calculation, which is used to evaluate the reliability of the white point determination results for each region. Its value range is [0, 1]. The closer the value is to 1, the more reliable the white point determination results for the corresponding region are, and vice versa.

[0047] In this embodiment, a confidence level is calculated for each region based on the three-dimensional white point, spectral gradient value, and structural label. The closer the confidence level of a region is to 1, the more reliable the white point determination result of that region is, and the more suitable it is as the weight basis for subsequent global white point fusion calculation.

[0048] S106. Based on the three-dimensional white points and the confidence level of all regions in each frame of the image, determine the global white points of each frame of the image.

[0049] The global white point of each frame can be determined by weighted least squares calculation. The weighted least squares calculation means that the confidence level of each region is used as a weight coefficient to perform the best fit calculation on the three-dimensional white points output by all regions of each frame. Its core logic is to give higher weight to high confidence regions and weaken the interference of low confidence regions, thereby suppressing the bias effect of unreliable white point data on the global result and finally outputting accurate global white point parameters.

[0050] "Global white point" refers to the three-dimensional numerical vector obtained by weighted fitting of the three-dimensional white points of all regions in each frame of the image through the weighted least squares method. It is used to characterize the white balance benchmark of the entire frame of the image. This parameter is the unified basis for color deviation correction of the whole image and can be directly applied to the ISP color restoration process to achieve automatic white balance effect of the whole image.

[0051] It is understandable that the global white point of each of the above frames can be represented by the following formula:

[0052] in, This represents the global white point of each frame of the image, which includes calibration parameters for the red, green, and blue color channels; Represents each frame of the image One region; This represents the confidence level of the i-th region; This represents the three-dimensional white point of the i-th region.

[0053] In this embodiment, the global white point of each frame is calculated using weighted least squares based on the three-dimensional white point and confidence level of all regions in each frame. This global white point can be used for subsequent white balance gain calculation, ISP color restoration, and RAW / RGB domain color correction, effectively avoiding the problem of overall white point drift caused by local color cast.

[0054] The image white point determination method provided in this application adopts a hierarchical processing approach of "multi-dimensional feature extraction - three-dimensional white point determination - global white point determination". In the early stage, image region segmentation, multispectral feature extraction and determination of structural labels, spectral gradient values ​​and reliability labels are completed (such as steps S101, S102 and S103), which builds a comprehensive and reliable feature foundation for three-dimensional white point determination. Then, the accurate output of regional three-dimensional white points is achieved through the regional white point determination model (step S104). In the later stage, only the regional three-dimensional white points and confidence scores are used to determine the global white points using the weighted least squares method (such as steps S105 and S106). This can make full use of multi-dimensional features to improve the accuracy of global white point determination and effectively avoid the problem of large errors caused by single features or direct global determination, thereby ensuring the reliability and accuracy of the image white point determination results and meeting the application needs of high-precision image color correction and other scenarios.

[0055] Figure 2 The specific process for determining multispectral vectors provided in embodiments of this application. In some embodiments, such as Figure 2 As shown, S101 specifically includes: S1011. Obtain the image sequence; S1012. Perform region segmentation on each frame of the image sequence to generate N regions for each frame of the image; In some embodiments, the N regions obtained by segmenting each frame of image are numbered in order from left to right and from top to bottom, with the numbering value being i=1, 2, ..., N; the number of each frame of image is associated with the corresponding region and stored to form a "frame number-region number" mapping table.

[0056] S1013. According to the preset number of multispectral channels, multispectral features are extracted from each of the N regions to generate a multispectral vector corresponding to each region of each frame image.

[0057] It is understandable that the multispectral vector of each of the above regions can be represented by the following formula:

[0058] in, This represents the multispectral vector of the i-th region; i is the region number; K is the number of multispectral channels; This represents the signal intensity value of the i-th region in the first spectral channel, and so on. This represents the signal intensity value of the i-th region in the k-th spectral channel.

[0059] In some embodiments, for the i-th region, the signal intensity values ​​of the region under K spectral channels are extracted respectively to obtain the signal intensity values ​​of the region under K spectral channels; the signal intensity values ​​corresponding to the K spectral channels are arranged in order according to the number of spectral channels to obtain the multispectral vector of the i-th region.

[0060] In this embodiment, firstly, a sequence of temporally correlated images is continuously acquired using a multispectral imaging sensor at a preset frame rate. Secondly, for each frame in the image sequence, a region segmentation operation is performed using a uniform grid partitioning rule to generate N independent, non-overlapping regions. Finally, according to a preset number of K multispectral channels, multispectral feature extraction is performed on each of the N regions of each frame to generate a multispectral vector for the corresponding region. The region segmentation described above can effectively avoid interference from local reflections or local light sources on the global analysis.

[0061] Figure 3 The specific process for determining structural labels provided in embodiments of this application. In some embodiments, such as Figure 3 As shown, S102 specifically includes: S1021. Identify the type of each region; In some embodiments, a lightweight semantic model, an edge model, or color rules are used to identify the type of each region. The lightweight semantic model-based identification process involves inputting each region into a pre-trained lightweight semantic model. The model learns the spectral and texture features of different region types and outputs the corresponding region type. The edge model-based identification process involves extracting the edge contour features of each region and performing similarity matching with preset edge templates of various types (e.g., straight edges of walls, irregular edges of grass, and rectangular edges of screens). The type with the highest matching degree is selected and determined as the region type. The color rule-based identification process involves comparing the color features corresponding to the multispectral vector of each region with preset color threshold ranges of various types (e.g., skin regions correspond to specific skin color spectral ranges, and sky regions correspond to blue-violet spectral ranges) to determine the type of the corresponding region.

[0062] S1022. Determine the target region type that matches each region from the preset region type set; The “preset set of region types” refers to predefined region types that cover all possible scenarios (such as sky, wall, skin, metal, etc.). During the structure label generation stage, the region type can be determined by matching the features of each region with the type features in this set.

[0063] "Target region type" refers to the output result of the region type determination process in the image. Specifically, it refers to comparing and matching the identified region type with the standard types in the preset region type set one by one, and finally determining the region type that completely corresponds to the region.

[0064] In some embodiments, taking a frame of an image from an indoor mixed light source scene as an example, multispectral features are extracted from each region obtained after region segmentation of the image. For one region, multispectral feature information with a color temperature of 6500K, high brightness, and spectral energy distribution concentrated in the blue light band is extracted. Based on this multispectral feature information, the type of the region is initially identified as a blue high-brightness emitting region. The preset region type set includes typical types such as screen, light strip, sky, and wall, and each type has corresponding feature information. For example, the feature information corresponding to the screen is a color temperature of 5000K-7000K, high brightness, and a high proportion of blue light. The multispectral feature information of the above region is compared and matched with the feature information corresponding to the screen in the region type set. When the matching degree of the two reaches a preset threshold (such as 90%), the target region type that matches the region can be determined to be the screen.

[0065] S1023. Generate a structural label containing the target region type.

[0066] It is understandable that the above structural tags can be represented by the following formula: Ti∈{Sky,Skin,Wall,Grass,Metal,Display,...} Where Ti represents the structure label of the i-th region; {Sky,Skin,Wall,Grass,Metal,Display,...} represents the preset set of region types.

[0067] In this embodiment, firstly, at least one method selected from lightweight semantic models, edge models, or color rules is used to perform type recognition operations on each region of each frame of the image. Secondly, the type of each identified region is matched with a preset set of region types, and the type that completely matches the region is selected as the target region type. Finally, a structural label containing the target region type is generated. This structural label, as a carrier of prior structural knowledge, can be used to determine whether each region is suitable for white point calculation, thereby effectively reducing the interference of untrusted regions on the AWB algorithm.

[0068] Figure 4 The specific process for determining spectral gradient values ​​and reliability labels provided in embodiments of this application. In some embodiments, such as Figure 4 As shown, S103 specifically includes: S1031. Obtain two consecutive frames of multispectral vectors for each region; Among them, "two consecutive frame multispectral vectors" refers to two sets of multispectral vectors formed by the multispectral features of two temporally adjacent frames of the same region, which are the input data for calculating the spectral gradient value.

[0069] It is understandable that the multispectral vectors of two consecutive frames for each of the above regions can be represented by the following formula:

[0070]

[0071] in, This represents the multispectral vector of the i-th region in the t-th frame; This represents the multispectral vector of the i-th region in the (t-1)-th frame.

[0072] In some embodiments, taking the i-th region of an image as an example, the multispectral vector of the i-th region in frame t and frame t-1 is obtained.

[0073] S1032. Based on the two consecutive frames of multispectral vectors, perform a spectral gradient calculation operation to determine the spectral gradient value of each region. In some embodiments, taking the i-th region of an image as an example, the spectral gradient of the multispectral vector of the i-th region in frame t and frame t-1 is calculated to determine the spectral gradient value of the image in the i-th region.

[0074] S1033. Determine the reliability of the spectral gradient value based on the spectral gradient threshold and output the corresponding reliability label.

[0075] The "spectral gradient threshold" can be a pre-set value used to determine the temporal stability of the spectral characteristics of each region in the image. The value of this spectral gradient threshold can range from 0.05. The threshold value is 0.10, but the specific value needs to be determined based on the inherent noise characteristics of the multispectral imaging sensor. If the noise of the multispectral imaging sensor is higher, a higher spectral gradient threshold should be selected to avoid misjudging small spectral fluctuations caused by noise as unstable regions; if the noise of the multispectral imaging sensor is lower, a lower spectral gradient threshold should be selected to improve the accuracy of stable region determination.

[0076] In this embodiment, firstly, spectral gradient calculation is performed on two consecutive frames of multispectral vectors for each region to determine the spectral gradient value of the corresponding region; then, the spectral gradient value of each region is compared with a preset spectral gradient threshold to generate a corresponding reliability label, thereby determining the spectral stability of the region.

[0077] Figure 5 The specific process for determining spectral gradient values ​​provided in embodiments of this application. In some embodiments, such as Figure 5 As shown, S1032 specifically includes: S10321. According to the preset number of multispectral channels, the multispectral vectors of the two consecutive frames are subtracted channel by channel and the absolute value is taken to generate the multispectral difference vector of each region in each spectral channel. It is understandable that the above multispectral difference vector can be represented by the following formula:

[0078] in, Let represent the multispectral difference vector of the i-th region.

[0079] In some embodiments, taking the i-th region of an image as an example, the difference between the multispectral vectors of the i-th region in frame t and frame t-1 in the first spectral channel is calculated and the absolute value is taken to generate the multispectral difference vector of the i-th region of the corresponding image in the first spectral channel.

[0080] S10322. Sum the squares of the multispectral difference vectors of all multispectral channels for each region to generate the spectral gradient value for each region.

[0081] It is understandable that the above spectral gradient values ​​can be expressed using the following formula:

[0082] in, This represents the spectral gradient value of the i-th region; This represents the multispectral vector of the i-th region in the K-th spectral channel and the t-th frame; This represents the multispectral vector of the i-th region in the K-th spectral channel and the (t-1)-th frame.

[0083] In this embodiment, firstly, according to a preset number of multispectral channels, the multispectral vectors of two consecutive frames are subtracted channel by channel and their absolute values ​​are taken to generate a multispectral difference vector for each region in each spectral channel. Secondly, the values ​​of each channel of the multispectral difference vector are squared and summed to obtain the spectral gradient value of that region. The above spectral gradient value can distinguish between brightness changes and spectral changes in the corresponding region, providing a quantitative basis for subsequent region reliability determination.

[0084] Figure 6 The specific process for determining reliability labels provided in embodiments of this application. In some embodiments, such as Figure 6 As shown, S1033 specifically includes: S10331. When the spectral gradient value is greater than the spectral gradient threshold, the corresponding region is determined to be an unstable region, and an unreliable label is output. "Unstable regions" refer to areas where the spectral gradient value is greater than the spectral gradient threshold, specifically including unstable light source regions, reflection regions, and screen flickering regions. The spectral characteristics of these regions fluctuate significantly between adjacent frames, resulting in low reliability of white point determination results calculated based on them.

[0085] "Unreliable label" refers to the reliability indicator output from the unstable region (e.g., ...). =1), used to mark the area as unsuitable as a reference for subsequent white point calculations.

[0086] S10332. When the spectral gradient value is less than or equal to the spectral gradient threshold, the corresponding region is determined to be a stable region, and a reliable label is output.

[0087] Among them, the "stable region" refers to the region where the spectral gradient value is less than or equal to the spectral gradient threshold. The spectral characteristics of this type of region fluctuate less between adjacent frames, and the white point determination results calculated based on it have high reliability.

[0088] "Reliability label" refers to the reliability identifier output from the stable region (e.g., ...). =0), used to mark the area as suitable as a reference for subsequent white point calculations.

[0089] In this embodiment, taking the i-th region of an image as an example, the spectral gradient value of the i-th region is... With the preset spectral gradient threshold Compare; if If the i-th region is determined to be an unstable region, then an unreliable label is output. =1; if If the i-th region is determined to be a stable region, a reliable label is output. =0. By comparing the differences in multispectral features of the same region in two consecutive frames, the system can distinguish between simple brightness changes and true spectral (color temperature) changes, further improving the stability of white point determination.

[0090] Figure 7 The specific process for determining three-dimensional white points provided in embodiments of this application. In some embodiments, such as Figure 7 As shown, S104 specifically includes: S1041. Construct an input feature vector by using the multispectral vector, the structural label, the spectral gradient value, and the reliability label as input features; It is understandable that the above input feature vector can be represented by the following formula:

[0091] in, This represents the input feature vector of the i-th region.

[0092] In some embodiments, taking the i-th region of an image as an example, the multispectral vector, structural label, spectral gradient value, and reliability label corresponding to the i-th region are used as input features to construct the input feature vector of the i-th region.

[0093] S1042. Input the input feature vector into the white point determination model of the region, and output the three-dimensional white point of each region.

[0094] The length of the input feature vector is the sum of the number of spectral channels, the number of structural labels, and a preset constant; the region white point determination model is a multilayer perceptron; and the three dimensions of the three-dimensional white point correspond to the calibration parameters of the red, green, and blue color channels, respectively.

[0095] It is understandable that the above three-dimensional white point can be represented by the following formula:

[0096] in, Represents the three-dimensional white point of the i-th region; The model for determining white points in the region; This represents the calibration parameters of the 3D white point in the i-th region in the red channel; This represents the calibration parameters of the 3D white point in the i-th region in the green channel; This represents the calibration parameters of the 3D white point in the i-th region in the blue channel.

[0097] In some embodiments, taking a spectral channel count of 9, a structural label count of 6, and a preset constant of 2 as an example, the length of the input feature vector is 9+6+2=17.

[0098] In this embodiment, taking the i-th region of an image as an example, the input feature vector of the i-th region is input into the region white point determination model, and the calibration parameters of the three-dimensional white point of the i-th region in the red, green and blue channels are output.

[0099] Figure 8 The specific process for determining confidence levels is provided for embodiments of this application. In some embodiments, such as Figure 8 As shown, S105 specifically includes: S1051. Based on the first exponential decay function and the spectral gradient value, perform a weighted mapping calculation operation to determine the first weight value of each region; Among them, the "first weight value" refers to the quantization parameter calculated based on the weighted mapping of the first exponential decay function and the spectral gradient value, which is used to characterize the spectral stability of the corresponding region.

[0100] It is understandable that the aforementioned first weight value can be expressed using the following formula:

[0101] in, This represents the first weight value of the i-th region; This represents the first exponential decay function.

[0102] In some embodiments, taking the i-th region of an image as an example, the spectral gradient value of the i-th region is weighted and mapped with the first exponential decay function to obtain the spectral stability value of the i-th region, which is the first weight value.

[0103] S1052. From the preset mapping table, query the second weight value that matches the structural label of each region; The “second weight value” refers to a parameter obtained from a preset mapping table. Its value matches the structural label of the corresponding region and is used to characterize the structural reliability of the corresponding region.

[0104] It is understandable that the aforementioned second weight value can be expressed using the following formula:

[0105] in, This represents the second weight value of the i-th region; This represents a preset mapping table.

[0106] In some embodiments, taking the i-th region of an image as an example, the structural reliability value that matches the structural label of the i-th region is queried from a preset mapping table, thus obtaining the second weight value.

[0107] S1053. Based on the second exponential decay function and the spectral gradient value, perform a weighted mapping calculation operation to determine the third weight value of each region; Among them, the "third weight value" refers to the quantization parameter calculated based on the weighted mapping of the second exponential decay function and the spectral gradient value, which is used to characterize the temporal consistency of the corresponding region.

[0108] It is understandable that the aforementioned third weight value can be expressed using the following formula:

[0109] in, Indicates the third weight value; This represents the second exponential decay function.

[0110] In some embodiments, taking the i-th region of an image as an example, the spectral gradient value of the i-th region is weighted and mapped with the second exponential decay function to obtain the temporal consistency value of the i-th region, which is the third weight value.

[0111] S1054. Based on the three-dimensional white points of all regions and the three-dimensional white points of each region in each frame of the image, perform a deviation calculation operation to determine the fourth weight value of each region; Among them, the "fourth weight value" refers to the parameter calculated based on the deviation between the three-dimensional white points of all regions and the three-dimensional white points of a single region in each frame of the image, which is used to characterize the global consistency of the corresponding region.

[0112] In some embodiments, taking N regions of an image as an example, the deviation between the three-dimensional white points of the N regions of the image and the three-dimensional white points of each region is calculated to obtain the global consistency value of the image, which is the fourth weight value.

[0113] S1055. Based on the first weight value, the second weight value, the third weight value, and the fourth weight value, perform a product operation to determine the confidence level of each region.

[0114] It is understandable that the confidence level of each of the above regions can be expressed by the following formula:

[0115] in, This represents the confidence level of the i-th region.

[0116] In this embodiment, taking the i-th region of an image as an example, the first weight value, the second weight value, the third weight value and the fourth weight value of the i-th region are multiplied to obtain the confidence level of the i-th region.

[0117] Figure 9 The specific process for determining the target spectral gradient value is provided for embodiments of this application. In some embodiments, such as Figure 9 As shown, the method for determining white points in an image also includes: S201. When the spectral gradient value is greater than the spectral gradient threshold, obtain two consecutive frames of spectral gradient values ​​for the corresponding region. In some embodiments, taking the i-th region of an image as an example, when the spectral gradient value of the i-th region is greater than the spectral gradient threshold, the spectral gradient values ​​of the i-th region in frame t and frame t-1 are obtained.

[0118] S202. Based on the spectral gradient values ​​of the two consecutive frames and the preset weighting coefficients, perform a weighted fusion operation to determine the target spectral gradient value for each region. It is understandable that the above target spectral gradient value can be expressed by the following formula:

[0119] in, This represents the target spectral gradient value for the i-th region; This represents the spectral gradient value of the i-th region in the t-th frame; This represents the spectral gradient value of the i-th region in the (t-1)-th frame; This indicates the preset weighting coefficients.

[0120] S203. Update the spectral gradient value of the corresponding region with the target spectral gradient value.

[0121] The smoothness of the target spectral gradient value is higher than that of the spectral gradient value before the weighted fusion operation.

[0122] In this embodiment, taking the i-th region of an image as an example, when the spectral gradient value of the i-th region is greater than the spectral gradient threshold, the spectral gradient values ​​of the i-th region in two consecutive frames, t and t-1, are first obtained. Then, these two spectral gradient values ​​are weighted and fused with a preset weighting coefficient to obtain the target spectral gradient value of the i-th region. Finally, the spectral gradient value of the i-th region is updated with this target spectral gradient value. That is, the target spectral gradient value is updated to the spectral gradient value of the i-th region. The multispectral vector, structural label, updated spectral gradient value, and reliability label are input into the region white point determination model, and the three-dimensional white point of each region is output. Based on the three-dimensional white point, the updated spectral gradient value, and structural label, the confidence level of each region is determined. Since the smoothness of the target spectral gradient value is higher than that of the spectral gradient value before the weighted fusion operation, the influence of random noise on the spectral gradient value can be effectively reduced, thereby avoiding the overall white point drift caused by local color cast.

[0123] Figure 10 This is a schematic diagram of the architecture of the image white point determination device provided in an embodiment of this application. Figure 10 As shown, the image white point determination device 300 includes: a first processing module 310, a second processing module 320, a third processing module 330, a fourth processing module 340, a fifth processing module 350, and a sixth processing module 360, wherein: The first processing module 310 is configured to: sequentially perform region segmentation and multispectral feature extraction on each frame of the obtained image sequence, and determine the multispectral vector corresponding to each region of each frame of the image; The second processing module 320 is configured to: identify the type of each region and determine the structural label of each region; The third processing module 330 is configured to: perform a spectral gradient calculation operation, determine the spectral gradient value of each region, and output the corresponding reliability label; The fourth processing module 340 is configured to: input the multispectral vector, the structural label, the spectral gradient value, and the reliability label into the white point determination model of the input region, and output the three-dimensional white point of each region; The fifth processing module 350 is configured to: determine the confidence level of each region based on the three-dimensional white points, the spectral gradient values, and the structural labels; The sixth processing module 360 ​​is configured to: determine the global white point of each frame image based on the three-dimensional white point and the confidence level of all regions in each frame image.

[0124] In some embodiments, the first processing module 310 is specifically used to: acquire the image sequence; perform region segmentation on each frame of the image sequence to generate N regions for each frame of the image; and extract multispectral features from each of the N regions according to a preset number of multispectral channels to generate a multispectral vector corresponding to each region of each frame of the image.

[0125] In some embodiments, the third processing module 330 includes: a first processing unit, a second processing unit, and a third processing unit, wherein: the first processing unit is configured to: acquire two consecutive frames of multispectral vectors for each region; the second processing unit is configured to: perform a spectral gradient calculation operation based on the two consecutive frames of multispectral vectors to determine the spectral gradient value for each region; and the third processing unit is configured to: perform a reliability determination on the spectral gradient value based on a spectral gradient threshold and output a corresponding reliability label.

[0126] In some embodiments, the second processing unit is specifically used to: calculate the difference between the multispectral vectors of two consecutive frames and take the absolute value according to a preset number of multispectral channels to generate a multispectral difference vector for each region in each spectral channel; and sum the squares of the multispectral difference vectors of each region in all multispectral channels to generate a spectral gradient value for each region; wherein the spectral gradient value is used to distinguish between brightness changes and spectral changes.

[0127] In some embodiments, the third processing unit is specifically configured to: determine that the corresponding region is an unstable region and output an unreliable label when the spectral gradient value is greater than the spectral gradient threshold; and determine that the corresponding region is a stable region and output a reliable label when the spectral gradient value is less than or equal to the spectral gradient threshold.

[0128] In some embodiments, the fourth processing module 340 is specifically used to: construct an input feature vector by using the multispectral vector, the structural label, the spectral gradient value, and the reliability label as input features; input the input feature vector into a region white point determination model, and output the three-dimensional white point of each region; wherein the length of the input feature vector is the sum of the number of spectral channels, the number of structural labels, and a preset constant; the region white point determination model is a multilayer perceptron; and the three dimensions of the three-dimensional white point correspond to the calibration parameters of the red, green, and blue color channels, respectively.

[0129] In some embodiments, the fifth processing module 350 is specifically configured to: perform a weighted mapping calculation operation based on a first exponential decay function and the spectral gradient value to determine a first weight value for each region; query a preset mapping table for a second weight value that matches the structural label of each region; perform a weighted mapping calculation operation based on a second exponential decay function and the spectral gradient value to determine a third weight value for each region; perform a deviation calculation operation based on the three-dimensional white points of all regions in each frame image and the three-dimensional white points of each region to determine a fourth weight value for each region; and perform a product operation based on the first weight value, the second weight value, the third weight value, and the fourth weight value to determine the confidence level of each region; wherein, the first weight value characterizes the spectral stability of the corresponding region; the second weight value characterizes the structural reliability of the corresponding region; the third weight value characterizes the temporal consistency of the corresponding region; and the fourth weight value characterizes the global consistency of the corresponding region.

[0130] In some embodiments, the image white point determination device 300 further includes: a seventh processing module, an eighth processing module, and a ninth processing module, wherein: the seventh processing module is configured to: acquire two consecutive frames of spectral gradient values ​​for the corresponding region when the spectral gradient value is greater than the spectral gradient threshold; the eighth processing module is configured to: perform a weighted fusion operation based on the two consecutive frames of spectral gradient values ​​and a preset weighting coefficient to determine a target spectral gradient value for each region; and the ninth processing module is configured to: update the spectral gradient value of the corresponding region with the target spectral gradient value; wherein the smoothness of the target spectral gradient value is higher than the spectral gradient value before the weighted fusion operation.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the functional modules described above can be referred to the corresponding method steps in the foregoing embodiments, and will not be repeated here. Those skilled in the art can use different methods to implement the described functions for each specific application. For example, a computer software program containing the steps of the above method embodiments can be stored in a computer-readable storage medium so that, when executed, the program can implement one or more steps of the above method embodiments.

[0132] Figure 11 The diagram shows the structure of an electronic device according to an embodiment of this application. This embodiment does not limit the specific implementation of the electronic device.

[0133] like Figure 11 As shown, the electronic device 400 may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440.

[0134] The processor 410, communication interface 420, and memory 430 communicate with each other via communication bus 440. Communication interface 420 is used for communication connections with other external devices. The processor 410 executes program 450 to implement the image white point determination method described in one or more of the above embodiments.

[0135] Specifically, program 450 may include program code that includes computer operation instructions. When program 450 is invoked, processor 410 executes the computer operation instructions to implement the steps in the image white point determination method of one or more embodiments.

[0136] Depending on the actual application scenario, the processor 410 can be of the appropriate type, including but not limited to mainstream embedded processors such as microcontrollers (MCUs) and ARM architecture processors, as well as other types of processors such as digital signal processors (DSPs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs), as long as they can provide the computing and control capabilities required by the actual application scenario.

[0137] The memory 430 is used to store the program 450. It includes a program storage area and a data storage area. The program storage area is used to store firmware programs, embedded applications, and various functional modules; the data storage area is used to store data and calculation results during program execution. For example, the memory 430 may include: on-chip RAM (for temporary data storage during program execution); on-chip Flash memory (for storing program code and configuration data); and EEPROM or other types of non-volatile memory (for storing parameters that need to be retained when power is off).

[0138] This application also provides a computer-readable storage medium. This computer-readable storage medium can be a non-volatile computer-readable storage medium. This computer-readable storage medium stores a computer program.

[0139] When executed by a processor, the computer program implements one or more steps of the image white point determination method disclosed in the embodiments of this application. A complete computer program product is embodied on one or more computer-readable storage media containing the computer program disclosed in the embodiments of this application.

[0140] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above. For the sake of brevity, they are not provided in detail. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining white spots in an image, characterized in that, include: Each frame of the obtained image sequence is sequentially segmented into regions and multispectral features are extracted to determine the multispectral vector corresponding to each region of each frame. Identify the type of each region and determine the structural label of each region; Perform spectral gradient calculation to determine the spectral gradient value of each region and output the corresponding reliability label; The multispectral vector, the structural label, the spectral gradient value, and the reliability label are input into the region white point determination model, and the three-dimensional white point of each region is output. The confidence level of each region is determined based on the three-dimensional white points, the spectral gradient values, and the structural labels. The global white point of each frame image is determined based on the three-dimensional white point and the confidence level of all regions in each frame image.

2. The image white point determination method according to claim 1, characterized in that, The step of sequentially performing region segmentation and multispectral feature extraction on each frame of the obtained image sequence to determine the multispectral vector corresponding to each region of each frame includes: Obtain the image sequence; Each frame of the image sequence is segmented into regions to generate N regions for each frame. Based on a preset number of multispectral channels, multispectral features are extracted from each of the N regions to generate a multispectral vector corresponding to each region of each frame image.

3. The image white point determination method according to claim 1, characterized in that, The step of performing spectral gradient calculation to determine the spectral gradient value of each region and outputting the corresponding reliability label includes: Obtain the multispectral vectors of two consecutive frames for each region; Based on the two consecutive frames of multispectral vectors, a spectral gradient calculation operation is performed to determine the spectral gradient value of each region. The reliability of the spectral gradient value is determined based on the spectral gradient threshold, and the corresponding reliability label is output.

4. The image white point determination method according to claim 3, characterized in that, The step of performing spectral gradient calculation based on the two consecutive frames of multispectral vectors to determine the spectral gradient value of each region includes: According to the preset number of multispectral channels, the multispectral vectors of the two consecutive frames are subtracted channel by channel and the absolute value is taken to generate the multispectral difference vector of each region in each spectral channel; The spectral gradient value for each region is generated by summing the squares of the multispectral difference vectors across all multispectral channels. The spectral gradient value is used to distinguish between brightness changes and spectral changes.

5. The image white point determination method according to claim 3, characterized in that, The reliability determination of the spectral gradient value based on the spectral gradient threshold, and the output of the corresponding reliability label, includes: When the spectral gradient value is greater than the spectral gradient threshold, the corresponding region is determined to be an unstable region, and an unreliable label is output. When the spectral gradient value is less than or equal to the spectral gradient threshold, the corresponding region is determined to be a stable region, and a reliable label is output.

6. The image white point determination method according to claim 1, characterized in that, The step of inputting the multispectral vector, the structural label, the spectral gradient value, and the reliability label into the white point determination model of the input region, and outputting the three-dimensional white point of each region, includes: The multispectral vector, the structural label, the spectral gradient value, and the reliability label are used as input features to construct an input feature vector; The input feature vector is input into the region white point determination model, and the three-dimensional white point of each region is output. The length of the input feature vector is the sum of the number of spectral channels, the number of structural labels, and a preset constant; the region white point determination model is a multilayer perceptron; and the three dimensions of the three-dimensional white point correspond to the calibration parameters of the red, green, and blue color channels, respectively.

7. The image white point determination method according to claim 1, characterized in that, The process of determining the confidence level of each region based on the three-dimensional white points, the spectral gradient values, and the structural labels includes: Based on the first exponential decay function and the spectral gradient value, a weighted mapping calculation operation is performed to determine the first weight value of each region; From the preset mapping table, query the second weight value that matches the structural label of each region; Based on the second exponential decay function and the spectral gradient value, a weighted mapping calculation operation is performed to determine the third weight value of each region; Based on the three-dimensional white points of all regions and the three-dimensional white points of each region in each frame of the image, a deviation calculation operation is performed to determine the fourth weight value of each region. Based on the first weight value, the second weight value, the third weight value, and the fourth weight value, a product operation is performed to determine the confidence level of each region; Wherein, the first weight value represents the spectral stability of the corresponding region; the second weight value represents the structural reliability of the corresponding region; the third weight value represents the temporal consistency of the corresponding region; and the fourth weight value represents the global consistency of the corresponding region.

8. The image white point determination method according to any one of claims 3 to 7, characterized in that, The method further includes: When the spectral gradient value is greater than the spectral gradient threshold, two consecutive frames of spectral gradient values ​​for the corresponding region are obtained; Based on the spectral gradient values ​​of the two consecutive frames and the preset weighting coefficients, a weighted fusion operation is performed to determine the target spectral gradient value for each region. Update the spectral gradient value of the corresponding region with the target spectral gradient value; The smoothness of the target spectral gradient value is higher than that of the spectral gradient value before the weighted fusion operation.

9. An electronic device, characterized in that, include: The processor is communicatively connected to the memory, and the memory stores computer program instructions that, when invoked by the processor, cause the processor to execute the image white point determination method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores processor-executable computer program instructions, which, when executed by the processor, cause the computer to perform the image white point determination method as described in any one of claims 1-8.

Images