White balance processing method, apparatus, device, and storage medium

Abstract

This application provides a white balance processing method, apparatus, device, and storage medium. The method includes: determining a first landing point and a second landing point of an image in a target coordinate system, wherein the first landing point corresponds to a first region of a target color within the image, and the second landing point corresponds to a second region within the image other than the first region; determining a first color temperature based on the first landing point and a first gain parameter based on the first color temperature when the first landing point satisfies preset conditions; determining a second color temperature and a second gain parameter based on the second landing point; determining a white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; and performing white balance processing on the image based on the white balance gain parameter. This application can improve the white balance processing effect of images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a white balance processing method, apparatus, device and storage medium. Background Technology

[0002] Automatic white balance is a core function of digital imaging systems, designed to automatically adjust image color gain under different lighting conditions to present objectively realistic color effects. Related technologies widely employ prior white balance methods based on white point detection. These typically define a prior white area range covering multiple hardware references in a preset coordinate system and calculate white balance gain parameters by statistically analyzing the image points falling within the white area (i.e., the reference color region). However, to accommodate hardware differences, existing prior white area ranges are usually set quite large. When the image to be processed contains large areas of light-colored non-white regions (such as a blue sky scene), the coordinate distribution characteristics of the pixels in this area are highly similar to the standard white point. This makes it easy to misjudge these non-white points as standard white points and include them in the global gain calculation, leading to objective deviations in the final white balance parameters, causing distortion of key image colors, and resulting in poor white balance performance. Summary of the Invention

[0003] This application provides a white balance processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve the white balance processing effect of images.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a white balance processing method, including: Determine the first and second landing points of the image in the target coordinate system. The first landing point corresponds to the first region of the target color in the image, and the second landing point corresponds to the second region in the image other than the first region. If the first landing point meets the preset conditions, a first color temperature is determined based on the first landing point, and a first gain parameter is determined based on the first color temperature; Based on the second landing point, determine the second color temperature and the second gain parameter; The white balance gain parameter is determined based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; The image is subjected to white balance processing based on the white balance gain parameters.

[0005] This application also provides a white balance processing device, including: The first determining module is used to determine the first landing point and the second landing point of the image in the target coordinate system. The first landing point corresponds to the first region of the target color in the image, and the second landing point corresponds to the second region in the image other than the first region. The second determining module is used to determine a first color temperature based on the first landing point and a first gain parameter based on the first color temperature, provided that the first landing point meets the preset conditions. The third determining module is used to determine the second color temperature and the second gain parameter based on the second landing point; The fourth determining module is used to determine the white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; The white balance module is used to perform white balance processing on the image based on the white balance gain parameters.

[0006] This application also provides an electronic device, including: Memory is used to store executable instructions for a computer; The processor, when executing computer-executable instructions stored in the memory, implements the white balance processing method provided in the embodiments of this application.

[0007] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the white balance processing method provided in this application.

[0008] This application also provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the white balance processing method provided in this application.

[0009] The embodiments of this application have the following beneficial effects: By classifying, extracting, and specifically quantizing the pixel landing points in different regions of the image, the white balance processing is refined and differentiated. By determining the first landing point corresponding to the target color and the second landing point corresponding to the remaining regions, the potential interference of specific color regions on global color statistics can be effectively eliminated. When the first landing point meets the preset conditions, the first color temperature, first gain parameter, second color temperature, and second gain parameter are independently determined, and the final white balance gain parameter is determined by combining the above multi-dimensional parameters. This mechanism achieves adaptive fusion of the target color distribution and the conventional color distribution, effectively solving the problem of white balance parameter calculation deviation caused by large areas of specific color regions being similar to the reference color features. Finally, the image is corrected using the determined white balance gain parameter, which significantly enhances the robustness of the automatic white balance algorithm to complex scenes, effectively restores the objective and true colors of the image, and improves the white balance processing effect of the image. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the architecture of the white balance processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the first process of the white balance processing method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the second process of the white balance processing method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the color temperature region and color temperature trajectory line in the target coordinate system provided in the embodiments of this application; Figure 6 This is a schematic diagram of the third process of the white balance processing method provided in the embodiments of this application; Figure 7 This is a schematic diagram of image block processing provided in an embodiment of this application; Figure 8 This is a schematic diagram of the first mask provided in an embodiment of this application; Figure 9 This is a schematic diagram of the second mask provided in an embodiment of this application; Figure 10 This is a schematic diagram of the mapping position of the first color temperature provided in the embodiments of this application; Figure 11 This is a schematic diagram of the fourth process of the white balance processing method provided in the embodiments of this application; Figure 12 This is a schematic diagram illustrating the processing effect of the white balance processing method provided in the embodiments of this application.

[0011] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0015] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of a larger module or unit that includes the functionality of the module or unit.

[0016] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0017] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0018] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0019] 1) The target coordinate system refers to a two-dimensional mapping space used to quantify and characterize the proportional relationship between different color channels of image pixels. It belongs to the logical architecture of colorimetric analysis in the field of image processing. In the operation of this technical solution, it plays the role of data standardization. By converting the raw color channel values ​​collected by the image sensor into mutually independent coordinate points, it realizes the geometric description and statistical analysis of the color distribution characteristics of the image to be processed under a unified benchmark, thereby providing basic support for subsequent color temperature determination and parameter calculation. In some embodiments of this application, the target coordinate system is specifically manifested as a two-dimensional rectangular coordinate system composed of the ratio of red-green channels and the ratio of blue-green channels, such as, but not limited to, other coordinate systems. A coordinate system, where K is a preset constant designed to simplify the computational complexity of the hardware.

[0020] 2) The color temperature zone of the target color refers to a closed geometric region pre-defined within the target coordinate system, corresponding to the color distribution pattern of a specific non-neutral color object under different standard light sources. As a pre-defined set of prior data, it plays the role of physical feature verification and preliminary screening during the execution of this technical solution. By determining whether the mapped coordinates of the pixel to be processed fall within the boundary range of the closed geometric region, the object region with specific spectral reflectance characteristics in the image can be identified, thereby assisting in locating specific scene content that is prone to white balance color cast interference. In some embodiments of this application, the color temperature zone of the target color is composed of multiple minimum convex hull geometric regions obtained from multiple shooting devices under standard color temperatures such as HZ, A, TL84, D50, D65 and D95, for example, the blue sky color temperature zone corresponding to the distribution characteristics of color block No. 3 in the standard 24 color card.

[0021] 3) Color temperature trajectory line refers to a multi-segment model or fitted curve data structure in the target coordinate system, which is formed by connecting a series of reference center points corresponding to known standard color temperatures in the order of color temperature evolution. In this technical solution, it serves as a geometric reference for color temperature positioning and coordinate mapping, and is used to objectively describe the dynamic evolution law of specific color components with the change of ambient light color temperature. By calculating the geometric positional relationship between the data points to be processed and the trajectory line, the estimation of ambient color temperature and the correlation mapping between coordinates of different color spaces are realized. In some embodiments of this application, the color temperature trajectory line includes a first color temperature trajectory line corresponding to the target color (such as the Planck line for blue sky) and a second color temperature trajectory line corresponding to the reference color (such as the Planck line for white). In terms of data logic, it is represented as a broken line model formed by connecting the statistical average values ​​of the median material landing points under different standard light sources in sequence.

[0022] 4) White balance gain parameters refer to a set of product coefficients used to linearly scale or compensate the original color channel values ​​of the image to be processed. These are correction control parameters in the image signal processing flow. In the color correction stage of this technical solution, this term functions to restore true colors. By applying the gain coefficients calculated based on the color temperature trajectory to the red and blue channel pixel components of the image, it compensates for the color shift caused by the ambient light source's color temperature, thus making the processed image visually closer to an objectively realistic color distribution. In some embodiments of this application, the white balance gain parameter is a globally consistent gain formed by fusing the first gain parameter and the second gain parameter based on the color temperature difference and the proportion of the first landing point, specifically including the gain for the red channel. And the gain for the blue channel .

[0023] This application provides a white balance processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the white balance processing effect of images. The following is a detailed description of the embodiments of this application based on the above explanation of the terms and concepts used.

[0024] The white balance processing system provided in the embodiments of this application is described below. See also... Figure 1 , Figure 1 This is a schematic diagram of the architecture of the white balance processing system provided in this application embodiment. To support an exemplary application, the white balance processing system 100 includes: a server 200, a network 300, and a terminal 400. The terminal 400 is connected to the server 200 via the network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both, using wireless or wired links for data transmission.

[0025] Here, in response to a white balance processing instruction for an image, terminal 400 sends a white balance processing request for the image to server 200; server 200, in response to the white balance processing request sent by terminal 400, acquires the image to be processed; determines a first landing point and a second landing point of the image in the target coordinate system, the first landing point corresponding to a first region of the target color within the image, and the second landing point corresponding to a second region within the image excluding the first region; if the first landing point meets preset conditions, determines a first color temperature based on the first landing point, and determines a first gain parameter based on the first color temperature; determines a second color temperature and a second gain parameter based on the second landing point; determines a white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; performs white balance processing on the image based on the white balance gain parameter to obtain a white balance processed image; returns the white balance processed image to terminal 400; terminal 400 receives the white balance processed image returned by server 200; and displays the white balance processed image.

[0026] The white balance processing method provided in this application embodiment is implemented by an electronic device. For example, it can be implemented by a terminal alone, by a server alone, or by a terminal and a server working together. The electronic device implementing the white balance processing method provided in this application embodiment can be various types of terminals or servers. The server (e.g., server 200) can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal (e.g., terminal 400) can be a laptop, tablet, desktop computer, smartphone, smart voice interaction device (e.g., smart speaker), smart home appliance (e.g., smart TV), smartwatch, vehicle terminal, wearable device, virtual reality (VR) device, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application embodiment does not impose any restrictions on this.

[0027] The following describes an electronic device implementing the white balance processing method provided in an embodiment of this application. See also... Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 500 provided in this embodiment can be a terminal or a server. Figure 2 As shown, electronic device 500 includes at least one processor 510, memory 550, at least one network interface 520, and user interface 530. The various components in electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 540.

[0028] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0029] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0030] Memory 550 may be removable, non-removable, or a combination thereof. Memory 550 may include one or more storage devices physically located away from processor 510. Memory 550 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0031] In some embodiments, memory 550 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, as illustrated below. Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks; network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including Bluetooth, Wireless Fidelity (Wi-Fi), and Universal Serial Bus (USB); presentation module 553 is used to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 associated with user interface 530 (e.g., a display screen, a speaker, etc.); input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0032] In some embodiments, the white balance processing device provided in this application can be implemented in software. Figure 2A white balance processing device 555 stored in memory 550 is shown. It can be software in the form of programs and plug-ins, including the following software modules: a first determination module 5551, a second determination module 5552, a third determination module 5553, a fourth determination module 5554, and a white balance module 5555. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0033] The white balance processing method provided in the embodiments of this application is described below. As mentioned above, the white balance processing method provided in the embodiments of this application is implemented by an electronic device, such as a server or terminal alone, or a server and terminal working together. Therefore, the executing entity of each step will not be described again below. See Figure 3 , Figure 3 This is a first flowchart illustrating the white balance processing method provided in this application embodiment. The white balance processing method provided in this application embodiment includes: Step 101: Determine the first and second landing points of the image in the target coordinate system.

[0034] The first landing point corresponds to the first region of the target color within the image, and the second landing point corresponds to the second region within the image other than the first region.

[0035] Step 101 is the process of classifying and extracting image landing points and dividing them into regions. The target coordinate system is a two-dimensional coordinate system used to represent the ratio relationship of color channels, which can be exemplarily a... The coordinate system is defined as follows: K is a preset constant, and R, G, and B represent the red, green, and blue color channels, respectively. The first landing point is the coordinate point mapped in the target coordinate system to an image patch with a specific target color semantics. For example, the first landing point could be the landing point of an image patch with a blue sky color. The second landing point is the coordinate point mapped in the target coordinate system to other image patches in the image besides those with the target color. The first region is a set of connected or disconnected image patches within the image composed of the target color. For example, the first region could be a blue sky region. The second region is the set of remaining image patches within the image that do not belong to the first region, and the second region includes image patches with a base color (such as white).

[0036] In some embodiments, see Figure 4Before step 101, "determining the first and second landing points of the image in the target coordinate system," the following steps 201-205 can also be performed: Step 201, for each of the multiple standard color temperatures, acquire reference images of the reference object obtained by multiple shooting devices under the standard color temperature, and determine the statistical points of the multiple reference images captured under the standard color temperature in the target coordinate system; Step 202, for each standard color temperature, determine the minimum convex hull geometric region of the multiple statistical points corresponding to the standard color temperature, and generate the closed polygon boundary corresponding to the standard color temperature based on the minimum convex hull geometric region; Step 203, for each standard color temperature, from the standard... Among the multiple statistical points corresponding to the standard color temperature, the reference statistical point corresponding to the reference shooting device is determined, and the average coordinate value of the reference statistical point is determined. The point indicated by the average coordinate value is taken as the center point corresponding to the standard color temperature. The multiple shooting devices include the reference shooting device. Step 204: When the reference object is a color block of the target color, the boundary of each closed polygon is determined as the color temperature zone of the target color, and multiple center points are connected to obtain the first color temperature trajectory line of the target color. Step 205: When the reference object is a test gray card, the boundary of each closed polygon is determined as the color temperature zone of the reference color, and multiple center points are connected to obtain the second color temperature trajectory line of the reference color.

[0037] Here, standard color temperature refers to preset temperature parameters used to simulate different ambient lighting conditions, specifically including HZ (2300K), A (2800K), TL84 (4000K), D50 (5000K), D65 (6500K), and D95 ​​(9500K). The reference object is a physical chart providing a standard color reflectance reference under a specific light source, specifically including a test gray chart and color patches representing the target color. The reference imaging device is a camera module with standard hardware physical characteristics, specifically including a median sensor and a median infrared filter. Statistical points are the discrete coordinates of the pixel mean of the reference image mapped in the target coordinate system. The minimum convex occlusion geometric region is the convex polygon with the smallest area that can completely contain all discrete points within a specific plane. The reference image is the raw, unprocessed data, i.e., the RAW image. The RAW image contains electrical signal data recorded by the shooting device during exposure that has not been processed by the Image Signal Processing Pipeline (ISP Pipeline). It has not undergone automatic compression and modification by parameters such as white balance, contrast, and color space of the camera.

[0038] Step 201: For each of the multiple standard color temperatures, acquire reference images of the reference object captured by multiple shooting devices, including extreme value shooting devices, at the current standard color temperature. Extract the mean values ​​of the three color channels of each reference image and map them to a preset target coordinate system to determine multiple statistical points that are clustered under the current standard color temperature.

[0039] Step 202: For each standard color temperature, calculate the minimum convex hull geometric region containing all statistical points generated by all shooting devices under the current standard color temperature. Based on the calculated minimum convex hull geometric region, generate the closed polygon boundary corresponding to the current standard color temperature. Overlapping regions are allowed between the closed polygon boundaries corresponding to adjacent standard color temperatures.

[0040] Step 203: For each standard color temperature, select the reference statistical points generated by the reference shooting device from multiple statistical points corresponding to the current standard color temperature, calculate the average coordinate value (i.e., statistical average value) of all reference statistical points, and determine the geometric coordinate point indicated by the average coordinate value as the center point corresponding to the current standard color temperature.

[0041] Step 204: When the reference object is a color block representing the target color (such as color block No. 3 of a 24-color chart, which is blue sky), the boundary of each closed polygon generated based on statistical points is determined as the color temperature zone of the target color under the corresponding color temperature. Multiple center points are connected sequentially according to the color temperature sequence to obtain a multi-segment polyline model and it is determined as the first color temperature trajectory line of the target color, such as the Planck line of blue sky.

[0042] Step 205: When the reference object is a test gray card, the boundary of each closed polygon generated based on statistical points is determined as the color temperature zone of the reference color under the corresponding color temperature. Multiple center points are connected in sequence according to the color temperature sequence to obtain a multi-segment polyline model and it is determined as the second color temperature trajectory line of the reference color, such as the white Planck line.

[0043] As an example, see Figure 5 , Figure 5 The target coordinate system is shown. The reference color (i.e., white) in the coordinate system falls within the color temperature range (i.e., white area) of standard color temperatures HZ (2300K), A (2800K), TL84 (4000K), D50 (5000K), D65 (6500K), and D95 ​​(9500K). The target color (i.e., blue sky) falls within the color temperature range (i.e., blue sky area) of standard color temperatures HZ (2300K), A (2800K), TL84 (4000K), D50 (5000K), D65 (6500K), and D95 ​​(9500K). The first color temperature trajectory line of the target color (Planck line for blue sky) and the second color temperature trajectory line of the reference color (Planck line for white) are also shown.

[0044] Applying the above embodiments, this calibration process effectively accommodates physical discreteness, including extreme value devices, by using the minimum convex hull algorithm to generate polygonal boundaries, thus constructing a highly reliable color temperature zone model. By independently acquiring reference charts to construct dual color temperature trajectory lines, independent and accurate quantification of the color temperature variation of specific colors is achieved. This provides an objective physical quantification benchmark for subsequent landing point classification estimation and cross-trajectory mapping in complex scenarios from the underlying data structure level.

[0045] In some embodiments, see Figure 6 Step 101, "Determining the first and second landing points of the image in the target coordinate system," can be achieved by executing the following steps 1011-1014: Step 1011, dividing the image into blocks to obtain multiple image blocks; Step 1012, for each image block, determining the average color channel value of the image block in each color channel, and determining the landing point of the image block in the target coordinate system based on the average color channel value; Step 1013, identifying the first region of the target color in the image, and taking the region in the image other than the first region as the second region; Step 1014, taking the landing point of the image block belonging to the first region as the first landing point, and taking the landing point of the image block belonging to the second region as the second landing point.

[0046] Here, block processing is a preprocessing procedure that divides the pixel matrix of the entire image into multiple sub-regions to reduce computational complexity. The first region is a set of pixels in the image with specific color attributes, for example, the blue sky region. The second region is a set of background or object pixels excluding the first region.

[0047] Step 1011: Obtain the image to be processed and perform block processing. Divide the image into blocks. Image blocks of the same size are exemplarily divided into Image patches. As an example. Figure 7 The diagram illustrates the process of dividing an image into blocks, which are then divided into 8x8 image blocks.

[0048] Step 1012: Calculate the average color channel value for each color channel for each image block. The average value of the red channel is obtained by summing the color channel component values ​​of each pixel within the image block and taking the arithmetic mean. Average value of green channel and the average value of the blue channel The landing point of the image patch is calculated in the target coordinate system based on the average value of the color channels. The x-coordinate of the landing point... The vertical coordinate of the landing point ,in, Indicates the x-coordinate of the landing point. Represents the vertical coordinate of the landing point. This is a preset constant. In practical applications, abnormal image blocks with excessively high brightness (i.e., brightness above the threshold) or excessively low brightness (brightness below the threshold) can be removed first, and then the remaining image blocks can be processed in steps 1012-1014.

[0049] Step 1013: Identify the first region of the target color within the image. A first mask is output using a pre-trained AI recognition model. Regions identified as the target color are marked as 1 in the first mask, and regions not identified as the target color are marked as 0. Simultaneously, a second mask is output based on whether the image patch falls within a preset color temperature frame of the target color and whether the brightness of the image patch is less than a preset brightness threshold. Specifically, for image patches that simultaneously meet the first preset condition (i.e., the image patch falls within the preset color temperature frame of the target color) and the second preset condition (i.e., the brightness of the image patch is less than the preset brightness threshold), their identifier value is marked as 1 in the second mask; for image patches that do not simultaneously meet both the first and second preset conditions, their identifier value is marked as 0 in the second mask. The specific color temperature frame of the target color exemplarily includes... Figure 5 The target color is defined by color temperatures of D95, D65, and D50. The intersection of the first and second masks, both marked as 1, is taken as the first region of the target color, and the region within the image excluding the first region is taken as the second region.

[0050] Step 1014: Take the landing point of the image block belonging to the first region as the first landing point, and take the landing point of the image block belonging to the second region as the second landing point.

[0051] By applying the above embodiments, the computational load on a massive number of pixels is significantly reduced through block processing, and the color distribution features of the image are effectively extracted using multi-color channel ratio mapping. By combining artificial intelligence semantic recognition with a dual discrimination mechanism of physical optical features (color temperature frame and brightness threshold), large-area target color regions can be identified with high precision, effectively solving the misjudgment problem caused by the high similarity between light-colored non-white areas and the baseline white point features in traditional algorithms. This refined region division and point classification provides accurate data support for the subsequent implementation of targeted differentiated white balance strategies, ensuring the baseline accuracy of color reproduction in complex scenes.

[0052] In some embodiments, the step 1013 of "identifying the first region of the target color in the image" can be achieved by performing the following steps: identifying the target scene in the image using a neural network model to obtain a first mask for multiple image blocks, wherein the target scene has a target color; determining the target color temperature frame of the target color in the target coordinate system for each landing point, and determining the brightness of each image block; generating a second mask for multiple image blocks based on the target color temperature frame and the brightness; and determining the first region based on the first mask and the second mask.

[0053] Here, the first mask is a two-dimensional matrix representing whether a pixel block belongs to the target scene (such as a blue sky scene, a green plant scene, a land scene, etc.) after semantic feature extraction of the image using artificial intelligence algorithms. For example, it is... Figure 8 The example shown is an AIMASK. A color temperature frame is a closed polygonal boundary defined within the color temperature range of the target color, corresponding to different specific color temperature intervals. Examples include... Figure 5 The target color is shown with D95, D65, and D50 color temperature frames. The target color temperature frame is the color temperature frame of the target color where the point falls within the target coordinate system. The second mask is a two-dimensional matrix representing the physical attribute characteristics of pixel blocks, generated after dual-condition filtering of image blocks based on physical coordinate distribution and optical brightness attributes. An example is... Figure 9 The skymask shown. The first region is a set of image patches with target color attributes, which were finally determined after dual verification of semantic and physical features.

[0054] Specifically, the image to be processed is input into a pre-trained neural network model for target scene recognition, outputting a first mask of multiple image patches with dimensions consistent with the image block dimensions. It should be noted that this pre-trained neural network model can recognize target scenes with the target color (e.g., a blue sky). In the first mask, image patches identified as target scenes are marked with a value of 1, while image patches not identified as target scenes are marked with a value of 0. For example... Figure 8 The image shown is a visualization example of the first mask, Aimask. The black area represents the image block with a label value of 0, and the white area represents the image block with a label value of 1.

[0055] Continuing, for each pre-divided image block, it is determined whether the target color temperature frame containing the corresponding landing point of the image block in the target coordinate system falls within a specific color temperature frame (such as the D95, D65, and D50 color temperature frames within the target color temperature range) of the target color (i.e., whether the landing point of the image block in the target coordinate system is within a specific color temperature frame within the target color temperature range). If the landing point is within a specific color temperature frame, the first preset condition is satisfied. Simultaneously, the brightness of each image block is determined, and the formula for calculating brightness is: Formula (1) in, Indicates the brightness of an image patch. This represents the average value of the red channel in the image patch. This represents the average value of the green channel in the image patch. This represents the average value of the blue channel in the image patch.

[0056] A preset brightness threshold is set to distinguish high-brightness environmental color temperature interference areas. When the brightness of an image block is less than the preset brightness threshold, it is determined that a second preset condition is met. Therefore, a second mask for multiple image blocks is generated based on the distribution of the landing points within the color temperature frame and the calculated brightness. Specifically, for image blocks that simultaneously meet both the first and second preset conditions, their identifier value is marked as 1 in the second mask; for image blocks that do not simultaneously meet both preset conditions, their identifier value is marked as 0 in the second mask. Figure 9 The image shown is a visualization example of the second skymask. Black areas represent image blocks with a flag value of 0, and white areas represent image blocks with a flag value of 1. Next, the first and second masks are obtained and compared. Image blocks with a flag value of 1 at corresponding positions in the first and second masks are extracted. The set of all extracted image blocks is defined as the first region.

[0057] By applying the above embodiments, and combining the semantic recognition advantages of neural network models with the physical verification mechanism of landing point distribution and brightness features, dual and accurate identification of target scenes within images is achieved. The combined judgment using color temperature boxes and brightness thresholds effectively eliminates interference areas in the image that have similar landing point features to the target color but significant brightness differences, such as bright white clouds. This overcomes the technical deficiency of relying solely on coordinate system landing point statistics, which easily leads to large-area misjudgments. It significantly improves the accuracy of target color region extraction in complex scenes and provides a high-confidence data source for subsequent white balance gain parameter calculations.

[0058] It should be noted that the image and each image block obtained by dividing the image are raw, unprocessed data, i.e., RAW images. In other words, the color channels (R, G, B color channels) of the image and each image block are raw data that have not been processed by the ISPPipeline.

[0059] Step 102: If the first landing point meets the preset conditions, determine the first color temperature based on the first landing point, and determine the first gain parameter based on the first color temperature.

[0060] Step 102 aims to trigger a gain calculation mechanism for the target color when the distribution of target color points within the image reaches a specific scale, thus preventing interference from affecting global color correction. The preset condition is a threshold used to determine whether the distribution ratio of the first point in the image reaches the threshold for triggering independent gain calculation. The first color temperature is a color temperature index obtained by vector projection and interpolation of the coordinates of the first point based on the Planck line of a pre-calibrated target color. The first gain parameter is a value obtained by mapping the first color temperature to the Planck line of the reference color and then performing coordinate system transformation to correct a specific color channel of the image. Before performing gain calculation, the total number of first points is first counted, and the proportion of the total number of first points to the total number of all points in the image is calculated. When the proportion is greater than or equal to a preset proportion threshold, the first point is determined to meet the preset condition. For example, the preset proportion threshold can be 20%. If the first point meets the preset condition, the first color temperature is determined based on the first point, and the first gain parameter is determined based on the first color temperature.

[0061] In some embodiments, step 102, "determining the first color temperature based on the first landing point," can be achieved by performing the following steps: determining the first average coordinate of the landing point coordinates and using the point indicated by the first average coordinate as the first color temperature reference point; determining the distance from the first color temperature reference point to each of the multiple line segments included in the first color temperature trajectory line, and determining the first line segment with the smallest corresponding distance from the multiple line segments, wherein the first color temperature trajectory line corresponds to the target color; and interpolating the endpoint color temperatures of the two endpoints of the first line segment based on the first projection parameters of the first color temperature reference point on the first line segment to obtain the first color temperature.

[0062] Here, the first average coordinates are the geometric center coordinates of the first set of landing points in the target coordinate system. The first color temperature reference point is the geometric reference point determined using the first average coordinates for calculating the color temperature, exemplarily the equivalent gain point of the blue sky landing point. The first color temperature trajectory is a pre-calibrated broken line model in the target coordinate system representing the variation of the target color with color temperature, exemplarily the Planck line of the blue sky. The first line segment is the broken line segment in the first color temperature trajectory that is closest to the first color temperature reference point. The first projection parameter represents the position of the orthogonal projection point of the first color temperature reference point on the first line segment. The first color temperature is the final value of the color temperature estimation of the target color based on the position of the projection point.

[0063] Specifically, calculate the average x-coordinate and average y-coordinate of all first landing points in the target coordinate system, and determine this as the first average coordinate, i.e., the first average coordinate is ( , ),in, The x-coordinate of the first average coordinate is represented. The ordinate of the first average coordinate is represented by the vertical axis. Indicates the x-coordinate of each first landing point. Represents the ordinate of each first landing point. This represents the total number of the first landing points. The coordinate point indicated by the first average coordinate is taken as the first color temperature reference point and denoted as the reference point. The first color temperature trajectory line corresponds to the target color and is formed by sequentially connecting multiple Planck points with known calibrated color temperatures. The distances from the first color temperature reference point to each of the multiple line segments included in the first color temperature trajectory line are calculated. Vector distance calculation is used to obtain the distance values, and the first line segment with the smallest corresponding distance is determined from among the multiple line segments.

[0064] Assume the two endpoints of the first line segment are the endpoints. and endpoints The endpoint color temperature corresponding to endpoint A is The endpoint color temperature corresponding to endpoint B is Construct a line segment vector pointing from endpoint A to endpoint B. and the reference vector pointing from endpoint A to the first color temperature reference point. The first projection parameters of the first color temperature reference point on the first line segment are calculated based on the vector projection method. The formula for calculating the first projection parameters is as follows: Formula (2) in, For the first projection parameter, This represents the dot product of the line segment vector and the reference vector. It represents the square of the magnitude of the line segment vector.

[0065] Continuing, the endpoint color temperatures of the two endpoints of the first line segment are interpolated based on the first projection parameters to obtain the first color temperature. The interpolation calculation formula is as follows: Formula (3) in, This indicates the first color temperature.

[0066] It should be noted that when the calculated first color temperature exceeds the calibrated color temperature range contained in the first color temperature trajectory line, the endpoint color temperature of the endpoint closest to the first color temperature reference point is returned as the default first color temperature.

[0067] By applying the above embodiments, the global features of the target color are extracted by calculating the first average coordinates, effectively eliminating the interference of a single discrete landing point on the overall color temperature assessment. Using vector projection and linear interpolation algorithms, the two-dimensional coordinates are accurately mapped into a continuous one-dimensional color temperature index, achieving a quantitative estimation of the ambient light in the target color region. By constructing an independent first color temperature trajectory line for the target color, the structural limitations of traditional methods, which are easily misled by non-reference colors, are overcome, ensuring the objectivity of specific color assessments in complex scenes.

[0068] In some embodiments, the step 102 of "determining the first gain parameter based on the first color temperature" can be achieved by performing the following steps: mapping the first color temperature to the second color temperature trajectory line of the reference color, and determining the position coordinates of the mapping position of the first color temperature on the second color temperature trajectory line; obtaining the reference gain coefficient, and determining the first gain parameter based on the position coordinates and the reference gain coefficient.

[0069] The step "determine the position coordinates of the mapping position of the first color temperature on the second color temperature trajectory line" can be achieved by performing the following steps: from the multiple line segments included in the second color temperature trajectory line, determine the second line segment including the first color temperature; based on the endpoint color temperature and the first color temperature of the two endpoints of the second line segment, determine the interpolation coefficients; based on the interpolation coefficients, perform interpolation processing on the endpoint coordinates of the two endpoints of the second line segment to obtain the position coordinates.

[0070] Here, the first color temperature is the ambient light color temperature index of the target area calculated based on the first landing point. The second color temperature trajectory is a pre-calibrated broken line model in the target coordinate system representing the variation of the reference color with color temperature, exemplified by a white Planck line. The position coordinates are the two-dimensional coordinates of the mapped position on the second color temperature trajectory, where the first color temperature is mapped. The reference gain coefficient is a preset constant used to convert coordinate values ​​into image channel gain, typically 1024. The first gain parameter is a value calculated independently for the first landing point to adjust a specific color channel of the image.

[0071] Specifically, the first color temperature is mapped onto the second color temperature trajectory line of the reference color; that is, a second line segment containing the first color temperature is found within the second color temperature trajectory line. See, for an example. Figure 10 Point 1001 represents the first color temperature reference point (also known as the equivalent gain point) of the first landing point (such as the landing point of blue sky), and point 1002 is the mapping point of the same color temperature of the first color temperature reference point on the second color temperature trajectory line (such as the Planck line of white), that is, the mapping point obtained by mapping the first color temperature to the second color temperature trajectory line of the reference color.

[0072] Assume that the two endpoints of the second line segment containing the first color temperature are respectively the endpoints. and endpoints The endpoint color temperature corresponding to endpoint E is The endpoint color temperature corresponding to endpoint F is The position coordinates of the mapping position of the first color temperature on the second color temperature trajectory line are determined based on the first color temperature. The formula for calculating the interpolation parameters is as follows: Formula (4) in, Indicates the interpolation parameters. This indicates the first color temperature.

[0073] The position coordinates are determined using interpolation parameters, where the x-coordinate of the position coordinate is... The ordinate of the position coordinate is ,in, The x-coordinate represents the position coordinates. The vertical coordinate represents the position coordinates.

[0074] Continuing, the reference gain coefficient is obtained, with the constant value of the reference gain coefficient being 1024 for example. A first gain parameter is determined based on the position coordinates and the reference gain coefficient. The first gain parameter includes the first gain parameter for the red channel and the first gain parameter for the blue channel. The formula for calculating the first gain parameter for the red channel is: Formula (5) The formula for calculating the first gain parameter of the blue channel is: Formula (6) in, Represents the reference gain coefficient. This indicates that the first gain parameter includes the gain parameter of the red channel. This indicates that the first gain parameter includes the gain parameter of the blue channel.

[0075] It should be noted that in the image data domain, since the green channel value is usually greater than the red and blue channel values, color adjustments are based on the green channel value and the green channel value is not adjusted.

[0076] By applying the above embodiments, the ambient color temperature of the target color is equivalently converted into a color correction reference for the base color through a mapping mechanism across color temperature trajectories, thus fundamentally eliminating the objective interference of special target colors on global color statistics. Using linear interpolation algorithms and mathematical derivation of the reference gain coefficient, the continuous one-dimensional color temperature index is accurately restored to channel gain parameters, ensuring the objectivity and accuracy of gain parameter calculations in complex scenes. This lays a quantitative foundation for subsequent white balance correction to eliminate image color cast issues.

[0077] Step 103: Based on the second landing point, determine the second color temperature and the second gain parameter.

[0078] Step 103 involves independently calculating color correction parameters for all regular regions in the image except the target color region. The second landing point is the set of coordinate points mapped in the target coordinate system to the remaining image patch after excluding the first region corresponding to the target color. The second color temperature is a color temperature index obtained through interpolation based on the statistical distribution of the second landing points within the color temperature range of the reference color. The second gain parameter is a value calculated based on the second landing point for color correction of a specific color channel of the image.

[0079] Specifically, a standard white balance strategy is applied to the second landing point (excluding the first landing point) to obtain the second color temperature and second gain parameters. The color temperature range of the pre-calibrated reference color and its Planck line are obtained in the target coordinate system. The geometric boundaries of the second landing point and the reference color temperature range are compared, and landing points whose coordinates fall entirely within the reference color temperature range are selected as valid second landing points. The average x-coordinate and y-coordinate of all valid second landing points in the target coordinate system are calculated to obtain an average landing point representing the overall color distribution characteristics of the second region. The second gain parameters for each color channel are calculated using the average x-coordinate and y-coordinate of the average landing points. Specifically, the second gain parameters include those for the red channel. And the second gain parameter for the blue channel Interpolation is performed on the average landing point using the Planck line of the base color to obtain the color temperature value corresponding to the average landing point, and this color temperature value is determined as the second color temperature, specifically denoted as... The execution logic of the white balance strategy is applicable to both conventional white spot detection algorithms and white balance algorithms derived and optimized based on the white spot detection principle.

[0080] In some embodiments, step 103, "determining the second color temperature and the second gain parameter based on the second landing point," can be achieved by performing the following steps: determining the second average coordinates of the landing point coordinates of the second landing point located in the color temperature zone of the reference color; using the point indicated by the second average coordinates as the second color temperature reference point, and determining the distances from the second color temperature reference point to each of the multiple line segments included in the second color temperature trajectory line, and determining the third line segment corresponding to the smallest distance from the multiple line segments, wherein the second color temperature trajectory line corresponds to the reference color; interpolating the endpoint color temperatures of the two endpoints of the third line segment based on the first projection parameter of the second color temperature reference point on the third line segment to obtain the second color temperature; obtaining the reference gain coefficient, and determining the second gain parameter based on the second average coordinates and the reference gain coefficient.

[0081] This method accurately estimates the reference value of the current ambient light by statistically analyzing the distribution characteristics of the landing points within the reference color region and utilizing geometric relationships and interpolation algorithms. The color temperature region of the reference color is a pre-defined closed polygonal region in the target coordinate system used to determine the pixel distribution of the reference color; exemplarily, it is the white region (i.e., the white color temperature region). The second color temperature trajectory line is a polygonal line model in the target coordinate system composed of multiple nodes with known calibrated color temperatures connected sequentially; exemplarily, it is the white Planck line. The second average coordinate is the geometric center point of the second landing point in the image corresponding to a non-target color and located within the reference color temperature region. The reference gain coefficient is a preset constant used to convert coordinate ratios into channel gain; exemplarily, it is 1024.

[0082] Specifically, firstly, the second landing point whose coordinate position is within the color temperature range of the reference color is selected as the valid second landing point. The average x-coordinate and average y-coordinate of the selected valid second landing points are calculated to determine the second average coordinate, i.e., the second average coordinate is ( , ),in, and These represent the horizontal and vertical components of the second average coordinate, respectively. and The x and y coordinates of each second landing point within the color temperature range of the reference color are represented. This represents the total number of second landing points falling within the color temperature range of the base color. (This will be determined by...) The determined coordinate point is used as the second color temperature reference point.

[0083] Continuing, obtain the pre-calibrated second color temperature trajectory line. For each line segment contained in the second color temperature trajectory line, calculate the Euclidean distance from the second color temperature reference point to that line segment. Select the line segment with the smallest corresponding distance value from all line segments as the third line segment. Let the starting endpoint of the third line segment be... The termination endpoint is The endpoint color temperature corresponding to the starting endpoint is The endpoint color temperature corresponding to the termination endpoint is The projection position of the second color temperature reference point onto the third line segment is calculated to determine the first projection parameter. The formula for calculating the first projection parameter is: Formula (7) in, This represents the first projection parameter.

[0084] The second color temperature is obtained by interpolating the endpoint color temperatures of the two endpoints of the third line segment based on the first projection parameters. The calculation formula is as follows: Formula (8) in, This indicates the second color temperature.

[0085] Obtain the reference gain coefficient The second gain parameter is determined based on the second average coordinates and the baseline gain coefficient. In the RAW domain of the image file, the G channel value is larger than the R / B value, so adjustments are made based on the G channel value; the G channel value itself is not adjusted. Therefore, the second gain parameter includes the red channel gain parameter and the blue channel gain parameter, i.e.: Formula (9) Formula (10) in, This refers to the gain parameter of the red channel in the second gain parameter set. This refers to the gain parameter of the blue channel in the second gain parameter.

[0086] By applying the above embodiments, noise reduction of stray points is achieved by determining the second average coordinates, improving the stability of the baseline color feature extraction. The use of geometric projection and line segment interpolation algorithms solves the problem of accurately mapping continuous color temperature values ​​on discrete polyline trajectories, ensuring the objectivity of the second color temperature calculation. The coordinate inverse operation combined with the baseline gain coefficient provides a quantitative benchmark for subsequent multi-strategy fusion, effectively guaranteeing the baseline accuracy and consistency of color correction in conventional scenarios.

[0087] Step 104: Determine the white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter.

[0088] Step 104 is the process of strategically fusing or selecting two sets of gain parameters calculated from different point sets to determine the final color correction parameters applied to the entire image. The first color temperature is the color temperature value calculated based on the first point. The second color temperature is the color temperature value calculated based on the second point. The first gain parameter is the color channel gain calculated independently for the first point. The second gain parameter is the color channel gain calculated independently for the second point. The white balance gain parameter is a globally unified parameter ultimately used to adjust the pixel values ​​of each color channel in the image to be processed. The absolute value of the difference between the first and second color temperatures is calculated to determine the difference between the results obtained from the two parameter calculation methods. A preset color temperature difference threshold is set to measure the degree of difference; the preset color temperature difference threshold can be, for example, 500K. The absolute value of the difference is compared with the preset color temperature difference threshold. When the absolute value of the difference is greater than or equal to the preset color temperature difference threshold, the first gain parameter is determined as the final white balance gain parameter. When the absolute value of the difference is greater than or equal to the preset color temperature difference threshold, it is usually because there are few reference colors in the image, causing the second point to fail to effectively converge within the color temperature range of the reference colors. When the absolute value of the difference is less than the preset color temperature difference threshold, the proportion of the total number of the first landing points to the total number of all landing points in the image is obtained. The first gain parameter and the second gain parameter are weighted and averaged according to this proportion. The result of the weighted average calculation is determined as the final white balance gain parameter.

[0089] In some embodiments, step 104, "determining the white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter," can be achieved by performing the following steps: determining the color temperature difference between the first color temperature and the second color temperature; if the color temperature difference is greater than a color temperature difference threshold, using the first gain parameter as the white balance gain parameter; if the color temperature difference is less than or equal to the color temperature difference threshold, weighting the first gain parameter and the second gain parameter based on the proportion of the first landing point relative to all landing points of the image in the target coordinate system to obtain the white balance gain parameter.

[0090] Here, the first color temperature is the color temperature index representing the ambient light of a specific color region, calculated for the first landing point. The second color temperature is the color temperature index statistically obtained for the second landing point within the color temperature range of the reference color. The color temperature difference is the absolute deviation between two independent color temperature prediction values ​​(i.e., the first color temperature and the second color temperature). The proportion is the distribution weight of the first landing point in the total number of landing points in the image. Specifically, the absolute value of the difference between the first color temperature and the second color temperature is determined to obtain the color temperature difference value. A preset color temperature difference threshold is set to evaluate the dispersion of color distribution within the image; this color temperature difference threshold is exemplarily 500K. The color temperature difference value is compared with the color temperature difference threshold. If the color temperature difference value is greater than the color temperature difference threshold, the image is determined to belong to a special monochrome scene with very few reference colors (i.e., the proportion of reference colors in the image is less than the preset threshold), and the first gain parameter is determined as the white balance gain parameter. If the color temperature difference value is less than or equal to the color temperature difference threshold, the proportion of the first landing point relative to all landing points in the target coordinate system of the image is calculated. Based on this proportion The first gain parameter and the second gain parameter are linearly weighted to obtain the white balance gain parameter. The weighting calculation formula includes: Formula (11) Formula (12) in, The white balance gain parameter representing the red channel (R) The white balance gain parameter representing the blue channel (B) Representative percentage, and These represent the gain parameters of the red channel and the blue channel in the first gain parameter, respectively. and These represent the red channel gain parameter and the blue channel gain parameter in the second gain parameter, respectively.

[0091] By applying the above embodiments, adaptive determination of monochrome and multicolor scenes is achieved through color temperature difference logic. In extreme scenes where white point information (i.e. reference color information) is lacking, the accuracy of color reproduction is ensured through a mapping mechanism. At the same time, the weighted fusion mechanism ensures a smooth transition of parameter adjustment, eliminates visual abrupt changes that may be caused by algorithm strategy switching, and significantly enhances the robustness of automatic white balance in complex scenes.

[0092] Step 105: Perform white balance processing on the image based on the white balance gain parameter.

[0093] Step 105 is the execution step that uses the finally determined global color correction parameters (i.e., white balance gain parameters) to correct the pixel data of the image to be processed in order to restore the true colors. The white balance gain parameters are the final parameters used to adjust the values ​​of specific color channels in the image, obtained through fusion calculation or selection judgment. White balance processing is the process of applying the white balance gain parameters to the color components of each pixel in the image to eliminate the adverse effects of ambient light color temperature on the image color.

[0094] During white balance processing, the acquired white balance gain parameter is applied to all pixels in the image to be processed. The pixels of the image are adjusted based on the calculated white balance gain parameter. For example, the adjustment formula for the R, G, and B components in a RAW image is as follows: Formula (13) Formula (14) Formula (15) in, , , These are the color channel values ​​after white balance adjustment. , , These are the color channel values ​​before white balance adjustment. This refers to the white balance gain parameter for color channel R. This refers to the white balance gain parameter for color channel B. This is the reference gain coefficient, typically 1024.

[0095] See Figure 11 Taking blue sky as an example, the white balance processing method provided in this application includes: Step 301: Preprocessing stage. This includes: collecting gray card and blue sky color block data under various standard light sources, generating Planck curves based on this data, and determining white and blue sky areas; and pre-training a blue sky area recognition model.

[0096] Step 302: Acquire the image to be processed. The imaging device acquires a frame of the original image of the current scene that needs to be processed by automatic white balance.

[0097] Step 303: Image segmentation, calculation of the R / G / B average value for each segment, and removal of overexposed and underexposed segments. Specifically, the image to be processed is segmented (e.g., into 128×90 segments). The average values ​​of the R, G, and B channels of pixels in each segment are calculated, and invalid segments that are overexposed (too bright) or underexposed are removed. The coordinate points of the remaining segments are then calculated.

[0098] Step 304: Use the blue sky region recognition model to identify the image and obtain the mask aimask. Specifically, use the pre-trained blue sky region recognition model to segment the image into blue sky regions, with the segmentation grid consistent with the above block division (128×90). Blocks identified as blue sky regions are marked as 1, otherwise 0, and the visual mask aimask is output.

[0099] Step 305: Calculate the landing point and obtain the skymask by judging the blue sky area. Specifically, determine whether the image block landing point falls within the high color temperature blue sky area (such as within the D95, D65, D50 color temperature frame); calculate whether the brightness of the image block is less than a preset threshold (to distinguish bright white clouds); blocks that meet both of the above conditions are marked as 1, otherwise as 0, and output the visualization mask skymask.

[0100] Step 306: Merge skymask and aimask to obtain the mask. Specifically, perform a logical AND operation on aimask and skymask. Only image patches marked as 1 in both are ultimately identified as blue sky area image patches, and their corresponding landing points are defined as "blue sky landing points". The rest are "other landing points".

[0101] Step 307: Calculate the number of blue sky landing points and other landing points, and calculate the percentage w of blue sky landing points. Specifically, count the number of "blue sky landing points" and "other landing points" separately, and calculate the percentage of blue sky landing points to all valid landing points.

[0102] Step 308: Determine whether the percentage of blue sky landing points is greater than the percentage threshold. If yes, proceed to step 310; otherwise, proceed to step 309.

[0103] Step 309: Execute the standard white balance algorithm to obtain the gain. Specifically, if the proportion of blue sky does not meet the standard, it is considered that there is no risk of redness in the image. The average gain point is calculated directly for the entire image using traditional white area testing methods and other white balance algorithms to obtain the overall white balance gain parameters, and then proceed to step 316.

[0104] Step 310: Calculate the geometric mean of the blue sky landing point, find the closest point on the blue Planck line, and interpolate to obtain the color temperature. Specifically, the statistical average of all blue sky landing points is calculated to obtain the equivalent gain point of the blue sky; using the vector projection method, the distance from this equivalent gain point to each segment of the Planck line of the blue sky is calculated, the segment with the smallest distance is found, and linear interpolation is performed using the two endpoints of this segment to calculate the color temperature. .

[0105] Step 311: Obtain the color temperature by mapping the Planck curve to the white point. The white point gain is calculated as gain1. Specifically, it involves finding the value containing color temperature on the Planck line of the white point. The color temperature of the line segment is obtained through an interpolation algorithm. Coordinates of the gain point on the white Planck line The gain is then calculated. : , .

[0106] Step 312: Perform a white balance algorithm on other landing points to obtain the color temperature. and gain Specifically, after removing the blue sky landing points, the remaining "other landing points" are processed using a standard white balance algorithm to calculate their average landing point within the white area, and the corresponding color temperature is obtained accordingly. and the gain calculated independently (including) and ).

[0107] Step 313: Determine the difference If the difference is less than a threshold (e.g., 500K), proceed to step 315; otherwise, proceed to step 314. This prevents calculation errors caused by monochrome scenes with little or no white space.

[0108] Step 314: Calculate white balance gain .

[0109] Step 315: Calculate white balance gain .

[0110] Step 316: Obtain White Balance Gain .

[0111] Step 317: Adjust the white balance of the image.

[0112] Experimental verification, based on multiple collected images, calculated white balance data for different blue sky scenes and compared image effects, shows that the white balance method for blue sky scenes in this application can effectively reduce color cast issues, with smaller white balance deviations and more accurate color reproduction. Meanwhile, it has little or no impact on scenes with a small proportion of blue sky or non-blue sky scenes.

[0113] The white balance deviation is calculated as follows: a standard gray card is placed in the scene, and multiple images are taken. First, the R, G, and B channel values ​​of the gray card are made the same through manual white balance, and the white balance gain parameter is recorded at this time. , The white balance gain is considered to be the ideal gain. The effect is demonstrated by comparing the deviation between the gain obtained by the method in this embodiment and the ideal gain, and the deviation of other conventional white balance algorithms. The formula for calculating the white balance deviation is as follows: Formula (16) Formula (17) For example, the average deviation obtained after white balance adjustment using this embodiment and the average deviation obtained from other conventional white balance algorithms for different common blue sky scenes are shown in Table 1 below:

[0114] Table 1 For example, see Figure 12 In the figures, (a) shows the image effect obtained by other white balance methods, (b) shows the image effect obtained by the white balance method in this embodiment, and (c) shows the ideal image effect obtained by manual white balance. This embodiment can better restore the color effect of the image, especially for some scenes prone to color cast (large areas of blue sky scenes).

[0115] By applying the embodiments described above, and through the classification, extraction, and targeted quantization of pixel locations in different regions of an image, refined and differentiated white balance processing is achieved. By determining the first location corresponding to the target color and the second location corresponding to the remaining regions, the potential interference of specific color regions on global color statistics can be effectively eliminated. When the first location meets preset conditions, the final white balance gain parameter is determined by independently determining the first color temperature, first gain parameter, second color temperature, and second gain parameter, and combining these multi-dimensional parameters. This mechanism achieves adaptive fusion of the target color distribution and the conventional color distribution, effectively solving the problem of white balance parameter calculation deviation caused by the similarity between large areas of specific color regions and the reference color features. Finally, the image is corrected using the determined white balance gain parameter, significantly enhancing the robustness of the automatic white balance algorithm to complex scenes, effectively restoring the objective and true colors of the image, and improving the white balance processing effect.

[0116] The following description continues to illustrate the exemplary structure of the white balance processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the white balance processing device 555 in the memory 550 may include: a first determining module 5551, used to determine a first landing point and a second landing point of the image in the target coordinate system, wherein the first landing point corresponds to a first region of the target color in the image, and the second landing point corresponds to a second region in the image other than the first region; a second determining module 5552, used to determine a first color temperature based on the first landing point when the first landing point meets a preset condition, and to determine a first gain parameter based on the first color temperature; a third determining module 5553, used to determine a second color temperature and a second gain parameter based on the second landing point; a fourth determining module 5554, used to determine a white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; and a white balance module 5555, used to perform white balance processing on the image based on the white balance gain parameter.

[0117] In some embodiments, the second determining module 5552 is further configured to determine the first average coordinate of the landing point coordinates of the first landing point, and use the point indicated by the first average coordinate as the first color temperature reference point; determine the distance from the first color temperature reference point to each of the plurality of line segments included in the first color temperature trajectory line, and determine the first line segment corresponding to the smallest distance from the plurality of line segments, wherein the first color temperature trajectory line corresponds to the target color; and perform interpolation processing on the endpoint color temperatures of the two endpoints of the first line segment based on the first projection parameter of the first color temperature reference point on the first line segment to obtain the first color temperature.

[0118] In some embodiments, the second determining module 5552 is further configured to map the first color temperature to a second color temperature trajectory line of a reference color, and determine the position coordinates of the first color temperature at the mapping position of the second color temperature trajectory line; obtain a reference gain coefficient, and determine the first gain parameter based on the position coordinates and the reference gain coefficient.

[0119] In some embodiments, the second determining module 5552 is further configured to determine a second line segment including the first color temperature from the plurality of line segments included in the second color temperature trajectory line; determine interpolation coefficients based on the endpoint color temperatures of the two endpoints of the second line segment and the first color temperature; and perform interpolation processing on the endpoint coordinates of the two endpoints of the second line segment based on the interpolation coefficients to obtain the position coordinates.

[0120] In some embodiments, the third determining module 5553 is further configured to: determine the second average coordinates of the landing point coordinates of the second landing point located in the color temperature zone of the reference color; use the point indicated by the second average coordinates as the second color temperature reference point, and determine the distances from the second color temperature reference point to each of the plurality of line segments included in the second color temperature trajectory line, and determine the third line segment corresponding to the smallest distance from the plurality of line segments, wherein the second color temperature trajectory line corresponds to the reference color; perform interpolation processing on the endpoint color temperatures of the two endpoints of the third line segment based on the first projection parameter of the second color temperature reference point on the third line segment to obtain the second color temperature; obtain the reference gain coefficient, and determine the second gain parameter based on the second average coordinates and the reference gain coefficient.

[0121] In some embodiments, the fourth determining module 5554 is further configured to determine the color temperature difference between the first color temperature and the second color temperature; if the color temperature difference is greater than a color temperature difference threshold, the first gain parameter is used as the white balance gain parameter; if the color temperature difference is less than or equal to the color temperature difference threshold, the first gain parameter and the second gain parameter are weighted based on the proportion of the first landing point relative to all landing points of the image in the target coordinate system to obtain the white balance gain parameter.

[0122] In some embodiments, the first determining module 5551 is further configured to perform block processing on the image to obtain a plurality of image blocks; for each image block, determine the average color channel value of the image block in each color channel, and determine the landing point of the image block in the target coordinate system based on the average color channel value of each color channel; identify a first region of the target color in the image, and take the region in the image other than the first region as a second region; take the landing point of the image block belonging to the first region as the first landing point, and take the landing point of the image block belonging to the second region as the second landing point.

[0123] In some embodiments, the first determining module 5551 is further configured to identify a target scene in the image using a neural network model, obtain a first mask for the plurality of image blocks, wherein the target scene has the target color; determine a target color temperature frame for the target color in the target coordinate system for each landing point, and determine the brightness of each image block; generate a second mask for the plurality of image blocks based on the target color temperature frame and the brightness; and determine the first region based on the first mask and the second mask.

[0124] In some embodiments, the first determining module 5551 is further configured to, before determining the first landing point and the second landing point of the image in the target coordinate system, acquire, for each of the plurality of standard color temperatures, reference images obtained by a plurality of shooting devices respectively shooting a reference object at the standard color temperature, and determine the statistical points of the plurality of reference images shot at the standard color temperature in the target coordinate system; for each standard color temperature, determine the minimum convex hull geometric region of the plurality of statistical points corresponding to the standard color temperature, and generate the closed polygon boundary corresponding to the standard color temperature based on the minimum convex hull geometric region; for each standard color temperature, from the plurality of statistical points corresponding to the standard color temperature... Among the statistical points, a reference statistical point corresponding to the reference shooting device is determined, and the average coordinate value of the reference statistical points is determined. The point indicated by the average coordinate value is taken as the center point corresponding to the standard color temperature. The plurality of shooting devices includes the reference shooting device. When the reference object is a color block of the target color, each closed polygon boundary is determined as the color temperature zone of the target color, and the plurality of center points are connected to obtain the first color temperature trajectory line of the target color. When the reference object is a test gray card, each closed polygon boundary is determined as the color temperature zone of the reference color, and the plurality of center points are connected to obtain the second color temperature trajectory line of the reference color.

[0125] It should be noted that the description of the device embodiments in this application is similar to the description of the method embodiments described above, and has similar beneficial effects as the method embodiments, so it will not be repeated here. Any technical details not covered in the white balance processing device provided in the embodiments of this application can be understood based on the description of the technical details in the above method embodiments.

[0126] This application also provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the white balance processing method provided in this application.

[0127] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the white balance processing method provided in this application.

[0128] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0129] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0130] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0131] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0132] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A white balance processing method, characterized in that, The method includes: Determine the first and second landing points of the image in the target coordinate system. The first landing point corresponds to the first region of the target color in the image, and the second landing point corresponds to the second region in the image other than the first region. If the first landing point meets the preset conditions, a first color temperature is determined based on the first landing point, and a first gain parameter is determined based on the first color temperature; Based on the second landing point, determine the second color temperature and the second gain parameter; The white balance gain parameter is determined based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; The image is subjected to white balance processing based on the white balance gain parameters.

2. The method as described in claim 1, characterized in that, Determining the first color temperature based on the first landing point includes: Determine the first average coordinate of the first landing point, and use the point indicated by the first average coordinate as the first color temperature reference point; Determine the distances of each of the multiple line segments from the first color temperature reference point to the first color temperature trajectory line, and determine the first line segment corresponding to the smallest distance from the multiple line segments, wherein the first color temperature trajectory line corresponds to the target color; Based on the first projection parameters of the first color temperature reference point on the first line segment, the endpoint color temperatures of the two endpoints of the first line segment are interpolated to obtain the first color temperature.

3. The method as described in claim 1, characterized in that, The step of determining the first gain parameter based on the first color temperature includes: Map the first color temperature to the second color temperature trajectory line of the reference color, and determine the position coordinates of the first color temperature at the mapping position on the second color temperature trajectory line; Obtain the reference gain coefficient, and determine the first gain parameter based on the position coordinates and the reference gain coefficient.

4. The method as described in claim 3, characterized in that, Determining the position coordinates of the first color temperature on the mapping position of the second color temperature trajectory line includes: From the multiple line segments included in the second color temperature trajectory line, determine the second line segment that includes the first color temperature; Based on the endpoint color temperature of the two endpoints of the second line segment and the first color temperature, the interpolation coefficients are determined; Based on the interpolation coefficients, the endpoint coordinates of the two endpoints of the second line segment are interpolated to obtain the position coordinates.

5. The method as described in claim 1, characterized in that, The step of determining the white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter includes: Determine the color temperature difference between the first color temperature and the second color temperature; If the color temperature difference is greater than the color temperature difference threshold, the first gain parameter is used as the white balance gain parameter. When the color temperature difference is less than or equal to the color temperature difference threshold, the first gain parameter and the second gain parameter are weighted based on the proportion of the first landing point relative to all landing points of the image in the target coordinate system to obtain the white balance gain parameter.

6. The method as described in claim 1, characterized in that, Determining the first and second landing points of the image in the target coordinate system includes: The image is divided into blocks to obtain multiple image blocks; For each image patch, the average color channel value of the image patch in each color channel is determined, and based on the average color channel value, the landing point of the image patch in the target coordinate system is determined; Identify a first region of the target color within the image, and designate the region within the image other than the first region as a second region; The landing point of the image block belonging to the first region is taken as the first landing point, and the landing point of the image block belonging to the second region is taken as the second landing point.

7. The method as described in claim 6, characterized in that, The identification of the first region of the target color within the image includes: The target scene is identified in the image by a neural network model to obtain a first mask for the plurality of image blocks, wherein the target scene has the target color; Determine the target color temperature frame of the target color in the target coordinate system for each of the aforementioned points, and determine the brightness of each of the aforementioned image blocks; Based on the target color temperature frame and the brightness, a second mask is generated for the plurality of image blocks; The first region is determined based on the first mask and the second mask.

8. The method according to any one of claims 1-7, characterized in that, Before determining the first and second landing points of the image in the target coordinate system, the method further includes: For each of the multiple standard color temperatures, acquire reference images of the reference object captured by multiple shooting devices under the standard color temperature, and determine the statistical points of the multiple reference images captured under the standard color temperature in the target coordinate system. For each standard color temperature, determine the minimum convex hull geometric region of multiple statistical points corresponding to the standard color temperature, and generate the closed polygon boundary corresponding to the standard color temperature based on the minimum convex hull geometric region. For each standard color temperature, a reference statistical point corresponding to a reference shooting device is determined from multiple statistical points corresponding to the standard color temperature, and the average coordinate value of the reference statistical points is determined. The point indicated by the average coordinate value is taken as the center point corresponding to the standard color temperature. The multiple shooting devices include the reference shooting device. When the reference object is a color block of the target color, each closed polygon boundary is determined as the color temperature zone of the target color, and multiple center points are connected to obtain the first color temperature trajectory line of the target color. When the reference object is a test gray card, each closed polygon boundary is determined as the color temperature zone of the reference color, and multiple center points are connected to obtain the second color temperature trajectory line of the reference color.

9. A white balance processing device, characterized in that, The device includes: The first determining module is used to determine the first landing point and the second landing point of the image in the target coordinate system. The first landing point corresponds to the first region of the target color in the image, and the second landing point corresponds to the second region in the image other than the first region. The second determining module is used to determine a first color temperature based on the first landing point and a first gain parameter based on the first color temperature, provided that the first landing point meets the preset conditions. The third determining module is used to determine the second color temperature and the second gain parameter based on the second landing point; The fourth determining module is used to determine the white balance gain parameter based on the first color temperature, the second color temperature, the first gain parameter, and the second gain parameter; The white balance module is used to perform white balance processing on the image based on the white balance gain parameters.

10. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the white balance processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, the white balance processing method according to any one of claims 1 to 8 is implemented.

Images