Electronic device and white balance adjusting method thereof

TWI945672BActive Publication Date: 2026-11-11ASUSTEK COMPUTER INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
TW114101642
Authority / Receiving Office
TW · TW
Patent Type
Patents
Current Assignee / Owner
Filing Date
2025-01-15
Publication Date
2026-11-11
Estimated Expiration
2045-01-14

AI Technical Summary

Technical Problem

Traditional automatic white balance (AWB) methods in camera systems fail to accurately estimate color temperature changes caused by flash, leading to color casts and inconsistent color temperatures in images, degrading image quality.

Method used

A method involving a pre-flash image capture, an ambient light image capture, and a final white balance decision point determination based on these images, along with a processor to adjust the white balance of the target image using a final white balance decision point.

Benefits of technology

This method effectively avoids color casts by accurately adjusting white balance, ensuring good image quality even under varying lighting conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure TWG2TB001920424_001
    Figure TWG2TB001920424_001
  • Figure TWG2TB001920424_002
    Figure TWG2TB001920424_002
  • Figure TWG2TB001920424_003
    Figure TWG2TB001920424_003
Patent Text Reader

Abstract

This case discloses an electronic device and its white balance adjustment method. The method includes the following steps: emitting a pre-flash using a flash unit and capturing a pre-flash image. Determining a first white balance decision point based on the pre-flash image. Turning off the flash unit and capturing an ambient light image using an image sensor. Determining a second white balance decision point based on the ambient light image. Determining a final white balance decision point based on a third white balance decision point corresponding to the main flash of the flash unit, the first white balance decision point, and the second white balance decision point. emitting a main flash using the flash unit and capturing a target image, and applying the final white balance decision point to adjust the white balance of the target image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to an electronic device and its white balance adjustment method. Prior Technology

[0002] With the advancement of technology, electronic devices equipped with camera systems have become ubiquitous in modern life. Automatic White Balance (AWB) algorithms are a key function in camera systems, used to automatically adjust the white balance of images to make photos appear more natural, especially under different lighting conditions. This is achieved by detecting color deviations in the image and then adjusting the gain of the RGB channels to balance color distortion under different light sources. However, when shooting in different lighting environments, traditional AWB methods often fail to accurately estimate the color temperature changes caused by flash, leading to color casts or inconsistent color temperatures in the images. These problems can degrade image quality and negatively impact the end-user experience. Summary of the Invention

[0003] This invention provides a white balance adjustment method for an electronic device including an image sensor and a flash unit. The white balance adjustment method includes the following steps: emitting a pre-flash using the flash unit and capturing a pre-flash image; determining a first white balance decision point based on the pre-flash image; turning off the flash unit and capturing an ambient light image using the image sensor; determining a second white balance decision point based on the ambient light image; determining a final white balance decision point based on a third white balance decision point corresponding to the main flash of the flash unit, the first white balance decision point, and the second white balance decision point; emitting a main flash using the flash unit and capturing a target image, and applying the final white balance decision point to adjust the white balance of the target image.

[0004] This application also provides an electronic device including an image sensor, a flash unit, and a processor. The processor is coupled to the image sensor and the flash unit. The processor is configured to perform the following steps: emitting a pre-flash using the flash unit and capturing a pre-flash image; determining a first white balance decision point based on the pre-flash image; turning off the flash unit and capturing an ambient light image using the image sensor; determining a second white balance decision point based on the ambient light image; determining a final white balance decision point based on a third white balance decision point corresponding to the main flash of the flash unit, the first white balance decision point, and the second white balance decision point; emitting a main flash using the flash unit and capturing a target image, and applying the final white balance decision point to adjust the white balance of the target image.

[0005] Based on the above, in this disclosed embodiment, a pre-flash image and an ambient light image can be captured with the pre-flash device enabled and the flash device disabled, respectively. A first white balance decision point and a second white balance decision point are then obtained based on the pre-flash image and the ambient light image, respectively. Furthermore, the final white balance decision point can be determined based on the third white balance decision point, the first white balance decision point, and the second white balance decision point corresponding to the main flash. The final white balance decision point is then used to adjust the white balance of the target image captured with the main flash enabled. Therefore, even if the color temperature or intensity of the pre-flash differs from that of the main flash, this disclosure avoids color cast caused by shooting with the flash enabled and ensures good white balance adjustment results. Simple Explanation of the Diagram

[0006] Figure 1 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. Figure 2 is a flowchart illustrating a white balance adjustment method according to an embodiment of the present disclosure. Figure 3 is a schematic diagram illustrating a white balance adjustment method according to an embodiment of the present disclosure. Figure 4 is a flowchart illustrating the determination of the final white balance decision point according to an embodiment of the present disclosure. Figure 5 is a schematic diagram illustrating the determination of the final white balance decision point according to an embodiment of the present disclosure. Implementation

[0007] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component symbols are used in the drawings and description to denote the same or similar parts. These embodiments are only a part of the present disclosure and do not reveal all possible implementations of the present disclosure. Rather, these embodiments are merely examples of apparatuses and methods within the scope of the patent application disclosed herein.

[0008] Referring to Figure 1, the electronic device 100 may include a display 110, an image sensor 120, a storage device 130, a flash device 140, and a processor 150. In different embodiments, the electronic device 100 may be, for example, a laptop computer, a tablet computer, a game console, a mobile phone, or other electronic devices with image capturing capabilities, and this disclosure is not limited thereto.

[0009] The display 110 can be any type of display, such as a liquid crystal display (LCD), a light-emitting diode (LED) display, or an organic light-emitting diode (OLED), and this disclosure is not limited thereto. The display 110 can be used to display the program operation interface of a camera application, preview images, or captured images, etc.

[0010] Image sensor 120 is used to provide image sensing functionality. Image sensor 120 may include a photosensitive element, such as a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) element, or other elements, which are not limited herein. Image sensor 120 may be combined with other elements (such as a lens, aperture, etc.) to form a camera module of electronic device 100.

[0011] Storage device 130 can be used to store data, images, instructions, code, software modules, etc., and can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices, integrated circuits, and combinations thereof.

[0012] Flash unit 140 can be used to emit flashes to provide additional illumination during image capture. Flash unit 140 may include one or more flash units, and these flash units may have the same or different color temperatures. In the embodiment disclosed herein, flash unit 140 can emit a main flash and a pre-flash. The intensity of the main flash is higher than that of the pre-flash. Flash unit 140 can emit a pre-flash before actual shooting; for example, one of the flash units in flash unit 140 can flash briefly to help the camera meter the light. Through the pre-flash generated by flash unit 140, various shooting parameters, such as exposure parameters, can be determined based on ambient light and the reflection of the subject. Flash unit 140 can emit a main flash during actual shooting, which is used to enhance the brightness of the subject.

[0013] Processor 150 is coupled to display 110, image sensor 120, storage device 130, and flash device 140. It is, for example, a central processing unit (CPU), application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU), or other similar devices, integrated circuits, or combinations thereof. In some embodiments, processor 150 may execute instructions or program code in storage device 130 to implement the steps of the white balance adjustment method in this disclosed embodiment.

[0014] Figure 2 is a flowchart illustrating a white balance adjustment method according to an embodiment of the present disclosure. Referring to Figure 2, the method of this embodiment can be executed by the electronic device 100 of Figure 1. The details of each step in Figure 2 are explained below with reference to the components shown in Figure 1.

[0015] In step S210, the processor 150 emits a pre-flash using the flash unit 140 and captures the pre-flash image through the image sensor 120. That is, when the flash unit 140 emits a pre-flash to illuminate the shooting scene, the image sensor 120 captures the pre-flash image.

[0016] In step S220, the processor 150 determines a first white balance decision point based on the pre-flash image. Specifically, the processor 150 can determine the scene color temperature in a multi-light source (pre-flash and ambient light) scene based on the pre-flash image. In this disclosed embodiment, the processor 150 can execute an automatic white balance (AWB) algorithm to determine a first white balance decision point in a target plane based on the pre-flash image. The determination of the first white balance decision point can be considered as scene color temperature information estimated by the automatic white balance algorithm, and can be used to determine the color channel gain for white balance adjustment.

[0017] In some embodiments, the first white balance decision point may include two coordinate components, namely the red-green channel ratio (RG ratio) and the blue-green channel ratio (BG ratio). In some embodiments, the processor 150 may acquire multiple color channel values ​​of multiple pixels in the pre-flash image. The processor 150 determines the first white balance decision point in the coordinate plane based on the red-green channel ratio of the red channel values ​​and the blue-green channel ratio of the blue channel values ​​and the green channel values ​​of each pixel. In some embodiments, these pixels may be multiple white points detected based on white point detection using an automatic white balance algorithm.

[0018] In some embodiments, the processor 150 may calculate multiple candidate white balance decision points based on multiple white points in the pre-flash image, and converge these candidate white balance decision points to a first white balance decision point in the coordinate plane. For example, the processor 150 may calculate multiple red-green channel ratios for the multiple white points and determine that the average of these red-green channel ratios is the red-green channel ratio of the first white balance decision point. Furthermore, the processor 150 may calculate multiple blue-green channel ratios for the multiple white points and determine that the average of these blue-green channel ratios is the blue-green channel ratio of the first white balance decision point. Alternatively, the processor 150 may determine the first white balance decision point corresponding to the pre-flash image based on multiple candidate white balance decision points using other convergence algorithms.

[0019] In step S230, the processor 150 turns off the flash unit 140 and captures an ambient light image through the image sensor 120. That is, the image sensor 120 captures an ambient light image even when the flash unit 140 does not emit any flash to illuminate the shooting scene. In some embodiments, the ambient light image and the aforementioned pre-flash image may be images captured by the image sensor 120 in metering mode or preview mode.

[0020] In step S240, processor 150 determines a second white balance decision point based on the ambient light image. Specifically, processor 150 can determine the scene color temperature when the flash device 140 is turned off based on the ambient light image. In this disclosed embodiment, processor 150 can execute an automatic white balance (AWB) algorithm to determine a second white balance decision point in a target plane based on the ambient light image. The determination of the second white balance decision point can be considered as scene color temperature information estimated by the automatic white balance algorithm, and can be used to determine the color channel gain for white balance adjustment. It should be noted that the way processor 150 calculates the second white balance decision point based on the ambient light image is similar to the way it calculates the first white balance decision point based on the pre-flash image. That is, processor 150 can determine the second white balance decision point based on the proportions of multiple red-green channels and multiple blue-green channels of multiple white points in the ambient light image.

[0021] In step S250, the processor 150 determines a final white balance decision point based on the third white balance decision point, the first white balance decision point, and the second white balance decision point corresponding to the main flash of the flash unit 140. In this disclosure, the third white balance decision point corresponding to the main flash of the flash unit 140 can be determined through pre-testing and recorded in the storage device 130. In different embodiments, the processor 150 can determine the final white balance decision point based on the first white balance decision point, the second white balance decision point, and the third white balance decision point through table lookup or function calculation.

[0022] From another perspective, when the actual shooting environment includes ambient light and the main flash of the flash unit 140, the processor 150 can estimate the scene color temperature information for activating the main flash based on the color temperature information of the main flash and the ambient light, thereby determining a more ideal color channel gain. The third white balance decision point corresponding to the main flash can represent the color temperature information of the main flash, which can be obtained and recorded in the storage device 130 based on pre-testing. The second white balance decision point can represent the color temperature information of the ambient light, which can be estimated based on the real-time captured ambient light image. In addition, by referring to the color temperature information of the pre-flash shooting scene, this disclosure can make the color temperature estimation of the actual shooting scene with the main flash activated closer to the real situation, thereby determining a color channel gain that is closer to actual needs.

[0023] In step S260, the processor 150 uses the flash unit 140 to emit a main flash and the image sensor 120 to capture the target image, and applies a final white balance decision point to perform white balance operation on the target image. Specifically, after determining the final white balance decision point, the processor 150 can respond to the shutter signal by controlling the flash unit 140 to emit a main flash and driving the image sensor 120 to capture the target image. Furthermore, the processor 150 can use the final white balance decision point to determine multiple color channel gains, and use these color channel gains to adjust the color channel values ​​of each pixel in the target image to achieve white balance adjustment. For example, the processor 150 can use the final white balance decision point to determine the red channel gain, and use this red channel gain to adjust the red channel values ​​of each pixel in the target image.

[0024] Please refer to Figure 3, which is a schematic diagram illustrating a white balance adjustment method according to an embodiment of this disclosure. In some embodiments, the third white balance decision point associated with the main flash can be determined through pre-testing. In a dark test environment, the processor 150 can activate the main flash and capture a flash test image Img_MF. Specifically, in some embodiments, in a dark environment, the processor 150 uses the flash unit 140 to emit a main flash and captures a flash test image Img_MF through the image sensor 120. The processor 150 determines the third white balance decision point WP3 based on the flash test image Img_MF. That is, in operation 313, the processor 150 can execute the AWB algorithm based on the flash test image Img_MF to estimate the color temperature information of the main flash, thereby obtaining the third white balance decision point WP3.

[0025] In an actual shooting environment, processor 150 can activate the pre-flash to capture a pre-flash image Img_PF, and deactivate the flash unit 140 to capture an ambient light image Img_S. In operation 311, processor 150 can execute the AWB algorithm based on the pre-flash image Img_PF to obtain a first white balance decision point WP1. In operation 312, processor 150 can execute the AWB algorithm based on the ambient light image Img_S to obtain a second white balance decision point WP2.

[0026] In operation 314, processor 150 determines the final white balance decision point WP4 based on the first white balance decision point WP1, the second white balance decision point WP2, and the third white balance decision point WP3. That is, processor 150 determines a final white balance decision coordinate based on three white balance decision coordinates on the coordinate plane. Therefore, in response to receiving the shutter signal, processor 150 can use the flash unit 140 to emit a main flash and capture a target image Img_T1 through the image sensor 120. In operation 315, processor 150 can adjust the white balance of the target image Img_T1 based on the final white balance decision point WP4 to obtain the final target image Img_T2.

[0027] Please refer to Figure 4, which is a flowchart illustrating the determination of the final white balance decision point according to an embodiment of this disclosure. In some embodiments, at step S251, the processor 150 may obtain the connection between the third white balance decision point and the second white balance decision point. At step S252, the processor 150 determines the final white balance decision point located on the connection based on the first white balance decision point. That is, assuming the final white balance decision point is located between the third white balance decision point associated with the main flash and the second white balance decision point associated with the ambient light, the processor 150 may search for the final white balance decision point on the connection between the third white balance decision point and the second white balance decision point.

[0028] In some embodiments, step S252 may be implemented as steps S2521 to S2522. In step S2521, processor 150 calculates the reference distance between the first white balance decision point and the second white balance decision point. In step S2522, processor 150 obtains the final white balance decision point on the connecting line based on the reference distance. That is, processor 150 can refer to the position of the first white balance decision point to find the final white balance decision point on the connecting line between the third white balance decision point and the second white balance decision point.

[0029] For example, referring to Figure 5, through pre-testing, processor 150 can determine the third white balance decision point B on the coordinate plane associated with the main flash. In the actual shooting environment, processor 150 can obtain the second white balance decision point A on the coordinate plane by turning off the flash, and can obtain the first white balance decision point C on the coordinate plane by turning on the pre-flash. The horizontal axis of the coordinate plane represents the ratio of the red channel value to the green channel value (R / G), and the vertical axis of the coordinate plane represents the ratio of the blue channel value to the green channel value (B / G).

[0030] Next, processor 150 acquires the line L51 connecting the third white balance decision point B and the second white balance decision point A. Processor 150 calculates the reference distance Δd between the first white balance decision point C and the second white balance decision point A. Based on the reference distance Δd, processor 150 acquires the final white balance decision point E on line L51. In some embodiments, the distance between the final white balance decision point E and the second white balance decision point A is equal to the product of an adjustable parameter and the reference distance Δd.

[0031] In some embodiments, the processor 150 may obtain a reference point D on the connecting line L51 based on a reference distance Δd. The distance d1 between the reference point D and the second white balance decision point A is equal to the reference distance Δd. The processor 150 may determine the final white balance decision point E located on the connecting line L51 based on the reference point D and an adjustable parameter.

[0032] Please refer to Figure 5. The X-axis of the coordinate plane represents the ratio of the red channel value to the green channel value (R / G), and the Y-axis represents the ratio of the blue channel value to the green channel value (B / G). Assume the coordinates of the first white balance decision point C are ( The coordinates of the second white balance decision point A are ( The coordinates of the third white balance decision point B are ( Processor 150 can be used in line segments. Find a reference point D on the top, such that The coordinates of reference point D ( ) can be represented by the following formulas (1) and (2). Formula (1) Formula (2) Where t represents a proportion.

[0033] based on The processor 150 can calculate t as shown in formula (1). Formula (3)

[0034] Therefore, the processor 150 can obtain the coordinates of the final white balance decision point E according to the following formulas (4) and (5). Yes. Here, k represents an adjustable parameter, which is an integer greater than 0. Then, the processor 150 can determine the coordinates of the final white balance decision point E (…). The color channel gain applied to each color channel is determined by the final white balance decision point E. This process of determining the color channel gain for each color channel based on the final white balance decision point E can be accomplished using various automatic white balance algorithms without specific limitations. Formula (4) Formula (5)

[0035] In summary, in the embodiments disclosed herein, the final white balance decision point can be determined based on the third white balance decision point of the main flash decision, the first white balance decision point of the pre-flash decision, and the second white balance decision point of the ambient light decision. This final white balance decision point is then used to adjust the white balance of the target image captured with the main flash activated. Therefore, even if the color temperature or intensity of the pre-flash differs from that of the main flash, this disclosure avoids color cast caused by using the flash and ensures good white balance adjustment. Furthermore, compared to directly using the first white balance decision point corresponding to the pre-flash to adjust the white balance of the target image captured with the main flash, this disclosed method comprehensively considers the white balance decision results under different lighting conditions and selects the final white balance decision point for white balance adjustment, effectively avoiding color cast.

[0036] Although the invention has been described above with reference to embodiments, it is not intended to limit the scope of the invention. Anyone skilled in the art may make some modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the invention shall be determined by the appended claims.

[0037] 100: Electronic devices 110: Monitor 120: Image Sensor 130: Storage device 140: Flash unit 150: Processor 311~315: Operation Img_MF: Flash test image Img_PF: Preflash image Img_S: Ambient Light Image Img_T1: Target Image Img_T2: Final target image L51: Connection S210, S220, S230, S240, S250~S252, S260, S2521, S2522: Steps WP1: First White Balance Decision Point WP2: Second White Balance Decision Point WP3: Third White Balance Decision Point WP4: Final White Balance Decision Point

Claims

1. A white balance adjustment method for an electronic device including an image sensor and a flash unit, the method comprising: The steps include: emitting a pre-flash using the flash unit and capturing a pre-flash image; determining a first white balance decision point based on the pre-flash image; turning off the flash unit and capturing an ambient light image using the image sensor; determining a second white balance decision point based on the ambient light image; and determining a final white balance decision point based on a third white balance decision point corresponding to the main flash of the flash unit, the first white balance decision point, and the second white balance decision point. The steps also include: obtaining a line connecting the third white balance decision point and the second white balance decision point; determining the final white balance decision point located on the line based on the first white balance decision point; emitting the main flash using the flash unit and capturing a target image, and applying the final white balance decision point to adjust the white balance of the target image.

2. The white balance adjustment method as described in claim 1, wherein the step of determining the final white balance decision point located on the connecting line based on the first white balance decision point includes: Calculate the reference distance between the first white balance decision point and the second white balance decision point; And based on the reference distance, obtain the final white balance decision point on the connecting line.

3. The white balance adjustment method as described in claim 2, wherein the step of obtaining the final white balance decision point on the connecting line based on the reference distance includes: Based on the reference distance, a reference point is obtained on the connecting line, wherein the distance between the reference point and the second white balance decision point is equal to the reference distance; And based on the reference point and an adjustable parameter, the final white balance decision point located on the connecting line is determined.

4. The white balance adjustment method as described in claim 2, wherein the distance between the final white balance decision point and the second white balance decision point is equal to the product of an adjustable parameter and the reference distance.

5. The white balance adjustment method as described in claim 1 further includes: In a dark environment, the main flash is emitted using the flash device, and a flash test image is captured. And determine the third white balance decision point based on the flash test image.

6. The white balance adjustment method as described in claim 1, wherein the step of determining the first white balance decision point based on the pre-flash image includes: Obtain multiple color channel values ​​of multiple pixels in the pre-flash image; And based on the red-green channel ratio of the red channel value and the green channel value of each of the plurality of pixels and the blue-green channel ratio of the blue channel value and the green channel value of each of the plurality of pixels, the first white balance decision point in a landmark plane is determined.

7. An electronic device comprising: An image sensor; A flash unit; The system also includes a processor coupled to the image sensor and the flash unit, configured to: emit a pre-flash using the flash unit and capture a pre-flash image; determine a first white balance decision point based on the pre-flash image; turn off the flash unit and capture an ambient light image using the image sensor; determine a second white balance decision point based on the ambient light image; and determine a final white balance decision point based on a third white balance decision point corresponding to the main flash of the flash unit, the first white balance decision point, and the second white balance decision point, wherein the steps include: obtaining a line connecting the third white balance decision point and the second white balance decision point; Based on the first white balance decision point, determine the final white balance decision point located on the connecting line; and use the flash device to emit the main flash and capture a target image, and apply the final white balance decision point to adjust the white balance of the target image.

8. The electronic device of claim 7, wherein the processor is further configured to: calculate a reference distance between the first white balance decision point and the second white balance decision point; and obtain the final white balance decision point on the line connecting the two points based on the reference distance.

9. The electronic device of claim 8, wherein the processor is further configured to: obtain a reference point on the line based on the reference distance, wherein the distance between the reference point and the second white balance decision point is equal to the reference distance; and determine the final white balance decision point located on the line based on the reference point and an adjustable parameter.

10. The electronic device of claim 8, wherein the distance between the final white balance decision point and the second white balance decision point is equal to the product of an adjustable parameter and the reference distance.

11. The electronic device of claim 7, wherein the processor is further configured to: emit the main flash using the flash device in a dark environment and capture a flash test image; and determine the third white balance decision point based on the flash test image.

12. The electronic device of claim 7, wherein the processor is further configured to: acquire multiple color channel values ​​of multiple pixels in the pre-flash image; and determine a first white balance decision point in a landmark plane based on the red-green channel ratio of the red channel values ​​and green channel values ​​of each of the multiple pixels and the blue-green channel ratio of the blue channel values ​​and green channel values ​​of each of the multiple pixels.

Citation Information

Patent Citations

  • An image processing method, a terminal and a computer readable storage medium

    CN108093233A

  • Single flash lamp compensation method and device based on white balance, equipment and medium

    CN110944116A

  • Image capture methods and systems

    US20080094484A1