Electronic device and method for reducing infrared light interference thereof

Abstract

An electronic device and a method for reducing infrared light interference thereof. The method includes the following steps. Detecting ambient light information by using an image sensor, wherein the image sensor includes a plurality of sensing pixels for sensing visible light and infrared light. Determining infrared light interference intensity according to the ambient light information. Controlling the image sensor to perform image sensing according to one of an original exposure time and a new exposure time according to the infrared light interference intensity. In response to the infrared light interference intensity being not greater than a threshold value, controlling the image sensor to perform image sensing according to the original exposure time. In response to the infrared light interference intensity being greater than the threshold value, controlling the image sensor to perform image sensing according to the new exposure time, wherein the new exposure time is shorter than the original exposure time.

Description

Technical Field

[0001] This invention relates to an electronic device, and more particularly to an electronic device and a method for reducing infrared light interference. Background Technology

[0002] Image sensors have traditionally focused on sensing visible light, thus incorporating an infrared cut filter to filter out infrared light and prevent distortion of the visible light sensed by the photosensitive pixels. In recent years, the photographic functions of various consumer electronics products have become increasingly diverse, and users' demands for nighttime shooting and infrared image capture have gained importance. Therefore, a hybrid image sensor that simultaneously senses visible and infrared light has been proposed to meet these needs. Currently, hybrid image sensors are widely used, primarily because they reduce installation space requirements and lower costs, leading to their increasing adoption. However, when using a hybrid image sensor, the infrared cut filter must be removed or a filter that allows some infrared light to pass through must be used so that the infrared sensing pixels of the hybrid image sensor can detect infrared light. As a result, the visible light sensing pixels of the hybrid image sensor also receive infrared light, causing color shift in the sensed image.

[0003] Figure 1 This is the spectral response diagram of a hybrid image sensor. Please refer to... Figure 1 According to the response curves L1-L3 of the RGB channels, the RGB channels sense both visible light (wavelength 400nm-750nm) and infrared light. Similarly, according to the infrared light response curve L4, the infrared (IR) channel senses both infrared and visible light. Currently, one correction method involves an image signal processor (ISP) responsible for correcting the RGB channel sensing results. The ISP subtracts a certain amount of IR signal from the RGB channel sensing signal. For example, the 12-bit data generated by the digitized RGB channel sensing signal is directly subtracted from a preset level (e.g., 300), and then the subtraction result is mapped back to 8-bit data. However, this method can only roughly offset the influence of infrared light and cannot adjust for the actual impact of infrared light, thus affecting image quality. In other words, even if the infrared component of ambient light is very low, the entire sensed image will still lose a certain level of digital signal, thus adversely affecting the sensed image. Summary of the Invention

[0004] In view of this, the present invention proposes an electronic device and a method for reducing infrared light interference, which can reduce the adverse effects of infrared light on RGB images.

[0005] This invention provides a method for reducing infrared light interference, comprising the following steps: Detecting ambient light information using an image sensor, wherein the image sensor includes multiple sensing pixels for sensing visible light and infrared light. Determining the intensity of infrared light interference based on the ambient light information. Controlling the image sensor to perform image sensing based on either an original exposure time or a new exposure time based on the intensity of infrared light interference. The step of controlling the image sensor to perform image sensing based on either an original exposure time or a new exposure time based on the intensity of infrared light interference includes: responding to an infrared light interference intensity not exceeding a threshold value, controlling the image sensor to perform image sensing based on the original exposure time; and responding to an infrared light interference intensity exceeding a threshold value, controlling the image sensor to perform image sensing based on a new exposure time, wherein the new exposure time is shorter than the original exposure time.

[0006] This invention provides an electronic device including an image sensor, a storage device, and a processor. The image sensor includes a plurality of sensing pixels for sensing visible light and infrared light. The processor is coupled to the image sensor and the storage device and configured to perform the following steps: Detecting ambient light information using the image sensor. Determining the intensity of infrared light interference based on the ambient light information. Controlling the image sensor to perform image sensing based on either an original exposure time or a new exposure time based on the intensity of infrared light interference. If the intensity of infrared light interference is not greater than a threshold value, controlling the image sensor to perform image sensing based on the original exposure time. If the intensity of infrared light interference is greater than a threshold value, controlling the image sensor to perform image sensing based on a new exposure time, wherein the new exposure time is shorter than the original exposure time.

[0007] Based on the above, in embodiments of the present invention, the image sensor includes multiple sensing pixels for sensing visible light and infrared light, thereby generating RGB images and infrared images. Ambient light information of the environment in which the electronic device is located can be detected first, thereby determining the intensity of infrared light interference. Then, the exposure time of the image sensor is adjusted according to the intensity of infrared light interference. This reduces the adverse effects of infrared light on the RGB image. Attached Figure Description

[0008] Figure 1 This is the spectral response diagram of a hybrid image sensor.

[0009] Figure 2 This is a block diagram of an electronic device according to an embodiment of the present invention.

[0010] Figure 3 This is a schematic diagram of an image sensor according to an embodiment of the present invention.

[0011] Figure 4 This is a flowchart of an infrared light interference reduction method according to an embodiment of the present invention.

[0012] Figure 5 This is a flowchart of an infrared light interference reduction method according to an embodiment of the present invention.

[0013] in:

[0014] 100: Electronic devices;

[0015] 110: Image sensor;

[0016] 120: Monitor;

[0017] 130: Storage device;

[0018] 140: Processor;

[0019] 111: IR sensing pixel;

[0020] 112:R sensing pixels;

[0021] 113:G sensor pixels;

[0022] 114:B sensing pixels;

[0023] L1~L4: Response curves;

[0024] S410~S430, S510~S249, S510~S560: Steps. Detailed Implementation

[0025] Some embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Component symbols used in the following description, when appearing in different drawings, are considered to be the same or similar components. These embodiments are only a part of the present invention and do not disclose all possible implementations of the invention. More precisely, these embodiments are merely examples of the methods and apparatus described in the claims of the present invention.

[0026] Figure 2 This is a block diagram of an electronic device according to an embodiment of the present invention. Please refer to... Figure 2 The electronic device 100 includes an image sensor 110, a display 120, a storage device 130, and a processor 140. The electronic device 100 may be, for example, a laptop, smart TV, tablet computer, game console, mobile phone, or other electronic devices with photographic capabilities; this invention is not limited thereto.

[0027] Image sensor 110 is used to provide image sensing functionality. Image sensor 110 may include a photosensitive element, such as a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) element, or other elements, which are not limited by the present invention. Image sensor 110 may form a camera module of electronic device 100 with other elements (such as a lens, aperture, etc.).

[0028] It should be noted that, in the embodiments of the present invention, the image sensor 110 includes a plurality of sensing pixels for sensing visible light and infrared light, and these sensing pixels are used to convert light energy into electrical energy. More specifically, the image sensor 110 is a hybrid image sensor, also known as an RGB-Ir image sensor. Compared to conventional image sensors that only sense visible light, the image sensor 110 may employ a color filter array (CFA) that incorporates a near-infrared light channel. Figure 3 This is a schematic diagram of an image sensor according to an embodiment of the present invention. Please refer to... Figure 3 The image sensor 110 includes IR sensing pixels (e.g., IR sensing pixel 111) for sensing infrared light and RGB sensing pixels (e.g., R sensing pixel 112, G sensing pixel 113, B sensing pixel 114) for sensing visible light. In this way, the image sensor 110 can simultaneously generate both RGB and IR images in a single image sensing operation. However, Figure 3 This is merely an illustrative example, and the present invention does not limit the arrangement of IR sensing pixels and RGB sensing pixels.

[0029] The display 120 is used to display the image data sensed by the image sensor 110. It can be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or any other type of display. This invention does not limit the types of displays.

[0030] Storage device 130 can be used to store images, instructions, program code, software modules, and other data. It can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices, integrated circuits, and combinations thereof.

[0031] Processor 140 is coupled to image sensor 110, display 120, and storage device 130, and may be, for example, a central processing unit (CPU), application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU), or other similar devices, integrated circuits, and combinations thereof. Processor 140 can access and execute software modules stored in storage device 130 to implement the infrared light interference reduction method in this embodiment of the invention. The term "software module" can be broadly interpreted as instructions, instruction sets, code, source code, application programs, software suites, threads, programs, functions, etc., regardless of whether it is referred to as software, firmware, intermediate software, microcode, hardware description language, or otherwise.

[0032] Figure 4 This is a flowchart of an infrared light interference reduction method according to an embodiment of the present invention. Figure 4 The method and process can be provided by Figure 2 The various components of the electronic device 100 are implemented. Please also refer to... Figure 2 and Figure 4 The following are the pairings. Figure 1 The components of the electronic device 100 are described, and the steps of the infrared light interference reduction method of this embodiment are explained.

[0033] In step S410, the processor 140 detects ambient light information using the image sensor 110. This ambient light information may include, for example, the ambient light wavelength range, ambient light color temperature, or infrared light intensity. Based on the data generated by the image sensor 110, the processor 140 can analyze the ambient light information. In some embodiments, based on the RGB channel data generated by the sensing pixels in the image sensor 110 used to sense visible light, the processor 140 can determine ambient light information including the ambient light wavelength range.

[0034] In step S420, the processor 140 determines the infrared light interference intensity based on the ambient light information. The processor 140 can determine the infrared light interference intensity based on the ambient light information through a lookup table or function calculation. The infrared light interference intensity represents the degree of influence of infrared light on the RGB sensing pixels of the image sensor 110. The greater the infrared light interference intensity, the greater the interference from the infrared light component to the RGB sensing pixels of the image sensor 110. In some embodiments, the processor 140 can determine the infrared light interference intensity by looking up a table based on the ambient temperature. Alternatively, in some embodiments, the processor 140 can determine the infrared light interference intensity based on the infrared light intensity sensed by the IR sensing pixels of the image sensor 110. Alternatively, in some embodiments, the processor 140 can determine the infrared light interference intensity by looking up a table based on the ambient light wavelength range.

[0035] In step S430, the processor 140 controls the image sensor 110 to perform image sensing based on either the original exposure time or the new exposure time, according to the intensity of infrared light interference. In some embodiments, the processor 140 may determine whether to use an original exposure table recording the original exposure time or a new exposure table recording the new exposure time based on the intensity of infrared light interference, thereby controlling the image sensor 110 to perform image sensing based on either the original exposure time or the new exposure time.

[0036] In some embodiments, processor 140 may compare the intensity of infrared light interference with a threshold value to determine whether to use the original exposure time or a new exposure time. If the intensity of infrared light interference is not greater than the threshold value, processor 140 may control image sensor 110 to perform image sensing based on the original exposure time. If the intensity of infrared light interference is greater than the threshold value, processor 140 may control image sensor 110 to perform image sensing based on the new exposure time. It is particularly important to note that the new exposure time is shorter than the original exposure time. That is, when processor 140 determines that the intensity of infrared light interference is greater than the threshold value, image sensor 110 will use a shorter new exposure time for image sensing, thereby reducing the energy of the infrared light received by image sensor 110.

[0037] Figure 5 This is a flowchart of an infrared light interference reduction method according to an embodiment of the present invention. Figure 5 The method and process can be provided by Figure 2 The various components of the electronic device 100 are implemented. Please also refer to... Figure 2 and Figure 5 The following are the pairings. Figure 1 The components of the electronic device 100 are described, and the steps of the infrared light interference reduction method of this embodiment are explained.

[0038] In step S510, based on the RGB channel data generated by the sensing pixels in the image sensor 110 used for sensing visible light, the processor 140 determines ambient light information including the wavelength range of ambient light. The RGB channel sensing data includes red channel values, green channel values, and blue channel values. The processor 140 can analyze the ambient light wavelength range based on the RGB channel data.

[0039] In some embodiments, processor 140 may calculate the red-green ratio (r / g value) between the red channel value and the green channel value, and calculate the blue-green ratio (b / g value) between the blue channel value and the green channel value. Processor 140 determines the ambient light wavelength range based on the red-green ratio value and the blue-green ratio value.

[0040] Specifically, based on the red, green, and blue channel values ​​sensed by all or some of the RGB sensing pixels of the image sensor 110, the processor 140 can obtain a red-green ratio (r / g value) and a blue-green ratio (b / g value). In some embodiments, the processor 140 can first average the red channel values ​​generated by multiple R sensing pixels to obtain a red channel average value. The processor 140 can first average the green channel values ​​generated by multiple G sensing pixels to obtain a green channel average value. The processor 140 can first average the blue channel values ​​generated by multiple B sensing pixels to obtain a blue channel average value. Then, the processor 140 can divide the red channel average value by the green channel average value to obtain the r / g value, and divide the blue channel average value by the green channel average value to obtain the b / g value. In some embodiments, the processor 140 can calculate the sub-red-green ratio value and sub-blue-green ratio value corresponding to each pixel position based on the red, green, and blue channel values ​​corresponding to each pixel position. Then, the processor 140 performs statistical processing on these sub-red-green ratio values ​​to obtain red-green ratio values, and performs statistical processing on these sub-blue-green ratio values ​​to obtain blue-green ratio values.

[0041] In some embodiments, multiple preset ambient light wavelength ranges are each associated with a corresponding set of reference red-green ratio values ​​and reference blue-green ratio values. In other words, multiple sets of reference red-green ratio values ​​and reference blue-green ratio values ​​are each associated with a corresponding preset ambient light wavelength range. Therefore, by determining that the red-green ratio value is closest to one of the multiple reference red-green ratio values ​​and the blue-green ratio value is closest to one of the multiple reference blue-green ratio values, the processor 140 can find the corresponding value from the multiple preset ambient light wavelength ranges based on the closest reference red-green ratio value and the closest reference blue-green ratio value. For example, Table 1 records multiple preset ambient light wavelength ranges and the corresponding multiple sets of reference red-green ratio values ​​and reference blue-green ratio values.

[0042]

[0043] Table 1

[0044] After the processor 140 obtains the red-green ratio value and blue-green ratio value based on the RGB channel sensing data sensed by the image sensor 110, the processor 140 can determine which of the multiple reference red-green ratio values ​​recorded in Table 1 the red-green ratio value is closest to, and determine which of the multiple reference blue-green ratio values ​​recorded in Table 1 the blue-green ratio value is closest to. Finally, the processor 140 can determine the ambient light wavelength range from the multiple preset ambient light wavelength ranges in Table 1. Assuming that the processor 140 obtains an r / g value of 1.24 and a b / g value of 0.97, the processor 140 can determine that the ambient light wavelength range is 610–700 nm. The above Table 1 can be experimentally established in advance and stored in the storage device 130.

[0045] Subsequently, in step S520, the processor 140 determines the infrared light interference intensity based on the ambient light wavelength range and the infrared light sensing characteristics of the image sensor 110. In some embodiments, the infrared light interference intensity is determined by looking up a lookup table based on the ambient light wavelength range, and the lookup table is established based on the infrared light sensing characteristics of the image sensor 110. This lookup table can be pre-established and stored in the storage device 130 through experiments using the image sensor 110. In other words, different image sensors may use different lookup tables due to their different infrared light sensing characteristics. The infrared light interference intensity can be implemented, for example, as the ratio of the infrared light response to the visible light response. For instance, the processor 140 can obtain the infrared light interference intensity by looking up the ambient light wavelength range in the lookup table 2.

[0046] Ambient light wavelength range Infrared light interference intensity 450~500 (nm) 15％ 500–570 nm 12％ 570–590 nm 2％ 590–610 nm 5％ 610~700 (nm) 18％

[0047] Table 2

[0048] Table 2 is a table that was pre-created and stored in storage device 130 after experiments were conducted using image sensor 110.

[0049] In step S530, the processor 140 determines whether the infrared light interference intensity is greater than a threshold value. The threshold value can be set according to actual needs, and this invention does not limit it. For example, based on the examples in Table 2, the threshold value can be set to 10%.

[0050] In step S540, if the infrared light interference intensity is not greater than a threshold value, the processor 140 controls the image sensor 110 to perform image sensing based on the original exposure time. In step S550, if the infrared light interference intensity is greater than a threshold value, the processor 140 controls the image sensor 110 to perform image sensing based on a new exposure time. The new exposure time is shorter than the original exposure time. For example, based on the examples in Table 2, assuming the threshold value is 10%. When the ambient light wavelength range is 570–590 nm and 590–610 nm, the image sensor 110 uses a longer original exposure value for image sensing. When the ambient light wavelength range is 450–500 nm, 500–570 nm, and 610–700 nm, the image sensor 110 uses a shorter new exposure value for image sensing.

[0051] In some embodiments, the new exposure time can be determined based on the original exposure time, for example, by subtracting a preset time length from the original exposure time, or by multiplying the original exposure time by a proportion less than 1. In some embodiments, the processor 140 can determine the new exposure time by searching for a new exposure table that differs from the original exposure table.

[0052] It is worth mentioning that in step S560, in response to the infrared light interference intensity exceeding a threshold, the processor 140 increases the gain value of the image sensor 110 based on the ambient brightness and controls the image sensor 110 to perform image sensing according to the increased gain value. Specifically, because the processor 140 controls the image sensor 110 to take a picture based on a shorter new exposure time, the image brightness decreases. Therefore, the processor 140 can increase the gain value of the image sensor 110 to improve the image brightness. The programmable amplifier in the image sensor 110 amplifies the sensing signal output by the sensing pixel according to the increased gain value, thereby improving the image brightness.

[0053] In some embodiments, the original exposure table records multiple original exposure times and corresponding original gain values. The new exposure table records multiple new exposure times and corresponding new gain values. Thus, in response to infrared light interference intensity exceeding a threshold, the processor 140 can simultaneously obtain the new gain value and new exposure time by looking up the new exposure table. For example, Table 3 is an example of a new exposure table and an original exposure table.

[0054] Table 3

[0055]

[0056]

[0057] However, Table 3 is merely an example and is not intended to limit the invention. Specifically, the processor 140 may determine whether to look up a new exposure table or the original exposure table based on the intensity of infrared light interference during the execution of an automatic exposure procedure.

[0058] It is worth mentioning that when the ambient light of the environment in which the electronic device 100 is located contains a strong infrared component, the processor 140 can obtain an infrared interference intensity greater than a threshold value to control the image sensor 110 to perform image sensing based on a new exposure time shorter than the original exposure time. In this way, the infrared light energy sensed by the RGB sensing pixels of the image sensor 110 can be effectively reduced, thereby improving the color shift phenomenon of the RGB image. Furthermore, compared to conventional techniques, when performing signal removal operations for infrared components, this embodiment of the invention can select to remove a lower signal amount or a preset level, thereby retaining more visible light energy. For example, assuming that conventional techniques require subtracting a preset signal amount from each RGB channel data to improve infrared interference, this invention only needs to subtract half of the aforementioned preset signal amount from each RGB channel data.

[0059] In summary, in the embodiments of the present invention, the image sensor includes multiple sensing pixels for sensing visible light and infrared light. Ambient light information of the environment in which the electronic device is located can be detected first, thereby determining the intensity of infrared light interference. The exposure time of the image sensor is adjusted according to the intensity of infrared light interference. Thus, the exposure time of the image sensor can be adaptively adjusted according to the intensity of infrared light components in the scene. By shortening the exposure time of the image sensor, the infrared light energy received by the image sensor can be reduced, thereby effectively reducing the adverse effects of infrared light on RGB images. Furthermore, for environments with low brightness but abundant infrared light, the image quality of RGB images can be improved by reducing the exposure time and increasing the gain value. Moreover, when performing signal cancellation operations for infrared light components, embodiments of the present invention can select to cancel lower signal amounts or preset levels, thereby retaining more visible light energy.

[0060] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for reducing infrared light interference, the method comprising: An image sensor is used to detect ambient light information, wherein the image sensor includes a plurality of sensing pixels for sensing visible light and infrared light. The intensity of infrared light interference is determined based on the ambient light information; and The image sensor is controlled to perform image sensing based on either an original exposure time or a new exposure time, according to the intensity of the infrared light interference. The step of controlling the image sensor to perform image sensing based on either the original exposure time or the new exposure time according to the infrared light interference intensity includes: If the intensity of the infrared light interference is not greater than a critical value, the image sensor is controlled to perform image sensing based on the original exposure time. as well as In response to the infrared light interference intensity being greater than the threshold value, the image sensor is controlled to perform image sensing based on the new exposure time, wherein the new exposure time is shorter than the original exposure time.

2. The infrared light interference reduction method as described in claim 1, characterized in that, The method further includes: In response to the infrared light interference intensity being greater than the threshold value, the gain value of the image sensor is increased based on the ambient brightness, and the image sensor is controlled to perform image sensing based on the increased gain value.

3. The infrared light interference reduction method as described in claim 1, characterized in that, The steps for detecting ambient light information using the image sensor include: Based on the RGB channel data generated by the sensing pixels in the image sensor used to sense the visible light, ambient light information, including the ambient light wavelength range, is determined.

4. The infrared light interference reduction method as described in claim 3, characterized in that, The step of determining the intensity of infrared light interference based on the ambient light information includes: The intensity of infrared light interference is determined based on the ambient light wavelength range and the infrared light sensing characteristics of the image sensor.

5. The infrared light interference reduction method as described in claim 4, characterized in that, The intensity of the infrared light interference is determined by looking up a lookup table based on the wavelength range of the ambient light. The lookup table is established based on the infrared light sensing characteristics of the image sensor.

6. The infrared light interference reduction method as described in claim 3, characterized in that, The RGB channel sensing data includes red channel values, green channel values, and blue channel values, and the step of determining ambient light information, including the range of ambient light wavelengths, includes: Calculate the red-to-green ratio between the red channel value and the green channel value, and calculate the blue-to-green ratio between the blue channel value and the green channel value; and The ambient light wavelength range is determined based on the red-green ratio and the blue-green ratio.

7. An electronic device comprising: An image sensor, comprising multiple sensing pixels for sensing visible light and infrared light; Storage device; as well as A processor, coupled to the image sensor and the storage device, is configured to: The image sensor is used to detect ambient light information; The intensity of infrared light interference is determined based on the ambient light information. The image sensor is controlled to perform image sensing based on either an original exposure time or a new exposure time, according to the intensity of infrared light interference. In response to the infrared light interference intensity not exceeding a critical value, the image sensor is controlled to perform image sensing based on the original exposure time; and In response to the infrared light interference intensity being greater than the threshold value, the image sensor is controlled to perform image sensing based on the new exposure time, wherein the new exposure time is shorter than the original exposure time.

8. The electronic device as claimed in claim 7, characterized in that, The processor is configured to: In response to the infrared light interference intensity being greater than the threshold value, the gain value of the image sensor is increased based on the ambient brightness, and the image sensor is controlled to perform image sensing based on the increased gain value.

9. The electronic device as claimed in claim 7, characterized in that, The processor is configured to: Based on the RGB channel data generated by the sensing pixels in the image sensor used to sense the visible light, ambient light information, including the ambient light wavelength range, is determined.

10. The electronic device as claimed in claim 9, characterized in that, The processor is configured to: The intensity of infrared light interference is determined based on the ambient light wavelength range and the infrared light sensing characteristics of the image sensor.

11. The electronic device as claimed in claim 10, characterized in that, The intensity of the infrared light interference is determined by looking up a lookup table based on the wavelength range of the ambient light. The lookup table is established based on the infrared light sensing characteristics of the image sensor.

12. The electronic device as claimed in claim 9, characterized in that, The RGB channel sensing data includes red channel values, green channel values, and blue channel values, and the processor is configured to: Calculate the red-green ratio between the red channel value and the green channel value, and calculate the blue-green ratio between the blue channel value and the green channel value; as well as The ambient light wavelength range is determined based on the red-green ratio and the blue-green ratio.

Images