Method and system for reducing thermal shadowing in image sensors using on-chip localized heat sources

Abstract

A method and system for reducing thermal shadowing in an image sensor using an on-chip localized heat source is provided. The method includes: detecting one or more areas where thermal shadowing occurs in a CMOS image sensor, the CMOS image sensor including a plurality of heating elements in a chip; automatically switching a subset of the plurality of heating elements on based on the detected one or more areas; and automatically switching the subset of the plurality of heating elements off during an active power consumption phase of the CMOS image sensor.

Description

[0001] This application claims the priority of Indian Patent Application No. 201941033238 filed on August 17, 2019, in the Indian Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field

[0002] The disclosed embodiments relate to an image sensor, and more particularly, to an image sensor having a plurality of pixels arranged in rows and columns, which is used in, for example, a mobile phone equipped with a camera, an automotive image sensor, a surveillance image sensor, and any complementary metal oxide semiconductor (CMOS) image sensor to reduce thermal dark shading. Background Art

[0003] In an image sensor, each light receiving element outputs a signal having an integrated light component and an integrated dark current component. Dark current is an undesirable current that accumulates (intrgrate) as a charge in the photodiode of a pixel in a complementary metal oxide semiconductor (CMOS) image sensor and is temperature-dependent. Due to the large size and uneven power consumption density of image sensors, the temperature across the image sensor chip may be uneven, which results in uneven dark current across the chip. The accumulated charge caused by the dark current on each pixel causes an offset in the pixel signal voltage. This results in dark shadows (or thermal shadows) in the image sensor, in which the shadow of the dark image is caused by an uneven offset caused by uneven dark currents in pixels throughout the pixel array. Dark shadows cause shadow patterns in the image that are visible under low light conditions.

[0004] Therefore, there exists a need for a solution that overcomes the above-mentioned problems. Summary of the invention

[0005] This Summary is provided to introduce in a simplified form a selection of concepts that are further described in the disclosed detailed description. This Summary is not intended to identify key or essential inventive concepts of the disclosure, nor is it intended to determine the scope of the disclosure.

[0006] According to an aspect of an example embodiment, a method for reducing thermal shading in a complementary metal oxide semiconductor (CMOS) image sensor is provided, the method comprising: detecting one or more regions in the CMOS image sensor where thermal shading occurs, the CMOS image sensor comprising a plurality of heating elements in a chip; automatically switching a subset of the plurality of heating elements on based on the detected one or more regions; and automatically switching the subset of the plurality of heating elements off during an active power consumption phase of the CMOS image sensor.

[0007] According to an aspect of an example embodiment, a system for reducing thermal shadowing in a complementary metal oxide semiconductor (CMOS) image sensor is provided, the system comprising: a CMOS image sensor, the CMOS image sensor comprising a detector and comprising a plurality of heating elements in a chip; and a controller associated with the CMOS image sensor and configured to: automatically switch a subset of the plurality of heating elements based on a determination that thermal shadowing occurs; and automatically switch the subset of the plurality of heating elements to be disconnected during an active power consumption phase of the CMOS image sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other aspects, features and advantages of the disclosed specific example embodiments will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0009] Figure 1 A schematic block diagram of a system for reducing thermal dark shadows in a complementary metal oxide semiconductor (CMOS) image sensor according to an example embodiment is shown;

[0010] Figure 2 A schematic block diagram illustrating a configuration of a CMOS image sensor array according to an example embodiment is shown;

[0011] Figure 3 shows a circuit diagram of a heating element according to an example embodiment;

[0012] Figure 4 shows the power consumption of a heating element according to an example embodiment;

[0013] Figure 5 shows the temperature difference in a chip without a heating element;

[0014] Figure 6 illustrates a temperature difference in a chip including a heating element according to example embodiments; and

[0015] Figure 7 A flow chart illustrating a method for reducing thermal dark shadows in a CMOS image sensor according to example embodiments is shown. DETAILED DESCRIPTION

[0016] For the purpose of promoting an understanding of the principles of the disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the embodiments. It will be understood, however, that no limitation of the scope of the disclosure is intended thereby, and such changes and further modifications in the illustrated systems and such further applications of the principles of the disclosure as illustrated herein would normally occur to one skilled in the art to which the disclosure relates.

[0017] It will be understood by those skilled in the art that both the foregoing general description and the following detailed description are explanatory of the disclosure and are not intended to be restrictive of the disclosure.

[0018] Throughout this specification, references to "on one hand," "on the other hand," or similar language indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment disclosed. Thus, appearances of the phrases "in one embodiment," "in another embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0019] The terms "include", "comprises", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process or method comprising a list of steps includes not only those steps, but also other steps not expressly listed or inherent to such a process or method. Similarly, one or more devices or subsystems or elements or structures or components preceded by "comprising ... one (kind)" do not exclude the presence of other devices or other subsystems or other elements or other structures or other components or additional devices or additional subsystems or additional elements or additional structures or additional components without more constraints.

[0020] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.The systems, methods, and examples provided herein are illustrative only and are not intended to be limiting.

[0021] Hereinafter, disclosed example embodiments will be described in detail with reference to the accompanying drawings.

[0022] Figure 1 A schematic block diagram of a system 100 for reducing thermal dark shadows (thermal shadows or dark shadows) in a complementary metal oxide semiconductor (CMOS) image sensor is shown according to an example embodiment.

[0023] In an example embodiment, the system 100 can be configured to reduce thermal dark shadows in the CMOS image sensor 102 by embedding a plurality of heating elements 104 in the CMOS image sensor 102. As a result, the problem of dark shadows caused by uneven temperatures throughout the image sensor chip is solved. In an example, the system 100 can be implemented in a mobile phone equipped with a camera, an automotive image sensor, a surveillance image sensor, and any large CMOS image sensor of higher resolution and / or large die size, etc.

[0024] In one example embodiment, the system 100 includes a CMOS image sensor 102, a plurality of heating elements 104, and a controller 108. The system 100 may also include a detector 106. Figure 1106 and the controller 108 are described as being included in the CMOS image sensor 102 , but the disclosure is not limited thereto, and the detector 106 and the controller 108 may be disposed separately from the CMOS image sensor 102 and electrically connected to the CMOS image sensor 102 .

[0025] In one example, the CMOS image sensor 102 may be configured to capture an image. In one embodiment, the CMOS image sensor 102 may be a chip sensor or a three-dimensional (3D) structure image sensor in which two or more chips are stacked. For example, Figure 2 An example of two chips being stacked to provide a 3D structure image sensor is shown. Figure 2 , the CMOS image sensor 102 may include a top chip 204 and a bottom chip 202 that are connected to each other (eg, glued together).

[0026] In one example embodiment, the top chip 204 of the CMOS image sensor 102 includes an array of active pixel sensor elements, such as photodiodes, that convert optical signals into electrical signals.

[0027] In one example embodiment, the bottom chip 202 of the CMOS image sensor 102 contains a conversion circuit that converts an electrical signal into a digital signal. The bottom chip 202 may include a plurality of heating elements 104, a detector 106, and a controller 108.

[0028] like Figure 3 As shown in FIG. 2 , in the bottom chip 202, the plurality of heating elements 104 may be implemented in the form of a circuit. For example, each of the plurality of heating elements 104 may include a resistor (e.g., R, R / 2, R / 4, R / 8, or R / 16) and a metal oxide semiconductor field effect transistor (MOSFET) (e.g., TUNE[0], TUNE[1], TUNE[2], TUNE[3], or TUNE[4]). Figure 3 Each of the plurality of heating elements 104 may include a resistor between a power supply (eg, VDDD) and ground of the chip and a negative channel metal oxide semiconductor (NMOS) transistor connected in series to the resistor, and branches of the plurality of resistors and NMOS transistors are connected in parallel between the power supply and ground.

[0029] In one embodiment, an NMOS transistor can be configured to function as a switching element in each branch of a resistor and an NMOS transistor of the plurality of heating elements 104. Thus, each of the plurality of heating elements 104 can be controlled to be turned on / off by the controller 108 by turning on / off the NMOS transistor of each branch. For example, by selectively switching the NMOS, one branch (e.g., a branch including R / 2) can be activated (or turned on) and another branch (e.g., a branch including R / 4) can be turned off. Thus, the heat generation of the plurality of heating elements 104 can be controlled.

[0030] Assuming that the voltage drop across the NMOS transistors is negligible, the power consumption of the plurality of heating elements 104 may be represented as follows, where TUNE[0]*R may represent the resistance of the branch including R and TUNE[0], TUNE[1]*R / 2 may represent the resistance of the branch including R / 2 and TUNE[1], TUNE[2]*R / 4 may represent the resistance of the branch including R / 4 and TUNE[2], TUNE[3]*R / 8 may represent the resistance of the branch including R / 8 and TUNE[3], and TUNE[4]*R / 16 may represent the resistance of the branch including R / 16 and TUNE[4]:

[0031]

[0032] Multiple heating elements 104 may be interspersed in the conversion circuit in the bottom chip 202 to balance the thermal gradient in the bottom chip 202. The thermal gradient in the bottom chip 202 of the CMOS image sensor 102 affects the photosensitive elements in the top chip 204.

[0033] Furthermore, multiple heating elements 104 may also be implemented via various energy dissipation blocks such as inverter chains, ring oscillators, or any energy dissipation blocks with small areas to generate local heating instead of resistor ladders and NMOS transistor on / off.

[0034] The detector 106 may be embedded in the bottom chip 202 of the CMOS image sensor 102 and may be configured to detect one or more areas on the CMOS image sensor 102 where thermal shadowing occurs under low light conditions. In one embodiment, the system may embed the detector 106 in the bottom chip 202. In another embodiment, during the testing of the bottom chip 202, the system 100 may detect one or more areas on the CMOS image sensor 102 where thermal shadowing occurs under low light conditions and store information related to the detection. This information may be used to turn on / off a subset of the plurality of heating elements 104 (e.g., any one or any combination of any two or more of the plurality of heating elements 104) to provide local heating within the CMOS image sensor.

[0035] In an example embodiment, the controller 108 is linked to (or associated with) the CMOS image sensor 102 and the detector 106. The controller 108 may be configured to automatically turn on a subset of the plurality of heating elements 104 to provide local heating in areas of the CMOS image sensor 102 where thermal shadowing occurs under low-light image capture conditions. The controller 108 may be configured to automatically turn off a subset of the heating elements 104 during an active power consumption phase of the CMOS image sensor 102. The controller 108 may be implemented by a processor, such as a microcontroller (MCU) or a central processing unit (CPU), firmware or hardware logic, such as a field programmable gate array (FPGA), or any combination thereof.

[0036] Figure 4 The power consumption of the CMOS image sensor 102 during an active period (or active power consumption phase) and a low power phase according to an example embodiment is shown. A subset of the heating elements 104 may be controlled to be turned on or off according to the needs of the CMOS image sensor 102. For example, a subset of the heating elements 104 may be turned on during a low power phase (e.g., a vertical blank (V-Blank) phase) of sensor operation. In low light conditions, the low power phase of the CMOS image sensor 102 may allow for higher light capture, and therefore, the operation of driving a subset of the plurality of heating elements 104 may not increase power consumption during the highest power consumption mode (e.g., bright light conditions). Figure 5 shows the temperature difference in a chip without the heating element 104, Figure 6 1 shows the temperature difference in a chip having a heating element 104 according to an example embodiment. Figure 5 and Figure 6 For the chip without the heating element 104 , the difference ΔT between the higher temperature and the lower temperature reaches 3.342° C., while for the chip with the heating element 104 , the difference ΔT between the higher temperature and the lower temperature is 1.854° C.

[0037] Figure 7 A flow chart illustrating a method 700 for reducing hot dark shadows in a CMOS image sensor according to an example embodiment is shown. Figures 1 to 6 The features explained in detail in the description of Figure 7 is not explained in detail in the description.

[0038] At step 702, method 700 includes embedding a plurality of heating elements in a chip of a CMOS image sensor. In method 700, the embedding step further includes generating a map of the plurality of heating elements in the chip. In the method, the embedding step further includes enabling a control for turning on one or more heating elements among the plurality of heating elements for the detected hot dark shadow region. For example, the operation of enabling a control for turning on one or more heating elements may be performed by referring to the map of the plurality of heating elements.

[0039] At step 704, method 700 includes detecting one or more regions on the CMOS image sensor that are subject to thermal shading during low light conditions. In an example embodiment, detector 106 may be configured to detect one or more regions on the CMOS image sensor that are subject to thermal shading during low light conditions.

[0040] In one example embodiment, the step of detecting a hot dark shadow area on the CMOS image sensor includes at least one selected from: distributing a plurality of temperature sensors on the chip to detect a temperature difference on the chip; and capturing a dark image to detect a hot dark shadow on the CMOS image sensor. In addition, the step of detecting a hot dark shadow area on the CMOS image sensor may further include processing an image of the CMOS image sensor using artificial intelligence (AI) and / or machine learning to detect the hot dark shadow area.

[0041] At step 706, method 700 includes automatically turning on a subset of heating elements to provide local heating within the CMOS image sensor based on one or more areas subject to thermal shadowing during low-light image capture conditions. At step 708, method 700 includes automatically turning off a subset of heating elements during an active power consumption phase of the CMOS image sensor. In an example embodiment, controller 108 may be configured to automatically turn on a subset of heating elements to provide local heating within the CMOS image sensor based on areas subject to thermal shadowing during low-light image capture conditions, and automatically turn off a subset of heating elements during an active power consumption phase of the CMOS image sensor. In one embodiment, local heating is triggered by power consumption caused by thermal resistance.

[0042] Benefits, other advantages, and solutions to problems have been described above with respect to example embodiments. However, the benefits, advantages, solutions to problems, and any components that may cause any benefit, advantage, or solution to occur or become more significant are not to be construed as key, required, or essential features or components of any or all claims.

[0043] At least by means of various example embodiments, the present disclosure achieves at least the following advantages:

[0044] 1. Each heating element uses a very small area;

[0045] 2. The power consumption of each heating element is programmable;

[0046] 3. The circuit of each heating element is based on components already available in the manufacturing process;

[0047] 4. Easily integrate multiple instances across electronic devices to allow correction of any dark shadow outlines; and

[0048] 5. It has no effect on the power consumption in the highest power consumption mode.

[0049] At least one of the components, elements, modules or units described here can be implemented as various numbers of hardware, software and / or firmware structures for performing the various functions described above according to the example embodiments. For example, at least one of these components, elements or units can use a direct circuit structure (such as, memory, processor, logic circuit, lookup table, etc.) that can perform the corresponding function by the control of one or more microprocessors or other control devices. In addition, at least one of these components, elements or units can be specifically implemented by a part of a module, program or code containing one or more executable instructions for performing a specific logical function, and can be performed by one or more microprocessors or other control devices. In addition, at least one of these components, elements or units can also include a processor (such as, a central processing unit (CPU) that performs the corresponding function), a microprocessor, etc., or can also be realized by a processor (such as, a central processing unit (CPU) that performs the corresponding function), a microprocessor, etc. Two or more of these components, elements or units can be combined into a single component, element or unit that performs all operations or functions of two or more components, elements or units combined. In addition, at least a part of the function of at least one component, element or unit of these components, elements or units can be performed by another component, element or unit in these components, elements or units. In addition, although the bus is not shown in the block diagram, the communication between the components, elements or units can be performed via the bus. The functional aspects of the above example embodiments can be implemented in algorithms executed on one or more processors. In addition, the components, elements or units represented by the blocks or processing steps can adopt any number of existing technologies for electronic device configuration, signal processing and / or control, data processing, etc.

[0050] While the inventive concepts have been particularly shown and described with reference to example embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the claims.

Claims

1. A method for reducing thermal shadowing in a complementary metal oxide semiconductor image sensor, the method comprising: detecting one or more areas of thermal shading in a complementary metal oxide semiconductor image sensor, the complementary metal oxide semiconductor image sensor including a plurality of heating elements in a chip; automatically switching on a subset of the plurality of heating elements to provide localized heating based on the detected one or more zones; as well as A subset of the plurality of heating elements is automatically switched off during an active power consumption phase of the complementary metal oxide semiconductor image sensor.

2. The method according to claim 1, wherein: The step of automatically switching on a subset of the plurality of heating elements comprises: generating a map of the plurality of heating elements in the chip; and Switching of a subset of the plurality of heating elements is controlled based on the detected one or more zones and also based on a map.

3. The method according to claim 1 or 2, wherein: The step of detecting includes at least one of the following: By using multiple temperature sensors to detect temperature differences on the chip; detecting the one or more regions where thermal shadowing occurs based on the captured dark image; as well as The one or more areas where thermal shadowing occurs are detected by processing images captured by a complementary metal oxide semiconductor image sensor using artificial intelligence and / or machine learning.

4. The method according to claim 1 or 2, wherein: Each of the plurality of heating elements includes a resistor between a power source and a ground and a transistor connected in series to the resistor.

5. The method according to claim 1 or 2, wherein: Automatically switching a subset of the plurality of heating elements on is triggered in response to power consumption caused by the thermal resistance.

6. The method according to claim 4, wherein: The plurality of heating elements include a plurality of branches connected in parallel, each of the plurality of branches including a resistor and a transistor connected in series to the resistor.

7. The method according to claim 4, wherein: The transistor is a metal oxide semiconductor field effect transistor, and the switching of each of the plurality of heating elements can be controlled by switching the metal oxide semiconductor field effect transistor included in each of the plurality of heating elements.

8. The method according to claim 1 or 2, wherein: The plurality of heating elements includes an energy dissipation block including at least one of an inverter chain and a ring oscillator.

9. A system for reducing thermal shadowing in a complementary metal oxide semiconductor image sensor, the system comprising: a complementary metal oxide semiconductor image sensor, the complementary metal oxide semiconductor image sensor including a detector and including a plurality of heating elements in a chip; as well as A controller is associated with the complementary metal oxide semiconductor image sensor and is configured to: automatically switching a subset of the plurality of heating elements on to provide localized heating based on a determination that thermal shadowing has occurred, and A subset of the plurality of heating elements is automatically switched off during an active power consumption phase of the complementary metal oxide semiconductor image sensor.

10. The system according to claim 9, wherein: The detector is configured to detect one or more regions in the complementary metal oxide semiconductor image sensor where thermal shadowing occurs.

11. The system according to claim 10, wherein: The controller is further configured to automatically switch on a subset of the plurality of heating elements based on the one or more regions detected by the detector.

12. The system according to claim 10, wherein: The controller is further configured to generate a map of the plurality of heating elements in the chip and control switching of a subset of the plurality of heating elements based on the detected one or more regions and also based on the map.

13. The system according to claim 10, wherein: The step of detecting includes at least one of the following: By using multiple temperature sensors to detect temperature differences on the chip; detecting the one or more regions where thermal shadowing occurs based on the captured dark image; as well as The one or more areas where thermal shadowing occurs are detected by processing images captured by a complementary metal oxide semiconductor image sensor using artificial intelligence and / or machine learning.

14. The system according to any one of claims 9 to 13, wherein: Each of the plurality of heating elements includes a resistor between a power source and a ground and a transistor connected in series to the resistor.

15. The system according to any one of claims 9 to 13, wherein: Automatically switching a subset of the plurality of heating elements on is triggered in response to power consumption caused by the thermal resistance.

16. The system of claim 14, wherein: The plurality of heating elements include a plurality of branches connected in parallel, each of the plurality of branches including a resistor and a transistor connected in series to the resistor.

17. The system of claim 14, wherein: The transistor is a metal oxide semiconductor field effect transistor, and the controller is further configured to control switching of each of the plurality of heating elements by switching the metal oxide semiconductor field effect transistor included in each of the plurality of heating elements.

18. The system of claim 9, wherein: The plurality of heating elements includes an energy dissipation block including at least one of an inverter chain and a ring oscillator.

Images