Image sensors and image sensor pixels

Abstract

An image sensor and a pixel of the image sensor are disclosed. The pixel of the image sensor includes a resistive microbolometer sensor portion, a visible light image sensor portion, and an output path. The resistive microbolometer sensor portion outputs a signal corresponding to an infrared (IR) image sensed by the resistive microbolometer sensor portion. The resistive microbolometer sensor portion does not use a bias current. The visible light image sensor portion outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion. The output path is shared by the resistive microbolometer sensor portion and the visible light image sensor portion and can be controlled to selectively output a signal corresponding to the IR image, a signal corresponding to the visible light image, or a fused image based on the IR image and the visible light image. The resistive microbolometer sensor portion can sense a near-infrared image or a long-wave infrared image.

Description

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 030,904, filed on May 27, 2020, and U.S. Application No. 16 / 937,572, filed on July 23, 2020, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0002] The subject matter disclosed herein relates to image sensors. More specifically, the subject matter disclosed herein relates to hybrid pixels for image sensors that use a resistive microbolometer combined with a complementary metal oxide semiconductor (CMOS) image sensor (CIS) into a single pixel to provide visible / near infrared (NIR) and long wave infrared (LWIR) imaging. Background Art

[0003] Because LWIR sensors do not require active illumination and may be able to sense human subjects and / or other objects obscured by obstacles, low-cost thermal LWIR sensors may be useful for advanced driver assistance systems (ADAS) and autonomous driving applications. Most thermal cameras include resistive bolometers that use a DC bias current for readout (the DC bias current can cause the bolometer to self-heat), and the readout circuitry for resistive bolometers can be complex. Combining visible light imaging and thermal imaging capabilities into a single sensor can significantly reduce system cost, achieve improved night vision, increase object recognition accuracy, and reduce processing power consumption.

[0004] Most typical thermal cameras can only provide thermal images with lower relative resolution than visible light image cameras. Typical imaging systems that include separate visible light and thermal imaging capabilities can be bulky and difficult to integrate into small form factor applications. Single thermal imaging cameras also cannot detect important information (such as signals and symbols) in visible light images. Summary of the Invention

[0005] Example embodiments provide a pixel of an image sensor, which may include a resistive microbolometer sensor portion, a visible light image sensor portion, and an output path. The resistive microbolometer sensor portion may output a signal corresponding to an infrared (IR) image sensed by the resistive microbolometer sensor portion. The visible light image sensor portion may output a signal corresponding to a visible light image sensed by the visible light image sensor portion. The output path may be shared by the resistive microbolometer sensor portion and the visible light image sensor portion. The output path may be controlled to selectively output a signal corresponding to the IR image, a signal corresponding to the visible light image, or a fused image based on the IR image and the visible light image. In one embodiment, the resistive microbolometer sensor portion does not use a bias current.

[0006] Example embodiments provide an image sensor that may include a pixel array. At least one first pixel may include a resistive microbolometer and a visible light image sensor portion. The resistive microbolometer sensor portion may output a signal corresponding to an IR image sensed by the resistive microbolometer sensor portion. The visible light image sensor portion may output a signal corresponding to a visible light image sensed by the visible light image sensor portion of the first pixel. At least one second pixel may include a visible light image sensor portion that outputs a visible light image sensed by the visible light image sensor portion of the second pixel. An output path may be shared by the resistive microbolometer sensor portion, the visible light image sensor portion of the first pixel, and the visible light image sensor portion of the second pixel. The output path may be controlled to selectively output a signal corresponding to the IR image, a signal corresponding to the visible light image sensed by the visible light image sensor portion of the first pixel, a fused image based on the IR image and the visible light image of the first pixel, or a signal corresponding to the visible light image sensed by the visible light image sensor portion of the second pixel. In one embodiment, the resistive microbolometer sensor portion does not use a bias current. In another embodiment, the output path may be shared by the resistive microbolometer sensor portion, the visible light image sensor portion of the first pixel, and the visible light image sensor portion of the second pixel. In yet another embodiment, the output path may be shared by the resistive microbolometer sensor portion and four visible light image sensor portions. In yet another embodiment, the output path may be shared by the resistive microbolometer sensor portion and eight visible light image sensor portions.

[0007] Example embodiments provide a pixel of an image sensor, which may include a resistive microbolometer sensor portion, a visible light image sensor portion, and an output path. The resistive microbolometer sensor portion may output a signal corresponding to an IR image sensed by the resistive microbolometer sensor portion. The visible light image sensor portion may output a signal corresponding to a visible light image sensed by the visible light image sensor portion, and the visible light image sensor portion may include a photodiode. The output path may be shared by the resistive microbolometer sensor portion and the visible light image sensor portion, and the output path may be controlled to selectively output a signal corresponding to the IR image, a signal corresponding to the visible light image, or a fused image based on the IR image and the visible light image of the pixel. In one embodiment, the pixel may be one pixel in a pixel array. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In the following sections, aspects of the subject matter disclosed herein will be described with reference to exemplary embodiments shown in the accompanying drawings, in which:

[0009] Figure 1 depicts a perspective view of an example embodiment of a resistive microbolometer that may be used in an example embodiment of a pixel and readout circuitry of an image sensor according to the subject matter disclosed herein;

[0010] Figure 2A is a schematic diagram of an example embodiment of a hybrid pixel including a resistive microbolometer and a photodiode according to the subject matter disclosed herein;

[0011] Figure 2B 1. When operated to output separate, combined or fused 2D / NIR images or 2D / LWIR images according to the subject matter disclosed herein Figure 2A Relative timing diagram of an example embodiment of hybrid pixels shown in FIG;

[0012] Figure 2C 1. FIG. 1 shows an example of a method according to the subject matter disclosed herein when operated to output separate 2D (or 2D / NIR) image information and LWIR image information. Figure 2A Relative timing diagram of an example embodiment of hybrid pixels shown in FIG;

[0013] Figure 3A is a schematic diagram of another example embodiment of a hybrid pixel of an image sensor according to the subject matter disclosed herein. The hybrid pixel can be operated to output image information of a separate, combined, or fused 2D / NIR image or a 2D / LWIR image;

[0014] Figure 3B 1. When operated to output separate, combined or fused 2D / NIR images or 2D / LWIR images according to the subject matter disclosed herein Figure 3A Relative timing diagram of an example embodiment of hybrid pixels shown in FIG;

[0015] Figure 3C 1. FIG. 1 shows an example of a method according to the subject matter disclosed herein when operated to output separate 2D (or 2D / NIR) image information and LWIR image information. Figure 3A Relative timing diagram of an example embodiment of hybrid pixels shown in FIG;

[0016] Figure 4A is a schematic diagram of an example embodiment of a hybrid pixel of an image sensor in which a resistive microbolometer is shared with two photodiodes according to the subject matter disclosed herein;

[0017] Figure 4B is a schematic diagram of an example embodiment of a hybrid pixel of an image sensor in which a resistive microbolometer is shared with four photodiodes according to the subject matter disclosed herein;

[0018] Figure 5A is a schematic diagram of another example embodiment of a hybrid pixel of an image sensor in which a resistive microbolometer is shared with two photodiodes according to the subject matter disclosed herein;

[0019] Figure 5B is a schematic diagram of another example embodiment of a hybrid pixel of an image sensor in which a resistive microbolometer is shared with four photodiodes according to the subject matter disclosed herein;

[0020] Figures 6A to 6J Depict example embodiments of arrays of different physical arrangements or placements of hybrid pixels, or visible light pixels (R, G, B) and microbolometer pixels (IR), respectively, according to the subject matter disclosed herein;

[0021] 7A to 7D Describes example embodiments of optical devices that may be used with the hybrid pixels disclosed herein; and

[0022] Figure 8 An electronic device including an image processing unit including hybrid pixels according to the subject matter disclosed herein is depicted. DETAILED DESCRIPTION

[0023] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. However, those skilled in the art will appreciate that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail in order not to obscure the subject matter disclosed herein.

[0024] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment disclosed herein. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "according to an embodiment" (or other phrases of similar meaning) in different places throughout this specification may not necessarily all refer to the same embodiment. Furthermore, in one or more embodiments, the particular features, structures, or characteristics may be combined in any suitable manner. In this regard, as used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" should not be construed as necessarily preferred or advantageous over other embodiments. Furthermore, in one or more embodiments, the particular features, structures, or characteristics may be combined in any suitable manner. Furthermore, depending on the context of the discussion herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. Similarly, hyphenated terms (e.g., "two-dimensional," "predetermined," "specific-pixel," etc.) may occasionally be used interchangeably with corresponding non-hyphenated versions (e.g., "two-dimensional," "predetermined," "specific pixel," etc.), and capitalized terms (e.g., "Counter Clock," "Row Select," "PIXOUT," etc.) may be used interchangeably with corresponding non-capitalized versions (e.g., "counter clock," "row select," "pixout," etc.). Such occasional interchangeable usage should not be considered inconsistent with each other.

[0025] In addition, depending on the context of the discussion herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. It should also be noted that the various figures (including component diagrams) shown and discussed herein are for illustrative purposes only and are not drawn to scale. Similarly, various waveform diagrams and timing diagrams are shown for illustrative purposes only. For example, for clarity, the size of some elements may be exaggerated relative to other elements. In addition, if deemed appropriate, reference numerals are repeated in the drawings to indicate corresponding and / or similar elements.

[0026] The terms used herein are for the purpose of describing some example embodiments only and are not intended to limit the claimed subject matter. Unless the context clearly indicates otherwise, the singular form as used herein is also intended to include the plural form. It will also be understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of stated features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups. Unless clearly defined in this way, the terms "first", "second", etc. as used herein are used as labels for the nouns following them and do not indicate any type of order (e.g., spatial, temporal, logical, etc.). In addition, the same reference numerals may be used between two or more figures to represent parts, components, blocks, circuits, units or modules with the same or similar functions. However, such usage is only for simplicity of description and ease of discussion and does not mean that the construction or architectural details of such components or units are the same between all embodiments or that such commonly referenced components / modules are the only way to implement some example embodiments in the example embodiments disclosed herein.

[0027] It will be understood that when an element or layer is referred to as being on, "connected to," or "coupled to" another element or layer, it can be directly on, directly connected to, or directly coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element or layer is referred to as being "directly on," "directly connected to," or "directly coupled to" another element or layer, there are no intervening elements or layers. Like reference numerals refer to like elements throughout. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0028] Unless clearly defined otherwise, the terms "first," "second," and the like used herein are used as labels for the nouns that follow them and do not indicate any type of order (e.g., spatial, temporal, logical, etc.). Furthermore, the same reference numerals may be used between two or more figures to indicate parts, components, blocks, circuits, units, or modules having the same or similar functions. However, such usage is merely for simplicity of illustration and ease of discussion and does not imply that the construction or structural details of such components or units are the same across all embodiments or that such commonly referenced components / modules are the only way to implement some of the example embodiments disclosed herein.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the subject matter belongs. It will also be understood that, unless expressly defined otherwise herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal manner.

[0030] The subject matter disclosed herein relates to image sensors that may include hybrid pixels that include a resistive microbolometer and a photodiode in the same pixel, wherein the resistive microbolometer and the photodiode share the same readout circuitry. In addition to a visible light image separated from a NIR and / or LWIR image, the hybrid pixel may also provide a fused visible light / NIR or a fused visible light / LWIR image. In one embodiment, an array of hybrid pixels may provide a fused visible light / NIR image, a fused visible light / LWIR image, and / or separate visible light, NIR, and / or LWIR images. A metalens may be used with a pixel array including the hybrid pixels disclosed herein to enable visible light / NIR / LWIR imaging in a single sensor, wherein the visible light / NIR / LWIR image information uses the same readout architecture, circuitry, and path. The visible light and infrared image information output from the hybrid pixel (whether fused or separate) may be used for, but not limited to, ADAS, mobile, industrial, and robotics applications.

[0031] The hybrid visible light / infrared imaging pixel disclosed herein includes a resistive microbolometer with low self-heating characteristics due to the absence of a DC bias and includes a low-power readout. The low self-heating and low-power characteristics of the hybrid image sensor provide improved accuracy for infrared images compared to conventional resistive bolometer-based infrared imagers.

[0032] Figure 1A perspective view depicts an example embodiment of a resistive microbolometer 100 that can be used in example embodiments of pixels and readout circuitry of an image sensor according to the subject matter disclosed herein. Microbolometer 100 can be formed on a substrate SUB and can include an infrared (IR) absorbing element 101, two conductive support members 102a and 102b, and two electrodes 103a and 103b. IR absorbing element 101 can be formed from an amorphous silicon (a-Si) layer and / or a vanadium oxide (V2O5) layer. The resistivity of IR absorbing element 101 can vary depending on the amount of IR radiation absorbed by the IR absorbing element. In one embodiment, IR absorbing element 101 can be approximately 50 μm by 50 μm in size and approximately 0.5 μm thick. Other sizes and shapes are possible. Electrodes 103a and 103b can be electrically connected to a supply voltage (or ground) 104 and source / drain (S / D) diffusion regions or terminals 105, respectively. In some embodiments, a reflector (not shown) may be located behind the IR absorbing element 101 (ie, between the IR absorbing element 101 and the substrate SUB) such that the reflector is positioned away from the IR source (not shown) relative to the IR absorbing element.

[0033] Figure 2A is a schematic diagram of an example embodiment of a hybrid pixel 200 including a resistive microbolometer and a photodiode according to the subject matter disclosed herein. The hybrid pixel 200 can be operated to output image information of a separate, combined, or fused two-dimensional / NIR (2D / NIR) image or a 2D / LWIR image. Alternatively, the hybrid pixel 200 can be operated to output separate 2D image information and NIR (or LWIR) image information. The hybrid pixel 200 can include a capacitor C b , resistive microbolometer R(T), photodiode PD1 and n-channel metal oxide semiconductor field effect transistors (MOSFETs) 201 to 206.

[0034] Capacitor C b It can be used to change the discharge time that represents IR information. In other words, IR information is generated by C b and the time constant formed by the resistive microbolometer R(T). Capacitor C b The capacitor C may include a first terminal and a second terminal. b The first terminal of MOSFET 201 may be connected to a first potential (such as ground). MOSFET 201 may include a first terminal connected to a capacitor C b The gate terminal of MOSFET 201 can be connected to the TM_RST signal. In the present disclosure, "a terminal is connected to a potential or a signal" can mean that the terminal receives the potential or signal.

[0035] MOSFET 202 may include a first S / D terminal connected to the second S / D terminal of MOSFET 201. The node where the first S / D terminal of MOSFET 202 and the second S / D terminal of MOSFET 201 are connected may be a floating diffusion (FD) node. The floating diffusion node may include a parasitic capacitance C fd .

[0036] The gate terminal of MOSFET 202 can be connected to the TM signal. The second S / D terminal of MOSFET 202 can be connected to the first terminal of the resistive microbolometer R(T). The second terminal of the resistive microbolometer R(T) can be connected to a second potential (such as analog ground AGND).

[0037] A first S / D terminal of MOSFET 203 may be connected to a node at the connection of the first S / D terminal of MOSFET 202 and the second S / D terminal of MOSFET 201. A second S / D terminal of MOSFET 203 may be connected to a first terminal of photodiode PD1, and a gate terminal of MOSFET 203 may be connected to a TX signal.

[0038] Photodiode PD1 may be an electronic positive-intrinsic-negative (PIN) diode or a shallow positive-negative (PN) junction for detecting 2D images. Resistive microbolometer R(T) may be located in the same semiconductor die as photodiode PD1 or in a different semiconductor die.

[0039] The first S / D terminal of MOSFET 204 can be connected to the second S / D terminal of MOSFET 201 and the first S / D terminal of MOSFET 202. The second S / D terminal of MOSFET 204 can be connected to a third potential (such as VAAPIX). The gate terminal of MOSFET 204 can be connected to the RST signal.

[0040] The gate terminal of MOSFET 205 may be connected to the FD node. A first S / D terminal of MOSFET 205 may be connected to a third potential (i.e., VAAPIX). A second S / D terminal of MOSFET 205 may be connected to a first S / D terminal of MOSFET 206. A gate terminal of MOSFET 206 may be connected to the SEL signal, and a second S / D terminal of MOSFET 206 may be connected to the PIXOUT line.

[0041] Figure 2B1. shows image information according to the subject matter disclosed herein when operated to output a combined or fused 2D / NIR image or 2D / LWIR image. Figure 2A A relative timing diagram 210 of an example embodiment of a hybrid pixel 200 is shown in FIG.

[0042] At 211, when the RST signal, TX signal, TM_RST signal, and TM signal are simultaneously true (high), all nodes of the hybrid pixel 200 are reset. At 212, after releasing the TX signal, RST signal, TM_RST signal, and TM signal, integration begins to obtain 2D image information. After the integration is completed at 213, at 214, all nodes are reset to read the reference level V rst , thereby performing a correlated double sampling (CDS) operation to remove kT / C noise and resetting the analog-to-digital converter (ADC) (not shown) of the pixel 200. rst Then, at 215, the charge is transferred from the photodiode PD1 to the floating diffusion FD node through the MOSFET 203. At 216, the MOSFET 202 is turned on to fuse the 2D image with the LWIR image information. At 217, the V rst -V signal -V thermal The ADC operation is performed on this signal, where V signal is the voltage representing 2D image information, V thermal is the voltage representing the NIR image information and is generated by C b and R(T) (or more precisely, C b +C fd At 213 , the SEL signal turns on the MOSFET 206 to output the fused 2D / NIR image information from the PIXOUT node.

[0043] Figure 2C 2D image information and NIR (or LWIR) image information according to the subject matter disclosed herein when operated to output separate 2D image information and NIR (or LWIR) image information Figure 2A A relative timing diagram 220 of an example embodiment of the hybrid pixel 200 is shown in FIG.

[0044] At 221, when the RST signal, TX signal, TM_RST signal, and TM signal are simultaneously true (high), all nodes of the hybrid pixel 200 are reset. At 222, after releasing the TX signal, RST signal, TM_RST signal, and TM signal, integration begins to obtain 2D image information. After the integration is completed at 223, at 224, the FD node is reset to read the reference level V rst, thereby performing CDS operation to remove kT / C noise and resetting the ADC for 2D imaging. rst After that, at 225, the charge is transferred from PD1 to the FD node through MOSFET 203, and an ADC operation is performed at 226 to obtain 2D image information. At 227, the TM_RST signal, the RST signal, and the TM signal turn on MOSFETs 201, 204, and 202, respectively, to read the reset level for IR imaging. At 228, the ADC is reset for IR imaging. At 229, the MOSFET 202 is turned on for a predetermined time, and at 230, an ADC operation is performed on the IR image information, which is obtained by C b and R(T) (or more precisely, C b +C fd At 223 , the SEL signal turns on the MOSFET 206 to output 2D imaging information at 226 and NIR image information at 230 .

[0045] Figure 3A is a schematic diagram of another example embodiment of a hybrid pixel 300 of an image sensor according to the subject matter disclosed herein. The hybrid pixel 300 can be operated to output image information of a combined or fused 2D / NIR image or a 2D / LWIR image. Alternatively, the hybrid pixel 300 can be operated to output separate 2D image information and NIR (or LWIR) image information. The hybrid pixel 300 may include a capacitor C b , resistive microbolometer R(T), photodiode PD2 and p-channel MOSFETs 301 to 306.

[0046] Capacitor C b It can be used to change the discharge time that represents IR information. In other words, IR information is generated by C b and the time constant formed by the resistive microbolometer R(T). Capacitor C b The capacitor C may include a first terminal and a second terminal. b The first terminal of MOSFET 301 may be connected to a first potential (such as VAAPIX). MOSFET 301 may include a first terminal connected to a capacitor C b The gate terminal of MOSFET 301 may be connected to the TM_RST signal.

[0047] MOSFET 302 may include a first S / D terminal connected to the second S / D terminal of MOSFET 301. The node at which the first S / D terminal of MOSFET 302 and the second S / D terminal of MOSFET 301 are connected may be a floating diffusion (FD) node. The floating diffusion node may include a parasitic capacitance C fd .

[0048] The gate terminal of MOSFET 302 can be connected to the TM signal. The second S / D terminal of MOSFET 302 can be connected to the first terminal of the resistive microbolometer R(T). The second terminal of the resistive microbolometer R(T) can be connected to a second potential (such as analog ground VAAPIX).

[0049] A first S / D terminal of MOSFET 303 may be connected to a node at the connection of the first S / D terminal of MOSFET 302 and the second S / D terminal of MOSFET 301. A second S / D terminal of MOSFET 303 may be connected to a first terminal of photodiode PD2, and a gate terminal of MOSFET 303 may be connected to a TX signal.

[0050] The photodiode PD2 may be a hole-type PIN photodiode or a shallow PN junction photodiode that can be used to detect 2D images. The resistive microbolometer R(T) may be placed in the same semiconductor die as the photodiode PD2 or in a different semiconductor die.

[0051] A first S / D terminal of MOSFET 304 may be connected to a second S / D terminal of MOSFET 301 and a first S / D terminal of MOSFET 302. A second S / D terminal of MOSFET 304 may be connected to a third potential (such as AGND). A gate terminal of MOSFET 304 may be connected to an RST signal.

[0052] The gate terminal of MOSFET 305 may be connected to the FD node. The first S / D terminal of MOSFET 305 may be connected to a third potential (i.e., AGND). The second S / D terminal of MOSFET 305 may be connected to the first S / D terminal of MOSFET 306. The gate terminal of MOSFET 306 may be connected to the SEL signal, and the second S / D terminal of MOSFET 306 may be connected to the PIXOUT line.

[0053] Figure 3B 1. shows image information according to the subject matter disclosed herein when operated to output a combined or fused 2D / NIR image or 2D / LWIR image. Figure 3A A relative timing diagram 310 of an example embodiment of a hybrid pixel 300 is shown in FIG.

[0054] At 311, when the RST signal, TX signal, TM_RST signal, and TM signal are simultaneously true (low), all nodes of the hybrid pixel 300 are reset. At 312, after releasing the TX signal, RST signal, TM_RST signal, and TM signal, integration begins to obtain 2D image information. After the integration is completed at 313, at 314, all nodes are reset to read the reference level V rst , in order to perform CDS operation to remove kT / C noise and reset the ADC (not shown). rst Then, at 315, the charge is transferred from PD2 to the FD node through MOSFET 303. At 316, MOSFET 302 is turned on to merge the 2D image information with the LWIR image information. At 317, the V rst -V signal -V thermal The ADC operation is performed on this signal, where V signal is the voltage representing 2D image information, V thermal is the voltage representing the NIR image information and is determined by C b and R(T) (or more precisely, C b +C fd At 313 , the SEL signal turns on the MOSFET 306 to output the fused 2D / NIR image information from the PIXOUT node.

[0055] Figure 3C 2D image information and NIR (or LWIR) image information according to the subject matter disclosed herein when operated to output separate 2D image information and NIR (or LWIR) image information Figure 3A A relative timing diagram 320 of an example embodiment of a hybrid pixel 300 is shown in FIG.

[0056] At 321, when the RST signal, TX signal, TM_RST signal, and TM signal are simultaneously true (low), all nodes of the hybrid pixel 300 are reset. At 322, after releasing the TX signal, RST signal, TM_RST signal, and TM signal, integration begins to obtain 2D image information. After the integration is completed at 323, at 324, the FD node is reset to read the reference level V rst , thereby performing CDS operation to remove kT / C noise and resetting the ADC for 2D imaging. rstAfter that, the charge is transferred from PD2 to the FD node through MOSFET 303 at 325, and an ADC operation is performed on the 2D image at 326. At 327, the TM_RST signal, the RST signal, and the TM signal are switched to true (low) to read the reset level for IR imaging, and then the ADC is reset for IR image information at 328. At 329, the MOSFET 302 is turned on for a predetermined time, and an ADC operation is performed on the IR image information at 330. The IR image information is obtained by C b and R(T) (or more precisely, C b +C fd At 323 , the SEL signal turns on the MOSFET 306 to output 2D imaging information at 326 and NIR image information at 330 .

[0057] Figure 4A is a schematic diagram of an example embodiment of a hybrid pixel 400 of an image sensor in which a resistive microbolometer is shared with two photodiodes according to the subject matter disclosed herein. Figure 4A The embodiment 400 shown in FIG. Figure 2A , and includes a MOSFET 401 and a second photodiode PD3 for forming a shared PD / microbolometer architecture. The portion of embodiment 400 that is a hybrid pixel 200 is indicated by a dashed line. It should be understood that the resistive microbolometer R(T) can be shared as many times as possible (i.e., 2 to N times, where N is an integer greater than 2). A 2-1 sharing arrangement can be a typical (two PDs and one resistive microbolometer) sharing arrangement ( Figure 4A ). 4-1 sharing arrangement 410 may include four PDs and one resistive microbolometer, and Figure 4B The embodiment 410 includes a hybrid pixel 200 ( Figure 2A ), and includes MOSFETs 401 to 403 and photodiodes PD3 to PD5. The portion of embodiment 410 that is the hybrid pixel 200 is indicated by the dotted line. The 8-1 sharing arrangement would include eight PDs and one resistive microbolometer.

[0058] Figure 5A is a schematic diagram of another example embodiment of a hybrid pixel 500 of an image sensor in which a resistive microbolometer is shared with two photodiodes according to the subject matter disclosed herein. Figure 5A The embodiment 500 shown in FIG. Figure 3A, and includes a MOSFET 501 and a second photodiode PD6 for forming a shared PD / microbolometer architecture. The portion of embodiment 500 that is a hybrid pixel 300 is indicated by the dashed line. It should be understood that the resistive microbolometer R(T) can be shared as many times as possible (i.e., 2 to N times). The 2-1 sharing arrangement can be a typical (two PDs and one resistive microbolometer) arrangement ( Figure 5A ). The 4-1 sharing arrangement 510 may include four PDs and one resistive microbolometer, and Figure 5B The embodiment 510 includes a hybrid pixel 300 ( Figure 3A ), and includes MOSFETs 501 to 503 and photodiodes PD6 to PD8. The portion of embodiment 510 that is a hybrid pixel 300 is indicated by the dotted line. The 8-1 sharing arrangement would include eight PDs and one resistive microbolometer.

[0059] Figure 6A An example embodiment of a portion of another array 600 of hybrid pixels 601 according to the subject matter disclosed herein is depicted. Array 600 can be used with backside illuminated (BSI) image sensors or stacked image sensors in which peripheral components can be located on a different die or substrate than the die or substrate on which array 600 is located.

[0060] In array 600, hybrid pixel 601 includes four visible light sensor sections and one microbolometer section that share an output path. For example, hybrid pixel 601 includes four visible light sensor sections indicated as red (R), green (Gb and Gr), and blue (B). Hybrid pixel 601 also includes one microbolometer section indicated as IR. In one embodiment, the IR pixel can be physically the same size as the four visible light sensor sections, where the IR pixel shares an output path with the four visible light sensor sections.

[0061] Peripheral components may include a row decoder 602, which may be connected to the array 600 as depicted, and a column decoder 603, including, for example, a column memory and / or a column scanner. One embodiment may include peripheral components 604 for correlated double sampling (CDS), column amplifiers, and / or column analog-to-digital converters (ADCs). Rows 605 may extend from the row decoder 602, and an output path 606 (e.g., a PIXOUT path) may extend from the hybrid pixel 601 to the peripheral components 604 and / or the column decoder 603. The output may be available at OUT.

[0062] Figure 6BAn example embodiment of a portion of an array 610 of visible light pixels 611 and capacitive microbolometer pixels 612 in accordance with the subject matter disclosed herein is depicted. Array 610 can be used with a front-side illuminated (FSI) image sensor or a back-side illuminated (BSI) image sensor.

[0063] In array 610, four visible light CIS pixels and one microbolometer IR pixel share an output path. For example, four visible light CIS pixels indicated as R, Gb, Gr, and B and an IR pixel share an output path. In one embodiment, the IR pixel can be positioned adjacent to the four visible light CIS pixels.

[0064] Peripheral components of array 610 may include a row decoder 602 and a column decoder 603 as depicted, which may be connected to array 610. One embodiment may include peripheral components 604 for correlated double sampling, column amplifiers, and / or column ADCs. Rows 605 may extend from row decoder 602, and output paths 606 (e.g., PIXOUT paths) may extend from hybrid pixels to peripheral components 604 and / or column decoder 603. Output may be available at OUT.

[0065] Figures 6C to 6J Example embodiments of arrays of different physical arrangements or placements of hybrid pixels, or visible light pixels (R, G, B) and microbolometer pixels (IR), respectively, are depicted according to the subject matter disclosed herein. Figures 6C to 6J The array depicted in can be used with an FSI image sensor. Figure 6C Depicts an example array 620 in which each pixel is similar to a pixel in FIG. Figure 2A and Figure 3A The mixed pixels 200 or 300 are depicted in FIG. Figure 6D An example array 630 is depicted where four visible light CIS pixels share one microbolometer pixel IR. Figure 6E An example array 640 is depicted where eight visible light CIS pixels share one microbolometer pixel IR.

[0066] Figure 6F and Figure 6G Example arrays 650 and 660 , respectively, are depicted where four visible light CIS pixels share one microbolometer pixel IR. Figure 6H An example array 670 is depicted where two CIS pixels share one microbolometer pixel IR. Figure 6I An example array 680 is depicted where four CIS pixels share one microbolometer pixel IR. Figure 6JAn example array 690 depicting two CIS pixels sharing one microbolometer pixel IR. In the above, M CIS pixels sharing one microbolometer pixel IR is equivalent to M CIS pixels sharing one output path with one microbolometer pixel IR, where M is a positive integer. In addition, although Figures 6C to 6J The case where M is an even number is described, but those skilled in the art will appreciate that M may also be an odd number. Figures 6A to 6J As shown, at least one hybrid pixel may share an output path. For example, the CIS pixel and / or microbolometer pixel IR of one hybrid pixel may share an output path with the CIS pixel and / or microbolometer pixel IR of at least one additional hybrid pixel.

[0067] Figure 7A An example embodiment of an optical device that can be used with the hybrid pixels disclosed herein is depicted. The optical device can include a global metalens 701 that focuses infrared radiation having a wavelength, for example, between about 8 μm and about 12 μm, toward an array of hybrid pixels 702. In one embodiment, the global metalens 701, which can include a thin, flat on-chip metasurface lens, can be used for LWIR imaging, NIR imaging, or both LWIR imaging and NIR imaging. The metasurface lens can be formed from a nanostructure comprising amorphous Si (silicon) (a-Si) and / or any other dielectric material that is transparent to wavelengths of the desired spectrum. Such a metasurface lens can be fabricated on a flat, transparent substrate at low cost using single-step conventional ultraviolet (UV) fabrication techniques. The nanostructures can be fabricated using different geometric sizes and arrangements to focus light at the same focal length for each wavelength, or at different focal lengths at the same spatial location or at different spatial locations, depending on the desired application. In another embodiment, a thin, flat metasurface may include dielectric nanostructures that can be used to focus light at visible / NIR wavelengths (0.4 μm-1 μm) and LWIR wavelengths (8 μm-12 μm) for simultaneous imaging in the visible / NIR and LWIR.

[0068] Figure 7BAn example embodiment of an optical device that can be used with the hybrid pixels disclosed herein is depicted. The optical device can include a global metalens 711 that focuses light or radiant energy of different wavelengths to different locations on one or more arrays 712 and 713 of hybrid pixels. Array 712 can include, for example, hybrid pixels, and array 713 can also include, for example, hybrid pixels. Global metalens 711 can focus light of wavelengths between, for example, about 0.4 μm and about 1 μm to one or more different locations on array 712, and can focus light of wavelengths between, for example, about 8 μm and about 12 μm to one or more locations on array 713. Although arrays 712 and 713 are depicted as two separate arrays, it should be understood that in some embodiments, arrays 712 and 713 can be a single array.

[0069] Figure 7C Another example embodiment of an optical device that can be used with the hybrid pixels disclosed herein is depicted. The optical device may include a global lens 721 that focuses different wavelengths of visible light and infrared energy onto one or more different microlenses (e.g., metasurface microlenses) 722 and 723. The metasurface microlenses 722 and 723, in turn, can focus different wavelengths of visible light and infrared energy onto one or more different arrays 724 and 725. For example, microlens 722 can focus light of wavelengths between approximately 8 μm and approximately 12 μm onto one or more locations on array 724, which can include one or more hybrid pixels. Similarly, microlens 723 can focus light of wavelengths between approximately 0.4 μm and approximately 1 μm onto one or more different locations on array 725, which can include hybrid pixels. Although arrays 724 and 725 are depicted as two separate arrays, it should be understood that in some embodiments, arrays 724 and 725 can be a single array.

[0070] Figure 7D An example embodiment of a layout of an array 730 is depicted containing hybrid pixels 731 where resistive microbolometers 732 may be shared with two or more photodiodes. Other layout arrangements are possible.

[0071] Figure 8An electronic device 800 is depicted that includes an image processing unit including hybrid pixels according to the subject matter disclosed herein. The electronic device 800 may be used in, but is not limited to, a computing device, a personal digital assistant (PDA), a laptop computer, a mobile computer, a web tablet, a wireless phone, a mobile phone, a smartphone, a digital music player, or a wired or wireless electronic device. The electronic device 800 may also be, but is not limited to, part of an ADAS, a mobile device imaging system, an industrial imaging system, a robot, or the like. The electronic device 800 may include a controller 810, input / output (I / O) devices 820 (such as, but not limited to, a keypad, a keyboard, a display, a touch screen display, a camera, and / or an image sensor), a memory (or memory device) 830, an interface 840, a graphics processor (GPU) 850, and an image processing unit 860, all connected to each other via a bus 870. The controller 810 may include, for example, at least one microprocessor, at least one digital signal processor, at least one microcontroller, or the like. The memory 830 may be configured to store command codes or user data to be used by the controller 810.

[0072] The electronic device 800 and various system components of the electronic device 800 may include an image processor 860 that includes hybrid pixels according to the subject matter disclosed herein. The interface 840 may be configured to include a wireless interface configured to send data to or receive data from a wireless communication network using RF signals. The wireless interface 840 may include, for example, an antenna, a wireless transceiver, and the like. The electronic device 800 can also be used in communication interface protocols of communication systems (such as, but not limited to, Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), North American Digital Communications (NADC), Extended Time Division Multiple Access (E-TDMA), Wideband CDMA (WCDMA), CDMA2000, Wi-Fi, Municipal Wi-Fi (Muni Wi-Fi), Bluetooth, Digital Enhanced Cordless Telecommunications (DECT), Wireless Universal Serial Bus (Wireless USB), Fast Low Latency Access Orthogonal Frequency Division Multiplexing with Seamless Handover (Fast OFDM), IEEE 802.20, General Packet Radio Service (GPRS), iBurst, Wireless Broadband (WiBro), WiMAX, WiMAX Advanced, Universal Mobile Telecommunications Service-Time Division Duplex (UMTS-TDD), High Speed ​​Packet Access (HSPA), Evolution Data Optimized (EVDO), Long Term Evolution Advanced (LTE-Advanced), Multi-channel Multipoint Distribution Service (MMDS), etc.).

[0073] The embodiments of the subject matter and operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer storage medium to be executed by a data processing device or to control the operation of the data processing device. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical signal, optical signal, or electromagnetic signal) that is generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium can be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access storage array or device, or a combination thereof. In addition, although the computer storage medium is not a propagation signal, the computer storage medium can be the source or destination of the computer program instructions encoded in the artificially generated propagation signal. The computer storage medium can also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices). Furthermore, the operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0074] Although this specification may contain many specific implementation details, the implementation details should not be interpreted as limiting the scope of any claimed subject matter, but rather should be interpreted as descriptions of specific features of particular embodiments. Specific features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination, and even initially claimed as such, in some cases, one or more features from that combination may be deleted from the claimed combination, and the claimed combination may involve sub-combinations or variations of sub-combinations.

[0075] Similarly, although operations are described in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, in order to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged in multiple software products.

[0076] Thus, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0077] As those skilled in the art will appreciate, the innovative concepts described herein can be modified and varied in a wide range of applications. Therefore, the scope of the claimed subject matter should not be limited to any specific exemplary teachings discussed above, but is instead defined by the appended claims.

Claims

1. A pixel of an image sensor, comprising: a resistive microbolometer sensor portion that outputs a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion; a visible light image sensor portion that outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion; as well as An output path shared by the resistive microbolometer sensor portion and the visible light image sensor portion is controlled to selectively output a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion, a signal corresponding to a visible light image sensed by the visible light image sensor portion, or a fused image based on the infrared image sensed by the resistive microbolometer sensor portion and the visible light image sensed by the visible light image sensor portion to peripheral components for correlated double sampling, a column amplifier, and / or a column analog-to-digital converter.

2. The pixel according to claim 1, wherein The resistive microbolometer sensor section does not use a bias current.

3. The pixel according to claim 2, wherein: The visible light image sensor portion includes a photodiode.

4. The pixel according to claim 1, wherein The pixel corresponds to a pixel in a pixel array.

5. The pixel according to claim 4, wherein The pixel array includes one or more additional pixels, each additional pixel including: a resistive microbolometer sensor portion that outputs a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion of the additional pixel; a visible light image sensor portion that outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion of the additional pixel; and An output path is shared by the resistive microbolometer sensor portion and the visible light image sensor portion of the additional pixel, and the output path is controlled to selectively output a signal corresponding to the infrared image of the additional pixel, a signal corresponding to the visible light image of the additional pixel, or a fused image based on the infrared image and the visible light image of the additional pixel. The pixel according to claim 1 , wherein: The one output path is also shared with the visible light image sensor portion of each of the at least one additional pixel, the visible light image sensor portion of each of the at least one additional pixel outputting a signal corresponding to a visible light image sensed by the visible light image sensor portion of the additional pixel, and Wherein, the one output path is further controlled to selectively output a signal corresponding to the infrared image, a signal corresponding to the visible light image of the visible light image sensor portion of the pixel, a fused image based on the infrared image and the visible light image of the pixel, or a signal corresponding to the visible light image of each of the at least one additional pixel.

7. The pixel according to claim 6, wherein: The one output path is shared by two visible light image sensor portions.

8. The pixel according to claim 6, wherein The one output path is shared by four visible light image sensor sections or eight visible light image sensor sections.

9. The pixel according to any one of claims 1 to 8, wherein: The resistance of the resistive microbolometer sensor portion changes based on the amount of infrared energy absorbed by the resistive microbolometer sensor portion.

10. The pixel according to claim 9, wherein: The resistive microbolometer sensor section senses near-infrared images.

11. The pixel according to claim 9, wherein The resistive microbolometer sensor section senses long-wave infrared images.

12. An image sensor comprising a pixel array, the pixel array comprising: at least one first pixel, each first pixel comprising: a resistive microbolometer sensor portion and a visible light image sensor portion, the resistive microbolometer sensor portion outputting a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion, and the visible light image sensor portion outputting a signal corresponding to a visible light image sensed by the visible light image sensor portion of the first pixel; at least one second pixel, each second pixel including a visible light image sensor portion that outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion of the second pixel; and An output path is shared by the resistive microbolometer sensor portion of each of the at least one first pixel, the visible light image sensor portion of each of the at least one first pixel, and the visible light image sensor portion of each of the at least one second pixel, the one output path being controlled to selectively output a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion of each of the at least one first pixel, a signal corresponding to a visible light image sensed by the visible light image sensor portion of each of the at least one first pixel, a fused image based on the infrared image sensed by the resistive microbolometer sensor portion of each of the at least one first pixel and the visible light image sensed by the visible light image sensor portion of each of the at least one first pixel, or a signal corresponding to a visible light image sensed by the visible light image sensor portion of each of the at least one second pixel to peripheral components for correlated double sampling, a column amplifier, and / or a column analog-to-digital converter.

13. The image sensor according to claim 12, wherein: The resistive microbolometer sensor section does not use a bias current.

14. The image sensor according to claim 13, wherein: The resistive microbolometer sensor portion of each first pixel is shared by two visible light image sensor portions of the first pixel, four visible light image sensor portions of the first pixel, or eight visible light image sensor portions of the first pixel.

15. The image sensor according to claim 13, wherein: The one output path is shared by the resistive microbolometer sensor portion and the four visible light image sensor portions.

16. The image sensor according to claim 13, wherein: The one output path is shared by the resistive microbolometer sensor portion and the eight visible light image sensor portions.

17. A pixel of an image sensor, comprising: a resistive microbolometer sensor portion that outputs a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion; a visible light image sensor portion that outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion, the visible light image sensor portion including a photodiode; as well as An output path is shared by the resistive microbolometer sensor portion and the visible light image sensor portion, and is controlled to selectively output a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion, a signal corresponding to a visible light image sensed by the visible light image sensor portion, or a fused image of the infrared image sensed by the resistive microbolometer sensor portion and the visible light image sensed by the visible light image sensor portion based on the pixel to peripheral components for correlated double sampling, a column amplifier, and / or a column analog-to-digital converter.

18. The pixel according to claim 17, wherein: The pixel corresponds to a pixel in a pixel array.

19. The pixel according to claim 18, wherein The pixel array includes one or more additional pixels, each additional pixel including: a resistive microbolometer sensor portion that outputs a signal corresponding to an infrared image sensed by the resistive microbolometer sensor portion of the additional pixel; a visible light image sensor portion that outputs a signal corresponding to a visible light image sensed by the visible light image sensor portion of the additional pixel; and A second output path is shared by the resistive microbolometer sensor portion of the additional pixel and the visible light image sensor portion of the additional pixel, and the second output path is controlled to selectively output a signal corresponding to the infrared image sensed by the resistive microbolometer sensor portion of the additional pixel, a fused image based on the infrared image and visible light image of the additional pixel, or a signal corresponding to the visible light image sensed by the visible light image sensor portion of the additional pixel.

20. The pixel according to claim 17 or 18, wherein: The one output path is shared with at least one additional pixel.

Images