Image sensors, pixels, and methods for manipulating pixels

By introducing a discrete photodiode structure and a capacitor-controlled transistor into the image sensor, and employing high conversion gain and low conversion gain mode switching, the problems of insufficient dynamic range and signal-to-noise ratio of the image sensor are solved, achieving efficient image capture under different illumination conditions and improving the overall performance of the image sensor.

CN114727038BActive Publication Date: 2026-11-17SAMSUNG ELECTRONICS CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202111657756.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-01-04
Filing Date
2021-12-31
Publication Date
2026-11-17
Estimated Expiration
2041-12-31

AI Technical Summary

Technical Problem

Existing image sensors have shortcomings in terms of dynamic range and signal-to-noise ratio (SNR), especially in effectively capturing images with high dynamic range under different illumination conditions.

Method used

By introducing a discrete photodiode structure into the image sensor, combined with a capacitor and a capacitor-controlled transistor, and employing switching between high conversion gain (HCG) and low conversion gain (LCG) modes, signal readout is optimized under different illumination conditions. The capacitance of the floating diffusion node is increased to adjust the conversion gain.

Benefits of technology

This improves the dynamic range and signal-to-noise ratio (SNR) of the image sensor under different illumination conditions, enabling efficient image capture in both low-light and high-light environments and enhancing the overall performance of the image sensor.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114727038B_ABST
    Figure CN114727038B_ABST
Patent Text Reader

Abstract

An image sensor includes a pixel array in which a plurality of pixels are arranged and a row driver for controlling the plurality of pixels. Each of the plurality of pixels includes a first photodiode, a second photodiode having a light-receiving area larger than that of the first photodiode, a first floating diffusion node storing charge generated by the first photodiode, a first capacitor connected to the first floating diffusion node, and a capacitor control transistor having one end connected in series with the first capacitor. For each of the plurality of pixels, during the readout cycle of the first photodiode, for each of a plurality of preset operating modes, the row driver adjusts the capacitance of the first floating diffusion node by using the capacitor control transistor.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2021-0000444, filed with the Korean Intellectual Property Office on January 4, 2021, the disclosure of which is incorporated herein by reference in its entirety. Background Technology

[0003] This disclosure relates to image sensors, pixels, and methods for operating pixels, and more specifically, to image sensors, pixels in image sensors, and methods for operating pixels that ensure dynamic range by using capacitors in split photodiode (PD) pixels and capacitor control transistors.

[0004] An image sensor is a device that captures two-dimensional or three-dimensional images of an object. Image sensors generate images of objects by using photoelectric conversion elements that react to the intensity of light reflected from the object. With the recent development of Complementary Metal-Oxide-Semiconductor (CMOS) technology, CMOS image sensors are widely used. Recently, to increase the dynamic range of image sensors, techniques for adding capacitors to pixels have been developed. Summary of the Invention

[0005] This disclosure provides an image sensor in which the signal-to-noise ratio (SNR) is improved by controlling a capacitor connected to a small photodiode to increase the dynamic range in a discrete photodiode structure; pixels in the image sensor; and a method for manipulating the pixels.

[0006] According to one aspect of this disclosure, an image sensor includes: a pixel array in which a plurality of pixels are arranged; and a row driver for controlling the plurality of pixels, wherein each of the plurality of pixels includes: a first photodiode; a second photodiode having a light-receiving area larger than that of the first photodiode; a first floating diffusion node in which charge generated by the first photodiode is stored; a first capacitor connected to the first floating diffusion node; and a capacitor control transistor including a capacitor control transistor connected in series to one end of the first capacitor, wherein for each of the plurality of pixels, the row driver adjusts the capacitance of the first floating diffusion node by using the capacitor control transistor, and wherein the row driver adjusts the capacitance for each of a plurality of preset operating modes during a first readout cycle of the first photodiode.

[0007] According to one aspect of this disclosure, a pixel having a discrete photodiode structure includes: a first photodiode; a second photodiode having a light-receiving area larger than that of the first photodiode; a first floating diffusion node in which charge generated by the first photodiode is stored; a first photodiode transfer transistor including one end connected to the first photodiode and the other end connected to the first floating diffusion node; a second floating diffusion node in which charge generated by the second photodiode is stored; a second photodiode transfer transistor including one end connected to the second photodiode and the other end connected to the second floating diffusion node; a first capacitor connected to the first floating diffusion node; a capacitor control transistor including one end connected to the first capacitor and the other end connected to a first capacitor power supply unit, the capacitor control transistor adjusting the capacitance of the first floating diffusion node; and a switching transistor turned on during readout operation of the first photodiode.

[0008] According to one aspect of this disclosure, a method of operating a pixel includes: during a first readout cycle of a first photodiode, turning off a capacitor control transistor in a first time period, the capacitor control transistor being connected in series to a first capacitor connected to a first floating diffusion node; operating the pixel in a high conversion gain (HCG) mode when the capacitor control transistor is turned off, wherein the conversion gain of the first floating diffusion node is relatively high in the high conversion gain mode; turning on the capacitor control transistor in a second time period of the first readout cycle; and operating the pixel in a low conversion gain (LCG) mode when the capacitor control transistor is turned on, wherein the conversion gain of the first floating diffusion node is relatively low in the low conversion gain mode, wherein during the first time period, the illuminance of the pixel signal corresponding to the first photodiode is less than a preset illuminance, and wherein during the second time period, the illuminance of the pixel signal corresponding to the first photodiode is greater than or equal to the preset illuminance.

[0009] According to one aspect of this disclosure, an image sensor includes: a plurality of pixels, at least one of the plurality of pixels including: a first photodiode; a second photodiode; a first charge storage region where charge generated by the first photodiode is stored; a second charge storage region where charge generated by the second photodiode is stored; a first transistor configured to adjust the capacitance of the first charge storage region; and a second transistor configured to adjust the capacitance of the second charge storage region, wherein the image sensor further includes a control circuit configured to control the first transistor and the second transistor based on pixel illumination. Attached Figure Description

[0010] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, wherein:

[0011] Figure 1 This is a block diagram of an image sensor according to an embodiment;

[0012] Figure 2 This is a circuit diagram illustrating an example of an implementation of pixels according to an embodiment;

[0013] Figures 3A to 3D This is a diagram illustrating the operation of pixels according to an embodiment;

[0014] Figure 4A and 4B This is a graph showing the signal-to-noise ratio (SNR) of pixels according to an embodiment;

[0015] Figure 5A and 5B This is a diagram illustrating the separate photodiode structure of the pixel according to an embodiment;

[0016] Figure 6 This is a timing diagram illustrating the operation of an image sensor according to an embodiment;

[0017] Figure 7 This is a flowchart illustrating the operation of pixels according to an embodiment;

[0018] Figure 8A This is an exploded perspective view of the image sensor according to an embodiment. Figure 8B yes Figure 8A A planar view of the pixel array of the image sensor, and Figure 8C yes Figure 8A A plan view of the logic circuitry of an image sensor;

[0019] Figure 9 This is a block diagram of an electronic device including a multi-camera module according to an embodiment; and

[0020] Figure 10 yes Figure 9 A detailed block diagram of the camera module in an electronic device. Detailed Implementation

[0021] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0022] It should be understood that when a component or layer is described as being "over," "above," "on," "below," "under," "below," "connected to," or "attached to" another component or layer, it can be directly above, above, above, directly below, below, connected to, or attached to the other component or layer, or there can be intermediate components or layers. In contrast, when a component is described as being directly "over," "above," "below," "below," "below," or directly "connected to" or "attached to" another component or layer, there are no intermediate components or layers. The same number always refers to the same component.

[0023] Spatially relative terms, such as “directly above,” “above,” “over,” “up,” “directly below,” “below,” “under,” “down,” etc., may be used herein to describe the relationship of an element or feature as shown in the figures to one or more other elements or features. It will be understood that, in addition to the orientations depicted in the figures, spatially relative terms are intended to cover different orientations of the device in use or operation. For example, if the device in the figures is flipped, an element described as “below” or “under” other elements or features will be oriented “above” other elements or features. Thus, the term “below” can include both above and below orientations. The device may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially relative descriptive terms used herein are interpreted accordingly.

[0024] For the sake of brevity, this article may or may not describe the conventional components of semiconductor devices in detail.

[0025] Figure 1 This is a block diagram of an image sensor 100 according to an example embodiment.

[0026] Image sensor 100 can be mounted on electronic devices with image or light sensing capabilities. For example, image sensor 100 can be mounted on electronic devices such as cameras, smartphones, wearable devices, Internet of Things (IoT) devices, home appliances, tablet PCs, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, drones, and advanced driver assistance systems (ADAS). Furthermore, image sensor 100 can be mounted on electronic devices provided as components in vehicles, furniture, manufacturing facilities, doors, and various measuring devices.

[0027] refer to Figure 1The image sensor 100 may include a pixel array 110, a row driver 120, a readout circuit 130, a ramp signal generator 140, a timing controller 150, and a signal processor 190. The readout circuit 130 may include an analog-to-digital converter (ADC) circuit 131 and a data bus 132.

[0028] The pixel array 110 includes a plurality of pixels PX arranged in rows and columns, and a plurality of row lines RL and a plurality of column lines CL connected to the plurality of pixels PX.

[0029] Each of the multiple row lines RL can extend in the row direction and connect to a pixel PX arranged in the same row. For example, each of the multiple row lines RL can transmit control signals (e.g., signals RS, STS, LTS, SEL, CGS, SWS, and CCS described below) output from the row driver 120 to each transistor of the pixel circuit, such as... Figure 2 As shown.

[0030] Each of the plurality of pixels PX according to the example embodiment may include at least one photoelectric conversion element (or light sensing element). The photoelectric conversion element can sense light and convert the sensed light into photocharge. For example, the photoelectric conversion element may be a light sensing element comprising organic or inorganic materials, such as an inorganic photodiode, an organic photodiode, a perovskite photodiode, a phototransistor, a photogate, or a pinned photodiode. In the embodiment, each of the plurality of pixels PX may include a plurality of photoelectric conversion elements.

[0031] Microlenses for converging light can be arranged above each of the plurality of pixels PX or above each group of pixels including adjacent pixels PX. Each of the plurality of pixels PX can detect light within a specific spectral range from the light received through the microlenses. For example, pixel array 110 may include red pixels for converting light within the red spectral range into electrical signals, green pixels for converting light within the green spectral range into electrical signals, and blue pixels for converting light within the blue spectral range into electrical signals. Color filters for transmitting light within a specific spectral range may be arranged on each of the plurality of pixels PX. However, this disclosure is not limited thereto, and pixel array 110 may include pixels that convert light within spectral ranges other than the red, green, and blue spectral ranges into electrical signals.

[0032] In this embodiment, each of the plurality of pixels PX may have a multi-layer structure. A pixel with a multi-layer structure may include a plurality of stacked photoelectric conversion elements that convert light in different spectral ranges into electrical signals, and electrical signals corresponding to different colors may be generated by the plurality of photoelectric conversion elements. In other words, electrical signals corresponding to multiple colors can be output from a single pixel PX.

[0033] An array of color filters for transmitting light within a specific spectral range can be arranged above multiple pixels PX, and the detectable color of a corresponding pixel can be determined based on the color filters arranged above each of the multiple pixels. However, this disclosure is not limited thereto. In embodiments, a specific photoelectric conversion element can convert light within a specific wavelength band into an electrical signal based on the level of the electrical signal applied to the photoelectric conversion element.

[0034] A pixel PX can be a pixel PX with a discrete photodiode structure, which includes at least two photodiodes configured to be exposed to one or more light bursts from a light source, wherein the at least two photodiodes can be exposed or blanked independently of each other. For example, such as Figure 2 As shown, a pixel PX can include a small photodiode (SPD) with a small light-receiving area and a large photodiode (LPD) with a larger light-receiving area than the small photodiode. The large photodiode (LPD) provides the low-light pixel signal. The large photodiode (LPD) can operate in high conversion gain (HCG) mode during the lowest light-light period and in low conversion gain (LCG) mode during general low-light periods. The small photodiode (SPD) can operate during high-light periods with extended exposure times. During the highest light-light period, sensitivity can be reduced to maintain a long exposure time, and a large-capacity capacitor can be used to store a large amount of charge.

[0035] Photodiodes can be operated using either single-exposure or multiple-exposure methods. While multiple-exposure methods offer advantages in dynamic range or SNR characteristics, motion artifacts or LED flickering can occur due to the synthesis of images with different exposure times.

[0036] In the embodiments, the pixel PX can use a single exposure method with a split photodiode structure, and for some illumination periods, a combination of multiple exposure methods and single exposure methods can be used.

[0037] For example, a pixel PX can use a single-exposure method, in which a large photodiode (LPD) is used to read out the pixel signal during a first time period for reading out the pixel signal corresponding to the lowest illuminance, and a small photodiode (LPD) is used to read out the pixel signal during a second time period for reading out the pixel signal corresponding to illuminance higher than that in the first time period. Alternatively, a pixel PX can use a multiple-exposure method, in which additional exposure is performed using either a large photodiode (LPD) or a small photodiode (SPD) to read out the pixel signal corresponding to illuminance higher than that in the second time period, thereby ensuring dynamic range.

[0038] Each of the multiple column lines CL can extend in the column direction and can be connected to the pixels PX arranged in the same column. Each of the multiple column lines CL can transmit the reset signal and sensing signal of the pixels PX of the pixel array 110 in rows to the readout circuit 130.

[0039] The timing controller 150 can control the timing of the row driver 120, the read circuit 130, and the ramp signal generator 140. The timing controller 150 can provide timing signals indicating the operating timing to each of the row driver 120, the read circuit 130, and the ramp signal generator 140.

[0040] The row driver 120, under the control of the timing controller 150, generates control signals for driving the pixel array 110 and provides these control signals to each of the plurality of pixels PX in the pixel array 110 via multiple row lines RL. The row driver 120 can control the plurality of pixels PX in the pixel array 110 to sense incident light simultaneously or row-wise. Furthermore, the row driver 120 can select pixels PX from the plurality of pixels PX row-wise and can control the selected pixels PX (e.g., pixels in a row) to output reset and sensing signals via multiple column lines CL.

[0041] The line driver 120 can output control signals used to control pixel signals (e.g., see below for reference). Figure 2 The described signals RS, STS, LTS, SEL, CGS, SWS, and CCS are transmitted to the pixel array 110, and the pixel PX can output a pixel signal by operating in response to a control signal. In this case, the pixel signal may include a sensing signal and a reset signal. In an embodiment, the row driver 120 can generate control signals for controlling the pixel PX to operate continuously during the readout cycle in the HCG mode (i.e., LPD-H mode) of the large photodiode LPD, the LCG mode (i.e., LPD-L mode) of the large photodiode LPD, the HCG mode (i.e., SPD-H mode) of the small photodiode SPD, and the LCG mode (i.e., SPD-L mode) of the small photodiode SPD, and can provide control signals to the pixel array 110. Reference will be made later. Figure 2 Figure 4 describes the control of pixel PX in detail.

[0042] The ramp signal generator 140 can generate a ramp signal RAMP that increases or decreases with a specific slope, and can provide the ramp signal RAMP to the ADC circuit 131 of the readout circuit 130.

[0043] The readout circuit 130 can read the reset signal and the sensing signal from the pixels PX of a row selected from a plurality of pixels PX by the row driver 120. The readout circuit 130 can convert the reset signal and the sensing signal received from the pixel array 110 through a plurality of column lines CL into digital data based on the ramp signal RAMP from the ramp signal generator 140, and thus can generate and output pixel values ​​corresponding to the plurality of pixels PX on a row-by-row basis.

[0044] The ADC circuit 131 may include a plurality of ADCs corresponding to a plurality of column lines CL, and each of the plurality of ADCs may compare each of a reset signal and a sensing signal received through the corresponding column line CL with a ramp signal RAMP and generate a pixel value based on the comparison result. For example, the ADC may remove the reset signal from the sensing signal and generate a pixel value indicating the amount of light sensed by pixel PX.

[0045] The multiple pixel values ​​generated by the ADC circuit 131 can be output as image data IDT via the data bus 132. For example, the image data IDT can be provided to an image signal processor inside or outside the image sensor 100.

[0046] Data bus 132 can temporarily store pixel values ​​output from ADC circuit 131, and then output the temporarily stored pixel values. Data bus 132 may include multiple column memories and column decoders. Multiple pixel values ​​stored in multiple column memories can be output as image data IDT under the control of the column decoders.

[0047] The ADC circuit 131 may include multiple correlated double sampling (CDS) circuits and multiple counter circuits. The ADC circuit 131 can convert pixel signals (e.g., pixel voltages) input from the pixel array 110 into pixel values ​​as digital signals. Each pixel signal received through each of the multiple column lines CL is converted into a pixel value, which is a digital signal, by the CDS circuit and the counter circuits.

[0048] The CDS circuit compares the pixel signal received via column line CL with the ramp signal RAMP and outputs the comparison result. When the level of the ramp signal RAMP is the same as the level of the pixel signal, the CDS circuit can output a comparison signal that transitions from a first level (e.g., logic high) to a second level (e.g., logic low). The timing of the level transition of the comparison signal can be determined based on the level of the pixel signal.

[0049] The CDS circuit can sample and hold the pixel signal provided from the pixel PX according to the CDS method, and can double sample a specific noise level (e.g., a reset signal) and a level according to the image signal (e.g., a sensing signal), and generate a comparison signal based on the level corresponding to the difference between the reset signal and the sensing signal.

[0050] In an embodiment, the CDS circuit may include one or more comparators. For example, each comparator may be implemented as an operational transconductance amplifier (OTA) (or a differential amplifier).

[0051] The ADC circuit 131 may include multiple delta-reset sampling (DRS) circuits. The DRS circuits can sample pixel signals provided by first reading out pixel signals and then reading out reset signals according to the DRS method.

[0052] In the pixel array 110 according to an embodiment, pixel PX may have dual conversion gain. Dual conversion gain includes LCG and HCG. Conversion gain (the unit of conversion gain may be μV / e) refers to... Figure 2 The rate at which the charge accumulated in the floating diffusion nodes FD1, FD2, or FD3 is converted into voltage. The charge generated by the photoelectric conversion element can be transferred and accumulated in the floating diffusion nodes, and the charge accumulated in the floating diffusion nodes can be converted into voltage according to the conversion gain. In this case, the conversion gain can vary depending on the capacitance of the floating diffusion node. When its capacitance increases, the conversion gain decreases, and when its capacitance decreases, the conversion gain increases.

[0053] The pixel PX can operate in either LCG mode with high floating diffusion node capacitance or HCG gain mode with low floating diffusion node capacitance. Even though the charge accumulated in the floating diffusion node FD is the same in HCG gain mode as in LCG gain mode, the voltage of the floating diffusion node FD is higher in HCG gain mode than in LCG gain mode. See below for further details. Figure 2 Describe in detail the configuration of pixel PX and the operation of the conversion gain mode based on pixel PX.

[0054] For example, in the first image data generated when the large photodiode LPD in the plurality of pixels PX of pixel array 110 operates in HCG mode (i.e., LPD-H mode), for example, during a first time period when the illuminance is less than a first value, the first region (the darkest region) can be clearly displayed. In the second image data generated when the large photodiode LPD in the plurality of pixels PX of pixel array 110 operates in LCG mode (i.e., LPD-L mode), for example, during a second time period when the illuminance is greater than or equal to the first value and less than the second value, the second region (the region brighter than the first region) can be clearly displayed. In the third image data generated when the small photodiode SPD in the plurality of pixels PX of pixel array 110 operates in HCG mode (i.e., SPD-H mode), for example, during a third time period when the illuminance is greater than or equal to the second value and less than the third value, the third region (the region brighter than the second region but darker than the fourth region described below) can be clearly displayed. In the fourth image data generated when the small photodiodes SPD in the multiple pixels PX of the pixel array 110 operate in LCG mode (i.e., SPD-L mode), for example, during a fourth time period when the illuminance is greater than or equal to the third value, the fourth region (the brightest region) can be clearly displayed.

[0055] In this embodiment, during a frame in which the pixel array 110 is scanned, each of the plurality of pixels PX can operate continuously in LPD-H mode, LPD-S mode, SPD-H mode, and SPD-L mode during the readout period. Therefore, first to fourth image data (corresponding to LPD-H mode, LPD-S mode, SPD-H mode, and SPD-L mode) can be generated within one frame period. The first to fourth image data can be merged to create a one-shot image with high dynamic range, in which bright areas (high-illuminance areas) and dark areas (low-illuminance areas) are clearly defined.

[0056] In an embodiment, the image sensor 100 may further include a bias current controller for increasing or decreasing the bias current during the readout cycle when the switching mode of the pixel PX changes (e.g., from LCG mode to HCG mode or vice versa). In this case, multiple bias current controllers may be connected to multiple column lines CL, respectively. The bias current changes as the switching mode of the pixel PX changes, and the amount of the bias current increases or decreases depending on whether the pixel PX is operating in LCG or HCG mode. For example, when the pixel PX is operating in HCG mode, the amount of the bias current may be less than when the pixel PX is operating in LCG mode.

[0057] The signal processor 190 can perform noise reduction, gain adjustment, waveform shaping, interpolation, white balance, gamma adjustment, edge enhancement, and binning on image data. In an embodiment, when the pixel array 100 operates in HCG mode (i.e., LPD-H mode), LCG mode (i.e., LPD-L mode) of a large photodiode LPD, HCG mode (i.e., SPD-H mode) of a small photodiode SPD, and LCG mode (i.e., SPD-L mode) of a small photodiode SPD within a frame period, the signal processor 190 can receive first image data according to LPD-H mode, second image data according to LPD-L mode, third image data according to SPD-H mode, and fourth image data according to SPD-L mode from the data bus 132, and can generate an image with high dynamic range by binning the first to fourth image data. In an embodiment, the signal processor 190 can be located in a processor external to the image sensor 100.

[0058] Figure 2 This is a circuit diagram illustrating an example implementation of a pixel PX according to an example embodiment.

[0059] A pixel PX may include multiple photodiodes, such as a small photodiode SPD and a large photodiode LPD. A pixel PX may include multiple transistors, such as a transfer transistor STG for the small photodiode SPD, a transfer transistor LTG for the large photodiode LPD, a reset transistor RG, a drive transistor DX, a select transistor SX, a gain control transistor DRG (or switching gain control transistor), a switching transistor SW, and a capacitor control transistor CCTR; and a capacitor C1.

[0060] The voltage applied to pixel PX can include pixel voltage VPIX, capacitor power supply voltage VMIM, and reset power supply voltage VRD. Each of the capacitor power supply voltage VMIM and reset power supply voltage VRD can be supplied in the same manner as pixel voltage VPIX or can be supplied by a separate circuit.

[0061] Capacitors such as parasitic capacitors can be formed from floating diffusion nodes FD1, FD2, and FD3 (i.e., charge storage regions). Capacitor C1 can be a passive element with fixed or variable capacitance, or it can be a capacitor used to adjust the capacitance of the floating diffusion node FD1.

[0062] Photodiodes (i.e., large photodiodes (LPDs) and small photodiodes (SPDs) convert externally incident light into electrical signals. Photodiodes generate charge based on light intensity. The amount of charge generated by a photodiode varies depending on the image shooting environment (low or high illumination). For example, a photodiode may generate enough charge in a high-illumination environment to reach its full-well capacity (FWC), but not in a low-illumination environment. Based on their relative light-receiving area, photodiodes can be classified into large photodiodes (LPDs) and small photodiodes (SPDs).

[0063] One end of the transfer transistor STG can be connected to the small photodiode SPD, and the other end can be connected to the floating diffusion node FD1. Therefore, the charge accumulated in the small photodiode SPD can be stored in the floating diffusion node FD1. One end of the switching transistor SW can be connected to the floating diffusion node FD1, and the other end can be connected to the floating diffusion node FD2. The switching transistor SW can be turned on when the small photodiode SPD is used, and turned off when the large photodiode LPD is used. Capacitor C1 can be connected in parallel with the floating diffusion node FD1 to increase the capacitance of the floating diffusion node FD1. One end of the capacitor control transistor CCTR can be connected to capacitor C1, and the other end can be connected to the capacitor power supply voltage VMIM.

[0064] One end of the transfer transistor LTG can be connected to the large photodiode LPD, and the other end can be connected to the floating diffusion node FD3. Therefore, the charge accumulated in the large photodiode LPD can be stored in the floating diffusion node FD3. One end of the reset transistor RG can be connected to the reset power supply voltage VRD, and the other end can be connected to the floating diffusion node FD2. Therefore, when a reset control signal RS is applied to the gate of the reset transistor RG, the pixel PX can be reset. One end of the gain control transistor DRG can be connected to the floating diffusion node FD2, and the other end can be connected to the floating diffusion node FD3. Therefore, the conversion gain can be adjusted during the operation of the large photodiode LPD.

[0065] The transfer transistor STG for small photodiode SPD, the transfer transistor LTG for large photodiode LPD, the reset transistor RG, the drive transistor DX, the select transistor SX, the gain control transistor DRG, the switch transistor SW, and the capacitor control transistor CCTR can operate in response to control signals provided from the row driver 120, such as the reset control signal RS, the transfer control signals STS and LTS, the select signal SEL, the gain control signal CGS, the switch signal SWS, and the capacitor control signal CCS, respectively.

[0066] The reset transistor RG can be turned on in response to a reset control signal RS applied to its gate terminal, and can reset floating diffusion nodes FD1, FD2, and FD3 based on the reset supply voltage VRD. In this case, the switching transistor SW can be turned on based on a gain control signal SWS received at its gate terminal, so the reset supply voltage VRD can be applied to the floating diffusion node FD1, and the floating diffusion node FD1 can be reset.

[0067] The transfer transistor STG can be turned on in response to a transfer control signal STS applied to its gate terminal, and can transfer the charge generated by the small photodiode SPD to the floating diffusion node FD1. The transferred charge can be accumulated (i.e., stored) in the floating diffusion node FD1. In other words, the charge can accumulate in the capacitor formed in the floating diffusion node FD1.

[0068] The transfer transistor LTG can be turned on in response to a transfer control signal LTS applied to its gate terminal, and can transfer the charge generated by the large photodiode LPD to the floating diffusion node FD3. The charge can accumulate (i.e., be stored) in a capacitor formed in the floating diffusion node FD3. In this case, when the gain control transistor DRG is turned on, the charge can also accumulate in a capacitor formed in the floating diffusion node FD2. In other words, when the gain control transistor DRG is turned on, the capacitors formed in the floating diffusion node FD3 and the floating diffusion node FD2 can be connected in parallel to increase the capacitance.

[0069] The driving transistor DX can operate as a source follower based on the bias current IL generated by the current source CS connected to the column line CL, and the output voltage corresponding to the voltage of the floating diffusion node can be selected by the transistor SX.

[0070] Select transistor SX can select pixel PX. Select transistor SX can be turned on in response to the select signal SEL applied to its gate terminal and can output the voltage (or current) from drive transistor DX to column line CL.

[0071] During the operation of a large photodiode (LPD), the gain control transistor (DRG) can be turned on or off based on the gain control signal (CGS) received at its gate terminal. The conversion gain when the DRG is off can be higher than the conversion gain when the DRG is on. When the DRG is off, it is referred to as LPD-H mode, and when the DRG is on, it is referred to as LPD-L mode.

[0072] During the operation of the large photodiode LPD of pixel PX, the large photodiode LPD can operate in one of LPD-H mode and LPD-L mode according to the on and off states of the gain control transistor DRG. In LPD-H mode, because the conversion gain of pixel PX increases, the gain of the circuitry used to process the voltage output from pixel PX (e.g., ADC circuit 131) is relatively lower compared to LPD-L mode. Therefore, Figure 1 The SNR of the image sensor 100 increases, thus reducing the minimum detectable light intensity and improving the low-intensity detection performance of the image sensor 100. In LPD-L mode, because the capacitance of the floating diffusion node of pixel PX is relatively large, the full-well capacity (FWC) can be increased. Therefore, the high-intensity detection performance of the image sensor 100 can be improved.

[0073] During the operation of a small photodiode (SPD), the capacitor control transistor (CCTR) can be turned on or off based on the capacitor control signal (CCS) received at its gate terminal, and the conversion gain when the CCTR is off can be higher than the conversion gain when the CCTR is on. (See later...) Figures 3A to 3D This describes the detailed principle of how the conversion gain is changed based on the on / off state of the capacitor control transistor (CCTR). When the CCTR is in the off state, it is called SPD-H mode, and when the CCTR is in the on state, it is called SPD-L mode. In SPD-H mode, the increased conversion gain improves the image sensor's surface-to-noise ratio (SNR).

[0074] In this way, the pixel PX can sense low-intensity light and high-intensity light by using large and small photodiodes LPD and SPD with dual conversion gain, thus expanding (or increasing) the dynamic range of the image sensor 100. For example, during the readout of the large photodiode LPD, the large photodiode LPD can operate in HCG mode and LCG mode, which are set by using the gain control transistor DRG. During the readout of the small photodiode SPD, the small photodiode SPD can operate in HCG mode and LCG mode, which are set by using the capacitor control transistor CCTR.

[0075] In addition, as mentioned above Figure 1As described, during the readout period, pixel PX can continuously operate in the HCG mode (i.e., LPD-H mode) of the large photodiode LPD, the LCG mode (i.e., LPD-L mode) of the large photodiode LPD, the HCG mode (i.e., SPD-H mode) of the small photodiode SPD, and the LCG mode (i.e., SPD-L mode) of the small photodiode SPD. Furthermore, in sensor 100, for example, Figure 1 The signal processor 190 can generate an image with high dynamic range by merging all images according to each mode. In particular, in the HCG mode of the small photodiode (i.e., SPD-H mode), the conversion gain can be increased by controlling the transistor CCTR by the cutoff capacitor, and the SNR of the image sensor 100 can be improved by reducing the capacitance of the floating diffusion node FD1.

[0076] Figures 3A to 3D This is a diagram illustrating the operation of pixels according to an example embodiment. Specifically, Figures 3A to 3D It is shown Figure 2 A diagram showing a partial configuration of the pixel PX. Figure 3B This is a diagram illustrating the SPD-L operation of pixel PX, and Figure 3C and 3D This is a diagram illustrating the SPD-H operation of pixel PX.

[0077] refer to Figure 2 and Figure 3A Pixel PX may include a capacitor control transistor (CCTR) connected to one end of capacitor C1, which is used to improve the dynamic range of the small photodiode (SPD). During the readout operation of the SPD, pixel PX can reduce or increase the conversion gain of the SPD by turning the capacitor control transistor (CCTR) on or off.

[0078] refer to Figure 3B In SPD-L mode, the capacitor control transistor CCTR can be turned on by the capacitor control signal CCS. Therefore, the capacitance formed in the floating diffusion node FD1 is formed by the parallel connection of the parasitic capacitance of the floating diffusion node FD1 and the capacitor C1. Furthermore, the capacitance of capacitor C1 can be greater than the capacitance of the parasitic capacitance of the floating diffusion node FD1. Therefore, when the capacitor control transistor CCTR is turned on, the capacitance of the floating diffusion node FD1 can be similar to the capacitance of capacitor C1.

[0079] Capacitor C1 can increase the capacitance of the floating diffusion node FD1. In this case, the capacitance of the floating diffusion node FD1 can include the capacitance of capacitor C1 connected in series with the floating diffusion node FD1 or the parasitic capacitance of the capacitor control transistor CCTR.

[0080] refer to Figure 3C In SPD-H mode, the capacitor control transistor CCTR can be turned off by the capacitor control signal CCS, and in this case, the capacitor control transistor CCTR can operate as a parasitic capacitor C_tr. Therefore, the capacitance formed in the floating diffusion node FD1 is formed by the series connection of capacitor C1 and parasitic capacitor C_tr. Because the capacitance of parasitic capacitor C_tr is smaller than the capacitance of capacitor C1, the capacitance of the floating diffusion node FD1 when the capacitor control transistor CCTR is turned off can be substantially similar to the capacitance of parasitic capacitor C_tr.

[0081] refer to Figure 3D When the capacitance of the parasitic capacitor C_tr is significantly smaller than that of the capacitor C1, the capacitor of the floating diffusion node FD1 in SPD-H mode can act as the parasitic capacitor C_tr, which has a capacitance smaller than that of the capacitor C1. Therefore, in SPD-H mode, the conversion gain of the small photodiode SPD can be increased.

[0082] As will be referred to later Figure 6 In SPD-H mode, a CDS readout method that first outputs a reset signal and then a sensing signal can be used. When using the CDS readout method, noise caused by leakage components from floating diffusion nodes can be removed, and pixel / ADC noise components can be largely preserved. Pixel / ADC noise components decrease with increasing conversion gain, and therefore, the SNR characteristics in SPD-H mode can be improved. While SNR characteristics may degrade when the sensitivity of the small photodiode SPD is reduced to improve dynamic range, in SPD-H mode, the sensitivity can be adjusted by increasing the capacitance as much as by increasing the conversion gain of the small photodiode SPD to improve SNR characteristics.

[0083] Figure 4A and 4B This is a diagram illustrating the SNR of pixels according to an example embodiment.

[0084] The dynamic range of a pixel (PX) can be increased by increasing the capacitance of the floating diffusion node or decreasing the sensitivity of the photodiode. Increasing the capacitance of the floating diffusion node degrades the SNR characteristics due to a proportional increase in noise components. Decreasing the sensitivity of the photodiode degrades the SNR characteristics due to a proportional decrease in signal components.

[0085] According to an embodiment, the pixel PX can extend its dynamic range by connecting a capacitor C1 to the floating diffusion node FD1, which accumulates the charge of the small photodiode SPD in SPD-L mode. Furthermore, the pixel PX can increase its SNR by reducing the capacitance of the floating diffusion node FD1 in SPD-H mode by turning off the capacitor control transistor CCTR connected to capacitor C1. For example, see reference... Figure 4A When the gain of the capacitor-controlled transistor CCTR cutoff increases in SPD-H mode, the SNR characteristic can be improved to n1 within the corresponding period.

[0086] Furthermore, according to the embodiments, the pixel PX can use a small photodiode SPD with low sensitivity to further increase the dynamic range. For example, reference Figure 4B When using a small photodiode (SPD) with low sensitivity, the dynamic range increases by d in SPD-L mode. In this case, the SNR characteristic deteriorates by n2 due to the use of the small photodiode with low sensitivity, but the deterioration of the SNR characteristic can be reduced by using a capacitor-controlled transistor (CCTR).

[0087] In other words, reference Figure 3A The pixel structure shown demonstrates that the conversion gain in SPD-H mode can be prevented from affecting the capacitance of capacitor C1 designed for SPD-L mode by using a capacitor control transistor (CCTR). Therefore, even when increasing the dynamic range in SPD-L mode by increasing the capacitance of capacitor C1, the impact on the SNR in SPD-H mode can be reduced. Thus, considering the permissible SNR level in SPD-L mode, the dynamic range can be improved by increasing the capacitance of capacitor C1.

[0088] Figure 5A and 5B This is a diagram illustrating the separate photodiode structure of a pixel according to an example embodiment.

[0089] refer to Figure 5A The pixel array 110 may include a plurality of pixels PX arranged in rows and columns in a plan view. Each of the plurality of pixels PX may include a photoelectric conversion section and a pixel circuit section. The photoelectric conversion section may vertically overlap the pixel circuit section.

[0090] The photoelectric conversion section may include microlenses, color filters, and photoelectric conversion elements. In some embodiments, the photoelectric conversion section may further include a protective layer and an insulating structure. Microlenses may be disposed on the photoelectric conversion element and may be configured to converge externally incident light and allow the converged light to enter the photoelectric conversion element.

[0091] Photoelectric conversion elements can include large photodiodes (LPDs) and small photodiodes (SPDs). In this case, such as... Figure 5A As shown, on multiple pixels (PX), large photodiodes (LPDs) can be arranged in a wide area, while small photodiodes (SPDs) can be arranged in a narrow area.

[0092] refer to Figure 5B Large photodiodes (LPDs) have a large light-receiving area, thus they can saturate faster than small photodiodes (SPDs). Therefore, large LPDs can be used in low-light areas, while small SPDs can be used in high-light areas.

[0093] Color filters can selectively transmit light signals of a specific wavelength band and can be located between the microlens and the photoelectric conversion element. The pixel circuitry can be formed below the photoelectric conversion element and may include floating diffusion nodes, source / drain electrodes, load resistors, gates, via contacts, and wiring structures.

[0094] Figure 6 This is a timing diagram illustrating the operation of an image sensor according to an example embodiment.

[0095] Figure 6 It is from Figure 2 A timing diagram of an image sensor during pixel readout (PX). During one readout cycle (or horizontal cycle), signals can be read from the pixel array (e.g., ...). Figure 1 Multiple voltages (or multiple currents) are read from multiple pixels PX in at least one row of the pixel array 110.

[0096] Let's refer to each other. Figure 2 and Figure 6 Throughout the readout cycle, the select signal SEL can be at an active level, such as logic high, and the select transistor SX can be turned on in response to the select signal SEL, thus allowing pixel PX to be connected to column line CL. In this case, the active level of the signal refers to the level at which the transistor applying the signal can be turned on. In this embodiment, it is assumed that logic high represents an active level and logic low represents an inactive level.

[0097] A readout cycle can include multiple cycles, and each of the multiple cycles can correspond to each of multiple different readout modes, depending on the photodiode and conversion gain of the output pixel signal. For example, refer to Figure 6 The multiple readout modes may include LPD-H, LPD-L, SPD-H, and SPD-L modes. Each of the multiple readout modes may include a signal period S for outputting a sense signal and a reset period R for outputting a reset signal.

[0098] In LPD-H mode, a reset signal based on the reset period R can be output first, followed by a sensing signal based on the signal period S. For example, refer to... Figure 6 During the reset period R in LPD-H mode, in response to the active level of the reset control signal RS, the reset transistor RG is turned on and the floating diffusion node FD3 is reset. During the reset period R, the reset signal corresponding to the reset level of pixel PX (e.g., the voltage of the already reset floating diffusion node FD3) can be read. In this case, because the large photodiode LPD of pixel PX operates in HCG mode, the reset signal indicating the reset level can be read in the HCG mode of the large photodiode LPD.

[0099] In LPD-H mode, during signal period S, the large photodiode LPD of pixel PX can operate in HCG mode and can read out a sensing signal corresponding to the signal level of pixel PX. When the transfer control signal LTS transitions to an active level (e.g., logic high), the charge generated by the large photodiode LPD can be transferred to and stored in the floating diffusion node FD3. The driving transistor DX can output a sensing signal based on the voltage of the floating diffusion node FD3 according to the amount of charge transferred from the large photodiode LPD.

[0100] In LPD-H mode, the gain control signal CGS can be at logic low, the switching signal SWS can be at logic low, the transfer control signal STS can be at logic low, and the capacitor control signal CCS can be at logic high.

[0101] When the gain control signal CGS transitions to an active level (e.g., logic high), the gain control transistor DRG can be turned on, and the pixel PX can switch from LPD-H mode to LPD-L mode.

[0102] In LPD-L mode, a sense signal based on signal period S can be output first, followed by a reset signal based on reset period R. For example, refer to... Figure 6 In LPD-L mode, when the transfer control signal LTS transitions to an active level (e.g., logic high), the charge generated by the large photodiode LPD during signal period S can be transferred to and stored in the floating diffusion node FD3. The driving transistor DX can output a sensing signal based on the voltage of the floating diffusion node FD3, according to the amount of charge transferred from the large photodiode LPD.

[0103] In SPD-H mode, a reset signal based on the reset period R can be output first, followed by a sensing signal based on the signal period S. For example, refer to... Figure 6During the reset period R in SPD-H mode, in response to the active level of the reset control signal RS, the reset transistor RG can be turned on and the floating diffusion node FD1 can be reset. During the reset period R, the reset signal corresponding to the reset level of pixel PX (e.g., the voltage of the already reset floating diffusion node FD1) can be read. In this case, the switch signal SWS, which transitions to logic high, can be applied to the gate of the switch transistor SW of pixel PX, and the capacitor control signal CCS, which transitions to logic low, can be applied to the gate of the capacitor control transistor CCTR. The switch signal SWS transitions to logic high after the reset control signal RS transitions to logic low.

[0104] In SPD-H mode, a sensing signal corresponding to the signal level of pixel PX can be read out during signal period S. When the transfer control signal STS becomes active (e.g., logic high), the charge generated by the small photodiode SPD can be transferred to and stored in the floating diffusion node FD1. The driving transistor DX can output a sensing signal based on the voltage of the floating diffusion node FD1 according to the amount of charge transferred from the small photodiode SPD.

[0105] In this way, in SPD-H mode, the reset signal can be read out first during the reset period R, followed by the sensing signal during the signal period S. Furthermore, a signal from which noise for each pixel has been removed can be generated by comparing the two signals (i.e., the sensing signal and the reset signal). This readout method can be called the CDS readout method.

[0106] When the capacitor control signal CCS transitions to an active level (e.g., logic high), the capacitor control transistor CCTR can be turned on and the pixel PX can switch from SPD-H mode to SPD-L mode.

[0107] In SPD-L mode, a sensing signal based on signal period S can be output first, followed by a reset signal based on reset period R. For example, refer to... Figure 6 During the signal period S in SPD-L mode, a sensing signal corresponding to the signal level of pixel PX can be read out.

[0108] During the reset period R in SPD-L mode, in response to the active level of the reset control signal RS, the reset transistor RG can be turned on and the floating diffusion node FD1 can be reset. During the reset period R, the reset signal corresponding to the reset level of pixel PX can be read out.

[0109] In this way, in SPD-L mode, the sensing signal can be read out first during the signal period S, followed by the reset signal during the reset period R. Furthermore, a signal from which noise for each pixel has been removed can be generated by comparing the two signals (i.e., the reset signal and the sensing signal). This readout method can be called the DRS readout method.

[0110] The operational sequence of the various readout modes of a pixel PX (e.g., LPD-H, LPD-L, SPD-H, and SPD-L modes) is not limited to the examples described above. For instance, the readout operation of a small photodiode SPD can be performed before the readout operation of a large photodiode LPD, and the operation of an LCG mode can be performed before the operation of an HCG mode. Furthermore, in the readout operation of a pixel PX, the order of the reset period R and the signal period S can be changed in each mode, or the signal period S in both HCG and LCG modes can be performed after the reset period R in LCG mode and the reset period R in HCG mode are executed first. The large photodiode LPD can operate in only one conversion gain mode, while the small photodiode SPD can operate in two conversion gain modes.

[0111] In another embodiment, after pixel PX operates with a single exposure in which at least one of the modes LPD-H, LPD-L, SPD-H, and SPD-L is combined, an additional photographic exposure can be added to extend the dynamic range of the portion with very high illumination. For example, a small photodiode SPD can be used to add an image captured with a short exposure time. In this case, the exposure time and exposure interval for the added photographic exposure can be set by taking illumination information into account.

[0112] The pixel PX can operate independently of illuminance in LPD-H, LPD-L, SPD-H, and SPD-L modes, and can select the data portion to use based on illuminance information from the readout sensing signal. However, the order or method of operation is not limited to this.

[0113] Figure 7 This is a flowchart illustrating the operation of pixels according to an example embodiment.

[0114] refer to Figure 7 The operations described for the pixels can be performed by Figure 2 The pixel PX is executed. (See reference) Figure 7The capacitance of the first floating diffusion node can be reduced by controlling the transistor via a cutoff capacitor during a first time period of the readout cycle of the first photodiode (operation S110). The first photodiode can be a small photodiode with a small light-receiving area. The first floating diffusion node storing the charge generated by the first photodiode can be connected in parallel to the first capacitor. The capacitor control transistor can be connected in series to the first capacitor to control the first capacitor.

[0115] When the capacitor control transistor is off during the first time period, a reset signal can be applied to the reset transistor to reset the pixel. When the pixel is reset, the switching transistor can be turned on and the reset signal can be transferred to the readout circuit. Furthermore, a transfer control signal can be applied to the first photodiode transfer transistor, and the sensing signal can be transferred to the readout circuit (operation S120).

[0116] During the second time period of the first photodiode's readout cycle, the capacitance of the first floating diffusion node can be increased by controlling the transistor through the on-conductor capacitor (operation S130).

[0117] When the capacitor control transistor is turned on in the second time period, the sensing signal can be transferred to the readout circuit. Then, a reset signal can be applied to the reset transistor to reset the pixel. When the pixel is reset, the reset signal can be transferred to the readout circuit (operation S140).

[0118] The pixel operation method may further include the following steps: during the readout cycle of the second photodiode, the pixel operates in HCG mode and LCG mode, wherein the gain control transistor is turned off in HCG mode and turned on in LCG mode. The second photodiode may be a photodiode having a larger light-receiving area than the first photodiode.

[0119] In this scenario, the first time period can be the period during which pixel signals with illuminance less than a preset illuminance are read out during the illuminance cycle of the first photodiode's operation, and the second time period can be the period during which pixel signals with illuminance equal to or greater than the preset illuminance are read out during the illuminance cycle of the first photodiode's operation. In other words, the first time period can be the period during which the first photodiode operates and the illuminance of the pixels during that period is less than the preset illuminance, and the second time period can be the period during which the first photodiode operates and the illuminance of the pixels during that period is greater than or equal to the preset illuminance.

[0120] Figure 8A This is an exploded perspective view of the image sensor 100a according to an example embodiment. Figure 8B This is a plan view of the image sensor 100a, and Figure 8C yes Figure 8AA plan view of the logic circuitry of the image sensor.

[0121] refer to Figure 8A , 8B And 8C, the image sensor 100a may have a structure in which a first chip CH1 and a second chip CH2 are stacked. Figure 1 The pixel core (e.g., at least one photoelectric conversion element and pixel circuitry) of each of the plurality of pixels PX in the pixel array 110 can be formed in the first chip CH1, and the logic circuitry, for example, Figure 1 The row driver 120, readout circuit 130, ramp signal generator 140 and timing controller 150 can be formed in the second chip CH2.

[0122] like Figure 8B and Figure 8C As shown, the first chip CH1 and the second chip CH2 may each include an active region AA and a logic region LA respectively arranged at the center of the first chip CH1 and the center of the second chip CH2, and may also include a peripheral region PERR and a peripheral region PEI respectively arranged around the edges of the first chip CH1 and the edges of the second chip CH2. In the active region AA of the first chip CH1, photoelectric conversion elements and pixel circuits can be arranged in a two-dimensional array structure. The logic circuits can be arranged in the logic region LA of the second chip CH2.

[0123] Vias extending in the third direction (Z direction) can be arranged in the peripheral region PERR of the first chip CH1 and the peripheral region PEI of the second chip CH2. The first chip CH1 and the second chip CH2 can be electrically connected to each other through the vias. Wiring lines and vertical contacts extending in the first direction (X direction) or the second direction (Y direction) can be further formed in the peripheral region PERR of the first chip CH1. Multiple wiring lines extending in the first direction (X direction) and the second direction (Y direction) can also be arranged in the wiring layer of the second chip CH2, and these wiring lines can be connected to logic circuits.

[0124] Although a structure in which the first chip CH1 and the second chip CH2 are electrically connected to each other through vias has been described, this disclosure is not limited thereto. For example, the first chip CH1 and the second chip CH2 can be implemented with various connection structures, such as copper (Cu)-Cu bonding, connection between vias and Cu pads, connection between vias and external connection terminals, and connection through an integral via.

[0125] Figure 9 This is a block diagram of an electronic device 1000 including a multi-camera module according to an example embodiment. Figure 10 yes Figure 9 Detailed block diagram of the camera module in the electronic device 1000.

[0126] refer to Figure 9 The electronic device 1000 may include a camera module group 1100, an application processor 1200, a power management integrated circuit (PMIC) 1300, and an external memory 1400.

[0127] Camera module group 1100 may include multiple camera modules 1100a, 1100b, and 1100c. Although Figure 9 Three camera modules 1100a, 1100b, and 1100c are shown, but this disclosure is not limited thereto. In some embodiments, camera module group 1100 may be modified to include only two camera modules. In some embodiments, camera module group 1100 may be modified to include “k” camera modules, where “k” is any natural number of 4 or greater.

[0128] The following will refer to Figure 10 This describes the detailed configuration of camera module 1100b. The following description also applies to other camera modules 1100a and 1100c.

[0129] refer to Figure 10 The camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140, and a storage device 1150.

[0130] The prism 1105 may include a reflective surface 1107 of reflective material and may change the path of light L incident from the outside.

[0131] In some embodiments, prism 1105 can change the path of light L incident in the first direction (X direction) to a second direction (Y direction) perpendicular to the first direction (X direction). Prism 1105 can rotate the reflective surface 1107 of the reflective material about its central axis 1106 in direction A, or rotate the central axis 1106 in direction B, to change the path of light L incident in the first direction (X direction) to a second direction (Y direction) perpendicular to the first direction (X direction). In this case, OPFE 1110 can move in a third direction (Z direction) perpendicular to the first direction (X direction) and the second direction (Y direction).

[0132] In some embodiments, such as Figure 10 As shown, the maximum rotation angle of prism 1105 in direction A can be less than or equal to 15 degrees in the positive (+)A direction and greater than 15 degrees in the negative (-)A direction. However, this disclosure is not limited thereto.

[0133] In some embodiments, prism 1105 can move by an angle of about 20 degrees in the positive or negative B direction, or within a range of about 10 degrees to about 20 degrees or from about 15 degrees to about 20 degrees. In this case, the angle by which prism 1105 moves in the positive B direction can be the same as or similar to the angle by which prism 1105 moves in the negative B direction, differing by about 1 degree.

[0134] In some embodiments, the prism 1105 can move the reflective surface 1107 of the reflective material in a third direction (Z direction) parallel to the extension direction of the central axis 1106.

[0135] OPFE 1110 may include, for example, "m" optical lenses, where "m" is a natural number. These "m" lenses can be moved in a second direction (Y direction) and change the optical zoom ratio of camera module 1100b. For example, when the default optical zoom ratio of camera module 1100b is Z, the optical zoom ratio of camera module 1100b can be changed to 3Z, 5Z, or greater by moving the "m" optical lenses included in OPFE 1110.

[0136] Actuator 1130 can move OPFE 1110 or optical lens to a specific position. For example, actuator 1130 can adjust the position of optical lens so that image sensor 1142 is located at the focal length of optical lens for accurate sensing.

[0137] Image sensing device 1140 may include image sensor 1142, control logic 1144, and memory 1146. Image sensor 1142 can sense an image of an object using light L provided through an optical lens. Image sensor 1142 can generate image data with a high range of motion by merging HCG image data and LCG image data.

[0138] Control logic 1144 can control all operations of camera module 1100b. For example, control logic 1144 can control the operation of camera module 1100b according to control signals provided through control signal line CSLb.

[0139] The memory 1146 may store information required for the operation of the camera module 1100b, such as calibration data 1147. Calibration data 1147 may include information required by the camera module 1100b to generate image data using light L provided externally. For example, calibration data 1147 may include information about rotation, focal length, optical axis, etc. When the camera module 1100b is implemented as a multi-state camera where the focal length varies with the position of the optical lens, calibration data 1147 may include the focal length value for each position (or state) of the optical lens and information about autofocus.

[0140] Storage device 1150 can store image data sensed by image sensor 1142. Storage device 1150 can be disposed externally to image sensing device 1140 and can be stacked with the sensor chip of image sensing device 1140. In some embodiments, although storage device 1150 may include electrically erasable programmable read-only memory (EEPROM), this disclosure is not limited thereto.

[0141] refer to Figure 9 and 10 In some embodiments, each of camera modules 1100a, 1100b, and 1100c may include a corresponding actuator 1130. Therefore, camera modules 1100a, 1100b, and 1100c may include calibration data 1147, which may be the same or different among camera modules 1100a, 1100b, and 1100c depending on the operation of the corresponding actuator 1130 included in each of the camera modules 1100a, 1100b, and 1100c.

[0142] In some embodiments, one of camera modules 1100a, 1100b, and 1100c (e.g., camera module 1100b) may be a folding lens type including prism 1105 and OPFE 1110, while the other camera modules (e.g., camera modules 1100a and 1100c) may be a vertical type excluding prism 1105 and OPFE 1110. However, this disclosure is not limited thereto.

[0143] In some embodiments, one of camera modules 1100a, 1100b, and 1100c (e.g., camera module 1100c) may include a vertical depth camera that extracts depth information using infrared (IR). In this case, application processor 1200 can generate a three-dimensional (3D) depth image by merging image data provided from the depth camera (e.g., camera module 1100c) with image data provided from another camera module (e.g., camera module 1100a or 1100b).

[0144] In some embodiments, at least two camera modules (e.g., 1100a and 1100b) of camera modules 1100a, 1100b, and 1100c may have different fields of view. In this case, for example, two camera modules (e.g., 1100a and 1100b) of camera modules 1100a, 1100b, and 1100c may each have different optical lenses. However, this disclosure is not limited thereto.

[0145] In some embodiments, camera modules 1100a, 1100b, and 1100c may have different fields of view from each other. In this case, although camera modules 1100a, 1100b, and 1100c may each have different optical lenses, this disclosure is not limited thereto.

[0146] In some embodiments, camera modules 1100a, 1100b, and 1100c may be physically separated from each other. In other words, the sensing area of ​​image sensor 1142 is not divided and used by camera modules 1100a, 1100b, and 1100c, but rather the respective image sensor 1142 may be independently included in each of camera modules 1100a, 1100b, and 1100c.

[0147] Return to reference Figure 9 The application processor 1200 may include an image processing unit 1210, a memory controller 1220, and internal memory 1230. The application processor 1200 may be implemented separately from the camera modules 1100a, 1100b, and 1100c. For example, the application processor 1200 and the camera modules 1100a, 1100b, and 1100c may be implemented in different semiconductor chips.

[0148] The image processing unit 1210 may include a plurality of sub-image processors 1212a, 1212b and 1212c; an image generator 1214 and a camera module controller 1216.

[0149] The image processing unit 1210 may include as many sub-image processors 1212a, 1212b and 1212c as camera modules 1100a, 1100b and 1100c.

[0150] Multiple image data lines generated by camera modules 1100a, 1100b, and 1100c can be provided to sub-image processors 1212a, 1212b, and 1212c respectively via separate image signal lines ISLa, ISLb, and ISLc. For example, image data generated by camera module 1100a can be provided to sub-image processor 1212a via image signal line ISLa, image data generated by camera module 1100b can be provided to sub-image processor 1212b via image signal line ISLb, and image data generated by camera module 1100c can be provided to sub-image processor 1212c via image signal line ISLc. Such image data transmission can be performed using, for example, a Camera Serial Interface (CSI) of a Mobile Industrial Processor Interface (MIPI). However, this disclosure is not limited thereto.

[0151] In some embodiments, a single sub-image processor can be provided for multiple camera modules. For example, sub-image processors 1212a and 1212c can be integrated into a single sub-image processor instead of being separate, and image data provided from camera module 1100a or camera module 1100c can be selected by a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor.

[0152] Image data provided to each of the sub-image processors 1212a, 1212b, and 1212c can be provided to the image generator 1214. The image generator 1214 can generate an output image based on image generation information or a pattern signal using the image data provided from each of the sub-image processors 1212a, 1212b, and 1212c.

[0153] Specifically, the image generator 1214 can generate an output image by merging at least some portions of multiple image data generated by camera modules 1100a, 1100b, and 1100c, which have different fields of view, based on image generation information or a pattern signal. Alternatively, the image generator 1214 can generate an output image by selecting one of the image data generated by camera modules 1100a, 1100b, and 1100c, which have different fields of view, based on image generation information or a pattern signal.

[0154] In some embodiments, image generation information may include a zoom signal or zoom factor. In some embodiments, the mode signal may be based on a user-selected mode.

[0155] When the image generation information includes a zoom signal or zoom coefficient and camera modules 1100a, 1100b, and 1100c have different fields of view, image generator 1214 can perform different operations based on different types of zoom signals. For example, when the zoom signal is a first signal, image generator 1214 can merge image data output from camera module 1100a and image data output from camera module 1100c, but not merge image data output from camera module 1100b, and then generate an output image using the merged image signal and the image data output from camera module 1100b. When the zoom signal is a second signal different from the first signal, image generator 1214 can generate an output image by selecting one of the multiple image data output from camera modules 1100a, 1100b, and 1100c respectively, instead of performing a merge. However, this disclosure is not limited to this, and the method of processing image data can be changed whenever necessary.

[0156] In some embodiments, the image generator 1214 may receive multiple image data with different exposure times from at least one of the sub-image processors 1212a, 1212b and 1212c and perform high dynamic range (HDR) processing on the multiple image data to generate merged image data with increased dynamic range.

[0157] The camera module controller 1216 can provide control signals to each of the camera modules 1100a, 1100b, and 1100c. The control signals generated by the camera module controller 1216 can be provided to a corresponding one of the camera modules 1100a, 1100b, and 1100c via a corresponding one of the separate control signal lines CSLa, CSLb, and CSLc.

[0158] One of camera modules 1100a, 1100b, and 1100c (e.g., camera module 1100b) can be designated as the master camera based on a mode signal or an image generation signal including a zoom signal, while other camera modules (e.g., camera modules 1100a and 1100c) can be designated as slave cameras. Such designation information can be included in control signals and provided to each of camera modules 1100a, 1100b, and 1100c via a corresponding one of the separate control signal lines CSLa, CSLb, and CSL.

[0159] The camera module operating as a master or slave can be changed based on the zoom ratio or operating mode signal. For example, when the field of view of camera module 1100a is larger than that of camera module 1100b and the zoom ratio indicates a low zoom ratio, camera module 1100a can operate as a master, while camera module 1100b can operate as a slave. Conversely, when the zoom ratio indicates a high zoom ratio, camera module 1100b can operate as a master, while camera module 1100a can operate as a slave.

[0160] In some embodiments, the control signals provided from the camera module controller 1216 to each of the camera modules 1100a, 1100b, and 1100c may include a synchronization enable signal. For example, when camera module 1100b is the main camera and camera module 1100a is the slave camera, the camera module controller 1216 may send a synchronization enable signal to camera module 1100b. Camera module 1100b, which provides the synchronization enable signal, may generate a synchronization signal based on the synchronization enable signal and provide the synchronization signal to camera modules 1100a and 1100c via the synchronization signal line SSL. Camera modules 1100a, 1100b, and 1100c may be synchronized using the synchronization signal and may send image data to the application processor 1200.

[0161] In some embodiments, control signals provided from camera module controller 1216 to each of camera modules 1100a, 1100b, and 1100c may include mode information based on mode signals. Camera modules 1100a, 1100b, and 1100c may operate in a first operating mode or a second operating mode related to sensing speed based on the mode information.

[0162] In the first operating mode, camera modules 1100a, 1100b, and 1100c can generate image signals at a first speed (e.g., at a first frame rate), encode the image signals at a second speed higher than the first speed (e.g., at a second frame rate higher than the first frame rate), and send the encoded image signals to application processor 1200. In this case, the second speed can be 30 times or less than the first speed.

[0163] Application processor 1200 can store the received image signal (i.e., the encoded image signal) in its internal memory 1230 or in external memory 1400. Subsequently, application processor 1200 can read the encoded image signal from internal memory 1230 or external memory 1400, decode the encoded image signal, and display image data generated based on the decoded image signal. For example, one of the sub-image processors 1212a, 1212b, and 1212c of image processing unit 1210 can perform decoding and image processing on the decoded image signal.

[0164] In the second operating mode, camera modules 1100a, 1100b, and 1100c can generate image signals at a third speed (e.g., a third frame rate lower than the first frame rate) and send the image signals to application processor 1200. The image signals provided to application processor 1200 may not be encoded. Application processor 1200 may perform image processing on the image signals or store the image signals in internal memory 1230 or external memory 1400.

[0165] PMIC 1300 can supply power, such as power supply voltage, to each of camera modules 1100a, 1100b, and 1100c. For example, under the control of application processor 1200, PMIC 1300 can supply a first power to camera module 1100a via power signal line PSLa, a second power to camera module 1100b via power signal line PSLb, and a third power to camera module 1100c via power signal line PSLc.

[0166] PMIC 1300 can generate and adjust the power level corresponding to each of camera modules 1100a, 1100b, and 1100c in response to a power control signal PCON from application processor 1200. The power control signal PCON can include a power adjustment signal for each operating mode of camera modules 1100a, 1100b, and 1100c. For example, the operating mode can include a low-power mode. In this case, the power control signal PCON can include information about the camera module operating in low-power mode and the power level to be set. The same or different power levels can be provided to camera modules 1100a, 1100b, and 1100c respectively. The power level can be dynamically changed.

[0167] While this disclosure has been specifically shown and described with reference to embodiments thereof, it should be understood that various changes in form and detail may be made without departing from the spirit and scope of the appended claims.

Claims

1. An image sensor, comprising: A pixel array, in which multiple pixels are arranged; and A row driver is used to control the plurality of pixels. Each of the plurality of pixels includes: First photodiode; A second photodiode having a larger light-receiving area than the first photodiode; The first floating diffusion node, wherein the charge generated by the first photodiode is stored in the first floating diffusion node; A first capacitor connected to the first floating diffusion node; Includes a capacitor control transistor connected in series to one end of the first capacitor. A first photodiode transfer transistor is connected between the first photodiode and the first floating diffusion node; A switching transistor is connected between the first floating diffusion node and the second floating diffusion node; and A reset transistor is connected between the second floating diffusion node and the reset power supply voltage; Specifically, for each of the plurality of pixels, the row driver adjusts the capacitance of the first floating diffusion node by using the capacitor control transistor, and The row driver adjusts the capacitance for each of a plurality of preset operating modes during the first readout cycle of the first photodiode.

2. The image sensor of claim 1, wherein, The row driver reduces the capacitance of the first floating diffusion node by turning off the capacitor control transistor during a first time period of the first readout cycle, and During the first time period, the illuminance of the pixel signal corresponding to the first photodiode is less than the preset illuminance.

3. The image sensor according to claim 2, wherein The reset transistor includes one end connected to the second floating diffusion node and the other end to which a reset power supply voltage is applied. The row driver turns on the switching transistor during the first time period. The row driver resets the first floating diffusion node by turning on the reset transistor during the reset period of the first time period, and controls the reset signal to be output to the readout circuit. The reset signal corresponds to the voltage of the reset first floating diffusion node. In this process, the row driver turns on the first photodiode transfer transistor during the sensing period of the first time period to store the charge generated by the first photodiode in the first floating diffusion node, and controls the sensing signal to be output to the readout circuit. The sensing signal corresponds to the first floating diffusion node storing the charge.

4. The image sensor of claim 1, wherein, The row driver turns on the capacitor control transistor during a second time period of the first readout cycle, and stores the charge generated in the first photodiode in both the first floating diffusion node and the first capacitor. During the second time period, the illuminance of the pixel signal corresponding to the first photodiode is greater than or equal to a preset illuminance.

5. The image sensor according to claim 4, wherein, The reset transistor of each of the plurality of pixels resets the first floating diffusion node based on the reset power supply voltage, and During the second time period, the row driver controls the sensing signal to be output to the readout circuit. The sensing signal corresponds to the first floating diffusion node. The driver turns on the reset transistor to reset the first floating diffusion node and controls the reset signal to be output to the readout circuit. The reset signal corresponds to the voltage of the reset first floating diffusion node.

6. The image sensor according to claim 2, wherein, The sensitivity of the first photodiode to the light source is set based on the change in signal-to-noise ratio (SNR) that occurs during the first time period when the row driver reduces the capacitance of the first floating diffusion node.

7. The image sensor according to claim 1, wherein, Each of the plurality of pixels also includes: The gain control transistor is connected between the third floating diffusion node and the second floating diffusion node. The row driver controls each of the plurality of pixels to operate in either a high conversion gain (HCG) mode or a low conversion gain (LCG) mode during the second readout cycle of the second photodiode, wherein the gain control transistor is turned off in the high conversion gain mode and turned on in the low conversion gain mode.

8. The image sensor according to claim 7, wherein, For each of the plurality of pixels, the row driver, in HCG mode, turns off the gain control transistor, turns on the reset transistor to reset the pixel, controls the first reset signal to be sent to the readout circuit when the pixel is reset, turns on the second photodiode transfer transistor when the first reset signal is sent to the readout circuit and stores the charge generated by the second photodiode in the third floating diffusion node, and controls the first sensing signal to be output to the readout circuit, the first sensing signal corresponding to the third floating diffusion node storing the charge, and Specifically, for each of the plurality of pixels, the row driver turns on the gain control transistor in LCG mode, turns on the second photodiode transfer transistor to store the charge generated by the second photodiode in the third floating diffusion node, controls the second sensing signal to be output to the readout circuit, the second sensing signal corresponding to the third floating diffusion node with accumulated charge, and controls the second reset signal to be sent to the readout circuit when the second sensing signal is sent to the readout circuit.

9. The image sensor according to claim 1, wherein, The readout of the first photodiode uses at least one of correlated double sampling (CDS) readout, which samples the reset signal and then samples the sense signal, and incremental reset sampling (DRS) readout, which samples the sense signal and then samples the reset signal.

10. The image sensor according to claim 1, further comprising: Includes a multiple exposure function that allows for multiple exposures in a single photograph.

11. A pixel of an image sensor, the pixel comprising: First photodiode; A second photodiode having a larger light-receiving area than the first photodiode; The first floating diffusion node, wherein the charge generated by the first photodiode is stored in the first floating diffusion node; The first photodiode transfer transistor includes one end connected to the first photodiode and the other end connected to the first floating diffusion node; The second floating diffusion node, wherein the charge generated by the second photodiode is stored in the second floating diffusion node; The second photodiode transfer transistor includes one end connected to the second photodiode and the other end connected to the second floating diffusion node; A first capacitor is connected to the first floating diffusion node; A capacitor control transistor includes one end connected to the first capacitor and the other end connected to the first capacitor power supply unit, the capacitor control transistor adjusting the capacitance of the first floating diffusion node; and The switching transistor that is turned on during the readout operation of the first photodiode.

12. The pixel according to claim 11, wherein, During a first time period of the readout cycle of the first photodiode, the capacitor-controlled transistor is turned off to reduce the capacitance of the first floating diffusion node, and During the first time period, the illuminance of the pixel signal corresponding to the first photodiode is less than the preset illuminance.

13. The pixel according to claim 12, in, The switching transistor is connected between the first floating diffusion node and the third floating diffusion node. The pixel further includes a reset transistor, which has one end connected to the third floating diffusion node and the other end to which a reset power supply voltage is applied. Specifically, when the capacitor control transistor is off during the first time period, the pixel turns on the reset transistor according to the received reset control signal. When the pixel is reset by the turn on of the reset transistor, the switching transistor is turned on, and a reset signal is sent to the readout circuit. The pixel turns on the first photodiode transfer transistor to store the charge generated by the first photodiode in the first floating diffusion node, and outputs a sensing signal to the readout circuit. The sensing signal corresponds to the first floating diffusion node where the charge is stored.

14. The pixel according to claim 11, wherein, During the second time period of the readout cycle of the first photodiode, the capacitor controls the transistor to turn on, and During the second time period, the illuminance of the pixel signal corresponding to the first photodiode is greater than or equal to a preset illuminance.

15. The pixel of claim 14, further comprising: The reset transistor includes one end connected to the third floating diffusion node and the other end to which a reset power supply voltage is applied. When the capacitor control transistor is turned on during the second time period, the pixel outputs a sensing signal to the readout circuit, turns on the reset transistor according to the received reset control signal, and outputs a reset signal to the readout circuit when the pixel is reset by turning on the reset transistor.

16. A method for manipulating pixels, the method comprising: During the first readout cycle of the first photodiode, the capacitor control transistor is turned off during a first time period, the capacitor control transistor being connected in series with a first capacitor connected to the first floating diffusion node; When the capacitor control transistor is off, the pixel operates in a high conversion gain (HCG) mode, in which the conversion gain of the first floating diffusion node is relatively high. The capacitor control transistor is turned on during the second time period of the first readout cycle; and When the capacitor control transistor is turned on, the pixel operates in a low conversion gain (LCG) mode, in which the conversion gain of the first floating diffusion node is relatively low. During the first time period, the illuminance of the pixel signal corresponding to the first photodiode is less than a preset illuminance, and During the second time period, the illuminance of the pixel signal corresponding to the first photodiode is greater than or equal to a preset illuminance. The pixels include: A first photodiode transfer transistor is connected between the first photodiode and the first floating diffusion node; A switching transistor is connected between the first floating diffusion node and the second floating diffusion node; and A reset transistor is connected between the second floating diffusion node and the reset power supply voltage.

17. The method according to claim 16, wherein, Operating the pixel in HCG mode includes: When the capacitor control transistor is turned off during the first time period, a reset control signal is applied to the reset transistor to reset the pixel. When the pixel is reset, the switching transistor is turned on, and a reset signal is sent to the readout circuit; Apply the transfer control signal to the first photodiode transfer transistor; and The sensing signal is transmitted to the readout circuit.

18. The method according to claim 16, wherein, Operating the pixel in LCG mode includes: When the capacitor control transistor is turned on during the second time period, the sensing signal is sent to the readout circuit; A reset control signal is applied to the reset transistor to reset the pixel; and When the pixel is reset, a reset signal is sent to the readout circuit.

19. The method of claim 16, wherein, The pixel further includes a second photodiode having a larger light-receiving area than the first photodiode, and The method further includes operating the pixel in HCG mode or LCG mode during the second readout cycle of the second photodiode, wherein the gain control transistor is turned off in HCG mode and turned on in LCG mode.

Citation Information

Patent Citations

  • Head protector

    KR1020210000444A

  • Image sensor

    KR1020190089694A

  • Image sensor, pixel array, and operation method of the image sensor

    US20200007808A1