Image sensor

Abstract

To provide an image sensor for compensating the difference of image signals generated by the parallax of microlenses.SOLUTION: The image sensor includes a pixel array in which first, second, and third pixel groups each including a plurality of first, second, and third unit pixels and respectively corresponding to color filters of different colors are arranged in a matrix form, a readout circuit configured to output a plurality of image signals based on a plurality of pixel signals output from the pixel array, and an image signal processor configured to perform one or more image processes on the plurality of image signals and output an image-processed image signal, the image signal processor may change a target image signal corresponding to a target pixel in response to saturation of one or more first first unit pixels among the plurality of first unit pixels in the ROI, and the target pixel may be a second unit pixel among the plurality of second unit pixels or a third unit pixel among the plurality of third unit pixels.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor), and more particularly to an image sensor that compensates for differences in image signals caused by parallax of microlenses. [Background technology]

[0002] An image sensor is a device that converts optical signals into electrical signals. The image sensor utilizes an autofocusing (AF) function that automatically detects the focus. Phase difference autofocusing (PAF) adjusts the focal length based on the phase difference between optical signals detected at different positions.

[0003] The daily challenge for image sensors is to develop and improve them to enable faster autofocusing while increasing resolution. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,646,338 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above-mentioned problems with conventional image sensors, and an object of the present invention is to provide an image sensor that compensates for differences in image signals caused by parallax of microlenses. [Means for solving the problem]

[0006] In order to achieve the above object, an image sensor according to the present invention includes a pixel array in which first pixel groups, second pixel groups, and third pixel groups, each including a plurality of first unit pixels, a plurality of second unit pixels, and a plurality of third unit pixels, are arranged in a matrix shape, and each first pixel group, second pixel group, and third pixel group, each corresponding to a color filter of a different color, are arranged in a matrix shape; a readout circuit that outputs a plurality of image signals based on a plurality of pixel signals output from the pixel array; and an image signal processor that performs one or more image processing operations on the plurality of image signals and outputs the image-processed image signals, wherein the image signal processor changes a target image signal corresponding to a target pixel in response to saturation of one or more first unit pixels among the plurality of first unit pixels within a region of interest, and the target pixel is a second unit pixel among the plurality of second unit pixels or a third unit pixel among the plurality of third unit pixels.

[0007] In order to achieve the above object, an image sensor according to the present invention includes: a pixel array in which first pixel groups, second pixel groups, and third pixel groups, each including a plurality of first unit pixels, a plurality of second unit pixels, and a plurality of third unit pixels, are arranged in a matrix, and each first pixel group corresponds to a color filter of a different color from each other; a readout circuit that outputs a plurality of image signals based on the plurality of pixel signals output from the pixel array; and an image signal processor that generates a plurality of first image signals, a plurality of second image signals, and a plurality of third image signals by performing white balancing on the image signals, and changes at least one of the second image signals and the third image signals in response to saturation of one or more first unit pixels and desaturation of one or more second unit pixels among the first unit pixels, wherein the plurality of first image signals are based on pixel signals of the plurality of first unit pixels of the first pixel group, the plurality of second image signals are based on pixel signals of the plurality of second unit pixels of the second pixel group, and the plurality of third image signals are based on pixel signals of the plurality of third unit pixels of the third pixel group.

[0008] In order to achieve the above object, an image sensor according to the present invention includes a pixel array including a plurality of pixel groups arranged in a matrix shape; a readout circuit that outputs a plurality of image signals based on a plurality of pixel signals output by the pixel array; and an image signal processor that generates a plurality of first image signals, a plurality of second image signals, and a plurality of third image signals by performing white balancing on the image signals, and outputs a processed image signal obtained by modifying at least one of the plurality of second image signals and the plurality of third image signals based on one or more of the plurality of first image signals, the plurality of second image signals, and the plurality of third image signals reaching a saturation level, wherein the plurality of pixel groups include a first pixel group, a second pixel group, and a third pixel group associated with different color channels, and the plurality of first image signals, the plurality of second image signals, and the plurality of third image signals are based on the plurality of pixel signals of the first pixel group, the plurality of pixel signals of the second pixel group, and the plurality of third image signals, respectively, in that order. [Effects of the Invention]

[0009] According to the image sensor of the present invention, it is possible to compensate for differences in image signals caused by parallax of microlenses, to compensate for differences in image signals based on multiple pixels sharing a microlens, and to generate image signals in which color differences caused by parallax of microlenses are compensated. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of an image sensor according to an embodiment of the present invention; [Figure 2] FIG. 2 is a plan view showing a schematic configuration of a pixel unit according to an embodiment of the present invention. [Figure 3]10A and 10B are diagrams illustrating phases of unit pixels included in a pixel group according to an embodiment of the present invention; [Figure 4] FIG. 2 is a plan view showing a schematic configuration of a pixel group according to an embodiment of the present invention. [Figure 5] 5 is a cross-sectional view of a pixel group taken along line II' according to the embodiment of FIG. 4. [Figure 6] 5 is a cross-sectional view of a pixel group according to the embodiment of FIG. 4 taken along line II-II'. [Figure 7] FIG. 2 is a schematic circuit diagram of a pixel group according to an embodiment of the present invention. [Figure 8A] 4 is a graph for schematically illustrating a compensation process for an image signal according to an embodiment of the present invention; [Figure 8B] 4 is a graph for schematically illustrating a compensation process for an image signal according to an embodiment of the present invention; [Figure 8C] 4 is a graph for schematically illustrating a compensation process for an image signal according to an embodiment of the present invention; [Figure 8D] 4 is a graph for schematically illustrating a compensation process for an image signal according to an embodiment of the present invention; [Figure 9] 1 is a block diagram showing a schematic configuration of an image signal processor according to an embodiment of the present invention; [Figure 10A] 1 is a block diagram showing a schematic configuration of an image signal processor according to an embodiment of the present invention; [Figure 10B] 1 is a block diagram showing a schematic configuration of an image signal processor according to an embodiment of the present invention; [Figure 11] 1 is a block diagram showing a schematic configuration of a color channel compensation circuit of an image signal processor according to an embodiment of the present invention; [Figure 12] 4 is a diagram illustrating compensation for a color channel difference in a color channel compensation circuit according to an embodiment of the present invention; [Figure 13] 1 is a block diagram showing a schematic configuration of an IPO overflow compensation circuit of an image signal processor according to an embodiment of the present invention; [Figure 14] 10A and 10B are diagrams for explaining the operation of an IPO overflow compensation circuit according to one embodiment of the present invention. [Figure 15] 10A and 10B are diagrams for explaining the operation of an IPO overflow compensation circuit according to one embodiment of the present invention. [Figure 16] 10A and 10B are diagrams for explaining the operation of an IPO overflow compensation circuit according to one embodiment of the present invention. [Figure 17] 1 is a perspective view showing a schematic configuration of an image sensor according to an embodiment of the present invention; [Figure 18] 1 is a partial perspective view showing a schematic configuration of an image sensor according to an embodiment of the present invention; [Figure 19] 1 is a block diagram showing a schematic configuration of an electronic device according to an embodiment of the present invention; [Figure 20] 1 is a flowchart illustrating an operation method of an image sensor according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, specific examples of embodiments of the image sensor according to the present invention will be described with reference to the drawings. DETAILED DESCRIPTION OF THE INVENTION In the following, the embodiments of the present invention are described clearly and in detail to the extent that those skilled in the art can easily practice the present invention.

[0012] FIG. 1 is a block diagram showing a schematic configuration of an image sensor 100 according to an embodiment of the present invention. The image signal processor 160 of the image sensor 100 according to an embodiment of the present invention compensates for differences in image signals caused by disparity of the microlenses. An image sensor 100 according to an embodiment of the present invention will be described in detail with reference to FIGS.

[0013] The image sensor 100 according to the embodiment of the present invention outputs a pixel signal PXS for each unit pixel. Alternatively, in one embodiment, the image sensor 100 may output pixel signals PXS for each pixel group. In this specification, the term "unit pixel" may be used interchangeably with "pixel." The image sensor 100 generates image data, which is visual information of an object captured through a lens, and the image signal processor 160 processes the image signal received from the readout circuit 150 and transmits the processed image signal to an external display device and / or an external storage device via an output interface.

[0014] The image sensor 100 includes a pixel array 110 , a row driver 120 , a timing controller 130 , a ramp signal generator 140 , and a readout circuit 150 . The readout circuit 150 includes an analog-to-digital converter (ADC) and an output buffer. The pixel array 110 includes a plurality of pixel units PXUs. The pixel unit PXU includes a plurality of pixel groups PXG, which will be described with reference to FIG. The pixel group PXG includes a plurality of unit pixels PX. This is explained in more detail below with reference to FIG. In the description of FIG. 1, it is assumed that the pixel unit PXU includes a plurality of pixel groups PXG shown in FIG. 2 and / or that the pixel group PXG includes a plurality of pixels PX shown in FIG.

[0015] The pixel array 110 receives a number of pixel drive signals CS from the row driver 120, such as a select signal for controlling a select transistor, a reset signal for controlling a reset transistor, and a transfer transistor control signal for controlling a transfer transistor. Each of the plurality of pixel units PXU of the pixel array 110 operates under the control of a pixel drive signal CS received from the row driver 120 . A plurality of unit pixels PX included in each pixel unit PXU are operated under the control of a pixel driving signal CS received from the row driver 120 . A plurality of pixel units PXU are arranged, for example, in a matrix form. Each of the pixel groups PXG and / or each of the pixels PX included in the pixel unit PXU is electrically connected to a row line and a column line.

[0016] In one embodiment, each pixel group PXG is based on a multi-pixel structure including multiple photodiodes. A pixel group PXG based on a multi-pixel structure shares at least a part of a pixel circuit among a plurality of unit pixels PX. For example, each pixel group PXG includes a number of transistors controlled by a row driver 120 . The unit pixels PX included in the same pixel group PXG share at least some of the driving transistors, the selection transistors, and the reset transistors with each other.

[0017] The row driver 120 drives one or more rows of the pixel array 110 under the control of the timing controller 130 . In this specification, a "row" refers to a plurality of unit pixels PX located in a first direction (eg, horizontal direction) among the plurality of unit pixels PX of the pixel array 110. Also, the term "column" refers to a plurality of unit pixels PX located in a second direction (for example, a vertical direction) among the plurality of unit pixels PX included in the pixel array 110. The row driver 120 drives at least one of the rows. The row driver 120 generates a selection signal to drive at least one of the rows. The row driver 120 activates the unit pixels PX and / or pixel groups PXG corresponding to the selected row. The pixel signals PXS of the unit pixels PX and / or pixel groups PXG of the selected row are transmitted to the readout circuit 150 through a plurality of column output lines.

[0018] The pixel signal PXS is the voltage of the floating diffusion region. The pixel signal PXS is a voltage that reflects the charges generated in the photodiodes PD included in the plurality of unit pixels PX. Alternatively, the pixel signal PXS may be a reference voltage used to perform correlated double sampling (CDS) with a voltage that reflects the charge generated in the photodiode PD. The reference voltage may be the voltage of the floating diffusion region. For example, the reference voltage is the voltage of the floating diffusion region reset by the reset voltage.

[0019] The timing controller 130 controls the pixel array 110 , the row driver 120 , the ramp signal generator 140 , and the readout circuit 150 . The timing controller 130 provides a timing control signal (TC) to the row driver 120 . In one embodiment, the timing control signal TC may be set differently based on the operation mode of the image sensor 100 . For example, the image sensor 100 operates in a signal output mode for each pixel unit PX included in the pixel group PXG, or in a signal output mode for each pixel group PXG included in the pixel unit PXU. For example, the signal output mode for each pixel group PXG is a binning mode in which pixel signals of unit pixels PX included in the same pixel group PXG are combined and output. The operating mode of image sensor 100 may be selected by a user, set by control of an external processor, or set by control of image signal processor 160.

[0020] The row driver 120 drives each of the plurality of pixels PX in a normal shooting mode or an HDR mode based on the timing control signal TC. In one embodiment, when the image sensor 100 operates in an exposure time-based HDR mode, the row driver 120 drives each of the plurality of pixels PX to generate at least two types of pixel signals PXS with different exposure times. For example, the row driver 120 generates a first pixel signal corresponding to a first exposure time, and then controls each of the plurality of pixels PX to generate a second pixel signal corresponding to a second exposure time. The first exposure time and the second exposure time are different from each other. The first exposure time may be longer or shorter than the second exposure time.

[0021] The timing controller 130 controls the ramp signal generator 140 through a ramp control signal (CS_RP). The lamp control signal (CS_RP) may include a lamp enable signal, a mode signal, and the like. The ramp signal generator 140 generates a ramp signal RAMP in response to the ramp control signal (CS_RP). The ramp signal generator 140 generates a ramp signal RAMP having a preset slope. The ramp signal generator 140 provides the generated ramp signal RAMP to the ADC of the readout circuit 150 . The ADC of the readout circuit 150 outputs an image signal IDT, which is a digital signal, based on a pixel signal that is based on the ramp signal RAMP. For example, the ADC outputs the pixel signal PXS as the image signal IDT based on the ramp signal RAMP in a correlated double sampling manner. The image signal IDT is provided to an image signal processor 160 . The image signal IDT is an intensity value corresponding to the pixel signal PXS.

[0022] The image signal processor 160 according to the embodiment of the present invention compensates for the difference due to the parallax of the microlens in the image signal IDT output from the readout circuit 150 . For example, the image signal processor 160 compensates for the difference between the image signal IDT of one color channel and the image signal IDT of another color channel due to the parallax of the microlens. Alternatively, the image signal processor 160 compensates for the difference between the image signal IDT of the pixel PX located on one side of the pixel group PXG and the image signal IDT of the pixel PX located on the other side thereof due to the parallax of the microlenses. The image signal processor 160 modifies at least a portion of the image signal IDT that is not saturated in response to at least a portion of the image signal IDT reaching a saturation level. For example, the image signal processor 160 determines the saturation of the pixels PX located on one side of the pixel group PXG corresponding to any one color channel of the unit pixel PX. In this case, the image signal processor 160 modifies at least a portion of the image signals IDT of the pixel groups PXG corresponding to the other color channels. Depending on the embodiment, it may additionally or alternatively be possible to determine whether a portion of the image signal IDT of the pixel group PXG corresponding to another color channel has reached a saturation level.

[0023] FIG. 2 is a plan view showing a schematic configuration of a pixel unit PXU according to an embodiment of the present invention. The pixel unit PXU according to the embodiment of FIG. 2 corresponds to the pixel unit PXU of FIG. As described with reference to FIG. 1, the pixel units PXU according to the embodiment of FIG. 2 are arranged in a matrix form in the pixel array of the image sensor 100 of FIG. A pixel unit PXU according to an embodiment of the present invention includes a plurality of pixel groups PXG. The number of pixel groups PXG included in a pixel unit PXU is based on the pattern of the color filter. For example, in the case of an image sensor 100 having a Bayer color pattern, four pixel groups PXG are included in a pixel unit PXU. Depending on the embodiment, the pixel unit PXU may be based on a color pattern other than Bayer.

[0024] In one embodiment, color filters located in adjacent pixel groups PXG among the plurality of pixel groups PXG transmit light of different spectrums. That is, the colors of the color filters located in adjacent pixel groups PXG are different from each other. According to an embodiment, some of the pixel groups PXG of the pixel unit PXU correspond to different color patterns, and some correspond to the same color pattern. For example, if a pixel unit PXU includes four pixel groups PXG, the two first pixel groups correspond to the first color channel, and the second and third pixel groups correspond to the second and third color channels, respectively. Color filters of different colors are arranged for each color channel. For example, referring to FIG. 2, if the pixel unit PXU is based on the Bayer color pattern, the first color channel is a green color channel, the second color channel is a red color channel, and the third color channel is a blue color channel. In the drawings, pixel groups PXG distinguished by different hatching or dot patterns are assumed to have different color filters.

[0025] The pixel group PXG includes a plurality of unit pixels PX. The unit pixels PX included in the pixel group PXG are arranged in an N×N matrix shape, with N pixels arranged in the same number in the horizontal and vertical directions. For example, the pixel group PXG may include unit pixels PX arranged in a 2x2 matrix, or may include unit pixels PX arranged in a 3x3 matrix. In this specification, the pixel group PXG is described assuming that the unit pixels PX are arranged in a 2x2 matrix, but this may differ in other embodiments. The unit pixels PX in the same pixel group PXG share one microlens. For example, referring to FIG. 2, the unit pixels (G1r, G2r, G3r, G4r) of the first pixel group PXG1A share the first microlens ML1, the other unit pixels (G1b, G2b, G3b, G4b) of the first pixel group PXG1B share the fourth microlens ML4, the unit pixels (R1, R2, R3, R4) of the second pixel group PXG2 share the second microlens ML2, and the unit pixels (B1, B2, B3, B4) of the third pixel group PXG3 share the third microlens ML3.

[0026] Each unit pixel PX includes a photoelectric conversion element that converts an incident optical signal into an electrical signal. Each of the pixels PX includes at least one photoelectric conversion element. Each of the unit pixels PX in the same pixel group PXG generates a photocharge in response to light passing through a color filter corresponding to the same color channel. The photoelectric conversion element may be a photodiode (PD). The photoelectric conversion element may be any one of a photodiode (PD), a photocapacitor, a photogate, a pinned photodiode (PPD), a partially pinned photodiode, an organic photodiode (OPD), and a quantum dot (QD), or a combination thereof. In the embodiments of this specification, the photoelectric conversion element is described on the assumption that it is a photodiode PD, but other photoelectric conversion elements described above can also be used, and the photoelectric conversion element is not limited to a photodiode PD.

[0027] FIG. 3 is a diagram illustrating the phases of unit pixels (PX1, PX2, PX3, PX4) included in a pixel group PXG according to an embodiment of the present invention. The pixel group PXG in FIG. 3 corresponds to each of the pixel groups (PXG1A, PXG1B, PXG2, PXG3) in FIG. Referring to FIG. 3, unit pixels (PX1, PX2, PX3, PX4) of a pixel group PXG according to an embodiment of the present invention share one microlens ML. The optical signal that has passed through the same microlens ML passes through a color filter and enters each of the unit pixels (PX1, PX2, PX3, PX4).

[0028] The optical signals incident on each of the unit pixels (PX1, PX2, PX3, PX4) have different optical paths depending on the refraction at the microlens ML. The optical signals incident on the unit pixels (PX1, PX2, PX3, PX4) each have a phase (P1, P2, P3, P4). Depending on the shooting environment, such as the position and focus of the object to be shot, the length of the optical path of an optical signal incident on one of the unit pixels (PX1, PX2, PX3, PX4) is different from the length of the optical path of an optical signal incident on the other unit pixels. For example, in a given shooting environment, due to differences in the optical path, the size of the image signal of the unit pixel (PX1, PX3) located on the left side of the pixel group PXG differs from the size of the image signal of the unit pixel (PX2, PX4) located on the right side. In this specification, one side of a pixel group PXG means a position close to one of the boundaries of the pixel group PXG. Therefore, one side of the pixel group PXG can be any one of the left side, right side, top side, and bottom side.

[0029] The following embodiment will be described assuming the left and right sides of the pixel group PXG, but the technical idea of ​​the present invention is not limited to this. In this specification, it is assumed that unit pixels located at the same position corresponding to the microlens ML in different pixel groups receive optical signals with the same phase. For example, in different pixel groups, unit pixels located at the position of the first unit pixel PX1 corresponding to the microlens ML receive optical signals with the same phase.

[0030] FIG. 4 is a plan view showing a schematic configuration of a pixel group according to an embodiment of the present invention. The pixel group PXG according to the embodiment of FIG. 2 corresponds to any one of the first pixel groups (PXG1A, PXG1B) in FIG. The embodiment described with reference to Figure 4 has been described based on the first pixel group (PXG1A, PXG1B) of Figure 2, but as described below, the second pixel group PXG2 and the third pixel group PXG3 of Figure 2 also have the same configuration except for the overflow area IPO. The pixel groups PXG are separated by element isolation structures SS. The pixel group PXG includes a 4-shared unit pixel including four photodiodes (PD1, PD2, PD3, PD4) formed inside a substrate. Each of the first to fourth photodiodes (PD1, PD2, PD3, PD4) constitutes a separate unit pixel. As described with reference to FIG. 2, the number of photodiodes included in the pixel group PXG may differ from that shown in FIG. 3 depending on the embodiment.

[0031] The unit pixels in the same pixel group PXG share at least a part of the pixel circuit. Referring to FIG. 4, the first to fourth photodiodes (PD1, PD2, PD3, PD4) share the same floating diffusion region FD. The first to fourth photodiodes (PD1, PD2, PD3, PD4) are arranged around the floating diffusion region FD in the pixel group PXG along the radial direction with the floating diffusion region FD at the center. The first to fourth photodiodes (PD1, PD2, PD3, PD4) transfer charges to the floating diffusion region FD through the corresponding first to fourth transfer gates (TG1, TG2, TG3, TG4), respectively. The unit pixels each including the first to fourth photodiodes PD1, PD2, PD3, and PD4 share a driving transistor, a selection transistor, and a reset transistor.

[0032] In one embodiment, referring to FIG. 4, unit pixels each including first to fourth photodiodes PD1, PD2, PD3, and PD4 are separated from one another by first to fourth isolation structures SS1, SS2, SS3, and SS4. According to an embodiment, the pixel group PXG does not include the first to fourth isolation structures (SS1, SS2, SS3, SS4). According to an embodiment of the present invention, any one of the pixel groups PXG includes an overflow area IPO that provides a path for overflowed charges to move between unit pixels, and the remaining pixel groups do not include the overflow area IPO. The overflow region IPO contacts each of the first to fourth photodiodes (PD1, PD2, PD3, PD4).

[0033] For example, the first pixel groups (PXG1A, PXG1B) in FIG. 2 each include an overflow region IPO. The second pixel group PXG2 and the third pixel group PXG3 do not include the overflow region IPO. If the pixel groups are based on the Bayer color pattern, the pixel groups PXG corresponding to the green color channel each include an overflow region IPO. Charges that exceed a full well capacity (FWC) and overflow from at least one of the first to fourth photodiodes (PD1, PD2, PD3, and PD4) are transferred to the remaining photodiodes through an overflow region IPO. Therefore, the FWC of the pixel group PXG is increased by the overflow region IPO.

[0034] FIG. 5 is a cross-sectional view of the pixel group PXG according to the embodiment of FIG. 4 taken along line II'. The cross-sectional view according to the embodiment of FIG. 4 corresponds to one of the first pixel groups (PXG1A, PXG1B) in FIG. The embodiment described with reference to Figure 5 is based on the first pixel group (PXG1A, PXG1B) of Figure 2, but the second pixel group PXG2 and the third pixel group PXG3 of Figure 2 also have the same configuration except for the overflow area IPO. Referring to FIG. 5, the pixel group PXG includes a first structure S1, a second structure S2, and a third structure S3.

[0035] In one embodiment, the first structure S1 includes a photodiode PD, transmission gates (TG1, TG4), and a floating diffusion region FD. In one embodiment, the pixel circuits of pixel group PXG are disposed in second structure S2. Alternatively, in one embodiment, a portion of the pixel circuits of the pixel group PXG may be disposed in the first structure S1, and another portion may be disposed in the second structure S2. That is, FIG. 5 exemplarily illustrates that the floating diffusion region FD of the first structure S1 is directly connected to the transistor TR of the second structure S2. However, unlike FIG. 5, the floating diffusion region FD of the first structure S1 can be electrically connected to the transistor TR of the second structure S2 through another pixel circuit of the first structure S1.

[0036] In one embodiment, the third structure S3 includes logic such as readout circuits, timing controllers, image signal processors, and interface circuits. In one embodiment, the first structure S1, the second structure S2, and the third structure S3 include a wiring layer WS for transmitting electrical signals. In one embodiment, the first structure S1 includes a first surface FS1 and a second surface BS1 facing each other. The first surface FS1 is the front surface of the first structure S1, and the second surface BS1 is the back surface of the first structure S1. For example, the image sensor is a backside illumination (BSI) image sensor in which light is incident on the rear surface of the first structure S1.

[0037] In one embodiment, the pixel group PXG arranged in the first structure S1 includes a plurality of photodiodes PD, a color filter CF, and a microlens ML. In one embodiment, a plurality of deep element isolation structures DTI are formed between the first surface FS1 and the second surface BS1 of the first substrate W1 of the first structure S1, extending from the second surface BS1 toward the first surface FS1. The pixel groups PXG are separated from each other by deep isolation structures DTI. In one embodiment, the first structure S1 includes a shallow isolation structure STI. In one embodiment, the shallow isolation structure STI extends from the first surface FS1 of the first substrate W1 toward the second surface BS1 to a predetermined depth and includes an insulating material. In this case, the shallow isolation structure STI is connected to the deep isolation structure DTI, and the boundary between the shallow isolation structure STI and the deep isolation structure DTI is unclear. In one embodiment, the shallow isolation structure STI is formed as a doped region having a predetermined depth from the first surface FS1 toward the second surface BS1 of the first substrate W1. The doped region is doped with a P-type material.

[0038] In one embodiment, the second structure S2 includes a second substrate W2. In one embodiment, the second substrate W2 may be a silicon on insulator (SOI) substrate. In this case, after the SOI substrate is bonded to the first structure S1, a portion of the SOI substrate may be separated by grinding, polishing, or ion-cutting. In this case, the second structure S2 includes an oxide layer OX and a buried oxide (BOX) layer. The second substrate W2 is referred to as the active layer. In one embodiment, unlike the exemplary embodiment shown in FIG. 5, the second substrate W2 does not include a buried oxide BOX layer. That is, the second substrate W2 may be a general semiconductor substrate other than an SOI substrate.

[0039] In one embodiment, the first structure S1 and the second structure S2 are electrically connected to each other through a deep contact structure DCNT. The deep contact structure DCNT is formed by a contact that crosses at least a portion of the first structure S1 and at least a portion of the second structure S2. The deep contact structure DCNT is formed after bonding the first structure S1 and the second structure S2. In one embodiment, the deep contact DCNT includes electrical connection paths made of tungsten. Alternatively, in one embodiment, unlike that shown in FIG. 5, the first structure S1 and the second structure S2 may be electrically connected to each other through a through silicon via (TSV).

[0040] Alternatively, in one embodiment, the first structure S1 and the second structure S2 may be electrically connected to each other through Cu-to-Cu (C2C) bonding contacts. Alternatively, in one embodiment, the first structure S1 and the second structure S2 may be electrically connected to each other through all of Cu-to-Cu (C2C) bonding contacts, deep contact structure DCNTs, and through-silicon electrodes. In one embodiment, the second structure S2 and the third structure S3 are electrically connected through Cu-to-Cu (C2C) bonding contacts. Alternatively, the second structure S2 and the third structure S3 may be electrically connected to each other through a through silicon electrode and / or a TSC. In one embodiment, the first surface FS1 of the first substrate W1 and the third surface BS2 of the second substrate W2 face each other, and the fourth surface FS2 of the second substrate W2 and the fifth surface FS3 of the third substrate W3 face each other.

[0041] The overflow region IPO of the pixel group PXG according to the embodiment of the present invention is disposed in the first structure S1. The overflow region IPO is disposed at a predetermined depth from the first surface FS1 of the first structure S1. In one embodiment, the overflow region IPO is formed as a semiconductor region doped with impurities of the same conductivity type as each of the photodiodes (PD1, PD4). For example, the overflow region IPO is formed as a semiconductor region doped with N-type impurities. In one embodiment, the impurity doping concentration of the photodiodes (PD1, PD4) and the impurity doping concentration of the overflow region IPO may be substantially the same or similar. Alternatively, in one embodiment, the impurity doping concentration of the photodiodes (PD1, PD4) is greater than the impurity doping concentration of the overflow region IPO. As described with reference to FIG. 4, some pixel groups of the same pixel unit do not include the overflow area IPO.

[0042] FIG. 6 is a cross-sectional view of the pixel group PXG according to the embodiment of FIG. 4 taken along line II-II'. Detailed descriptions of portions that overlap with or are similar to the embodiment described with reference to FIG. 5 will be omitted. Referring to FIG. 6, the pixel group includes a shallow isolation structure STI and a shallow isolation structure DTI between a first photodiode PD1 and a second photodiode PD2. The shallow isolation structures STI and DTI in FIG. 5 correspond to the first to fourth isolation structures (SS1, SS2, SS3, SS4) in FIG.

[0043] FIG. 7 is a schematic circuit diagram of a pixel group PXG according to one embodiment of the present invention. The pixel groups PXG in FIG. 7 correspond to the pixel groups in FIGS. Referring to FIG. 7, a pixel group PXG according to one embodiment of the present invention includes first to fourth photodiodes (PD1, PD2, PD3, PD4), first to fourth transfer gates (TG1, TG2, TG3, TG4), a floating diffusion region FD, a reset transistor RX, a drive transistor DX, and a selection transistor SX.

[0044] The first to fourth transmission gates (TG1, TG2, TG3, TG4), the reset transistor RX, the drive transistor DX, and the select transistor SX are each controlled by the row driver 120 of FIG. The first to fourth photodiodes (PD1, PD2, PD3, PD4) transfer charges to the floating diffusion region FD through the corresponding first to fourth transfer gates (TG1, TG2, TG3, TG4), respectively. The reset transistor RX connects the first pixel voltage power supply VDD1 and the floating diffusion region FD, and is controlled by a reset control signal RS. The drive transistor DX is a source follower transistor and is controlled by the voltage of the floating diffusion region FD. The drive transistor DX amplifies the voltage provided to the gate terminal and provides an output signal to one terminal of the select transistor SX. The selection transistor SX outputs the pixel signal Vout received from the drive transistor DX to the column line CL under the control of the selection signal SEL. According to an embodiment of the present invention, unit pixels included in the same pixel group PXG share a driving transistor DX, a selection transistor SX, and a reset transistor RX.

[0045] 8A to 8D are graphs for schematically illustrating a compensation process for an image signal in an image signal processor according to an embodiment of the present invention. Image signal compensation is performed by image signal processor 160 of FIG. 1, 2, 3, and 8A to 8D, compensation of the image signal by the image signal processor 160 will be described. FIG. 8A shows the change in image signal based on the pixel unit PXU according to the exposure time EIT of FIG. Code refers to the size of the digitized image signal. FIG. 8A shows the change in the image signal without compensation by image signal processor 160. FIG. 8A to 8D will be described on the assumption that the pixel signals have been converted into 10-bit image signals by the readout circuit 150 of FIG. The maximum value of the code may vary depending on the configuration of the readout circuit 150.

[0046] FIG. 8A shows first image signals (GL, GR) based on pixel signals of the first pixel group (PXG1A, PXG1B) of FIG. 2, and second image signals (RL, RR) based on pixel signals of the second pixel group PXG2. Although FIGS. 8A to 8D do not show the third image signal based on the pixel signals of the third pixel group PXG3, the third image signal is similar to the second image signal. The first image signal average code GS is an average code of the image signal GL of the unit pixel located on one left side of the first pixel group (PXG1A, PXG1B) and the image signal GR of the unit pixel located on one right side thereof. Similarly, the second image signal average code RS is the average code of the image signal RL of the unit pixel located on one left side of the second pixel group PXG2 and the image signal RR of the unit pixel located on one right side thereof. For convenience of explanation, FIGS. 8B to 8D only show the second image signal average code RS.

[0047] Referring to FIG. 8A, the first image signal average code GS, which is the average code of the first image signals (GL, GR), has different linearities for each time interval according to the exposure time. Linearity does not necessarily mean a perfect straight line, but can mean a statistically linear characteristic. The first image signal (GL, GR) is based on the pixel signal of the unit pixel corresponding to the green color channel. For example, referring to FIG. 8A, the first image signal average code GS increases at a first increasing rate until it reaches the first exposure time (GL_SAT) at which the unit pixel located on the left side of the first pixel group (PXG1A, PXG1B) is saturated.

[0048] As described with reference to FIG. 2, the unit pixels of a pixel group share the same microlens. Therefore, depending on the photographing environment, the unit pixels located on either side of the pixel group are saturated earlier than the unit pixels located on the other side due to the difference in the light path caused by the parallax of the microlenses. In particular, when the pixel group is based on the Bayer pattern, the unit pixels located on either side of the pixel group corresponding to the green color channel are saturated earlier than the unit pixels located on the other side. In this specification, it is assumed that the unit pixel located on the left side of the pixel group corresponding to the green color channel is saturated before the unit pixel located on the right side. However, this is for the convenience of explanation, and the technical idea of ​​the present invention is not limited to this. For example, depending on the shooting environment, the unit pixel located on the right side of the pixel group corresponding to the green color channel may be saturated earlier than the unit pixel located on the left side.

[0049] The first image signal average code GS increases at a second increasing rate from the first exposure time (GL_SAT) until the second exposure time (IPO_OF) is reached. The second exposure time (IPO_OF) is the point at which the charge overflowed from the unit pixel located on the left side of the first pixel group (PXG1A, PXG1B) by the overflow area IPO in Figures 4 and 5 begins to move to the unit pixel located on the right side. Through the second exposure time (IPO_OF) and until the third exposure time (GR_SAT) is reached, the first image signal average code GS increases at a third increasing rate. The third exposure time (GR_SAT) is the time when the unit pixels located on the right side of the first pixel group (PXG1A, PXG1B) are also saturated. At least some of the first increase rate, the second increase rate, and the third increase rate may be different from one another.

[0050] In contrast, the image signals of the second pixel group PXG2 and the third pixel group PXG3 in FIG. 2 have the same linearity as a function of exposure time. The second image signals (RL, RR) are based on the pixel signals of the unit pixels corresponding to the red color channel. A third image signal (not shown) is based on the pixel signal of the unit pixel corresponding to the blue color channel. For example, referring to FIG. 8A, the second image signals (RL, RR) increase at substantially the same rate from the first exposure time (GL_SAT), through the second exposure time (IP0_OF), to the third exposure time (GR_SAT). As a result, the second image signal average code RS also increases at the same rate as the exposure time EIT changes. The third image signal is similar to this.

[0051] FIG. 8B shows an image signal after white balancing has been performed by image signal processor 160 of FIG. 8A. For example, referring to FIG. 8A, the image signal processor 160 multiplies the second image signals (RL, RR) by weighting values. In one embodiment, the values ​​of the weights may be preset. Depending on the embodiment, the white balancing method may vary. This specification will be described on the assumption that white balancing is performed by multiplying the second and third image signals by weighting values. However, the technical idea of ​​the present invention is not limited to this.

[0052] Referring to FIG. 8B, in one embodiment, the image signal processor 160 performs white balancing by multiplying the second image signal (RL, RR) by a weight based on the first increase rate. White balancing is performed on the third image signal in a similar manner. The third image signal will not be described further below, but is processed in the same manner as the second image signal. Referring to FIG. 8B, the second image signal average code RS has a value greater than the first image signal average code GS after the first exposure time (GL_SAT) due to white balancing. Depending on the shooting environment, the second image signal average code RS has a value greater than the first image signal average code GS before or after the first exposure time (GL_SAT) due to white balancing.

[0053] For ease of explanation, it is assumed that the second image signal average code RS has a value greater than the first image signal average code GS after the first exposure time (GL_SAT) due to white balancing. In this case, due to the second image signal average code RS (average code of the second image signal) being over-compensated by white balancing, some of the image signals output from the image signal processor 160 have redder values ​​than they actually are. That is, some pixels in the output image appear redder than they actually are. This phenomenon occurs because a part of the second image signals (RL, RR) is converted to a very high value by white balancing. For example, this is due to the result that a part of the second image signals (RL, RR) is converted to a value exceeding the saturation level (code 1023 when digitized at 10 bits).

[0054] Therefore, the image signal processor 160 performs color channel difference compensation CP1 on the white balanced image signal. For example, when at least a portion of the second image signals (RL, RR) reaches a saturation level, the image signal processor 160 changes the value of the second image signals (RL, RR) that have reached the saturation level. As a result, after the first exposure time (GL_SAT), at least a portion of the second image signals (RL, RR) are modified so that the rate of increase of the second image signal average code RS is similar to the second rate of increase of the first image signal average code GS.

[0055] FIG. 8C shows an image signal in which color channel difference compensation CP1 has been performed on the image signal of FIG. 8B by image signal processor 160. Referring to FIG. 8C, the second image signal average code RS has a smaller value than the first image signal average code GS after the second exposure time (IPO_OF) due to the color channel difference compensation CP1. For example, this is because the size of the first image signal GR increases at a larger rate after the second exposure time (IPO_OF) than before, due to the movement of the overflowed charges from the overflow region IPO in Figures 4 and 5 to the unit pixel located to the right of the first pixel group. In this case, some of the image signals output from the image signal processor 160 have greener values ​​than they actually are due to the first image signal average code GS (average code of the first image signal). That is, some pixels in the output image may appear greener than they actually are.

[0056] Therefore, the image signal processor 160 performs compensation CP2 for differences due to overflow (referred to herein as IPO differences) on the image signal on which compensation CP1 for color channel differences has been performed. For example, if a portion of the first image signals (e.g., GL) has a saturation level and the second image signals (RL, RR) are smaller than a predetermined standard than another portion of the first image signals (e.g., GR), the image signal processor 160 changes the value of at least a portion of the second image signals (RL, RR). Consequently, after a third exposure time (GR_SAT), at least a portion of the second image signals (RL, RR) are modified so that the rate of increase of the second image signal average code RS is similar to the third rate of increase of the first image signal average code GS.

[0057] FIG. 8D shows the image signal after image signal processor 160 has completed color channel difference compensation CP1 and IPO difference compensation CP2. It can be seen that the first image signal average code GS and the second image signal average code RS have similar linearity. Similarly, the third image signal average code (not shown) has similar linearity to the first image signal average code GS.

[0058] Referring to Figures 8A to 8D, when some of the first image signals (e.g., GL) reach a saturation level and other parts of the first image signals (e.g., GR) do not reach the saturation level, the image signal processor 160 changes the values ​​of at least some of the second image signals and / or the third image signals. That is, the image signal processor 160 compensates for color channel differences and / or IPO differences that occur when only the unit pixels located on either side of the unit pixels of the first pixel group (PXG1A, PXG1B) of Figure 2 are saturated due to the parallax of the microlens. The image signal processor 160 compensates for the color channel difference and / or the IPO difference by modifying at least a portion of the image signals corresponding to the unit pixels of the second pixel group PXG2 and / or the third pixel group PXG3 in FIG.

[0059] FIG. 9 is a block diagram showing a schematic configuration of an image signal processor 160 according to an embodiment of the present invention. The image signal processor 160 of FIG. 9 corresponds to the image signal processor 160 of FIG. Referring to FIG. 9, the image signal processor 160 receives the image signal IDT, performs at least one image process on the image signal IDT, and outputs an image signal pIDT. In one embodiment, image signal processor 160 includes a front-end circuit 161 , a compensation circuit 163 , a formatting circuit 165 , and an output interface circuit 167 .

[0060] The front-end circuit 161 performs noise processing, white balancing, etc. on the image signal IDT. The compensation circuit 163 performs color channel difference compensation CP1 and IPO difference compensation CP2 on the image signal IDT on which white balancing has been performed. The formatting circuit 165 demosaices the image signal IDT on which the color channel difference compensation CP1 and the IPO difference compensation CP2 have been performed. Depending on the embodiment, the formatting circuit 165 may combine the image signals IDT on which the color channel difference compensation CP1 and the IPO difference compensation CP2 have been performed. Depending on the embodiment, the compensation circuit 163 may perform demosaicing together with color channel difference compensation CP1 and / or IPO difference compensation CP2. In this specification, the description will be given on the assumption that the format circuit 165 performs demosaicing, but the technical idea of ​​the present invention is not limited to this.

[0061] The output interface circuit 167 transmits the image signal pIDT that has undergone image processing to the outside based on the protocol of the output interface. For example, the output interface exchanges data with an external device based on the Camera Serial Interface (CSI) and C-PHY defined by the Mobile Industry Processor Interface (MIPI) Alliance. The output interface is not limited to CSI and C-PHY.

[0062] 10A and 10B are block diagrams showing a schematic configuration of a compensation circuit according to one embodiment of the present invention. The compensation circuit of FIGS. 10A and 10B corresponds to compensation circuit 163 of FIG. Referring to FIG. 10A, the compensation circuit serially performs the color channel difference compensation CP1 and the IPO difference compensation CP2 described with reference to FIGS. 8A to 8D. The compensation circuit includes a color channel compensation circuit 163a that receives the image signal IDT and performs color channel difference compensation CP1 on the image signal IDT, and an IPO overflow compensation circuit 163b that performs IPO difference compensation CP2 on the image signal cIDT on which color channel difference compensation CP1 has been performed.

[0063] Referring to FIG. 10B, the compensation circuit may perform color channel difference compensation CP1 and IPO difference compensation CP2 in parallel. The compensation circuits include a color channel compensation circuit 163a that receives the white-balanced image signal IDT and performs color channel difference compensation CP1, and an IPO overflow compensation circuit 163b that performs IPO difference compensation CP2. The compensation circuit includes an image signal merging circuit 163c that merges the image signal cIDT on which the color channel difference compensation CP1 has been performed and the image signal iIDT on which the IPO difference compensation CP2 has been performed. 10 to 16 are described on the assumption that the compensation circuit performs IPO difference compensation CP2 on the image signal that has undergone color channel difference compensation CP1, but the technical idea of ​​the present invention is not limited to this.

[0064] FIG. 11 is a block diagram showing a schematic configuration of a color channel compensation circuit 163a of an image signal processor according to an embodiment of the present invention. Compensation circuit 163a corresponds to compensation circuit 163 of FIGS. 10A and 10B. Referring to FIG. 11, in one embodiment, the compensation circuit 163a includes a first saturation level determination circuit (163_1), a clipping circuit (163_2), and a selection circuit (163_3). The image signal wIDT on which white balancing has been performed is input to a first saturation level determination circuit 163_1, a clipping circuit 163_2, and a selection circuit 163_3.

[0065] The first saturation level determination circuit 163_1 compares the size of the image signal wIDT on which white balancing has been performed with the size of the image signal cpIDT obtained by clipping the image signal wIDT based on the saturation level. The first saturation level determination circuit (163_1) transmits a control signal DS to the selection circuit (163_3) based on the result of the comparison. The clipping circuit (163_2) clips the image signal wIDT, which has been subjected to white balancing, based on the saturation level. For example, if the readout circuit 150 of FIG. 1 performs digital conversion of a pixel signal based on 10 bits, the clipping circuit 163_2 changes the image signal wIDT to a value smaller than 1023, which is the maximum value that can be expressed in 10 bits, and outputs the clipped image signal cpIDT. The selection circuit 163_3 selects one of the clipped image signal cpIDT and the white-balanced image signal wIDT based on the control signal DS and outputs it as the image signal chIDT that has undergone the color channel difference compensation CP1.

[0066] FIG. 12 is a diagram for explaining the color channel difference compensation CP1 of the color channel compensation circuit according to an embodiment of the present invention. The color channel difference compensation CP1 is performed in the color channel compensation circuit 163a of FIG. The color channel difference compensation CP1 of the color channel compensation circuit 163a will be described with reference to FIGS. The color channel compensation circuit 163a performs color channel difference compensation CP1 for each image signal of the target pixel in the region of interest. The region of interest is a unit pixel within a preset range from the target pixel PS, where the color channel difference compensation CP1 is performed, among the unit pixels of the pixel array.

[0067] For example, referring to FIG. 12, the region of interest is a first region of interest ROI1 that includes unit pixels within a certain range from the target pixel PS. Alternatively, the region of interest may be a second region of interest ROI2 corresponding to pixel units (PXU1, PXU2, PXU3, PXU5, PXU7, PXU8, PXU9) surrounding pixel unit PXU4 in which the target pixel PS is included. The description with reference to FIGS. 12 to 16 will be given on the assumption that the region of interest corresponds to the second region of interest ROI2. However, depending on the embodiment, the size of the region of interest may be set differently.

[0068] The image signals of the unit pixels corresponding to the region of interest are stored in a line memory device. The selection circuit 163_3 performs the color channel difference compensation CP1 of the image signal by referring to the line memory device. The following description will be given on the assumption that the image signal stored in the line memory device is an image signal that has been subjected to white balancing. The clipping circuit 163_2 compares the image signal wIDT of the target pixel PS with a saturation level, modifies the image signal wIDT to the smaller value between the saturation level and the image signal wIDT of the target pixel PS, and outputs a clipped image signal cpIDT.

[0069] In one embodiment, the first saturation level determination circuit 163_1 determines a unit pixel (comparison pixel) having the same color channel as the target pixel PS. That is, the comparison pixel is a unit pixel included in a pixel group of the same type as the target pixel PS. The comparison pixels include the target pixel PS. For example, the comparison pixels are the R1, R2, R3, and R4 unit pixels of each of the pixel units (PXU1, PXU2, PXU3, PXU4, PXU5, PXU7, PXU8, and PXU9). In one embodiment, the first saturation level determination circuit 163_1 determines a unit pixel (comparison pixel) that has the same phase as the target pixel PS and the same corresponding color channel. The comparison pixels include the target pixel PS. For example, the comparison pixel is the R1 pixel of each of the pixel units (PXU1, PXU2, PXU3, PXU4, PXU5, PXU7, PXU8, PXU9).

[0070] In one embodiment, the first saturation level determination circuit 163_1 determines a unit pixel (comparison pixel) that has the same color channel as the target pixel PS and is located at the same position in each pixel group. The comparison pixels include the target pixel PS. For example, the comparison pixels are the R1 and R3 pixels of each of the pixel units (PXU1, PXU2, PXU3, PXU4, PXU5, PXU7, PXU8, and PXU9). The first saturation level decision circuit (163_1) compares the size of the average of the image signal wIDT of the comparison pixel with the size of the average of the clipped image signal cpIDT of the comparison pixel. Alternatively, depending on the embodiment, the first saturation level determination circuit 163_1 compares the size of the sum of the image signals wIDT of the comparison pixel with the size of the sum of the clipped image signals cpIDT of the comparison pixel. The first saturation level determination circuit (163_1) outputs the comparison result as a control signal DS.

[0071] The selection circuit 163_3 selects the smaller image signal from the clipped image signal cpIDT and the white-balanced image signal wIDT based on the control signal DS and outputs it as the image signal chIDT that has undergone color channel difference compensation CP1. For example, if the average of the image signal wIDT of the comparison pixel is smaller than the average of the clipped image signal cpIDT of the comparison pixel, the image signal wIDT of the target pixel PS is output as the image signal chIDT on which the color channel difference compensation CP1 has been performed.

[0072] FIG. 13 is a block diagram showing a schematic configuration of an IPO overflow compensation circuit 163b of an image signal processor according to an embodiment of the present invention. The IPO overflow compensation circuit 163b corresponds to the IPO overflow compensation circuit 163b of FIGS. 10A and 10B. Referring to FIG. 13, in one embodiment, the IPO overflow compensation circuit 163b includes a first UV level determination circuit (163_4), a second saturation level selection circuit (163_5), a second UV level determination circuit (163_6), and a merging circuit (163_7).

[0073] The first UV level determination circuit 163_4, the second saturation level selection circuit 163_5, the second UV level determination circuit 163_6, and the merging circuit 163_7 each receive the image signal chIDT that has undergone the color channel difference compensation CP1. In the description with reference to FIG. 13, the image signal chIDT on which the color channel difference compensation CP1 has been performed is simply referred to as the target image signal chIDT. The unit pixel corresponding to the target image signal chIDT is called a target pixel. The first UV level determination circuit (163_4) outputs a first result W1 obtained by comparing a value obtained by adding a part of the U signal and the V signal based on the target image signal chIDT of the region of interest with a first threshold value. The U and V signals are based on YUV encoding. In one embodiment of the present invention, the Y signal is determined by the same code value as the image signal of the green channel, the U signal is determined by a code value corresponding to the difference between the image signal of the blue channel and the image signal of the green channel, and the V signal is determined by a code value corresponding to the difference between the image signal of the red channel and the image signal of the green channel. However, depending on the embodiment, the Y, U, and V signals may be determined in a different manner and still fall within the spirit of the present invention based on YUV encoding.

[0074] In one embodiment, the first UV level determination circuit 163_4 outputs a first result W1 obtained by comparing the sum (absolute value) of the U signal and the V signal of a unit pixel (comparison pixel) located at a different position from the target pixel in the pixel group with a first threshold value. For example, if the target pixel is located on the right side of the pixel group, the sum of the U signal and the V signal of the unit pixel located on the left side of the pixel group is compared with the first threshold. In one embodiment, the first UV level determination circuit 163_4 outputs a first result W1 obtained by comparing the sum (absolute value) of the U signal and the V signal of a unit pixel (comparison pixel) located at a different position from the target pixel in a pixel group corresponding to the same color channel as the target pixel with a first threshold. For example, if the target pixel is a unit pixel located on the left side of the second pixel group, the sum of the U and V signals of unit pixels located on the right side of the second pixel group within the region of interest is compared with the first threshold.

[0075] The first threshold value is set by taking into consideration that a majority of the image signals of the corresponding unit pixel have reached a saturation level. Therefore, referring to FIG. 8C, the first UV level determination circuit 163_4 determines whether the target image signal chIDT has a corresponding code after the first exposure time GL_SAT. For example, if the target pixel is a unit pixel of the second pixel group, unit pixels located at positions different from the target pixel in the second pixel group have similar codes in saturation levels due to white balancing. Also, among the first image signals, an image signal corresponding to a first unit pixel located on either side of the first pixel group may have a code that has reached a saturation level. Therefore, if the sum (absolute value) of the U signal and V signal of the comparison pixel is smaller than the first threshold, it can be determined that the target image signal chIDT has the corresponding code after the first exposure time (GL_SAT).

[0076] The second saturation level selection circuit (163_5) outputs a second result W2 obtained by comparing the sum or average value of the image signals of at least some of the unit pixels (comparison pixels) of the first pixel group within the region of interest with a second threshold value. In one embodiment, the second saturation level selection circuit (163_5) outputs a second result W2 obtained by comparing the sum or average value of the image signals of all the unit pixels (comparison pixels) of the first pixel group within the region of interest with a second threshold value. The second saturation level selection circuit (163_5) outputs a second result W2 obtained by comparing the sum or average value of the image signals of unit pixels (comparison pixels) located at the same position as the target pixel in the first pixel group within the region of interest with a second threshold value.

[0077] The second threshold is set in consideration of the fact that a majority of the image signals of the corresponding unit pixel have reached a saturation level. Therefore, referring to FIG. 8C, the second saturation level selection circuit 163_5 determines whether the target image signal chIDT has a corresponding code after the first exposure time GL_SAT. That is, if the sum (average value) of the image signals of the comparison pixels is greater than the second threshold, it is determined that the target image signal chIDT has the corresponding code after the first exposure time GL_SAT. For example, if the region of interest corresponds to the second region of interest in FIG. 12, the second threshold for comparison with the average value of the image signal of the comparison pixel may be 999 or greater.

[0078] The second UV level determination circuit (163_6) compares the sum of parts of the U signal and the V signal based on the target image signal chIDT of the region of interest with a third threshold value and outputs a third result W3. In one embodiment, the second UV level determination circuit 163_6 outputs a third result W3 obtained by comparing the sum (absolute value) of the U signal and the V signal of a unit pixel (comparison pixel) located at the same position as the target pixel in the pixel group with a third threshold. For example, the sum of the U signal and the V signal of the unit pixel located on the left side of the pixel group is compared with the first threshold. In one embodiment, the second UV level determination circuit 163_6 outputs a third result W3 obtained by comparing the sum (absolute value) of the U signal and the V signal of a unit pixel (comparison pixel) located at the same position as the target pixel in a pixel group corresponding to the same color channel as the target pixel with a third threshold. For example, if the target pixel is a unit pixel located to the left of the second pixel group, the sum of the U and V signals of the unit pixel located to the left of the second pixel group within the region of interest is compared with the third threshold.

[0079] The third threshold value is set in consideration of the large difference between the majority of the image signals of the comparison pixels and the first image signal (the image signal of the first pixel group). Therefore, referring to FIG. 8C, the second UV level determination circuit 163_6 determines whether the target image signal chIDT has a corresponding code after the second exposure time (IPO_OF). That is, if the sum (absolute value) of the U signal and the V signal of the comparison pixel is greater than the third threshold, it is determined that the target image signal chIDT has the corresponding code after the second exposure time (IPO_OF).

[0080] The merging circuit (163_7) determines whether the target image signal chIDT has a corresponding code after the second exposure time (IPO_OF) of FIG. 8C based on all of the first result W1, the second result W2, and the third result W3. If the merging circuit (163_7) determines whether the target image signal chIDT has a corresponding code after the second exposure time (IPO_OF) of FIG. 8C, it changes the target image signal chIDT to the first image signal corresponding to the target pixel. For example, the merging circuit 163_7 calculates the average value of the image signals of the first unit pixel that is located in the same row as the target pixel, has the same phase as the target pixel, and is closest to the target pixel. The merging circuit 163_7 modifies the target image signal chIDT by the average value of the image signal of the first unit pixel.

[0081] If the target image signal chIDT does not have a corresponding code after the second exposure time (IPO_OF) in FIG. 8C, the merging circuit (163_7) outputs the target image signal chIDT as is. Therefore, if the IPO overflow compensation circuit 163b determines that the target image signal chIDT has a corresponding code after the second exposure time (IPO_OF) of Figure 8C, it changes the target image signal chIDT to the corresponding first image signal. As a result, the increase rate of the second image signal (image signal of the unit pixel included in the second pixel group) and / or the third image signal (image signal of the unit pixel included in the third pixel group) is changed to be the same as the increase rate of the first image signal (image signal of the unit pixel included in the first pixel group).

[0082] 14 to 16 are diagrams for explaining the IPO difference compensation CP2 operation of the IPO overflow compensation circuit according to one embodiment of the present invention. The IPO difference compensation CP2 operation is performed in the IPO overflow compensation circuit 163b of FIG. The IPO difference compensation CP2 operation of the IPO overflow compensation circuit 163b will be described with reference to FIGS. The IPO overflow compensation circuit 163b performs an IPO difference compensation CP2 for each image signal of each target pixel in the region of interest. As described with reference to FIG. 12, the region of interest is a unit pixel within a preset range from the target pixel PS.

[0083] FIG. 14 shows a schematic diagram of how the V signal is determined. The V signal may be predetermined by another circuit or, in one embodiment, is determined by the IPO overflow compensation circuit 163b. The method for determining the U signal is similar to the method for determining the V signal. It is assumed that the V signal is determined by a code value corresponding to the difference between the image signal of the red channel and the image signal of the green channel, and the Y signal is determined by the code value of the image signal of the green channel.

[0084] Referring to FIG. 14, in one embodiment, the V signal corresponding to the target pixel PS1 is determined based on the image signal of the first unit pixel that is located in the same row as the target pixel PS1 and has the same phase as the target pixel PS1 and is closest to the target pixel PS1. For example, the first unit pixel located in the same row as the target pixel PS1 and having the same phase as the target pixel PS1 has Y signals Y1 and Y3, respectively. The Y signal corresponding to the target pixel PS1 is calculated using Y2, which is the average value of Y1 and Y3, and the code value of the image signal of the green color channel corresponding to the target pixel PS1 is also calculated using Y2. Therefore, the V signal of the target pixel PS1 is determined by the difference between Y2 and the image signal of the target pixel PS1. In one embodiment, the V signal corresponding to the target pixel PS1 is determined based on the image signal of the first unit pixel located in the same row as the target pixel PS2 and closest to the target pixel PS2.

[0085] For example, the first unit pixel closest to the target pixel PS2, which is located in the same row and has the same phase as the target pixel PS2, has Y signals Y4 and Y6, respectively. Y5, which is the average value of Y4 and Y6, corresponds to the center position between the target pixel PS2 and the pixel located to the right of the target pixel PS2. A second image signal (Rmid) corresponding to the midpoint between the target pixel PS2 and the pixel located to the right of the target pixel PS2 is calculated as the average of the image signal of the target pixel PS2 and the image signal of the pixel located to the right of the target pixel PS2. Therefore, the V signal corresponding to the center position between the target pixel PS2 and the pixel located to the right of the target pixel PS2 is calculated as the difference between Y5 and Rmid. The V signal of the target pixel PS1 is calculated as the difference between Y5 and Rmid, or is determined as the difference between Y5 and the image signal of the target pixel PS2.

[0086] FIG. 15 is a diagram for explaining the operation of the first UV level determination circuit 163_4 according to an embodiment of the present invention. The operation of the first UV level determination circuit 163_4 will be described with reference to FIGS. The first UV level determination circuit (163_4) outputs a first result W1 obtained by comparing a value obtained by adding together parts of the U signal and the V signal based on the target image signal of the region of interest with a first threshold value. The target image signal is the image signal of the target pixel PS. The first UV level determination circuit 163_4 outputs a first result W1 for each image signal of the target pixel in the region of interest. The region of interest is a unit pixel within a preset range from the target pixel PS, where the compensation CP2 for the IPO difference is performed, among the unit pixels of the pixel array.

[0087] FIG. 15 illustrates an embodiment in which the first UV level determination circuit 163_4 outputs a first result W1 obtained by comparing the sum (absolute value) of the U and V signals of the target pixel and a unit pixel (comparison pixel) located at another position in a pixel group corresponding to the same color channel as the target pixel with a first threshold value. Referring to FIG. 15, the target pixel PS is a unit pixel located on the left side of the second pixel group of the fourth pixel unit PXU4. The first UV level determination circuit 163_4 determines the unit pixel located on the right side of the second pixel group of each of the pixel units PXU1, PXU2, PXU3, PXU4, PXU5, PXU7, PXU8, and PXU9 as a comparison pixel. The first UV level determination circuit 163_4 compares the sum (absolute value) of the V signals (V2, V4) of the second pixel group with the first threshold value and outputs a first result W1.

[0088] FIG. 16 is a diagram for explaining the operation of the second UV level determination circuit (163_6) according to an embodiment of the present invention. The operation of the second UV level determination circuit 163_6 will be described with reference to FIGS. The second UV level determination circuit (163_6) compares the sum of parts of the U signal and the V signal based on the target image signal of the region of interest with a third threshold value and outputs a third result W3. The target image signal is the image signal of the target pixel PS. The second UV level determination circuit (163_6) outputs a third result W3 for each image signal of the target pixel in the region of interest. The region of interest is a unit pixel within a preset range from the target pixel PS, where the compensation CP2 for the IPO difference is performed, among the unit pixels of the pixel array.

[0089] FIG. 16 is explained assuming that, in one embodiment, the second UV level determination circuit 163_6 outputs a third result W3 obtained by comparing the sum (absolute value) of the U signal and the V signal of a unit pixel (comparison pixel) located at the same position as the target pixel in a pixel group corresponding to the same color channel as the target pixel with a third threshold value. Referring to FIG. 16, the target pixel PS is a unit pixel located on the left side of the second pixel group of the fourth pixel unit PXU4. The second UV level determination circuit 163_6 determines the unit pixel located on the left side of the second pixel group of each of the pixel units PXU1, PXU2, PXU3, PXU4, PXU5, PXU7, PXU8, and PXU9 as a comparison pixel. The second UV level determination circuit 163_6 compares the sum (absolute value) of the V signals (V1, V3) of the second pixel group with a third threshold value and outputs a third result W3.

[0090] FIG. 17 is a perspective view showing a schematic configuration of an image sensor 100a according to one embodiment of the present invention. Detailed explanations of parts that overlap with those previously explained will be omitted. The image sensor 100a includes a first substrate 10a and a second substrate 20a that are stacked together. The first substrate 10a and the second substrate 20a are connected to each other through a wafer bonding process using C2C interconnections at the pixel group level. The first substrate 10a and the second substrate 20a are electrically connected not only through in-pixel contacts (IN_CT) within the unit pixel or pixel group PXa but also through a C2C (Cu-to-Cu) array located in the peripheral region of the substrates. Control signals are transmitted through the C2C array to control the pixel circuits. The pixel signals (or image signals) of the first substrate 10a are transmitted to a readout circuit (or image processor) of the second substrate 20a through the in-pixel contacts (IN_CT).

[0091] FIG. 18 is a partial perspective view showing a schematic configuration of an image sensor 100b according to an embodiment of the present invention. Detailed explanations of parts that overlap with those previously explained will be omitted. Referring to FIG. 18, an image sensor 100b includes a first substrate 10b, a second substrate 20b, and a third substrate 30b. The third substrate 30b, the second substrate 20b, and the first substrate 10b are stacked in this order in a direction D3 perpendicular to the plane of the substrates (a plane parallel to D1 and D2).

[0092] In one embodiment, a portion of the pixel circuits (PXb_1, PXb_2, PXb_3) is formed on each of the first substrate 10b and the second substrate 20b. A first partial circuit (PXb_1) of the pixel group is disposed on the first substrate 10b, and the remaining second partial circuits (PXb_2, PXb_3) of the pixel group are disposed on the second substrate 20b. The third board 30b may include logic such as a readout circuit, a timing controller, an image processor, and an interface circuit. The readout circuit includes an ADC. For example, the photodiode and transfer transistor are located on the first substrate 10b, and the remaining pixel circuitry is located on the second substrate 20d. The arrangement of the pixel circuits on the first substrate 10b and the second substrate 20b is not limited to this.

[0093] The first substrate 10b and the second substrate 20b are electrically connected to each other. In one embodiment, the first substrate 10b and the second substrate 20b transmit pixel signals or control signals through through silicon vias TSVs located in the peripheral regions of the substrates. In one embodiment, the first partial circuit (PXa_1) of the pixel group on the first substrate 10b and the second partial circuit (PXb_2) of the pixel group on the second substrate 20b are also electrically connected through the first inter-substrate connection structure INTC1. The inter-substrate connection structure INTC1 is a Cu-to-Cu (C2C) bonding contact, deep contact structure. The deep contact structure includes a through silicon electrode. The inter-substrate connection structure (INTC_1) electrically connects the in-pixel contact (IN_CT1) electrically connected to the element of the first partial circuit (PXa_1) of the pixel group and the in-pixel contact (IN_CT2) electrically connected to the element of the second partial circuit (PXb_2) of the pixel group to each other.

[0094] In one embodiment, the first substrate 10b and / or the second substrate 20b are electrically connected to the third substrate 30b through a through silicon via TSV and / or a second inter-substrate connection structure (INTC_2). Signals from the first substrate 10b and / or the second substrate 20b are transmitted to a readout circuit (or image processor) on the third substrate 30b via the through silicon via TSV and / or the second inter-substrate connection structure (INTC_2). In one embodiment, the second part circuit (PXb_2) of the pixel group is electrically connected to the circuit of the third substrate 30b through a Cu-to-Cu (C2C) bonding contact. The second inter-substrate connection structure (INTC_2) includes a Cu-to-Cu (C2C) bonding contact. In one embodiment, the third partial circuit (PXb_3) of the pixel group is electrically connected to the circuit of the third substrate 30b through a TSC (Thru-silicon Copper).

[0095] FIG. 19 is a block diagram showing a schematic configuration of an electronic device according to one embodiment of the present invention. Detailed explanations of parts that overlap with those previously explained will be omitted. The electronic device 1000 includes an image capture unit 1100 , an image sensor 1200 , a processor 1300 , a display device 1400 , and a storage device 1500 . The processor 1300 controls the overall operation of the electronic device 1000 . The processor 1300 provides control signals to the lens driver 1120 to control the position of the lens 1110 . As a result, the focal length is controlled. The imaging unit 1100 includes a lens 1110 and a lens driver 1120 as components that receive light. The lens 1110 may include multiple lenses.

[0096] The lens driver 1120 moves the lens 1110 in a direction in which the distance from the object S increases or decreases based on a control signal from the processor 1300 . The image sensor 1200 generates an image signal and / or phase data based on the incident light. The image sensor 1200 includes a pixel array 1210, a timing controller 1220, a readout circuit 1230, and an image signal processor 1240. A pixel group of the pixel array 1210 includes at least one photoelectric conversion element.

[0097] Each of the pixel groups in pixel array 1210 according to an embodiment of the present invention shares the same microlens. Some of the pixel groups include overflow areas. The image signal processor 1240 performs color channel difference compensation or IPO compensation for the image signal transmitted by the readout circuit 1230 . The image signal processor 1240 generates a mode control signal MC based on the imaging mode (signal MODE) transmitted by the processor 1300 . The pixel group operates in a signal output mode for each unit pixel PX or a signal output mode for each pixel group PXG based on a mode control signal MC transmitted from the image signal processor 1240 . The image signal processor 1240 provides a mode control signal MC to the timing controller 1220 . The timing controller 1220 controls the operation of the pixel array 1210 based on the mode control signal MC.

[0098] FIG. 20 is a flowchart illustrating a method of operating an image sensor according to an embodiment of the present invention. Detailed explanations of parts that overlap with those previously explained will be omitted. The method of operation of the image sensor of FIG. 20 may be implemented in the image sensor 100 of FIG. In step S110, the pixel array of the image sensor 100 outputs pixel signals, and the readout circuit outputs image signals based on the pixel signals.

[0099] In step S120, the image signal processor of the image sensor 100 performs color channel difference compensation for the image signal. For example, the color channel compensation circuit 163a of FIG. 11 performs compensation for color channel differences of the image signal as described with reference to FIGS. In step S130, the image signal processor of the image sensor 100 performs IPO compensation on the image signal. For example, the IPO overflow compensation circuit 163b of FIG. 13 performs IPO compensation on the image signal as described with reference to FIGS.

[0100] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical scope of the present invention. [Explanation of symbols]

[0101] 100 image sensors 110 pixel array 120 Low Driver 130 Timing Controller 140 Ramp Signal Generator 150 Readout circuit 160 Image Signal Processor 161 Front-end circuit 163 Compensation circuit 163a Color channel compensation circuit 163b IPO overflow compensation circuit 163c Image signal merging circuit 163_1 First saturation level judgment circuit 163_2 Clipping circuit 163_3 Selection circuit 165 Formatting Circuit 167 Output Interface Circuit

Claims

1. a pixel array including a plurality of first unit pixels, a plurality of second unit pixels, and a plurality of third unit pixels, each of which corresponds to a color filter of a different color from each other, and in which first pixel groups, second pixel groups, and third pixel groups are arranged in a matrix shape; a readout circuit that outputs a plurality of image signals based on the plurality of pixel signals output from the pixel array; an image signal processor that performs one or more image processes on the plurality of image signals and outputs the image-processed image signals; The image signal processor modifies a target image signal corresponding to a target pixel in response to saturation of one or more first unit pixels of the plurality of first unit pixels within a region of interest; The target pixel is a second unit pixel among the plurality of second unit pixels or a third unit pixel among the plurality of third unit pixels.

2. 2. The image sensor of claim 1, wherein the image signal processor modifies the target image signal in response to saturation of one or more first unit pixels located on one side and desaturation of one or more second unit pixels located on the other side of each of the first pixel groups within the region of interest.

3. each of the first pixel groups includes an overflow region in contact with a plurality of photodiodes included in the plurality of first unit pixels and providing a path for transfer of overflowed charges between the plurality of photodiodes of the plurality of first unit pixels; 2. The image sensor of claim 1, wherein the image signal processor modifies the target image signal based on an overflow of charges through the overflow region between the plurality of first unit pixels.

4. the image signal processor modifies the target image signal based on a result of adding sizes of at least some of U signals or V signals based on the plurality of image signals corresponding to the plurality of first to third unit pixels within the region of interest; 4. The image sensor of claim 3, wherein the U signal and the V signal are based on YUV encoding.

5. 5. The image sensor of claim 4, wherein the image signal processor modifies the target image signal based on a result of adding the sizes of the U signal or the V signal corresponding to the first, second, third unit pixels located on one side of each of the first, second, third pixel groups within the region of interest.

6. 4. The image sensor of claim 3, wherein the image signal processor modifies the target image signal based on whether at least some of the plurality of image signals corresponding to the plurality of first unit pixels within the region of interest have reached a saturation level.

7. 4. The image sensor of claim 3, wherein the image signal processor modifies the target image signal based on a plurality of image signals of the plurality of first unit pixels within the region of interest.

8. A plurality of unit pixels in the same pixel group share one microlens; The plurality of unit pixels of the same pixel group generate charges based on light passing through a color filter corresponding to the same color channel; 2. The image sensor of claim 1, wherein color filters disposed in adjacent pixel groups among the first pixel group, the second pixel group, and the third pixel group transmit light of different spectrums.

9. 2. The image sensor of claim 1, wherein the image signal processor modifies the target image signal based on the size of the image signal of a pixel group having the same color filter as the color filter of the target pixel within the region of interest.

10. 10. The image sensor of claim 9, wherein the image signal processor modifies the target image signal based on a comparison between a first value obtained by adding the image signals of a pixel group having the same color filter as the color filter of the target pixel in the region of interest and a second value obtained by adding a value obtained by clipping the image signals of a pixel group having the same color filter as the color filter of the target pixel in the region of interest based on a saturation level.

11. a pixel array including a plurality of first unit pixels, a plurality of second unit pixels, and a plurality of third unit pixels, each of which corresponds to a color filter of a different color from each other, and in which first pixel groups, second pixel groups, and third pixel groups are arranged in a matrix shape; a readout circuit that outputs a plurality of image signals based on the plurality of pixel signals output from the pixel array; an image signal processor that generates a plurality of first image signals, a plurality of second image signals, and a plurality of third image signals by performing white balancing on the image signals, and that changes at least one of the second image signals and the third image signals in response to saturation of one or more first unit pixels and desaturation of one or more second unit pixels among the first unit pixels, the plurality of first image signals are based on a plurality of pixel signals of the plurality of first unit pixels of the first pixel group; the plurality of second image signals are based on a plurality of pixel signals of the plurality of second unit pixels of the second pixel group; The image sensor, wherein the third image signals are based on pixel signals of the third unit pixels of the third pixel group.

12. the first pixel group, the second pixel group, and the third pixel group correspond to a green color filter, a red color filter, and a blue color filter, respectively, in order; the first pixel group, the second pixel group, and the third pixel group are arranged in a Bayer pattern in the pixel array; The image sensor of claim 11, wherein a plurality of unit pixels in the same pixel group share one microlens.

13. 12. The image sensor of claim 11, wherein the image signal processor modifies the one or more second image signals based on one or more of the second image signals reaching a saturation level, or modifies the one or more third image signals based on one or more of the third image signals reaching a saturation level.

14. the image signal processor modifies one or more of the second image signal and the third image signal based on an overflow of charges between the plurality of first unit pixels through an overflow region; 12. The image sensor of claim 11, wherein the overflow region contacts each of the plurality of photodiodes included in the plurality of first unit pixels and provides a path for transfer of overflowed charges between the plurality of photodiodes of the plurality of first unit pixels.

15. 15. The image sensor of claim 14, wherein the image signal processor modifies one or more of the second image signals or the third image signals that do not reach a saturation level based on the first image signals.

16. 16. The image sensor of claim 15, wherein the image signal processor modifies one or more of the second image signals or the third image signals that do not reach a saturation level based on the first image signals corresponding to unit pixels that are not saturated among the first unit pixels.

17. a pixel array including a plurality of pixel groups arranged in a matrix shape; a readout circuit that outputs a plurality of image signals based on the plurality of pixel signals output by the pixel array; an image signal processor that generates a plurality of first image signals, a plurality of second image signals, and a plurality of third image signals by performing white balancing on the image signals, and outputs a processed image signal obtained by modifying at least one of the plurality of second image signals and the plurality of third image signals based on whether one or more of the plurality of first image signals, the plurality of second image signals, and the plurality of third image signals have reached a saturation level; the plurality of pixel groups include a first pixel group, a second pixel group, and a third pixel group associated with different color channels; the plurality of first image signals, the plurality of second image signals, and the plurality of third image signals are based on the plurality of pixel signals of the first pixel group, the plurality of pixel signals of the second pixel group, and the plurality of pixel signals of the third pixel group, respectively, in order.

18. the first pixel group, the second pixel group, and the third pixel group each include a plurality of first unit pixels, a plurality of second unit pixels, and a plurality of third unit pixels, in that order; 18. The image sensor of claim 17, wherein the first pixel group includes an overflow region that contacts each of the photodiodes of the first unit pixels and provides a path for overflowed charges to move between the photodiodes of the first unit pixels.

19. 20. The image sensor of claim 18, wherein the image signal processor modifies at least some of the second image signals and the third image signals based on an overflow of charges between the first unit pixels through the overflow region.

20. 20. The image sensor of claim 19, wherein the image signal processor modifies at least some of the second image signals or the third image signals based on a first result that determines that at least some of the first image signals corresponding to a region of interest of a predetermined size have reached the saturation level and a second result that compares the first image signals with the second image signals and the third image signals corresponding to the region of interest.

Images