An apparatus and method for compensating crosstalk in an image sensor having a multi-color filter array

Abstract

An electronic device is disclosed, which includes a processing block, a crosstalk compensation block, and a dark level compensation block. The processing block receives image data from an active pixel region of an image sensor and performs preprocessing on the image data. The crosstalk compensation block performs crosstalk compensation on the preprocessed image data. The dark level compensation block performs crosstalk compensation on dark level data received from an optically black region of the image sensor, and performs a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data.

Description

[0001] Cross - reference to related applications

[0002] This application claims the priority of Korean Patent Application No. 10 - 2020 - 0067910, filed on Jun. 4, 2020 with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical field

[0003] The present invention relates to an electronic device, and more particularly, to an electronic device for compensating crosstalk in a multi - color filter array of an image sensor. Background art

[0004] An image sensor may be a charge - coupled device (CCD) image sensor, a complementary metal - oxide - semiconductor (CMOS) image sensor (CIS), etc. A CMOS image sensor includes pixels formed of CMOS transistors and converts light energy into an electrical signal by using a photoelectric conversion element included in each pixel. The CMOS image sensor obtains information related to a captured / taken image by using the electrical signal generated at each pixel.

[0005] These days, image sensors having a multi - color filter array are adopted in various fields, and pixels formed to share a floating diffusion region (or, so - called floating diffusion node) are also adopted in various fields. However, the order of manufacturing each color filter constituting the multi - color filter array varies depending on the type of the color filter, thereby causing a height (or, thickness) difference between the color filters. In an image sensor having a general Bayer pattern, this height difference may not be a problem. However, in the case of a Bayer pattern (e.g., a quad - Bayer pattern) in which multiple pixels share one floating diffusion region, the height (or, thickness) difference of the color filters may cause optical crosstalk. Summary of the invention

[0006] The technical idea of the present disclosure provides an electronic device and a method for compensating crosstalk caused by a height difference between color filters in a multi - color filter array.

[0007] According to an exemplary embodiment, an electronic device includes: a processing block that receives image data from an active pixel region of an image sensor and performs pre - processing on the image data; a crosstalk compensation block that performs crosstalk compensation on the pre - processed image data; and a dark - level compensation block that performs crosstalk compensation on dark - level data received from an optically black region of the image sensor and performs a subtraction operation on the crosstalk - compensated image data and the crosstalk - compensated dark - level data.

[0008] According to an exemplary embodiment, an electronic device includes: an image sensor including an active pixel region and an optical black region, the active pixel region including a plurality of unit pixel groups, each of the plurality of unit pixel groups including a plurality of pixels sharing a floating diffusion region, the image sensor outputting image data corresponding to the active pixel region and outputting dark level data corresponding to the optical black region; an image signal processor (ISP) front-end block performing crosstalk compensation on the image data, performing crosstalk compensation on the dark level data, and performing a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data; and an image signal processor receiving and processing the image data on which the subtraction operation has been performed.

[0009] According to an exemplary embodiment, a method for processing signals output from an image sensor including an active pixel region and an optical black region, the active pixel region including a first pixel sharing a first floating diffusion region and a second pixel sharing a second floating diffusion region, the method includes: generating image data based on signals output from pixels selected from the first pixel, generating dark level data based on signals output from the optical black region, performing crosstalk compensation on the image data according to a height difference between a first color filter covering the first pixel and a second color filter covering the second pixel, performing crosstalk compensation on the dark level data, and performing a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects and features of the inventive concept will become apparent by describing exemplary embodiments of the inventive concept in detail with reference to the accompanying drawings.

[0011] Figure 1 An exemplary configuration of an image processing block according to an embodiment of the present disclosure is shown.

[0012] Figure 2 An exemplary configuration of Figure 1 an image sensor according to an embodiment is shown.

[0013] Figure 3 An exemplary configuration of Figure 1 an image sensor according to an embodiment is shown.

[0014] Figure 4 An exemplary cross-sectional view of a pixel array taken along Figure 3 lines I-I' and II-II' of

[0015] Figure 5 An exemplary circuit diagram of Figure 2 one of the pixel groups of

[0016] Figure 6A andFigure 6B Conceptually shows the influence of peripheral pixels on Figures 2 to 4 a pixel group of an image sensor.

[0017] Figure 7 Shows an exemplary configuration of an ISP front-end block according to an embodiment of the present disclosure.

[0018] Figure 8A and Figure 8B Shows a schematic operation of correlated double sampling.

[0019] Figure 9 Schematically shows image data processed by Figure 7 the ISP front-end block.

[0020] Figure 10 Shows a plan view of an Figure 1 image sensor according to one embodiment.

[0021] Figure 11 is a cross-sectional view of the image sensor taken along Figure 10 line III-III'.

[0022] Figure 12 Shows Figure 10 an exemplary circuit diagram of one of the pixel groups of

[0023] Figure 13A and Figure 13B Conceptually shows the influence of peripheral pixels on Figures 10 to 12 a pixel group of an image sensor.

[0024] Figures 14A to 14C Schematically shows various methods for determining calibration values.

[0025] Figure 15 Shows a method for processing a signal output from an image sensor according to an embodiment of the present disclosure.

[0026] Figure 16 Shows an exemplary configuration of an electronic device including a multi-camera module applying crosstalk compensation of the present disclosure.

[0027] Figure 17 Shows Figure 16 an exemplary configuration of a camera module of Detailed Description

[0028] Hereinafter, embodiments of the inventive concept are described in detail and clearly to such an extent that an ordinary skilled person in the art can implement the inventive concept.

[0029] In the detailed description, components described with reference to terms such as "unit", "module", "block", "device", or "element", and functional blocks shown in the drawings will be implemented using software, hardware, or a combination thereof. For example, software can be machine code, firmware, embedded code, and application software. For example, hardware can include circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, pressure sensors, inertial sensors, microelectromechanical systems (MEMS), passive components, or a combination thereof.

[0030] Figure 1 An exemplary configuration of an image processing block according to an embodiment of the present disclosure is shown. The image processing block 10 can be implemented as part of various electronic devices such as smartphones, digital cameras, laptop computers, and desktop computers. The image processing block 10 can include a lens 12, an image sensor 14, an image signal processor (ISP) front-end block 16, and an image signal processor 18.

[0031] In operation, light is reflected by an object or a landscape that is the shooting target, and the lens 12 receives the reflected light. The image sensor 14 generates an electrical signal based on the light received through the lens 12. The ISP front-end block 16 processes the electrical signal output from the image sensor 14 to be suitable for processing by the image signal processor 18. The image signal processor 18 generates image data associated with the photographed object and landscape by appropriately processing the electrical signal processed by the ISP front-end block 16.

[0032] The image sensor 14 can include an active pixel sensor (APS) area and an optically black area. The pixels in the active pixel sensor area convert light into an electrical signal, while the pixels in the optically black area output dark current regardless of the light. The dark current output from the optically black area can be used to compensate for the dark level. For example, the image sensor 14 can be implemented using a complementary metal oxide semiconductor (CMOS) image sensor or the like.

[0033] In Figure 1 One lens 12 and one image sensor 14 are shown. However, in another example, the image processing block 10 can include multiple lenses, multiple ISP front-end blocks, and multiple image sensors. Multiple image sensors can be provided to have different functions, different performances, and / or different characteristics. In some embodiments, the image sensors can respectively include lenses with different fields of view (FOV).

[0034] The ISP front-end block 16 can perform preprocessing on the signal output from the image sensor 14. For example, the ISP front-end block 16 can perform crosstalk compensation, dark level compensation, etc. on the signal output from the image sensor 14. In particular, in the automatic dark level compensation (ADLC) for the signal output from the image sensor 14, the ISP front-end block 16 of the present disclosure takes into account crosstalk compensation according to the height difference of the color filters of the pixels constituting the image sensor 14. For example, the ISP front-end block 16 compensates the dark level based on pre-calculated crosstalk calibration data. Therefore, fixed pattern noise (FPN) can be removed.

[0035] The image signal processor 18 can generate image data associated with an object, a landscape, etc. based on the data processed by the ISP front-end block 16. To this end, the image signal processor 18 can perform various processes such as color interpolation, color correction, automatic white balance, gamma correction, color saturation correction, formatting, defective pixel correction, and hue correction.

[0036] Figure 2 An exemplary configuration of the Figure 1 image sensor 14 is shown. The image sensor 100 can include a pixel array 110, a row driver 120, a ramp signal generator 130, an analog-to-digital conversion block (hereinafter referred to as "ADC block") 140, a timing controller 150, and a buffer 160.

[0037] The pixel array 110 can include a plurality of pixels arranged in a matrix form along rows and columns, and can be divided into an active pixel region 110a and an optically black region 110b. Each of the plurality of pixels can include a photoelectric conversion element. The pixels in the active pixel region 110a can sense light by using the photoelectric conversion element, and can convert the sensed light into an electrical signal (hereinafter referred to as "pixel signal"). The pixels in the optically black region 110b can output an electrical signal (i.e., a dark level offset signal) regardless of the incident light. For example, the photoelectric conversion element can include a photodiode, a phototransistor, a photogate, a pinned photodiode, etc. In Figure 1 an embodiment is shown in which the optically black region 110b is provided in the top row of the pixel array 110, but the present disclosure is not limited thereto.

[0038] The pixel array 110 can include a plurality of pixel groups. Each pixel group PG can include at least two or more pixels. In Figure 2 an embodiment is shown in which the pixel group PG includes four pixels arranged in two rows and two columns. The pixels constituting one pixel group PG can share a floating diffusion region (or, a floating diffusion node). However, the present disclosure is not limited thereto. For example, the pixels constituting one pixel group PG can share a plurality of floating diffusion regions. Additionally, in Figure 2An example is shown in which the pixel array 110 includes pixel groups arranged in four rows and two columns (i.e., 4×2 pixel groups). However, the present disclosure is not limited thereto.

[0039] The pixel group PG may include pixels of the same color. For example, the pixel group PG may include red pixels “R” for converting light in the red spectral range into an electrical signal, green pixels Gr / Gb for converting light in the green spectral range into an electrical signal, and blue pixels “B” for converting light in the blue spectral range into an electrical signal. For example, the pixels constituting the pixel array 110 may be arranged in the form of a four - Bayer pattern.

[0040] A plurality of color filters may be formed in the pixel groups PG constituting the pixel array 110. For example, a multi - color filter array (multi - CFA) may be formed thereon. The height at which the color filter is formed (e.g., the height of the top surface of the color filter) may vary depending on the processing order. For example, the color filter formed relatively earlier may have a relatively lower height from the substrate. In contrast, the color filter formed relatively later may have a relatively higher height from the substrate. The color filter formed relatively earlier may be affected by the color filter formed relatively later, thereby causing crosstalk between pixels. This will be described more fully with reference to Figure 3 this.

[0041] The pixel groups arranged along one column may be alternately connected to two column lines. For example, some of the pixel groups arranged in the first column may be connected to the first column line CL1, while other pixel groups may be connected to the second column line CL2. As described above, some of the pixel groups arranged in the second column may be connected to the third column line CL3, while other pixel groups may be connected to the fourth column line CL4.

[0042] The row driver 120 is configured to select and drive the rows of the pixel array 110. The row driver 120 may decode the address and / or control signals generated by the timing controller 150 and may generate control signals for selecting and driving the rows of the pixel array 110. For example, the control signals may include signals for selecting pixels, signals for resetting the floating diffusion region, signals for selecting column lines, etc.

[0043] The ramp signal generator 130 is configured to generate a ramp signal. The ramp signal generator 130 may operate under the control of the timing controller 150. For example, the ramp signal generator 130 may operate in response to control signals such as a ramp enable signal or a mode signal. When the ramp enable signal is activated, the ramp signal generator 130 may generate a ramp signal having a slope set based on the mode signal.

[0044] The ADC block 140 is configured to convert an analog signal (i.e., a pixel signal or a dark level offset signal) output from the pixel array 110 into a digital signal. For example, the ADC block 140 may convert the pixel signal into a digital signal to generate values including a signal level and a dark level. The ADC block 140 may convert the dark level offset signal output from the optically black area 110b into a digital signal to generate a value including the dark level.

[0045] In an embodiment, the ADC module 140 may include four ADCs 140_1, 140_2, 140_3, and 140_4, each of which includes a comparator COMP and a counter CNT. The comparator COMP may compare the pixel signal output through a column line (i.e., one of CL1 to CL4) connected to the comparator COMP with a ramp signal RAMP (refer to Figure 8A ), and may output a comparison result. For example, the comparator COMP may operate based on the correlated double sampling (CDS) technique to obtain a reset signal and an image signal regarding the pixel signal, and extract the difference between the reset signal and the image signal as an effective signal component.

[0046] The counter CNT may operate under the control of the timing controller 150, and may count the pulses of the output signal of the comparator COMP. For example, the counter CNT may operate in response to control signals such as a counter clock signal, a counter reset signal for controlling the reset of the counter CNT, and an inversion signal for inverting the internal bits of the counter CNT. The counter CNT may count the comparison result signal depending on the counter clock signal, and may output the counted result as a digital signal.

[0047] The counter CNT may include an up / down counter, a bitwise inversion counter, etc. The operation of the bitwise counter may be similar to the operation of the up / down counter. For example, the bitwise counter may perform the following functions: only perform the function of up counting; and the function of inverting all the internal bits of the counter in response to a specific signal to obtain the one's complement when the specific signal is received. The bitwise counter may perform a reset count, and may invert the result of the reset count to the one's complement, i.e., a negative value.

[0048] The timing controller 150 is configured to generate control signals and / or clocks for controlling the operation and / or timing of each of the row driver 120, the ramp signal generator 130, the ADC module 140, and the counter CNT.

[0049] The buffer 160 may include a memory MEM (e.g., memory cells) and a set 162 of sense amplifiers SA. Each of the memories MEM may store a digital signal output from a corresponding ADC. The sense amplifier SA may sense and amplify the digital signal stored in the memory MEM. The sense amplifier SA may output the amplified digital signal as image data IDAT. For example, the image data IDAT may be formed of 11 bits. Although not shown in the drawings, the sense amplifier SA may amplify the digital signal (i.e., the second code value) output from the ADC and may output the amplified signal as dark level data. The dark level data may be used in dark level compensation.

[0050] Figure 3 A plan view of an image sensor is shown. In an embodiment, Figure 1 is shown. Figure 3 A plan view of the pixel array 110 of the image sensor 100 shown in Figure 2 is shown. The image sensor 100 may include pixel groups PG1 to PG4. However, for the purpose of showing a cross-sectional view of the image sensor 100 taken along line I-I' and line II-II', in addition to the pixel groups PG1 to PG4, in Figure 3 other pixels adjacent to the pixel groups PG1 to PG4 are also shown. The pixel groups PG1 to PG4 may be repeatedly formed on the substrate of the image sensor 100 along the X-axis and the Y-axis. Each of the pixel groups PG1 to PG4 may include 2×2 pixels arranged along the X-axis and the Y-axis. Each pixel may include a photoelectric conversion element PD.

[0051] Color filters CF1 to CF4 for passing light of a specific type (e.g., a specific wavelength band) may be formed on the pixel groups PG1 to PG4, respectively. For example, the first color filter CF1 may pass blue light, the second color filter CF2 and the third color filter CF3 may pass green light, and the fourth color filter CF4 may pass red light. As Figure 3 shown in, since the second color filter CF2 is adjacent to the first color filter CF1 in the X-axis direction and the third color filter CF3 is adjacent to the fourth color filter CF4 in the X-axis direction, a Bayer pattern may be formed. A microlens ML may be formed on the color filter formed on each pixel.

[0052] Figure 4 A cross-sectional view of the pixel array taken along line I-I' and line II-II' of Figure 3 is shown.

[0053] The pixel array 110 may include a substrate SUB, the substrate SUB including a first surface 1a and a second surface 1b facing each other. The substrate SUB may include or may be a single-crystalline substrate or an epitaxial layer. The substrate SUB may include a region 1 doped with impurities of a first conductivity type (e.g., P-type) and a region 2 doped with impurities of a second conductivity type (e.g., N-type). The first doped region 1 and the second doped region 2 may form a photoelectric conversion element PD of each pixel. When light is incident on the photoelectric conversion element PD through a microlens ML and a color filter (e.g., one of CF1 to CF4), electron-hole pairs EHP corresponding to the intensity of the absorbed light may be generated.

[0054] A fixed charge layer 3 may be formed on the first surface 1a of the substrate SUB. The fixed charge layer 3 may include various types of metal oxides and / or metal fluorides. For example, the fixed charge layer 3 may include at least one or more of Al2O3, HfOX, SiO2, and SiN. An interlayer insulating layer 6 may be formed on the second surface 1b of the substrate SUB. For example, the interlayer insulating layer 6 may include a plurality of insulating layers (which may be referred to as sub-layers). The interlayer insulating layer 6 may be covered with a passivation layer 8. For example, the passivation layer 8 may include or may be a silicon nitride layer.

[0055] Meanwhile, each of the pixels PX1 to PX7 may include a floating diffusion region FD and a transfer transistor TG. Additionally, although not shown for simplicity, each of the pixels PX1 to PX7 may further include a reset transistor, a driving transistor, and a selection transistor. The pixels PX1 to PX7 may be separated from each other by deep trench isolators. When a transfer signal is applied to the gate electrode of the transfer transistor TG, the transfer transistor TG may be turned on, and thus, the charges generated in the first doped region 1 and the second doped region 2 may move to the floating diffusion region FD. The charges in the floating diffusion region FD may be transmitted to the outside through an internal wire 7.

[0056] A light-shielding pattern 4 may be formed on the fixed charge layer 3. The light-shielding pattern 4 may include tungsten, titanium, etc. The light-shielding pattern 4 may prevent crosstalk between adjacent pixels. In a plan view, the light-shielding pattern 4 may have a grid shape. The light-shielding pattern 4 may include, for example, linear patterns parallel to each other and extending in the X-axis direction and linear patterns parallel to each other and extending in the Y-axis direction. The color filters CF1 to CF4 may be formed between the light-shielding patterns 4 on the fixed charge layer 3. A planarization layer 5 may be formed on the color filters CF1 to CF4, and a microlens ML may be formed on the planarization layer 5. The planarization layer 5 may be formed of a transparent material such as an electrically insulating material. The light-shielding pattern 4 for preventing the light passing through a specific color filter from affecting any other color filter may not be able to prevent the influence caused by the height or thickness difference between the color filters CF1 to CF4.

[0057] In an embodiment, it is assumed that a first light L1 is incident between a first pixel PX1 and a second pixel PX2, a second light L2 is incident between a third pixel PX3 and a fourth pixel PX4, a third light L3 is incident between a fifth pixel PX5 and the third pixel PX3, and a fourth light L4 is incident between a sixth pixel PX6 and a seventh pixel PX7. The lights L1 to L4 may be from one light source (e.g., natural light) or multiple light sources, and it should be understood that this classification is made for convenience depending on the position where the light is incident on the pixel array 110 of the image sensor 100.

[0058] Since the thickness d1 of the first color filter CF1 or the height of the first color filter CF1 from the fixed charge layer 3 (e.g., the height of the uppermost surface of the first color filter CF1 from the fixed charge layer 3) is different from the thickness d3 of the third color filter CF3 or the height of the third color filter CF3 from the fixed charge layer 3 (e.g., the height of the uppermost surface of the third color filter CF3 from the fixed charge layer 3), when the light L1 is incident, the shadow caused by the first color filter CF1 of the first pixel PX1 may affect the region "a" of the third color filter CF3 of the second pixel PX2. When the light L2 is incident, the shadow caused by the first color filter CF1 of the fourth pixel PX4 may affect the region "b" of the third color filter CF3 of the third pixel PX3.

[0059] As described above, since the thickness d3 of the third color filter CF3 or the height of the third color filter CF3 from the fixed charge layer 3 (e.g., the height of the uppermost surface of the third color filter CF3 from the fixed charge layer 3) is different from the thickness d4 of the fourth color filter CF4 or the height of the fourth color filter CF4 from the fixed charge layer 3 (e.g., the height of the uppermost surface of the fourth color filter CF4 from the fixed charge layer 3), when the light L3 is incident, the shadow caused by the fourth color filter CF4 of the fifth pixel PX5 may affect the region "c" of the third color filter CF3 of the third pixel PX3. When the light L4 is incident, the shadow caused by the fourth color filter CF4 of the seventh pixel PX7 may affect the region "d" of the third color filter CF3 of the sixth pixel PX6.

[0060] Due to the thickness or height differences of the color filters CF1 to CF4 described above, the intensity of the light actually incident on each pixel may be different, thereby reducing the dynamic range and / or fixed pattern noise. Therefore, the ISP front-end block 16 of the present disclosure (refer to Figure 1)configured to consider crosstalk according to the height difference of the color filters CF1 to CF4 in dark level compensation. For example, the ISP front-end block 16 may multiply the code levels of the signal level including the image data and the dark level by a calibration value for compensating crosstalk. Additionally, even when compensating the dark level, the ISP front-end block 16 may multiply the code level of the dark level including the dark level data by calibration data for compensating crosstalk.

[0061] An embodiment is shown in which the fourth color filter CF4 for red light is the thickest and the third color filter CF3 for green light is the thinnest, but the present disclosure is not limited thereto. For example, the thickness of the color filters may vary differently depending on the order in which the color filters are formed. For example, a color filter formed earlier may be relatively thin, while a color filter formed later may be relatively thick.

[0062] Figure 5 shows Figure 2 An exemplary circuit diagram of one of the pixel groups PG1 to PG4. In an embodiment, the third pixel group PG3 may include photoelectric conversion elements PD1 to PD4, transfer transistors TG1 to TG4, a dual conversion transistor DCT, a reset transistor RT, a driving transistor DT, and a selection transistor ST. The first pixel Gr1 may include a first photoelectric conversion element PD1 and a first transfer transistor TG1. The second pixel Gr2 may include a second photoelectric conversion element PD2 and a second transfer transistor TG2, and the remaining pixels Gr3 and Gr4 may each include similar components / elements. The first pixel Gr1 to the fourth pixel Gr4 may share the dual conversion transistor DCT, the reset transistor RT, the driving transistor DT, the selection transistor ST, and the floating diffusion region FD.

[0063] When the dual conversion transistor DCT is turned off, the floating diffusion region FD may be connected to the first floating diffusion capacitor CFD1. When the dual conversion transistor DCT is turned on, the floating diffusion region FD may be connected to the second floating diffusion capacitor CFD2 and the first floating diffusion capacitor CFD1. For example, the floating diffusion capacitors CFD1 and CFD2 may be parasitic capacitors and / or capacitor elements. The second floating diffusion capacitor CFD2 may be provided to prevent saturation.

[0064] The transfer transistors TG1 to TG4 may transfer the charges generated by the photoelectric conversion elements PD1 to PD4 to the floating diffusion region FD or the floating diffusion region FD' expanded when the dual conversion transistor DCT is turned on. For example, the first ends of the transfer transistors TG1, TG2, TG3, and TG4 may be connected to the photoelectric conversion elements PD1, PD2, PD3, and PD4, respectively, and their second ends may be commonly connected to the floating diffusion region FD. The transfer transistors TG1, TG2, TG3, and TG4 may be respectively driven by the row driver 120 (refer toFigure 1 ) Controlled by the received transmission signals VTG1, VTG2, VTG3, and VTG4.

[0065] The floating diffusion region FD or the extended floating diffusion region FD' can accumulate charges corresponding to the amount of incident light. The floating diffusion region FD can have a capacitance corresponding to the first floating diffusion capacitor CFD1. During the time when the transfer transistors TG1, TG2, TG3, and TG4 are turned on by the transmission signals VTG1, VTG2, VTG3, and VTG4 respectively, the charges provided by the photoelectric conversion elements PD1, PD2, PD3, and PD4 can be accumulated at the floating diffusion region FD or the extended floating diffusion region FD'. The floating diffusion region FD can be connected to the gate terminal of the driving transistor DT that serves as a source follower amplifier. As a result, a voltage potential corresponding to the charges accumulated in the floating diffusion region FD can be formed.

[0066] The reset transistor RT can be turned on by the reset signal VRST and can supply a reset voltage (e.g., the power supply voltage VDD) to the floating diffusion region FD or the extended floating diffusion region FD'. As a result, the charges accumulated in the floating diffusion region FD or the extended floating diffusion region FD' can be moved to the terminal of the power supply voltage VDD, and the voltage of the floating diffusion region FD or the extended floating diffusion region FD' can be reset.

[0067] The driving transistor DT can amplify the change in the potential of the floating diffusion region FD or the extended floating diffusion region FD' and can generate a voltage (i.e., the pixel signal PIX) corresponding to the amplified result. The selection transistor ST can be driven by the selection signal VSEL and can select the pixels to be read in units of rows. When the selection transistor ST is turned on, the pixel signal PIX can be output through the column line CL.

[0068] Figure 6A and Figure 6B Conceptually shows the influence of the peripheral pixels on Figures 2 to 4 the pixel group of the image sensor.

[0069] First, refer to together Figures 2 to 4 and Figure 6A, each of the pixels Gr1 to Gr4 that make up the third pixel group PG3 may be affected by a color filter covering at least one adjacent pixel. For example, the thickness (or height) of the third color filter CF3 covering the pixels Gr1 to Gr4 may be less than the thickness (or height) of other adjacent color filters. For example, the pixel Gr1 may be affected by the shadow caused by the fourth color filter CF4 covering the pixel R2 and the shadow caused by the first color filter CF1 covering the pixel B3. In an embodiment, the crosstalk experienced by the pixel Gr1 is labeled "C1".

[0070] As described above, the remaining pixels Gr2 to Gr4 may be affected by adjacent color filters, and the crosstalk experienced by the pixels Gr2, Gr3, and Gr4 may be "C2", "C3", and "C4". Since the surrounding environment (i.e., the color filter height) of the pixels Gr1 to Gr4 is the same, the effects experienced by the pixels Gr1 to Gr4 may be approximately the same. That is, the values of the crosstalk C1 to C4 may be approximately the same. However, since the height of the color filters covering the pixels Gb1 to Gb4 is the same as the height of the third color filter CF3 covering the pixels Gr1 to Gr4, there may be no effect on the pixels Gb1 to Gb4 due to the thickness difference of the color filters.

[0071] Next, referring together to Figures 2 to 4 and Figure 6B , the height of the first color filter CF1 covering the pixels B1 to B4 is greater than the height of the color filters covering the pixels Gr1 to Gr4 and Gb1 to Gb4. Therefore, the signals associated with the pixels B1 to B4 may not be affected by the color filters covering the pixels Gr1 to Gr4 and Gb1 to Gb4. However, the height of the first color filter CF1 covering the pixels B1 to B4 may be less than the height of the color filters covering the pixels R1 to R4. Therefore, the pixel B1 may be slightly affected by the color filter covering the pixel R4. In an embodiment, the crosstalk experienced by the pixel B1 is labeled "C5". As described above, the remaining pixels B2, B3, and B4 may be affected by the color filters covering the pixels R3, R2, and R4 (C6 to C8), respectively.

[0072] According to the above description, the pixels Gb1 to Gb4 of the second pixel group PG2 and the pixels Gr1 to Gr4 of the third pixel group PG3 may be most affected by the color filters covering the surrounding pixels. The pixels B1 to B4 of the first pixel group PG1 may be slightly affected by the color filters covering the surrounding pixels R1 to R4, while the pixels R1 to R4 of the fourth pixel group PG4 may not be affected by the color filters covering the surrounding pixels. According to the present disclosure, relative to the optically black region 110b (see Figure 2)The black level of the output dark level offset signal compensates for the crosstalk caused by the difference in the thickness (or height) of the color filter. As a result, fixed pattern noise can be removed.

[0073] Figure 7 An exemplary configuration of the ISP front-end block 16 according to an embodiment of the present disclosure is shown. For example, the ISP front-end block 200 can be implemented with Figure 1 the ISP front-end block 16. The ISP front-end block 200 may include a CDS processing block 210, a crosstalk compensation block 220, a black level compensation block 230, and a calibration data decoder 240.

[0074] The CDS processing block 210 is configured to receive and process the image data IDAT output from the image sensor 100 (refer to Figure 2 ). For example, the CDS processing block 210 may allow specific range of data to be used as valid data (or, effective data) within the output range of the image data IDAT. For example, the CDS processing block 210 may determine whether to process the data of the image data IDAT, and the data of the image data IDAT belongs to a region of a specific value or a region within a range greater than a certain amount and less than a certain amount of the specific value.

[0075] In an embodiment, the CDS processing block 210 may process the image data IDAT output in a low light environment. To remove the negative values of the image data IDAT, the CDS processing block 210 may add an ADC offset value (e.g., a constant such as 128) to the image data IDAT. Referring to Figure 8A which shows a schematic operation of correlated double sampling, since the amount of charge accumulated at the floating diffusion region FD or FD' (refer to Figure 5 ) in a low light environment is small, therefore, according to the conduction of the transfer transistors TG1 to TG4 (refer to Figure 5 ), the change "b" in the pixel signal PIX may be very small. In this case, an operation of adding a specific value to the image data IDAT may be required to remove the negative values included in the image data IDAT.

[0076] In one embodiment, the CDS processing block 210 may determine a minimum value max_pos_value (refer to Figure 9 ) for processing the image data IDAT as black. For example, the CDS processing block 210 may invert the sign of the image data IDAT having a negative value, and may determine the two's complement of the value of the image data IDAT having the inverted sign as the minimum value max_pos_value. For example, the minimum value max_pos_value may have a value approximately "2047", which is the maximum value of the 11-bit image data IDAT.

[0077] In an embodiment, the CDS processing block 210 may process image data IDAT output in a high illuminance environment. Referring together to the Figure 8B which schematically shows the operation of correlated double sampling, since in a high illuminance environment the amount of charge collected at the floating diffusion region FD or FD’ (refer to Figure 5 ) is large, therefore, according to the conduction of the transfer transistors TG1 to TG4 (refer to Figure 5 ), the change “b” in the pixel signal PIX may be very large. In this case, the CDS processing block 210 may determine a boundary value clip_up_value (refer to Figure 9 ) for processing the image data IDAT as white. A range equal to or greater than the boundary value clip_up_value may be considered a saturation state.

[0078] In addition, in the case where the image data IDAT is distorted due to light entering the optically black region 110b (refer to Figure 2 ) (where light is normally not blocked) in a high illuminance environment, the CDS processing block 210 may perform various operations to calibrate the distortion.

[0079] To describe the operation of the CDS processing block 210, the correlated double sampling operation is schematically shown in Figure 8A and Figure 8B . The correlated double sampling operation generates a count value CNT as a result of counting the comparison result by comparing the pixel signal PIX with a ramp signal RAMP and using a count clock CNT_CLK. However, since correlated double sampling itself is well known, additional description will be omitted to avoid redundancy.

[0080] Returning to Figure 7 , the crosstalk compensation block 220 may receive the image data IDAT_pre preprocessed by the CDS processing block 210. The crosstalk compensation block 220 may multiply the code level of the image data IDAT_pre by a calibration value α for compensating crosstalk caused by a difference in height (or, thickness) of color filters. As a result, the dark level may also be multiplied by the calibration value α, which is the same as the signal level included in the code level. For example, as described in reference to Figure 6A and Figure 6B , the pixels Gr1 to Gr4 of the third pixel group PG3 may be significantly affected by color filters covering adjacent pixels. Therefore, the code level of the image data IDAT may be different from the expected original value, resulting in a reduced dynamic range. For example, the calibration value α represented by the following equation 1 and the code level of the image data IDAT_pre may be multiplied together to widen the reduced dynamic range again. The crosstalk compensation block 220 may output the crosstalk-compensated image data IDAT_XT.

[0081] [Equation 1]

[0082]

[0083] In the above Equation 1, "Value_Gr1" indicates the pixel value (i.e., code level) of the image data output from pixel Gr1, and "Value_Gr2" indicates the pixel value of the image data output from pixel Gr2. "Value_Gr3" indicates the pixel value of the image data output from pixel Gr3, and "Value_Gr4" indicates the pixel value of the image data output from pixel Gr4. "Value_Gri" (where i is from 1 to 4) indicates the pixel value of the image data output from the pixel that is the target of crosstalk compensation.

[0084] The dark level compensation block 230 can subtract the dark level data from the image data IDAT_XT to generate the dark level compensated image data IDAT_DK. For example, the dark level data, which is a digital signal corresponding to the dark level offset signal output from the optical black area 110b (reference Figure 2 ) may include the dark level. However, the dark level compensation block 230 may not only subtract the dark level data from the image data IDAT_XT, but also apply a calibration value α to the dark level data.

[0085] However, in the absence of crosstalk compensation for the dark level data, a reduction in the dynamic range may be inevitable. The reason is that when performing crosstalk compensation on the image data, the correction value is applied even to the dark level included in the code level of the image data, so the dynamic range is reduced by as much as the dark level of the image data. According to the dark level compensation of the present disclosure, the multiplied dark level of the crosstalk compensated image data IDAT_XT and the dark level of the dark level data can be completely canceled out, so that the dark level can be removed from the code level of the image data IDAT.

[0086] The dark level compensation block 230 can add a reference value having a specific value (e.g., a constant such as 64) to the dark level compensated image data IDAT_DK. For example, the added reference value can be used to prevent the dark level compensated image data IDAT_DK from having a negative value. After performing the above set of operations, the dark level compensation block 230 can clip the dark level compensated image data IDAT_DK. For example, the dark level compensation block 230 can clip the dark level compensated image data IDAT_DK to be suitable for processing by the image signal processor 18 (reference Figure 1 ). As a result, 10-bit image data IDAT_DK can be obtained, and the obtained image data IDAT_DK can be provided to the image signal processor 18 (reference Figure 2 ).

[0087] The crosstalk calibration data decoder 240 can receive crosstalk calibration data from an external source (e.g., a memory). For example, the calibration values, which are pre-measured values, can be compressed, and the compressed values can be stored as crosstalk calibration data in an external memory. The crosstalk calibration data decoder 240 can decode the crosstalk calibration data and output a calibration value α corresponding to each of the pixels that make up the pixel array 110 (refer to Figure 2 ). For example, the crosstalk calibration data can include representative values (e.g., average value, maximum value, or minimum value) of the calibration values of the pixels in each row (or column) of the pixel array 110, and the crosstalk calibration data decoder 240 can generate the calibration values of the pixels in each row (or column) through an interpolation scheme or the like.

[0088] However, in another embodiment, the calibration values of the pixels in each row (or column) of the pixel array 110 can be directly received from an external source (e.g., a memory). In this case, the calibration values can be pre-measured values and can be provided to the ISP front-end block 200 without encoding and decoding processes. Therefore, in the embodiment where the calibration values are directly received from an external source, the crosstalk calibration data decoder 240 shown in Figure 7 may not be required.

[0089] Figure 9 Schematically shows the image data processed by the Figure 7 ISP front-end block 200.

[0090] Referring together to Figure 7 and 9 , the CDS processing block 210 can add an ADC offset value to the image data IDAT received from the image sensor 14 (refer to Figure 2 ). The CDS processing block 210 can determine Max_pos_value for black processing of pixels when the image data IDAT includes negative values, and can determine Clip_up_value for white processing of the image data IDAT in a high illuminance environment. Although not shown in the drawings, the code level of the image data IDAT_pre output from the CDS processing block 210 can be multiplied by the calibration value α, and thus, crosstalk caused by the thickness (or height) of the color filter can be compensated.

[0091] The dark level compensation block 230 can compensate for the dark level of the image data IDAT_pre. For example, in dark level compensation, the dark level compensation block 230 can also multiply the subtracted dark level offset signal by the calibration value α. This is in Figure 9is represented by "Dark Xα". The dark level compensation block 230 can add a reference value to the image data IDAT_DK after dark level compensation and can output 10-bit image data IDAT_DK suitable for processing by the image signal processor 18.

[0092] Figure 10 shows a Figure 1 planar view of an image sensor according to an embodiment. The image sensor 300 may include pixel groups PG1 to PG4, and each of these pixel groups may include unit pixel groups UPG1 to UPG3. The pixel groups PG1 to PG4 may be repeatedly formed on the substrate of the image sensor 300 along the X-axis and the Y-axis. Each of the pixel groups PG1 to PG4 may include 3×3 pixels arranged along the X-axis and the Y-axis, but the present disclosure is not limited thereto. Each pixel may include a photoelectric conversion element PD.

[0093] The arrangement of the unit pixel groups UPG1 to UPG3 will be described with reference to the first pixel group PG1. Each of the unit pixel groups UPG1 to UPG3 may include 1×3 pixels arranged along the X-axis and the Y-axis. A unit pixel group may be defined as a group of pixels that share the same floating diffusion region. For example, even if the unit pixel groups belong to the same pixel group, the floating diffusion region shared by the pixels belonging to one unit pixel group may not be shared by the pixels belonging to another unit pixel group. The unit pixel groups UPG1 to UPG3 may be adjacent to each other in the X-axis direction. Therefore, as Figure 10 shown, in the case where one pixel group includes three unit pixel groups, one pixel group may include 3×3 pixels arranged along the X-axis and the Y-axis.

[0094] Color filters CF1 to CF4 for passing light of a specific wavelength band may be formed on the pixel groups PG1 to PG4, respectively. As in the Figure 3 embodiment, the first color filter CF1 may pass blue light, the second color filter CF2 and the third color filter CF3 may pass green light, and the fourth color filter CF4 may pass red light. A microlens corresponding to the pixel may be provided on each of the color filters.

[0095] Figure 11 is a cross-sectional view of the image sensor taken along the Figure 10 line III-III'. Figure 11 The cross-sectional view of Figure 4 is substantially similar to the cross-sectional view of Figure 4 and the pixel structure is similar to the pixel structure of Figure 11, light L1 can be incident between the third pixel PX3 and the fourth pixel PX4, and light L2 can be incident between the sixth pixel PX6 and the seventh pixel PX7. Of course, lights L1 to L2 can come from one light source or multiple light sources, and it is understood that this classification is for convenience.

[0096] Since the thickness d1 of the first color filter CF1 or the height of the first color filter CF1 from the fixed charge layer 3 is different from the thickness d3 of the third color filter CF3 or the height of the third color filter CF3 from the fixed charge layer 3, the signal output from the fourth pixel PX4 may be affected by the first color filter CF1. The area of the fourth pixel PX4 affected by the first color filter CF1 is marked by "a". As described above, since the thickness d4 of the fourth color filter CF4 or the height of the fourth color filter CF4 from the fixed charge layer 3 is different from the thickness d3 of the third color filter CF3 or the height of the third color filter CF3 from the fixed charge layer 3, the signal output from the sixth pixel PX6 may be affected by the fourth color filter CF4. The area of the sixth pixel PX6 affected by the fourth color filter CF4 is marked by "b".

[0097] However, different from Figure 2 the image sensor 100 shown, even if the pixels belong to one pixel group (i.e., PG3), due to the different environments around the pixels, the degree of crosstalk experienced by each pixel may be different. For example, since the thickness (or height) of the first color filter CF1 affecting the fourth pixel PX4 is different from the thickness (or height) of the fourth color filter CF4 affecting the sixth pixel PX6, the crosstalk experienced by the fourth pixel PX4 may be different from the crosstalk experienced by the sixth pixel PX6.

[0098] In addition, since the pixel set in the center of the pixels in the third pixel group PG3 (e.g., PX5) is not affected by adjacent color filters, crosstalk caused by the height (or thickness) difference of the color filters may not occur at the pixel PX5. Therefore, various methods for compensating crosstalk can be considered, and these methods will be described more comprehensively with reference to Figure 13A and Figures 13B to 15 more fully.

[0099] Figure 12 shows Figure 10 an exemplary circuit diagram of one of the pixel groups PG1 to PG4 of Figure 10The second unit pixel group UPG2 of the first pixel group PG1 will be described as an example. The second unit pixel group UPG2 includes pixels PX1 to PX3. The second unit pixel group UPG2 may include photoelectric conversion elements PD1 to PD3, transfer transistors TG1 to TG3, dual conversion transistors DCT, reset transistors RT, drive transistors DT, and selection transistors ST. The basic operations of the components of each pixel are substantially similar to those described with reference to Figure 5 Therefore, the differences between them will be mainly described.

[0100] When the transfer transistors TG1 to TG3 are turned on, the charges generated by the photoelectric conversion elements PD1 to PD3 can be transferred to the first floating diffusion region FD1, and a signal corresponding to the potential of the first floating diffusion region FD1 can be output through the second column line CL2. However, the first floating diffusion capacitor CFD1 of the first floating diffusion region FD1 can be easily saturated. In this case, the first floating diffusion region FD1 and the second floating diffusion region FD2 can be electrically connected by turning on the dual conversion transistor DCT. As a result, the capacitance of the first floating diffusion region FD1 can be extended to the sum of the capacitance of the first floating diffusion region FD1 and the capacitance of the second floating diffusion region FD2.

[0101] In addition, the second floating diffusion region FD2 can be electrically connected to the floating diffusion region of an adjacent unit pixel group (not shown) through the line L1. In this case, the capacitance of the first floating diffusion region FD1 can be further extended, thereby preventing saturation.

[0102] Figure 13A and Figure 13B conceptually shows the influence of peripheral pixels on Figures 10 to 12 the pixel group of the image sensor. Referring together to Figure 10 , Figure 11 and Figure 13A , each of the remaining pixels among the pixels Gr1 to Gr9 that make up the third pixel group PG3 except for the fifth pixel Gr5 may be affected by a color filter covering at least one adjacent pixel.

[0103] First, the pixel Gr1 may be affected by a color filter that passes red light and covers the pixel R3 and a color filter that passes blue light and covers the pixel B7, and this influence is marked by crosstalk C1. As described above, the pixels Gr3, Gr7, and Gr9 may be affected by a color filter that passes red light and a color filter that passes blue light, and this influence is marked by crosstalk C3, crosstalk C7, and crosstalk C9, respectively.

[0104] Pixel Gr2 may be affected by a color filter that allows blue light to pass through and covers pixel B8, and this effect is labeled by crosstalk C2. Pixel Gr8 may be affected by a color filter that allows blue light to pass through and covers pixel B2, and this effect is labeled by crosstalk C8.

[0105] Pixel Gr4 may be affected by a color filter that allows red light to pass through and covers pixel R6, and this effect is labeled by crosstalk C4. Pixel Gr6 may be affected by a color filter that allows red light to pass through and covers pixel R4, and this effect is labeled by crosstalk C6.

[0106] The degrees of crosstalk C1, C3, C7, and C9 experienced by pixels Gr1, Gr3, Gr7, and Gr9 respectively may be roughly similar, the degrees of crosstalk C2 and C8 experienced by pixels Gr2 and Gr8 respectively may be roughly similar, and the degrees of crosstalk C4 and C6 experienced by pixels Gr4 and Gr6 respectively may be roughly similar. Therefore, the values of the signals output from pixels Gr1, Gr3, Gr7, and Gr9 with the greatest influence of the surrounding color filters can be minimized roughly, and the signal magnitudes can increase successively from the values of the signals output from pixels Gr4 and Gr6 to the values of the signals output from pixels Gr2 and Gr8. Of course, the value of the signal output from pixel Gr5, which is not affected by the surrounding color filters, can be the largest roughly.

[0107] Next, referring together to Figure 10 、 Figure 11 and Figure 13B , the height of the first color filter CF1 covering pixels B1 to B9 is less than the height of the color filters covering pixels R1, R3, R7, and R9. Therefore, pixel B1 may be slightly affected by the color filter covering pixel R9, and this filter is labeled by crosstalk C1. As described above, the other pixels B3, B7, and B9 may be affected by the surrounding color filters, which are labeled by crosstalk C3, crosstalk C7, and crosstalk C9 respectively.

[0108] According to the above description, the pixels Gb1 to Gb9 of the second pixel group PG2 and the pixels Gr1 to Gr9 of the third pixel group PG3 may be most affected by the color filters covering the surrounding pixels. The pixels B1, B3, B7, and B9 of the first pixel group PG1 may be slightly affected by the color filters covering the surrounding pixels R1, R3, R7, and R9, while the pixels R1 to R9 of the fourth pixel group PG4 may not be affected by the color filters covering the surrounding pixels. According to the present disclosure, crosstalk caused by the difference in thickness (or, height) of the color filters is compensated for with respect to the dark level measured by the optically black region 110b (refer to Figure 2 ). As a result, fixed pattern noise can be removed.

[0109] Figures 14A to 14CSchematically shows various methods for determining calibration values.

[0110] First, referring to Figure 14A , the horizontal axis represents pixel values (i.e., code levels). In the embodiment of Figure 14A , the calibration value α can be determined based on the average value MV1 of all pixel values Value_Gr1 to Value_Gr9. For example, pixel values placed to the left of the average value MV1 indicate values output from pixels with a relatively high crosstalk effect. However, the average value MV1 is exemplary. For example, depending on the height difference of various color filters, the average value MV1 can be a value between the pixel value Value_Gr1 and the pixel value Value_Gr4. In this case, the calibration value α can be determined by Equation 2 below.

[0111] [Equation 2]

[0112]

[0113] For example, in the case of compensating the signal output from pixel Gr1, the magnitude of the pixel value Value_Gr1 output from pixel Gr1 can be less than the average value MV1. The code level of the image data IDAT_pre (refer to Figure 7 ) can be multiplied by the calibration value α so that the pixel value Value_Gr1 is equal to the average value MV1 (in Equation 2 above, i is an integer between 1 and 9). Conversely, in the case of performing dark level compensation on the signal output from pixel Gr2, the magnitude of the pixel value Value_Gr2 output from pixel Gr2 can be greater than the average value MV1. Therefore, the code level of the image data IDAT_pre (refer to Figure 7 ) can be multiplied by the calibration value α so that the pixel value Value_Gr2 is equal to the average value MV1. Moreover, the code level of the dark level data can be multiplied by the calibration value α. As an example, the calibration value α for a single specific pixel can be determined by dividing the average value MV1 of all pixels in the pixel group (e.g., the average code level corresponding to the intensity or strength of the signal) by the value of the specific pixel. Therefore, if a specific pixel has more crosstalk and a level lower than the average level, the calibration value α will be greater than 1, and if the crosstalk of the specific pixel is smaller (less than the average value), the calibration value α will be less than 1. Then, when an image is received, for a specific pixel, any code level received for that pixel is multiplied by a pre-determined calibration value α, thereby appropriately compensating the pixel code level.

[0114] In Figure 14BIn an embodiment, the calibration value α can be determined based on the average value MV2 of the pixel values Value_Gr2 and Value_Gr8. For example, the pixel value located to the left of the average value MV2 indicates a value output from a pixel with a relatively high crosstalk effect. In this case, the calibration value α can be determined by Equation 3 below.

[0115] [Equation 3]

[0116]

[0117] For example, in the case of compensating the signal output from pixel Gr1, the magnitude of the pixel value Value_Gr1 output from pixel Gr1 can be smaller than the average value MV2. Therefore, the code level of the image data IDAT_pre (refer to Figure 7 ) can be multiplied by the calibration value α so that the pixel value Value_Gr1 is equal to the average value MV2. Even if pixel Gr5 is not affected by the color filter covering other adjacent pixel groups, crosstalk compensation and dark level compensation can still be performed on the signal output from pixel PX5. For example, the magnitude of the pixel value Value_Gr5 output from pixel Gr5 can be larger than the average value MV2. Therefore, the code level of the image data IDAT_pre (refer to Figure 7 ) can be multiplied by the calibration value α so that the pixel value Value_Gr5 is equal to the average value MV2. Moreover, the code level of the dark level data can be multiplied by the calibration value α.

[0118] Finally, in Figure 14C 's embodiment, the calibration value α can be determined based on the pixel value Value_Gr5. In this case, considering that pixel Gr5 is not affected by the color filter covering the adjacent pixel group, dark level compensation may not be performed on the pixel value Value_Gr5. In this case, the calibration value α can be determined by Equation 4 below.

[0119] [Equation 4]

[0120]

[0121] For example, in the case of compensating the signal output from pixel Gr1, the magnitude of the pixel value Value_Gr1 output from pixel Gr1 can be smaller than the pixel value Value_Gr5. Therefore, the code level of the image data IDAT_pre (refer to Figure 7 ) can be multiplied by the calibration value α so that the pixel value Value_Gr1 is equal to the pixel value Value_Gr5. Moreover, the code level of the dark level data can be multiplied by the calibration value α. In the above Figures 14A to 14CIn three examples, the highest value of the equation for determining the calibration value α can be described as a pixel group reference value, such that the calibration value α is determined by dividing the pixel group reference value by the pixel value output from a specific pixel for which the calibration value α is determined. However, it should be noted that the calibration value α can also be inverted (e.g., dividing the pixel value output from a specific pixel for which the calibration value α is determined by the pixel group reference value), as long as the same equation is used to determine all the calibration values α for each pixel in the pixel group.

[0122] Although not described as a separate embodiment, the calibration value α can be determined based on the average of the pixel values Value_Gr4 and Value_Gr6. Additionally, the calibration value α can be determined by using various values.

[0123] Figure 15 A method for processing signals output from an image sensor according to an embodiment of the present disclosure is shown. For example, as described in the specification, an image sensor can be implemented in such a way that multiple pixels share a floating diffusion region. For better understanding, reference will be made together Figure 2 and Figure 7 for a description.

[0124] In operation S110, the image sensor generates image data based on signals output from pixels selected from the pixels in the active pixel region. Specifically, the ADC block 140 can convert the pixel signals output from the selected pixels into digital signals to generate image data. For example, the image data can include a code level that includes a signal level and a dark level. The signal level can correspond to the relative intensity of the received light. The dark level can be based on the current generated by the pixel regardless of the intensity of the received light.

[0125] In operation S120, the image sensor generates dark level data based on signals output from the optically black region. Specifically, the ADC block 140 can convert the dark level offset signals output from the pixels in the optically black region into digital signals to generate dark level data. For example, the dark level data can correspond to the dark level. Although steps S110 and S120 are shown in a specific order, they can occur in the reverse order.

[0126] In operation S130, the ISP front-end block 200 receives crosstalk calibration data from the outside (e.g., from outside the image sensor 14 or outside the image processing block 10). For example, the crosstalk calibration data can be data encoded according to pre-measured calibration values. For example, the crosstalk calibration data can be stored in a memory located inside the image processing block 10 (refer to Figure 1 ) or stored in a memory located outside the image processing block 10 (e.g., a host memory).

[0127] In operation S140, the crosstalk calibration data decoder 240 of the ISP front-end block 200 decodes the crosstalk calibration data to generate calibration values. For example, the calibration data may include representative values of calibration values for specific rows or columns, and the calibration values for specific rows or columns may be generated through an interpolation scheme or the like.

[0128] In operation S150, crosstalk compensation may be performed on the image data. For example, the crosstalk compensation may be associated with compensating for optical crosstalk caused by a height difference between a first color filter covering pixels sharing a floating diffusion region and a second color filter covering other pixels sharing another floating diffusion region. For example, the crosstalk compensation may be based on the average value of the image data of pixels sharing the floating diffusion region or the ratio of the pixel group reference value to the image data of pixels selected from the pixels sharing the floating diffusion region.

[0129] In operation S160, crosstalk compensation may be performed on the dark level data. Since the dark level included in the code level of the image data is multiplied by the calibration value in operation S150, even if crosstalk compensation is performed on the image data, the dynamic range may be reduced as much as the dark level of the code level. Therefore, in the dark level compensation, the dark level of the dark level data may be multiplied by the calibration value.

[0130] In operation S170, a subtraction operation may be performed on the crosstalk-compensated image data and the crosstalk-compensated dark level data (e.g., subtracting the crosstalk-compensated dark level data from the crosstalk-compensated image data). As a result, the dark level may be completely removed from the code level of the image data.

[0131] Figure 16 An exemplary configuration of an electronic device including a multi-camera module to which the crosstalk compensation of the present disclosure is applied is shown. Figure 17 Shown is Figure 16 an exemplary configuration of the camera module.

[0132] Referring to Figure 16 , the electronic device 1000 may include a camera module group 1100, an application processor 1200, a PMIC 1300, and an external memory 1400.

[0133] The camera module group 1100 may include a plurality of camera modules 1100a, 1100b, and 1100c. Shown in Figure 16 is an electronic device including three camera modules 1100a, 1100b, and 1100c, but the present disclosure is not limited thereto. In some embodiments, the camera module group 1100 may be modified to include only two camera modules. Moreover, in some embodiments, the camera module group 1100 may be modified to include "n" camera modules (n is a natural number of 4 or greater).

[0134] Next, with reference to Figure 17 the detailed configuration of the camera module 1100b will be described more comprehensively, but the following description can be equally applied to the remaining camera modules 1100a and 1100c.

[0135] With reference to Figure 17 , the camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140, and a storage device 1150.

[0136] The prism 1105 may include a reflection plane 1107 of a light reflecting material and may change the path of light "L" incident from the outside.

[0137] In some embodiments, the prism 1105 may change the path of the light "L" incident in the first direction "X" to a second direction "Y" perpendicular to the first direction "X". Moreover, the prism 1105 may change the path of the light "L" incident in the first direction "X" to a path in the second direction "Y" perpendicular to the first direction "X" by rotating the reflection plane 1107 of the light reflecting material about the central axis 1106 in the direction "A" or by rotating the reflection plane 1107 of the light reflecting material about the central axis 1106 in the direction "B". In this case, the OPFE 1110 may move in a third direction "Z" perpendicular to the first direction "X" and the second direction "Y".

[0138] In some embodiments, as shown, the maximum rotation angle of the prism 1105 in the "A" direction may be less than or equal to 15 degrees in the positive A direction and greater than 15 degrees in the negative A direction, but the present disclosure is not limited thereto.

[0139] In some embodiments, the prism 1105 may move within approximately 20 degrees, between 10 degrees and 20 degrees, or between 15 degrees and 20 degrees in the positive or negative B direction; herein, the prism 1105 may move at the same angle in the positive or negative B direction, or may move at a similar angle within approximately 1 degree.

[0140] In some embodiments, the prism 1105 may move the reflection plane 1107 of the light reflecting material in a third direction (e.g., the Z direction) parallel to the direction in which the central axis 1106 extends.

[0141] The OPFE 1110 may include, for example, an optical lens composed of "m" groups (where "m" is a natural number). Here, the "m" lenses may be moved in the second direction "Y" to change the optical zoom ratio of the camera module 1100b. For example, when the default optical zoom ratio of the camera module 1100b is "Z", the optical zoom ratio of the camera module 1100b may be changed to 3Z, 5Z, or an optical zoom ratio of 5Z or higher by moving the "m" optical lenses included in the OPFE 1110.

[0142] The actuator 1130 may move the OPFE 1110 or the optical lens (hereinafter referred to as the "optical lens") to a specific position. For example, the actuator 1130 may adjust the position of the optical lens such that the image sensor 1142 is placed at the focal length of the optical lens for precise sensing.

[0143] The image sensing device 1140 may include an image sensor 1142, control logic 1144, and a memory 1146. The image sensor 1142 may sense an image of a sensing target by using the light "L" provided through the optical lens. The control logic 1144 may control the overall operation of the camera module 1100b. For example, the control logic 1144 may control the operation of the camera module 1100b based on a control signal provided through the control signal line CSLb. Additionally, the control logic 1144 may include an ISP front-end block for performing dark level compensation of the present disclosure.

[0144] The memory 1146 may store information for the operation of the camera module 1100b, such as calibration data 1147. The calibration data 1147 may include information for the camera module 1100b to generate image data by using the light "L" provided from the outside. The calibration data 1147 may include, for example, information about the above-mentioned rotation degree, information about the focal length, information about the optical axis, etc. In the case where the camera module 1100b is implemented in the form of a multi-state camera in which the focal length varies depending on the position of the optical lens, the calibration data 1147 may include the focal length value of each position (or state) of the optical lens and information about autofocus. The calibration data 1147 may include crosstalk calibration data for performing dark level compensation of the present disclosure (for example, refer to Figure 7 ).

[0145] The storage 1150 may store the image data sensed by the image sensor 1142. The storage 1150 may be arranged outside the image sensing device 1140 and may be implemented in a shape in which the storage 1150 and the sensor chip constituting the image sensing device 1140 are stacked. In some embodiments, the storage 1150 may be implemented with an electrically erasable programmable read-only memory (EEPROM), but the present disclosure is not limited thereto.

[0146] Refer together Figure 16 and Figure 17 In some embodiments, each of the plurality of camera modules 1100a, 1100b, and 1100c may include an actuator 1130. In this way, the same calibration data 1147 or different calibration data 1147 may be included in the plurality of camera modules 1100a, 1100b, and 1100c depending on the operation of the actuator 1130 therein.

[0147] In some embodiments, one of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., 1100b) may be a camera module in the shape of a folding lens, in which the above-described prism 1105 and OPFE 1110 are included, and the remaining camera modules (e.g., 1100a and 1100c) may be camera modules in a vertical shape, in which the above-described prism 1105 and OPFE 1110 are not included; however, the present disclosure is not limited thereto.

[0148] In some embodiments, one of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., 1100c) may be a vertical-shaped depth camera that extracts depth information by using infrared rays (IR). In this case, the application processor 1200 may combine the image data provided from the depth camera and the image data provided from any other camera module (e.g., 1100a or 1100b), and may generate a three-dimensional (3D) depth image.

[0149] In some embodiments, at least two of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., 1100a and 1100b) may have different fields of view. In this case, at least two of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., 1100a and 1100b) may include different optical lenses, but are not limited thereto.

[0150] Moreover, in some embodiments, the fields of view of the plurality of camera modules 1100a, 1100b, and 1100c may be different. In this case, the plurality of camera modules 1100a, 1100b, and 1100c may include different optical lenses, but are not limited thereto.

[0151] In some embodiments, the plurality of camera modules 1100a, 1100b, and 1100c may be set to be physically separated from each other. For example, the plurality of camera modules 1100a, 1100b, and 1100c may not use the sensing area of one image sensor 1142, but the plurality of camera modules 1100a, 1100b, and 1100c may each include an independent image sensor 1142 therein.

[0152] Return to Figure 16 Figure 16 , the application processor 1200 may include an image processing device 1210, a memory controller 1220, and an internal memory 1230. The application processor 1200 may be implemented separately from the plurality of camera modules 1100a, 1100b, and 1100c. For example, the application processor 1200 and the plurality of camera modules 1100a, 1100b, and 1100c may be implemented with separate semiconductor chips.

[0153] The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b, and 1212c, an image generator 1214, and a camera module controller 1216. The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b, and 1212c, and the number of these sub-image processors corresponds to the number of the plurality of camera modules 1100a, 1100b, and 1100c.

[0154] Image data respectively generated from the camera modules 1100a, 1100b, and 1100c may be respectively provided to the corresponding sub-image processors 1212a, 1212b, and 1212c through separate image signal lines ISLa, ISLb, and ISLc. For example, the image data generated from the camera module 1100a may be provided to the sub-image processor 1212a through the image signal line ISLa, the image data generated from the camera module 1100b may be provided to the sub-image processor 1212b through the image signal line ISLb, and the image data generated from the camera module 1100c may be provided to the sub-image processor 1212c through the image signal line ISLc. For example, this image data transmission may be performed by using a camera serial interface (CSI) based on MIPI (Mobile Industry Processor Interface), but the present disclosure is not limited thereto.

[0155] In some embodiments, one sub-image processor may be set to correspond to a plurality of camera modules. For example, the sub-image processor 1212a and the sub-image processor 1212c may be integrally implemented instead of being separated from each other as shown in Figure 12 ; in this case, one of the image data respectively provided from the camera module 1100a and the camera module 1100c may be selected by a selection element (e.g., a multiplexer), and the selected image data may be provided to the integrated sub-image processor.

[0156] Image data respectively provided to the sub-image processors 1212a, 1212b, and 1212c can be provided to the image generator 1214. Depending on the generation information (or, image generation information) or the mode signal, the image generator 1214 can generate an output image by using the image data respectively provided from the sub-image processors 1212a, 1212b, and 1212c.

[0157] Specifically, depending on the generation information or the mode signal, the image generator 1214 can generate an output image by combining at least a part of the image data respectively generated from the camera modules 1100a, 1100b, and 1100c having different fields of view. In addition, depending on the generation information or the mode signal, the image generator 1214 can generate an output image by selecting one of the image data respectively generated from the camera modules 1100a, 1100b, and 1100c having different fields of view.

[0158] In some embodiments, the generation information can include a zoom signal or a zoom factor. Moreover, in some embodiments, the mode signal can be, for example, a signal based on a mode selected by a user.

[0159] In the case where the generation information is a zoom signal (or, zoom factor) and the camera modules 1100a, 1100b, and 1100c have different fields of view, the image generator 1214 can perform different operations depending on the type of the zoom signal. For example, in the case where the zoom signal is a first signal, the image generator 1214 can combine the image data output from the camera module 1100a with the image data output from the camera module 1100c, and can generate an output image by using the combined image signal and the image data output from the camera module 1100b that is not used in the combining operation.

[0160] In the case where the zoom signal is a second signal different from the first signal, without an image data combining operation, the image generator 1214 can select one of the image data respectively output from the camera modules 1100a, 1100b, and 1100c, and can output the selected image data as the output image. However, the present disclosure is not limited thereto, and if necessary, the manner of processing the image data can be modified without limitation.

[0161] In some embodiments, the image generator 1214 can generate combined image data with an increased dynamic range by receiving a plurality of image data having different exposure times from at least one of the plurality of sub-image processors 1212a, 1212b, and 1212c and performing high dynamic range (HDR) processing on the plurality of image data.

[0162] The camera module controller 1216 may provide control signals to the camera modules 1100a, 1100b, and 1100c, respectively. The control signals generated from the camera module controller 1216 may be provided to the corresponding camera modules 1100a, 1100b, and 1100c through the separate control signal lines CSLa, CSLb, and CSLc, respectively.

[0163] One of the plurality of camera modules 1100a, 1100b, and 1100c may be designated as a main camera (e.g., 1100b) depending on the generation information including a zoom signal or a mode signal, and the remaining camera modules (e.g., 1100a and 1100c) may be designated as slave cameras. The above designation information may be included in the control signal, and the control signal including the designation information may be provided to the corresponding camera modules 1100a, 1100b, and 1100c through the separate control signal lines CSLa, CSLb, and CSLc, respectively.

[0164] The camera modules serving as the main camera and the slave cameras may be changed depending on a zoom factor or an operation mode signal. For example, when the field of view of the camera module 1100a is wider than the field of view of the camera module 1100b and the zoom factor indicates a low zoom ratio, the camera module 1100b may operate as the main camera, and the camera module 1100a may operate as the slave camera. On the contrary, when the zoom factor indicates a high zoom ratio, the camera module 1100a may be used as the main camera, and the camera module 1100b may operate as the slave camera.

[0165] In some embodiments, the control signals provided from the camera module controller 1216 to each of the camera modules 1100a, 1100b, and 1100c may include a synchronization enable signal. For example, when the camera module 1100b serves as the main camera and the camera modules 1100a and 1100c serve as the slave cameras, the camera module controller 1216 may transmit the synchronization enable signal to the camera module 1100b. The camera module 1100b provided with the synchronization enable signal may generate a synchronization signal based on the provided synchronization enable signal, and may provide the generated synchronization signal to the camera modules 1100a and 1100c through the synchronization signal line SSL. The camera module 1100b and the camera modules 1100a and 1100c may be synchronized with the synchronization signal to transmit image data to the application processor 1200.

[0166] In some embodiments, the control signals provided to each of the camera modules 1100a, 1100b, and 1100c from the camera module controller 1216 may include mode information according to a mode signal. Based on the mode information, the plurality of camera modules 1100a, 1100b, and 1100c may operate in a first operation mode and a second operation mode regarding sensing speed.

[0167] In the first operation mode, the plurality of camera modules 1100a, 1100b, and 1100c may generate an image signal at a first speed (e.g., may generate an image signal at a first frame rate), may encode the image signal at a second speed (e.g., may encode an image signal at a second rate higher than the first rate), and transmit the encoded image signal to the application processor 1200. In this case, the second speed may be 30 times or less than the first speed.

[0168] The application processor 1200 may store the received image signal (i.e., the encoded image signal) in an internally provided internal memory 1230 or place it in an external memory 1400 outside the application processor 1200. Thereafter, the application processor 1200 may read the encoded image signal from the internal memory 1230 or the external memory 1400 and decode the encoded image signal, and may display the image data generated based on the decoded image signal. For example, a corresponding one of the sub-image processors 1212a, 1212b, and 1212c of the image processing device 1210 may perform decoding and may also perform image processing on the decoded image signal.

[0169] In the second operation mode, the plurality of camera modules 1100a, 1100b, and 1100c may generate an image signal at a third speed (e.g., may generate an image signal at a third frame rate lower than the first frame rate) and transmit the image signal to the application processor 1200. The image signal provided to the application processor 1200 may be an unencoded signal. The application processor 1200 may perform image processing on the received image signal or may store the image signal in the internal memory 1230 or the external memory 1400.

[0170] The PMIC 1300 may supply power, such as a power voltage, to the plurality of camera modules 1100a, 1100b, and 1100c, respectively. For example, under the control of the application processor 1200, the PMIC 1300 may supply a first power to the camera module 1100a through a power signal line PSLa, supply a second power to the camera module 1100b through a power signal line PSLb, and supply a third power to the camera module 1100c through a power signal line PSLc.

[0171] In response to a power control signal PCON from the application processor 1200, the PMIC 1300 may generate power corresponding to each of the plurality of camera modules 1100a, 1100b, and 1100c and may adjust the power level. The power control signal PCON may include power adjustment signals for each operation mode of the plurality of camera modules 1100a, 1100b, and 1100c. For example, the operation mode may include a low power mode. In this case, the power control signal PCON may include information related to the camera module operating in the low power mode and the set power level. The power levels respectively provided to the plurality of camera modules 1100a, 1100b, and 1100c may be the same as or different from each other. Moreover, the power level may be changed dynamically.

[0172] According to the present disclosure, crosstalk caused by a height difference of a multi-color filter array may be compensated. In particular, even in automatic dark level compensation (ADLC), fixed pattern noise (FPN) may be removed by compensating for crosstalk.

[0173] Although the inventive concept has been described with reference to exemplary embodiments of the inventive concept, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made thereto without departing from the spirit and scope of the inventive concept as set forth in the appended claims.

Claims

1. An electronic device, comprising: A processing block configured to receive image data from an active pixel region of an image sensor and perform preprocessing on the image data, wherein the active pixel region includes a first pixel sharing a first floating diffusion region and a second pixel sharing a second floating diffusion region; A crosstalk compensation block configured to perform crosstalk compensation on the preprocessed image data; and A dark level compensation block configured to perform the crosstalk compensation on dark level data received from an optically black region of the image sensor, and perform a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data.

2. The electronic device according to claim 1, further comprising: A decoder configured to decode received crosstalk calibration data and generate a calibration value for the crosstalk compensation.

3. The electronic device according to claim 1, further comprising: The image sensor, wherein the image sensor includes: A pixel array including the active pixel region and the optically black region; An analog-to-digital converter configured to convert a pixel signal output from the active pixel region into a digital signal to output a first value, and convert a dark level offset signal output from the optically black region into a digital signal to output a second value; and A buffer configured to amplify the first value to output the image data, and amplify the second value to output the dark level data.

4. The electronic device according to claim 3, wherein, The pixel array includes: A first pixel group including pixels sharing the first floating diffusion region; A second pixel group including pixels sharing the second floating diffusion region; A third pixel group including pixels sharing a third floating diffusion region; A first color filter formed on the first pixel group and configured to allow light of a first type to pass through; A second color filter formed on the second pixel group and configured to allow light of a second type to pass through; and A third color filter formed on the third pixel group and configured to allow light of a third type to pass through.

5. The electronic device according to claim 4, wherein, Performing crosstalk compensation for each specific pixel of the first pixel group based on a ratio of a pixel group reference value of image data of pixels of the first pixel group to the image data of a specific pixel of the first pixel group.

6. The electronic device according to claim 5, wherein, A first height of the first color filter formed on the first pixel group is different from a second height of the second color filter formed on the second pixel group or a third height of the third color filter formed on the third pixel group.

7. The electronic device according to claim 3, wherein, The pixel array includes: A first unit pixel group including pixels sharing the first floating diffusion region; A second unit pixel group including pixels sharing the second floating diffusion region; A third unit pixel group including pixels sharing a third floating diffusion region; A first color filter formed on the first unit pixel group to the third unit pixel group and configured to allow light of a first type to pass through; A fourth unit pixel group including pixels sharing a fourth floating diffusion region; A fifth unit pixel group including pixels sharing a fifth floating diffusion region; A sixth unit pixel group including pixels sharing a sixth floating diffusion region; and A second color filter, formed on the fourth to sixth unit pixel groups and configured to allow light of a second type to pass through.

8. The electronic device according to claim 7, wherein, Crosstalk compensation is performed for each pixel in the pixels of the first to third unit pixel groups based on the following ratio: the average value of the image data of at least some of the pixels in the pixels of the first to third unit pixel groups divided by the image data of each pixel in the pixels of the first to third unit pixel groups.

9. The electronic device according to claim 8, wherein, The first height of the first color filter formed on the first to third unit pixel groups is different from the second height of the second color filter formed on the fourth to sixth unit pixel groups.

10. An electronic device, comprising: An image sensor, including an active pixel region and an optically black region, the active pixel region including a plurality of unit pixel groups, each unit pixel group of the plurality of unit pixel groups including a plurality of pixels sharing a floating diffusion region, and the image sensor outputting image data corresponding to the active pixel region and outputting dark level data corresponding to the optically black region; An image signal processor ISP front-end block, configured to perform crosstalk compensation on the image data, perform crosstalk compensation on the dark level data, and perform a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data; And An image signal processor, configured to receive and process the image data on which the subtraction operation has been performed.

11. The electronic device according to claim 10, wherein, The image sensor further includes: A first color filter, formed on a first pixel group including at least one unit pixel group of the plurality of unit pixel groups; and A second color filter, formed on a second pixel group including at least one unit pixel group of the plurality of unit pixel groups.

12. The electronic device according to claim 11, wherein, Crosstalk compensation is performed for each individual pixel in the first pixel group based on the ratio of the pixel group reference value of the image data of the pixels in the first pixel group to the image data of each individual pixel in the first pixel group.

13. The electronic device according to claim 12, wherein, The first height of the first color filter formed on the first pixel group is different from the second height of the second color filter formed on the second pixel group.

14. The electronic device according to claim 10, wherein, The image sensor further includes: An analog-to-digital converter, configured to convert a pixel signal output from the active pixel region into a digital signal to output a first value, and convert a dark level offset signal output from the optically black region into a digital signal to output a second value; and A buffer, configured to amplify the first value to output the image data, and amplify the second value to output the dark level data.

15. The electronic device according to claim 10, wherein, The ISP front-end block includes: A correlated double sampling CDS processing block, configured to receive the image data from the active pixel region and perform preprocessing on the image data; A crosstalk compensation block, configured to perform the crosstalk compensation on the preprocessed image data; and A dark level compensation block, configured to perform the crosstalk compensation on the dark level data, and perform a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data.

16. A method for processing a signal output from an image sensor, the image sensor including an active pixel region and an optically black region, the active pixel region including a first pixel sharing a first floating diffusion region and a second pixel sharing a second floating diffusion region, the method comprising: generating image data based on a signal output from a pixel selected from the first pixel; generating dark level data based on a signal output from the optically black region; performing crosstalk compensation on the image data according to a height difference between a first color filter covering the first pixel and a second color filter covering the second pixel; performing the crosstalk compensation on the dark level data; and performing a subtraction operation on the crosstalk-compensated image data and the crosstalk-compensated dark level data.

17. The method according to claim 16, wherein The execution of the crosstalk compensation on the image data is performed based on the following ratio: a ratio of an average value of the image data of the first pixel to the image data of the pixel selected from the first pixel.

18. The method according to claim 17, wherein, A first height of the first color filter formed on the first pixel is different from a second height of the second color filter formed on the second pixel.

19. The method according to claim 16, further comprising: before performing the crosstalk compensation on the image data, receiving crosstalk calibration data from outside the image sensor; and decoding the crosstalk calibration data to generate a calibration value for the crosstalk compensation.

20. The method according to claim 16, further comprising: adding a reference value to the image data that has undergone the subtraction operation.

Images