Image sensor and operating method thereof

Abstract

An image sensor and a method of operating the image sensor are disclosed. An image sensor according to the technical concept of the present disclosure may include a pixel array comprising a plurality of pixels arranged in rows and columns, wherein each of the plurality of pixels includes a microlens and a first photoelectric conversion element and a second photoelectric conversion element arranged side by side in a first direction below the microlens, and a row decoder that controls the output of a first image signal generated from the first photoelectric conversion element and a summated image signal generated from the first photoelectric conversion element and the second photoelectric conversion element in sequence from a first pixel arranged in a first row of the pixel array during a first readout period, and controls the output of a second image signal generated from the second photoelectric conversion element and the summated image signal in sequence from a second pixel arranged in a second row of the pixel array during a second readout period.

Description

Technology Field

[0001] The technical concept of the present disclosure relates to an image sensor, and more specifically, to an image sensor that performs an autofocus function and a method for reading the image sensor. Background Technology

[0002] Recently, autofocus (AF) methods that automatically detect the focus of image sensors have become widely utilized. In particular, due to the characteristic of fast focus detection speeds, various studies are being conducted on phase difference autofocus (PAF) technology. In PAF, light transmitted through the imaging lens is split and detected at different locations, and the focusing lens is automatically driven to adjust the focal length so that the detected signals have the same intensity and phase. Generally, autofocus setting is performed using a separate AF sensor that is much smaller than the image sensor, or by placing focus detection pixels separately from image detection pixels on a part of the image sensor and using an AF module inside the image sensor. Recently, technologies are being researched to increase focus detection speed by configuring the focus detection pixels or all of the image detection pixels as a pair of photoelectric conversion elements. By performing phase difference autofocus detection operations on a pixel-by-pixel basis or on a certain number of adjacent pixels, the speed and accuracy of focus detection can be improved. The problem to be solved

[0003] The problem that the technical concept of the present disclosure aims to solve is to provide an image sensor that provides a high frame rate and high-speed autofocusing function, and a method of operating the image sensor. means of solving the problem

[0004] To achieve the above objectives, an image sensor according to the technical concept of the present disclosure may include a pixel array comprising a plurality of pixels arranged in rows and columns, wherein each of the plurality of pixels includes a microlens and a first photoelectric conversion element and a second photoelectric conversion element arranged side by side in a first direction below the microlens, and a row decoder that controls the sequential reading of a first image signal generated from the first photoelectric conversion element and a summated image signal generated from the first photoelectric conversion element and the second photoelectric conversion element from a first pixel arranged in a first row of the pixel array during a first reading period, and controls the sequential reading of a second image signal generated from the second photoelectric conversion element and the summated image signal from a second pixel arranged in a second row of the pixel array during a second reading period.

[0005] An image sensor according to the technical concept of the present disclosure comprises a plurality of row lines extended in a first direction, a plurality of column lines arranged in a second direction perpendicular to the first direction, and a plurality of pixels arranged in rows and columns connected to the plurality of row lines and the plurality of column lines, wherein each of the plurality of pixels comprises a pixel array including a first subpixel and a second subpixel arranged side by side, and a row decoder providing a plurality of first control signals to a first pixel arranged in a first row of the pixel array and providing a plurality of second control signals to a second pixel arranged in a second row of the pixel array, wherein the pixel array outputs, in response to a plurality of first control signals during a first readout period, a first image signal from the first subpixel of the first pixel and a first summed image signal from the first subpixel and the second subpixel of the first pixel through a first column line, and in response to a plurality of second control signals during a second readout period, a second image signal from the second subpixel of the second pixel and the first subpixel and The second summed image signal from the second subpixel can be output through the first column line.

[0006] An image sensor according to the technical concept of the present disclosure comprises a pixel array including a plurality of first pixels arranged in a first row and a plurality of second pixels arranged in a second row, a row decoder providing a plurality of control signals to the pixel array to the plurality of first pixels through a plurality of first row lines, and a readout circuit receiving a plurality of signals from the pixel array through a plurality of column lines and converting the plurality of signals from analog to digital, wherein the plurality of first pixels output a first reset signal, a first image signal from the first photoelectric conversion element, and a first summation signal from the first photoelectric conversion element and the second photoelectric conversion element in sequence to the readout circuit during a first readout period, and the plurality of second pixels output a second reset signal, a second image signal from the second photoelectric conversion element, and a second summation signal from the first photoelectric conversion element and the second photoelectric conversion element in sequence to the readout circuit during a second readout period.

[0007] A method of operating an image sensor comprising a pixel array including a first pixel disposed in a first row and a second pixel disposed in a second row according to the technical concept of the present disclosure, wherein each of the first pixel and the second pixel comprises a first photoelectric conversion element and a second photoelectric conversion element disposed in parallel, the method comprising: a step in which the first pixel outputs a first image signal from the first photoelectric conversion element; and a step in which the first pixel outputs a first summed image signal from the first photoelectric conversion element and the second photoelectric element.

[0008] The method may include the step of the second pixel outputting a second image signal from the second photoelectric conversion element, the step of the second pixel outputting a second summed image signal from the first photoelectric conversion element and the second photoelectric element, and the step of generating autofocus data based on the first image signal and the second image signal. Effects of the invention

[0009] According to the technical concept of the present disclosure, an image sensor and a method of operating the image sensor include two photoelectric conversion elements arranged side by side, each of which is a plurality of pixels of a pixel array, pixels arranged in a predetermined row output a first image signal generated from a first photoelectric conversion element, and pixels arranged in another row output a second image signal generated from a second photoelectric conversion element, and autofocusing data for an autofocusing function can be generated based on the first image signal and the second image signal. Accordingly, a plurality of pixels of the pixel array can be used to generate autofocusing data, so that an imaging device equipped with an image sensor can perform high-speed autofocusing, and the time required to convert each of the first image signal and the second image signal into a digital value without signal distortion can be reduced, and thus the frame rate of the image sensor can be increased. Brief explanation of the drawing

[0010] FIG. 1 is a block diagram showing a digital imaging device according to an exemplary embodiment of the present disclosure. FIG. 2 is a block diagram showing an image sensor according to an embodiment of the present disclosure. FIG. 3 shows a pixel array according to an exemplary embodiment of the present disclosure. FIGS. 4A and FIGS. 4B show a front view and a vertical cross-sectional view of a pixel provided in the pixel array of FIG. 3. Figure 5 is an equivalent circuit diagram of the pixel in Figure 3. FIG. 6 shows a timing diagram of an image sensor for reading a pixel signal from a pixel of FIG. 5 according to an exemplary embodiment of the present disclosure. FIG. 7 illustrates a method for outputting an image signal for generating autofocus data in a pixel array according to an exemplary embodiment of the present disclosure. FIG. 8 shows a timing diagram of an image sensor for reading a pixel signal from a pixel array according to an exemplary embodiment of the present disclosure. Figure 9 shows the transfer of photocharges in pixels according to the transmission control signals of Figure 8. FIGS. 10a and FIGS. 10b schematically show a block diagram and a timing diagram of a row decoder according to an exemplary embodiment of the present disclosure. FIG. 11a shows a pixel array according to an exemplary embodiment of the present disclosure, and FIG. 11b shows a front view of a pixel provided in the pixel array of FIG. 11a. FIG. 12 shows a pixel array according to an exemplary embodiment of the present disclosure. FIG. 13a shows a pixel array according to an exemplary embodiment of the present disclosure, and FIG. 13b shows a front view of a pixel provided in the pixel array of FIG. 13a. Figure 14 is an equivalent circuit diagram of the pixel in Figure 13a. FIG. 15 shows a timing diagram of transmission control signals applied to the pixel array of FIG. 13a according to an example of the present disclosure. FIG. 16 is a flowchart illustrating a method of operation of an image sensor according to an exemplary embodiment of the present disclosure. FIGS. 17 and FIGS. 18 are block diagrams of an electronic device including a multi-camera module. FIG. 19 is a detailed block diagram of the camera module of FIG. 17. Specific details for implementing the invention

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0013] FIG. 1 is a block diagram showing a digital imaging device (1000) according to an exemplary embodiment of the present disclosure.

[0014] A digital imaging device (1000) according to an exemplary embodiment of the present disclosure may include an imaging unit (1100), an image sensor (1200), and a processor (1300). The digital imaging device (1000) may be equipped with an autofocus detection function (auto focusing; hereinafter referred to as AF function).

[0015] The operations of the digital imaging device (1000) can be controlled by a processor (1300). The processor (1300) can provide control signals for the operation of each component to a lens drive unit (1120), an aperture drive unit (1140), a control unit (1220), etc.

[0016] The imaging unit (1100) is a component that receives light and may include a lens (1110), a lens driving unit (1120), an aperture (1130), and an aperture driving unit (1140). The lens (1110) may be equipped with a plurality of lenses.

[0017] The lens driving unit (1120) can communicate information regarding focus detection with the processor (1300) and can adjust the position of the lens (1110) according to a control signal provided by the processor (1300). The lens driving unit (1120) can adjust the position of the lens (1110) by moving the lens (1110). For example, the lens driving unit (1120) can move the lens (1110) in a direction in which the distance from the object increases or decreases, and the distance between the lens (1110) and the object can be adjusted. Depending on the position of the lens (1110), the object may be in focus or blurred.

[0018] The image sensor (1200) can convert incident light into an image signal. The image sensor (1200) may include a pixel array (1210), a control unit (1220), and a signal processing unit (1230). An optical signal transmitted through a lens (1110) and an aperture (1130) reaches the light-receiving surface of the pixel array (1210) and can form an image of a subject.

[0019] The pixel array (1210) may be a Complementary Metal Oxide Semiconductor Image Sensor (CIS) that converts an optical signal into an electrical signal. The sensitivity of such a pixel array (1210) may be controlled by a control unit (1220). The pixel array (1210) may include a plurality of pixels arranged in rows and columns, and each of the plurality of pixels may include a microlens and at least two photoelectric conversion elements arranged side by side below the microlens. Each of the plurality of pixels may include at least one first photoelectric conversion element and at least one second photoelectric conversion element arranged side by side. The pixel may output a first image signal generated from the first photoelectric conversion element or a second image signal generated from the second photoelectric conversion element. Additionally, the pixel may output a summed image signal generated from the first photoelectric conversion element and the second photoelectric conversion element.

[0020] The signal processing unit (1230) can generate a pair of phase detection signals used for phase difference calculation based on a first image signal and a second image signal output from at least two pixels placed in adjacent different rows and the same column on the pixel array (1210). When out of focus, the phases of the first image signal and the second image signal may be different. For example, the signal strengths of the first image signal and the second image signal may be different. When in focus, the phases of the first image signal and the second image signal may be the same.

[0021] The signal processing unit (1230) can generate a plurality of phase detection signal pairs based on a plurality of first image signals and a plurality of second image signals output from the pixel array (1210). The plurality of phase detection signal pairs, or the first image and the second image generated by the plurality of phase detection signal pairs, can be provided to the processor (1300) as auto-focusing data.

[0022] The processor (1300) can receive image data from the image sensor (1200). The image data may include frame-unit images and / or autofocus data. The processor (1300) can perform phase difference calculations for the AF function using the autofocus data. In an embodiment, the processor (1300) can perform phase difference calculations based on a plurality of phase detection signal pairs included in the autofocus data. For example, the processor (1300) can generate a first image based on a plurality of first phase detection signals among the plurality of phase detection signal pairs, generate a second image based on a plurality of second phase detection signals, and calculate a phase difference between the first image and the second image.

[0023] The processor (1300) can obtain the position of the focus, the direction of the focus, and / or the distance between the object and the image sensor (1200), where the signal intensity of the two phase detection signals included in the phase detection signal pair matches (i.e., the phases of the first image and the second image match), through phase difference calculation.

[0024] The processor (1300) can generate a control signal to control the lens driving unit (1120) to move the position of the lens (1110) based on the result of the phase difference calculation, and output the control signal to the lens driving unit (1120).

[0026] FIG. 2 is a block diagram showing an image sensor (100) according to an embodiment of the present disclosure.

[0027] An image sensor (100) according to an embodiment of the present disclosure may be mounted on an electronic device having an image or light sensing function, and in particular may be mounted on an electronic device having an auto-focusing function. For example, the image sensor (100) may be mounted on an electronic device such as a camera, a smartphone, a wearable device, the Internet of Things (IoT), a tablet PC (Personal Computer), a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), a navigation device, etc. Additionally, the image sensor (100) may be mounted on an electronic device equipped as a component in a vehicle, furniture, manufacturing equipment, a door, various measuring instruments, etc.

[0028] The image sensor (100) may include a pixel array (110), a row decoder (120), a ramp signal generator (130), a comparison circuit (140), a clock signal generator (150), a counter circuit (160), a timing generator (170), a column decoder (180), and a signal processing unit (190). The comparison circuit (140) and the counter circuit (160) may be referred to as readout circuits (or analog-to-digital conversion circuits).

[0029] The pixel array (110) is connected to a plurality of row lines (RL), a plurality of column lines (CL), and a plurality of row lines (RL) and a plurality of column lines (CL), and includes a plurality of pixels (PX) arranged in rows and columns.

[0030] A plurality of pixels (PX) can detect light received using a photoelectric conversion element and output an image signal, which is an electrical signal corresponding to the detected light. The plurality of pixels (PX) may include red pixels, green pixels, and blue pixels. A red pixel may generate an image signal (or charges) corresponding to a red color signal in response to wavelengths in the red region of the visible light region. A green pixel may generate an image signal (or charges) corresponding to a green color signal in response to wavelengths in the green region of the visible light region. A blue pixel may generate an image signal (or charges) corresponding to a blue color signal in response to wavelengths in the blue region of the visible light region. However, it is not limited thereto, and the plurality of pixels (PX) may further include a white pixel. As another example, the plurality of pixels (PX) may include a cyan pixel, a yellow pixel, a magenta pixel, or a white pixel.

[0031] Micro-lenses and color filters may be stacked on the upper surface of each of the multiple pixels (PX), and multiple color filters of the multiple pixels (PX) may form a color filter array. The color filter can transmit light of a specific color, that is, wavelengths of a specific color region, among the light incident through the micro-lens. The color that the pixel (PX) can detect may be determined according to the color filter provided in the pixel (PX). However, it is not limited thereto, and in the embodiment, the photoelectric conversion element provided in the pixel (PX) may convert light corresponding to wavelengths of a color region into an electrical signal according to the level of the applied electrical signal, e.g., voltage level, and accordingly, the color that the pixel (PX) can detect may be determined according to the level of the electrical signal applied to the photoelectric conversion element.

[0032] Each of the plurality of pixels (PX) may include at least two photoelectric conversion elements (or light sensing elements). For example, the photoelectric conversion elements may include a photodiode, a phototransistor, a photogate, or a pinned photodiode. The at least two photoelectric conversion elements can generate individual image signals by independently generating photocharges.

[0033] Each of the plurality of pixels (PX) may include at least one first photoelectric conversion element positioned to the left (or upward) direction centered on the optical axis of the micro-lens and at least one second photoelectric conversion element positioned to the right (or downward) direction centered on the optical axis of the lens. Each of the plurality of pixels (PX) may output a first image signal generated from the first photoelectric conversion element or a second image signal generated from the second photoelectric conversion element. Among two pixels (PX) positioned in adjacent different rows and the same column, one pixel (PX) may output a first image signal generated from the first photoelectric conversion element, and the other pixel (PX) may output a second image signal generated from the second photoelectric conversion element. At this time, the two pixels (PX) may detect the same color. Based on the first image signal and the second image signal output from the two pixels (PX), autofocus data, such as a phase detection signal pair, which is used for phase difference calculation for AF function, may be generated.

[0034] Additionally, the pixel (PX) can output a summed image signal generated from at least one first photoelectric conversion element and at least one second photoelectric conversion element. The summed image signal can be used to generate an image on a frame-by-frame basis.

[0035] The row decoder (120) can generate a plurality of control signals that can control the operation of pixels (PX) placed in each row according to the control of the timing controller (170). The row decoder (120) can provide a plurality of control signals to each of the plurality of pixels (PX) of the pixel array (110) through a plurality of row lines (RL). In response to the plurality of control signals provided by the row decoder (120), the pixel array (110) can be driven in row units. In other words, the plurality of pixels (PX) of the pixel array (110) can output pixel signals in order in row units. At this time, the pixel signals may include a reset signal indicating the reset level of the pixel (PX) and an image signal generated from a photoelectric conversion element.

[0036] The ramp signal generator (130) can generate a ramp signal (e.g., ramp voltage) whose level rises or falls at a predetermined slope under the control of the timing controller (170). The ramp signal (RAMP) can be provided to each of the plurality of comparators (141) of the comparison circuit (140).

[0037] The comparison circuit (140) includes a plurality of comparators (141) and can convert a plurality of pixel signals received from a pixel array (110) through a plurality of column lines (CL) into a plurality of digital signals. The comparator (141) can convert the received pixel signals into digital signals based on a ramp signal (RAMP). The comparator (141) can output a comparison signal that transitions from a first level (e.g., logic high) to a second level (e.g., logic low) when the level of the ramp signal (RAMP) and the level of the pixel signal are the same. The point in time when the level of the comparison signal transitions can be determined according to the level of the pixel signal.

[0038] The comparator (141) can be implemented as a circuit to which the correlation double sampling (CDS) method is applied, such as a CDS circuit. The comparator (141) can sample and hold a pixel signal provided from a pixel (PX), and can generate a comparison signal based on the level corresponding to the difference by double-sampling a specific noise level (e.g., a reset signal) and a level corresponding to an image signal. For example, the comparator (141) may include one or more OTAs (Operational Transconductance Amplifiers) (or differential amplifiers), two sample and hold capacitors connected to two input terminals of the OTA, and two switches connecting each of the two input terminals and two output terminals of the OTA during auto-zero operation. Here, the reset noise of the pixel signal and the offset of the comparator (141) can be removed by the auto-zero operation.

[0039] A clock signal generator (150) can generate a counting clock signal (CCLK) to be provided to a counter circuit (160). The generation timing and frequency of the counting clock signal (CCLK) can be controlled by a timing controller (170). In an embodiment, the clock signal generator (150) can be implemented as a Gray code generator. The clock signal generator (150) can generate a plurality of code values ​​having a resolution according to a set number of bits as a counting clock signal (CCLK). For example, if a 10-bit code is set, the clock signal generator (150) can generate a counting clock signal (CCLK) containing 1024 code values, and if an 11-bit code is set, the clock signal generator (150) can generate a counting clock signal (CCLK) containing 2048 code values.

[0040] The counter circuit (160) may include a plurality of counters (CNTR) (161). Each of the plurality of counters (161) may count the level transition point of a comparison signal output from a corresponding comparator (141) based on a counting clock signal (CCLK) and output a count value. In an embodiment, the counter (161) may include a latch circuit and an operation circuit. The latch circuit may latch a code value received as a counting clock signal (CCLK) at the time when the level of the comparison signal received from the comparator (141) transitions. The latch circuit may latch a code value corresponding to a reset signal, e.g., a reset value, and a code value corresponding to an image signal, e.g., an image signal value. The operation circuit may perform an operation on the reset value and the image signal value to generate an image signal value with the reset level of the pixel (PX) removed. The counter (161) may output the image signal value with the reset level removed as a pixel value. However, it is not limited to this, and the counter (161) may be implemented as an up-counter and operation circuit in which the count value increases sequentially based on the counting clock signal (CCLK), or an up / down counter, or a bit-wise inversion counter.

[0041] In an embodiment, the counter circuit (160) may include a plurality of memories (162), and the plurality of memories (162) may each store pixel values ​​output from a plurality of counters (161). The memory (162) may be implemented as a static random access memory (SRAM), a latch, a flip-flop, or a combination thereof, but is not limited thereto.

[0042] The column decoder (180) can control the output timing of pixel values ​​stored in a plurality of memories (162) according to the control of the timing controller (180). In an embodiment, the plurality of memories (162) can sequentially output a plurality of pixel values ​​according to the control of the column decoder (180). The plurality of pixel values ​​may be output to the outside of the image sensor (100) or to the signal processing unit (190).

[0043] The timing controller (170) can generate control signals that control the operation of the row decoder (120), ramp signal generator (130), comparator circuit (140), clock signal generator (150), counter circuit (160), and column decoder (180). The timing controller (170) can be implemented as the control unit (1220) of FIG. 1 or as a part of the control unit (1220).

[0044] The signal processing unit (190) can perform signal processing on image data received from the counting circuit (160), such as a plurality of pixel values. The signal processing unit (130) can generate auto-focusing data based on a plurality of first pixel values ​​(e.g., a plurality of first image signal values) corresponding to a plurality of first image signals and a plurality of second pixel values ​​(e.g., a plurality of second image signal values) corresponding to a plurality of second image signals.

[0045] Additionally, the signal processing unit (130) can perform noise reduction processing, gain adjustment, waveform shaping processing, interpolation processing, white balance processing, gamma processing, edge enhancement processing, binning, etc. on the image data. In an embodiment, the signal processing unit (190) may be provided in a processor (1300, FIG. 1) outside the image sensor (100).

[0047] FIG. 3 shows a pixel array according to an exemplary embodiment of the present disclosure.

[0048] Referring to FIG. 3, a pixel array (110a) includes a plurality of pixels (PXa) arranged in rows and columns. For example, the plurality of pixels (PXa) may include a plurality of green pixels (G), a plurality of blue pixels (B), and a plurality of red pixels (R). Two green pixels (G), blue pixels (B), and red pixels (R) may be arranged in a 2 x 2 row and column, and two green pixels (G) may be arranged diagonally. Such pixel patterns may be repeatedly arranged within the pixel array (110a). FIG. 3 illustrates, by way of example, 16 pixels (PXa) arranged in the first row to the fourth row (ROW1~ROW4) and the first column to the fourth column (COL1~COL1), but this is illustrated by way of example for convenience of explanation. The number of the plurality of pixels (PXa) may be determined according to the resolution of the pixel array (110a).

[0049] Each of the plurality of pixels (PXa) may include a microlens (ML) and two subpixels, such as a first subpixel (SPX1) and a second subpixel (SPX2). The subpixels, such as the first subpixel (SPX1) and the second subpixel (SPX2), may refer to a single photoelectric conversion element or a structure (or circuit) comprising a single photoelectric conversion element and an element connected one-to-one to the single photoelectric conversion element. The first subpixel (SPX1) and the second subpixel (SPX2) may be arranged side by side in the row direction, such as the X-axis direction (first direction). For example, the first subpixel (SPX1) may be placed on the left within the pixel (PXa) and the second subpixel (SPX2) may be placed on the right within the pixel (PXa).

[0050] A plurality of first subpixels (SPX1) of a plurality of pixels (PXa) may operate in response to a plurality of first transmission control signals (TG11, TG21, TG31, TG41), and a plurality of second subpixels (SPX2) may operate in response to a plurality of second transmission control signals (TG12, TG22, TG32, TG42). Here, the operation of the first subpixel (SPX1) and the second subpixel (SPX2) means that photoelectric charges generated in the photoelectric conversion elements (e.g., the first photodiode (PD1) and the second photodiode (PD2) of FIG. 4a) included in each of the first subpixel (SPX1) and the second subpixel (SPX2) are transmitted to a floating diffusion node (FD of FIG. 4a) within the pixel (PXa).

[0051] A plurality of first transmission control signals (TG11, TG21, TG31, TG41) and a plurality of second transmission control signals (TG12, TG22, TG32, TG42) may be distinct signals provided from a row decoder (120 in FIG. 2) through different row lines (RL). The first and second subpixels (SPX1, SPX2) of a pixel (PXa) placed in the same row may receive the first transmission control signal and the second transmission control signal through the same row lines (RL). The connection relationship between the row lines (RL) and the subpixels (PSX1, SPX2) is indicated through a connector (CNT).

[0052] Multiple rows of a pixel array (110a), such as first to fourth rows (ROW1 to ROW4), may be read out in sequence during multiple reading periods (or referred to as multiple horizontal periods). In other words, multiple signals may be read out in row units from multiple pixels (PXa) placed in the first to fourth rows (ROW1 to ROW4). For example, pixel signals may be output from pixels (PXa) placed in the first row (ROW1) during the first reading period, pixel signals may be output from pixels (PXa) placed in the second row (ROW2) during the second reading period, pixel signals may be output from pixels (PXa) placed in the third row (ROW3) during the third reading period, and pixel signals may be output from pixels (PXa) placed in the fourth row (ROW4) during the fourth reading period. However, this is not limited thereto, and the order in which multiple rows are read out may vary.

[0053] Meanwhile, the pixel array (110a) may be divided into a plurality of pixel groups (PG), and each of the plurality of pixel groups (PG) may include at least two pixel patterns (e.g., pixels (PXa) arranged in a 2 x 4 matrix) arranged adjacently in the Y direction (e.g., a second direction). The pixel group (PG) may be a pixel unit for generating autofocus data. For example, based on first image signals (e.g., left image signals) and second image signals (e.g., right image signals) generated from pixels (PXa) included in the pixel group (PG), a pair of phase detection signals for autofocus data, such as phase difference calculation for adjusting the left and right directions of focus, may be generated. At this time, first image signals generated in the first subpixel (SPX1) may be output from pixels (PXa) placed in a predetermined row among a plurality of rows, for example, the first row to the fourth row (ROW1~ROW4), and second image signals generated in the second subpixel (PSX2) may be output from pixels (PXa) placed in another predetermined row.

[0055] FIGS. 4A and FIGS. 4B show a front view and a vertical cross-sectional view of a pixel provided in the pixel array of FIG. 3. FIG. 4B shows a vertical cross-sectional view along line A-A' in FIG. 4A.

[0056] Referring to FIGS. 4a and 4b, a color filter (CF) and a microlens (ML) may be disposed on top of the first photodiode (PD1) and the second photodiode (PD2). The first subpixel (SPX1) may include the first photodiode (PD1), and the second subpixel (PSX2) may include the second photodiode (PD2).

[0057] A pixel (PXa) may include a first layer (L1) and a second layer (L2) stacked in the Z-axis direction (e.g., a third direction). The first layer (L1) may be referred to as a photoelectric conversion layer and may include a color filter (CF) and a microlens (ML) formed on a substrate (SUB), and two photoelectric conversion elements formed on the substrate (SUB), such as a first photodiode (PD1) and a second photodiode (PD2). The second layer (L2) may be referred to as a wiring layer, and a plurality of wirings (WS) may be formed on the second layer (L2).

[0058] The substrate (SUB) may be a silicon wafer, a Silicon on Insulator (SOI) substrate, or a semiconductor epitaxial layer. The substrate (SUB) may include a first surface (Sf) and a second surface (Sb) facing each other. For example, the first surface (Sf) may be the front surface of the substrate (SUB), and the second surface (Sb) may be the back surface of the substrate (SUB). Light may be incident on the second surface (Sb).

[0059] A plurality of pixel isolation films (SEP1, SEP2) (e.g., Deep Trench Isolation (DTI) or P-type ion implantation region) are formed on a substrate (SUB) extending from a second surface (Sb) of the substrate (SUB) toward a first surface (Sf). A pixel region (APX) in which a pixel (PXa) is formed is separated by a plurality of first pixel isolation films (SEP1) that are relatively long among the plurality of pixel isolation films (SEP1, SEP2), and the pixel region (APX) is separated by a relatively short first pixel isolation film (SEP1) into a first region (A1) and a second region (A2) in which a first subpixel (PSX1) and a second subpixel (PSX2) are formed. In an exemplary embodiment, each of the first region (A1) and the second region (A2) may be doped with an impurity of a first conductivity type (e.g., P-type). A first photodiode (PD1) and a second photodiode (PD2) may be formed in each of the first region (A1) and the second region (A2). For example, well regions doped with impurities of the second conductivity type (e.g., N-type) may be formed as the first photodiode (PD1) and the second photodiode (PD2).

[0060] As described, the first photodiode (PD1) and the second photodiode (PD2) can be arranged side by side in a first direction (e.g., X direction) or a second direction (e.g., Y direction) around the optical axis (MLX) of the microlens (ML).

[0061] A floating diffusion node (FD) may be formed between the first photodiode (PD1) and the second photodiode (PD2). In an embodiment, gate and source terminals (not shown) of transistors (TX1, TX2, RX, DX, SX in FIG. 5) may be formed between the first photodiode (PD1) and the second photodiode (PD2) and the first plane (Sf). Signals may be transmitted to and received from the transistors (TX1, TX2, RX, DX, SX in FIG. 5) through a plurality of wires (WS) of the wiring layer (L2).

[0062] Figure 5 is an equivalent circuit diagram of the pixel in Figure 3.

[0063] Referring to FIG. 5, the pixel (PXa) may include a first photodiode (PD1), a second photodiode (PD2), a first transfer transistor (TX), a second transfer transistor (TX2), a reset transistor (RX), a drive transistor (DX), and a select transistor (SX).

[0064] A floating diffusion node (FD) (or a floating diffusion region) may be shared by a first photodiode (PD1) and a second photodiode (PD2), a first transfer transistor (TX1), and a second transfer transistor (TX2). The first photodiode (PD1) and the first transfer transistor (TX1) may be referred to as a first subpixel (SPX1), and the second photodiode (PD2) and the second transfer transistor (TX2) may be referred to as a second subpixel (SPX2).

[0065] Each of the first photodiode (PD1) and the second photodiode (PD2) can generate photocharges that vary according to the intensity of light. For example, each of the first photodiode (PD1) and the second photodiode (PD2) is a PN junction diode and can generate charges, namely electrons, which are negative charges, and holes, which are positive charges, in proportion to the amount of incident light. Each of the first photodiode (PD1) and the second photodiode (PD2) may be at least one of a phototransistor, a photogate, a pinned photodiode (PPD), and combinations thereof, as an example of a photoelectric conversion device.

[0066] The floating diffusion node (FD) can operate as a capacitor. When the first transfer transistor (TX1) is turned on in response to a first transfer control signal (TS1) applied to its gate terminal, the charge (e.g., photocharge) generated in the first photodiode (PD1) can be transferred to the floating diffusion node (FD1) and stored in the floating diffusion node (FD). The second transfer transistor (TX2) is turned on in response to a second transfer control signal (TS2) applied to its gate terminal, so that the charge generated in the first photodiode (PD12) can be transferred to the floating diffusion node (FD). The first transmission control signal (TS1) and the second transmission control signal (TS2) are separate signals, and thus the turn-on times of the first transmission transistor (TX) and the second transmission transistor (TX2) can be independently controlled by each of the first transmission control signal (TS1) and the second transmission control signal (TS2).

[0067] The reset transistor (RX) can periodically reset the charges accumulated in the floating diffusion node (FD). The source electrode of the reset transistor (RX) is connected to the floating diffusion node (FD), and the drain electrode can be connected to the power supply voltage (VPIX). When the reset transistor (RX) is turned on in response to a reset control signal (RS) applied to its gate terminal, the power supply voltage (VPIX) connected to the drain electrode of the reset transistor (RX) is transferred to the floating diffusion node (FD). When the reset transistor (RX) is turned on, the charges accumulated in the floating diffusion node (FD) are discharged, and the floating diffusion node (FD) can be reset.

[0068] The driving transistor (DX) can operate as a source follower. The driving transistor (DX) can receive a signal corresponding to the charge amount of the floating diffusion node (FD), that is, the potential of the floating diffusion node (FD), at its gate terminal, and can buffer the received signal and output it. The selection transistor (SX) can be turned on in response to a selection signal (SEL) applied to its gate terminal, and when the selection transistor (SX) is turned on, the buffered signal output from the driving transistor (DX) can be output as a pixel signal (VOUT) through the column line (CL).

[0069] FIG. 6 shows a timing diagram of an image sensor (100) for reading a pixel signal from a pixel of FIG. 5 according to an exemplary embodiment of the present disclosure.

[0070] During the readout period (or referred to as the horizontal period), multiple pixel signals (VOUT) can be read from multiple pixels (PXa) placed in one row of the pixel array (110 in FIG. 3). In other words, multiple pixel signals (VOUT) are output from multiple pixels (PXa) placed in one row, and the multiple pixel signals (VOUT) can be converted from analog to digital in a readout circuit, such as a comparison circuit (140 in FIG. 2) and a counter circuit (160 in FIG. 2), and can be generated as pixel values ​​that are digital values.

[0071] Referring to FIG. 6, the readout period can be divided into first to third sub-periods (SP1, SP2, SP3) according to a signal output from a pixel (PXa) and converted from analog to digital. In the first sub-period (SP1), a reset signal (RST) corresponding to a reset level (e.g., reset voltage) is output as a pixel signal (VOUT); in the second sub-period (SP2), a first image signal (IMG1) (e.g., first signal voltage) generated from the first photodiode (PD1) of the pixel (PXa) or a second image signal (IMG2) (e.g., second signal voltage) generated from the second photodiode (PD2) is output as a pixel signal (VOUT); and in the third sub-period (SP3), a summated image signal (SIMG) (e.g., summated signal voltage) generated from the first photodiode (PD1) and the second photodiode (PD2) of the pixel (PXa) can be output as a pixel signal (VOUT).

[0072] The reset level of a pixel (PXa) may differ for multiple pixels (PXa) and may also differ over time even for a single pixel (PXa). Accordingly, during the readout period, a reset signal (RST) may be read out first from a pixel (PXa), and then the reset signal (RST) may be subtracted (or added) from an image signal read out subsequently, such as a first image signal (IMG1) (or a second image signal (ING2)) and a summation image signal (SIMG). Accordingly, actual image signals may be read out, and signal deviation between image signals output from multiple pixels (PXa) may be reduced.

[0073] In this way, the reset signal (RST), the first image signal (IMG1) (or the second image signal IMG2), and the summed image signal (SIMG) generated at the pixel (PXa) during the readout period can be readout in sequence. This readout method may be referred to as the RSS readout method (reset-signal-signal).

[0074] Specifically, referring to FIGS. 5 and FIGS. 6 together, during the readout period, the select signal (SEL) may be at an active level, e.g., logic high, and in response to the select signal (SEL), the select transistor (SX) may be turned on so that the pixel (PXa) can be connected to the column line (CL). Here, the active level of the signal refers to a level at which the transistor to which the signal is applied can be turned on. In the present disclosure, logic high is assumed to be the active level and logic low is assumed to be the inactive level.

[0075] When the readout period begins, as the reset control signal (RS) transitions from logic low to logic high, the reset transistor (RX) is turned on, and the floating diffusion node (FD) can be reset. During the first sub-period (SP1), a reset signal (RST) corresponding to the reset level of the floating diffusion node (FD) can be output through the column line (CL) as a pixel signal (VOUT), and a comparator (141 in FIG. 2) connected to the column line (CL) can compare the ramp signal (RAMP) and the pixel signal (VOUT) and output the comparison result as a comparison signal. The ramp signal (RAMP) decreases in level with a predetermined slope, and when the level of the ramp signal (RAMP) becomes lower than the level of the pixel signal (VOUT), the level of the comparison signal can be transitioned. During the first sub-period (SP1), the comparator (141) can compare the ramp signal (RAMP) received as the pixel signal (VOUT) and output a comparison signal.

[0076] A counter (161 in FIG. 2) can count pixel signals (VOUT) based on a counting clock (CCLK). Specifically, the counter (161) can count ramp signals (RAMP) by counting from the point where the level of the ramp signal (RAMP) falls until the point where the level of the comparison signal transitions, that is, until the point where the level of the ramp signal (RAMP) becomes equal to the level of the pixel signal (VOUT). In an embodiment, the counter (161) receives a Gray code in which code values ​​increase sequentially as a counting clock (CCLK) and outputs the code value at the point where the level of the comparison signal transitions as a counting value.

[0077] A reset signal (RST) is counted during the first sub-period (SP1), and a counting value (hereinafter referred to as the reset value) corresponding to the level of the reset signal can be stored in the internal latch (or memory) or memory (162 in FIG. 1) of the counter (161). Thus, the reset signal (RST) can be read out during the first sub-period (SP1).

[0078] As the first transmission control signal (TG1) transitions to an active level during the second sub-period (SP2), the charge generated in the first photodiode (PD1) can be transmitted and stored in the floating diffusion node (FD). Based on the potential of the floating diffusion node (FD) according to the amount of charge transmitted from the first photodiode (PD1), the driving transistor (DX) can output a first image signal (IMG1), and the first image signal can be output through the column line (CL) as a pixel signal (VOUT).

[0079] A comparator (141) compares a first image signal (IMG1) and a ramp signal (RAMP), and a counter (161) can count the first image signal (IMG1) by counting the comparison signal output from the comparator (141) based on a counting clock (CCLK). By generating a counting value (hereinafter referred to as the first image signal value) corresponding to the level of the first image signal (IMG1), the first image signal can be read out.

[0080] As the first transmission control signal (TG1) and the second transmission control signal (TG2) transition to an active level during the third sub-period (SP3), the charge generated in the first photodiode (PD1) and the second photodiode (PD1) can be transmitted and stored in the floating diffusion node (FD). Based on the potential of the floating diffusion node (FD) according to the amount of transmitted charge, the driving transistor (DX) can output a summed image signal (SIMG), and the summed image signal (SIMG) can be output through the column line (CL) as a pixel signal (VOUT).

[0081] The comparator (141) compares the summed image signal (SIM) and the ramp signal (RAMP), and the counter (161) counts the summed image signal (SIMG) by counting the comparison signal output from the comparator (141) based on the counting clock (CCLK). The summed image signal (SIMG) can be read out by generating a counting value (hereinafter referred to as the summed image signal value) corresponding to the level of the summed image signal.

[0082] Meanwhile, the first image signal (IMG1) is generated from a single photodiode, e.g., the first photodiode (PD1), and the summed image signal (SIMG) is generated from two photodiodes, e.g., the first and second photodiodes (PD1, PD2), so the signal range (e.g., voltage range) of the summed image signal (SIMG) is wider than the signal range of the first image signal (IMG1). For example, the signal range of the summed image signal (SIMG) may be twice the signal range of the first image signal (IMG1). Accordingly, the range of code values ​​provided as a counting clock (CCLK) in the third sub-period (SP3) (in other words, the number of toggled clocks) may be wider than the range of code values ​​provided in the second sub-period (SP2). For example, the comparator (141) can count comparison signals based on N code values ​​(CDV) in the second sub-period (SP2) and count comparison signals based on 2N code values ​​(CDV) in the third sub-period (SP3).

[0084] FIG. 7 illustrates a method for outputting an image signal for generating autofocus data in a pixel array according to an exemplary embodiment of the present disclosure.

[0085] Referring to FIG. 7, a first image signal (e.g., an image signal resulting from a photocharge generated by a first photodiode PD1) is read from a pixel (PXa) placed on the left among a plurality of rows (ROW1 to ROW4), and a second image signal (e.g., an image signal resulting from a photocharge generated by a second photodiode PD2) is read from a pixel (PXa) placed on the right among a plurality of rows, and autofocus data can be generated based on the read first image signal and the second image signal, that is, the first image signal value and the second image signal value.

[0086] In an embodiment, a first image signal from a first subpixel (SPXa) is output in response to a first transmission control signal (TG11, TG41) at each of a plurality of pixels (PXa) placed in a first row (ROW1) and a fourth row (ROW4), and a second image signal from a second subpixel (SPXb) is output in response to a second transmission control signal (TG22, TG32) at each of a plurality of pixels (PXa) placed in a second row (ROW2) and a third row (ROW3).

[0087] In an embodiment, a first image signal from a first subpixel (SPXa) is output in response to a first transmission control signal (TG11, TG21) at each of a plurality of pixels (PXa) placed in a first row (ROW1) and a second row (ROW2), and a second image signal from a second subpixel (SPXb) is output in response to a second transmission control signal (TG32, TG42) at each of a plurality of pixels (PXa) placed in a third row (ROW3) and a fourth row (ROW4).

[0088] For example, a first image signal (e.g., a first red image signal) may be output from a first subpixel (SPX1) of a red pixel (R) placed in a first row (ROW1), and a second image signal (e.g., a second red image signal) may be output from a second subpixel (SPX2) of a red pixel (R) placed in a third row (ROW3). Based on the first red image signal and the second red image signal, a pair of phase detection signals corresponding to the red pixel (R) of the pixel group (PG) may be generated.

[0089] Additionally, a second image signal (e.g., a second blue image signal) may be output from a first subpixel (SPX1) of a blue pixel (B) placed in the second row (ROW2), and a first image signal (e.g., a first blue image signal) may be output from a second subpixel (SPX2) of a blue pixel (B) placed in the fourth row (ROW4). Based on the first blue image signal and the second blue image signal, a pair of phase detection signals corresponding to the blue pixel (B) of the pixel group (PG) may be generated.

[0090] In this way, auto-focusing data can be generated based on a plurality of first image signals and a plurality of second image signals output from a plurality of pixel groups (PG) of a pixel array (110a).

[0092] FIG. 8 shows a timing diagram of an image sensor for reading a pixel signal from a pixel array according to an exemplary embodiment of the present disclosure, and FIG. 9 shows photocharge movement in pixels according to the transmission control signals of FIG. 8. FIG. 8 shows a reset control signal (RS), transmission control signals (TG11, TG12, TG21, TG22), ramp signal (RAMP), pixel signal (VOUT), and counting clock signal (CCLK) during two readout periods (1H, 2H) for convenience of explanation. The method of reading an image signal from a pixel (PXa) described with reference to FIG. 6 may be applied to the present embodiment.

[0093] Referring to FIGS. 3, 5 and 8, during the first readout period (1H), a pixel (PXa) (hereinafter referred to as the first pixel) of one row of the pixel array (110a), for example, the first row (ROW1), can output a pixel signal (VOUT), and during the second readout period (2H), a pixel (PXa) of another row, for example, the second row (hereinafter referred to as the second pixel), can output a pixel signal (VOUT).

[0094] In the first sub-period (SP1) of the first readout period (1H) and the second readout period (2H), the reset transistors (RX) of the first pixel and the second pixel are turned on in response to a reset signal (RST), thereby resetting the floating diffusion node (FD) as shown in FIG. 9. The first pixel and the second pixel output a reset signal according to the reset floating diffusion node (FD), and the reset signals of the first pixel and the second pixel can be output and counted as pixel signals (VOUT), respectively.

[0095] In the second sub-period (SP2) of the first readout period (1H), the first transmission transistor (TX1) of the first pixel is turned on in response to the first transmission control signal (TG11), so that the charge (photocharge) generated in the first photodiode (PD1) can be transmitted and stored in the floating diffusion node (FD). The first pixel can output a first image signal (IMG1), such as a left image signal, corresponding to the charge stored in the floating diffusion node (FD). Accordingly, the first image signal (IMG1) can be output and counted from the first photodiode (PD1) (or the first sub-pixel (SPX1)) of the first pixel.

[0096] During the third sub-period (SP3) of the first readout period (1H), the first transfer transistor (TX1) and the second transfer transistor (TX2) of the first pixel are turned on in response to the first transfer control signal (TG11) and the second transfer control signal (TG12), and the charge generated in the second photodiode (PD2) (and the charge remaining in the first photodiode (PD1)) can be transferred and stored in the floating diffusion node (FD). The charge transferred from the first photodiode (PD1) during the second sub-period (SP2) and the charge transferred from the second photodiode (PD2) (and the first photodiode (PD1)) during the third sub-period (SP3) can be stored in the floating diffusion node (FD). The first pixel can output a summed image signal corresponding to the charge stored in the floating diffusion node (FD), such as the first summed image signal (SIMG1) of the first pixel. Accordingly, a first summed image signal (SIMG1) from the first pixel can be output and counted.

[0097] Meanwhile, during the second sub-period (SP2) of the second readout period (2H), the second transmission transistor (TX2) of the second pixel is turned on in response to the second transmission control signal (TG22), so that the charge (photocharge) generated in the second photodiode (PD2) can be transmitted and stored in the floating diffusion node (FD). The second pixel can output a second image signal (IMG2), such as a light image signal, corresponding to the charge stored in the floating diffusion node (FD). Accordingly, the second image signal (IMG2) can be output and counted from the second photodiode (PD2) (or the second sub-pixel (SPX2)) of the second pixel.

[0098] In the third sub-period (SP3) of the second readout period (2H), the first transmission transistor (TX1) and the second transmission transistor (TX2) of the second pixel are turned on in response to the first transmission control signal (TG21) and the second transmission control signal (TG22), so that the charge generated in the first photodiode (PD1) and the second photodiode (PD2) can be transmitted and stored in the floating diffusion node (FD). The second pixel can output a summed image signal corresponding to the charge stored in the floating diffusion node (FD), such as the first summed image signal (SIMG1) of the first pixel. Accordingly, the first summed image signal (SIMG1) from the first pixel can be output and counted.

[0099] In this way, during the second sub-period (SP2) of the first readout period (1H), a pixel (PXa) of the first row, for example, the first pixel, responds to the first transmission control signal (TG11) to output a first image signal (IMG1), for example, a left image signal, and during the second sub-period (SP2) of the second readout period (2H), a pixel (PXa) of the second row, for example, the second pixel, responds to the second transmission control signal (TG22) to output a second image signal (IMG2), for example, a right image signal.

[0100] In FIG. 8, it is described as an example that a pixel signal (VOUT) is read from the first row (ROW1) during the first readout period (1H) and a pixel signal (VOUT) is read from the second row (ROW2) during the second readout period (2H). However, this is not limited thereto, and for example, a pixel signal (VOUT) may be read from one of the first row (ROW1) and the fourth row (ROW4) of FIG. 3 during the first readout period (1H), and a pixel signal (VOUT) may be read from one of the second row (ROW2) and the third row (ROW3) during the second readout period (2H). Alternatively, during the first horizontal period (1H), a pixel signal (VOUT) may be read from one of the first row (ROW1) and the second row (ROW2) of FIG. 3, and during the second horizontal period (2H), a pixel signal (VOUT) may be read from one of the third row (ROW3) and the fourth row (ROW4).

[0101] As described above, according to an exemplary embodiment of the present disclosure, an image sensor (100) and a method of operation thereof, each of a plurality of pixels (PX) of a pixel array (110) includes at least two photoelectric conversion elements arranged side by side, such as a first photodiode (PD1) and a second photodiode (PD2), and a predetermined row of pixels outputs a first image signal generated from the first photoelectric conversion element and a predetermined row of pixels outputs a second image signal generated from the second photoelectric conversion element, and autofocus data for an autofocus function can be generated based on the first image signal value and the second image signal value, which are converted into digital values.

[0102] Meanwhile, when a first image signal, such as a left image signal, is output from a pixel (PX), and a summation image signal is output from a pixel (PX), and a second image signal value is calculated by subtracting the first image signal value corresponding to the first image signal from the summation image signal value corresponding to the summation image signal, the summation image signal may be counted based on a number of code values ​​greater than the number of code values ​​required to count the summation image signal when reading the summation image signal, in order to prevent signal distortion of the second image signal value. An increase in the number of code values ​​implies an increase in reading time, and therefore, the time required to convert the image signal output from the pixel into a digital value may increase.

[0103] However, according to the method of operation of the image sensor (100) according to an exemplary embodiment of the present disclosure, a first image signal and a second image signal are read from at least two adjacent pixels of the same color, and autofocus data for an autofocus function can be generated based on the read first image signal and the second image signal, that is, the first image signal value and the second image signal value. Accordingly, the time required to convert each of the first image signal and the second image signal into a digital value without signal distortion can be reduced, and thus the frame rate of the image sensor (100) can be increased. In addition, a plurality of pixels (PX) of the pixel array (110), such as all pixels (PX) provided in the pixel array (110), can be used to generate autofocus data, so that high-accuracy autofocus data is generated, and thus an imaging device (e.g., 1000 of FIG. 1) equipped with the image sensor (100) can perform high-speed autofocus.

[0105] FIGS. 10a and FIGS. 10b schematically show a block diagram and a timing diagram of a row decoder (120) according to an exemplary embodiment of the present disclosure.

[0106] Referring to FIG. 10a, the row decoder (120) may include a predecoder (121), a logic circuit (122), and a driver (123).

[0107] The predecoder (121) can generate control signals provided to the pixel array (110 in FIG. 2), such as a reset control signal, a transmission control signal, and a selection signal. The predecoder (121) can generate a first reference control signal (TGR1) and a second reference control signal (TGR2), and generate a first transmission control signal (TG1) and a second transmission control signal (TG2) based on the first reference control signal (TGR1) and the second reference control signal (TGR2).

[0108] The pre-decoder (121) may include a first multiplexer (121-1) and a second multiplexer (121-2), and the first multiplexer (121-1) receives a first reference control signal (TGR1) and a second reference control signal (TGR2), and can select one of the first reference control signal (TGR1) and the second reference control signal (TGR2) as a first transmission control signal (TG1) based on a swapping enable signal (SEN). The second multiplexer (121-2) can select one of the first reference control signal (TGR1) and the second reference control signal (TGR2) as a second transmission control signal (TG2) based on a swapping enable bar signal (SENB). At this time, the swapping enable bar signal (SENB) may be a signal with opposite phase to the swapping enable signal (SEN). Accordingly, in the first multiplexer (121-1) and the second multiplexer (121-2), one of the first reference control signal (TGR1) and the second reference control signal (TGR2) may be selected as the first transmission control signal (TG1), and the other may be selected as the second transmission control signal (TG2).

[0109] Referring to FIG. 10b, the first reference control signal (TGR1) and the second reference control signal (TGR2) may be toggled at least once during a readout period, e.g., the first readout period (1H) and the second readout period (2H), from a first level, e.g., logic low, to a second level, e.g., logic high. The first reference control signal (TGR1) may be toggled ahead of the second reference control signal (TGR2), e.g., the first reference control signal (TGR1) may be toggled during the second sub-period (SP2). The second reference control signal (TGR2) may be toggled during the third sub-period (SP3), and in an embodiment, the first reference control signal (TGR1) may also be toggled during the third sub-period (SP3).

[0110] During the first readout period (1H), the swapping enable signal (SEN) may be at a first level, e.g., logic low, and in response to the swapping enable signal (SEN) (or swapping enable bar signal (SENB)), the first multiplexer (121-1) may output a first reference control signal (TGR1) as a first transmission control signal (TG1), and the second multiplexer (121-2) may output a second reference control signal (TGR2) as a second transmission control signal (TG2).

[0111] During the second readout period (2H), the swapping enable signal (SEN) may be at a second level, e.g., logic high, and in response to the swapping enable signal (SEN) (or swapping enable bar signal (SENB)), the first multiplexer (121-1) may output a second reference control signal (TGR2) as a first transmission control signal (TG2), and the second multiplexer (121-2) may output a first reference control signal (TGR1) as a second transmission control signal (TG2).

[0112] The logic circuit (122) can generate a reset control signal, a transmission control signal, and a selection signal corresponding to each of a plurality of rows of the pixel array (110a in FIG. 3) based on control signals received from the predecoder (121), such as a reset control signal, a transmission control signal, and a selection signal. For example, the logic circuit (122) can generate a first transmission control signal (TG11) and a second transmission control signal (TG12) applied to a first row (ROW1) during a first horizontal period (1H) based on a first transmission control signal (TG1) and a second transmission control signal (TG2) received from the predecoder (121), and generate a first transmission control signal (TG21) and a second transmission control signal (TG22) applied to a second row (ROW2) during a second horizontal period (2H).

[0113] The driver (123) can level-shift control signals for each of the plurality of rows received from the logic circuit (122), such as a reset control signal, a transmission control signal, and a selection signal, based on a power supply voltage corresponding to an active level and an inactive level, and output the level-shifted control signals to the pixel array (110a).

[0115] FIG. 11a shows a pixel array according to an exemplary embodiment of the present disclosure, and FIG. 11b shows a front view of a pixel provided in the pixel array of FIG. 11a. The pixel array (110b) of FIG. 11a may be a modified example of the pixel array (110b) of FIG. 3. Accordingly, the explanation will focus on the differences.

[0116] Referring to FIGS. 11a and 11b, each of the plurality of pixels (PXb) may include a microlens (ML) and two subpixels, such as a first subpixel (SPX1) and a second subpixel (SPX2). The first subpixel (SPX1) and the second subpixel (SPX2) may be arranged side by side in a column direction, such as the Y-axis direction (second direction). For example, the first subpixel (SPX1) may be placed at the top within the pixel (PXb) and the second subpixel (SPX2) may be placed at the bottom within the pixel (PXa). The first subpixel (SPX1) and the second subpixel (SPX2) may each include a first photodiode (PD1) and a second photodiode (PD2).

[0117] A plurality of first subpixels (SPX1) of a plurality of pixels (PXa) may operate in response to a plurality of first transmission control signals (TG11, TG21, TG31, TG41), and a plurality of second subpixels (SPX2) may operate in response to a plurality of second transmission control signals (TG12, TG22, TG32, TG42).

[0118] As described with reference to FIGS. 3 to 9, pixel signals can be read from a pixel array (110b), and first image signals generated in a first subpixel (SPX1) can be output from pixels (PXb) placed in a predetermined row among the first to fourth rows (ROW1 to ROW4), and second image signals generated in a second subpixel (PSX2) can be output from pixels (PXb) placed in another predetermined row. Based on the first image signals (e.g., top image signals) and the second image signals (e.g., bottom image signals), a pair of phase detection signals for autofocus data, such as phase difference calculation for adjusting the vertical direction of focus, can be generated.

[0120] FIG. 12 shows a pixel array according to an exemplary embodiment of the present disclosure. The pixel array (110c) of FIG. 12 may be a variation of the pixel array (110b) of FIG. 3 and the pixel array (110c) of FIG. 12a. Accordingly, the explanation will focus on the differences.

[0121] Referring to FIG. 12, the pixel array (110c) may include a plurality of first type pixels (PXa) in which a first subpixel (SPX1) and a second subpixel (SPX2) are arranged side by side in a row direction, e.g., the X-axis direction (first direction), and a plurality of second type pixels (PXb) in which the first subpixel (SPX1) and the second subpixel (SPX2) are arranged side by side in a column direction, e.g., the Y-axis direction (second direction). The first type pixels (PXa) and the second type pixels (PXb) may be arranged in the same row, and the first type pixels (PXa) and the second type pixels (PXb) arranged in the same row, e.g., the first row (ROW1), may each receive a first transmission control signal (TG11) and a second transmission control signal (TG12) through the same row lines (RL).

[0122] The pixel array (110c) may be divided into a plurality of first pixel groups (PG1) and a plurality of second pixel groups (PG2), the first pixel group (PG1) may include first type pixels (PXa) arranged in a 2 x 4 matrix, and the second pixel group (PG2) may include second type pixels (PXb) arranged in a 2 x 4 matrix.

[0123] Pixel signals can be read from the pixel array (110b), and first image signals generated in the first subpixel (SPX1) can be output from the first type of pixels (PXa) placed in a predetermined row among the first to fourth rows (ROW1~ROW4), and third image signals generated in the first subpixel (SPX1) can be output from the second type of pixels (PXb). Second image signals generated in the second subpixel (PSX2) can be output from the first type of pixels (PXa) placed in another predetermined row, and fourth image signals generated in the second subpixel (SPX2) can be output from the second type of pixels (PXb). The first image signal and the second image signal correspond to the left image signal and the right image signal, respectively, and the third image signal and the fourth image signal correspond to the upper image signal and the lower image signal, respectively.

[0124] A pair of phase detection signals for phase difference calculation for left-right direction adjustment of focus can be generated based on first image signals and second image signals generated from first type pixels (PXa) included in the first pixel group (PG1). Additionally, a pair of phase detection signals for phase difference calculation for left-right direction adjustment of focus can be generated based on third image signals and fourth image signals generated from second type pixels (PXb) included in the second pixel group (PG2).

[0125] Meanwhile, in FIG. 12, the pixel array (110c) is illustrated as including a first type of pixel (PXa) in which a first subpixel (SPX1) and a second subpixel (SPX2) are arranged side by side in the row direction, and a second type of pixel (PXb) in which a first subpixel (SPX1) and a second subpixel (SPX2) are arranged side by side in the column direction, but is not limited thereto. In an embodiment, the pixel array (110c) may include a third type of pixel and a fourth type of pixel in which a first subpixel (SPX1) and a second subpixel (SPX2) are arranged in the upper left direction (or lower right direction) and the upper right direction (or lower left direction), respectively.

[0127] FIG. 13a shows a pixel array according to an exemplary embodiment of the present disclosure, and FIG. 13b shows a front view of a pixel provided in the pixel array of FIG. 13a.

[0128] Referring to FIGS. 13a and 13b, each of the plurality of pixels (PXc) may include a microlens (ML) and four subpixels, such as a first subpixel (SPX1), a second subpixel (SPX2), a third subpixel (SPX3), and a fourth subpixel (SPX4). The first subpixel (SPX1) may be positioned at the top-left within the pixel (PXc), the second subpixel (SPX2) may be positioned at the top-right within the pixel (PXa), the third subpixel (SPX3) may be positioned at the bottom-left within the pixel (PXc), and the fourth subpixel (SPX4) may be positioned at the bottom-right within the pixel (PXc). In other words, the first subpixel (SPX1) and the second subpixel (SPX2), and the third subpixel (SPX3) and the fourth subpixel (SPX4) may be arranged side by side in the row direction, e.g., the X-axis direction (first direction), and the first subpixel (SPX1) and the third subpixel (SPX3), and the second subpixel (SPX2) and the fourth subpixel (SPX4) may be arranged side by side in the column direction, e.g., the Y-axis direction (second direction).

[0129] The first subpixel (SPX1), the second subpixel (SPX2), the third subpixel (SPX3), and the fourth subpixel (SPX4) may each include a photoelectric conversion element, such as a first photodiode (PD1), a second photodiode (PD2), a third photodiode (PD3), and a fourth photodiode (PD4).

[0130] A plurality of first subpixels (SPX1) of a plurality of pixels (PXc) may operate in response to a plurality of first transmission control signals (TG11, TG21, TG31, TG41), a plurality of second subpixels (SPX2) may operate in response to a plurality of second transmission control signals (TG12, TG22, TG32, TG42), a plurality of third subpixels (SPX1) may operate in response to a plurality of third transmission control signals (TG13, TG23, TG33, TG43), and a plurality of fourth subpixels (SPX4) may operate in response to a plurality of fourth transmission control signals (TG14, TG24, TG34, TG44).

[0131] A plurality of first transmission control signals (TG11, TG21, TG31, TG41), a plurality of second transmission control signals (TG12, TG22, TG32, TG42), a plurality of third transmission control signals (TG13, TG23, TG33, TG43), and a plurality of fourth transmission control signals (TG14, TG24, TG34, TG44) may be distinct signals provided through different row lines (RL) from a row decoder (120 in FIG. 2). The first to fourth subpixels (SPX1 to SPX4) of pixels (PXc) placed in the same row may receive the first to fourth transmission control signals through the same row lines (RL).

[0132] The pixel array (110c) may be divided into multiple pixel groups (PG), and each of the multiple pixel groups (PG) may include, for example, pixels (PXc) arranged in a 2 x 4 matrix.

[0133] Figure 14 is an equivalent circuit diagram of the pixel in Figure 13a.

[0134] Referring to FIG. 14, the pixel (PXc) may include a first photodiode (PD1), a second photodiode (PD2), a third photodiode (PD2), a fourth photodiode (PD4), a first transmission transistor (TX1), a second transmission transistor (TX2), a third transmission transistor (TX3), a fourth transmission transistor (TX4), a reset transistor (RX), a drive transistor (DX), and a select transistor (SX).

[0135] A floating diffusion node (FD) (or a floating diffusion region) can be shared by a first photodiode (PD1), a second photodiode (PD2), a third photodiode (PD2), a fourth photodiode (PD4), a first transfer transistor (TX1), a second transfer transistor (TX2), a third transfer transistor (TX3), and a fourth transfer transistor (TX4).

[0136] The first transmission transistor (TX1), the second transmission transistor (TX2), the third transmission transistor (TX3), and the fourth transmission transistor (TX4) can be independently turned on or turned off in response to a transmission control signal applied to each of the gate terminals of the first transmission control signal (TS1), the second transmission control signal (TS2), the third transmission control signal (TS3), and the fourth transmission control signal (TS4). When the first transmission transistor (TX1), the second transmission transistor (TX2), the third transmission transistor (TX3), or the fourth transmission transistor (TX4) is turned on, the photocharge generated in the corresponding photodiode can be transmitted and stored in the floating diffusion node (FD).

[0137] The control signals applied to the reset transistor (RX), the driving transistor (DX), and the select transistor (SX), and their operation are as described with reference to FIG. 5.

[0138] FIG. 15 shows a timing diagram of transmission control signals applied to the pixel array of FIG. 13a according to an example of the present disclosure.

[0139] Referring to FIGS. 13a, 14 and 15, during the first readout period (1H), a pixel (PXc) (hereinafter referred to as the first pixel) of one row of the pixel array (110c), for example, the first row (ROW1), can output a pixel signal (VOUT), and during the second readout period (2H), a pixel (PXa) (hereinafter referred to as the second pixel) of another row, for example, the second row, can output a pixel signal (VOUT).

[0140] In the first sub-period (SP1) of the first readout period (1H) and the second readout period (2H), the first pixel and the second pixel can output a reset signal.

[0141] A first transmission control signal (TG11) and a third transmission control signal (TG13) can be toggled during the second sub-period (SP2) of the first readout period (1H). By turning on the first transmission transistor (TX1) and the third transmission transistor (TX3) of the first pixel in response to the first transmission control signal (TG11) and the third transmission control signal (TG13), the charge (photocharge) generated in the first photodiode (PD1) and the third photodiode (PD3) can be transmitted and stored in the floating diffusion node (FD). The first pixel can output a first image signal, such as a left image signal, corresponding to the charge stored in the floating diffusion node (FD). Accordingly, the first pixel can output a first image signal generated from the first photodiode (PD1) and the third photodiode (PD3) (or the first subpixel (SPX1) and the third subpixel (SPX3)).

[0142] The first to fourth transmission control signals (TG11~TG14) may be toggled during the third sub-period (SP3) of the first readout period (1H). The first to fourth transmission transistors (TX1~TX4) of the first pixel are turned on in response to the first to fourth transmission control signals (TG11~TG14), and charges generated in the first to fourth photodiodes (PD1~PD4) may be transmitted and stored in the floating diffusion node (FD). The first pixel may output a summed image signal corresponding to the charges stored in the floating diffusion node (FD), such as the first summed image signal of the first pixel. Accordingly, the first summed image signal may be output and counted from the first pixel.

[0143] Meanwhile, during the second sub-period (SP2) of the second readout period (2H), the second transfer transistor (TX2) and the fourth transfer transistor (TX4) of the second pixel are turned on in response to the second transfer control signal (TG22) and the fourth transfer control signal (TG24), so that the charge generated in the second photodiode (PD2) and the fourth photodiode (PD4) can be transferred and stored in the floating diffusion node (FD). The second pixel can output a second image signal, such as a light image signal, corresponding to the charge stored in the floating diffusion node (FD). Accordingly, the second pixel can output a second image signal generated from the second photodiode (PD2) and the fourth photodiode (PD4) (or the first sub-pixel (SPX2) and the fourth sub-pixel (SPX4)).

[0144] The first to fourth transmission control signals (TG21 to TG24) may be toggled during the third sub-period (SP3) of the second readout period (2H). The first to fourth transmission transistors (TX1 to TX4) of the first pixel are turned on in response to the first to fourth transmission control signals (TG21 to TG24), and charges generated in the first to fourth photodiodes (PD1 to PD4) may be transmitted and stored in the floating diffusion node (FD). The second pixel may output a summed image signal corresponding to the charges stored in the floating diffusion node (FD), for example, a second summed image signal of the second pixel. Accordingly, the second summed image signal may be output and counted from the second pixel.

[0145] In this way, during the second sub-period (SP2) of the first readout period (1H), a pixel (PXc) of the first row, such as the first pixel, responds to the first and third transmission control signals (TG11, TG13) to output a first image signal, such as a left image signal, and during the second sub-period (SP2) of the second readout period (2H), a pixel (PXc) of the second row, such as the second pixel, responds to the second and fourth transmission control signals (TG22, TG24) to output a second image signal, such as a right image signal.

[0146] In another embodiment, the first transmission control signal (TG11, TG21) and the fourth transmission control signal (TG14, TG24) are identical, and the second transmission control signal (TG12, TG22) and the third transmission control signal (TG13, TG23) may be identical. Accordingly, in the second sub-period (SP2) of the first readout period (1H), the pixel (PXc) of the first row, for example, the first pixel, responds to the first and fourth transmission control signals (TG11, TG14) to output a third image signal, for example, an upper image signal, generated from the first photodiode (PD1) and the fourth photodiode (PD4), and in the second sub-period (SP2) of the second readout period (2H), the pixel (PXc) of the second row (ROW2), for example, the second pixel, responds to the second and third transmission control signals (TG22, TG23) to output a fourth image signal, for example, a lower image signal, generated from the twelfth photodiode (PD2) and the third photodiode (PD3).

[0148] FIG. 16 is a flowchart illustrating a method of operation of an image sensor according to an exemplary embodiment of the present disclosure. The method of FIG. 16 can be performed on the image sensor (100) of FIG. 2.

[0149] Referring to FIG. 16, each of the first pixels of the first row of the pixel array (110) outputs a reset signal, a first image signal, and a summation signal in sequence according to the RSS reading method according to steps S110, S120, and S130, and subsequently, each of the second pixels of the second row of the pixel array (110) outputs a reset signal, a second image signal, and a summation signal in sequence according to the RSS reading method according to steps S140, S150, and S160. The first pixel and the second pixel may be pixels of the same color placed in the same column.

[0150] The first pixel of the first row can output a first reset signal (S110). The first pixel can output a first reset signal in response to transmission control signals from the row driver (120).

[0151] A first pixel of a first row outputs a first image signal from a first photoelectric conversion element (S120), and then, a first pixel of a first row can output a first summed image signal from a first photoelectric conversion element and a second photoelectric conversion element (S130).

[0152] Subsequently, the second pixel of the second row outputs a second reset signal, and the second pixel of the second row outputs a second image signal from the second photoelectric conversion element (S150), and subsequently, the second pixel of the second row can output a second summed image signal from the first photoelectric conversion element and the second photoelectric conversion element (S160).

[0153] When steps S110 to S160 are performed, signals output from the first pixel and the second pixel, such as a first reset signal, a first image signal, a first summed image signal, a second reset signal, a second image signal, and a second summed image signal, can be converted into digital values, such as counting values, in the comparison circuit (140) and the counter circuit (160). Based on the first reset signal value and the second reset signal value, a first image signal value, a second image signal value, a first summed image signal value, and a second summed image signal value with the reset level removed can be generated.

[0154] The signal processor (190) can generate autofocus data based on the first image signal and the second image signal (S170). The signal processor (190) can generate a phase detection signal pair based on the first image signal value and the second image signal value from which the reset level has been removed. A plurality of phase detection signal pairs can be generated based on a plurality of first image signals and a plurality of second image signals output from the pixel array (110).

[0156] FIGS. 17 and 18 are block diagrams of an electronic device including a multi-camera module. FIG. 19 is a detailed block diagram of the camera module of FIG. 17.

[0157] Referring to FIG. 17, the electronic device (1000) may include a camera module group (2100), an application processor (2200), a PMIC (2300), and an external memory (2400).

[0158] The camera module group (2100) may include a plurality of camera modules (2100a, 2100b, 2100c). Although an embodiment in which three camera modules (2100a, 2100b, 2100c) are arranged is illustrated in the drawings, the embodiments are not limited thereto. In some embodiments, the camera module group (2100) may be modified to include two cameras or n camera modules (where n is a natural number greater than or equal to 4).

[0159] Hereinafter, with reference to FIG. 19, the detailed configuration of the camera module (2100b) will be described in more detail, but the following description may be applied equally to other camera modules (2100a, 2100b) according to the embodiment.

[0160] Referring to FIG. 19, the camera module (2100b) may include a prism (2105), an optical path folding element (OPFE, hereinafter referred to as "OPFE") (2110), an actuator (2130), an image sensing device (2140), and a storage unit (2150).

[0161] The prism (2105) can modify the path of light (L) incident from the outside by including a reflective surface (2107) of a light-reflecting material.

[0162] In some embodiments, the prism (2105) can change the path of light (L) incident in a first direction (X) to a second direction (Y) perpendicular to the first direction (X). Additionally, the prism (2105) can change the path of light (L) incident in the first direction (X) to a second direction (Y) perpendicular to the first direction (X) by rotating the reflective surface (2107) of the light-reflecting material in direction A around the central axis (2106) or by rotating the central axis (2106) in direction B. At this time, the OPFE (2110) can also move to a third direction (Z) perpendicular to the first direction (X) and the second direction (Y).

[0163] In some embodiments, as illustrated, the maximum rotation angle in the A direction of the prism (2105) may be 15 degrees or less in the plus (+) A direction and greater than 15 degrees in the minus (-) A direction, but the embodiments are not limited thereto.

[0164] In some embodiments, the prism (2105) can move in the plus (+) or minus (-) B direction by about 20 degrees, or between 10 and 20 degrees, or between 15 and 20 degrees, where the angle of movement can be moved by the same angle in the plus (+) or minus (-) B direction, or by a nearly similar angle within a range of about 1 degree.

[0165] In some embodiments, the prism (2105) can move the reflective surface (2106) of the light-reflecting material in a third direction (e.g., Z direction) parallel to the extension direction of the central axis (2106).

[0166] In some embodiments, the camera module (2100b) may be composed of two or more prisms, thereby allowing the path of light (L) incident in a first direction (X) to be varied to a second direction (Y) perpendicular to the first direction (X), then to the first direction (X) or a third direction (Z), and then to the second direction (Y).

[0167] OPFE (2110) may include, for example, groups of m (where m is a natural number) optical lenses. The m lenses can be moved in a second direction (Y) to change the optical zoom ratio of the camera module (2100b). For example, when the basic optical zoom ratio of the camera module (2100b) is Z, moving the m optical lenses included in the OPFE (2110) may change the optical zoom ratio of the camera module (2100b) to 3Z or 5Z or an optical zoom ratio of 5Z or more.

[0168] The actuator (2130) can move the OPFE (2110) or the optical lens (hereinafter referred to as the optical lens) to a specific position. For example, the actuator (2130) can adjust the position of the optical lens so that the image sensor (2142) is positioned at the focal length of the optical lens for accurate sensing.

[0169] The image sensing device (2140) may include an image sensor (2142), control logic (2144), and memory (2146). The image sensor (2142) may sense an image of a sensing target using light (L) provided through an optical lens. The image sensor (100) described with reference to FIGS. 2 to 16 may be applied as the image sensor (2142). It includes at least one first photoelectric conversion element and at least one second photoelectric conversion element arranged side by side below the micro-lens of a plurality of pixels (PX) of a pixel array (110), and a first image signal from the first photoelectric conversion element, such as a left image signal (or an upper image signal), may be output from pixels (PX) arranged in a predetermined row among the plurality of pixels (PX), and a second image signal from the second photoelectric conversion element, such as a right image signal (or a lower image signal), may be output from pixels (PX) arranged in another predetermined row. Auto-focusing data for an AF function, such as a pair of phase detection signals, can be generated based on a first image signal and a second image signal output from at least two pixels (PX) placed in adjacent rows and the same column.

[0170] The control logic (2144) can control the overall operation of the camera module (2100b) and process the sensed image. For example, the control logic (2144) can control the operation of the camera module (2100b) according to a control signal provided through the control signal line (CSLb). For example, the control logic (2144) can control the actuator (2130) so that the optical lens is positioned at the focal distance according to AF by a focus adjustment signal received from the application processor (2200). The control logic (2144) can extract image data corresponding to a specific image (e.g., a person's face, arm, leg, etc. within the image) from the sensed image.

[0171] In some embodiments, the control logic (2144) can perform image processing such as encoding and noise reduction of the sensed image.

[0172] The memory (2146) can store information necessary for the operation of the camera module (2100b), such as calibration data (2147). Calibration data (2147) is information necessary for the camera module (2100b) to generate image data using light (L) provided from the outside, and may include, for example, information regarding the degree of rotation, information regarding the focal length, information regarding the optical axis, etc. If the camera module (2100b) is implemented in the form of a multi-state camera in which the focal length changes according to the position of the optical lens, the calibration data (2147) may include focal length values ​​for each position (or state) of the optical lens and information related to auto-focusing. In some embodiments, the image sensor (2142) is configured as a first chip, and the control logic (2144), storage unit (2150), and memory (2146) are configured as a second chip, so that the two chips can be implemented in a stacked form.

[0173] The storage unit (2150) can store image data sensed through the image sensor (2142). The storage unit (2150) may be placed outside the image sensing device (2140) and may be implemented in a stacked form with the sensor chip constituting the image sensing device (2140). In some embodiments, the storage unit (2150) may be implemented as an EEPROM (Electrically Erasable Programmable Read-Only Memory), but the embodiments are not limited thereto. In some embodiments, the image sensor (2142) is composed of a pixel array, and the control logic (2144) may include an analog-to-digital converter and an image signal processing unit for processing the sensed image.

[0174] Referring to FIG. 17 and FIG. 19 together, in some embodiments, each of the plurality of camera modules (2100a, 2100b, 2100c) may include an actuator (2130). Accordingly, each of the plurality of camera modules (2100a, 2100b, 2100c) may include identical or different calibration data (2147) according to the operation of the actuator (2130) included therein.

[0175] In some embodiments, one of the plurality of camera modules (2100a, 2100b, 2100c) camera module (e.g., 2100b) is a camera module in the form of a folded lens that includes the previously described prism (2105) and OPFE (2110), and the remaining camera modules (e.g., 2100a, 2100c) may be camera modules in the form of a vertical camera module that do not include the prism (2105) and OPFE (2110), but are not limited thereto.

[0176] In some embodiments, one of the plurality of camera modules (2100a, 2100b, 2100c) (e.g., 2100c) may be a vertical depth camera that extracts depth information using, for example, IR (Infrared Ray). In this case, the application processor (2200) may generate a 3D depth image by merging image data provided from this depth camera with image data provided from another camera module (e.g., 2100a or 2100b).

[0177] In some embodiments, at least two of the plurality of camera modules (2100a, 2100b, 2100c) may have different field of view angles. In this case, for example, the optical lenses of at least two of the plurality of camera modules (2100a, 2100b, 2100c) may be different from each other, but are not limited thereto.

[0178] Additionally, in some embodiments, the field of view of each of the plurality of camera modules (2100a, 2100b, 2100c) may be different from one another. For example, camera module (1100a) may be an ultrawide camera, camera module (1100b) may be a wide camera, and camera module (1100c) may be a tele camera, but is not limited thereto. In this case, the optical lenses included in each of the plurality of camera modules (2100a, 2100b, 2100c) may also be different from one another, but are not limited thereto.

[0179] In some embodiments, each of the plurality of camera modules (2100a, 2100b, 2100c) may be physically separated from one another. That is, instead of the plurality of camera modules (2100a, 2100b, 2100c) dividing and using the sensing area of ​​a single image sensor (1142), an independent image sensor (2142) may be placed inside each of the plurality of camera modules (2100a, 2100b, 2100c).

[0180] Referring again to FIG. 17, the application processor (2200) may include an image processing device (1210), a memory controller (2220), and an internal memory (2230). The application processor (2200) may be implemented separately from a plurality of camera modules (2100a, 2100b, 2100c). For example, the application processor (2200) and the plurality of camera modules (2100a, 21100b, 2100c) may be implemented separately from each other as separate semiconductor chips.

[0181] The image processing device (1210) may include a plurality of sub-image processors (2212a, 2212b, 2212c), an image generator (2214), and a camera module controller (2216).

[0182] The image processing device (2210) may include a plurality of sub-image processors (2212a, 2212b, 2212c) corresponding to the number of camera modules (2100a, 2100b, 2100c).

[0183] Image data generated from the camera module (2100a) may be provided to the sub-image processor (2212a) via the image signal line (ISLa), image data generated from the camera module (2100b) may be provided to the sub-image processor (2212b) via the image signal line (ISLb), and image data generated from the camera module (2100c) may be provided to the sub-image processor (2212c) via the image signal line (ISLc). Such image data transmission may be performed, for example, using a Camera Serial Interface (CSI) based on the Mobile Industry Processor Interface (MIPI), but is not limited thereto.

[0184] Meanwhile, in some embodiments, a single sub-image processor may be arranged to correspond to a plurality of camera modules. For example, the sub-image processor (2212a) and the sub-image processor (2212c) are not implemented separately as illustrated, but are integrated into a single sub-image processor, and image data provided from the camera module (2100a) and the camera module (2100c) may be selected through a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor. At this time, the sub-image processor (2212b) is not integrated and may receive image data from the camera module (2100b).

[0185] Additionally, in some embodiments, image data generated from a camera module (2100a) may be provided to a sub-image processor (2212a) via an image signal line (ISLa), image data generated from a camera module (1100b) may be provided to a sub-image processor (2212b) via an image signal line (ISLb), and image data generated from a camera module (2100c) may be provided to a sub-image processor (2212c) via an image signal line (ISLc). Furthermore, image data processed by the sub-image processor (2212b) may be provided directly to an image generator (2214), but image data processed by the sub-image processor (2212a) and image data processed by the sub-image processor (2212c) may be provided to the image generator (2214) after one of them is selected through a selection element (e.g., a multiplexer).

[0186] Each sub-image processor (2212a, 2212b, 2212c) can perform image processing such as bad pixel correction, 3A adjustment (Auto-focus correction, Auto-white balance, Auto-exposure), noise reduction, sharpening, gamma control, and remosaic on image data provided from the camera module (2100a, 2100b, 2100c).

[0187] In some embodiments, remosaic signal processing may be performed in each camera module (2100a, 2100b, 2100c) and then provided to a sub-image processor (2212a, 2212b, 2212c).

[0188] Image data processed by each sub-image processor (2212a, 2212b, 2212c) can be provided to an image generator (2214). The image generator (2214) can generate an output image using image data provided from the image processors (2212a, 2212b, 2212c) according to image generating information or a mode signal.

[0189] Specifically, the image generator (2214) can generate an output image by merging at least some of the image data generated from camera modules (2100a, 2100b, 2100c) having different viewing angles according to image generation information or a mode signal. Additionally, the image generator (2214) can generate an output image by selecting any one of the image data generated from camera modules (2100a, 2100b, 2100c) having different viewing angles according to image generation information or a mode signal.

[0190] In some embodiments, the image generation information may include a zoom signal (or zoom factor). Additionally, in some embodiments, the mode signal may be a signal based, for example, on a mode selected by a user.

[0191] When the image generation information is a zoom signal (zoom factor) and each camera module (2100a, 2100b, 2100c) has a different viewing angle (angle of view), the image generator (2214) can perform different operations depending on the type of zoom signal. For example, when the zoom signal is a first signal, among the image data output from the sub-image processor (2212a) and the image data output from the sub-image processor (2212c), the image data output from the sub-image processor (2212a) and the image data output from the sub-image processor (2212b) can be used to generate an output image. If the zoom signal is a second signal different from the first signal, the image generator (2214) can generate an output image by using the image data output from the sub-image processor (2212c) and the image data output from the sub-image processor (2212b), among the image data output from the sub-image processor (2212a) and the image data output from the sub-image processor (2212c). If the zoom signal is a third signal different from the first and second signals, the image generator (2214) can generate an output image by selecting any one of the image data output from each sub-image processor (2212a, 2212b, 2212c) without performing such image data merging. However, the embodiments are not limited thereto, and the method of processing image data can be modified as needed.

[0192] Referring to FIG. 18, in some embodiments, the image processing device (2210) may further include a selection unit (2213) that selects the output of sub-image processors (2212a, 2212b, 2212c) and transmits it to an image generator (2214).

[0193] In this case, the selection unit (2213) can perform different operations depending on the zoom signal or zoom factor. For example, when the zoom signal is the fourth signal (e.g., the zoom magnification is the first magnification), the selection unit (2213) can select one of the outputs of the sub-image processors (2212a, 2212b, 2212c) and transmit it to the image generator (2214).

[0194] Additionally, the selection unit (2213) can sequentially transmit p outputs (p is a natural number greater than or equal to 2) among the outputs of sub-image processors (2212a, 2212b, 2212c) to the image generator (2214) when the zoom signal is a fifth signal different from the fourth signal (e.g., the zoom magnification is the second magnification). For example, the selection unit (2213) can sequentially transmit the outputs of sub-image processor (2212b) and sub-image processor (2212c) to the image generator (2214). Additionally, the selection unit (2213) can sequentially transmit the outputs of sub-image processor (2212a) and sub-image processor (2212b) to the image generator (2214). The image generator (2214) can generate a single output image by merging the p outputs received sequentially.

[0195] Here, image processing such as demosaicing, downscaling to a video / preview resolution size, gamma correction, and HDR (High Dynamic Range) processing is performed in advance in sub-image processors (2212a, 2212b, 2212c), and then the processed image data is transmitted to the image generator (2214). Therefore, even if the processed image data is provided to the image generator (2214) as a single signal line through the selection unit (2213), the image merging operation of the image generator (2214) can be performed at high speed.

[0196] In some embodiments, the image generator (2214) receives multiple image data with different exposure times from at least one of a plurality of sub-image processors (2212a, 2212b, 2212c) and performs HDR (high dynamic range) processing on the multiple image data to generate merged image data with increased dynamic range.

[0197] The camera module controller (2216) can provide control signals to each camera module (2100a, 2100b, 2100c). The control signals generated from the camera module controller (2216) can be provided to the corresponding camera modules (2100a, 2100b, 2100c) through separate control signal lines (CSLa, CSLb, CSLc).

[0198] One of the plurality of camera modules (2100a, 2100b, 2100c) may be designated as a master camera (e.g., 1100b) according to image generation information including a zoom signal or a mode signal, and the remaining camera modules (e.g., 2100a, 2100c) may be designated as slave cameras. This information may be included in a control signal and provided to the corresponding camera modules (2100a, 2100b, 2100c) through separate control signal lines (CSLa, CSLb, CSLc).

[0199] The camera module operating as a master and slave may be changed according to the zoom factor or operation mode signal. For example, if the field of view of the camera module (2100a) is wider than the field of view of the camera module (2100b) and the zoom factor indicates a low zoom magnification, the camera module (2100a) may operate as a master and the camera module (2100b) may operate as a slave. Conversely, if the zoom factor indicates a high zoom magnification, the camera module (2100b) may operate as a master and the camera module (2100a) may operate as a slave.

[0200] In some embodiments, the control signal provided from the camera module controller (2216) to each camera module (2100a, 2100b, 2100c) may include a sync enable signal. For example, if the camera module (2100b) is a master camera and the camera modules (2100a, 2100c) are slave cameras, the camera module controller (2216) may transmit a sync enable signal to the camera module (2100b). The camera module (2100b) that receives this sync enable signal may generate a sync signal based on the received sync enable signal and provide the generated sync signal to the camera modules (2100a, 2100c) through a sync signal line (SSL). The camera module (2100b) and camera modules (2100a, 2100c) can be synchronized with this sync signal to transmit image data to the application processor (2200).

[0201] In some embodiments, a control signal provided from a camera module controller (2216) to a plurality of camera modules (2100a, 2100b, 2100c) may include mode information according to a mode signal. Based on this mode information, the plurality of camera modules (2100a, 2100b, 2100c) may operate in a first operation mode and a second operation mode with respect to the sensing speed.

[0202] A plurality of camera modules (2100a, 2100b, 2100c) can, in a first operating mode, generate an image signal at a first speed (e.g., generate an image signal at a first frame rate) and encode it at a second speed higher than the first speed (e.g., encode an image signal at a second frame rate higher than the first frame rate), and transmit the encoded image signal to an application processor (2200). At this time, the second speed may be 30 times or less of the first speed.

[0203] The application processor (2200) stores the received image signal, that is, the encoded image signal, in a memory (2230) provided internally or in a storage (2400) outside the application processor (2200), and subsequently reads the encoded image signal from the memory (2230) or the storage (2400) to decode it, and can display image data generated based on the decoded image signal. For example, a corresponding sub-image processor among a plurality of sub-image processors (2212a, 2212b, 2212c) of the image processing device (2210) can perform decoding and can also perform image processing on the decoded image signal.

[0204] A plurality of camera modules (2100a, 2100b, 2100c) can generate an image signal at a third speed lower than a first speed in a second operation mode (e.g., generate an image signal at a third frame rate lower than a first frame rate) and transmit the image signal to an application processor (2200). The image signal provided to the application processor (2200) may be an unencoded signal. The application processor (2200) may perform image processing on the received image signal or store the image signal in memory (2230) or storage (2400).

[0205] The PMIC (2300) can supply power, such as power voltage, to each of the plurality of camera modules (2100a, 2100b, 2100c). For example, the PMIC (2300) can supply first power to the camera module (2100a) through a power signal line (PSLa), supply second power to the camera module (2100b) through a power signal line (PSLb), and supply third power to the camera module (2100c) through a power signal line (PSLc), under the control of the application processor (2200).

[0206] The PMIC (2300) can generate power corresponding to each of the plurality of camera modules (2100a, 2100b, 2100c) and adjust the power level in response to a power control signal (PCON) from the application processor (2200). The power control signal (PCON) may include power adjustment signals for each operating mode of the plurality of camera modules (2100a, 2100b, 2100c). For example, the operating mode may include a low power mode, and in this case, the power control signal (PCON) may include information about the camera module operating in the low power mode and the power level being set. The power levels provided to each of the plurality of camera modules (2100a, 2100b, 2100c) may be the same or different from each other. Additionally, the power level may be changed dynamically.

[0207] As described above, exemplary embodiments have been disclosed in the drawings and specification. Although specific terms have been used to describe the embodiments in this specification, they are used only for the purpose of explaining the technical concept of this disclosure and are not intended to limit the meaning or the scope of this disclosure as defined in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this disclosure should be determined by the technical concept of the appended claims.

Claims

Claim 1 A pixel array comprising a plurality of pixels arranged in a matrix, wherein each of the plurality of pixels includes a microlens and a first photoelectric conversion element and a second photoelectric conversion element arranged side by side in a row direction below the microlens, and wherein the first photoelectric conversion element is positioned ahead of the second photoelectric conversion element along the row direction; An image sensor comprising a row decoder that controls image signals to be output from a first pixel placed in a first row of the pixel array during a first readout period, controls a first image signal generated from the first photoelectric conversion element of the first pixel to be output during a first sub-period of the first readout period, controls a first summed image signal generated from the first photoelectric conversion element and the second photoelectric conversion element of the first pixel to be output during a second sub-period of the first readout period, controls image signals to be output from a second pixel placed in a second row of the pixel array during a second readout period, controls a second image signal generated from the second photoelectric conversion element of the second pixel to be output during a first sub-period of the second readout period, and controls a second summed image signal generated from the first photoelectric conversion element and the second photoelectric conversion element of the second pixel to be output during a second sub-period of the second readout period. Claim 2 An image sensor according to claim 1, characterized in that a pair of phase detection signals for autofocus is generated based on each of the first image signal and the second image signal. Claim 3 An image sensor according to claim 1, wherein the row decoder provides a plurality of first control signals to the first pixel circuit to control the sequential output of a first reset signal indicating the reset level of the first pixel, the first image signal, and the first summation signal from the first pixel during the first readout period, and provides a plurality of second control signals to the second pixel circuit to control the sequential output of a second reset signal indicating the reset level of the second pixel, the second image signal, and the second summation signal from the second pixel during the second readout period. Claim 4 An image sensor according to claim 1, wherein each of a plurality of pixels further comprises: a floating diffusion node; a first transfer transistor that is turned on to transfer a photocharge generated in the first photoelectric conversion element to the floating diffusion node; a second transfer transistor that is turned on to transfer a photocharge generated in the second photoelectric conversion element to the floating diffusion node; a reset transistor that is turned on to reset the floating diffusion node to a power supply voltage; and a driving transistor that outputs a pixel signal corresponding to the potential of the floating diffusion node, wherein during the first readout period, after the first transfer transistor of the first pixel is turned on, the second transfer transistor of the first pixel is turned on, and during the second readout period, after the second transfer transistor of the second pixel is turned on, the first transfer transistor of the second pixel is turned on. Claim 5 An image sensor according to claim 4, wherein the row decoder provides a first transmission control signal and a second transmission control signal to the first transmission transistor and the second transmission transistor of the first pixel, respectively, during the first readout period, and provides a third transmission control signal and a fourth transmission control signal to the first transmission transistor and the second transmission transistor of the second pixel, respectively, during the second readout period, wherein the first transmission control signal and the fourth transmission control signal are identical and the second transmission control signal and the third transmission control signal are identical. Claim 6 An image sensor according to claim 5, wherein the row decoder comprises a predecoder that, during a first readout period, selects the first reference control signal among the first reference control signal and the second reference control signal as the first transmission control signal and selects the second reference control signal as the second transmission control signal, and during a second readout period, selects the second reference control signal among the first reference control signal and the second reference control signal as the third transmission control signal and selects the first reference control signal as the fourth transmission control signal. Claim 7 An image sensor according to claim 1, further comprising a readout circuit that converts the first image signal and the second image signal into digital signals based on N counting clocks (N is a positive integer greater than or equal to 2), and converts the first summing signal and the second summing signal into digital signals based on 2N counting clocks. Claim 8 An image sensor according to claim 1, wherein the first pixel and the second pixel convert an optical signal of the same wavelength band into an electrical signal. Claim 9 An image sensor according to claim 1, wherein the first pixel and the second pixel are arranged in the same column within the pixel array. Claim 10 In claim 1, each of the plurality of pixels further includes a third photoelectric conversion element and a fourth photoelectric conversion element arranged parallel to the first photoelectric conversion element and the second photoelectric conversion element, respectively, in a column direction perpendicular to the row direction, and the row decoder controls such that a third image signal generated from the first photoelectric conversion element and the third photoelectric conversion element is output from the first pixel during the first sub-period of the first readout period, and a third summed image signal generated from the first photoelectric conversion element, the second photoelectric conversion element, the third photoelectric conversion element, and the fourth photoelectric conversion element is output during the second sub-period of the first readout period, and a fourth image signal generated from the second photoelectric conversion element and the fourth photoelectric conversion element is output from the second pixel during the first sub-period of the second readout period, and during the second sub-period of the second readout period, the first photoelectric conversion element, the second photoelectric conversion element, and the third photoelectric conversion element An image sensor characterized by controlling the output of a fourth summed image signal generated from the element and the fourth photoelectric conversion element. Claim 11 A pixel array comprising a plurality of row lines extended in a first direction, a plurality of column lines extended in a second direction perpendicular to the first direction, and a plurality of pixels connected to the plurality of row lines and the plurality of column lines and arranged in a matrix, wherein each of the plurality of pixels includes a first subpixel and a second subpixel arranged side by side, wherein the first subpixel is arranged on a first side and the second subpixel is arranged on a second side opposite to the first side; An image sensor comprising a row decoder that provides a plurality of first control signals to a first pixel placed in a first row of the pixel array and provides a plurality of second control signals to a second pixel placed in a second row of the pixel array, wherein during a first readout period, the first pixel outputs image signals through a first column line among a plurality of column lines in response to the plurality of first control signals, outputs a first image signal from the first subpixel during a first sub-period during the first readout period, outputs a first summed image signal from the first subpixel and the second subpixel during a second sub-period during the first readout period, and during a second readout period, the second pixel outputs image signals through the first column line in response to the plurality of second control signals, outputs a second image signal from the second subpixel during a first sub-period during the second readout period, and outputs a second summed image signal from the first subpixel and the second subpixel during a second sub-period during the second readout period. Claim 12 An image sensor according to claim 11, further comprising a signal processing unit that performs a phase difference operation for autofocus based on a pair of phase detection signals generated based on the first image signal and the second image signal. Claim 13 An image sensor according to claim 11, wherein the first subpixel and the second subpixel each comprise a photoelectric conversion element, and the first subpixel and the second subpixel share a single microlens. Claim 14 An image sensor according to claim 11, wherein the first subpixel and the second subpixel are arranged side by side in the first direction. Claim 15 An image sensor according to claim 11, wherein the first subpixel and the second subpixel are arranged side by side in the second direction. Claim 16 A pixel array comprising a plurality of pixels arranged in a matrix, wherein the plurality of pixels comprises a plurality of first pixels arranged in a first row and a plurality of second pixels arranged in a second row, wherein each of the plurality of first pixels and the plurality of second pixels comprises a first photoelectric conversion element and a second photoelectric conversion element arranged parallel to each other in the row direction, and wherein the first photoelectric conversion element is positioned ahead of the second photoelectric conversion element along the row direction; and a row decoder providing a plurality of first control signals to the plurality of first pixels through a plurality of first row lines of the pixel array and providing a plurality of second control signals to the plurality of second pixels through a plurality of second row lines of the pixel array; An image sensor comprising a readout circuit that receives a plurality of pixel signals from a pixel array through a plurality of column lines of the pixel array and converts the plurality of pixel signals from analog to digital, wherein each of the plurality of first pixels outputs pixel signals to the readout circuit during a first readout period, outputs a first reset signal during a first sub-period, outputs a first image signal from the first photoelectric conversion element during a second sub-period, and outputs a first summation signal from the first photoelectric conversion element and the second photoelectric conversion element during a third sub-period, and each of the plurality of second pixels outputs pixel signals to the readout circuit during a second readout period, outputs a second reset signal during a first sub-period, outputs a second image signal from the second photoelectric conversion element during a second sub-period, and outputs a second summation signal from the first photoelectric conversion element and the second photoelectric conversion element during a third sub-period. Claim 17 An image sensor according to claim 16, characterized in that the first photoelectric conversion element and the second photoelectric conversion element are configured to be connected to the same floating diffusion node. Claim 18 An image sensor according to claim 16, characterized in that a first phase detection signal of a phase detection signal pair is generated based on a plurality of first image signals output from a plurality of first pixels, and a second phase detection signal of a phase detection signal pair is generated based on a plurality of second image signals output from a plurality of second pixels. Claim 19 An image sensor according to claim 16, wherein the pixel array further comprises a plurality of third pixels arranged in a third row and a plurality of fourth pixels arranged in a fourth row, wherein each of the plurality of third pixels outputs pixel signals to the readout circuit during a third readout period, outputs a third reset signal during a first sub-period, outputs a third image signal from the first photoelectric conversion element during a second sub-period, and outputs a third summation signal from the first photoelectric conversion element and the second photoelectric conversion element during a third sub-period, and each of the plurality of fourth pixels outputs pixel signals to the readout circuit during a fourth readout period, outputs a fourth reset signal during a first sub-period, outputs a fourth image signal from the second photoelectric conversion element during a second sub-period, and outputs a fourth summation signal from the first photoelectric conversion element and the second photoelectric conversion element during a third sub-period. Claim 20 An image sensor according to claim 19, wherein a first phase detection signal of a phase detection signal pair is generated based on a plurality of first image signals output from a plurality of first pixels and a plurality of third image signals output from a plurality of third pixels, and a second phase detection signal of a phase detection signal pair is generated based on a plurality of second image signals output from a plurality of second pixels and a plurality of fourth image signals output from a plurality of fourth pixels.

Images