Image Sensor, Binning Method, and Image Processing System

Abstract

To provide an image sensor, and a binning method of an image sensor.SOLUTION: The present invention relates to an image sensor and a binning method of an image sensor. The binning method of an image sensor includes the steps of: simultaneously reading out a plurality of pixel signals from at least two rows of each of a plurality of areas of pixel arrays, which are segmented into the plurality of areas including a plurality of pixels arranged in a (2n)×(2n) matrix, (where n is an integer equal to or greater than 2); generating first image data by performing analog-to-digital conversion on the plurality of pixel signals having been read out, the first image data including a plurality of binning areas corresponding to the plurality of areas of the pixel arrays; generating a first summation value of each of a plurality of binning areas based on two pixel values corresponding to a same color in each of the plurality of binning areas, the plurality of binning areas corresponding to the plurality of areas of the pixel arrays; and generating a second summation value of each of two binning areas, based on two first summation values corresponding to a same color in the two binning areas, the two binning areas being adjacent to each other in a column direction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image sensor, and more particularly, to a binning method of an image sensor, an image sensor that performs the method, and an image processing system.

Background Art

[0002] With the increase in the resolution of image sensors, the size of the image data generated by the image sensors has also been increasing. However, the larger the size of the image data generated by the image sensor, the more difficult it is to maintain a high frame rate in the video mode, and the power consumption also increases. For example, the frame rate is a frame rate achievable based on the computational bandwidth and circuit interface bandwidth related to preventing artifacts such as zigzag artifacts or false colors when changing the sample rate. If the amount of calculation required to avoid artifacts is large or the circuit interface bandwidth is not wide, there will be a problem that the achievable frame speed is limited. In order to increase the frame rate and maintain the image quality, a binning technique is used.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The problem to be solved by the present invention is to provide a binning method of an image sensor that can increase the frame rate, reduce the data size, and maintain the image quality, and an image sensor that performs the method.

Means for Solving the Problems

[0004] To achieve the above object, a binning method of an image sensor according to an exemplary embodiment of the present disclosure is such that a pixel array is divided into a plurality of regions each including a plurality of pixels (n is an integer of 2 or more) arranged in a (2n)×(2n) matrix, and for each of the plurality of regions, simultaneously reading out a plurality of pixel signals in at least two rows of row units, generating first image data by analog-to-digital converting the read-out plurality of pixel signals, the first image data including a plurality of binning regions corresponding to the plurality of regions of the pixel array, for each of the plurality of binning regions, generating a first summation value based on two pixel values having the same hue within the binning region, and in two binning regions adjacent in the column direction, generating a second summation value for each of the two binning regions based on the two first summation values indicating the same hue.

[0005] To achieve the above object, an image sensor according to an exemplary embodiment of the present disclosure includes a pixel array divided into a plurality of rectangular regions each including pixels (n is an integer of 2 or more) arranged in a (2n) / (2n) matrix, an analog-to-digital conversion circuit that reads out a plurality of pixel signals received from the pixel array via a plurality of column lines and converts the read-out plurality of pixel signals into a plurality of pixel values, a row driver that provides a control signal via a plurality of row lines connected to the pixel array and generates the control signal for controlling the pixel array to output a plurality of pixel signals simultaneously in at least two rows of row units, a line buffer that stores the first image data output from the analog-to-digital conversion circuit in a predetermined line unit, and a processor that performs binning on the first image data stored in the line buffer.

[0006] To achieve the above object, an image processing system according to an exemplary embodiment of the present disclosure includes an image sensor that senses an optical signal to generate image data, and a processor that receives and processes the image data from the image sensor. The image sensor includes a pixel array divided into a plurality of rectangular regions including pixels arranged in a 4X4 matrix, an analog-to-digital conversion circuit that reads out a plurality of pixel signals received from the pixel array via a plurality of column lines and converts the read-out plurality of pixel signals into a plurality of pixel values, a row driver that provides a control signal via a plurality of row lines connected to the pixel array and generates the control signal to control the pixel array to output a plurality of pixel signals simultaneously at least in units of two rows, a line buffer that stores the first image data output from the analog-to-digital conversion circuit in a predetermined line unit, and a processor that performs binning on the first image data stored in the line buffer.

Advantages of the Invention

[0007] According to the binning method of an image sensor according to an exemplary embodiment of the present disclosure, by simultaneously reading out at least two rows by a vertical analog summation method, the frame rate is increased, and by performing binning on the generated image data, it is possible to prevent false color and zigzag noise from occurring in the image frame.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 5C

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12

Figure 13

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0010] FIG. 1 is a block diagram schematically showing an image sensor according to an exemplary embodiment of the present disclosure.

[0011] The image sensor 100 can be mounted on an electronic device having an image or light sensing function. For example, the image sensor 100 can also be mounted on electronic devices such as cameras, smartphones, wearable devices, Internet of Things (IoT) devices, home appliances, tablet PCs (personal computers), PDAs (personal digital assistants), PMPs (portable multimedia players), navigation devices, drones, and advanced driver assistance systems (ADAS). In addition, the image sensor 100 can also be mounted on electronic devices equipped as components in vehicles, furniture, manufacturing equipment, doors, various measuring instruments, and the like.

[0012] Referring to FIG. 1, the image sensor 100 may include a pixel array 110, a row driver 120, an analog-to-digital converter (ADC) circuit 130, a lamp signal generator 140, a timing controller 150, a line buffer 160, and a processor 170.

[0013] The pixel array 110 includes a plurality of pixels PX arranged in a matrix, a plurality of row lines RL connected to the plurality of pixels PX, and a plurality of column lines CL.

[0014] Each of the plurality of pixels PX includes at least one photoelectric conversion element (or light sensing element), and the photoelectric conversion element can sense light and convert the sensed light into photo charges. For example, the photoelectric conversion element is also a light sensing element composed of an organic or inorganic substance, such as an inorganic photodiode, an organic photodiode, a perovskite photodiode, a phototransistor, a photogate, or a pinned photodiode. In one embodiment, each of the plurality of pixels PX may include a plurality of photoelectric conversion elements. The plurality of light sensing elements are arranged in the same layer or stacked in directions perpendicular to each other.

[0015] A microlens for light collection can be arranged on the top of each of the plurality of pixels PX or on the top of each pixel group composed of adjacent pixels PX. Each of the plurality of pixels PX can sense light in a specific spectral region from the light received through the microlens. For example, the pixel array 110 may include red pixels that convert light in the red spectral region into an electrical signal, green pixels for converting light in the green spectral region into an electrical signal, and blue pixels for converting light in the blue spectral region into an electrical signal. A color filter for transmitting light in a specific spectral region can be arranged on the top of each of the plurality of pixels PX. However, without being limited thereto, the pixel array 110 may include pixels that convert light in other spectral regions into electrical signals in addition to red, green, and blue.

[0016] In one embodiment, the plurality of pixels 111 can also have a multilayer structure. The pixels 111 of the multilayer structure include stacked photosensing elements that convert light in mutually different spectral regions into electrical signals, and electrical signals corresponding to mutually different hues can be generated from the photosensing elements. In other words, electrical signals corresponding to a plurality of colors can be output from one pixel PX.

[0017] The pixel array 110 can have a Bayer pattern in which a first pixel, a second pixel, and a third pixel that sense signals of mutually different hues are repeatedly arranged in the column direction and the row direction.

[0018] FIG. 2 exemplarily shows the pattern of the pixel array in FIG. 1.

[0019] Referring to FIG. 2, in the pixel array 110, a row in which red pixels PX_R, green pixels, for example, a first green pixel PX_Gr are arranged, and a row in which still other green pixels, for example, a second green pixel PX_Gb and blue pixels PX_B are arranged are repeatedly arranged, and the green pixels, for example, the first green pixel PX_Gr and the second green pixel PX_Gb can be located diagonally to each other. The green pixels, for example, the first green pixel PX_Gr and the second green pixel PX_Gb are closely related to luminance, are arranged in all rows, and the red pixels PX_R and the blue pixels PX_B are also arranged alternately for each row.

[0020] Such a pattern is referred to as an RGB Bayer pattern. Hereinafter, in the present disclosure, an example in which the pixel array 110 has an RGB Bayer pattern will be described. However, the technical idea of the present disclosure is not limited thereto, and various patterns in which pixels of three different hues are repeatedly arranged and the second pixel, for example, the pixel related to luminance, is matched in all rows while the second pixel has a structure located diagonally in adjacent rows are also applicable to the pixel array 110. For example, an RYYB pattern composed of a red pixel, two yellow pixels, and a blue pixel is also applicable to the pixel array 110.

[0021] The pixel array 110 is divided into a plurality of regions AR, and each of the plurality of regions AR may include a plurality of pixels PX arranged in a (2n)×(2n) matrix (n is an integer of 2 or more). For example, as shown in the drawing, each of the plurality of regions AR may include a plurality of pixels PX arranged in a 4×4 matrix. At this time, each of the plurality of regions AR is a basic unit to which the readout method according to the exemplary embodiment of the present disclosure is applied when the image sensor 100 operates in the first mode of performing binning, and may correspond to each of a plurality of binning regions of image data generated based on the readout signal. According to the readout method according to the exemplary embodiment of the present disclosure, for each of the plurality of regions AR, a plurality of pixel signals are simultaneously read out in at least two rows of row units. For example, in one horizontal period, a plurality of pixel signals of a plurality of pixels PX corresponding to at least two rows of rows may be read out. The readout method according to the exemplary embodiment of the present disclosure will be described later with reference to FIGS. 4A to 5B.

[0022] On the other hand, when the image sensor 100 operates in the second mode, for example, the normal mode without performing binning, the pixel array 110 may have a plurality of pixel signals read out in order in row units.

[0023] Subsequently, referring to FIG. 1, each of the plurality of row lines RL extends in the row direction and is also connected to the pixels PX arranged in the same row. For example, each of the plurality of row lines RL can transmit a control signal output from the row driver 120 to an element provided in the pixel PX, for example, each of the plurality of transistors.

[0024] Each of the plurality of column lines CL extends in the column direction and is also connected to the pixels PX arranged in the same column. Each of the plurality of column lines CL can transmit pixel signals output from the pixels PX, such as reset signals and sensing signals, to the ADC circuit 130 in units of rows of the pixel array 110. As described above, when the image sensor 100 operates in the first mode, some of the plurality of column lines CL can transmit pixel signals in units of at least two rows.

[0025] The timing controller 150 can control the timing of other components of the image sensor 100, such as the row driver 120, the ADC circuit 130, the ramp signal generator 140, the line buffer 160, and the processor 170. The timing controller 150 can provide timing signals indicating the operation timing to each of the row driver 120, the ADC circuit 130, the ramp signal generator 140, the line buffer 160, and the processor 170.

[0026] Under the control of the timing controller 150, the row driver 120 can generate a control signal for driving the pixel array 110 and provide the control signal to each of the plurality of pixels PX of the pixel array 110 via the plurality of row lines RL. The row driver 120 can control the plurality of pixels PX of the pixel array 110 to sense incident light simultaneously or in units of rows. Also, the row driver 120 can select the pixels PX in units of rows or in units of at least two rows among the plurality of pixels PX and control the selected pixels PX to output pixel signals via the plurality of column lines CL.

[0027] The ramp signal generator 140 can generate a ramp signal RAMP that increases and decreases at a predetermined gradient and provide the ramp signal RAMP to the ADC circuit 130.

[0028] The ADC circuit 130 can receive a plurality of pixel signals read out from a plurality of pixels PX of the row selected by the loud driver 120 among the plurality of pixels PX and convert them into a plurality of pixel values that are digital data.

[0029] The ADC circuit 130 can generate and output first image data IDT1, for example, raw image data, in row units by converting a plurality of pixel signals received from the pixel array 110 via a plurality of column lines CL into digital data based on the ramp signal RAMP from the ramp signal generator 140.

[0030] The ADC circuit 130 includes a plurality of ADCs corresponding to a plurality of column lines CL. Each of the plurality of ADCs can compare a pixel signal received via a corresponding column line CL with each of the ramp signals RAMP and generate a pixel value based on the comparison result. For example, the ADC can remove a reset signal from a sensing signal in a correlated double sampling (CDS) method and generate a pixel value indicating the amount of light sensed by the pixel PX.

[0031] The line buffer 160 includes a plurality of line memories and can store a plurality of pixel values output from the ADC circuit 130 in predetermined row units. In other words, the line buffer 160 can store the first image data IDT1 output from the ADC circuit 130 in predetermined row units. For example, the line buffer 160 includes three line memories corresponding to three rows of the pixel array 110, and can store a plurality of pixel values corresponding to three rows of the first image data IDT1 output from the ADC circuit 130 in the three line memories.

[0032] The processor 170 can process a plurality of pixel values corresponding to a plurality of rows of the first image data IDT1 stored in the line buffer 160. The processor 170 can perform image quality compensation, binning, downsampling, etc. on the first image data IDT1 in a predetermined row unit with respect to the line buffer 160. As a result, the image-processed output image data OIDT is also generated and output in a predetermined row unit.

[0033] In one embodiment, the processor 170 can process the first image data IDT1 by hue. For example, when the first image data IDT1 includes red pixel values, green pixel values, and blue pixel values, the processor 170 can process each of the red pixels, green pixels, and blue pixels in parallel or in series. In one embodiment, the processor 170 may include a plurality of processing circuits in order to perform such hue-by-hue processing in parallel. However, it is not limited thereto, and one processing circuit may be repeatedly used.

[0034] The processor 170 can generate output image data OIDT with a reduced data size by performing a binning method according to an exemplary embodiment of the present disclosure described later.

[0035] The output image data OIDT is output to an external processor, for example, an application processor, and the application processor can store, image-process, or display the output image data OIDT.

[0036] According to an exemplary embodiment of the present disclosure, when the image sensor 100 operates in the first operation mode, a plurality of pixel signals of at least two rows of rows can be simultaneously read out while being analog-summed in the vertical direction (e.g., the column direction). According to such a vertical analog summing method, at least two rows of rows are simultaneously read out in one horizontal period, and the frame rate can be increased by at least two times or more.

[0037] By the vertical analog summing method, first image data IDT1 is generated, and the processor 170 can perform binning on the first image data IDT1. Thereby, the data amount of the output image data OIDT can be reduced, and the occurrence of false color and zigzag noise due to the sampling frequency difference can be alleviated.

[0038] FIG. 3A is a flowchart showing vertical analog summing and interpolation of an image sensor according to an exemplary embodiment of the present disclosure. The binning method in FIG. 3A is also performed in the image sensor 100 of FIG. 1.

[0039] Referring to FIG. 3A, pixels generate photoelectric charges (S010). The method of generating the photoelectric charges will be described with reference to FIG. 1, and more detailed content will be described later with reference to FIGS. 10 and 12.

[0040] The photoelectric charges are analog-summed on the column wiring (S020). For example, the photoelectric charges are also summed on the column line CL in FIG. 1. The summing result is also displayed as IDT0.

[0041] The voltage generated from the charge summing is also converted into a digital value by analog-to-digital conversion (ADC) (S030). The digital value is generated by, for example, the ADC circuit 130 in FIG. 1 and can form the image IDT1 in FIG. 1. This will be described later with reference to FIG. 4B as an example.

[0042] The pixels of the image IDT1 are interpolated even when using weighting values (S040). This will be described later with reference to FIG. 6 as an example.

[0043] The interpolated digital value is output, for example, as image data OIDT (FIG. 1) (S050).

[0044] FIG. 3B is a flowchart showing a binning method of an image sensor according to an exemplary embodiment of the present disclosure. The operations in FIG. 3B are also performed in the image sensor 100 of FIG. 1.

[0045] Referring to FIGS. 1 and 3B, the image sensor 100 can simultaneously read out a plurality of pixel signals in at least two rows for each of the plurality of regions AR of the pixel array 110 in units of rows. Thereby, as described above, at least two pixel signals output from at least two pixels arranged in the same column of at least two rows are also analog-summed.

[0046] The image sensor 100 can perform analog-to-digital conversion on the read-out plurality of pixel signals to generate first image data IDT1. For example, the ADC circuit 130 can generate the first image data IDT1 by performing analog-to-digital conversion on the plurality of pixel signals received via the plurality of column lines CL. Thereafter, digital binning is performed.

[0047] The first image data IDT1 is divided into a plurality of binning regions, and the image sensor 100 can perform first binning for each of the plurality of binning regions of the first image data IDT1 based on the pixel values within the binning region (S130). The image sensor 100 can weighted-sum pixel values having the same hue as each other within the binning region. In other words, a weighting value set for each pixel value can be added, and the values added with the weighting value can be averaged.

[0048] The image sensor 100 can perform interpolation based on the pixel values of two binning regions adjacent to each other in the column direction (S140). The image sensor 100 can perform weighted summation of pixel values having the same hue in the two binning regions.

[0049] Thereby, second image data with a size reduced from the first image data IDT1 can be output (S150). For example, when a plurality of binning regions include pixel values arranged in a 4×4 matrix, the size of the second image data can correspond to 1 / 4 times the resolution of the pixel array 110.

[0050] Hereinafter, with reference to FIGS. 4A to 9, the binning method according to the exemplary embodiment of the present disclosure will be described in more detail.

[0051] FIGS. 4A and 4B are drawings for explaining a readout method according to the exemplary embodiment of the present disclosure, and FIGS. 5A to 5C schematically show first image data generated by the readout method according to the exemplary embodiment of the present disclosure.

[0052] FIGS. 4A and 4B show the readout for one region AR of the pixel array 110. The region AR may include a plurality of pixels PX arranged in a 4×4 matrix.

[0053] Referring to FIG. 4A, in the first horizontal period, the first row Row1 and the third row Row3 can be read out simultaneously. The pixel signals of the first green pixels Gr1, Gr3 arranged in the first column C1 are also output via the first column line CL1, and the pixel signals of the red pixels R2, R4 arranged in the fourth column C4 are also output via the fourth column line CL4.

[0054] When pixel signals of two pixels, for example, the first green pixels Gr1 and Gr3, are output via the first column line CL1, the pixel signals can be added together. However, when a pixel signal is output from a pixel PX, the pixel PX can operate as a source follower, and due to the parasitic resistance inside the pixel PX, an added signal having a value adjacent to a relatively high value in the pixel signals of the first green pixels Gr1 and Gr3 can be provided to the ADC circuit 130 via the first column line CL1 as the pixel signal corresponding to the first green pixels Gr1 and Gr3.

[0055] On the other hand, among the red pixels R1 and R3 arranged in the second column C2, the pixel signal of the red pixel R1 arranged relatively on the outer contour of the region AR is also output via the second column line CL2, and among the first green pixels Gr2 and Gr4 arranged in the third column C3, the pixel signal of the green pixel Gr2 arranged relatively on the outer contour of the region AR is also output via the third column line CL3. In other words, in the second column C2 and the third column C3, among the pixels having the same hue arranged in the first row Row1 and the second row Row2, the pixel signals of the pixels arranged relatively on the outer contour are also led out. The pixel signals of the pixels arranged relatively inside are ignored without being led out.

[0056] The ADC circuit 130 can convert the received pixel signal into pixel values PGr13, PR1, PGr2, and PR24 which are digital values. In one embodiment, the pixel values PGr13, PR1, PGr2, and PR24 generated in the first horizontal period are also stored in the same line memory, for example, the first line memory LM1 of the line buffer 160, and can constitute a part of the first image data IDT1. However, the pixel values PGr13, PR1, PGr2, and PR24 do not correspond to the same row of the first image data IDT1.

[0057] Referring to FIG. 4B, in the second horizontal period following the first horizontal period, Row 2 and Row 4 can be read out simultaneously. Pixel signals of blue pixels B1, B3 arranged in the first column C1 are also output via the first column line CL1, and pixel signals of the second green pixels Gb2, Gb4 arranged in the fourth column C4 are also output via the fourth column line CL4. At this time, as described with reference to FIG. 4A, two pixel signals output to the same column line are added together, and the added signal is also provided to the ADC circuit 130.

[0058] Among the second green pixels Gb1, Gb3 arranged in the second column C2, the pixel signal of the first green pixel Gb3, which is relatively arranged on the outer contour of the region AR, is also output via the second column line CL2. Among the blue pixels B2, B4 arranged in the third column C3, the pixel signal of the block pixel B4, which is relatively arranged on the outer contour of the region AR, is also output via the third column line CL3.

[0059] The ADC circuit 130 can convert the received pixel signals into pixel values PB13, PGb3, PB4, PGb24, which are digital values. In one embodiment, the pixel values PB13, PGb3, PB4, PGb24 generated in the second horizontal period are also stored in the same line memory, for example, the second line memory LM2 of the line buffer 160, and can form a part of the first image data IDT1. In one embodiment, the pixel values PGr13, PR1, PGr2, PR24 generated in the first horizontal period are moved from the first line memory LM1 to the second line memory LM2, and the pixel values PB13, PGb3, PB4, PGb24 generated in the second horizontal period are also stored in the first line memory LM1. The line buffer 160 also includes a third line memory LM3 similar to the first line memory LM1 and the second line memory LM2, and the third line memory LM3 is also used continuously with the first line memory LM1 and the second line memory LM2.

[0060] Referring to FIG. 5A, the pixel values PGr13, PR1, PGr2, PR24 stored in the first line memory LM1 can form the first row Row1 and the second row Row2 within the binning region BA of the first image data IDT1. Since each of the pixel values PGr13, PR24 is the sum value of the pixel signals of two pixels located in the first row Row1 and the third row Row3 of one region AR of the pixel array 110, for example, it can indicate the pixel value corresponding to the sampling position at the midpoint between two pixels, such as two pixels Gr1 and Gr3, or pixels R2 and R4 (FIG. 4A). The pixel values PR1, PGr2 can indicate the pixel values corresponding to the positions of the corresponding pixels R1, Gr2 (FIG. 4A).

[0061] Referring to FIG. 5B, the pixel values PB13, PGb3, PB4, PGb24 stored in the second line memory LM2 can form the third row Row3 and the fourth row Row4 within the binning region BA of the first image data IDT1. Since each of the pixel values PB13, PGb24 is the sum value of the pixel signals of, for example, two pixels B1 and B3, or pixels Gb2 and Gb4 (FIG. 4B) located in the second row Row2 and the fourth row Row4 of one region AR of the pixel array 110, it can indicate the pixel value corresponding to the sampling position at the midpoint between the two pixels. The pixel values PGb3, PB4 can indicate the pixel values corresponding to the positions of the corresponding pixels Gb3, B4 (FIG. 4B).

[0062] Referring to FIG. 5C, according to an exemplary embodiment of the present disclosure, by reading out the region AR of the pixel array 110, as shown, the pixel values related to the binning region BA of the first image data IDT1 can be determined.

[0063] FIG. 6 shows that in the binning method according to an exemplary embodiment of the present disclosure, for each of the plurality of binning regions of the first image data, the first binning is performed based on the pixel values within the binning region.

[0064] Referring to FIG. 6, in the binning region BA, first binning is also performed by summing pixel values corresponding to the same hue.

[0065] For example, pixel values PGr13 and PGr2 corresponding to the first green hue are summed, and the pixel value corresponding to the sampling position S11 is also calculated. At this time, weighting values preset for each of the pixel values PGr13 and PGr2 are given, and the values with the weighting values are averaged and summed. The weighting value can be preset in consideration of the sampling position. In other words, the weighting value is also set so that the average sum value of the values with the weighting value is located at the preset sampling position S11. For example, if the distance between the centers of the pixels indicated by the pixel values PGr13 and PGr2 is 10 and the sampling position S11 is located at a distance of about 3 from the pixel value PGr13, the ratio of the weighting values added to each of the pixel values PGr13 and PGr2 is also 7 to 3. In other words, a higher weighting value can be added to the pixel value PGr13.

[0066] In this way, weighting values are given so as to be located at the sampling positions S12, S13, and S14 of the other pixel values corresponding to the same hue, and the values with the weighting values can be averaged and summed. Thereby, pixel values PGr_b, PR_b, PB_b, and PGb_b corresponding to the sampling positions S11, S12, S13, and S14 can be calculated.

[0067] FIG. 7 shows interpolation being performed based on pixel values of two adjacent binning regions in the binning method according to an exemplary embodiment of the present disclosure.

[0068] Referring to FIG. 7, the first image data IDT1 may include a plurality of binning regions BAn-1, BAn, and BAn+1. Interpolation is performed among the plurality of binning regions BAn-1, BAn, and BAn+1, and thereby, pixel values corresponding to the target sampling positions TS1, TS2, TS3, and TS4 are also generated.

[0069] In the binning region BAn, an example will be given to illustrate generating pixel values corresponding to target sampling positions. As described with reference to FIG. 6, in the binning region BAn, pixel values PGr_b, PR_b, PB_b, Gb_b corresponding to sampling positions S11, S12, S13, S14 are calculated, and the pixel values PGr_b, PR_b, PB_b, PGb_b are also added to the pixel values corresponding to the closest positions among the pixel values of the same hue in adjacent other binning regions BAn-1, BAn+1. A set weight value is added to each of the added pixel values, and the weight value is set in consideration of the position of each pixel value and the target sampling position. The closer the distance between the position corresponding to the pixel value and the target sampling position, the higher the weight value can be set. In this way, by interpolation, pixel values PGr_t, PR_t, PB_t, PGb_t corresponding to target sampling positions TS1, TS2, TS3, TS4 in the binning region BAn are also calculated.

[0070] Thereby, second image data including pixel values arranged in a 2×2 matrix can be generated from a plurality of pixels PX arranged in a 4×4 matrix through vertical analog addition and binning by readout.

[0071] FIG. 8 is a flowchart showing a binning method according to an exemplary embodiment of the present disclosure. The binning method of FIG. 8 is also performed in the image sensor 100 of FIG. 1. Steps S210, S220, S230, S240 are respectively the same as steps S110, S120, S130, and S140 of FIG. 3. Therefore, duplicate descriptions are omitted.

[0072] In step S220, after generating the first image data IDT1, the image sensor 100 can perform second binning on the green pixels (S250). For each of the multiple binning regions of the first image data IDT1, the image sensor 100 can sum at least two green pixel values within the binning region and the green pixel values in other binning regions adjacent to the binning region. For example, binning regions different from each other can be adjacent to each other in the column direction. For each pixel, a weighting value can be added in consideration of the sampling position, and the values with the weighting values added can be averaged and summed. Referring to FIG. 9, the second binning will be described in detail.

[0073] FIG. 9 is a drawing for explaining the second binning applied to green pixels in a binning method according to an exemplary embodiment of the present disclosure.

[0074] Referring to FIG. 9, binning region BAn and binning region BAn-1 are the closest adjacent to each other in the column direction.

[0075] The pixel values PGr13, PGr2 corresponding to the first green pixel in binning region BAn and the pixel value PGb3 corresponding to the second green pixel closest to the first green pixel in binning region BAn in binning region BAn-1 can be summed. At this time, weighting values are assigned to the respective pixel values so that the sum value is located at the target sampling position TS1, and the values with the weighting values added can be averaged and summed. Thereby, the pixel value PGr_t' of the green pixel corresponding to the target sampling position TS1 can be generated. In such a manner, in binning region BAn, the pixel value of the green pixel is determined at the target sampling position corresponding to the green pixel, and thus the third image data IDT3 including the pixel value of the green pixel is also generated.

[0076] When performing interpolation in step S240, as shown in FIG. 7, the distance between the pixel values to be summed is far. However, according to the second binning method, in the binning region BAn, the pixel value of the green pixel at the closest distance can be summed with the pixel value of the binning region BAn. Therefore, binning will also be performed based on the pixel values corresponding to further adjacent pixels.

[0077] Referring to FIG. 8 again, based on the pixel value difference between the first green pixel and the second green pixel, the second image data and the third image data can be merged (S270). As described above, the third image data may include pixel values related to green pixels. Therefore, in the second image data, the pixel values related to green pixels and in the third image data, the pixel values related to green pixels can be merged. In step S280, the merged image can be output.

[0078] At this time, based on the difference in the pixel values of the first green pixel and the second green pixel used during the second binning operation, which correspond to the closest adjacent pixels, the second image data and the third image data can be merged.

[0079] For example, when the difference in pixel values is less than the first reference value, in other words, when the difference in pixel values is very small, the pixel value of the green pixel in the third image data will also be applied to the output image data. In other words, the pixel values related to the red pixels and blue pixels in the second image data, and the pixel values related to the green pixels in the third image data are also included in the output image data. Conversely, when the difference in pixel values exceeds the second reference value, in other words, when the difference in pixel values is very large, the second image data will also be selected as the output image data. In other words, the green pixel value of the third image data is not reflected in the output image data. The comparison between the difference value and the threshold value can provide a non-linear stage useful for reducing artifacts such as zigzag artifacts and false color artifacts.

[0080] When the difference in pixel values is greater than or equal to the first reference value and less than the second reference value, the difference is converted to a value of 1 or less based on the first reference value and the second reference value, and weights corresponding to the converted value are applied to the second image data and the third image data respectively, and the values to which the weights are applied can be added together. For example, when the converted value of the difference in pixel values is 0.6, 0.4 is applied to the pixel value of the green pixel in the second image data, and a weight value of 0.6 is applied to the pixel value of the green pixel in the third image data, and the values to which the weights are applied can be added together. Based on the combined pixel value of the green pixels and the pixel values of the red pixels and blue pixels in the second image data, output image data can be generated.

[0081] FIG. 10 shows an example of a pixel according to an exemplary embodiment of the present disclosure.

[0082] Referring to FIG. 10, pixel PX includes a photoelectric conversion element 11 and a pixel circuit 12, and the pixel circuit 12 may include a plurality of transistors, for example, a transfer transistor TX, a reset transistor RX, a drive transistor DX, and a switching transistor SX.

[0083] The photoelectric conversion element 11 is also, for example, a photodiode. The photodiode generates photo charges that are variable according to the intensity of incident light. The transfer transistor TX can transfer the photo charges to the floating diffusion node FD by a transfer control signal TS provided from a low driver 120 (FIG. 1). The drive transistor DX can amplify and output a voltage due to the photo charges accumulated in the floating diffusion node FD. The drive transistor DX can operate as a source follower. When the drain node of the selection transistor SX is connected to the source node of the drive transistor DX and the selection transistor SX is turned on in response to a selection signal SEL output from the low driver 120, a pixel signal APS having a level corresponding to the voltage level of the floating diffusion node FD can be output to a column line CL connected to the pixel PX. The reset transistor RX can reset the floating diffusion node FD based on a power supply voltage VDD by a reset signal RS provided from the low driver 120.

[0084] As described with reference to FIGS. 4A and 4B, in the pixel array 110, at least two rows of rows are simultaneously read out. At this time, the pixels located at the center of one region AR of the pixel array 110 are not read out. Therefore, when two rows of rows are simultaneously read out, the pixels are not selected. The pixels to be read out respond to the first selection signal SEL1 at the active level and are connected to the column line CL, and the pixels not to be read out respond to the second selection signal SEL2 at the inactive level, and the connection with the column line CL is also blocked. Thereby, even in the pixels located in the same row, the pixel signals are selectively output or not output.

[0085] FIG. 11A shows a pixel array of a tetra pattern, and FIG. 11B shows an example in which the pixel array of the tetra pattern is applied to an image sensor according to an exemplary embodiment of the present disclosure.

[0086] Referring to FIG. 11A, the pixel array 110a has a tetra pattern. The red pixel PX_R, the first green pixel PX_Gr, the second green pixel PX_Gb, and the blue pixel PX_B are arranged in a 2×2 matrix, respectively, and the pattern can be repeated in a matrix. Such a pattern is also referred to as a quad Bayer pattern.

[0087] In one embodiment, the pixels arranged in a 2×2 matrix each include a photoelectric conversion element and can share a floating diffusion node and a pixel circuit, as illustrated in FIG. 12. Thereby, as illustrated in FIG. 11B, the pixels arranged in a 2×2 matrix can operate as, for example, one big pixel PX_R1, PX_Gr1, PX_Gb1, PX_B1. The big pixel can form a Bayer pattern. Thereby, as described above, the binning method according to the exemplary embodiment of the present disclosure can be applied. In the readout stage, in the first horizontal period, the first row Row1 and the third row Row3 are simultaneously read out, and in the second horizontal period, the second row Row2 and the fourth row Row4 are simultaneously read out.

[0088] FIG. 12 shows an example of a pixel according to an exemplary embodiment of the present disclosure.

[0089] Referring to FIG. 12, the pixel PXa may include a plurality of photoelectric conversion elements 22a, 22b, 22c, 22d and a pixel circuit 12. For example, the pixel PXa includes four photoelectric conversion elements 22a, 22b, 22c, 22d, and in some embodiments, may include four photodiodes PD1A, PD1B, PD1C, PD1D as the photoelectric conversion elements. Microlenses may be disposed on top of each of the plurality of photoelectric conversion elements 22a, 22b, 22c, 22d. Therefore, the combination of the microlens and the photoelectric conversion element is also referred to as one pixel, and thereby, the pixel PXa in FIG. 12 is also regarded as four pixels.

[0090] The pixel circuit 12 may include first to fourth transmission transistors Tx1 to TX4, a reset transistor RX1, a drive transistor DX1, and a selection transistor SX1 connected to each of the plurality of photoelectric conversion elements 22a, 22b, 22c, 22d.

[0091] The floating diffusion node FD is also shared by four photoelectric conversion elements 22a, 22b, 22c, 22d and four transmission transistors TX1, TX2, TX3, TX4. The transmission transistors TX1, TX2, TX3, TX4 can connect or disconnect each of the four first photodiodes PD1A, PD1B, PD1C, PD1D to / from the floating diffusion node FD1 according to the voltages of the transfer gates TG1, TG2, TG3, TG4.

[0092] The light incident on the first photodiodes PD1A, PD1B, PD1C, PD1D is also accumulated as charges in the photodiodes PD1A, PD1B, PD1C, PD1D by photoelectric conversion. When the charges accumulated in the photodiodes PD1A, PD1B, PD1C, PD1D are transmitted to the floating diffusion node FD1, they are output to the outside as the first analog voltage V1out through the drive transistor DX1 and the selection transistor SX1. The first analog voltage V1out corresponding to the voltage change of the floating diffusion node FD1 is also transmitted to an external lead-out circuit (not shown).

[0093] The pixel PXa according to this embodiment can be applied to the pixel array 110a in FIG. 11A. For example, the four photoelectric conversion elements 22a, 22b, 22c, 22d of the pixel PXa can correspond to each pixel arranged in a 2X2 pattern. In other words, each pixel arranged in a 2X2 pattern can share the floating diffusion node FD as in the pixel PXa of FIG. 12. The transmission transistors TX1, TX2, TX3, TX4 can be turned on or off simultaneously so that the pixels arranged in a 2X2 pattern can operate as one big pixel as shown in FIG. 11B. In one embodiment, when operating as a big pixel, only some of the transmission transistors TX1, TX2, TX3, TX4 can be turned on or off, and the rest can maintain the off state.

[0094] FIG. 13 is a block diagram of an electronic device including a multi-camera module to which an image sensor according to an exemplary embodiment of the present disclosure is applied. FIG. 14 is a detailed block diagram of the camera module of FIG. 13.

[0095] Referring to FIG. 13, the electronic device 1000 may include a camera module group 1100, an application processor 1200, a PMIC (power management integrated circuit) 1300, and an external memory 1400.

[0096] The camera module group 1100 may include a plurality of camera modules 1100a, 1100b, 1100c. Even though an embodiment in which three camera modules 1100a, 1100b, 1100c are arranged is illustrated in the drawings, the present embodiment is not limited thereto. In some embodiments, the camera module group 1100 may be modified or implemented to include only two camera modules. Also, in some embodiments, the camera module group 1100 may be modified or implemented to include n (n is a natural number of 4 or more) camera modules.

[0097] Hereinafter, referring to FIG. 14, the detailed configuration of the camera module 1100b will be described more specifically. However, according to the present embodiment, the following description may be equally applied to the other camera modules 1100a, 1100b.

[0098] Referring to FIG. 14, the camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140, and a storage unit 1150.

[0099] The prism 1105 includes a reflecting surface 1107 of a light reflecting material and can deform the path of the light L incident from the outside.

[0100] In some embodiments, the prism 1105 can change the path of the light L incident in the first direction X to the second direction Y perpendicular to the first direction X. Also, the prism 1105 can rotate the reflecting surface 1107 of the light reflecting substance about the central axis 1106 in the A direction or rotate the central axis 1106 in the B direction to change the path of the light L incident in the first direction X to the perpendicular second direction Y. At this time, the OPFE 1110 can also move in the third direction Z perpendicular to the first direction X and the second direction Y.

[0101] In some embodiments, as shown in the figure, the maximum rotation angle of the prism 1105 in the A direction is 15° or less in the plus (+) A direction and greater than 15° in the minus (-) A direction, but this embodiment is not limited thereto.

[0102] In some embodiments, the prism 1105 can move between approximately 20°, or between 10° and 20°, or between 15° and 20° in the plus (+) B direction or the minus (-) B direction, where the moving angle can move at the same angle in the plus (+) B direction or the minus (-) B direction or move to approximately similar angles within a range of about 1°.

[0103] In some embodiments, the prism 1105 can move the reflecting surface 1106 of the light reflecting substance in a direction parallel to the extension direction of the central axis 1106, for example, in the third direction Z.

[0104] The OPFE 1110 may include, for example, an optical lens composed of m (where m is a natural number) groups. The m lenses can move in the second direction Y to change the optical zoom ratio of the camera module 1100b. For example, when the basic optical zoom ratio of the camera module 1100b is z, when the m optical lenses included in the OPFE 1110 are moved, the optical zoom ratio of the camera module 1100b is also changed to an optical zoom ratio of 3z or 5z, or 5z or more.

[0105] The actuator 1130 can move the OPFE 1110 or the optical lens (hereinafter referred to as "optical lens") to a specific position. For example, the actuator 1130 can adjust the position of the optical lens so that the image sensor 1142 is located at the focal length of the optical lens for accurate sensing.

[0106] The image sensing device 1140 may include an image sensor 1142, control logic 1144, and a memory 1146. The image sensor 1142 can utilize the light L provided through the optical lens to sense an image of the sensing target. The image sensor 100 (FIG. 1) that performs the binning method according to the exemplary embodiment of the present disclosure described above is also applicable as the image sensor 1142. Therefore, when the image sensing device 1140 operates in the first mode, the frame rate is increased, the size of the generated image data is reduced, and the image quality is improved. For example, the frame rate is also a frame rate achievable based on the circuit interface bandwidth and the calculation bandwidth. The image sensor 100 (which provides exemplary details of the image sensor 1142) can allow a high frame rate.

[0107] The control logic 1144 can control the overall operation of the camera module 1100b. For example, the control logic 1144 can control the operation of the camera module 1100b by a control signal provided via the control signal line CSLb.

[0108] Memory 1146 can store information necessary for the operation of camera module 1100b, such as calibration data 1147. Calibration data 1147 may include information necessary for camera module 1100b to generate image data using externally provided light L. Calibration data 1147 may include, for example, information related to the aforementioned degree of rotation, information related to the focal length, information related to the optical axis, and the like. When camera module 1100b is implemented in a multi-state camera form in which the focal length changes depending on the position of the optical lens, calibration data 1147 may include focal length values for each position (or state) of the optical lens and information related to auto focusing.

[0109] Storage unit 1150 can store image data sensed via image sensor 1142. Storage unit 1150 is disposed outside image sensing device 1140 and is also implemented in a stacked form with the sensor chip constituting image sensing device 1140. In some embodiments, storage unit 1150 is also implemented by an EEPROM (electrically erasable programmable read only memory), but the present embodiment is not limited thereto.

[0110] Referring to both FIGS. 13 and 14, in some embodiments, each of the plurality of camera modules 1100a, 1100b, 1100c may include an actuator 1130. Thereby, each of the plurality of camera modules 1100a, 1100b, 1100c may include calibration data 1147 that is the same as or different from each other due to the operation of the actuator 1130 included therein.

[0111] In some embodiments, among the plurality of camera modules 1100a, 1100b, 1100c, for example, one camera module 1100b is a folded lens-shaped camera module including the aforementioned prism 1105 and OPFE 1110. For example, the remaining camera modules 1100a, 1100c are also vertical camera modules that do not include the prism 1105 and OPFE 1110, but the present embodiment is not limited thereto.

[0112] In some embodiments, among the plurality of camera modules 1100a, 1100b, 1100c, for example, one camera module 1100c is also a vertical depth camera that uses, for example, IR (infrared ray) to extract depth information. In that case, the application processor 1200 can merge, for example, the image data provided from such a depth camera and the image data provided from a different camera module 1100a or 1100b to generate a three-dimensional depth image (3D depth image).

[0113] In some embodiments, among the plurality of camera modules 1100a, 1100b, 1100c, for example, at least two camera modules 1100a, 1100b can have different observation fields of view (field of view angles). In that case, for example, among the plurality of camera modules 1100a, 1100b, 1100c, for example, the optical lenses of at least two camera modules 1100a, 1100b can be different from each other, but it is not limited thereto.

[0114] Also, in some embodiments, the viewing angles of the plurality of camera modules 1100a, 1100b, 1100c may also be different from each other. In that case, the optical lenses included in each of the plurality of camera modules 1100a, 1100b, 1100c may also be different from each other, but are not limited thereto.

[0115] In some embodiments, each of the plurality of camera modules 1100a, 1100b, 1100c may be physically separated and arranged from each other. That is, instead of dividing and using the sensing area of one image sensor 1142 by the plurality of camera modules 1100a, 1100b, 1100c, independent image sensors 1142 may be arranged inside each of the plurality of camera modules 1100a, 1100b, 1100c.

[0116] Referring to FIG. 13 again, the application processor 1200 may include an image processing device 1210, a memory controller 1220, and an internal memory 1230. The application processor 1200 may be implemented separately from the plurality of camera modules 1100a, 1100b, 1100c. For example, the application processor 1200 and the plurality of camera modules 1100a, 1100b, 1100c may be implemented separately from each other as separate semiconductor chips.

[0117] The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b, 1212c, an image generator 1214, and a camera module controller 1216.

[0118] The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b, 1212c corresponding to the number of the plurality of camera modules 1100a, 1100b, 1100c.

[0119] The image data generated from each of the camera modules 1100a, 1100b, and 1100c can be provided to the corresponding sub-image processors 1212a, 1212b, and 1212c via the mutually separated image signal lines ISLa, ISLb, and ISLc. For example, the image data generated from the camera module 1100a is also provided to the sub-image processor 1212a via the image signal line ISLa, the image data generated from the camera module 1100b is also provided to the sub-image processor 1212b via the image signal line ISLb, and the image data generated from the camera module 1100c is also provided to the sub-image processor 1212c via the image signal line ISLc. Such image data transmission is also performed, for example, using a camera serial interface (CSI) based on MIPI (mobile industry processor interface). This embodiment is not limited thereto.

[0120] In some embodiments, one sub-image processor is also arranged to correspond to a plurality of camera modules. For example, the sub-image processor 1212a and the sub-image processor 1212c are not embodied separately from each other as shown in the figure, but are integrated into one sub-image processor. The image data provided from the camera module 1100a and the camera module 1100c is also provided to the integrated sub-image processor after being selected via a selection element (e.g., a multiplexer).

[0121] The image data provided to each of the sub-image processors 1212a, 1212b, 1212c is also provided to the image generator 1214. The image generator 1214 can utilize the image data provided from each of the sub-image processors 1212a, 1212b, 1212c based on generating information or a mode signal to generate an output image.

[0122] Specifically, the image generator 1214 can merge at least a part of the image data generated from the camera modules 1100a, 1100b, 1100c having different viewing angles with each other based on generating information or a mode signal to generate an output image. Also, the image generator 1214 can select any one of the image data generated from the camera modules 1100a, 1100b, 1100c having different viewing angles with each other based on generating information or a mode signal to generate an output image.

[0123] In some embodiments, the generating information may include a zoom signal (zoom signal or zoom factor). Also, in some embodiments, the mode signal is also a signal based on, for example, a mode selected by the user.

[0124] When the image generation information is a zoom signal (zoom factor) and each of the camera modules 1100a, 1100b, and 1100c has a different observation field of view (field of view angle), the image generator 1214 can perform different operations depending on the type of the zoom signal. For example, when the zoom signal is a first signal, after merging the image data output from the camera module 1100a and the image data output from the camera module 1100c, an output image can be generated using the merged image signal and the image data output from the camera module 1100b that was not used for the merging. If the zoom signal is a second signal different from the first signal, the image generator 1214 can select any one of the image data output from each of the camera modules 1100a, 1100b, and 1100c without performing such image data merging and generate an output image. However, this embodiment is not limited thereto, and if necessary, the method of processing the image data can be implemented in any modified manner.

[0125] In some embodiments, the image generator 1214 can receive a plurality of image data with different exposure times from at least one of the plurality of sub-image processors 1212a, 1212b, and 1212c, and perform HDR (high dynamic range) processing on the plurality of image data to generate merged image data with an increased dynamic range.

[0126] The camera module controller 1216 can provide control signals to each of the camera modules 1100a, 1100b, and 1100c. The control signals generated from the camera module controller 1216 are also provided to the corresponding camera modules 1100a, 1100b, and 1100c via the separated control signal lines CSLa, CSLb, and CSLc.

[0127] Any one of the plurality of camera modules 1100a, 1100b, 1100c is also designated as, for example, a master camera 1100b by image generation information including a zoom signal or a mode signal. For example, the remaining camera modules 1100a, 1100c are also designated as slave cameras. Such information is included in a control signal and is also provided to the corresponding camera modules 1100a, 1100b, 1100c via separate control signal lines CSLa, CSLb, CSLc.

[0128] The camera modules operating as master and slave can be changed by a zoom factor or an operation mode signal. For example, when the field of view angle of the camera module 1100a is wider than that of the camera module 1100b and the zoom factor indicates a low zoom ratio, the camera module 1100b can operate as a master and the camera module 1100a can operate as a slave. Conversely, when the zoom factor indicates a high zoom ratio, the camera module 1100a can operate as a master and the camera module 1100b can operate as a slave.

[0129] In some embodiments, the control signals provided from the camera module controller 1216 to the respective camera modules 1100a, 1100b, 1100c may include a sync enable signal. For example, when the camera module 1100b is the master camera and the camera modules 1100a, 1100c are slave cameras, the camera module controller 1216 can transmit a sync enable signal to the camera module 1100b. The camera module 1100b provided with such a sync enable signal can generate a sync signal based on the provided sync enable signal, and provide the generated sync signal to the camera modules 1100a, 1100c via the sync signal line SSL. The camera module 1100b and the camera modules 1100a, 1100c are synchronized with such a sync signal and can transmit image data to the application processor 1200.

[0130] In some embodiments, the control signals provided from the camera module controller 1216 to the plurality of camera modules 1100a, 1100b, 1100c may include mode information by a mode signal. Based on such mode information, the plurality of camera modules 1100a, 1100b, 1100c can be related to the sensing speed and operate in a first operation mode and a second operation mode.

[0131] The plurality of camera modules 1100a, 1100b, 1100c generate an image signal at a first speed (for example, generate an image signal at a first frame rate) in the first operation mode, encode it at a second speed higher than the first speed (for example, encode an image signal at a second frame rate higher than the first frame rate), and can transmit the encoded image signal to the application processor 1200. At this time, the second speed is also 30 times or less of the first speed.

[0132] The application processor 1200 can store the received image signal, that is, the encoded image signal, in the internal memory 1230 provided therein or the external memory 1400 outside the application processor 1200, and then read and decode the encoded image signal from the internal memory 1230 or the external memory 1400, and display the image data generated based on the decoded image signal. For example, in the plurality of sub-processors 1212a, 1212b, 1212c of the image processing device 1210, the corresponding sub-processor can perform decoding, and also perform image processing on the decoded image signal.

[0133] In the second operation mode, the plurality of camera modules 1100a, 1100b, 1100c can generate an image signal (for example, generate an image signal with a third frame rate lower than the first frame rate) at a third speed lower than the first speed, and transmit the image signal to the application processor 1200. The image signal provided to the application processor 1200 is also an unencoded signal. The application processor 1200 can perform image processing on the received image signal or store the image signal in the internal memory 1230 or the external memory 1400.

[0134] The PMIC 1300 can supply power, for example, a power supply voltage, to each of the plurality of camera modules 1100a, 1100b, 1100c. For example, under the control of the application processor 1200, the PMIC 1300 can supply the first power to the camera module 1100a via the power signal line PSLa, supply the second power to the camera module 1100b via the power signal line PSLb, and supply the third power to the camera module 1100c via the power signal line PSLc.

[0135] The PMIC 1300 can generate power corresponding to each of the plurality of camera modules 1100a, 1100b, 1100c in response to the power control signal PCON from the application processor 1200, and can also adjust the power level. The power control signal PCON may include a power adjustment signal for each operation mode of the plurality of camera modules 1100a, 1100b, 1100c. For example, the operation mode includes a low power mode, and at this time, the power control signal PCON may include information on the camera module operating in the low power mode and the set power level. The power levels provided to each of the plurality of camera modules 1100a, 1100b, 1100c may be the same as each other or different from each other. Also, the power level can be dynamically changed.

[0136] As described above, exemplary embodiments have been disclosed in the drawings and the specification. In this specification, specific terms have been used to describe this embodiment, but they are used only for the purpose of explaining the technical idea of the present disclosure, and are not used to limit the meaning or the scope of the present disclosure described in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present disclosure is defined by the technical idea of the claims.

Description of Reference Numerals

[0137] 100 Image Sensor 110 Pixel Array 120 Loudspeaker 130 Analog-to-Digital Conversion (ADC) Circuit 140 Lamp Signal Generator 150 Timing Controller 160 Line Buffer 170 Processor PX, PXa Pixel

Claims

1. In a binning method of an image sensor, a pixel array is divided into a plurality of regions including a plurality of pixels (n is an integer of 2 or more) arranged in a (2n)×(2n) matrix, and the binning method includes: for each of the plurality of regions, simultaneously reading out a plurality of pixel signals in at least two rows of row units; generating first image data by performing analog-to-digital conversion on the plurality of read-out pixel signals, wherein the first image data includes a plurality of binning regions corresponding to the plurality of regions of the pixel array; for each of the plurality of binning regions, generating a first summation value based on two pixel values having the same hue within the binning region; a method including, in two binning regions adjacent in the column direction, generating a second summation value for each of the two binning regions based on the two first summation values indicating the same hue.

2. The step of reading out includes: simultaneously reading out pixel signals from at least two first pixels having a first hue arranged in the first column of the at least two rows of rows, and reading out a pixel signal from one second pixel among at least two second pixels having a second hue arranged in the second column of the at least two rows of rows, the method according to claim 1.

3. The one second pixel: is arranged relatively on the outer periphery within the region among the at least two second pixels, the method according to claim 2.

4. The step of generating the first image data includes: an analog-to-digital conversion circuit receiving a summation signal corresponding to the sum of the pixel signals of the at least two first pixels via a first column line, performing analog-to-digital conversion on the summation signal, and generating the summation signal as a pixel value corresponding to a first sampling position at the midpoint of the at least two first pixels, the method according to claim 2.

5. The step of generating the first summation value includes: respectively assigning weighting values to the two pixel values and averaging the values to which the weighting values are assigned, the method according to any one of claims 1 to 4.

6. The weighting value assigned to each of the two pixel values is set based on a second sampling position where the first summation value based on the result of the average summation is located, according to the method of claim 5.

7. The step of generating the second summation value: The method according to any one of claims 1 to 6, characterized in that weighting values are respectively assigned to the two first summation values and averaged, and the second summation value based on the result of the average summation is generated as a pixel value related to a third sampling position.

8. For each of a plurality of binning regions of the first image data, generating a third summation value based on at least two first pixel values within the binning region and first pixel values within another binning region adjacent to the binning region; The method according to any one of claims 1 to 7, further comprising the step of merging the second summation value and the third summation value for at least two first pixels.

9. The merging step: The method according to claim 8, characterized in that it is performed based on the difference between the first pixel value of the binning region and the first pixel value of the other binning region. If the difference is less than a first threshold value, the third summation value is included in the output image data. If the difference exceeds a second threshold value, the second summation value is included in the output image data. If the difference is greater than or equal to the first threshold value and less than or equal to the second threshold value, weighting values based on the difference are added to each of the second summation value and the third summation value, and the summation value of the values with the weighting values added is included in the output image data.

10. A pixel array divided into a plurality of rectangular regions including pixels arranged in a (2n)×(2n) matrix (n is an integer of 2 or more); An analog-to-digital conversion circuit that reads out a plurality of pixel signals received from the pixel array via a plurality of column lines and converts the read-out plurality of pixel signals into a plurality of pixel values; A row driver that provides a control signal via a plurality of row lines connected to the pixel array and generates the control signal for controlling such that a plurality of pixel signals are simultaneously output from the pixel array in units of at least two rows. A line buffer that stores the first image data output from the analog-digital conversion circuit in a predetermined row unit, A processor that performs binning on the first image data stored in the line buffer, The processor, For each of a plurality of binning regions included in the first image data, based on two pixel values having the same hue within the binning region, generates a first sum value, An image sensor that, in two binning regions adjacent in the column direction among the plurality of binning regions, based on the two first sum values indicating the same hue, generates a second sum value for each of the two binning regions.

11. The row driver, The image sensor according to claim 10, characterized in that at least two first pixels arranged in the first column of the at least two rows of rows output pixel signals simultaneously, and one of the second pixels arranged in the second column of the at least two rows of rows outputs a pixel signal.

12. The analog-digital conversion circuit, Receives a sum signal corresponding to the sum of the pixel signals of the at least two first pixels via a first column line, performs analog-digital conversion on the sum signal, and generates it as a pixel value related to a first sampling position corresponding to an intermediate point of the at least two first pixels. The image sensor according to claim 11.

13. The processor, For the two pixel values, assigns a weighting value set based on a second sampling position where the first sum value is located, and performs average summation on the values to which the weighting value is assigned. The image sensor according to claim 10.

14. The processor, For each of the plurality of binning regions of the first image data, calculates a third sum value based on at least two first pixel values within the binning region and first pixel values within other binning regions adjacent to the binning region, and merges the second sum value and the third sum value for at least two first pixels. The image sensor according to claim 10.

15. Each of the plurality of binning regions, The image sensor according to any one of claims 10 to 14, characterized by including a Bayer pattern in which red pixels, two green pixels, and blue pixels are repeatedly arranged.

16. An image sensor that senses an optical signal to generate image data, A processor that receives and processes the image data from the image sensor, and includes: The image sensor is A pixel array divided into a plurality of rectangular regions including pixels arranged in a 4X4 matrix, An analog-to-digital conversion circuit that reads out a plurality of pixel signals received from the pixel array via a plurality of column lines, and converts the read-out plurality of pixel signals into a plurality of pixel values, A row driver that provides a control signal via a plurality of row lines connected to the pixel array, and generates the control signal for controlling the pixel array to output a plurality of pixel signals simultaneously in at least two rows of row units, A line buffer that stores the first image data output from the analog-to-digital conversion circuit in a predetermined row unit, And a processor that performs binning on the first image data stored in the line buffer. The processor is For each of a plurality of binning regions of the first image data, based on two pixel values having the same hue within the binning region, a first combined value is generated, and in two binning regions adjacent in the column direction among the plurality of binning regions, based on the two first combined values indicating the same hue, for each of the two binning regions, a second combined value is generated. An image processing system.

17. The processor calculates a third combined value based on at least two first pixel values within the binning region and first pixel values within other binning regions adjacent to the binning region for each of a plurality of binning regions of the first image data, and combines the second combined value and the third combined value for at least two first pixels. The image processing system according to claim 16, characterized by this.

Images