Image sensing system

Abstract

To more effectively compensate for a difference of photosensitivity for each color of a color filter.SOLUTION: An image sensing system 10 includes: an image sensor 100 which includes a plurality of unit pixels that converts an optical signal to a subject into an electric signal and is configured to generate and output raw image data from pixel signals output from the plurality of unit pixels; a sensing controller 200 which is configured to control, on the basis of a sensing parameter, a sensing condition of the image sensor 100 for each color of the unit pixels; an additional unit 300 which is configured to add the sensing parameter to the raw image data output from the image sensor 100 and output the data with the parameter; and a sensing parameter calculator 400 which is configured to calculate a distribution of pixel values for each color of the unit pixels by using the data with the parameter output from the additional unit 300, calculate a new sensing parameter on the basis of the distribution of pixel values, and provide the new sensing parameter to the sensing controller 200 and the additional unit 300.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image sensing system capable of compensating for differences in photosensitivity for different colors of a color filter.

Background Art

[0002] An image sensor is an element that converts an optical image into an electrical signal. Recently, due to the development of the computer industry and the communication industry, there has been an increasing demand for image sensors with improved integration and performance in various fields such as digital cameras, camcorders, PCS (Personal Communication System), video game devices, security cameras, medical micro cameras, and robots.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Embodiments of the present invention aim to provide an image sensing system capable of more effectively compensating for differences in photosensitivity for different colors of a color filter.

[0004] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned should be clearly understood by those skilled in the art from the following description.

Means for Solving the Problems

[0005] An image sensing system according to one embodiment of the present invention may include an image sensor that includes a plurality of unit pixels that convert optical signals to a subject into electrical signals, and generates and outputs raw image data from pixel signals output from the plurality of unit pixels; a sensing control unit that controls the sensing conditions of the image sensor according to the color of the unit pixels based on sensing parameters; an addition unit that adds the sensing parameters to the raw image data output from the image sensor and outputs it; and a sensing parameter calculation unit that uses the parameter addition data output from the addition unit to calculate the distribution of pixel values ​​according to the color of the unit pixels, calculates new sensing parameters based on the distribution of pixel values, and provides them to the sensing control unit and the addition unit. [Effects of the Invention]

[0006] The image sensing system according to an embodiment of the present invention can obtain images with less noise by more effectively compensating for the differences in light sensitivity of different colors of the color filter. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram schematically showing the configuration of an image sensing system according to one embodiment of the present invention. [Figure 2] This is a block diagram that shows the structure of the image sensor in Figure 1 in more detail. [Figure 3a] Figure 2 is a circuit diagram illustrating an exemplary structure of a unit pixel formed in the pixel array shown in Figure 2. [Figure 3b] Figure 2 is a circuit diagram illustrating an exemplary structure of a unit pixel formed in the pixel array shown in Figure 2. [Figure 4] This is a flowchart illustrating the operation of the image sensing system shown in Figure 1. [Figure 5] Figure 1 illustrates how the sensing parameter calculation unit calculates a histogram for red color pixel values. [Modes for carrying out the invention]

[0008] Hereinafter, some embodiments of the present invention will be described in detail with reference to illustrative figures. When assigning reference numerals to the components in each figure, it should be noted that, as far as possible, identical components will have the same reference numeral, even if they are shown in other figures. Furthermore, in the description of embodiments of the present invention, if it is determined that a specific description of a related known configuration or function would hinder the understanding of the embodiments of the present invention, such a detailed description will be omitted.

[0009] Figure 1 is a block diagram schematically showing the configuration of an image sensing system according to one embodiment of the present invention.

[0010] The image sensing device 10 may include an image sensor 100, a sensing control unit 200, an add-on unit 300, a sensing parameter calculation unit 400, an image processor 500, a display unit 600, and a storage unit 700.

[0011] The image sensor 100 can capture a subject and generate raw image data (RIMD) by converting the optical signals of the captured subject into electrical signals. For example, the image sensor 100 may include multiple unit pixels that convert the optical signals obtained from capturing a subject into electrical signals (pixel signals), and generate raw image data (RIMD) by converting the pixel signals output from the multiple unit pixels into digital signals using an analog-to-digital converter (ADC). In such an image sensor 100, the sensing conditions for capturing a subject may change based on the control of the sensing control unit 200.

[0012] The unit pixels of the image sensor 100 may include color filters for filtering light of a specific color from incident light. In this case, the photosensitivity of the color filters may differ from one another depending on the color. For example, a unit pixel may include an RGB color filter. However, since the green (G) filter has a higher photosensitivity than the red (R) and blue (B) filters, the green (G) unit pixels may saturate faster than the red (R) and blue (B) unit pixels. For this reason, if the exposure time is set so that the green (G) unit pixels do not saturate, the amount of light in the red (R) and blue (B) unit pixels is low, which may cause a problem in which the signal-to-noise ratio (SNR) characteristics of the pixel signals deteriorate. Therefore, the image sensor 100 can compensate for the difference in photosensitivity of the color filters by adjusting at least one of the exposure time, conversion gain, and analog gain based on the control of the sensing control unit 200 to capture the subject.

[0013] The sensing control unit 200 can control at least one of the exposure time, conversion gain, and analog gain of the image sensor 100 for each color of the unit pixel based on the sensing parameters (SP). For example, the sensing parameters (SP) may include information on the exposure time, conversion gain, and analog gain for each color of the color filter for the unit pixel, and the sensing control unit 200 can receive the sensing parameters (SP) from the sensing parameter calculation unit 400. When the sensing control unit 200 receives new sensing parameters (SP) from the sensing parameter calculation unit 400, it will again control at least one of the exposure time, conversion gain, and analog gain of the image sensor 100 based on the new sensing parameters (SP). The sensing control unit 200 can adjust the exposure time and conversion gain by controlling the generation of the transmission signal and the conversion gain control signal for the unit pixels of the image sensor 100 based on the sensing parameters (SP). Furthermore, the sensing control unit 200 can adjust the analog gain by controlling the amplitude of the ramp signal for analog-to-digital conversion at the image sensor 100 based on the sensing parameter (SP).

[0014] The add-on unit 300 can receive sensing parameters (SP) from the sensing parameter calculation unit 400 and raw image data (RIMD) from the image sensor 100. The add-on unit 300 can add the sensing parameters (SP) received from the sensing parameter calculation unit 400 to the raw image data (RIMD) output from the image sensor 100 and output it. That is, the add-on unit 300 can add sensing parameters (SP) to the raw image data (RIMD) output from the image sensor 100 to indicate under what sensing conditions the data was captured. Hereinafter, the image data with sensing parameters (SP) added to the raw image data (RIMD) will be referred to as parameter-added data (RIMD+SP). The parameter-added data (RIMD+SP) can be output to the sensing parameter calculation unit 400, the image processor 500, and the storage unit 700.

[0015] The sensing parameter calculation unit 400 can analyze the parameter addition data (RIMD+SP) provided by the addition unit 300 and generate sensing parameters (SP) for correcting the color-specific light sensitivity of unit pixels. For example, the sensing parameter calculation unit 400 can separate the parameter addition data (RIMD+SP) provided by the addition unit 300 into original image data (RIMD) and sensing parameters (SP), and can classify the original image data (RIMD) by color. That is, the sensing parameter calculation unit 400 can classify the pixel values ​​for multiple unit pixels by color. Using the pixel values ​​classified by color, the sensing parameter calculation unit 400 can calculate a histogram representing the distribution state of the color-specific pixel values, and can find pixel values ​​corresponding to already set reference values ​​(reference pixel values) from the histogram. For example, the sensing parameter calculation unit 400 can find the pixel values ​​corresponding to the top 10% in the histogram of each color as the reference pixel values ​​for that color.

[0016] The sensing parameter calculation unit 400 can calculate new sensing parameters for each color using the reference pixel value for each color, the current sensing parameter (SP), and the already set maximum pixel value. For example, the sensing parameter calculation unit 400 can determine the exposure time, conversion gain, and analog gain for each color unit pixel using the reference pixel value for each color, the current gain of the image sensor 100 obtained using the current sensing parameter (SP), and the already set maximum pixel value. When determining the values ​​of exposure time, conversion gain, and analog gain, the sensing parameter calculation unit 400 can determine the values ​​in the order of conversion gain, analog gain, and exposure time. The specific method by which the sensing parameter calculation unit 400 calculates the new sensing parameters will be described later.

[0017] The sensing parameter calculation unit 400 may include a change determination unit (not shown) that compares previous raw image data with the current raw image data to determine whether or not there is a change (movement) in the subject. If the sensing parameter calculation unit 400 determines that there is a change in the subject, it may choose not to control the exposure time when calculating new sensing parameters. Alternatively, the sensing parameter calculation unit 400 may make the difference in exposure time for each color when there is no change in the subject shorter than the difference in exposure time for each color when there is a change in the subject.

[0018] The image processor 500 can receive the parameter-added data (RIMD+SP) from the addition unit 300, separate the original image data (RAW IMD) and the sensing parameter (SP), and generate an image (IMG) by performing image processing on the original image data (RIMD) based on the sensing parameter (SP). That is, the image processor 500 can use the sensing parameter (SP) to determine under what sensing conditions the current original image data (RIMD) was captured, and generate an image (IMG) of the subject by performing image processing on the original image data (RIMD) so as to be suitable for the sensing conditions.

[0019] The display unit 600 can output the image (IMG) generated by the image processor 500 to the screen.

[0020] The storage unit 700 can store the parameter-added data (RIMD+SP) generated by the addition unit 300 and the image (IMG) generated by the image processor 500.

[0021] FIG. 2 is a block diagram more specifically showing the structure of the image sensor 100 in FIG. 1.

[0022] As shown in FIG. 2, the image sensor 100 can include a pixel array 110, a row driver 120, a correlated double sampler (CDS) 130, an analog-to-digital converter (ADC) 140, an output buffer 150, a column driver 160, and a timing controller 170. Here, each component of the image sensing device 100 is merely exemplary, and at least some components may be added or omitted as necessary.

[0023] The pixel array 110 can include a plurality of unit pixels arranged in a plurality of rows and a plurality of columns. In one embodiment, the plurality of unit pixels may be arranged in a two-dimensional pixel array including a plurality of rows and a plurality of columns. In other embodiments, the plurality of unit pixels may be arranged in a three-dimensional pixel array. The plurality of unit pixels can convert an optical signal in units of unit pixels or pixel groups to generate an electrical signal, and the unit pixels within a pixel group can share at least specific internal circuits. The plurality of unit pixels can each include a light sensing element. For example, the light sensing element can include a photo diode, a phototransistor, a photogate, or a pinned photodiode, etc.

[0024] The pixel array 110 can receive from the load driver 120 the provision of drive signals such as a row selection signal, a reset signal, a transmission signal, and a conversion gain control signal. The unit pixel is activated when the drive signal is received and can operate in response to the drive signal. At this time, the transmission signal and the conversion gain control signal change based on the control of the sensing control unit 200, so that the exposure time and the conversion gain of the unit pixel can be adjusted to correspond to the color of the unit pixel (the color of the color filter).

[0025] The low driver 120 can operate unit pixels based on control signals provided by a control circuit such as a timing controller 170. The low driver 120 can select at least one unit pixel connected to at least one low line of the pixel array 110. The low driver 120 can generate a low selection signal to select at least one low line from a plurality of low lines. The low driver 120 can enable a conversion gain control signal, a reset signal, and a transmission signal for the unit pixels of the selected low line. In this case, the on / off timing of the conversion gain control signal and the timing of the transmission signal may be controlled separately for each color of the unit pixel based on the control of the timing controller 170. The pixel signals generated by the unit pixels of the selected low line may be output to a correlated duplex sampler 130.

[0026] The correlated double sampler 130 can remove unwanted offset values ​​from unit pixels using the correlated double sampling (CDS) method. For example, the correlated double sampler 130 can remove unwanted offset values ​​from unit pixels by comparing the output voltage of a unit pixel obtained before and after the photocharge generated by incident light is accumulated in the sensing node (floating diffusion node). Through this, a pixel signal free of noise components and generated only by incident light can be obtained. Based on a clock signal provided by the timing controller 170, the correlated double sampler 130 can sequentially sample and hold the voltage level of a reference signal and the voltage level of a pixel signal received from the pixel array 110 via multiple column lines. The correlated double sampler 130 can output the reference signal and the pixel signal as correlated double sampling (CDS) signals to the analog-to-digital converter 140.

[0027] The analog-to-digital converter 140 can convert the CDS signal received from the correlated duplex sampler 130 into a digital signal. The analog-to-digital converter 140 may include a ramp-comparison type analog-to-digital converter. The analog-to-digital converter 140 can compare the ramp signal provided by the timing controller 170 with the CDS signal provided by the correlated duplex sampler 130 to generate a comparison signal. The analog-to-digital converter 140 can count the level transition time of the comparison signal based on the ramp signal provided by the timing controller 170 and output the count value to the output buffer 150.

[0028] The output buffer 150 can temporarily store the data for each column provided by the analog-to-digital converter 140, based on the control of the timing controller 170. The output buffer 150 can also act as an interface to compensate for differences in transmission (or processing) speed between the image sensor 100 and the connected add-on 300.

[0029] The column driver 160 can select a column in the output buffer 150 based on the control of the timing controller 170 and sequentially output the data temporarily stored in the selected column of the output buffer 150. When the column driver 160 receives an address signal from the timing controller 170, it can generate a column selection signal based on that address signal and select a column in the output buffer 150, thereby controlling the output of data from the selected column of the output buffer 150 as raw image data (RIMD).

[0030] The timing controller 170 can generate signals to control the operation of the low driver 120, analog-to-digital converter 140, output buffer 150, and column driver 160. The timing controller 170 can provide the low decoder 120, column driver 160, analog-to-digital converter 140, and output buffer 150 with clock signals required for the operation of each component of the image sensing device, control signals for timing control, and address signals for selecting low or column. Based on the control of the sensing control unit 200, the timing controller 170 can adjust the timing of the transmission signal, the on / off status of the conversion gain control signal, and the amplitude of the ramp signal on a color-by-color basis for each unit pixel. Depending on the embodiment, the timing controller 170 may include a logic control circuit, a phase lock loop (PLL) circuit, a timing control circuit, and a communication interface circuit.

[0031] Figures 3a and 3b are circuit diagrams illustrating the exemplary structure of a unit pixel formed in the pixel array shown in Figure 2.

[0032] As shown in Figure 3a, the unit pixels of the pixel array 110 may include a photoelectric converter (PD), a transfer transistor (TX), a reset transistor (RX), a drive transistor (DX), a selection transistor (SX), a conversion gain transistor (CGX), and a conversion gain capacitor (CAP).

[0033] A photoelectric converter (PD) can generate and store photocharges corresponding to incident light. For example, a photoelectric converter (PD) may be a photodiode, phototransistor, photogate, pinned photodiode (PPD), or a combination thereof.

[0034] A transfer transistor (TX) can transmit photocharges accumulated in a photoelectric converter (PD) to a floating diffusion node (FD) based on a transmission signal (TG).

[0035] A floating diffusion node (FD) receives and stores photocharges generated by a photoelectric converter (PD). The amount of photocharge stored in the floating diffusion node (FD) can control the drive transistor (DX).

[0036] A reset transistor (RX) can periodically reset the floating diffusion node (FD). When the reset signal (RG) is enabled and the reset transistor (RX) is turned on, the pixel power supply voltage (VPIX) is transmitted to the floating diffusion node (FD). This discharges any charge accumulated in the floating diffusion node (FD), and the floating diffusion node (FD) can be reset.

[0037] The drive transistor (DX) may be a source follower buffer amplifier that generates a source-drain current proportional to the amount of charge in a floating diffusion node (FD) input to its gate electrode. The drive transistor (DX) amplifies the potential change in the floating diffusion node (FD), and the amplified signal may be output to an output line (column line) (OUT) through a selection transistor (SX). The source terminal of the drive transistor (DX) may be connected to the pixel power supply voltage (VPIX), and the drain terminal of the drive transistor (DX) may be connected to the source terminal of the selection transistor (SX). In this case, the conversion gain of the drive transistor (DX) that outputs an electrical signal proportional to the amount of charge stored in the floating diffusion node (FD) may vary depending on the capacitance of the floating diffusion node (FD).

[0038] The selection transistor (SX) can select unit pixels to read out on a column-by-column basis. When the selection transistor (SX) is turned on by the selection signal (SG), the electrical signal (pixel signal) output to the drain electrode of the drive transistor (DX) can be output to the output line (OUT).

[0039] A conversion gain transistor (CGX) can be connected in series between a floating diffusion node (FD) and a conversion gain capacitor (CAP). The conversion gain transistor (CGX) can selectively connect a conversion gain capacitor (CAP) to the floating diffusion node (FD) based on a conversion gain control signal (CG1 / CG2). When the conversion gain capacitor (CAP) is connected to the floating diffusion node (FD), the capacitance of the floating diffusion node (FD) increases, thereby increasing the input impedance of the drive transistor (DX), and the unit pixel may have a low conversion gain. Conversely, when the conversion gain capacitor (CAP) is electrically isolated from the floating diffusion node (FD), the unit pixel may have a high conversion gain.

[0040] As shown in Figure 3b, the pixel array 110 may include a structure in which multiple unit pixels having the same color filter share a single floating diffusion node (FD). That is, multiple unit pixels can convert optical signals in groups to generate electrical signals.

[0041] A pixel group may include first to fourth photoelectric elements (PD1, PD2, PD3, PD4) and first to fourth transfer transistors (TX1, TX2, TX3, TX4). The first to fourth photoelectric elements (PD1, PD2, PD3, PD4) may share a single floating diffusion node (FD). The first to fourth photoelectric elements (PD1, PD2, PD3, PD4) may be coupled to the first to fourth transfer transistors (TX1, TX2, TX3, TX4), respectively. For example, the first to fourth photoelectric elements (PD1, PD2, PD3, PD4) may be arranged adjacent to each other in a 2x2 structure.

[0042] The first to fourth transfer transistors (TX1, TX2, TX3, TX4) can each transmit the photocharge accumulated in the first to fourth photoelectric conversion elements (PD1, PD2, PD3, PD4) to the floating diffusion node (FD) based on the first to fourth transmission signals (TG1, TG2, TG3, TG4).

[0043] The reset transistor (RX), drive transistor (DX), selection transistor (SX), conversion gain transistor (CGX), and conversion gain capacitor (CAP) may have the same function as the transistors and capacitors shown in Figure 3a.

[0044] Figure 4 is a flowchart illustrating the operation of the image sensing device shown in Figure 1. Figure 5 is an illustrative diagram showing how the sensing parameter calculation unit in Figure 1 calculates a histogram for red-colored pixel values.

[0045] When the image sensing device 10 starts operating (for example, when the power is turned on), the sensing parameter calculation unit 400 can initialize the sensing parameters (SP) to the initial values ​​that have already been set (S100).

[0046] The sensing parameters (SP) may include information on the exposure time, conversion gain, and analog gain for each color of the image sensor 100, and the sensing parameter calculation unit 400 can initialize the exposure time, conversion gain, and analog gain for each color of the unit pixel. For example, the sensing parameters (SP) may be set so that the conversion gain, analog gain, and exposure time for green color unit pixels are all 1.0x, while for red and blue color unit pixels, the conversion gain is 1.0x, the analog gain is 1.4x, and the exposure time is 1.0x. That is, the gain for green color unit pixels may be initialized to 1.0, and the gains for red and blue color unit pixels may be initialized to 1.4.

[0047] The sensing parameter calculation unit 400 can output the initialized sensing parameters (SP) to the sensing control unit 200 and the add-on unit 300.

[0048] The sensing control unit 200 can initialize the exposure time, conversion gain, and analog gain of the image sensor 100 for each unit pixel color by controlling the timing controller 170 of the image sensor 100 based on the sensing parameters (SP) provided by the sensing parameter calculation unit 400 (S200).

[0049] For example, the timing controller 170 of the image sensor 100 can adjust the timing of the transmission signal (TG) to the unit pixel, the on / off state of the conversion gain control signals (CG1 / CG2), and the amplitude of the ramp signal based on the control of the sensing control unit 200. For example, the timing controller 170 can control the conversion gain to be 1.0 by controlling all the conversion gain transistors (CGX) of the unit pixels to be either on or off based on the control of the sensing control unit 200, and can also control the exposure time to be 1.0 by controlling the exposure time of all unit pixels in the same way. Furthermore, the timing controller 170 can control the amplitude of the ramp signal so that the analog gain for red and blue unit pixels is 1.4 times the analog gain for green unit pixels.

[0050] The image sensor 100 can generate raw image data (RIMD) by capturing images of a subject according to the sensing conditions initialized by the sensing control unit 200 (S300).

[0051] The image sensor 100 can generate and output original image data (RIMD) of a captured subject by converting an optical signal into an electrical signal, which is a pixel signal, and then converting the pixel signal into a digital signal.

[0052] The addition unit 300 can add sensing parameters (SP) provided by the sensing parameter calculation unit 400 to the original image data (RIMD) output from the image sensor 100, and output parameter-added data (RIMD+SP) (S400).

[0053] In other words, the add-on unit 300 can add sensing parameters (SP) to the original image data (RIMD) output from the image sensor 100, indicating the sensing conditions under which the RIMD was captured.

[0054] The parameter-added data (RIMD+SP) output from the addition unit 300 can be transmitted to the sensing parameter calculation unit 400 and the image processor 500. The parameter-added data (RIMD+SP) can also be transmitted to and stored in the storage unit 700.

[0055] The sensing parameter calculation unit 400 can calculate new sensing parameters (SP) based on the original image data (RIMD) included in the parameter-added data (RIMD+SP) and the current sensing parameters (SP) (S500).

[0056] To this end, the sensing parameter calculation unit 400 first determines a new gain for the image sensor 100 using the original image data (RIMD) and the current sensing parameters (SP), and then determines the exposure time, conversion gain, and analog gain necessary to achieve this gain.

[0057] First, in order to determine a new gain for the image sensor 100, the sensing parameter calculation unit 400 divides the original image data (RIMD) by color and then calculates a histogram for each pixel value for each color, as shown in Figure 5. The sensing parameter calculation unit 400 can also find a pixel value that corresponds to a previously set reference value (reference pixel value) from the calculated histogram for each color. For example, the sensing parameter calculation unit 400 can set the pixel values ​​in the top 10% of the histogram as the reference pixel values.

[0058] Next, the sensing parameter calculation unit 400 can calculate a new gain for each color using the reference pixel value for each color found via the original image data (RIMD) and the already set maximum pixel value. For example, the sensing parameter calculation unit 400 can find the Y value at which the reference pixel value is less than 1 / Y of the maximum pixel value, and determine the new gain as the largest gain among the configurable gains of the image sensor 100 that does not exceed Y times the current gain.

[0059] For example, the configurable gains for the image sensor 100 are 1.4, 2.0, 2.8, 4.0, 5.6, and as shown in Figure 5, we assume that the maximum pixel value for a red unit pixel is 1023. Since each unit pixel has a photoelectric converter (PD) of the same size, all unit pixels can have the same maximum pixel value regardless of color.

[0060] The sensing parameter calculation unit 400 first separates the pixel values ​​for red color unit pixels from the original image data (RIMD) included in the parameter-added data (RIMD+SP), then calculates a histogram for those pixel values ​​as shown in Figure 5, and from the calculated histogram, it can be seen that the reference pixel value for red color is 405. Furthermore, the sensing parameter calculation unit 400 can see that the current gain is 1.4 from the sensing parameters (SP) extracted from the parameter-added data (RIMD+SP).

[0061] Through this, the sensing parameter calculation unit 400 can determine the Y value at which the reference pixel value (405) is less than 1 / Y of the maximum pixel value (1023). In other words, the sensing parameter calculation unit 400 can determine that the Y value must be less than 1023 / 405 (≒2.53). Once the range of Y values ​​is determined, the sensing parameter calculation unit 400 can determine the range of values ​​that are Y times the current gain (1.4) (<1.4 × 2.53 ≒3.52).

[0062] The sensing parameter calculation unit 400 compares the range of values ​​that are Y times the current gain (1.4) with the configurable gains of the image sensor 100, and can find the largest gain among the configurable gains that does not exceed Y times the current gain (1.4). In this embodiment, the configurable gains of the image sensor 100 are... 1.4, 2.0, 2.8, 4.0, 5.6..., so the largest value among the values ​​that do not exceed 3.52 is 2.8. Therefore, the sensing parameter calculation unit 400 can determine the new gain to be 2.8.

[0063] In other words, if the new gain for the red color is set to 2.8, the new gain (2.8) will be twice the current gain (1.4). Therefore, the top 10% reference pixel value for the pixel values ​​expected to be obtained with the new gain will be 810, which is twice the top 10% reference pixel value for the pixel values ​​obtained with the current gain (405). Since the reference pixel value of 810 in this case does not exceed the maximum pixel value of 1023, a gain of 2.8 can be used as the new gain. The gains that can be set for the image sensor 100 are... 1.4, 2.0, 2.8, 4.0, 5.6... so if a gain one step higher than 2.8 (4.0) is set as the new gain, the new gain (4.0) will be approximately 2.86 times the current gain (1.4). The top 10% of the reference pixel values ​​expected to be obtained with a gain of 4.0 are expected to be 1158.3 (405 × 2.86). Since such reference pixel values ​​exceed the maximum pixel value (1023), they are not suitable as a new gain. Therefore, the new gain for red color can be determined to be 2.8.

[0064] In the embodiment described above, the reference value for determining the reference pixel value was set to the top 10%, but the reference value is adjustable. For example, if you want to reduce the number of saturated pixels, you can set the reference value higher than the top 10%.

[0065] Once the new gain is determined, the sensing parameter calculation unit 400 can determine the conversion gain, analog gain, and exposure time necessary to achieve this new gain. In other words, since the gain of the image sensor 100 is achieved through a combination of the conversion gain, analog gain, and exposure time, the sensing parameter calculation unit 400 can determine the conversion gain, analog gain, and exposure time necessary to achieve the new gain.

[0066] Generally, with image sensors, analog gain can be set in 13 or 19 steps from 1x to 64x, while conversion gain is limited to only two steps: 1x and 2x. Furthermore, while exposure time can be precisely adjusted, similar to analog gain, large differences in exposure time between colors can cause motion artifacts, so these differences must be kept within a predetermined range.

[0067] Due to these constraints, the sensing parameter calculation unit 400 can determine the values ​​in the order of conversion gain, analog gain, and exposure time. For example, as mentioned above, if the new gain is set to 2.8, the sensing parameter calculation unit 400 can set the conversion gain to 2.0 times, the analog gain to 1.4 times, and the exposure time to 1.0 times. If the gain is set to 4.0, the sensing parameter calculation unit 400 can set the conversion gain to 2.0 times, the analog gain to 2.0 times, and the exposure time to 1.0 times.

[0068] In the embodiment described above, since the new gain for the red-colored unit pixel was determined to be 2.8, the sensing parameter calculation unit 400 can calculate and output new sensing parameters (SP) for the red-colored unit pixel that result in a conversion gain of 2.0 times, an analog gain of 1.4 times, and an exposure time of 1.0 times. Although the embodiment described above only described the case for the red-colored unit pixel, the sensing parameter calculation unit 400 can calculate and output new sensing parameters for unit pixels of other colors in the same manner.

[0069] The process from S200 to S500 described above may be repeated, for example, until the power to the image sensor 100 is turned off.

[0070] As the calculation of sensing parameters is repeatedly performed, the sensing parameter calculation unit 400 can compare the previous raw image data with the current raw image data to determine whether or not there is a change (movement) in the subject. If the sensing parameter calculation unit 400 determines that there is a change in the captured subject, it can ensure that the exposure time is not controlled by the sensing control unit 200 when calculating the new sensing parameters. In other words, the sensing parameter calculation unit 400 can ensure that the exposure time is not reflected in the realization of the new gain, and only the conversion gain and analog gain are reflected. Alternatively, the sensing parameter calculation unit 400 can make the difference in exposure time for each color when there is no change in the subject shorter than the difference in exposure time for each color when there is a change in the subject.

[0071] The parameterized data (RIMD+SP) output from the add-on unit 300 in S400 can also be transmitted to the image processor 500. The image processor 500 separates the parameterized data (RIMD+SP) output from the add-on unit 300 into raw image data (RAW IMD) and sensing parameters (SP), and generates an image (IMG) by performing an image processing process on the raw image data (RIMD) based on the sensing parameters (SP) (S600).

[0072] The image processor 500 determines, through sensing parameters (SP), the current raw image data (RIMD) under what sensing conditions, i.e., what gain was applied to each color when it was captured, and then processes the raw image data (RIMD) according to those sensing conditions. The image processing method can be any one of the many conventionally used image processing methods.

[0073] The image (IMG) processed by the image processor 500 is transmitted to the display unit 600 and the storage unit 700, and can be stored in the storage unit 700 while being displayed on the screen (S700).

[0074] The above explanation is merely illustrative in describing the technical concept of the present invention, and any person with ordinary skill in the art to which the present invention belongs should be able to make various modifications and variations within the bounds of the essential characteristics of the present invention.

[0075] Therefore, the embodiments disclosed herein are for illustrative purposes only, and not to limit, the technical concept of the present invention, and such embodiments do not limit the scope of the technical concept of the present invention. The scope of protection of the present invention must be interpreted in accordance with the claims, and all technical concepts within an equivalent scope must be interpreted as being included within the scope of the rights of the present invention. [Explanation of symbols]

[0076] 100 Image Sensors 200 Sensing Control Unit 300 Add-on part 400 Sensing parameter calculation unit 500 Image Processors 600 Display 700 Storage section

Claims

1. An image sensor that includes multiple unit pixels that convert optical signals to a subject into electrical signals, and generates and outputs original image data from the pixel signals output from the multiple unit pixels, A sensing control unit that controls the sensing conditions of the image sensor according to the color of the unit pixel based on sensing parameters, An addition unit that adds the sensing parameters to the raw image data output from the image sensor and outputs it, A sensing parameter calculation unit calculates the distribution of pixel values ​​for each color of the unit pixel using the parameter addition data output from the addition unit, calculates new sensing parameters based on the distribution of pixel values, and provides these parameters to the sensing control unit and the addition unit. An image sensing system, including [the following].

2. The sensing control unit, The image sensing system according to claim 1, characterized in that at least one of the exposure time, conversion gain, and analog gain for the image sensor is controlled separately for the color of the unit pixel.

3. The sensing control unit, The image sensing system according to claim 2, characterized in that it controls the exposure time and conversion gain by controlling the generation of a transmission signal and a conversion gain control signal to the unit pixel, and controls the analog gain by controlling the amplitude of a ramp signal for analog-to-digital conversion of the image sensor.

4. The sensing parameter calculation unit is: The image sensing system according to claim 1, characterized in that the pixel values ​​of the unit pixels are separated by color from the original image data included in the parameter-added data, and a histogram is calculated for the pixel values ​​of each color.

5. The sensing parameter calculation unit is: The image sensing system according to claim 4, characterized in that it finds a reference pixel value corresponding to a previously set reference value from the histogram, and calculates a new gain for each color using the reference pixel value, the sensing parameter, and the previously set maximum pixel value.

6. The sensing parameter calculation unit is: The image sensing system according to claim 5, characterized in that the values ​​of exposure time, conversion gain and analog gain are determined so that the aforementioned new gain can be achieved.

7. The sensing parameter calculation unit is: The image sensing system according to claim 6, characterized in that when determining the values ​​of exposure time, conversion gain, and analog gain, the values ​​are determined in the order of conversion gain, analog gain, and exposure time.

8. The sensing parameter calculation unit is: The image sensing system according to claim 5, characterized in that the new gain is set to the largest gain among the configurable gains of the image sensor, to the extent that the reference pixel value does not exceed the maximum pixel value with respect to the pixel value expected to be obtained with the new gain.

9. The sensing parameter calculation unit is: The image sensing system according to claim 5, characterized in that the Y value is determined such that the reference pixel value is less than 1 / Y of the maximum pixel value, and the new gain is determined to be the largest gain among the configurable gains of the image sensor that does not exceed Y times the current gain.

10. The sensing parameter calculation unit is: The image sensing system according to claim 1, characterized in that it compares previous raw image data with current raw image data to determine whether or not there has been a change in the subject, and if there has been a change in the subject, the exposure time is not controlled when calculating the new sensing parameters.

11. The sensing parameter calculation unit is: The image sensing system according to claim 1, characterized in that it compares previous raw image data with current raw image data to determine whether or not there has been a change in the subject, and makes the difference in exposure time for each color when there has been no change in the subject shorter than the difference in exposure time for each color when there has been a change in the subject.

12. An image processor receives the parameter addition data from the addition unit, separates the original image data and the sensing parameters, processes the original image data based on the sensing parameters, and generates an image. A display unit that outputs the image output from the image processor to the screen, A storage unit that stores the parameter data and the image, The image sensing system according to claim 1, further comprising the following:

Images