Image sensing system

Abstract

The present application relates to an image sensing system. An image sensing system includes an image sensor, a sensing controller, an adding unit, and a sensing parameter calculator. The image sensor includes a plurality of unit pixels for converting an optical signal of an object into an electrical signal, generates raw image data from pixel signals output from the plurality of unit pixels, and outputs the raw image data. The sensing controller controls a sensing condition of the image sensor for each color of the unit pixels based on a sensing parameter. The adding unit adds the sensing parameter to the raw image data output from the image sensor and outputs result data corresponding to a sum of the sensing parameter and the raw image data. The sensing parameter calculator calculates a distribution of pixel values of each color of the unit pixels using data of the added parameter received from the adding unit, calculates a new sensing parameter based on the distribution of the pixel values, and transmits the calculated new sensing parameter to the sensing controller and the adding unit.

Description

Technical Field

[0001] The technology and implementation disclosed in this patent document generally relate to an image sensing system that includes pixel units having color filters corresponding to different colors. Background Technology

[0002] Image sensors are used in electronic devices to convert optical images into electrical signals. With recent advancements in the automotive, medical, computer, and communications industries, the demand for highly integrated, higher-performance image sensors is rapidly increasing in various electronic devices such as digital cameras, camcorders, personal communication systems (PCS), video game consoles, surveillance cameras, medical miniature cameras, and robots. Summary of the Invention

[0003] Various embodiments of the disclosed technology relate to an image sensing system capable of more effectively compensating for differences in photosensitivity corresponding to different colors of color filters.

[0004] According to an embodiment of the disclosed technology, an image sensing system may include: an image sensor comprising a plurality of unit pixels, each unit pixel configured to convert an optical signal of an object into a pixel signal and including a color filter to allow incident light of a specific color to pass through for generating an optical signal; the image sensor being configured to generate raw image data indicating an image of the object based on the pixel signals output from the plurality of unit pixels; a sensing controller communicatively coupled to the image sensor and configured to control sensing conditions of the image sensor for each group of unit pixels comprising one or more unit pixels corresponding to the same color based on sensing parameters; a parameter addition circuit communicatively coupled to the image sensor and configured to obtain data of added parameters by adding the sensing parameters to the raw image data output from the image sensor; and a sensing parameter calculator communicatively coupled to the sensing controller and the parameter addition circuit and configured to calculate the distribution of pixel values ​​of each group of unit pixels based on the data of added parameters received from the parameter addition circuit, calculate updated sensing parameters based on the distribution of pixel values, and send the updated sensing parameters to the sensing controller and the parameter addition circuit.

[0005] According to an embodiment of the disclosed technology, an image sensing system may include: an image sensor including a first group of unit pixels configured to detect light of a first color in incident light from an object to be imaged and a second group of unit pixels configured to detect light of a second color in incident light from the object to be imaged; a sensing controller communicatively connected to the image sensor and configured to control at least one of an exposure time, a conversion gain, or an analog gain for each of the first group of unit pixels and the second group of unit pixels; and a sensing parameter calculator communicatively connected to the sensing controller and configured to calculate a first sensing parameter for the first group of unit pixels and a second sensing parameter for the second group of unit pixels, respectively, and to provide the first sensing parameter and the second sensing parameter to the sensing controller such that at least one of an exposure time, a conversion gain, or an analog gain for each of the first group of unit pixels and the second group of unit pixels is based on the sensing parameters, wherein the first sensing parameter and the second sensing parameter are different from each other.

[0006] It should be understood that the foregoing general description of the disclosed technology and the following detailed description are both illustrative and explanatory, and are intended to provide a further explanation of the claimed disclosure. Attached Figure Description

[0007] The above and other features and advantages of the disclosed technology will become apparent when considered in conjunction with the accompanying drawings and with reference to the following detailed description.

[0008] Figure 1 This is a block diagram illustrating examples of image sensing systems based on some implementations of the disclosed techniques.

[0009] Figure 2 These are examples of some implementations based on the disclosed technology. Figure 1 A block diagram of an exemplary structure of an image sensor is shown.

[0010] Figure 3A and Figure 3B This illustrates the formation of some implementations based on the disclosed technology. Figure 2 A circuit diagram of an exemplary structure of a unit pixel in the pixel array shown.

[0011] Figure 4 These are examples of some implementations based on the disclosed technology for operation. Figure 1 The flowchart shows an example of a method for an image sensing system.

[0012] Figure 5 These are examples of some implementations based on the disclosed technology for use via Figure 1 The graph shown is a curve illustrating how the sensor parameter calculator calculates the histogram of red pixel values. Detailed Implementation

[0013] This patent document provides implementations and examples of image sensing systems, and demonstrates that the disclosed features can be implemented to achieve one or more advantages in a wider range of applications. Some implementations of the disclosed techniques propose designs for image sensing systems capable of effectively compensating for sensitivity differences between color filters corresponding to different colors. The disclosed techniques provide various implementations of image sensing systems capable of obtaining images with less noise by more effectively compensating for sensitivity differences in each color of the color filters.

[0014] Reference will now be made in detail to certain embodiments, examples of which are shown in the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. Embodiments of the disclosed technology can provide insights into various effects, either directly or indirectly, through the disclosed technology.

[0015] Figure 1 This is a block diagram illustrating some implementations of image sensing systems based on the disclosed techniques.

[0016] Reference Figure 1 The image sensing system 10 may include an image sensor 100, a sensing controller 200, an adder unit 300, a sensing parameter calculator 400, an image processor 500, a display 600, and a storage device 700.

[0017] Image sensor 100 can capture images of a target scene or object (e.g., a target object to be captured) and can convert the optical signals of the captured object into electrical signals (which may be referred to as pixel signals), resulting in raw image data RIMD. For example, image sensor 100 may include multiple unit pixels, each of which converts the optical signals obtained through the captured object into electrical signals (i.e., pixel signals), and the pixel signals output from the multiple unit pixels can be converted into digital signals using an analog-to-digital converter (ADC), resulting in raw image data RIMD. Image sensor 100 can be configured such that the sensing conditions required for capturing the object can vary according to control signals received from sensing controller 200.

[0018] Each pixel of the image sensor 100 may include a color filter configured to filter light of a specific color from incident light to reach the photosensitive element in each pixel, while blocking light of other colors from being detected. The color filter may be configured to have different sensitivities depending on the corresponding color. For example, a pixel may include an RGB color filter. Since the green color filter (G) has a higher sensitivity than the red color filter (R) and the blue color filter (B), a pixel with the green color filter (G) can saturate faster than a pixel with the red color filter (R) and a pixel with the blue color filter (B). If the exposure time is set to a value that prevents saturation of pixels with the green color filter (G), the amount of light in pixels corresponding to the red color filter (R) and the blue color filter (B), respectively, becomes smaller, which degrades the signal-to-noise ratio (SNR) characteristics. The disclosed implementation of the technology provides an image sensor 100 that captures a target object by adjusting at least one of exposure time, conversion gain, and analog gain based on the control of a sensing controller 200, thereby compensating for the sensitivity differences of color filters corresponding to different colors.

[0019] Sensing controller 200 can control at least one of the exposure time, conversion gain, or analog gain of image sensor 100 for each color per unit pixel based on sensing parameters (SP). In some implementations, sensing controller 200 can control at least two of the exposure time, conversion gain, or analog gain of image sensor 100 for each color per unit pixel based on sensing parameters (SP). For example, sensing parameters (SP) may include information about the exposure time, conversion gain, and analog gain for each color of a color filter per unit pixel, and sensing controller 200 can receive sensing parameters (SP) from sensing parameter calculator 400. Upon receiving new sensing parameters (SP) from sensing parameter calculator 400, sensing controller 200 can again control at least one of the exposure time, conversion gain, and analog gain of the image sensor based on the new sensing parameters (SP). The sensor controller 200 can adjust the exposure time and conversion gain based on sensing parameters (SP) by controlling the generation of a unit pixel transmission signal and a conversion gain control signal of the image sensor. In some implementations, the sensor controller 200 can adjust the analog gain based on the sensing parameters (SP) by controlling the amplitude of a ramp signal used for analog-to-digital conversion (ADC) in the image sensor 100.

[0020] The adder unit 300 can receive sensing parameters (SP) from the sensing parameter calculator 400 and raw image data (RIMD) from the image sensor 100. The adder unit 300 can add the sensing parameters (SP) received from the sensing parameter calculator 400 to the raw image data (RIMD) output from the image sensor 100, thus outputting result data corresponding to the sum of the sensing parameters (SP) and the raw image data (RIMD). Therefore, the adder unit 300 can add the sensing parameters SP to the raw image data (RIMD), allowing the adder unit 300 to indicate the sensing conditions of the raw image data (RIMD) output from the image sensor 100. In the following text, for ease of description, the image data corresponding to the sum of the raw image data (RIMD) and the sensing parameters (SP) will be referred to as the data with added parameters (RIMD+SP). The data with added parameters (RIMD+SP) can be output to the sensing parameter calculator 400, the image processor 500, and the memory 700.

[0021] The sensing parameter calculator 400 can analyze the data (RIMD+SP) of added parameters received from the addition unit 300, and thus can generate sensing parameters (SP) for correcting the sensitivity of each color per unit pixel. For example, the sensing parameter calculator 400 can divide the data (RIMD+SP) of added parameters received from the addition unit 300 into raw image data (RIMD) and sensing parameters (SP), and can distinguish the raw image data (RIMD) for each color. Therefore, the sensing parameter calculator 400 can obtain pixel values ​​for multiple units of pixels based on the corresponding color of the unit pixel. The sensing parameter calculator 400 can use the pixel values ​​obtained for each color to calculate a histogram indicating the distribution of pixel values ​​for each color, and can search in the histogram for pixel values ​​(reference pixel values) that correspond to preset reference values. For example, the sensing parameter calculator 400 can search in the histogram for each color for the pixel values ​​corresponding to the top 10%, and can determine the reference pixel value for the corresponding color as the pixel value obtained as the result of the search.

[0022] The sensing parameter calculator 400 can calculate new sensing parameters for each color using reference pixel values, current sensing parameters (SP), and preset maximum pixel values. For example, the sensing parameter calculator 400 can determine the exposure time, conversion gain, and analog gain per pixel for each color using reference pixel values, the current gain of the image sensor 100 obtained using the current sensing parameters (SP), and preset maximum pixel values. When determining the exposure time, conversion gain, and analog gain, the sensing parameter calculator 400 can determine their values ​​sequentially in the order of exposure time → conversion gain → analog gain. A detailed method for enabling the sensing parameter calculator 400 to calculate new sensing parameters will be described later.

[0023] The sensing parameter calculator 400 may include a change determination unit (not shown) for determining whether a change in the object (e.g., object movement) has occurred or not by comparing previous raw image data with current raw image data. When the sensing parameter calculator 400 has determined that such a change (e.g., object movement) has occurred, the sensing parameter calculator 400 may not control the exposure time during the calculation of new sensing parameters. Alternatively, the sensing parameter calculator 400 may enable the exposure time difference for each color when there is an object change to be smaller than the exposure time difference for each color when there is no object change.

[0024] The image processor 500 can separate the original image data (RIMD) and the sensing parameters SP from each other when it receives data (RIMD+SP) with added parameters from the addition unit 300, and can perform image processing on the original image data (RIMD) based on the sensing parameters (SP) to form an image (IMG). In other words, the image processor 500 can use the sensing parameters (SP) to determine the sensing conditions of the current original image data (RIMD), and can perform image processing on the original image data (RIMD) according to the corresponding sensing conditions to form an image (IMG) of the object.

[0025] The display 600 can output an image (IMG) generated by the image processor 500 on the screen.

[0026] The memory 700 can store the data of added parameters (RIMD+SP) generated by the addition unit 300 and the image (IMG) generated by the image processor 500.

[0027] Figure 2 This is an example Figure 1 A block diagram of an exemplary structure of the image sensor 100 shown.

[0028] Reference Figure 2The image sensor 100 may include a pixel array 110, a row driver 120, a correlated dual sampler (CDS) 130, an analog-to-digital converter (ADC) 140, an output buffer 150, a column driver 160, and a timing controller 170. Figure 1 The components of the image sensor 100 shown are discussed by way of example only, and this patent document covers many other changes, substitutions, variations, alterations and modifications.

[0029] Pixel array 110 may include a plurality of unit pixels arranged in rows and columns. In one example, the plurality of unit pixels may be arranged as a two-dimensional (2D) pixel array including rows and columns. In another example, the plurality of unit pixels may be arranged as a three-dimensional (3D) pixel array. The plurality of unit pixels may convert optical signals into electrical signals based on individual pixels or based on groups of pixels, wherein unit pixels in a group of pixels share at least one internal circuit. Each unit pixel may include a photosensitive element. The photosensitive element may include a photodiode, a phototransistor, a photogate, a pinned photodiode, or a combination thereof.

[0030] The pixel array 110 can receive drive signals (e.g., row selection signal, reset signal, transfer (or transmit) signal, conversion gain control signal, etc.) from the row driver 120. Upon receiving a drive signal, a unit pixel can be activated to perform operations corresponding to the row selection signal, reset signal, transfer signal, and conversion gain control signal, etc. At this time, the transfer signal and conversion gain control signal can be changed based on the control of the sense controller 200, so that the exposure time and conversion gain of the unit pixel can be adjusted to correspond to the color of the corresponding unit pixel (i.e., the color of the color filter).

[0031] The row driver 120 can activate the pixel array 110 based on control signals provided by controller circuitry such as timing controller 170 to perform certain operations on unit pixels in the corresponding row. In some implementations, the row driver 120 can select one or more unit pixels arranged in one or more rows of the pixel array 110. The row driver 120 can generate a row selection signal to select one or more rows from multiple rows. The row driver 120 can sequentially enable conventional gain control signals, reset signals, and transfer signals for the unit pixels arranged in the selected row. At this time, upon receiving control signals from timing controller 170, the on / off operation of the conversion gain control signal and the timing of the transfer signal can be controlled for each color of the unit pixel. In this case, the on / off timing of the conversion gain control signal and the transfer signal can be controlled for each color of the unit pixel under the control of timing controller 170. The pixel signal generated by the unit pixels arranged in the selected row can be output to correlated dual sampler (CDS) 130.

[0032] The Correlated Double Sampler (CDS) 130 can use correlated double sampling to remove unwanted offset values ​​per unit pixel. In one example, the Correlated Double Sampler (CDS) 130 can remove unwanted offset values ​​per unit pixel by comparing the output voltage of the pixel signal (per unit pixel) obtained before and after the photocharge generated by the incident light is accumulated at the sensing node (i.e., the floating diffuse (FD) node). As a result, the CDS 130 can obtain pixel signals generated only by the incident light without introducing noise. In some implementations, upon receiving a clock signal from the timing controller 170, the CDS 130 can sequentially sample and hold the voltage levels of the reference signal and pixel signal provided from the pixel array 110 to each of the multiple column lines. That is, the CDS 130 can sample and hold the voltage levels of the reference signal and pixel signal corresponding to each column of the pixel array 110. In some implementations, the CDS 130 can transmit the reference signal and pixel signal of each column as a correlated double sample (CDS) signal to the ADC 140 based on control signals from the timing controller 170.

[0033] ADC 140 is used to convert the analog CDS signal received from CDS 130 into a digital signal. In some implementations, ADC 140 can be implemented as a ramp comparator ADC. The analog-to-digital converter (ADC) 140 can compare the ramp signal received from timing controller 170 with the CDS signal received from CDS 130, and therefore can output a comparison signal indicating the result of the comparison between the ramp signal and the CDS signal. The ADC 140 can count the level transition times of the comparison signal in response to the ramp signal received from timing controller 170, and can output a count value indicating the counted level transition times to output buffer 150.

[0034] The output buffer 150 can temporarily store column-based image data provided from the ADC 140 based on control signals from the timing controller 170. Image data received from the ADC 140 can be temporarily stored in the output buffer 150 based on control signals from the timing controller 170. The output buffer 150 can provide an interface to compensate for data rate differences or transmission rate differences between the image sensor 100 and the adder unit 300.

[0035] The column driver 160 can select a column of the output buffer 150 upon receiving a control signal from the timing controller 170, and sequentially output the image data temporarily stored in the selected column of the output buffer 150. In some implementations, upon receiving an address signal from the timing controller 170, the column driver 160 can generate a column selection signal based on the address signal. This column selection signal can be used to select a column of the output buffer 150, and the data received from the selected column of the output buffer 150 can be controlled to be output as raw image data (RIMD).

[0036] The timing controller 170 can generate signals for controlling the operation of the row driver 120, ADC 140, output buffer 150, and column driver 160. The timing controller 170 can provide the row driver 120, column driver 160, ADC 140, and output buffer 150 with clock signals required for the operation of the corresponding components of the image sensing device, control signals for timing control, and address signals for selecting rows or columns. Upon receiving control signals from the sensing controller 200, the timing controller 170 can adjust the timing of the transmitted signals, the on / off operation of the conversion gain control signals, and the amplitude of the ramp signals for each color per unit pixel. In some implementations, the timing controller 170 may include logic control circuitry, phase-locked loop (PLL) circuitry, timing control circuitry, communication interface circuitry, etc.

[0037] Figure 3A and Figure 3B This is an example of what is formed in Figure 2 A circuit diagram of an exemplary structure of a unit pixel in the pixel array shown.

[0038] Reference Figure 3A Each pixel of the pixel array 110 may include a photoelectric conversion element PD, a transmission transistor TX, a reset transistor RX, a drive transistor DX, a selection transistor SX, a conversion gain transistor CGX, and a conversion gain capacitor CAP.

[0039] A photoelectric conversion element (PD) can generate and accumulate photocharge corresponding to incident light. For example, a photoelectric conversion element (PD) may include a photodiode, a phototransistor, a photogate, a pinned photodiode (PPD), or a combination thereof, but is not limited thereto.

[0040] The transfer transistor TX can transfer the photocharge accumulated in the photoelectric conversion element PD to the floating diffusion node FD based on the transfer signal (TG).

[0041] The floating diffusion node (FD) can receive and accumulate photocharge generated by the photoelectric conversion element (PD). The driving transistor (DX) can be controlled based on the amount of photocharge accumulated in the floating diffusion node (FD).

[0042] The reset transistor TX can periodically reset the floating diffusion node FD. When the reset signal RG is enabled and the reset transistor RX is turned on, the pixel power supply voltage V... PIX The photocharge is transferred to the floating diffusion node (FD). Therefore, the photocharge accumulated in the floating diffusion node (FD) can be discharged, allowing the floating diffusion node (FD) to be reset.

[0043] The driving transistor DX can be a source follower buffer amplifier that generates a source-drain current proportional to the amount of charge applied to the floating diffuse node FD at the gate electrode. The driving transistor DX amplifies the potential change at the floating diffuse node FD, and the amplified signal can be output to the output (column) line (OUT) via the select transistor SX. The source terminal of the driving transistor DX can be connected to the pixel power supply voltage V. PIX Furthermore, the drain terminal of the driving transistor DX can be connected to the source terminal of the selection transistor SX. In this case, the switching gain of the driving transistor DX, which is configured to output an electrical signal proportional to the amount of charge accumulated in the floating diffusion node FD, can vary depending on the capacitance of the floating diffusion node FD.

[0044] The selector transistor SX can select the unit pixel to be read for each column. When the selector transistor SX is turned on by the select signal SG, the electrical signal (pixel signal) output to the drain electrode of the drive transistor DX can be output to the output line (OUT).

[0045] A conversion gain transistor CGX can be connected in series between the floating diffusion node FD and the conversion gain capacitor CAP. The conversion gain transistor CGX can selectively connect the conversion gain capacitor CAP to the floating diffusion node FD based on conversion gain control signals CG1 / CG2. When the conversion gain capacitor CAP is connected to the floating diffusion node FD, the capacitance of the floating diffusion node FD can be increased, and the input impedance of the driving transistor DX can also be increased, resulting in a low conversion gain for the corresponding unit pixel. Conversely, when the conversion gain capacitor CAP is electrically isolated from the floating diffusion node FD, a high conversion gain can be achieved for the unit pixel.

[0046] Reference Figure 3B The pixel array 110 may include a structure in which multiple unit pixels having the same color filter share a single floating diffusion node FD. In other words, multiple unit pixels can be grouped together to convert optical signals, thus generating electrical signals.

[0047] The pixel group may include first photoelectric conversion elements to fourth photoelectric conversion elements PD1, PD2, PD3, and PD4, and first transfer transistors to fourth transfer transistors TX1, TX2, TX3, and TX4. The first to fourth photoelectric conversion elements PD1, PD2, PD3, and PD4 may share a floating diffusion node FD. The first to fourth photoelectric conversion elements PD1, PD2, PD3, and PD4 may be connected to the first to fourth transfer transistors TX1, TX2, TX3, and TX4, respectively. For example, the first to fourth photoelectric conversion elements PD1, PD2, PD3, and PD4 may be arranged adjacent to each other in a (2×2) structure.

[0048] The first to fourth transmission transistors TX1, TX2, TX3 and TX4 can transmit the photocharge accumulated in the first to fourth photoelectric conversion elements PD1, PD2, PD3 and PD4 to the floating diffusion node FD based on the first to fourth transmission signals TG1, TG2, TG3 and TG4.

[0049] The reset transistor RX, drive transistor DX, select transistor SX, conversion gain transistor CGX, and conversion gain capacitor CAP are functionally compatible with... Figure 3A The corresponding transistors and capacitors shown are equal.

[0050] Figure 4 This is an example used for operation. Figure 1 The flowchart shows an example of a method for an image sensing system. Figure 5 This is an example used to pass through Figure 1 The graph shown is a curve illustrating how the sensor parameter calculator calculates the histogram of red pixel values.

[0051] When the image sensing system 10 starts operating (e.g., when the image sensing system 10 is powered on), the sensing parameter calculator 400 can initialize the sensing parameters (SP) to preset initial values ​​(S100). The sensing parameters (SP) initialized at operation 100 can be referred to as the initial sensing parameters (SP).

[0052] Sensing parameters (SP) may include information about the exposure time, conversion gain, and analog gain for each color of the image sensor 100, and the sensing parameter calculator 400 may initialize the exposure time, conversion gain, and analog gain for each color of a unit pixel using initial sensing parameters (SP). For example, for a unit pixel with green, the initial sensing parameters (SP) may be 1.0 times each of the conversion gain, analog gain, and exposure time. For each of a unit pixel with red and a unit pixel with blue, the initial sensing parameters (SP) may be 1.0 times the conversion gain, 1.4 times the analog gain, or 1.0 times the exposure time. Therefore, the gain for each unit pixel with green can be initialized to 1.0, and the gain for each of the unit pixels with red and blue can be initialized to 1.4.

[0053] The sensing parameter calculator 400 can output initial sensing parameters (SP) to the sensing controller 200 and the adder unit 300.

[0054] The sensing controller 200 can control the timing controller 170 of the image sensor 100 based on the initial sensing parameters (SP) received from the sensing parameter calculator 400, so that the sensing controller 200 can initialize the exposure time, conversion gain and analog gain of the image sensor 100 for each color per unit pixel.

[0055] For example, the timing controller 170 of the image sensor 100 can adjust the timing of the transfer signal (TG) of each unit pixel, the on / off operation of the conversion gain control signals (CG1 / CG2), and the amplitude of the ramp signal under the control of the sensing controller 200. For instance, the timing controller 170 can control the conversion gain transistor CGX of each unit pixel to be turned on or off based on the control of the sensing controller 200, such that the conversion gain of each pixel is 1.0 times, and the timing controller 170 can control each unit pixel to have the same exposure time, such that the exposure time of each unit pixel is 1.0 times. The timing controller 170 can control the amplitude of the corresponding ramp signal, such that the analog gain of each of the red and blue unit pixels is 1.4 times the analog gain of the green unit pixel.

[0056] The image sensor 100 can capture images of an object based on the sensing conditions initialized by the sensing controller 200, resulting in raw image data (RIMD) (S300).

[0057] The image sensor 100 can convert optical signals into pixel signals corresponding to electrical signals (pixel signals), convert pixel signals into digital signals, generate raw image data (RIMD) of the subject being photographed, and output the generated raw image data (RIMD).

[0058] The addition unit 300 can add the initial sensing parameters (SP) received from the sensing parameter calculator 400 to the raw image data (RIMD) output from the image sensor 100, and thus can output the data with added parameters (RIMD+SP) (S400).

[0059] In order to indicate the sensing conditions of the raw image data (RIMD) output from the image sensor 100, the adder unit 300 can add the initial sensing parameters (SP) indicating the sensing conditions to the corresponding raw image data (RIMD).

[0060] The data of added parameters (RIMD+SP) output from the adder unit 300 can be sent to the sensor parameter calculator 400 and the image processor 500. Alternatively, the data of added parameters (RIMD+SP) can be sent to and stored in the memory 700.

[0061] The sensing parameter calculator 400 can calculate a new sensing parameter (SP) based on both the original image data (RIMD) included in the data with added parameters (RIMD+SP) and the current sensing parameter (SP) (S500).

[0062] To this end, the sensing parameter calculator 400 can use raw image data (RIMD) and initial sensing parameters (SP) to calculate the target gain of the image sensor 100, and can determine the exposure time, conversion gain and analog gain used to achieve the target gain.

[0063] To obtain the target gain of the image sensor 100, the sensing parameter calculator 400 can classify the raw image data (RIMD) according to the corresponding color of each pixel, and can calculate a histogram of the pixel values ​​for each color, such as... Figure 5 As shown. Classifying the raw image data (RIMD) involves obtaining the raw image data corresponding to the corresponding color of each unit pixel. The sensing parameter calculator 400 can search for the pixel value (reference pixel value) corresponding to a preset reference value in the histogram of each color. For example, the sensing parameter calculator 400 can determine the pixel values ​​corresponding to the top 10% of the histogram as reference pixel values, but other implementations are also possible. For example, the reference pixel value can be determined as the pixel value corresponding to another value, and is not limited to the top 10% of the histogram.

[0064] Subsequently, after finding the reference pixel value for each color using the raw image data (RIMD), the sensing parameter calculator 400 can calculate the target gain for each color using the reference pixel value and the preset maximum pixel value for each color. For example, the sensing parameter calculator 400 can obtain a Y value where the reference pixel value is less than 1 / Y of the maximum pixel value, and can determine the maximum gain among the multiple configurable gains from the image sensor 100 that do not exceed Y times the current gain as the target gain.

[0065] For example, suppose the configurable gain of image sensor 100 is set to ..., 1.4, 2.0, 2.8, 4.0, 5.6, ... etc., and the maximum pixel value per pixel with the red filter is 1023, such as... Figure 5 As shown. Because each pixel has a photoelectric conversion element (PD) of the same size, all pixels can have the same maximum pixel value, regardless of the color type.

[0066] The Sensing Parameter Calculator 400 can classify the pixel values ​​of red units based on the original image data (RIMD) included in the data with added parameters (RIMD+SP), and can calculate a histogram of the corresponding pixel values, such as... Figure 5 As shown, the reference pixel value for red, which can be identified by the calculated histogram, is set to 405. Furthermore, the sensing parameter calculator 400 can identify that the current gain is set to 1.4 based on the sensing parameters (SP) extracted from the data with added parameters (RIMD+SP).

[0067] As a result, the sensing parameter calculator 400 can identify the Y value, where the reference pixel value 405 is less than 1 / Y of the maximum pixel value 1023. That is, the sensing parameter calculator 400 can identify that the Y value should be less than 1023 / 405 (≒2.53). If the range of Y values ​​is determined, the sensing parameter calculator 400 can identify the range of values ​​that are each a multiple of the current gain (1.4) (<1.4×2.53≒3.52).

[0068] The sensing parameter calculator 400 can compare a range of values, each a multiple of the current gain (1.4), with each configurable gain of the image sensor 100, allowing the calculator to search for the maximum gain among the configurable gains, each not exceeding a multiple of the current gain (1.4). In some implementations, the configurable gains of the image sensor 100 can be set to ..., 1.4, 2.0, 2.8, 4.0, 5.6, ..., such that the maximum value among values ​​not exceeding 3.52 can be set to 2.8. Therefore, the sensing parameter calculator 400 can determine the target gain as 2.8.

[0069] In other words, if the target gain for red is set to 2.8, then the target gain (2.8) is twice the current gain (1.4), making it possible to set the target gain of 810 to the top 10% of the reference pixel values ​​expected to be obtained with the target gain. This target gain is twice the reference pixel value 405, which is the top 10% of the reference pixel values ​​obtained with the current gain. In this case, the reference pixel value 810 does not exceed the maximum pixel value 1023, making it possible to use the gain (2.8) as the target gain. The configurable gains of the image sensor 100 are set to ..., 1.4, 2.0, 2.8, 4.0, 5.6, ... etc. If a gain (4.0), which is one level higher than the gain (2.8), is set as the target gain, then the gain (4.0) is approximately 2.86 times the current gain 1.4. The reference pixel value, which is expected to be the top 10% of the pixel values ​​obtained with a gain of (4.0), can be expected to be 1158.3 (405 × 2.86). This reference pixel value exceeds the maximum pixel value (1023), making reference pixel values ​​exceeding the maximum pixel value (1023) inappropriate. As a result, the target gain for red can be set to 2.8.

[0070] In the above embodiments, although the reference value used to determine the reference pixel value is set to the top 10% for ease of description, the scope or spirit of the disclosed technology is not limited thereto, and the reference value can be adjusted. For example, in order to reduce the number of saturated pixels, the reference value can be set to be higher than the top 10%.

[0071] If the target gain is determined, the sensing parameter calculator 400 can determine the conversion gain, analog gain, and exposure time to achieve such a target gain. That is, since the gain of the image sensor 100 is achieved through a combination of conversion gain, analog gain, and exposure time, the sensing parameter calculator 400 can determine the conversion gain, analog gain, and exposure time to achieve the target gain.

[0072] Generally, when determining the analog gain in an image sensor, the analog gain can be classified into 13 or 19 levels within the range of 1x to 64x, while the conversion gain can be classified into only two levels: 1x (1x) and 2x (2x). Furthermore, the exposure time can be precisely set in the same way as the analog gain. However, as the difference in exposure time for each color per unit pixel increases, motion artifacts may occur, necessitating that the difference in exposure time for each color per unit pixel be limited to a preset range.

[0073] Due to these limitations, the sensing parameter calculator 400 can determine the corresponding values ​​sequentially in the order of conversion gain → analog gain → exposure time. For example, as described above, if the target gain is determined to be 2.8, the sensing parameter calculator 400 can allow the conversion gain to be set to 2x, the analog gain to be set to 1.4x, and the exposure time to be set to 1.0x. If the gain is set to 4.0, the sensing parameter calculator 400 can control the conversion gain to be set to 2.0x, the analog gain to be set to 2.0x, and the exposure time to be set to 1.0x.

[0074] Since the target gain of the red unit pixel is set to 2.8 as described in the above embodiment, the sensing parameter calculator 400 can calculate new sensing parameters (SP). Using these sensing parameters, the conversion gain of the red unit pixel can be set to 2.0 times, the analog gain of the red unit pixel can be set to 1.4 times, and the exposure time of the red unit pixel can be set to 1.0 times, thereby outputting the new sensing parameters (SP). Although the above embodiment only discloses the red unit pixel for ease of description, the scope or spirit of the disclosed technology is not limited thereto. The sensing parameter calculator 400 can also calculate new sensing parameters (SP) for the remaining unit pixels of other colors in the same manner as described above, and thus can output new sensing parameters (SP).

[0075] For example, steps S200 to S500 can be repeated until the image sensor 100 is powered off.

[0076] While repeatedly performing this calculation of sensing parameters, the sensing parameter calculator 400 can compare previous raw image data with current raw image data and determine whether a change in the object (object movement) has occurred or not based on the comparison result. If a change in the object is detected, the sensing parameter calculator 400 can prevent the exposure time from being controlled by the sensing controller 200 when calculating new sensing parameters. That is, the sensing parameter calculator 400 can only allow conversion gain and analog gain to be reflected in the process of achieving a target gain, without reflecting the exposure time in the process of achieving such a target gain. Alternatively, the sensing parameter calculator 400 can control the difference in exposure time per unit pixel for each color when there is a change in the object to be smaller than the difference in exposure time per unit pixel for each color when there is no change in the object.

[0077] In step S400, the data of added parameters (RIMD+SP) output from the adding unit 300 can also be sent to the image processor 500. The image processor 500 can divide the data of added parameters (RIMD+SP) into raw image data (RIMD) and sensing parameters (SP), and can perform image processing on the raw image data (RIMD) based on the sensing parameters (SP) to form an image (IMG) (S600).

[0078] Image processor 500 can determine which sensing condition has been used to acquire the current raw image data (RIMD) by sensing parameters (SP), and can perform image processing on the raw image data (RIMD) according to the corresponding sensing condition. More specifically, image processor 500 can determine which gain value has been applied to each color per unit pixel, and can perform image processing on the raw image data (RIMD) according to the determined sensing condition. Here, any of several conventional image processing methods can be selected and used to implement the above image processing.

[0079] The image (IMG) processed by the image processor 500 can be sent to the display 600 and the storage 700, so that the image (IMG) can be displayed on the screen of the display 600 and stored in the storage 700 (S700).

[0080] It is evident from the above description that some implementations of the disclosed technology of image sensing systems can obtain images with less noise by more effectively compensating for the differences in the sensitivity of color filters of different colors.

[0081] The implementation of the disclosed technology can provide a variety of effects that can be directly or indirectly understood through the aforementioned patent documents.

[0082] Although several exemplary embodiments have been described, it should be understood that modifications or enhancements and other embodiments of the disclosed embodiments can be designed based on the description and / or illustrations in this patent document.

[0083] Cross-references to related applications

[0084] This patent document claims priority and benefit to Korean Patent Application No. 10-2021-0154999, filed on November 11, 2021, the entire contents of which are incorporated herein by reference as part of the disclosure of this patent document.

Claims

1. An image sensing system, the image sensing system comprising: An image sensor comprising a plurality of unit pixels, each unit pixel converting an optical signal of an object into a pixel signal and including a color filter to allow incident light of a specific color to pass through to generate the optical signal, the image sensor generating raw image data indicating an image of the object based on the pixel signals output from the plurality of unit pixels; A sensing controller, communicatively coupled to the image sensor and controlling the sensing conditions of the image sensor for each group of unit pixels based on sensing parameters, each group of unit pixels comprising one or more unit pixels corresponding to the same color; A parameter addition circuit, which is communicatively connected to the image sensor and obtains data for adding parameters by adding the sensed parameters to the raw image data output from the image sensor; as well as A sensing parameter calculator is communicatively connected to the sensing controller and the parameter addition circuit and calculates the distribution of pixel values ​​for each group in the unit pixel based on the data of the added parameters received from the parameter addition circuit, calculates updated sensing parameters based on the distribution of pixel values, and sends the updated sensing parameters to the sensing controller and the parameter addition circuit.

2. The image sensing system according to claim 1, wherein: The sensing controller controls at least one of the exposure time, conversion gain, and analog gain of the image sensor for each group of units of pixels.

3. The image sensing system according to claim 2, wherein: The sensing controller controls the exposure time and the conversion gain by controlling the generation of transmission signals and conversion gain control signals for the plurality of unit pixels, and controls the analog gain by controlling the amplitude of the ramp signal for the analog-to-digital converter (ADC) of the image sensor.

4. The image sensing system according to claim 1, wherein: The sensing parameter calculator also obtains the pixel values ​​of the plurality of unit pixels for each group of unit pixels based on the original image data included in the data of the added parameters, and calculates a histogram of the pixel values ​​of each group of unit pixels.

5. The image sensing system according to claim 4, wherein: The sensing parameter calculator also calculates the reference pixel value corresponding to the preset reference value in the histogram, and calculates the target gain of each group in the unit pixel based on the reference pixel value, the sensing parameters and the preset maximum pixel value.

6. The image sensing system according to claim 5, wherein: The sensing parameter calculator also determines the exposure time, conversion gain, and analog gain of the image sensor for each group of units based on the target gain.

7. The image sensing system according to claim 6, wherein: The sensing parameter calculator also sequentially determines the conversion gain, the analog gain, and the exposure time.

8. The image sensing system according to claim 5, wherein: The sensing parameter calculator also determines the target gain as the maximum gain among the configurable gains of the image sensor.

9. The image sensing system according to claim 5, wherein, The sensing parameter calculator performs an operation, the operation including: Calculate the Y value, wherein the reference pixel value is less than 1 / Y of the maximum pixel value; and The target gain is determined to be the maximum gain among the configurable gains of the image sensor that does not exceed the value obtained by multiplying the current gain by the Y value.

10. The image sensing system according to claim 1, wherein, The sensing parameter calculator also performs operations including: Receive second raw image data from the image sensor; The change in the object is determined based on a comparison between the original image data and the second original image data; and When the change is determined to occur, the exposure time of the corresponding group of units of pixels is maintained.

11. The image sensing system according to claim 1, wherein, The sensing parameter calculator performs an operation, the operation including: Receive second raw image data from the image sensor; The occurrence of a change in the object is determined based on a comparison between the original image data and the second original image data; and When a change is detected, the exposure time difference of each group in the unit pixel is adjusted to a first value, and when no change is detected, the exposure time difference of each group in the unit pixel is adjusted to a second value, wherein the first value is less than the second value.

12. The image sensing system according to claim 1, further comprising: An image processor receives data of the added parameters from the parameter addition circuit, obtains the original image data and the sensing parameters from the received data of the added parameters, and processes the original image data based on the sensing parameters to form an image.

13. The image sensing system according to claim 12, further comprising: A display that outputs the image received from the image processor on a screen.

14. The image sensing system according to claim 12, further comprising: A storage device that stores the data of the added parameters and the image.

15. An image sensing system, the image sensing system comprising: An image sensor, the image sensor including a first group of unit pixels configured to detect light of a first color in incident light from an object to be imaged and a second group of unit pixels configured to detect light of a second color in incident light from the object to be imaged. A sensing controller, communicatively coupled to the image sensor, controls at least one of the exposure time, conversion gain, or analog gain for each of the first unit pixel group and the second unit pixel group; as well as A sensing parameter calculator, communicatively connected to the sensing controller, calculates a first sensing parameter for the first unit pixel group and a second sensing parameter for the second unit pixel group, and provides the first and second sensing parameters to the sensing controller, such that at least one of the exposure time, the conversion gain, or the analog gain for each of the first and second unit pixel groups is based on the sensing parameters. The first sensing parameter and the second sensing parameter are different from each other.

16. The image sensing system according to claim 15, further comprising: A parameter addition circuit is communicatively connected to the image sensor and the sensing parameter calculator and obtains data of added parameters for the first and second unit pixel groups based on the raw image data output from the first and second unit pixel groups.

17. The image sensing system according to claim 15, wherein, The sensing parameter calculator also calculates the first sensing parameter and the second sensing parameter to compensate for the photosensitivity difference between the first unit pixel group and the second unit pixel group.

18. The image sensing system according to claim 16, wherein, The sensing parameter calculator also calculates the target gain of the first unit pixel group and the second unit pixel group based on the reference pixel value and the preset maximum pixel value.

19. The image sensing system according to claim 16, wherein, The sensing parameter calculator also detects changes in the object based on the original image data.

20. The image sensing system according to claim 19, wherein, The sensing parameter calculator also adjusts the exposure time of the first unit pixel group and the second unit pixel group so that the exposure time difference between the first unit pixel group and the second unit pixel group when a detected change in the object exists becomes smaller, compared to the exposure time difference between the first unit pixel group and the second unit pixel group when no change in the object is detected.

Images