Photographing Apparatus
The shooting device generates HDR images by controlling pixel sensitivity and exposure parameters to optimize dynamic range and reduce noise, addressing the challenges of capturing varying light conditions.
Patent Information
- Application Number
- KR1020210096596
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-07-22
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2041-07-22
AI Technical Summary
Existing image sensing devices struggle to generate high dynamic range (HDR) images efficiently while minimizing noise and maintaining optimal dynamic range without causing motion artifacts or saturation issues.
A shooting device that includes an image sensing device with multiple pixels of varying sensitivities, controlled by an HDR controller to adjust light exposure, exposure time, conversion gain, and analog gain, synthesizing images with different sensitivities to generate HDR images.
The device achieves HDR images with minimized noise and optimal dynamic range by controlling pixel sensitivity based on hardware characteristics, effectively capturing a wide range of light intensities without motion artifacts or saturation.
Smart Images

Figure 112021084985214-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a shooting device that generates high dynamic range (HDR) images. Background Technology
[0002] An image sensing device is a device that captures optical images by utilizing the properties of light-sensitive semiconductor materials that react to light. With the advancement of industries such as automotive, medical, computer, and telecommunications, the demand for high-performance image sensing devices is increasing in various fields, including smartphones, digital cameras, gaming devices, the Internet of Things, robots, security cameras, and medical micro-cameras.
[0003] Image sensing devices can be broadly classified into Charge Coupled Device (CCD) image sensing devices and Complementary Metal Oxide Semiconductor (CMOS) image sensing devices. While CCD image sensing devices provide better image quality compared to CMOS image sensing devices, they tend to be implemented in larger sizes and consume more power. On the other hand, CMOS image sensing devices can be implemented in smaller sizes and consume less power than CCD devices. Furthermore, since CMOS image sensing devices are manufactured using CMOS manufacturing technology, light-sensing elements and signal processing circuits can be integrated onto a single chip, enabling the production of compact image sensing devices at a lower cost. For these reasons, CMOS image sensing devices are being developed for many applications, including mobile devices. The problem to be solved
[0004] The technical concept of the present invention can provide a shooting device that generates HDR images using the optimal method among various methods.
[0005] The technical problems of the present invention are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below. means of solving the problem
[0006] A shooting device according to one embodiment of the present invention disclosed in this document may include an image sensing device comprising a plurality of pixels, each of which generates a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit that acquires the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit that acquires at least one controllable item among sensitivity items that determine the sensitivity of each of the plurality of pixels; and a set value calculation unit that calculates a set value for the at least one controllable item based on the image data of the target area and the at least one controllable item to generate a control signal.
[0007] A shooting device according to another embodiment of the present invention may include an image sensing device that includes a plurality of pixels, each of which generates a pixel signal corresponding to the intensity of incident light, and generates image data corresponding to the pixel signal; an HDR controller that calculates a set value for at least one controllable item among light exposure amount, exposure time, conversion gain, and analog gain based on the image data and generates a control signal; and an image signal processor that synthesizes image data having different sensitivities generated in a state where the sensitivity of each of the plurality of pixels is adjusted according to the control signal to generate an HDR image. Effects of the invention
[0008] According to the embodiments disclosed in this document, by identifying the luminance and controllable items of a subject and controlling the controllable items by reflecting the hardware characteristics (e.g., response characteristics) when controlling sensitivity, it is possible to generate an HDR image having the maximum dynamic range while minimizing noise.
[0009] In addition, various effects identified directly or indirectly through this document may be provided. Brief explanation of the drawing
[0010] FIG. 1 is a drawing showing a shooting device according to one embodiment of the present invention. Figure 2 is a diagram exemplarily showing the response according to illumination for high-sensitivity pixels and low-sensitivity pixels. Figure 3 is a block diagram showing the image sensing device and HDR controller illustrated in Figure 1 in more detail. FIG. 4 is a circuit diagram showing an example of a pixel included in the pixel array of FIG. 3. FIG. 5 is a diagram illustrating an embodiment of controlling the sensitivity of pixels by a difference in light transmittance. FIG. 6 is a diagram illustrating an embodiment of controlling pixel sensitivity by a difference in exposure time. FIG. 7 is a diagram illustrating an embodiment of controlling pixel sensitivity by the difference in conversion gain. FIG. 8 is a diagram illustrating an embodiment of controlling the sensitivity of a pixel by a difference in analog gain. FIG. 9 is a flowchart illustrating a method for generating an HDR image of a shooting device according to an embodiment of the present invention. Figure 10 is a flowchart showing step S30 of Figure 9 in more detail. Specific details for implementing the invention
[0011] Various embodiments will be described below with reference to the attached drawings. However, the present disclosure is not limited to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments. Embodiments of the present disclosure may provide various effects that can be recognized directly or indirectly through the present disclosure.
[0012] FIG. 1 is a drawing showing a shooting device according to an embodiment of the present invention. FIG. 2 is a drawing exemplarily showing the response according to illumination for high-sensitivity pixels and low-sensitivity pixels.
[0013] Referring to FIG. 1, the shooting device (1) may refer to a device such as a digital still camera that takes still images or a digital video camera that takes videos. For example, the shooting device (1) may be implemented as a Digital Single Lens Reflex (DSLR), a mirrorless camera, or a smartphone, but is not limited thereto, and the shooting device (1) may be a concept that includes a device capable of taking a subject and generating an image, including a lens and an image element.
[0014] The shooting device (1) may include a lens (10), an aperture (20), a lens driver (30), an aperture driver (40), an image sensing device (100), and an image signal processor (200).
[0015] The lens (10) may include at least one lens aligned with respect to the optical axis. The lens (10) may be positioned in front of the image sensing device (100) to transmit incident light to the image sensing device (100), and its position may be adjusted by a lens driver (30). For example, the lens (10) may be moved along the optical axis by the lens driver (30).
[0016] The aperture (20) is positioned at the front end of the image sensing device (100) and the degree of opening and closing is controlled by the aperture driver (40), thereby controlling the amount of light incident on the image sensing device (100). In FIG. 1, the aperture (20) is shown as being located at the rear end of the lens (10) to receive incident light passing through the lens (10), but the scope of the present invention is not limited thereto. For example, the aperture (20) may be located between the lenses included in the lens (10) or at the front end of the lens (10).
[0017] Incident light passing through the lens (10) and aperture (20) is incident on the light receiving surface of the image sensing device (100) to form an image of the subject.
[0018] The lens driver (30) can adjust the position of the lens (10) according to a control signal (CS) provided from the image signal processor (200). By adjusting the position of the lens (10), the lens driver (30) can perform operations such as autofocusing, zoom change, and focus change.
[0019] The aperture driver (40) can control the degree of opening and closing of the aperture (20) according to a control signal (CS) provided from the image signal processor (200). By controlling the degree of opening and closing of the aperture (20), the aperture driver (40) can control the amount of light exposure to the image sensing device (100).
[0020] The image sensing device (100) may be a Complementary Metal Oxide Semiconductor Image Sensor (CIS) that converts incident light into an electrical signal. The image sensing device (100) may have its on / off, exposure time, conversion gain, analog gain, etc. controlled by an image signal processor (200). The image sensing device (100) generates image data (IDATA) by converting incident light into an electrical signal on a pixel-by-pixel basis, and the image sensing device (100) may include multiple pixels with different sensitivities. Here, sensitivity may refer to the increase in image data (IDATA) (or the increase in response) in relation to the increase in the intensity of the incident light. That is, the higher the sensitivity, the greater the increase in image data (IDATA) for the increase in the intensity of incident light, and the lower the sensitivity, the smaller the increase in image data (IDATA) for the increase in the intensity of incident light. Sensitivity can be determined by light transmittance, conversion gain, exposure time, analog gain, etc. A more detailed configuration and operation of the image sensing device (100) will be described later with reference to FIG. 3.
[0021] The image signal processor (200) processes image data (IDATA) input from the image sensing device (100) and can control each component of the shooting device (10) according to the processing result or according to an external input signal. The image signal processor (200) can reduce noise on the image data (IDATA) and perform image signal processing for image quality improvement, such as gamma correction, color filter array interpolation, color matrix, color correction, and color enhancement. In addition, the image data generated by performing image signal processing for image quality improvement can be compressed to create an image file, or image data can be restored from said image file. The compression format of the image may be a reversible format or an irreversible format. As an example of a compression format, for a still image, the JPEG (Joint Photographic Experts Group) format or the JPEG 2000 format may be used. In addition, for video, a video file can be created by compressing multiple frames according to the MPEG (Moving Picture Experts Group) standard. For example, a video file can be created according to the Exif (Exchangeable image file format) standard.
[0022] Additionally, the image signal processor (200) can generate an HDR image by synthesizing at least two images with different sensitivities. The configuration for generating the HDR image can be defined as an image synthesis unit distinct from the HDR controller (300). For example, the image sensing device (100) can output a low-sensitivity image generated from pixels with relatively low sensitivity (low-sensitivity pixels) and a high-sensitivity image generated from pixels with relatively high sensitivity (high-sensitivity pixels), and the image signal processor (200) can generate an HDR image by synthesizing the low-sensitivity image and the high-sensitivity image. Here, low sensitivity and high sensitivity are relative concepts, and the image sensing device (100) can generate n (n is an integer greater than or equal to 2) or more image data (IDATA) with different sensitivities, and the image signal processor (200) can generate an HDR image using these.
[0023] The HDR image output from the image signal processor (200) can be stored in the internal memory or external memory of the shooting device (1) or displayed through a display, either automatically or at the user's request.
[0024] In addition, the image signal processor (200) can perform blur processing, blur processing, edge highlighting processing, image interpretation processing, image recognition processing, image effect processing, etc.
[0025] In addition, the image signal processor (200) can perform display image signal processing for display. For example, the image signal processor (200) can perform brightness level adjustment, color correction, contrast adjustment, contour enhancement adjustment, screen splitting processing, character image generation, and image synthesis processing.
[0026] The image signal processor (200) can control the lens driver (30), aperture driver (40), and image sensing device (100) according to control information automatically generated by real-time input image data (IDATA) or control information manually input by user operation.
[0027] In particular, the image signal processor (200) may include an HDR controller (300), and according to another embodiment, the HDR controller (300) may be implemented independently of the image signal processor (200). For example, the HDR controller (300) may be included in the image sensing device (100).
[0028] The HDR controller (300) can control at least one of the aperture driver (40) and the image sensing device (100) so that the pixels of the image sensing device (100) can have an optimal dynamic range.
[0029] Referring to FIG. 2, for a high-sensitivity pixel, which is a pixel with a relatively large increase in response with increasing intensity of incident light, and a low-sensitivity pixel, which is a pixel with a relatively small increase in response with increasing intensity of incident light, the response of the high-sensitivity pixel and the response of the low-sensitivity pixel according to the intensity of incident light incident on the pixel are illustrated. Here, the response may refer to the image data (IDATA) of the pixel.
[0030] The response can have an SNR (signal-to-noise ratio) limit level and a saturation level.
[0031] The SNR limit level may refer to a threshold value of a response capable of satisfying a predetermined reference SNR. A response below the SNR limit level becomes an invalid response that cannot satisfy the reference SNR, and a response above the SNR limit level becomes a valid response capable of satisfying the reference SNR. The reference SNR may be a value experimentally determined by considering the characteristics of the image sensing device (100) and the requirements of the system.
[0032] The saturation level may refer to the maximum response capable of representing the intensity of incident light. The saturation level can be determined by the pixel's ability to convert the intensity of incident light into photocharges (e.g., the capacitance of a photoelectric converter), its ability to convert photocharges into analog signals (e.g., the capacitance of a floating diffusion region), and its ability to convert analog signals into digital signals (e.g., the input range of an analog-to-digital converter). When the intensity of incident light increases, the response may increase in proportion to the intensity of the incident light until it reaches the saturation level. However, after the response reaches the saturation level, the response cannot increase beyond the saturation level even if the intensity of the incident light increases, and may have the same value as the saturation level.
[0033] If the effective response of a pixel is defined as a response capable of representing the intensity of incident light while satisfying the reference SNR, the range of incident light intensities corresponding to the effective response of the pixel can be defined as the pixel's dynamic range. In other words, the pixel's dynamic range can refer to the range of incident light intensities in which the pixel can have an effective response.
[0034] The response of a high-sensitivity pixel, which is a pixel with a relatively large increase in response with increasing intensity of incident light, increases with a relatively high slope in response to the increase in intensity of incident light, and after reaching the saturation level, it can have a level corresponding to the saturation level regardless of the intensity of incident light.
[0035] The response of a low-sensitivity pixel, which is a pixel with a relatively small increase in response with increasing intensity of incident light, increases with a relatively low slope in response to an increase in intensity of incident light, and after reaching a saturation level, it can have a level corresponding to the saturation level regardless of the intensity of incident light.
[0036] As shown in FIG. 2, the lower limit of the dynamic range (DR_H) of the high-sensitivity pixel is smaller than the lower limit of the dynamic range (DR_L) of the low-sensitivity pixel, and the upper limit of the dynamic range (DR_H) of the high-sensitivity pixel is smaller than the upper limit of the dynamic range (DR_L) of the low-sensitivity pixel. Therefore, in an illuminance range where the intensity of the incident light is relatively small (low illuminance), the high-sensitivity pixel is more suitable for detecting the intensity of the incident light, and in an illuminance range where the intensity of the incident light is relatively large (high illuminance), the low-sensitivity pixel may be more suitable for detecting the intensity of the incident light.
[0037] HDR can be implemented by utilizing the respective responses of high-sensitivity pixels suitable for low light conditions and low-sensitivity pixels suitable for high light conditions. That is, compared to the case where only one of the high-sensitivity or low-sensitivity pixels is used, when high-sensitivity and low-sensitivity pixels are used together, the entire pixel array can have a dynamic range corresponding to the range from the lower limit of the dynamic range of the high-sensitivity pixels to the upper limit of the dynamic range of the low-sensitivity pixels. To achieve this, at least a portion of the dynamic range of the high-sensitivity pixels and at least a portion of the dynamic range of the low-sensitivity pixels can overlap each other.
[0038] A method for synthesizing an HDR image corresponding to a dynamic range using high-sensitivity pixels and low-sensitivity pixels may utilize methods such as synthesizing by interpolating and performing operations (e.g., summing) on the responses of high-sensitivity pixels and low-sensitivity pixels, generating an image based on the response of high-sensitivity pixels in low light conditions and generating an image based on the response of low-sensitivity pixels in high light conditions, but the scope of the present invention is not limited thereto.
[0039] As shown in FIG. 2, by adjusting the sensitivity of a pixel (i.e., the slope of the response), the dynamic range of the corresponding pixel can be adjusted. The sensitivity of a pixel can be determined by at least one of the sensitivity items that determine the sensitivity of the pixel, namely light exposure amount, light transmittance, exposure time, conversion gain, and analog gain. Accordingly, the HDR controller (300) can adjust the dynamic range of the pixel by adjusting the controllable items among the light exposure amount, light transmittance, exposure time, conversion gain, and analog gain (i.e., sensitivity items).
[0040] While widening the dynamic range offers the advantage of obtaining an effective response corresponding to incident light over a wide illumination range, it can also lead to increased side effects. For example, excessive adjustment of the exposure time can cause significant differences in the timing of incident light capture between pixels, potentially increasing motion artifacts for fast-moving subjects. Alternatively, if the dynamic range is unnecessarily extended to the low-light range while capturing a very bright scene, the response of high-sensitivity pixels may become saturated, potentially degrading the overall image quality.
[0041] Accordingly, the HDR controller (300) can control at least one of the aperture driver (40) and the image sensing device (100) based on the characteristics of the scene and the characteristics of the controllable item so that the pixels of the image sensing device (100) can have an optimal dynamic range.
[0042] Figure 3 is a block diagram showing the image sensing device and HDR controller illustrated in Figure 1 in more detail.
[0043] Referring to FIG. 3, the image sensing device (100) may include a pixel array (110), a row driver (120), a Correlate Double Sampler (CDS, 130), an Analog-Digital Converter (ADC, 140), an output buffer (150), a column driver (160), and a timing controller (170). Here, each component of the image sensing device (100) is merely exemplary, and at least some components may be added or omitted as needed.
[0044] A pixel array (110) may include a plurality of pixels arranged in a plurality of rows and a plurality of columns. In one embodiment, the plurality of pixels may be arranged in a two-dimensional pixel array including rows and columns. In another embodiment, a plurality of unit image pixels may be arranged in a three-dimensional pixel array. The plurality of pixels may convert an optical signal into an electrical signal on a pixel-by-pixel basis or in a pixel group basis, and the unit pixels within a pixel group may share at least a specific internal circuit. The pixel array (110) may receive a pixel control signal including a row select signal, a pixel reset signal, and a transmission signal from a row driver (120), and the corresponding unit pixel of the pixel array (110) may be activated by the pixel control signal to perform an operation corresponding to the row select signal, the pixel reset signal, and the transmission signal.
[0045] The row driver (120) can activate the pixel array (110) to perform specific operations on the unit pixels included in the corresponding row based on commands and control signals supplied by the timing controller (170). In one embodiment, the row driver (120) can select at least one unit pixel arranged in at least one row of the pixel array (110). The row driver (120) can generate a row selection signal to select at least one row among a plurality of rows. The row driver (120) can sequentially enable a pixel reset signal and a transmission signal for the pixels corresponding to the selected at least one row. Accordingly, an analog reference signal and an image signal generated from each of the pixels of the selected row can be sequentially transmitted to the correlation double sampler (130). Here, the reference signal is an electrical signal provided to the correlation double sampler (130) when the sensing node (e.g., floating diffusion node) of a unit pixel is reset, and the image signal may be an electrical signal provided to the correlation double sampler (130) when photocharges generated by the unit pixel are accumulated in the sensing node. The reference signal representing the pixel-specific reset noise and the image signal representing the intensity of the incident light may be collectively referred to as the pixel signal.
[0046] A CMOS image sensing device may utilize correlated double sampling to eliminate unwanted offset values of pixels, such as fixed pattern noise, by sampling the pixel signal twice to eliminate the difference between two samples. For example, correlated double sampling can eliminate unwanted offset values by comparing pixel output voltages obtained before and after photocharges generated by incident light are accumulated at the sensing node, thereby allowing the pixel output voltage based solely on incident light to be measured. In one embodiment, the correlated double sampler (130) may sequentially sample and hold reference signals and image signals provided to each of a plurality of column lines from the pixel array (110). That is, the correlated double sampler (130) may sample and hold levels of reference signals and image signals corresponding to each of the columns of the pixel array (110).
[0047] The correlation double sampler (130) can transmit the reference signal and image signal of each column to the ADC (140) as a correlation double sampling signal based on a control signal from the timing controller (170).
[0048] The ADC (140) can convert the correlated double sampling signal for each column output from the correlated double sampler (130) into a digital signal and output it. In one embodiment, the ADC (140) can convert the correlated double sampling signal generated by the correlated double sampler (130) for each column into a digital signal and output it.
[0049] The ADC (140) may include a plurality of column counters corresponding to each of the columns of the pixel array (110). Each column of the pixel array (110) is connected to each column counter, and image data may be generated by converting the correlated double sampling signal corresponding to each of the columns into a digital signal using the column counters. According to another embodiment, the ADC (140) may include a global counter and convert the correlated double sampling signal corresponding to each of the columns into a digital signal using a global code provided by the global counter.
[0050] The output buffer (150) can temporarily hold and output image data (IDATA) in each column unit provided by the ADC (140). The output buffer (150) can temporarily store image data (IDATA) output from the ADC (140) based on a control signal from the timing controller (170). The output buffer (150) can operate as an interface that compensates for the difference in transmission speed (or processing speed) between the image sensing device (100) and other devices connected thereto.
[0051] The column driver (160) can select a column of the output buffer (150) based on a control signal from the timing controller (170) and control the output of image data (IDATA) temporarily stored in the selected column of the output buffer (150) sequentially. In one embodiment, the column driver (160) can receive an address signal from the timing controller (170), and the column driver (160) can generate a column selection signal based on the address signal and select a column of the output buffer (150) to control the output of image data (IDATA) from the selected column of the output buffer (150) to the outside.
[0052] The timing controller (170) can control at least one of the row driver (120), the correlated dual sampler (130), the ADC (140), the output buffer (150), and the column driver (160).
[0053] The timing controller (170) can provide clock signals required for the operation of each component of the image sensing device (100), control signals for timing control, and address signals for selecting a row or column to at least one of a row driver (120), a correlated dual sampler (130), an ADC (140), an output buffer (150), and a column driver (160). According to one embodiment, the timing controller (170) may include a logic control circuit, a phase lock loop (PLL) circuit, a timing control circuit, and a communication interface circuit, etc.
[0054] The timing controller (170) can control the sensitivity of each pixel of the pixel array (110). The sensitivity of each pixel can be determined by light transmittance, exposure time, conversion gain, and analog gain. Light transmittance may refer to the ratio of the intensity of light reaching the element that converts light into charge (photoelectric conversion element described later) inside the pixel to the intensity of light incident toward the pixel. Exposure time may refer to the time it takes to convert light incident on the pixel into charge. Conversion gain may refer to the ratio of the level of the pixel signal (i.e., voltage) to the amount of charge generated in the pixel. Analog gain may refer to the ratio of the digital value (i.e., image data) to which the pixel signal is converted to the level of the pixel signal output from the pixel. The higher the light transmittance, the shorter the exposure time, the higher the conversion gain, or the higher the analog gain, the higher the sensitivity of the pixel. Conversely, the lower the light transmittance, the longer the exposure time, the lower the conversion gain, or the lower the analog gain, the lower the sensitivity of the pixel may be.
[0055] Light transmittance can be fixed in advance for each pixel, and exposure time, conversion gain, and analog gain can be controllable items. The timing controller (170) can control a row driver (120) that supplies a control signal to the pixel array (110) to control the exposure time or conversion gain of each pixel. Additionally, the timing controller (170) can control an ADC (140) that performs analog-to-digital conversion to control the analog gain of each pixel.
[0056] The method by which the timing controller (170) controls the exposure time, conversion gain, and analog gain to control the sensitivity of each pixel will be described later with reference to FIG. 6 and below.
[0057] The HDR controller (300) may include a luminance acquisition unit (310), a controllable item acquisition unit (320), and a set value calculation unit (330).
[0058] The luminance acquisition unit (310) can acquire the luminance of a target area within a scene based on image data (IDATA) generated by the image sensing device (100). Here, the scene may refer to image data (IDATA) corresponding to a frame captured by the image sensing device (100) at once. That is, the scene is an image captured at once from the entire pixel array (110), and pixels corresponding to a target area, which is at least part of the scene, may be defined as target pixels. The target area is an area included in the scene and may refer to an area designated by a user (e.g., a specific subject), an area where image data (IDATA) with high luminance in the scene is concentrated (i.e., the brightest area), or an area corresponding to the entire scene. Additionally, luminance may refer to the value of the image data (IDATA). According to another embodiment, the luminance acquisition unit (310) may acquire the luminance of the target area using a separate device (e.g., a photometric sensor, another camera device).
[0059] The controllable item acquisition unit (320) can acquire controllable items among light exposure amount, exposure time, conversion gain, and analog gain. Here, the light exposure amount may be a value determined according to the degree of opening of the aperture (20) under the control of the aperture driver (40). Each of the light exposure amount, exposure time, conversion gain, and analog gain may be fixed in advance or controllable. For example, in the case of night shooting, the system (e.g., application processor) may fix the light exposure amount to the maximum. Or, when shooting video or high-speed shooting where the time allocated for one frame is short, the system may limit the exposure time to a certain time or forcibly fix it. Or, to reduce power consumption required for the control of the image sensing device (100) and image processing, the system may limit the control of at least one of the conversion gain and analog gain.
[0060] The setting value calculation unit (330) can calculate a setting value for a controllable item based on the brightness of the target area received from the brightness acquisition unit (310) and a controllable item received from the controllable item acquisition unit (320), and generate a control signal (CS) representing the calculated setting value. Here, the setting value may refer to information for controlling a controllable item (light exposure amount, exposure time, conversion gain, analog gain).
[0061] The detailed operation of the setting value calculation unit (330) will be described later with reference to FIG. 10.
[0062] FIG. 4 is a circuit diagram showing an example of a pixel included in the pixel array of FIG. 3.
[0063] Referring to FIG. 4, a pixel (PX) may be any one of a plurality of pixels included in a pixel array (110), and while FIG. 4 describes one pixel (PX), other pixels may also have substantially the same structure and operation as the pixel (PX).
[0064] A pixel (PX) may include a photoelectric conversion element (PD), a transfer transistor (TX), a reset transistor (RX), a floating diffusion region (FD), a CG (conversion gain) transistor (CX), first and second capacitors (C1 to C2), a source follower transistor (SF), and a select transistor (SX). In FIG. 4, the pixel (PX) is illustrated as having one photoelectric conversion element (PD), but according to other embodiments, it may be a shared pixel having a plurality of photoelectric conversion elements. In this case, a plurality of transfer transistors may be provided corresponding to the plurality of photoelectric conversion elements.
[0065] A photoelectric conversion device (PD) can generate and accumulate photocharges corresponding to the intensity of incident light. For example, the photoelectric conversion device (PD) can be implemented as a photodiode, a phototransistor, a photogate, a pinned photodiode, or a combination thereof.
[0066] When the photoelectric conversion element (PD) is implemented as a photodiode, it may be a region doped with an impurity of a second conductivity type (e.g., N-type) within a substrate having a first conductivity type (e.g., P-type).
[0067] A transfer transistor (TX) can be connected between a photoelectric conversion element (PD) and a floating diffusion region (FD). The transfer transistor (TX) can be turned on or turned off in response to a transfer control signal (TG), and the turned-on transfer transistor (TX) can transfer photocharges accumulated in the photoelectric conversion element (PD) to the floating diffusion region (FD).
[0068] The reset transistor (RX) is connected between the power supply voltage (VDD) and the floating diffusion region (FD), and can reset the voltage of the floating diffusion region (FD) to the power supply voltage (VDD) in response to the reset control signal (RG).
[0069] The floating diffusion region (FD) can accumulate photocharges transferred from the transfer transistor (TX). The floating diffusion region (FD) can be connected to a first capacitor (C1) connected to a ground terminal. For example, the floating diffusion region (FD) may be a region doped with impurities of a second conductivity type (e.g., N-type) within a substrate having a first conductivity type (e.g., P-type), and the substrate and the impurity-doped region may be modeled as a first capacitor (C1) which is a junction capacitor.
[0070] The CG transistor (CX) is connected between the floating diffusion region (FD) and the second capacitor (C2), and the second capacitor (C2) can be selectively connected to the floating diffusion region (FD) in response to the CG control signal (CG). The second capacitor (C2) may be composed of at least one of a Metal-Insulator-Metal (MIM) capacitor, a Metal-Insulator-Polysilicon (MIP) capacitor, a Metal-Oxide-Semiconductor (MOS) capacitor, and a junction capacitor. When the CG transistor (CX) is turned off, the floating diffusion region (FD) may have a capacitance corresponding to the capacitance of the first capacitor (C1). When the CG transistor (CX) is turned on, the floating diffusion region (FD) can have a capacitance corresponding to the sum of the capacitance of the first capacitor (C1) and the capacitance of the second capacitor (C2). That is, the CG transistor (CX) can control the capacitance of the floating diffusion region (FD).
[0071] In FIG. 4, the case where there is one CG transistor (CX) is described as an example, but according to other embodiments, there may be multiple CG transistors. In this case, the capacitance of the floating diffusion region (FD) may have a wider range of values.
[0072] The source follower transistor (SF) is connected between the power supply voltage (VDD) and the select transistor (SX), and can amplify the change in the electrical potential of the floating diffusion region (FD), which receives the photocharge accumulated in the photoelectric conversion element (PD), and transfer it to the select transistor (SX).
[0073] The select transistor (SX) is connected between the source follower transistor (SF) and the output signal line, and is turned on by the select control signal (SEL) to output the electrical signal transmitted from the source follower transistor (SF) as a pixel signal (PS).
[0074] FIG. 5 is a diagram illustrating an embodiment of controlling the sensitivity of pixels by a difference in light transmittance.
[0075] Referring to FIG. 5, an embodiment is illustrated in which high transmittance pixels (HPX) with relatively high light transmittance and low transmittance pixels (LPX) with relatively low light transmittance are arranged on the left side of FIG. 5. That is, three high transmittance pixels (HPX) and one low transmittance pixel (LPX) can be arranged in a 2x2 matrix (unit matrix). Each of the high transmittance pixels (HPX) and the low transmittance pixel (LPX) may have a structure corresponding to FIG. 4. According to another embodiment, each of the high transmittance pixels (HPX) and the low transmittance pixel (LPX) may independently include a photoelectric conversion element and a transfer transistor, while other components may be implemented as a shared pixel structure shared by four pixels.
[0076] According to one embodiment, high transmittance pixels (HPX) and low transmittance pixels (LPX) may be pixels that detect light of the same color (e.g., red, blue, or green). In this case, the pixel array (110) may form a quad Bayer pattern structure in units of a 2x2 matrix.
[0077] On the right side of FIG. 5, an example of a cross-section is shown by cutting along a cutting line (A-A') between a high transmittance pixel (HPX) and a low transmittance pixel (LPX) that are placed adjacent to each other.
[0078] The cross-sections of the high transmittance pixel (HPX) and the low transmittance pixel (LPX) may include a substrate (510), a photoelectric conversion element (520), an optical filter (530), a micro lens (540), and a light blocking structure (550).
[0079] The substrate (510) is a semiconductor substrate, for example, a P-type or N-type bulk substrate, a substrate on which a P-type or N-type epitaxial layer is grown on a P-type bulk substrate, or a substrate on which a P-type or N-type epitaxial layer is grown on an N-type bulk substrate.
[0080] The photoelectric conversion element (520) can be formed inside the substrate (510) and may correspond to the photoelectric conversion element (PD) of FIG. 4. That is, the photoelectric conversion element (520) can generate and accumulate photocharges corresponding to the intensity of incident light passing through the microlens (540) and the optical filter (530).
[0081] An optical filter (530) can selectively transmit light of a transmission wavelength band (e.g., Red, Green, Blue, Magenta, Yellow, Cyan, infrared, etc.). Here, the transmission wavelength band may refer to a wavelength band corresponding to the light that the optical filter intends to selectively transmit. For example, the optical filter (530) may include a colored photosensitive material corresponding to a specific color or may include alternating thin film layers. The optical filters included in the pixel array (110) can form an optical filter array by being arranged in correspondence with pixels arranged in a matrix comprising multiple rows and multiple columns.
[0082] A micro lens (540) can be formed on the upper part of an optical filter (530) and can increase the light gathering power for incident light to improve the light receiving efficiency of the photoelectric conversion element (520).
[0083] A light blocking structure (550) may be placed between one side of the substrate (510) and the optical filter (530) so as to block at least a portion of the incident light passing through the optical filter (530) in the low transmittance pixel (LPX) so as not to be transmitted to the photoelectric conversion element (520). The light blocking structure (550) may include at least one of a material with high light reflectivity (e.g., silver, aluminum) and a material with high light absorption (e.g., tungsten).
[0084] The total area of the low transmittance pixel (LPX) can be defined as the sum of the blocking area of the area where the light blocking structure (550) is not placed and the aperture area of the area where the light blocking structure (550) is placed. The light transmittance of the low transmittance pixel (LPX) can be determined according to the ratio between the blocking area and the aperture area.
[0085] The light transmittance of a high transmittance pixel (HPX) that does not include a light blocking structure (550) may be higher than the light transmittance of a low transmittance pixel (LPX) that includes a light blocking structure (550).
[0086] That is, when incident light of the same intensity is incident on a high transmittance pixel (HPX) and a low transmittance pixel (LPX), the intensity of light transmitted to the photoelectric conversion element (520) of the low transmittance pixel (LPX) may be smaller than the intensity of light transmitted to the photoelectric conversion element (520) of the high transmittance pixel (HPX).
[0087] Additionally, the light intensity transmitted to the photoelectric conversion element (520) of the low transmittance pixel (LPX) increases at a relatively low slope in response to an increase in the intensity of the incident light, and the light intensity transmitted to the photoelectric conversion element (520) of the high transmittance pixel (HPX) increases at a relatively high slope in response to an increase in the intensity of the incident light.
[0088] Since the light intensity transmitted to the photoelectric conversion element (520) of the low transmittance pixel (LPX) and the light intensity transmitted to the photoelectric conversion element (520) of the high transmittance pixel (HPX) are each converted into pixel signals, the response of the low transmittance pixel (LPX) follows the response of the low-sensitivity pixel shown in FIG. 2, and the response of the high transmittance pixel (HPX) follows the response of the high-sensitivity pixel shown in FIG. 2.
[0089] In FIG. 5, the light blocking structure (550) is illustrated as being placed at the edge of the low transmittance pixel (LPX), but the scope of the invention is not limited thereto, and for example, the light blocking structure (550) may be placed at any location of the low transmittance pixel (LPX), and may be placed over the entire area of the low transmittance pixel (LPX) without any part being open.
[0090] According to an image sensing device (100) according to one embodiment of the present invention, low-sensitivity pixels and high-sensitivity pixels can be implemented together within a single pixel array (110), so that an HDR image can be generated as a single image.
[0091] FIG. 6 is a diagram illustrating an embodiment of controlling pixel sensitivity by a difference in exposure time.
[0092] Referring to FIG. 6, it is assumed that a long exposure pixel with a relatively long exposure time and a short exposure pixel with a relatively short exposure time are placed together in one row of a pixel array (110). Since the pixel array (110) is driven row by row driver (120), the long exposure pixel and the short exposure pixel may receive the same reset signal (RG) and row select signal (SEL). However, the long exposure pixel and the short exposure pixel may receive different first transmission signals (TG_L) and second transmission signals (TG_S) to have different exposure times. Meanwhile, although not shown, the long exposure pixel and the short exposure pixel may receive the same CG signal (CG) or different CG signals (CG).
[0093] Each of the reset signal (RG), low select signal (SEL), first transmission signal (TG_L), and second transmission signal (TG_S) may have a logic low level (L) and a logic high level (H), and the transistor to which the signal having the logic low level (L) is input may be turned off, and the transistor to which the signal having the logic high level (H) is input may be turned on.
[0094] The operation period of each long exposure pixel and short exposure pixel may include a reset period (RS), an exposure period, and a readout period (RD). The operation period may further include a readout period (RD) for generating a reference signal after the reset period (RS), but for the sake of convenience of explanation, a description thereof will be omitted.
[0095] The reset period (RS) may be a period for removing residual photocharges within the corresponding pixel and resetting the floating diffusion region (FD) to the power supply voltage (VDD). In the reset period (RS), the reset signal (RG), the low select signal (SEL), the first transmission signal (TG_L), and the second transmission signal (TG_S) may each have a logic high level (H).
[0096] The exposure section (EX1, EX2) may be a section in which the photoelectric conversion element (PD) of each long exposure pixel and short exposure pixel generates and accumulates photocharges corresponding to the intensity of incident light, and transfers the accumulated photocharges to a floating diffusion region (FD).
[0097] After the reset period (RS), the photoelectric conversion element (PD) of the long exposure pixel generates and accumulates photocharges corresponding to the intensity of the incident light, and the transfer transistor of the long exposure pixel can transfer photocharges from the photoelectric conversion element (PD) of the long exposure pixel to the floating diffusion region (FD) of the long exposure pixel until the first transfer signal (TG_L) transitions from a logic high level (H) to a logic low level (L). That is, the long exposure pixel can accumulate photocharges generated during the first exposure period (EX1) in the floating diffusion region (FD).
[0098] After the reset period (RS), the photoelectric conversion element (PD) of the single exposure pixel generates and accumulates photocharges corresponding to the intensity of the incident light, and the transfer transistor of the single exposure pixel can transfer photocharges from the photoelectric conversion element (PD) of the single exposure pixel to the floating diffusion region (FD) of the single exposure pixel until the second transfer signal (TG_S) transitions from a logic high level (H) to a logic low level (L). That is, the single exposure pixel can accumulate photocharges generated during the second exposure period (EX2) in the floating diffusion region (FD). The exposure periods (EX1, EX2) may refer to the time during which photocharges are generated and accumulated in each pixel, and the transfer signal applied to each pixel may determine the length of the exposure period.
[0099] The readout period (RD) may be a period in which the long exposure pixel and the short exposure pixel each generate an electrical signal corresponding to the photocharge accumulated in the floating diffusion region (FD) and output it as a pixel signal (PS).
[0100] The point in time when the first transmission signal (TG_L) transitions from a logic high level (H) to a logic low level (L) may be later than the point in time when the second transmission signal (TG_S) transitions from a logic high level (H) to a logic low level (L). Therefore, the first exposure period (EX1) may be longer than the second exposure period (EX2).
[0101] That is, when incident light of the same intensity is incident on a long exposure pixel and a short exposure pixel, the amount of photocharge accumulated in the floating diffusion region (FD) of the short exposure pixel may be smaller than the amount of photocharge accumulated in the floating diffusion region (FD) of the long exposure pixel.
[0102] In addition, the amount of photocharge accumulated in the floating diffusion region (FD) of a short exposure pixel increases at a relatively low slope in response to an increase in the intensity of the incident light, and the amount of photocharge accumulated in the floating diffusion region (FD) of a long exposure pixel increases at a relatively high slope in response to an increase in the intensity of the incident light.
[0103] Since the amount of photocharge accumulated in the floating diffusion region (FD) of the short exposure pixel and the amount of photocharge accumulated in the floating diffusion region (FD) of the long exposure pixel are each converted into a pixel signal, the response of the short exposure pixel follows the response of the low-sensitivity pixel shown in FIG. 2, and the response of the long exposure pixel follows the response of the high-sensitivity pixel shown in FIG. 2.
[0104] According to an image sensing device (100) according to another embodiment of the present invention, low-sensitivity pixels and high-sensitivity pixels can be implemented together within a single pixel array (110), so that an HDR image can be generated as a single image.
[0105] FIG. 7 is a diagram illustrating an embodiment of controlling pixel sensitivity by the difference in conversion gain.
[0106] Referring to FIG. 7, it is assumed that a high CG pixel with a relatively large conversion gain and a low CG pixel with a relatively small conversion gain are placed together in one row of a pixel array (110). Since the pixel array (110) is driven row by row driver (120), the high CG pixel and the low CG pixel can receive the same reset signal (RG) and row select signal (SEL). Additionally, FIG. 7 assumes that the high CG pixel and the low CG pixel receive the same transmission signal (TG), but it is also possible for them to receive different transmission signals (TG_L, TG_S) respectively, as in FIG. 6. The high CG pixel and the low CG pixel can receive different first CG signals (CG_H) and second CG signals (CG_L) to have different conversion gains.
[0107] Each of the high CG pixel and low CG pixel operation periods may include a reset period (RS), an exposure period, and a readout period (RD). Since the operation in each of the reset period (RS), exposure period, and readout period (RD) is substantially the same as described in FIG. 6, a redundant description will be omitted.
[0108] During the reset period (RS), exposure period, and readout period (RD), the high CG pixel may receive a first CG signal (CG_H) that maintains a logic low level (L), and the low CG pixel may receive a second CG signal (CG_L) that maintains a logic high level (H). Accordingly, the capacitance of the floating diffusion region (FD) of the high CG pixel corresponds to the capacitance of the first capacitor (C1), and the capacitance of the floating diffusion region (FD) of the low CG pixel may correspond to the sum of the capacitance of the first capacitor (C1) and the capacitance of the second capacitor (C2).
[0109] In the readout period (RD), photocharges accumulated in the floating diffusion region (FD) cause a voltage change in the floating diffusion region (FD), and this voltage change in the floating diffusion region (FD) can be converted into an electrical signal by the source follower transistor (SF). At this time, the magnitude of the voltage change in the floating diffusion region (FD) can be determined by the capacitance of the floating diffusion region (FD). For the same amount of photocharge, the smaller the capacitance of the floating diffusion region (FD), the greater the voltage change in the floating diffusion region (FD), and the larger the capacitance of the floating diffusion region (FD), the smaller the voltage change in the floating diffusion region (FD).
[0110] That is, if the same amount of photocharge is accumulated in the floating diffusion region (FD) from high CG pixels and low CG pixels, the magnitude of the pixel signal generated by the high CG pixels may be greater than the magnitude of the pixel signal generated by the low CG pixels.
[0111] In addition, the magnitude of the pixel signal of low CG pixels increases with a relatively low slope in response to the increase in photocharge, and the magnitude of the pixel signal of high CG pixels increases with a relatively high slope in response to the increase in photocharge.
[0112] Since the magnitude of the pixel signal of the low CG pixel and the magnitude of the pixel signal of the high CG pixel are each converted into image data (IDATA), the response of the low CG pixel follows the response of the low-sensitivity pixel shown in FIG. 2, and the response of the high CG pixel follows the response of the high-sensitivity pixel shown in FIG. 2.
[0113] According to an image sensing device (100) according to another embodiment of the present invention, low-sensitivity pixels and high-sensitivity pixels can be implemented together within a single pixel array (110), so that an HDR image can be generated as a single image.
[0114] FIG. 8 is a diagram illustrating an embodiment of controlling the sensitivity of a pixel by a difference in analog gain.
[0115] Referring to FIG. 8, the ADC (140) may be a ramp-compare type ADC. In one embodiment, the ramp-compare type ADC may include a comparator that compares a ramp signal falling over time with an analog pixel signal, and a counter that performs a counting operation until the ramp signal is matched with the analog pixel signal. Additionally, ADCs (140) may be provided independently for each column line in which pixels belonging to the same column of the pixel array (110) are connected. Each ADC (140) may perform analog-to-digital conversion using the same or different ramp signals.
[0116] The X-axis of the graph shown in FIG. 8 may represent time, and the Y-axis may represent voltage. FIG. 8 shows first and second ramp signals (RAMP1, RAMP2), and each of the first and second ramp signals (RAMP1, RAMP2) may have a form in which it maintains a constant voltage until the first time point (t1), and then decreases linearly after the first time point (t1).
[0117] The slope of the first ramp signal (RAMP1) may be smaller than the slope of the second ramp signal (RAMP2). The slopes of the first and second ramp signals (RAMP1, RAMP2) may be adjusted by controlling the resistance value of a variable resistor included in the ramp circuit that generates the ramp signal according to the control of the timing controller (170), but the scope of the present invention is not limited thereto.
[0118] The interval from a first time point (t1), at which each of the first and second ramp signals (RAMP1, RAMP2) begins to decrease linearly, to a second time point (t2), after a predetermined time has elapsed, can be defined as a countable range. The countable range is the maximum time during which the counter of the ADC (140) can perform a counting operation, and can be determined according to the maximum number of counts of the counter and can represent the output range of the ADC (140).
[0119] The input range of the ADC (140) may mean a voltage range of a pixel signal that can be effectively converted into image data (IDATA) within a predetermined output range of the ADC (140) (e.g., digital number (DN) from 0 to 1023).
[0120] When the first ramp signal (RAMP1) is input to the comparator, the voltage range of the pixel signal that can be effectively converted into pixel data within the countable time determined by the output range of the ADC (140) may correspond to the first input range (IR1). In the example of FIG. 8, when the first ramp signal (RAMP1) and the pixel signal (PS) are input to the comparator, the counter starts counting from the first time point (t1), performs counting until the third time point (t3) at which the pixel signal (PS) begins to have a value greater than or equal to the first ramp signal (RAMP1), and then outputs the accumulated counting value as image data. The pixel signal (PS) can be effectively converted into image data as a signal within the first input range (IR1). Here, being effectively converted into image data may mean being converted into image data representing the voltage of the pixel signal (PS). If the pixel signal (PS) has a voltage lower than the voltage of the first ramp signal (RAMP1) at the second time point (t2), the pixel signal (PS) cannot be effectively converted into pixel data.
[0121] When the second ramp signal (RAMP2) is input to the comparator, the voltage range of the pixel signal that can be effectively converted into pixel data within the countable time determined by the output range of the ADC (140) may correspond to the second input range (IR2). In the example of FIG. 8, when the second ramp signal (RAMP2) and the pixel signal (PS) are input to the comparator, the counter starts counting from the first time point (t1) and performs counting until the fourth time point (t4) at which the pixel signal (PS) begins to have a value greater than or equal to the second ramp signal (RAMP2), and then outputs the accumulated counting value as image data.
[0122] A pixel connected to an ADC (140) that performs analog-to-digital conversion using a first ramp signal (RAMP1) is defined as a high AG pixel, and a pixel connected to an ADC (140) that performs analog-to-digital conversion using a second ramp signal (RAMP2) is defined as a low AG pixel.
[0123] That is, for a pixel signal (PS) having the same voltage, the image data output by the ADC (140) connected to the high AG pixel may be larger than the image data output by the ADC (140) connected to the low AG pixel.
[0124] In addition, the image data output by the ADC (140) connected to the low AG pixel increases at a relatively low slope in response to the increase in the pixel signal (wherein the increase means an increase in the absolute value of the pixel signal), and the image data output by the ADC (140) connected to the high AG pixel increases at a relatively high slope in response to the increase in the pixel signal.
[0125] Accordingly, the response of the low AG pixel follows the response of the low-sensitivity pixel shown in FIG. 2, and the response of the high AG pixel follows the response of the high-sensitivity pixel shown in FIG. 2.
[0126] According to an image sensing device (100) according to another embodiment of the present invention, low-sensitivity pixels and high-sensitivity pixels can be implemented together within a single pixel array (110), so that an HDR image can be generated as a single image.
[0127] The embodiments for controlling the sensitivity of a pixel by the difference between the light transmittance, exposure time, conversion gain, and analog gain described in FIGS. 5 to 8 can be combined with each other. For example, the CG transistor (CG) of a low transmittance pixel (LPX) with relatively low light transmittance can be turned on to make the slope of the response smaller, or the CG transistor (CG) of a low transmittance pixel (LPX) can be turned off to make the slope of the response larger. In addition, the sensitivity of a pixel can be controlled using at least two of the light transmittance, exposure time, conversion gain, and analog gain.
[0128] Additionally, for convenience of explanation, the light transmittance, exposure time, conversion gain, and analog gain described in FIGS. 5 to 8 are described as having only two types, large and small (e.g., high transmittance pixels or low transmittance pixels), but the scope of the present invention is not limited thereto and may have three or more types (e.g., medium transmittance pixels having an intermediate transmittance between high transmittance pixels or low transmittance pixels).
[0129] FIG. 9 is a flowchart illustrating a method for generating an HDR image of a shooting device according to an embodiment of the present invention. FIG. 10 is a flowchart illustrating step S30 of FIG. 9 in more detail.
[0130] Referring to FIG. 9, the luminance acquisition unit (310) can acquire the luminance of a target area in a scene based on image data (IDATA) generated by the image sensing device (100) (S10).
[0131] The controllable item acquisition unit (320) can acquire at least one controllable item among sensitivity items (e.g., light exposure amount, light transmittance, exposure time, conversion gain and analog gain) (S20). Here, although light transmittance belongs to the sensitivity items, since the light transmittance of each pixel must be fixed in hardware, the controllable item acquisition unit (320) may not consider light transmittance as a controllable item.
[0132] The setting value calculation unit (330) can calculate a setting value for a controllable item based on the brightness of the target area received from the brightness acquisition unit (310) and a controllable item received from the controllable item acquisition unit (320), and generate a control signal (CS) representing the calculated setting value (S30).
[0133] The setting value calculation unit (330) can calculate a setting value for a controllable item so that the pixels of the image sensing device (100) can have an optimal dynamic range. Here, the optimal dynamic range may mean a dynamic range that is as wide as possible while minimizing noise.
[0134] In order to maximize the dynamic range, the amount of photocharge generated in the photoelectric conversion element (PD) is increased to the maximum without saturating the photoelectric conversion element (PD), and image data is generated with the maximum gain (conversion gain and analog gain) without saturating the ADC (140).
[0135] Here, saturation in the photoelectric conversion element (PD) may mean that photoelectric charge is generated in excess of the full well capacity (FWC), which represents the maximum amount of photoelectric charge that the photoelectric conversion element (PD) can generate and accumulate. Additionally, saturation in the ADC (140) may mean that image data corresponding to the upper limit of the output range of the ADC (140) is generated.
[0136] A method to increase the amount of photocharge generated in the photoelectric conversion element (PD) may be a method in which the aperture driver (40) increases the degree of opening of the aperture (20) so that the amount of light exposure increases, or a method in which the low driver (120) increases the time during which the transmission signal has a logic high level so that the exposure time increases.
[0137] A method for increasing the gain for generating image data may be for the low driver (120) to turn on the CG transistor so that the conversion gain is increased, or for the timing controller (170) to decrease the slope of the ramp signal so that the analog gain is increased.
[0138] The setting value calculation unit (330) can determine whether each pixel included in the target area is saturated based on image data of the target area received from the luminance acquisition unit (310). The saturation of each pixel is a concept that includes saturation in the photoelectric conversion element (PD) and saturation in the ADC (140). In this disclosure, it is assumed that if the image data of each pixel corresponds to the upper limit value of the output range of the ADC (140), the pixel is determined to be saturated.
[0139] Additionally, the setting value calculation unit (330) can determine whether each pixel can be saturated through control of controllable items. To this end, the setting value calculation unit (330) can store in advance response characteristics in which the response of the corresponding pixel changes in response to controlling each of the controllable items (light exposure amount, exposure time, conversion gain, analog gain) to a specific setting value. These response characteristics can be experimentally determined by measuring the response of the pixel while changing the setting value of each item. For example, the setting value calculation unit (330) can determine whether the high transmittance pixel can be saturated when the setting value of the controllable items (e.g., light exposure amount, exposure time, conversion gain, analog gain) is controlled to maximize the sensitivity of the high transmittance pixel, based on the current image data of the high transmittance pixel within the target area and the response characteristics of the high transmittance pixel. If high transmittance pixels cannot be saturated, the current illuminance is very low, so even if the exposure time is shortened like a single exposure pixel, the effect of expanding the dynamic range cannot be expected, and instead, the signal-to-noise ratio may only worsen. Therefore, the setting value calculation unit (330) can exclude the exposure time from the controllable items and set the exposure time to the longest time.
[0140] According to another embodiment, if the average value (i.e., average luminance) or maximum value (i.e., maximum luminance) of the current image data of high transmittance pixels within the target area is smaller than a predetermined value (e.g., night shooting, video shooting, etc.), the setting value calculation unit (330) determines that the environment is one with very low illumination, excludes the exposure time from controllable items, and sets the exposure time to the longest time, which is the only value.
[0141] The embodiments for controlling pixel sensitivity presented in this disclosure may each have contrasting advantages and disadvantages.
[0142] A method of controlling the sensitivity of a pixel by controlling light transmittance allows for the spatial control of the amount of light incident on the pixel, so that even at high illumination levels where high transmittance pixels become saturated, low transmittance pixels do not become saturated and can generate an effective response, thereby substantially increasing the FWC and relatively greatly expanding the dynamic range of the pixel. However, light transmittance is a fixed item during the manufacturing of the image sensing device (100) and cannot be dynamically controlled.
[0143] The method of controlling pixel sensitivity by controlling exposure time allows for the generation of a valid response even at high illumination levels where long-exposure pixels become saturated, as low-exposure pixels do not become saturated by temporally controlling the amount of light incident on the pixels. This effectively increases the FWC and can relatively significantly expand the dynamic range of the pixels. However, if exposure time is excessively controlled, the timing of capturing incident light between pixels will vary significantly, which can lead to an increase in motion artifacts for fast-moving subjects and a significant deterioration in image quality.
[0144] A method of controlling pixel sensitivity by controlling the conversion gain allows for obtaining an amplified voltage according to a constant gain even in low-light conditions where generated photocharges are insufficient, thereby effectively suppressing noise by adjusting the gain through which photocharges generated in the pixel are converted into a pixel signal (i.e., voltage). However, for pixels where the photoelectric conversion element (PD) is saturated, an effective response cannot be obtained even by controlling the conversion gain, which limits the performance in extending the dynamic range of the pixel.
[0145] The method of controlling the sensitivity of a pixel by controlling the analog gain allows for obtaining amplified image data according to a certain gain even in low light conditions where the generated photocharge is insufficient, thereby having the effect of suppressing noise by controlling the gain through which the pixel signal is converted into image data (i.e., digital value). However, for pixels where the photoelectric conversion element (PD) is saturated, an effective response cannot be obtained even by controlling the analog gain, so there is a limit to the performance in expanding the dynamic range of the pixel. In addition, if the analog gain is increased, as described in FIG. 8, the input range, which is the voltage range of the pixel signal that can be effectively converted into image data by the ADC (140), is inevitably reduced, so there may be a side effect where the dynamic range is actually reduced depending on the light level.
[0146] The imaging device (1) can expand the dynamic range of image data by considering the characteristics of the items that control the sensitivity of the pixels. According to one embodiment, the imaging device (1) can expand the dynamic range by controlling the sensitivity of each pixel in a progressively decreasing order of sensitivity control using light transmittance, sensitivity control using conversion gain, sensitivity control using exposure time, and sensitivity control using analog gain. This is done considering the performance and side effects of expanding the dynamic range, and the order may be changed according to other embodiments. In addition, the imaging device (1) can calculate the setting value of the controllable item so that the intensity of light incident on each pixel is as large as possible when controlling the sensitivity of the pixels.
[0147] Referring to FIG. 10, an example of how a set value calculation unit (330) calculates a set value for a controllable item will be described. For convenience of explanation in this disclosure, it is assumed that the conversion gain and the analog gain are always controllable, and that the light exposure amount and the exposure time may each be controllable or uncontrollable. That is, the controllable item acquisition unit (320) can acquire a controllable item among the light exposure amount and the exposure time.
[0148] In addition, it is assumed that the arrangement of low transmittance pixels and high transmittance pixels follows the form exemplified in FIG. 5. That is, it is assumed that the unit matrix consists of one low transmittance pixel and three high transmittance pixels.
[0149] The setting value calculation unit (330) can determine whether the amount of light exposure is a controllable item (S300).
[0150] If the amount of light exposure is not a controllable item (No of S300), the setting value calculation unit (330) can determine whether the exposure time is a controllable item (S310).
[0151] If the exposure time is a controllable item (Yes in S310), the setting value calculation unit (330) can perform a first setting control (S320). The first setting control may refer to a control method for controllable items under conditions where the amount of light exposure is an uncontrollable item, and the exposure time, conversion gain, and analog gain are controllable items.
[0152] Specifically, the setting value calculation unit (330) can determine whether a required dynamic range can be obtained in the target area by controlling the conversion gain for high transmittance pixels and low transmittance pixels. Here, the required dynamic range may mean a dynamic range suitable for capturing the target area.
[0153] According to one embodiment, if, through the control of the conversion gain for high transmittance pixels and low transmittance pixels, the number of high transmittance pixels having a response of less than or equal to a predetermined value in the target area is less than a first ratio and the number of saturated low transmittance pixels in the target area is less than or equal to a second ratio, the setting value calculation unit (330) may determine that the required dynamic range can be obtained in the target area. Conversely, even through the control of the conversion gain for high transmittance pixels and low transmittance pixels, if the number of high transmittance pixels having a response of less than or equal to a predetermined value in the target area is greater than or equal to the first ratio or the number of saturated low transmittance pixels in the target area exceeds the second ratio, the setting value calculation unit (330) may determine that the required dynamic range cannot be obtained in the target area. Here, the first ratio may refer to the ratio of the number of high transmittance pixels having a response of less than or equal to a predetermined value to the total number of high transmittance pixels in the target area. The second ratio may refer to the ratio of the number of saturated low transmittance pixels to the total number of low transmittance pixels in the target area. The first ratio and the second ratio can be experimentally determined in advance according to the required dynamic range. For example, a predetermined value may mean the lower limit value of the dynamic range (DR_H) of the high-sensitivity pixel described in FIG. 2. Additionally, a condition in which high-transmittance pixels having a response of less than or equal to a predetermined value in the target area become less than the first ratio may be defined as the first condition, and a condition in which saturated low-transmittance pixels in the target area become less than or equal to the second ratio may be defined as the second condition.
[0154] According to another embodiment, the setting value calculation unit (330) may receive information regarding whether a required dynamic range can be obtained from another configuration of the image signal processor (200) (e.g., a configuration that synthesizes an HDR image).
[0155] In the first setting control, if the required dynamic range in the target area can be obtained through the control of the conversion gain for high transmittance pixels and low transmittance pixels, the setting value calculation unit (330) can set the maximum exposure time and the optimal analog gain for the high transmittance pixels and low transmittance pixels. Here, the maximum exposure time may be the maximum exposure time in the range where the high transmittance pixels having a response of less than or equal to a predetermined value in the target area become less than the first ratio and the low transmittance pixels saturated in the target area become less than or equal to the second ratio. Additionally, the minimum analog gain may be the lowest analog gain that can be set in the ADC (140). This is to prevent the dynamic range from being unintentionally limited, as the input range of the ADC (140) can be expanded as the analog gain is lower.
[0156] If the required dynamic range in the target area cannot be obtained by controlling the conversion gain for high transmittance pixels and low transmittance pixels, the set value calculation unit (330) can extend the dynamic range by controlling the exposure time. The length of the first exposure section (EX1) of the long exposure pixel described in FIG. 6 is defined as the first exposure time, and the length of the second exposure section (EX2) of the short exposure pixel is defined as the second exposure time.
[0157] The setting value calculation unit (330) can determine a first exposure time such that high transmittance pixels having a response of less than or equal to a predetermined value in the target area are less than a first ratio. Additionally, the setting value calculation unit (330) can determine a second exposure time such that saturated low transmittance pixels in the target area are less than or equal to a second ratio. However, the difference between the first exposure time and the second exposure time may be set to a minimum to reduce motion artifacts. In order to maximize the dynamic range by setting the sensitivity difference between low transmittance pixels and high transmittance pixels to the maximum using the first exposure time and the second exposure time having a minimum difference, the second exposure time may be set for low transmittance pixels and the first exposure time may be set for high transmittance pixels.
[0158] In addition, the conversion gain of the high CG pixel described in FIG. 7 is defined as the first conversion gain, and the conversion gain of the low CG pixel is defined as the second conversion gain. Furthermore, the analog gain of the high AG pixel described in FIG. 8 is defined as the first analog gain, and the analog gain of the low AG pixel is defined as the second analog gain. For convenience of explanation, the second analog gain is assumed to be the same as the lowest analog gain described earlier.
[0159] The setting value calculation unit (330) can set a second conversion gain, a second exposure time, and a second analog gain for a low transmittance pixel. Here, setting the second conversion gain and the second exposure time for a low transmittance pixel is intended to extend the dynamic range of the low transmittance pixel, and setting the second analog gain for a low transmittance pixel is intended to prevent the dynamic range from being limited.
[0160] The setting value calculation unit (330) can set a plurality of conversion gains including a first conversion gain, a plurality of exposure times including a first exposure time, and a second analog gain for high transmittance pixels. That is, the plurality of conversion gains can be set to include the first conversion gain and the plurality of exposure times can be set to include the first exposure time to expand the dynamic range of the high transmittance pixels, and setting a second analog gain for the high transmittance pixels is intended to prevent limitations on the dynamic range. In addition, by setting a combination of a plurality of conversion gains and a plurality of exposure times for the high transmittance pixels, the high transmittance pixels can be made to have various dynamic ranges. When image data of high transmittance pixels having various dynamic ranges is synthesized in this way, only image data with an excellent signal-to-noise ratio can be synthesized to generate an HDR image, thereby improving the quality of the image. For example, in a unit matrix, the setting value calculation unit (330) can set a first conversion gain and a first exposure time for one high transmittance pixel, set a second conversion gain and a second exposure time for another high transmittance pixel, and set a second conversion gain and a first exposure time for the remaining high transmittance pixel.
[0161] If the exposure time is not a controllable item (No of S310), the setting value calculation unit (330) can determine whether the low transmittance pixels saturated in the target area exceed the second ratio (S330).
[0162] If the low transmittance pixels saturated in the target area exceed the second ratio (Yes in S330), the set value calculation unit (330) can perform second setting control (S340). The second setting control may refer to a control method for items that are controllable under conditions where the light exposure amount and exposure time are uncontrollable items, the conversion gain and analog gain are controllable items, and the low transmittance pixels saturated in the target area exceed the second ratio. These conditions may refer to relatively high illumination conditions where even low transmittance pixels are saturated above a certain ratio.
[0163] The setting value calculation unit (330) can set a first conversion gain or a second conversion gain in a predetermined pattern for low transmittance pixels and high transmittance pixels. The setting value calculation unit (330) can set a first analog gain for pixels set to the first conversion gain. The setting value calculation unit (330) can set a second analog gain for pixels set to the second conversion gain. This is to achieve a large possible difference in sensitivity using the conversion gain and the analog gain. However, if pixels set to the first conversion gain are set to the first analog gain, the dynamic range may be limited. According to another embodiment, the setting value calculation unit (330) may set a second analog gain for pixels set to the first conversion gain, or selectively set the first analog gain or the second analog gain.
[0164] If the low transmittance pixels saturated in the target area are less than or equal to the second ratio (No of S330), the setting value calculation unit (330) can determine whether the high transmittance pixels having a response less than or equal to a predetermined value in the target area are less than the first ratio (S350).
[0165] If the high transmittance pixel having a response of less than or equal to a predetermined value in the target area is less than the first ratio (Yes of S350), the set value calculation unit (330) can perform a third setting control (S360). The third setting control may refer to a control method for items that are controllable under conditions where the light exposure amount and exposure time are uncontrollable items, the conversion gain and analog gain are controllable items, and the low transmittance pixel saturated in the target area is less than the second ratio and the high transmittance pixel having a response of less than or equal to a predetermined value is less than the first ratio. These conditions may refer to appropriate illumination conditions where the low transmittance pixel is saturated at less than the second ratio and the high transmittance pixel has a response of less than or equal to a predetermined value at less than the first ratio.
[0166] The setting value calculation unit (330) can determine whether the saturated low transmittance pixel in the target area will be maintained at a second ratio or lower when setting a first conversion gain for the low transmittance pixel. If the saturated low transmittance pixel can be maintained at a second ratio or lower, the setting value calculation unit (330) can set a first conversion gain for the low transmittance pixel. If the saturated low transmittance pixel exceeds the second ratio, the setting value calculation unit (330) can set a second conversion gain for the low transmittance pixel.
[0167] Meanwhile, the setting value calculation unit (330) can set a first conversion gain or a second conversion gain for high transmittance pixels in a predetermined pattern. Here, the predetermined pattern may be a pattern in which a high transmittance pixel belonging to the same row as a low transmittance pixel is set to the same conversion gain as a low transmittance pixel, and a high transmittance pixel belonging to a different row from a low transmittance pixel is set to a different conversion gain as a low transmittance pixel, but the scope of the present invention is not limited thereto.
[0168] With the conversion gain set for the low transmittance pixels and high transmittance pixels as described above, the setting value calculation unit (330) can set the maximum possible analog gain for the low transmittance pixels and high transmittance pixels within a range in which the condition is maintained that the low transmittance pixels saturated in the target area are at or below the second ratio and the high transmittance pixels having a response of or below a predetermined value are at or below the first ratio. This is to obtain a wide dynamic range while suppressing noise as much as possible.
[0169] If the number of high transmittance pixels having a response of less than or equal to a predetermined value in the target area is greater than or equal to the first ratio (No of S350), the set value calculation unit (330) can perform a fourth setting control (S370). The fourth setting control may refer to a control method for items that are controllable under conditions where the light exposure amount and exposure time are uncontrollable items, the conversion gain and analog gain are controllable items, and the number of low transmittance pixels saturated in the target area is less than or equal to the second ratio, and the number of high transmittance pixels having a response of less than or equal to a predetermined value is greater than or equal to the first ratio. These conditions may refer to relatively low illumination conditions where the high transmittance pixels have a response of less than or equal to a predetermined value at a ratio greater than or equal to the first ratio.
[0170] The setting value calculation unit (330) can set a first conversion gain for low transmittance pixels.
[0171] Meanwhile, the setting value calculation unit (330) can set a first conversion gain or a second conversion gain for high transmittance pixels in a predetermined pattern. Here, even if the high transmittance pixels are set to the first conversion gain, if it is determined that they have a response of less than or equal to a predetermined value at a first ratio, the setting value calculation unit (330) can set a first conversion gain for the high transmittance pixels that is not in a predetermined pattern. This is because, in such cases, even if a second conversion gain is set for the high transmittance pixels, an expansion of the dynamic range cannot be expected.
[0172] With the conversion gain set for the low transmittance pixels and high transmittance pixels as described above, the setting value calculation unit (330) can set the highest possible analog gain for the low transmittance pixels and high transmittance pixels as long as the condition is maintained that the low transmittance pixels saturated in the target area are at or below the second ratio. This is to obtain the widest possible dynamic range while suppressing noise as much as possible.
[0173] If the amount of light exposure is a controllable item (Yes of S300), the setting value calculation unit (330) can set the amount of light exposure so that the aperture (20) is opened as much as possible within the range where the low transmittance pixels saturated in the target area are less than or equal to the second ratio (S380).
[0174] When the amount of light exposure is fixed at a specific set value, the set value calculation unit (330) can determine whether the exposure time is a controllable item (S390).
[0175] If the exposure time is a controllable item (Yes in S390), the setting value calculation unit (330) can perform the first setting control (S320). This is because the exposure time is a controllable item under conditions where the amount of light exposure is fixed and not a controllable item.
[0176] If the exposure time is not a controllable item (No of S390), the setting value calculation unit (330) can perform a third setting control (S360). This is because, under conditions where the light exposure amount is not a fixed controllable item but the exposure time is a controllable item, the aperture (20) is set to open as much as possible within the range where the low transmittance pixels saturated in the target area become less than the second ratio, and there is a high probability that the high transmittance pixels having a response of less than or equal to a predetermined value in the target area will become less than the first ratio.
[0177] Referring again to FIG. 9, the setting value calculation unit (330) can transmit a control signal (CS) representing a setting value for a controllable item to the aperture driver (40) and the image sensing device (100). The aperture driver (40) can control the degree of opening of the aperture (20) to have an amount of light exposure corresponding to the control signal (CS). Additionally, the timing controller (170) of the image sensing device (100) can control the row driver (120) and the ADC (14) so that each pixel has an exposure time, conversion gain, and analog gain corresponding to the control signal (CS).
[0178] The image sensing device (100) captures a scene when control according to the control signal (CS) is completed and transmits image data (IDATA) to the image signal processor (200), and the image signal processor (200) can synthesize at least two images with different sensitivities to generate an HDR image (S40). The type of sensitivity of the image data (IDATA) can be determined in various ways by combining light transmittance, exposure time, conversion gain, and analog gain. For example, pixels included in the pixel array (110) may have any one of five sensitivities, and each of two pixels having different sensitivities may correspond to the previously described low-sensitivity pixel or high-sensitivity pixel according to the relationship between the sensitivities.
[0179] A shooting device (1) according to one embodiment of the present invention can generate an HDR image with a maximum dynamic range while minimizing noise by identifying the brightness and controllable items of a subject and controlling the controllable items by reflecting the hardware characteristics (e.g., response characteristics) when controlling sensitivity.
Claims
Claim 1 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit acquiring the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit acquiring at least one controllable item among sensitivity items determining the sensitivity of each of the plurality of pixels; and a setting value calculation unit generating a control signal by calculating a setting value for the at least one controllable item based on the image data of the target area and the at least one controllable item, wherein the sensitivity items include at least one of light exposure amount, light transmittance, exposure time, conversion gain, and analog gain, and the plurality of pixels include at least one high transmittance pixel having a relatively high light transmittance and at least one low transmittance pixel having a relatively low light transmittance, and the low transmittance pixel includes a light blocking structure that blocks at least a portion of the incident light. Claim 2 A shooting device according to claim 1, further comprising an image synthesis unit that synthesizes image data having different sensitivities generated in a state where the sensitivity of each of the plurality of pixels is adjusted according to the control signal to generate an HDR (high dynamic range) image. Claim 3 delete Claim 4 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit acquiring the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit acquiring at least one controllable item among sensitivity items determining the sensitivity of each of the plurality of pixels; and a setting value calculation unit generating a control signal by calculating a setting value for the at least one controllable item based on the image data of the target area and the at least one controllable item, wherein the sensitivity items include at least one of light exposure amount, light transmittance, exposure time, conversion gain, and analog gain, and further comprising an aperture driver that controls the degree of opening and closing of an aperture disposed at the front of the image sensing device, wherein the light exposure amount is controlled by controlling the degree of opening and closing of the aperture. Claim 5 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit acquiring the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit acquiring at least one controllable item among sensitivity items determining the sensitivity of each of the plurality of pixels; and a setting value calculation unit generating a control signal by calculating a setting value for the at least one controllable item based on the image data of the target area and the at least one controllable item, wherein the sensitivity items include at least one of light exposure amount, light transmittance, exposure time, conversion gain, and analog gain, and further comprising a low driver generating a transmission signal for transmitting a photocharge generated according to the intensity of incident light in each of the plurality of pixels to a floating diffusion area, wherein the exposure time is controlled by adjusting the time during which the transmission signal has a certain level. Claim 6 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit acquiring the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit acquiring at least one controllable item among sensitivity items determining the sensitivity of each of the plurality of pixels; and a setting value calculation unit generating a control signal by calculating a setting value for the at least one controllable item based on the image data of the target area and the at least one controllable item, wherein the sensitivity items include at least one of light exposure amount, light transmittance, exposure time, conversion gain, and analog gain, and further comprising a low driver generating a CG (conversion gain) signal that controls the capacitance of a floating diffusion area that accumulates photocharges generated according to the intensity of incident light in each of the plurality of pixels, and wherein the conversion gain is controlled by adjusting the level of the CG signal. Claim 7 A shooting device according to claim 1, further comprising an analog-digital converter (ADC) that generates image data according to a comparison result by comparing the pixel signal of each of the plurality of pixels with a ramp signal, wherein the analog gain is controlled by adjusting the slope of the ramp signal. Claim 8 delete Claim 9 In claim 1, the above-mentioned setting value calculation unit determines whether each of the target pixels is saturated through control of the at least one controllable item. Claim 10 In claim 9, if the high transmittance pixels included in the target pixels are not saturated, the setting value calculation unit sets the exposure time to a unique value. Claim 11 In claim 1, when the average value of the image data of the high transmittance pixels included in the target pixels is smaller than a predetermined value, the setting value calculation unit sets the exposure time to a unique value. Claim 12 In claim 1, wherein the at least one controllable item includes the exposure time, the conversion gain, and the analog gain, and the amount of light exposure is an uncontrollable item, the setting value calculation unit sets a plurality of conversion gains including a first conversion gain, a plurality of exposure times including a first exposure time, and a second analog gain for the high transmittance pixels, and sets a second conversion gain, a second exposure time, and a second analog gain for the low transmittance pixels, wherein the first conversion gain is greater than the second conversion gain, the first exposure time is longer than the second exposure time, and the second analog gain is smaller than the first analog gain. Claim 13 In claim 12, the above-mentioned setting value calculation unit determines the first exposure time such that the high transmittance pixels having a response of less than or equal to a predetermined value in the target area become less than the first ratio, and determines the second exposure time such that the low transmittance pixels saturated in the target area become less than or equal to the second ratio. Claim 14 A shooting device according to claim 13, wherein at least one controllable item includes the conversion gain and the analog gain, and the light exposure amount and the exposure time are uncontrollable items, and when the low transmittance pixels saturated in the target area exceed the second ratio, the setting value calculation unit sets the first conversion gain or the second conversion gain in a predetermined pattern for the low transmittance pixels and the high transmittance pixels, and the setting value calculation unit sets the first analog gain for the pixels set with the first conversion gain and sets the second analog gain for the pixels set with the second conversion gain. Claim 15 In claim 13, wherein the at least one controllable item includes the conversion gain and the analog gain, and the light exposure amount and the exposure time are uncontrollable items, and the low transmittance pixels saturated in the target area are at or below the second ratio, and the high transmittance pixels having a response at or below the predetermined value in the target area are at or below the first ratio, the setting value calculation unit sets the first conversion gain or the second conversion gain for the low transmittance pixels according to whether the low transmittance pixels saturated in the target area can be maintained at or below the second ratio when the first conversion gain is set for the low transmittance pixels, and sets the first conversion gain or the second conversion gain for the high transmittance pixels in a predetermined pattern, and the setting value calculation unit sets the maximum analog gain for the low transmittance pixels and the high transmittance pixels within the range in which the condition is maintained that the low transmittance pixels saturated in the target area are at or below the second ratio and the high transmittance pixels having a response at or below the predetermined value in the target area are at or below the first ratio. A shooting device for setting. Claim 16 A shooting device according to claim 13, wherein at least one controllable item includes the conversion gain and the analog gain, and the light exposure amount and the exposure time are uncontrollable items, and when the low transmittance pixels saturated in the target area are at least the second ratio and the high transmittance pixels having a response at least the predetermined value in the target area are at least the first ratio, the setting value calculation unit sets the first conversion gain for the low transmittance pixels and sets the first conversion gain or the second conversion gain for the high transmittance pixels in a predetermined pattern, and the setting value calculation unit sets the maximum analog gain for the low transmittance pixels and the high transmittance pixels within the range in which the condition that the low transmittance pixels saturated in the target area are at least the second ratio is maintained. Claim 17 In claim 13, when the above at least one controllable item includes the light exposure amount, the setting value calculation unit sets the light exposure amount such that the aperture is opened to the maximum extent within the range where the low transmittance pixel saturated in the target area becomes less than or equal to the second ratio. Claim 18 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; a luminance acquisition unit acquiring the image data of a target area corresponding to target pixels among the plurality of pixels; a controllable item acquisition unit acquiring at least one controllable item among sensitivity items determining the sensitivity of each of the plurality of pixels; and a setting value calculation unit generating a control signal by calculating a setting value for the at least one controllable item based on the image data of the target area and the at least one controllable item, wherein the sensitivity items include at least one of light exposure amount, light transmittance, exposure time, conversion gain, and analog gain, and the plurality of pixels include at least one high transmittance pixel having a relatively high light transmittance and at least one low transmittance pixel having a relatively low light transmittance, and the low transmittance pixel includes a light blocking structure that blocks at least a portion of the incident light, and the setting value calculation unit calculates the setting value so that the sensitivity of each of the plurality of pixels is adjusted in a priority order of decreasing in the order of light transmittance, conversion gain, exposure time, and analog gain. Claim 19 An image sensing device comprising a plurality of pixels, each generating a pixel signal corresponding to the intensity of incident light, and generating image data corresponding to the pixel signal; an HDR controller that calculates a set value for at least one controllable item among light exposure amount, exposure time, conversion gain, and analog gain based on the image data and generates a control signal; and an image signal processor that synthesizes image data having different sensitivities generated in a state where the sensitivity of each of the plurality of pixels is adjusted according to the control signal to generate an HDR (high dynamic range) image, wherein the plurality of pixels include at least one high transmittance pixel having a relatively high light transmittance and at least one low transmittance pixel having a relatively low light transmittance, and the low transmittance pixel includes a light blocking structure that blocks at least a portion of the incident light.
Citation Information
Patent Citations
Imaging device, imaging apparatus and imaging signal processing method
JP2017216646A
Imaging apparatus and imaging system
JP2019106574A
Imaging system for generating HDR image and operating method thereof
KR1020200090291A
Image sensor and driving method thereof
KR1020200098764A
