Imaging apparatus and method for controlling the same, program, and storage medium
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2023-05-24
- Publication Date
- 2026-06-17
AI Technical Summary
Existing imaging devices face challenges in achieving different gain outputs for pixel signals without increasing circuit size, particularly when using dual slope AD converters.
The imaging device employs a control mechanism that allows AD conversion with multiple reference signals of varying slopes to obtain different gain outputs for the same pixel, utilizing a first mode for single conversion and a second mode for multiple conversions to achieve desired gain levels without enlarging the circuit scale.
This approach enables high-quality high dynamic range imaging without increasing the circuit size, allowing for faster readout speeds and improved image quality through precise gain adjustments.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging apparatus and a control method thereof. [Background technology]
[0002] Conventionally, as a pixel signal readout circuit in an image sensor, a configuration is known in which pixel signals output from pixels to column output lines are AD converted (analog / digital converted) in a column circuit provided for each pixel column.
[0003] Patent Document 1 describes a dual-slope AD converter that increases the readout speed by switching the slope of a ramp signal depending on the signal level when performing AD conversion on a pixel signal.
[0004] Furthermore, Patent Document 2 discloses a technique for expanding the dynamic range by simultaneously amplifying signals from the same pixel with different gains. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2022-144244 A [Patent Document 2] JP 2015-128253 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, performing dual-slope AD conversion to increase the readout speed while also attempting to obtain outputs with different gains to expand the dynamic range poses the problem of an increase in the size of circuits such as wiring and selectors within the imaging device.
[0007] The present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an imaging device that is equipped with a dual-slope AD converter and is capable of obtaining outputs with different gains for signals of the same pixel without increasing the circuit size. [Means for solving the problem]
[0008] The imaging device of the present invention is characterized by comprising: a pixel that performs photoelectric conversion; an AD converter that performs AD conversion of the pixel signal; and control means for controlling the AD converter to perform AD conversion in either a first AD conversion mode in which the AD converter obtains an image signal by comparing the pixel signal with one of a plurality of reference signals having different slopes of the signal level change over time; or a second AD conversion mode in which the signal of one and the same pixel is compared with a plurality of reference signals having different slopes of the signal level change over time to perform multiple AD conversions to obtain a plurality of image signals with different gains. Effect of the Invention
[0009] According to the present invention, even with a dual-slope AD converter, it is possible to obtain outputs with different gains for signals of the same pixel without increasing the circuit scale. [Brief description of the drawings]
[0010] [Figure 1] 1 is a block diagram showing the configuration of an imaging apparatus according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a simplified diagram of the configuration of an image sensor. [Diagram 3] FIG. 2 is an equivalent circuit diagram of a unit pixel. [Figure 4] FIG. 2 is a diagram showing the configuration of an AD converter of the image sensor. [Diagram 5] FIG. 4 is a diagram showing shooting modes in the first embodiment. [Figure 6] 11 is a timing chart showing the operation of DS_AD at a low signal level. [Figure 7]11 is a timing chart showing the operation of DS_AD at a high signal level. [Figure 8] 11 is a timing chart showing the operation of HDR_SS_AD at a low gain. [Figure 9] 1 is a timing chart showing the operation of HDR_SS_AD at high gain. [Figure 10] FIG. 11 is a diagram showing shooting modes in the second embodiment. [Figure 11] 11 is a timing chart showing the operation of HDR_DS_AD at a low signal level. [Figure 12] 11 is a timing chart showing the operation of HDR_DS_AD at a high signal level. [Figure 13] FIG. 11 is a diagram showing shooting modes in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0012] (First embodiment) FIG. 1 is a block diagram showing the configuration of an image capturing apparatus 100 according to the first embodiment of the present invention.
[0013] 1, a lens unit 101 collects incident light from a subject and forms an image on an image sensor 102. The lens unit 101 is configured to include, for example, a plurality of lenses, an aperture, etc. The image sensor 102 photoelectrically converts the subject image formed by the lens unit 101 and outputs image data.
[0014] The control unit 112 controls the entire imaging device 100, including the driving of the imaging element 102 and the image processing circuit 109. The control unit 112 also controls the reception of instructions from the display unit 113 and the operation unit 116, and the transfer of data to the memory circuit 111 and the recording unit 110. The control unit 112 may be provided in the imaging element 102 or the image processing circuit 109.
[0015] The image processing circuit 109 performs development processes such as color matrix processing and gamma processing on the image data from the image sensor 102. In these processes, the image processing circuit 109 stores the image data in a memory circuit 111 as necessary. Then, the image processing circuit 109 outputs the processed image data to a display unit 113 or a recording unit 110.
[0016] The display unit 113 displays image data and the like output from the image processing circuit 109. The operation unit 116 sends an operation signal to the control unit 112 in accordance with a user's operation.
[0017] The bus 120 is a common path for the image sensor 102, image processing circuit 109, display unit 113, operation unit 116, recording unit 110, and memory circuit 111 to exchange data with one another.
[0018] FIG. 2 is a simplified configuration diagram of the image sensor 102.
[0019] A plurality of unit pixels 205 are arranged two-dimensionally (in a matrix) in the pixel region 208. The vertical arrangement of the unit pixels 205 in the pixel region 208 is called a "column", and the horizontal arrangement is called a "row".
[0020] The pixel signals output from each unit pixel 205 are output to a column output line 210, and further input to a column circuit 220. An analog / digital converter (hereinafter, AD converter) 220a is arranged in the column circuit 220. Details of the column circuit including the AD converter 220a will be described later with reference to FIG.
[0021] The AD converter 220a is connected to a reference signal generator 221 and a timing control unit 222. The reference signal generator 221 supplies a reference signal (ramp signal) to the AD converter 220a via a plurality of reference signal lines 223, 224, 225, and 226. The timing control unit 222 supplies a clock for counting via a clock line 227, and supplies a count reset signal via a reset signal line 228.
[0022] Furthermore, a register setting unit 230 is disposed in the image sensor 102 , and a shooting mode is designated by the control unit 112 , and the designated mode signal is output to a reference signal generator 221 .
[0023] The digital signal output from the column circuit 220 is input to a digital signal processing unit 211 where it is subjected to processes such as dark offset correction and gain correction, and is then output to the outside of the image sensor 102 via an output unit 212 .
[0024] FIG. 3 is an equivalent circuit diagram of the unit pixel 205. As shown in FIG.
[0025] Charges generated and accumulated in a photodiode 305 in response to incident light are transferred to a floating diffusion section (hereinafter, FD section) 307 by turning on a transfer switch 306 using a transfer control signal 301. A source follower amplifier 309 is configured together with a constant current source 311 connected to a column output line 210, and amplifies a voltage signal based on the charges accumulated in the FD section 307 and outputs it as a pixel signal. The output of the source follower amplifier 309 is output to the column output line 210 by turning on a row selection switch 310 using a row selection control signal 303.
[0026] When resetting unnecessary charges accumulated in the FD 307, the reset switch 308 is turned on by the reset control signal 302. Furthermore, when resetting the photodiode 305, the reset switch 308 is turned on and the transfer switch 306 is turned on by the transfer control signal 301. The transfer control signal 301, the reset control signal 302, and the row selection control signal 303 are output from the vertical scanning circuit 204 in response to commands from the control unit 112.
[0027] Fig. 4 is a diagram showing a circuit configuration of an AD converter of an image sensor. Fig. 4 shows a k-th column circuit 220_k and a k+1-th column circuit 220_k+1. Since the column circuits of each column have the same configuration, the k-th column circuit 220_k (sometimes simply referred to as column circuit 220) will be described as an example here. In addition, in terms of AD conversion methods, a method of AD conversion using two ramp signals with different slopes depending on the signal level will be described as Dual_Slope_AD (hereinafter, DS_AD), and a method of AD conversion using a single ramp signal will be described as Single_Slope_AD (hereinafter, SS_AD).
[0028] A reference signal generator 221 supplies a luminance determination reference signal 223, a DS_AD low luminance reference signal 224, a DS_AD high luminance reference signal 225, and an SS_AD low luminance reference signal 226 to each column circuit 220. These reference signals have different slopes of their signal waveforms. The switch SW400_k is a switch that selects either the luminance determination reference signal 223 or a reference signal selected by the reference signal selection unit 411_k.
[0029] The pixel signal from the column output line 210k is input to one side of the comparator 410_k, and a reference signal selected by SW400_k is input to the other side. The comparison result of the comparator 410_k is input to a signal level determination unit 412_k via a switch SW401_k. Similarly, the comparison result of the comparator 410_k is input to a counter circuit 413_k via a switch SW402_k.
[0030] In the counter circuit 413_k, a counter is driven in synchronization with the reference signal. In addition, a clock signal 227 for operating the counter circuit 413_k is supplied from the timing control section 222. Furthermore, a reset signal 228 for resetting the counter in the counter circuit 413_k is also supplied from the timing control section 222. The output of the signal level determination section 412_k is output to a reference signal selection section 411_k.
[0031] Furthermore, a mode setting signal for setting the shooting mode is input to the reference signal generator 221 and is reflected in the selection of the reference signal.
[0032] Next, the shooting modes in this embodiment will be described with reference to FIG.
[0033] As already explained, in FIG. 5, DS_AD (dual slope AD conversion) is a method of performing AD conversion by switching between ramp signals with two different slopes, whose signal level changes over time in a triangular waveform, depending on the signal level of the pixel signal.
[0034] SS_AD is a method of performing AD conversion using a ramp signal with one type of slope, in which the signal level changes in a triangular waveform.
[0035] HDR (high dynamic range) photography is a photography method that obtains a single image signal with an expanded dynamic range by synthesizing image signals of multiple images (two images in this embodiment) with different exposures. Note that normal photography in the following refers to photography that obtains an SDR (standard dynamic range) image.
[0036] In Figure 5, normal bit DS_AD is used for AD conversion in normal shooting mode. In addition, in HDR_SS mode (high dynamic range single slope mode), which is another AD conversion mode, low gain high bit SS_AD and high gain high bit SS_AD are performed. In other words, AD conversion is performed twice for the output signals of all pixels, and two images are synthesized after AD conversion. Normal bit DS_AD allows high speed readout. In contrast, HDR_SS mode performs high bit SS_AD with an increased number of bits twice, so although the readout time is slightly longer, it is possible to obtain high definition images.
[0037] Fig. 6 is a timing chart showing the operation of DS_AD when the image signal is at a low level in the normal bit DS_AD during normal shooting. Fig. 7 is a timing chart showing the operation of DS_AD when the pixel signal is at a high level. In DS_AD, the operation shown in either the timing chart of Fig. 6 or Fig. 7 is performed according to the signal level of each pixel. First, the operation of the DS_AD when the image signal is at a low level will be described with reference to the timing chart of FIG.
[0038] 6, the control unit 112 sets driving in the DS_AD normal bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the DS_AD low luminance reference signal 224 is selected.
[0039] At time T501, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal. Note that the bit precision varies depending on the scanning period of the clock period, and the longer the scanning period, the higher the bit precision.
[0040] At time T502, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0041] When the ramp signal has been scanned up to a predetermined level at time T503, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0042] At time T504, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0043] At time T505, the control unit 112 controls each switch signal to perform luminance judgment. The control unit 112 sets the switch SW401 to HIGH to input the result of the comparator 410 to the signal level judgment unit 412. The control unit 112 also sets the switch SW402 to LOW to cut off the connection to the counter circuit 413.
[0044] At time T506, the control unit 112 switches the switch SW400 to input the luminance evaluation reference signal 223 to the comparator 410.
[0045] At time T507, the signal level matches the luminance evaluation reference signal 223 within a predetermined time, so the output of the comparator 410 changes from HIGH to LOW. The example in Fig. 6 shows a case where the signal level is low, so the signal level evaluation unit 412 determines that the signal level is low (determination result), and the reference signal selection unit 411 selects the DS_AD low luminance reference signal 224.
[0046] At time T508, in order to end the luminance determination, the control unit 112 sets the switches SW400 and SW401 to LOW, sets the switch SW402 to HIGH, and sets the output of the comparator 410 to HIGH.
[0047] At time T509, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0048] At time T510, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0049] When the ramp signal has been scanned up to a predetermined level at time T511, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value and sets the output of the comparator 410 to HIGH.
[0050] At time T512, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. This operation is repeated for each row, thereby reading out the image signal from the image sensor 102.
[0051] Next, the operation of the DS_AD when the pixel signal is at a high level will be described with reference to the timing chart of FIG.
[0052] 7, the control unit 112 sets driving in the DS_AD normal bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the DS_AD low luminance reference signal 224 is selected.
[0053] At time T601, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0054] At time T602, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0055] When the ramp signal has been scanned up to a predetermined level at time T603, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0056] At time T604, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0057] At time T605, the control unit 112 controls each switch signal to perform a luminance determination. The control unit 112 sets the switch SW401 to HIGH to input the result of the comparator 410 to the signal level determination unit 412. The control unit 112 also sets the switch SW402 to LOW to cut off the connection to the counter circuit 413.
[0058] At time T606, the control unit 112 switches the switch SW400 to input the luminance evaluation reference signal 223 to the comparator 410.
[0059] Since there was no timing until time T607 at which the signal level matched the reference signal for luminance determination 223, the signal level determination unit 412 determined that the signal level was high, and the reference signal selection unit 411 selected the DS_AD high luminance reference signal 225.
[0060] At time T608, in order to finish the luminance determination, the control unit 112 sets the switches SW400 and SW401 to LOW, sets the switch SW402 to HIGH, and keeps the output of the comparator 410 at HIGH.
[0061] At time T609, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0062] At time T610, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0063] When the ramp signal has been scanned up to a predetermined level at time T611, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value and sets the output of the comparator 410 to HIGH.
[0064] At time T612, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. This operation is repeated for each row, thereby reading out the image signal from the image sensor 102.
[0065] On the other hand, FIG. 8 is a timing chart showing the operation of the SS_AD in the HDR mode when a low-gain reference signal is used.
[0066] At time T700, the control unit 112 sets driving of the SS_AD for HDR at low gain in the register setting unit 230. The setting is also reflected in the reference signal generator 221. Here, the DS_AD low luminance reference signal 224 is shared with the SS_AD when a low gain reference signal is used.
[0067] At time T701, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0068] At time T702, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0069] When the ramp signal has been scanned up to a predetermined level at time T703, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0070] At time T704, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0071] At time T705, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0072] At time T706, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0073] When the ramp signal is scanned to a predetermined level at time T707, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. Also, the output of the comparator 410 is set to HIGH. Note that in the SS_AD in HDR mode, the scanning time of the ramp signal is made longer than in the DS mode in order to obtain data with higher accuracy.
[0074] At time T708, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the HDR low gain image signal of the image sensor 102 is read out.
[0075] FIG. 9 is a timing chart showing the operation of the SS_AD in the HDR mode when a high gain reference signal is used.
[0076] At time T800, the control unit 112 sets HDR_SS_AD driving at high gain in the register setting unit 230. The setting is also reflected in the reference signal generator 221. Here, the SS_AD low luminance reference signal 226 is selected. At time T801, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal. The counter circuit 413 counts the clock signal.
[0077] At time T802, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0078] When the ramp signal has been scanned up to a predetermined level at time T803, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0079] At time T804, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0080] At time T805, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0081] At time T806, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. The LOW output of the comparator 410 acts as an enable for the counter circuit 413, so the counter circuit 413 stops counting when it becomes LOW. Note that although this is the same level as the signal level in Fig. 8, the slope of the reference signal is gentle and it is in a high gain state, so the output is large. All pixels show the same tendency.
[0082] When the ramp signal is scanned to a predetermined level at time T807, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. Also, the output of the comparator 410 is set to HIGH. Note that in the SS_AD in HDR mode, the scanning time of the ramp signal is made longer than in the DS mode in order to obtain data with higher accuracy.
[0083] At time T808, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the HDR high gain image signal of the image sensor 102 is read out.
[0084] In the high-bit SS_AD in HDR mode, readout is performed multiple times (twice in this embodiment) with different gains to obtain two images, a low-gain image as shown in FIG. 8 and a high-gain image as shown in FIG. 9, which are then appropriately combined by the image processing circuit 109. For example, a high-gain signal is selected for low-luminance parts, and a low-gain signal is selected for high-luminance parts. Alternatively, pixel values calculated from each pixel may be combined. The circuit that performs the combination may be the image processing circuit 109 or a circuit within the image sensor 102.
[0085] As described above, by switching between the DS_AD method during normal shooting and the high-bit SS_AD method in HDR mode, in an imaging device equipped with a DS_AD that has previously been used for high-speed readout, it is possible to obtain an HDR image synthesized from image data with higher accuracy than in the case of DS_AD in normal shooting by switching modes.
[0086] Second embodiment In the second embodiment, an embodiment in which DS_AD different from normal DS_AD is performed when generating an HDR image will be described.
[0087] Figure 10 shows the normal shooting and HDR_DS_AD driving modes. In the normal SDR shooting mode, normal bit DS_AD is performed, and in the HDR_DS_AD mode, high bit AD is performed once for all pixels in DS_AD, and low level and high level images are output.
[0088] Fig. 11 is a timing chart showing the operation of the high-bit HDR_DS_AD at low level. Fig. 12 is a timing chart showing the operation of the high-bit HDR_DS_AD at high level. In the DS_AD, the operation shown in either the timing chart of Fig. 11 or Fig. 12 is performed according to the signal level of each pixel. First, the operation of the high-bit HDR_DS_AD at a low level will be described with reference to the timing chart of FIG.
[0089] 11, the control unit 112 sets driving in the HDR_DS_AD high bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the SS_AD low luminance reference signal 226 is selected.
[0090] At time T1001, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0091] At time T1002, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0092] When the ramp signal has been scanned up to a predetermined level at time T1003, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0093] At time T1004, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0094] At time T1005, the control unit 112 controls each switch signal to perform luminance judgment. The control unit 112 sets the switch SW401 to HIGH in order to input the result of the comparator 410 to the signal level judgment unit 412. The control unit 112 also sets the switch SW402 to LOW to cut off the connection to the counter circuit 413.
[0095] At time T1006, the control unit 112 switches the switch SW400 to input the luminance evaluation reference signal 223 to the comparator 410.
[0096] At time T1007, the signal level matches the luminance evaluation reference signal 223 within a predetermined time, so the output of the comparator 410 changes from HIGH to LOW. The example in Fig. 11 shows a case where the signal level is low, so the signal level evaluation unit 412 determines that the signal level is low, and the reference signal selection unit 411 selects the SS_AD low luminance reference signal 226.
[0097] At time T1008, in order to end the luminance determination, the control unit 112 sets the switches SW400 and SW401 to LOW, sets the switch SW402 to HIGH, and sets the output of the comparator 410 to HIGH.
[0098] At time T1009, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0099] At time T1010, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0100] When the ramp signal is scanned to a predetermined level at time T1011, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. Also, the output of the comparator 410 is set to HIGH. Note that in HDR mode, the scanning time from T1009 to T1011 is set longer than in the Dual Slope mode for normal shooting.
[0101] At time T1012, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset signal level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the low-level image signal of the image sensor 102 is read out.
[0102] Next, the operation of the high-bit HDR_DS_AD at a high level will be described with reference to the timing chart of FIG.
[0103] 12, the control unit 112 sets driving in the HDR_DS_AD high bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the SS_AD low luminance reference signal 226 is selected.
[0104] At time T1101, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0105] At time T1102, the pixel reset release level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0106] When the ramp signal has been scanned up to a predetermined level at time T1103, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0107] At time T1104, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. After that, the pixel signal level is read out to the column output line 210.
[0108] At time T1105, the control unit 112 controls each switch signal to perform luminance judgment. The control unit 112 sets the switch SW401 to HIGH in order to input the result of the comparator 410 to the signal level judgment unit 412. The control unit 112 also sets the switch SW402 to LOW to cut off the connection to the counter circuit 413.
[0109] At time T 1106 , the control unit 112 switches the switch SW 400 to input the luminance evaluation reference signal 223 to the comparator 410 .
[0110] Since there was no timing until time T1107 at which the signal level matched with the luminance determination reference signal 223, the signal level determination unit 412 determined that the signal level was high, and the reference signal selection unit 411 selected the DS_AD low luminance reference signal 224. Here, the DS_AD low luminance reference signal 224 is shared with the high-bit HDR_DS_AD at the high level.
[0111] At time T1108, in order to end the luminance determination, the control unit 112 sets the switches SW400 and SW401 to LOW, sets the switch SW402 to HIGH, and keeps the output of the comparator 410 at HIGH.
[0112] At time T1109, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 to start scanning the ramp signal from the initial level. The control unit 112 also controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal.
[0113] At time T1110, the signal level and the ramp signal level match, so the output of the comparator 410 changes from HIGH to LOW. Since the LOW output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting at the point when it becomes LOW.
[0114] When the ramp signal has been scanned up to a predetermined level at time T1111, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value, and also sets the output of the comparator 410 to HIGH.
[0115] At time T1112, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the high-level image signal of the image sensor 102 is read out.
[0116] Since the HDR DS_AD mode has more pixel bit information than normal AD conversion, for example, a high gain is applied to low-level signals and a low gain is applied to high-luminance signals, and an HDR image may be obtained by performing processing at the judgment value by the image sensor. Also, data with more bits than normal may be sent to the image processing circuit 109, and an HDR image may be obtained by image processing.
[0117] As described above, in this embodiment, the bit precision of the AD is changed for the same DS_AD in the DS_AD during normal shooting and the DS_AD during HDR. This makes it possible to obtain high-quality HDR images faster than SS_AD in HDR mode in an imaging device equipped with a DS_AD function.
[0118] (Third embodiment) In the third embodiment, switching between the HDR_SS_AD mode shown in the first embodiment and the HDR_DS_AD mode shown in the second embodiment will be described. Fig. 13 is a diagram showing switching between the HDR_SS_AD mode and the HDR_DS_AD mode shown in the second embodiment.
[0119] In the HDR_SS_AD mode, the high-bit AD shown in the first embodiment is performed multiple times (twice in this embodiment) on all pixels in the SS_AD to obtain multiple output images (two in this embodiment), which are then composited.In the HDR_DS_AD mode, the high-bit AD shown in the second embodiment is performed once on all pixels in the DS_AD to obtain low-level and high-level images, which are then composited.
[0120] In HDR_SS_AD mode, high-bit AD is performed multiple times (twice in this embodiment), so the read speed is slightly slower, but multiple images (two images in this embodiment) are output, so the freedom of image processing increases. In HDR_DS_AD mode, high-bit AD only needs to be performed once, so the read speed is faster than in HDR_SS_AD mode.
[0121] For example, if you want to obtain an HDR image but there is a restriction on the readout time, you can select the HDR_DS_AD mode, and if there is no restriction on the readout time, you can select the HDR_SS_AD mode.
[0122] In the third embodiment, an example in which the HDR method is changed depending on the shooting mode has been described above. It is possible to change the AD conversion method for each shooting mode in accordance with the advantages of each mode.
[0123] The disclosure of this specification includes the following imaging device, control method, program, and storage medium.
[0124] (Item 1) A pixel that performs photoelectric conversion; an AD converter that performs AD conversion of the pixel signal; a control means for controlling the AD converter to perform AD conversion in either a first AD conversion mode in which an image signal is obtained by comparing a signal of the pixel with one of a plurality of reference signals having different slopes of the time change in signal level, or a second AD conversion mode in which a signal of one same pixel is compared with a plurality of reference signals having different slopes of the time change in signal level, and AD conversion is performed a plurality of times to obtain a plurality of image signals with different gains; An imaging device comprising:
[0125] (Item 2) In the first AD conversion mode, the AD converter performs AD conversion by comparing the signal of the pixel with a first reference signal having a first slope of the time change in signal level or a second reference signal having a slope of the time change in signal level that is greater than that of the first reference signal, and in the second AD conversion mode, the AD converter performs AD conversion a plurality of times by comparing the signal of one same pixel with a third reference signal having a third slope of the time change in signal level and a fourth reference signal having a slope of the time change in signal level that is greater than that of the third reference signal, The imaging device described in item 1, characterized in that when AD conversion is performed in the second AD conversion mode, at least one of the first reference signal and the second reference signal is used for at least one of the third reference signal and the fourth reference signal.
[0126] (Item 3) 3. The imaging device according to claim 1, wherein in the first AD conversion mode, AD conversion is performed on the pixel signal with an increased number of bits.
[0127] (Item 4) 4. The imaging device according to item 3, wherein an HDR image signal is generated from the image signal obtained by increasing the number of bits and performing AD conversion.
[0128] (Item 5) 5. The imaging device according to any one of items 1 to 4, wherein in the second AD conversion mode, the number of bits of the pixel signal is increased and the multiple AD conversions are performed.
[0129] (Item 6) 6. The imaging device according to any one of items 1 to 5, further comprising a determination unit for determining a signal level of the signal from the pixel.
[0130] (Item 7) 7. The imaging device according to item 6, wherein the determining means determines the signal level of the signal of the pixel by comparing the signal of the pixel with a reference signal for determining the signal level.
[0131] (Item 8) 7. The imaging device according to item 6, wherein the AD converter switches a reference signal used in the first AD conversion mode according to a determination result of the determination means.
[0132] (Item 9) 9. The imaging device according to any one of items 1 to 8, wherein in the first AD conversion mode, an image signal of SDR is obtained.
[0133] (Item 10) 10. The imaging device according to any one of items 1 to 9, further comprising a synthesis unit that synthesizes a plurality of image signals obtained by the plurality of AD conversions performed in the second AD conversion mode.
[0134] (Item 11) 11. The imaging device according to item 10, wherein the synthesizing means generates an HDR image signal by synthesizing a plurality of image signals obtained by the plurality of AD conversions.
[0135] (Item 12) A method for controlling an imaging device including a pixel that performs photoelectric conversion and an AD converter that performs AD conversion of a signal of the pixel, comprising: A control method for an imaging device, comprising a control step of controlling the AD converter to perform AD conversion in either a first AD conversion mode in which an image signal is obtained by comparing the signal of the pixel with one of a plurality of reference signals having different slopes of the signal level change over time, or a second AD conversion mode in which a signal of the same pixel is compared with each of a plurality of reference signals having different slopes of the signal level change over time, thereby performing AD conversion multiple times to obtain a plurality of image signals with different gains.
[0136] (Item 13) Item 13. A program for causing a computer to execute the control method according to item 12.
[0137] (Item 14) A computer-readable storage medium storing a program for causing a computer to execute the control method according to item 12.
[0138] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0139] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0140] 102: imaging element, 205: unit pixel, 210: column output line, 220: column circuit, 220a: AD converter, 221: reference signal generator, 222: timing control unit, 230: register setting unit
Claims
1. A pixel that performs photoelectric conversion; an AD converter that performs AD conversion of the pixel signal; a control means for controlling the AD converter to perform AD conversion in either a first AD conversion mode in which an image signal is obtained by AD converting the signal of the pixel by comparing it with one of a plurality of reference signals having different slopes of the time change in signal level, or a second AD conversion mode in which a signal of one same pixel is compared with a plurality of reference signals having different slopes of the time change in signal level by performing AD conversion a plurality of times to obtain a plurality of image signals having different gains; An imaging device comprising:
2. In the first AD conversion mode, the AD converter performs AD conversion by comparing the signal of the pixel with a first reference signal having a first slope of the time change in signal level or a second reference signal having a slope of the time change in signal level that is greater than that of the first reference signal, and in the second AD conversion mode, the AD converter performs AD conversion a plurality of times by comparing the signal of one same pixel with a third reference signal having a third slope of the time change in signal level and a fourth reference signal having a slope of the time change in signal level that is greater than that of the third reference signal, The imaging device according to claim 1, characterized in that when AD conversion is performed in the second AD conversion mode, at least one of the first reference signal and the second reference signal is used for at least one of the third reference signal and the fourth reference signal.
3. 2. The imaging device according to claim 1, wherein in the first AD conversion mode, the AD conversion is performed on the pixel signal with an increased number of bits.
4. 4. The imaging device according to claim 3, wherein an HDR image signal is generated from the image signal obtained by performing AD conversion with the number of bits increased.
5. 2. The imaging device according to claim 1, wherein in the second AD conversion mode, the number of bits of the pixel signal is increased and the multiple AD conversions are performed.
6. 2. The imaging apparatus according to claim 1, further comprising a determination unit for determining a signal level of the signal from the pixel.
7. 7. The image pickup apparatus according to claim 6, wherein the determining means determines the signal level of the signal of the pixel by comparing the signal of the pixel with a reference signal for determining the signal level.
8. 7. The imaging apparatus according to claim 6, wherein the AD converter switches a reference signal used in the first AD conversion mode in accordance with a result of the determination by the determination means.
9. 2. The imaging device according to claim 1, wherein an image signal in SDR format is obtained in the first AD conversion mode.
10. 2. The imaging apparatus according to claim 1, further comprising a synthesizing unit that synthesizes a plurality of image signals obtained by the plurality of AD conversions performed in the second AD conversion mode.
11. 11. The imaging device according to claim 10, wherein the synthesizing unit synthesizes a plurality of image signals obtained by the plurality of AD conversions to generate an HDR image signal.
12. A method for controlling an imaging device including a pixel that performs photoelectric conversion and an AD converter that performs AD conversion of a signal of the pixel, comprising: A control method for an imaging device, comprising a control step of controlling the AD converter to perform AD conversion in either a first AD conversion mode in which an image signal is obtained by comparing the signal of the pixel with one of a plurality of reference signals having mutually different slopes of the signal level change over time, or a second AD conversion mode in which a signal of the same pixel is compared with each of a plurality of reference signals having mutually different slopes of the signal level change over time to perform AD conversion multiple times, thereby obtaining multiple image signals with different gains.
13. A program for causing a computer to execute the control method according to claim 12.
14. A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 12.