Image pickup apparatus, control method of image pickup apparatus, storage medium, and program product
By using a dual-slope AD converter and multiple AD conversion modes in the camera device, AD conversion is performed using reference signals with different time-changing gradients of signal levels, the problem of increasing circuit scale in the prior art is solved and different Gain output and high-quality image signal.
Patent Information
- Application Number
- CN202410645967.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
When existing camera devices increase the readout speed and dynamic range, the circuit scale becomes larger, making it difficult to achieve outputs of different gains without increasing the circuit scale.
A dual slope type AD converter is adopted, and an AD conversion is performed by using reference signals with different time-changing gradients of the signal level in the first AD conversion mode and the second AD conversion mode to obtain image signals with different gains.
Without increasing the circuit scale, outputs with different gains are achieved, improving the quality and readout speed of the image signal.
Smart Images

Figure CN119031257A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an imaging device and a method for controlling the imaging device. Background Art
[0002] Conventionally, as a pixel signal reading circuit in an image sensor, there is known a structure in which a pixel signal output from a pixel to a column output line is AD-converted (analog / digital-converted) in a column circuit provided for each pixel column.
[0003] Japanese Patent Laid-Open No. 2022-144244 describes a dual slope type AD converter that increases a readout speed by switching the slope of a ramp signal according to a signal level when AD converting a pixel signal.
[0004] Furthermore, Japanese Patent Laid-Open No. 2015-128253 discloses a technique for extending a dynamic range by simultaneously amplifying signals of the same pixel using different gains.
[0005] However, there is a problem that if dual slope type AD conversion is performed to increase the readout speed and also different gain outputs are obtained to increase the dynamic range, the circuit scale of wiring or selectors or the like in the imaging apparatus becomes larger. Summary of the invention
[0006] The present disclosure has been made in view of the above-mentioned problems, and provides an image pickup apparatus which is equipped with a dual slope type AD converter and can obtain outputs of different gains for a signal of the same pixel without increasing the circuit scale.
[0007] According to a first aspect of the present invention, there is provided an imaging device, comprising: a pixel for performing photoelectric conversion; an AD converter for performing AD conversion of a signal of the pixel; and a control unit, which is configured to perform AD conversion in a first AD conversion mode or a second AD conversion mode using the AD converter, wherein in the first AD conversion mode, an image signal is obtained by performing AD conversion by comparing the signal of the pixel with one of a plurality of reference signals whose gradients of time changes in signal levels are different from each other, and in the second AD conversion mode, a plurality of image signals with different gains are obtained by performing multiple AD conversions by comparing the signal of a pixel with a plurality of reference signals whose gradients of time changes in signal levels are different from each other.
[0008] According to a second aspect of the present invention, a control method for an imaging device is provided, the imaging device comprising pixels for performing photoelectric conversion, and an AD converter for performing AD conversion of signals of the pixels, the control method comprising: controlling to utilize the AD converter to perform AD conversion in a first AD conversion mode or a second AD conversion mode, wherein in the first AD conversion mode, an image signal is obtained by performing AD conversion by comparing the signal of the pixel with one of a plurality of reference signals whose gradients of time changes of signal levels are different from each other, and in the second AD conversion mode, a plurality of image signals having different gains are obtained by performing multiple AD conversions by comparing the signal of a pixel with a plurality of reference signals whose gradients of time changes of signal levels are different from each other.
[0009] According to a third aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing a program for causing a computer to execute the above method.
[0010] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a block diagram illustrating the configuration of an image pickup apparatus in a first embodiment of the present invention.
[0012] Figure 2 It is a simplified structural diagram of the camera element.
[0013] Figure 3 It is the equivalent circuit diagram of a unit pixel.
[0014] Figure 4 This is a diagram illustrating the configuration of an AD converter of an image pickup element.
[0015] Figure 5 It is a diagram illustrating the imaging mode in the first embodiment.
[0016] Figure 6 is a timing diagram illustrating the operation at a low signal level of DS_AD.
[0017] Figure 7 is a timing diagram illustrating the operation at a high signal level of DS_AD.
[0018] Figure 8 is a timing diagram illustrating the operation at low gain of HDR_SS_AD.
[0019] Fig. 9 is a timing diagram illustrating the operation at high gain of HDR_SS_AD.
[0020] Fig.10It is a diagram illustrating an image pickup mode in the second embodiment.
[0021] Fig.11 is a timing diagram illustrating the operation at a low signal level of HDR_DS_AD.
[0022] Fig.12 is a timing diagram illustrating the operation at a high signal level of HDR_DS_AD.
[0023] Fig.13 It is a diagram illustrating an image pickup mode in the third embodiment. DETAILED DESCRIPTION
[0024] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A plurality of features are described in the embodiments, but are not limited to an invention requiring all such features, and a plurality of such features may be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar structures, and redundant descriptions thereof are omitted.
[0025] (First embodiment)
[0026] Figure 1 1 is a block diagram illustrating the configuration of an image pickup apparatus 100 in the first embodiment of the present invention.
[0027] exist Figure 1 In the present invention, the lens unit 101 condenses incident light from a subject and forms an image on the imaging element 102. The lens unit 101 is configured to include, for example, a plurality of lenses, and a diaphragm, etc. The imaging element 102 photoelectrically converts the subject image formed by the lens unit 101 and outputs image data.
[0028] The control unit 112 performs control of the entire imaging apparatus 100, such as driving of the imaging element 102 and control of the image processing circuit 109. The control unit 112 also controls acceptance of instructions from the display unit 113 and the operation unit 116, and transfer of data to the memory circuit 111 and the recording unit 110. Note that the control unit 112 may be provided in the imaging element 102 or the image processing circuit 109.
[0029] The image processing circuit 109 performs development processing such as color matrix processing and gamma processing on the image data from the image pickup element 102. In these processes, the image processing circuit 109 causes the memory circuit 111 to store the image data as needed. Then, the image processing circuit 109 outputs the processed image data to the display unit 113 and the recording unit 110.
[0030] The display unit 113 displays image data and the like output from the image processing circuit 109. The operation unit 116 sends an operation signal to the control unit 112 according to an operation performed by a user.
[0031] The bus 120 is a common path for the imaging element 102 , the image processing circuit 109 , the display unit 113 , the operation unit 116 , the recording unit 110 , and the memory circuit 111 to exchange data with each other.
[0032] Figure 2 1 is a simplified structural diagram of the imaging element 102 .
[0033] In the pixel region 208, a plurality of unit pixels 205 are arranged two-dimensionally (in a matrix). The vertical arrangement of the unit pixels 205 in the pixel region 208 is referred to as a "column", and the horizontal arrangement is referred to as a "row".
[0034] The pixel signal output from each unit pixel 205 is output to the column output line 210, and further input to the column circuit 220. In the column circuit 220, an analog-to-digital converter (hereinafter referred to as an AD converter) 220a is arranged. Figure 4 A detailed description will now be given of the column circuit including the AD converter 220a.
[0035] The AD converter 220a is connected to a reference signal generator 221 and a timing control unit 222. The reference signal generator 221 supplies a reference signal (ramp signal) to the AD converter 220a via a plurality of reference signal lines 223, 224, 225, and 226. The timing control unit 222 supplies a clock for counting via a clock line 227, and supplies a reset signal for counting via a reset signal line 228.
[0036] In addition, a register setting unit 230 is arranged in the imaging element 102 , an imaging mode is specified in the register setting unit 230 by the control unit 112 , and the specified mode signal is output from the register setting unit 230 to the reference signal generator 221 .
[0037] The digital signal output from the column circuit 220 is input to the digital signal processing unit 211 , subjected to processing such as dark shift correction and gain correction, and output to the outside of the imaging element 102 via the output unit 212 .
[0038] Figure 3 is an equivalent circuit diagram of the unit pixel 205 .
[0039] By turning on the transfer switch 306 using the transfer control signal 301, the charge generated and accumulated in the photodiode 305 in response to the incident light is transferred to the floating diffusion unit (hereinafter referred to as the FD unit) 307. The source follower amplifier 309 is constituted together with the constant current source 311 connected to the column output line 210, amplifies the voltage signal based on the charge accumulated in the FD unit 307, and outputs the amplified voltage signal as a pixel signal. When the row selection control signal 303 turns on the row selection switch 310, the output of the source follower amplifier 309 is output to the column output line 210.
[0040] When unnecessary charges accumulated in the FD 307 are reset, the reset switch 308 is turned on by the reset control signal 302. In addition, when the photodiode 305 is reset, the reset switch 308 is turned on, and the transfer switch 306 is turned on by the transfer control signal 301. The transfer control signal 301, the reset control signal 302, and the row selection control signal 303 are output from the vertical scanning circuit 204 in response to a command from the control unit 112.
[0041] Figure 4 FIG. 1 is a diagram illustrating a circuit configuration of an AD converter of an imaging element. Figure 4 , a column circuit 220_k of the kth column and a column circuit 220_k+1 of the k+1th column are illustrated. Since the column circuits of the respective columns have the same structure, the column circuit 220_k of the kth column (which may be simply referred to as the column circuit 220) will be described as an example. In addition, in the AD conversion method, a method of performing AD conversion by using two ramp signals whose slopes are different from each other according to the signal level is called Dual_Slope_AD (hereinafter referred to as DS_AD), and a method of performing AD conversion by using a single ramp signal is called Single_Slope_AD (hereinafter referred to as SS_AD).
[0042] The reference signal generator 221 supplies a brightness determination reference signal 223, a DS_AD low brightness reference signal 224, a DS_AD high brightness reference signal 225, and a SS_AD low brightness reference signal 226 to each column circuit 220. These reference signals are different in the gradient of the slope of the signal waveform. The switch SW 400_k is a switch that selects the brightness determination reference signal 223 or the reference signal selected by the reference signal selection unit 411_k.
[0043] The pixel signal from the column output line 210k is input to one input of the comparator 410_k, and the reference signal selected by the SW 400_k is input to the other input of the comparator 410_k. The comparison result of the comparator 410_k is input to the signal level determination unit 412_k via the switch SW401_k. Similarly, the comparison result of the comparator 410_k is input to the counter circuit 413_k via the switch SW402_k.
[0044] In the counter circuit 413_k, the counter is driven in synchronization with the reference signal. In addition, a clock signal 227 for operating the counter circuit 413_k is supplied from the timing control unit 222. In addition, a reset signal 228 for resetting the counter in the counter circuit 413_k is supplied from the timing control unit 222. The output of the signal level determination unit 412_k is output to the reference signal selection unit 411_k.
[0045] Furthermore, a mode setting signal for setting an imaging mode is input to the reference signal generator 221 and is reflected in the selection of the reference signal.
[0046] Next, refer to Figure 5 The image capturing mode according to the present embodiment will be described.
[0047] As already explained above, in Figure 5 In the present invention, dual slope AD conversion (DS_AD) is a method of performing AD conversion by switching two types of ramp signals whose signal levels change temporally in a triangular waveform according to the signal level of a pixel signal.
[0048] In addition, SS_AD is a method of performing AD conversion using a ramp signal having one type of slope in which a signal level changes in a triangular waveform.
[0049] In addition, high dynamic range (HDR) imaging is an imaging method in which multiple (two in this embodiment) image signals with different exposures are synthesized to obtain a single image signal with an expanded dynamic range. It should be noted that the ordinary imaging mentioned below refers to imaging that obtains a standard dynamic range (SDR) image.
[0050] exist Figure 5In the normal camera mode, for AD conversion in the normal camera mode, the normal bit DS_AD is used. In the high dynamic range single slope mode (HDR_SS mode) which is another AD conversion mode, a high-bit SS_AD with low gain and a high-bit SS_AD with high gain are performed. In other words, two AD conversions are performed on the output signals of all pixels, and the two images are synthesized after the AD conversion. In the normal bit DS_AD, readout can be performed at high speed. In contrast, in the HDR_SS mode, since the high-bit SS_AD with a bit number increase is performed twice, the readout time is slightly longer, but a high-definition image can be obtained.
[0051] Figure 6 This is a timing chart for when the image signal is at a low level in the normal bit DS_AD during normal imaging. Figure 7 is a timing diagram illustrating the operation of DS_AD when the pixel signal is at a high level. In DS_AD, the Figure 6 or Figure 7 The operations are illustrated in the timing diagram of FIG.
[0052] First, refer to Figure 6 A timing diagram is used to illustrate the operation of DS_AD when the image signal is at a low level.
[0053] exist Figure 6 At time T500, the control unit 112 makes a setting for driving in the DS_AD normal bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the DS_AD low brightness reference signal 224 is selected.
[0054] At time T501, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the period when the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal. Note that the bit accuracy varies depending on the scanning time period of the clock cycle, and increasing the scanning time period increases the bit accuracy.
[0055] Since the reset release level of the pixel and the ramp signal level are consistent with each other at time T502, the output of the comparator 410 changes from high (HIGH) to low (LOW). Since the low output of the comparator 410 is used as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0056] When the ramp signal is swept to a predetermined level at time T503, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0057] At time T504, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0058] At time T505, the control unit 112 controls the respective switch signals to perform brightness determination. To input the result of the comparator 410 to the signal level determination unit 412, the control unit 112 sets the switch SW401 to high. The switch SW402 is set to low to disconnect the counter circuit 413.
[0059] At time T506 , the control unit 112 switches the switch SW 400 and inputs the brightness determination reference signal 223 to the comparator 410 .
[0060] At time T507, since the brightness determination reference signal 223 and the signal level coincide with each other within a predetermined period of time, the output of the comparator 410 changes from high to low. Figure 6 In the example of , the signal level is low, the signal level determination unit 412 determines that the signal level is low (determination result), and the reference signal selection unit 411 selects the DS_AD low brightness reference signal 224 .
[0061] At time T508, in order to end the brightness determination, the control unit 112 sets the switches SW 400 and SW 401 to low and sets the switch SW 402 to high. The output of the comparator 410 is set to high.
[0062] At time T509, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and starts scanning the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0063] Since the signal level and the ramp signal level coincide with each other at time T510, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0064] When the ramp signal is swept to a predetermined level at time T511, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. In addition, the output of the comparator 410 is set to high.
[0065] At time T512, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the image signal of the imaging element 102 is read out.
[0066] Next, refer to Figure 7 A timing diagram is used to illustrate the operation of DS_AD when the image signal is at a high level.
[0067] exist Figure 7 At time T600, the control unit 112 makes a setting for driving in the DS_AD normal bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the DS_AD low brightness reference signal 224 is selected.
[0068] At time T601, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the period in which the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0069] Since the reset release level of the pixel and the ramp signal level coincide with each other at time T602, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0070] When the ramp signal is swept to a predetermined level at time T603, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0071] At time T604, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0072] At time T605, the control unit 112 controls the respective switch signals to perform brightness determination. To input the result of the comparator 410 to the signal level determination unit 412, the control unit 112 sets the switch SW401 to high. The switch SW402 is set to low to disconnect the counter circuit 413.
[0073] At time T606 , the control unit 112 switches the switch SW 400 and inputs the brightness determination reference signal 223 to the comparator 410 .
[0074] Since there is no timing at which the brightness determination reference signal 223 and the signal level coincide with each other until time T607 , the signal level determination unit 412 determines that the signal level is high, and the reference signal selection unit 411 selects the DS_AD high brightness reference signal 225 .
[0075] At time T607, in order to end the brightness determination, the control unit 112 sets the switches SW 400 and SW 401 to low and sets the switch SW 402 to high. The output of the comparator 410 is set to high.
[0076] At time T609, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and starts scanning the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0077] Since the signal level and the ramp signal level coincide with each other at time T610, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0078] When the ramp signal is swept to a predetermined level at time T611, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0079] At time T612, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the image signal of the imaging element 102 is read out.
[0080] Figure 8 is a timing diagram illustrating the operation of SS_AD when a low gain reference signal is used in HDR mode.
[0081] At time T700, the control unit 112 makes settings for driving with HDR SS_AD with low gain in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the DS_AD low brightness reference signal 224 is shared by the SS_AD when the low gain reference signal is used.
[0082] At time T701, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 while the reference signal generator 221 is scanning the ramp signal. The counter circuit 413 counts the clock signal.
[0083] Since the reset release level of the pixel and the ramp signal level coincide with each other at time T702, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0084] When the ramp signal is swept to a predetermined level at time T703, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0085] At time T704, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0086] At time T705, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and scans the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0087] Since the signal level and the ramp signal level coincide with each other at time T706, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0088] When the ramp signal is scanned to a predetermined level at time T707, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high. In SS_AD in HDR mode, the scanning time of the ramp signal is longer than that in DS mode to obtain more accurate data.
[0089] At time T708, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the HDR low gain image signal of the imaging element 102 is read out.
[0090] Fig. 9 is a timing diagram illustrating the operation of SS_AD when a high gain reference signal is used in HDR mode.
[0091] At time T800, the control unit 112 makes a setting for driving with HDR_SS_AD having a high gain in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the SS_AD low brightness reference signal 226 is selected.
[0092] At time T801, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the period in which the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0093] Since the reset release level of the pixel and the ramp signal level coincide with each other at time T802, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0094] When the ramp signal is swept to a predetermined level at time T803, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0095] At time T804, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0096] At time T805, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and scans the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0097] Since the signal level and the ramp signal level coincide with each other at time T806, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low. Note that although the signal level is consistent with Figure 8 The signal level in the same level, but because the slope of the reference signal is in a gentle high gain state, the output is a large output. All pixels tend to be the same.
[0098] When the ramp signal is scanned to a predetermined level at time T807, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high. In SS_AD in HDR mode, the scanning time of the ramp signal is longer than that in DS mode to obtain more accurate data.
[0099] At time T808, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the HDR high gain image signal of the imaging element 102 is read out.
[0100] In the high bit SS_AD in the HDR mode, multiple readings (twice in this embodiment) are performed using different gains, and each of them obtains the following Figure 8 The low gain image shown and Fig. 9 The high-gain images illustrated are synthesized appropriately by the image processing circuit 109. For example, a high-gain signal is selected for a low-luminance portion, and a low-gain signal is selected mainly for a high-luminance portion. Alternatively, the pixel values calculated from the individual pixels may be synthesized. Note that the circuit for performing synthesis may be the image processing circuit 109 or a circuit in the imaging element 102.
[0101] As described above, by switching between DS_AD during normal imaging and the high-bit SS_AD method in HDR mode, the mode is switched in an imaging device equipped with DS_AD which has heretofore been used for high-speed reading, a synthesized HDR image can be obtained from image data having higher accuracy than the image data of DS_AD of normal imaging.
[0102] (Second embodiment)
[0103] In the second embodiment, an embodiment in which DS_AD is performed in a manner different from that in a normal state when an HDR image is generated will be described.
[0104] Fig.10 The normal camera driving mode and the HDR_DS_AD driving mode are exemplified. In the normal SDR camera mode, normal DS_AD is performed, and in the HDR_DS_AD mode, high-level AD is performed once for each pixel with DS_AD, and a low-level image and a high-level image are output.
[0105] Fig.11 is a timing diagram illustrating the operation of the upper bit HDR_DS_AD at a low level. Fig.12 is a timing diagram illustrating the operation of the high-bit HDR_DS_AD at a high level. In DS_AD, the signal level of each pixel is determined. Fig.11 or Fig.12 The operations are illustrated in the timing diagram of FIG.
[0106] First, refer to Fig.11 The timing diagram is used to illustrate the operation of the high-bit HDR_DS_AD when the level is low.
[0107] exist Fig.11 At time T1000, the control unit 112 makes a setting for driving in the HDR_DS_AD high-bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the SS_AD low-brightness reference signal 226 is selected.
[0108] At time T1001, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the period in which the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0109] Since the reset release level of the pixel and the ramp signal level coincide with each other at time T1002, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0110] When the ramp signal is swept to a predetermined level at time T1003, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0111] At time T1004, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0112] At time T1005, the control unit 112 controls the respective switch signals to perform brightness determination. To input the result of the comparator 410 to the signal level determination unit 412, the control unit 112 sets the switch SW401 to high. The switch SW402 is set to low to disconnect the counter circuit 413.
[0113] At time T1006 , the control unit 112 switches the switch SW 400 and inputs the brightness determination reference signal 223 to the comparator 410 .
[0114] At time T1007, since the brightness determination reference signal 223 and the signal level coincide with each other within a predetermined period of time, the output of the comparator 410 changes from high to low. Fig.11 In the example of , the signal level is low, the signal level determination unit 412 determines that the signal level is low, and the reference signal selection unit 411 selects the SS_AD low brightness reference signal 226 .
[0115] At time T1008, in order to end the brightness determination, the control unit 112 sets the switches SW 400 and SW 401 to low and sets the switch SW 402 to high. The output of the comparator 410 is set to high.
[0116] At time T1009, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and scans the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0117] Since the signal level and the ramp signal level coincide with each other at time T1010, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output is set to low.
[0118] When the ramp signal is scanned to a predetermined level at time T1011, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high. In the HDR mode, the scanning time from T1009 to T1011 is made longer than in the dual slope mode of normal imaging.
[0119] At time T1012, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset signal level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the low-level image signal of the imaging element 102 is read out.
[0120] Next, refer to Fig.12 The timing diagram is used to illustrate the operation of the high-bit HDR_DS_AD when the level is high.
[0121] exist Fig.12 At time T1100, the control unit 112 makes a setting for driving in the HDR_DS_AD high-bit mode in the register setting unit 230. This setting is also reflected in the reference signal generator 221. Here, the SS_AD low-brightness reference signal 226 is selected.
[0122] At time T1101, the control unit 112 instructs the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the period in which the reference signal generator 221 scans the ramp signal. The counter circuit 413 counts the clock signal.
[0123] Since the reset release level of the pixel and the ramp signal level coincide with each other at time T1102, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0124] When the ramp signal is swept to a predetermined level at time T1103, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0125] At time T1104, the control unit 112 controls the timing control unit 222 to store the counter value of the reset release level in the reset release level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. Thereafter, the pixel signal level is read out to the column output line 210.
[0126] At time T1105, the control unit 112 controls the respective switch signals to perform brightness determination. To input the result of the comparator 410 to the signal level determination unit 412, the control unit 112 sets the switch SW401 to high. The switch SW402 is set to low to disconnect the counter circuit 413.
[0127] At time T1106 , the control unit 112 switches the switch SW 400 and inputs the brightness determination reference signal 223 to the comparator 410 .
[0128] Since there is no timing at which the brightness determination reference signal 223 and the signal level coincide with each other until time T1107, the signal level determination unit 412 determines that the signal level is high, and the reference signal selection unit 411 selects the DS_AD low brightness reference signal 224. Here, the DS_AD low brightness reference signal 224 is shared by the high bit HDR_DS_AD at the high level.
[0129] At time T1108, in order to end the brightness determination, the control unit 112 sets the switches SW 400 and SW 401 to low and sets the switch SW 402 to high. The output of the comparator 410 is set to high.
[0130] At time T1109, in order to AD convert the pixel signal level, the control unit 112 controls the reference signal generator 221 and starts scanning the ramp signal from the initial level. In addition, the control unit 112 controls the timing control unit 222 to output a clock signal of a predetermined period to the counter circuit 413 during the reference signal generator 221 scanning the ramp signal.
[0131] Since the signal level and the ramp signal level coincide with each other at time T1110, the output of the comparator 410 changes from high to low. Since the low output of the comparator 410 serves as an enable for the counter circuit 413, the counter circuit 413 stops counting when the output becomes low.
[0132] When the ramp signal is swept to a predetermined level at time T1111, the control unit 112 controls the timing control unit 222 to set the reference signal level to an initial value. The output of the comparator 410 is set to high.
[0133] At time T1112, the control unit 112 controls the timing control unit 222 to store the counter value of the signal level in the signal level memory in the column memory 414. The counter circuit 413 is reset to the initial level via the reset signal 228. The column memory 414 calculates the difference between the stored pixel signal level and the reset release level, and outputs the result to the digital signal processing unit 211. By repeating this operation for each row, the high-level image signal of the imaging element 102 is read out.
[0134] The HDR DS_AD mode has more pixel bit information than the normal AD conversion, so for example, a high gain can be applied to a low-level signal, and a low gain can be applied to a high-brightness signal, and an HDR image can be obtained by processing the joint judgment value portion in the image pickup element. In addition, an HDR image can be obtained by sending data with a larger number of bits than normal to the image processing circuit 109 and performing image processing.
[0135] As described above, in this embodiment, for DS_AD in normal shooting and DS_AD in HDR, the bit accuracy of AD is changed even for the same DS_AD. As a result, in a camera equipped with a DS_AD function, a high-quality HDR image can be obtained at a higher speed than using SS_AD in HDR mode.
[0136] (Third Embodiment)
[0137] In the third embodiment, switching between the HDR_SS_AD mode described in the first embodiment and the HDR_DS_AD mode described in the second embodiment will be described. Fig.13 2 is a diagram illustrating switching between the HDR_SS_AD mode and the HDR_DS_AD mode described in the second embodiment.
[0138] In the HDR_SS_AD mode, the high-bit AD described in the first embodiment is performed multiple times (twice in this embodiment) on all pixels with SS_AD, multiple output images (two in this embodiment) are obtained, and the output images are synthesized. In the HDR_DS_AD mode, the high-bit AD described in the second embodiment is performed once on all pixels with DS_AD, and a low-level image and a high-level image are obtained, and the low-level image and the high-level image are synthesized.
[0139] In the HDR_SS_AD mode, high-bit AD is performed multiple times (twice in this embodiment), making the readout speed slightly slower, but multiple images (two images in this embodiment) are output, so the degree of freedom of image processing is increased. In the HDR_DS_AD mode, high-bit AD only needs to be performed once, making the readout speed faster than in the HDR_SS_AD mode.
[0140] For example, when it is desired to obtain an HDR image but there is a restriction on the reading time, the HDR_DS_AD mode may be selected, and when there is no restriction on the reading time, the HDR_SS_AD mode may be selected.
[0141] In the third embodiment, as described above, an example is given of changing the HDR method in the image capturing mode. The AD conversion method may be changed for each image capturing mode according to respective advantages.
[0142] Other embodiments
[0143] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.
[0144] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.
Claims
1. A camera device, include: Pixels, used for photoelectric conversion; An AD converter, used for performing AD conversion of the signal of the pixel; as well as At least one processor or circuit configured to: A control unit is configured to perform AD conversion in a first AD conversion mode or a second AD conversion mode using the AD converter, wherein in the first AD conversion mode, an image signal is obtained by comparing the signal of the pixel with one of a plurality of reference signals whose gradients of time changes in signal levels are different from each other for AD conversion, and in the second AD conversion mode, a plurality of image signals having different gains are obtained by comparing the signal of one pixel with a plurality of reference signals whose gradients of time changes in signal levels are different from each other for AD conversion.
2. The imaging device according to claim 1, in, In the first AD conversion mode, AD conversion is performed by comparing the signal of the pixel with a first reference signal whose gradient of temporal change of signal level is a first gradient or a second reference signal whose gradient of temporal change of signal level is larger than the gradient of the first reference signal using the AD converter, and AD conversion is performed a plurality of times by comparing the signal of one pixel with each of a third reference signal whose gradient of temporal change of signal level is a third gradient and a fourth reference signal whose gradient of temporal change of signal level is larger than the gradient of the third reference signal using the AD converter in the second AD conversion mode, and When AD conversion is performed in the second AD conversion mode, at least one of the first reference signal and the second reference signal is used for at least one of the third reference signal and the fourth reference signal.
3. The imaging device according to claim 1, in, In the first AD conversion mode, the number of bits of the signal of the pixel is increased and AD conversion is performed.
4. The imaging device according to claim 3, in, An HDR image signal is generated from an image signal obtained by increasing the number of bits and performing AD conversion.
5. The imaging device according to claim 1, in, In the second AD conversion mode, the number of bits of the signal of the pixel is increased and AD conversion is performed a plurality of times.
6. The imaging device according to claim 1, in, The at least one processor or circuit also functions as a determination unit configured to determine a signal level of a signal of the pixel.
7. The imaging device according to claim 6, in, The determination unit determines a signal level of the signal of the pixel by comparing the signal of the pixel with a reference signal for determining a signal level.
8. The imaging device according to claim 6, in, The AD converter switches a reference signal used in the first AD conversion mode according to a determination result of the determination unit.
9. The imaging device according to claim 1, in, In the first AD conversion mode, an SDR image signal is obtained.
10. The imaging device according to claim 1, in, The at least one processor or circuit also functions as a synthesis unit configured to synthesize a plurality of image signals obtained through a plurality of AD conversions performed in the second AD conversion mode.
11. The imaging device according to claim 10, in, The synthesis unit generates an HDR image signal by synthesizing a plurality of image signals obtained through the plurality of AD conversions.
12. A control method for an image pickup device, the image pickup device comprising pixels for performing photoelectric conversion and an AD converter for performing AD conversion of signals of the pixels, the control method include: Control is performed to utilize the AD converter to perform AD conversion in a first AD conversion mode or a second AD conversion mode, wherein in the first AD conversion mode, an image signal is obtained by comparing the signal of the pixel with one of a plurality of reference signals whose gradients of time changes in signal levels are different from each other for AD conversion, and in the second AD conversion mode, a plurality of image signals with different gains are obtained by comparing the signal of one pixel with a plurality of reference signals whose gradients of time changes in signal levels are different from each other for multiple AD conversions. 13 . A non-transitory computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 12 .
14. A computer program product comprising a program for causing a computer to execute the control method according to claim 12.
