Imaging device and reference signal generation device

Abstract

To provide an imaging device capable of making the AD resolution of data used for a dark area different from the AD resolution of data used for a bright area and a reference signal generation device.SOLUTION: The imaging device includes: a pixel circuit for generating a first pixel signal corresponding to first illuminance and a second pixel signal corresponding to second illuminance higher than the first illuminance; an AD conversion part for AD-converting the first pixel signal and the second pixel signal to generate a first digital signal and a second digital signal respectively; and a processing part for synthesizing the first digital signal and the second digital signal to generate a synthetic image. In the AD conversion part, resolution with respect to the second pixel signal is lower than resolution with respect to the first pixel signal.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an imaging device and a reference signal generating device. [Background technology]

[0002] HDR (High Dynamic Range) technology is a technology that ensures dynamic range for both bright and dark areas when an image contains both. Conventionally, in HDR, AD conversion is performed at the same AD resolution for both the data used for the dark areas of the subject and the data used for the bright areas. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-051134 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, even though the visibility of bright and dark areas in an image differs and the required resolution may differ, AD conversion is performed at the same AD resolution, which may result in restrictions on the frame rate.

[0005] Therefore, the present disclosure provides an imaging device and a reference signal generating device that are capable of differentiating the AD resolution of data used in dark areas from that of data used in bright areas. [Means for solving the problem]

[0006] In order to solve the above problems, according to the present disclosure, a pixel circuit that generates a first pixel signal corresponding to a first illuminance and a second pixel signal corresponding to a second illuminance higher than the first illuminance; an AD conversion unit that performs AD conversion on the first pixel signal and the second pixel signal to generate a first digital signal and a second digital signal; a processing unit that performs a synthesis process on the first digital signal and the second digital signal to generate a synthesis image, The imaging device is provided such that the AD conversion section has a lower resolution for the second pixel signal than a resolution for the first pixel signal.

[0007] The pixel circuit may set a second exposure time when generating the second pixel signal to be shorter than a first exposure time when generating the first pixel signal.

[0008] the AD conversion unit generates the first digital signal based on a comparison between the first pixel signal and a first reference signal, and generates the second digital signal based on a comparison between the second pixel signal and a second reference signal; The first reference signal and the second reference signal may vary over time with different slopes.

[0009] The rate at which the value of the second reference signal changes over time may be greater than the rate at which the value of the first reference signal changes over time.

[0010] The wireless communication device may further include a reference signal generating unit that generates the first reference signal and the second reference signal.

[0011] The reference signal generation unit The first resistor, a current source section capable of varying a current supplied to the first resistor over time; an adder that generates the first reference signal and the second reference signal based on a potential of the first resistor by making different a rate of change of the current over time;

[0043]

[0012] the current source section connects a plurality of current sources to the first resistor via a corresponding plurality of switching elements; The current supplied to the first resistor may be changed over time by switching the switching elements between a connected state and a disconnected state.

[0013] A potential at one end of the first resistor may correspond to the first reference signal or the second reference signal, and the other end of the first resistor may be connected to a predetermined low potential.

[0014] The current source unit may reduce the second current supplied to the first resistor over time when generating the second reference signal by a rate greater than the rate at which it reduces the first current supplied to the first resistor over time when generating the first reference signal.

[0015] The power supply may further include a level adjustment section capable of supplying a further current to the first resistor.

[0016] The AD conversion unit is a comparator that compares a readout signal potential of the pixel circuit with the first reference signal or the second reference signal, and inverts an output level when the readout signal potential matches a level of the first reference signal or the second reference signal; a counter whose operation is controlled by an output of the comparator and which counts a comparison time between the read signal potential and the first reference signal or the second reference signal; It may have.

[0017] The AD conversion section may generate, as a digital signal, a difference between a first downcount of the counter at a reset potential and a second downcount of the counter at the read signal potential.

[0018] The processing unit may adjust resolutions of the first digital signal and the second digital signal to generate the composite image.

[0019] The processing unit may adjust the resolution of the first digital signal and the second digital signal, and add the first digital signal and the second digital signal at a ratio according to the value of the adjusted digital signal to generate the composite image with an expanded dynamic range.

[0020] the pixel circuit includes a first photoelectric conversion element having a first sensitivity corresponding to the first illuminance, and a second photoelectric conversion element having a second sensitivity corresponding to the second illuminance and lower than the first sensitivity; The first photoelectric conversion element may generate the first pixel signal, and the second photoelectric conversion element may generate the second pixel signal.

[0021] The pixel circuit includes: a photoelectric conversion element that receives light and generates and accumulates photocharges; a transfer transistor for transferring photocharges; a first charge accumulation unit connected to the photoelectric conversion element via the transfer transistor; A second charge storage section; a storage transistor having one end connected to the first charge storage unit and the other end connected to the second charge storage unit; The second pixel signal may be based on the photocharges accumulated in the first charge accumulation unit, and the first pixel signal may be based on the photocharges accumulated in the first charge accumulation unit and the second charge accumulation unit.

[0022] The pixel circuit may generate one of the first pixel signal and the second pixel signal, and then generate the other.

[0023] The pixel circuits are arranged in a matrix, An optical system that forms an optical image on the pixel circuits arranged in a plurality of locations, Further, it may be provided.

[0024] The AD conversion unit may be configured in plurality to correspond to a plurality of columns of the pixel circuits.

[0025] In order to solve the above problems, according to the present disclosure, The first resistor, a current source section capable of varying a current supplied to the first resistor over time; an adder that generates a first reference signal and a second reference signal that change over time with different slopes based on a potential of the first resistor by varying a rate of change of the current over time; A reference signal generating device is provided, comprising: [Brief description of the drawings]

[0026] [Figure 1] 1 is a block diagram showing an example of the configuration of an imaging device according to a first embodiment of the present technology. [Diagram 2] 1 is a block diagram showing a configuration example of a solid-state imaging element according to a first embodiment of the present technology. [Diagram 3] 2 is a diagram showing a detailed configuration example of a pixel circuit and a column signal processing unit 2. FIG. [Figure 4] FIG. 2 is a diagram showing an example of a circuit configuration of a DAC. [Diagram 5] FIG. 2 is a diagram showing an example of the configuration of a lamp DAC. [Figure 6] FIG. 4 is a diagram illustrating an example of a down-counting operation of the counter 261. [Figure 7] 13 is a diagram showing the state of the ramp DAC when the reference voltage Vramp is at its highest value. [Figure 8] FIG. 8 is a diagram showing the voltage of the output node in the state of FIG. 7; [Figure 9] A diagram showing the state when one switch is off. [Figure 10] 10 is a diagram showing the voltage of the output node in the state of FIG. 9; [Figure 11] FIG. 13 is a diagram showing a state in which n switches are turned off. [Figure 12] FIG. 12 is a diagram showing the voltage of the output node in the state of FIG. 11; [Figure 13] 4 is a time chart showing an example of imaging operation for one frame during long-term accumulation imaging. [Figure 14] 4 is a time chart showing an example of imaging operation for one frame during short accumulation imaging. [Figure 15] 11 is a timing chart in which a long-term accumulation imaging operation and a short-term accumulation imaging operation are superimposed. [Figure 16] 11 is a diagram showing a schematic diagram of an analog-to-digital conversion period in one frame of long-term accumulation data and short-term accumulation data. [Figure 17] FIG. 4 is a diagram showing a schematic processing order of HDR image acquisition processing. [Figure 18] FIG. 11 is a diagram showing a schematic concept of image synthesis of long-term accumulation data and short-term accumulation data. [Figure 19] FIG. 11 is a diagram showing an example of the configuration of a pixel circuit according to a second embodiment. [Figure 20] 4 is a timing chart showing an example of an accumulation operation of a pixel circuit. [Figure 21] 4 is a timing chart showing an example of a readout operation of a pixel circuit. [Figure 22] FIG. 11 is a diagram showing a detailed configuration example of a pixel circuit and a column signal processing unit according to a third embodiment. [Figure 23] 4 is a timing chart showing a drive signal at two levels. [Figure 24] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Diagram 25] 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, an embodiment of an imaging device and a reference signal generating device will be described with reference to the drawings. The following description will focus on the main components of the imaging device and the reference signal generating device, but the light-emitting element may have components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0028] (First embodiment) 1 is a block diagram showing an example of a configuration of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device for capturing an image of a subject and generating image data thereof, and is capable of generating HDR (High Dynamic Range) image data. The imaging device 100 includes an optical system 110, a solid-state imaging element 200, and a DSP (Digital Signal Processing) circuit 120. The imaging device 100 further includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, a processing unit 180, and a power supply unit 190. The imaging device 100 is assumed to be a camera mounted on a smartphone, an in-vehicle camera, or the like.

[0029] The optical system 110 guides light from a subject and forms an optical image on the light receiving surface of the solid-state imaging element 200. The solid-state imaging element 200 generates image data by photoelectric conversion. The solid-state imaging element 200 supplies the generated image data to the DSP circuit 120 via a signal line 209.

[0030] The DSP circuit 120 executes a predetermined signal processing on the image data and outputs the processed image data to a frame memory 160 or the like via a bus 150.

[0031] The display unit 130 displays image data. For example, a liquid crystal panel or an organic EL (Electro Luminescence) panel is assumed as the display unit 130. The operation unit 140 generates an operation signal in accordance with a user's operation.

[0032] The bus 150 is a common path for the optical system 110, solid-state imaging device 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, processing unit 180, and power supply unit 190 to exchange data with one another.

[0033] The frame memory 160 holds first image data based on a first digital signal, which will be described later, and second image data based on a second digital signal, which will be described later. The storage unit 170 stores various data such as the first image data and the second image data.

[0034] The processing unit 180 performs a synthesis process on the first image data on the dark side and the second image data on the bright side to generate HDR (High Dynamic Range) synthetic image data. That is, the processing unit 180 performs a synthesis process on the first digital signal and the second digital signal to generate a synthetic image digital signal. The power supply unit 190 supplies power to the solid-state imaging element 200, the DSP circuit 120, the display unit 130, and the like.

[0035] [Example of the configuration of a solid-state imaging element] 2 is a block diagram showing an example of a configuration of a solid-state imaging element 200 according to a first embodiment of the present technology. The solid-state imaging element 200 includes a vertical scanning circuit 210, a reference signal generating section (reference signal generating device) 215, a timing control section 220, a pixel array section 240, a column signal processing section 260, and a horizontal scanning circuit 270. The reference signal generating section 215 includes a DAC control section 222 and a DAC (Digital to Analog Converter) 230. The reference signal generating section 215 may include the DAC control section 222 and the DAC 230 in one semiconductor chip, or may include multiple semiconductor chips.

[0036] A plurality of pixel circuits 250 are arranged in a two-dimensional lattice in the pixel array section 240. Hereinafter, a set of pixel circuits 250 arranged in a predetermined horizontal direction will be referred to as a "row", and a set of pixel circuits 250 arranged in a vertical direction to the row will be referred to as a "column".

[0037] The vertical scanning circuit 210 drives the rows in sequence to output the pixel circuits 250. In this embodiment, for example, the exposure time is changed and imaging is performed twice in time series. In this way, two frames of image data are captured in sequence. The exposure time corresponding to the dark area is set to a first exposure time, and the exposure time corresponding to the bright area is set to a second exposure time shorter than the first exposure time. Hereinafter, imaging corresponding to the dark area may be referred to as "long accumulation" or "long accumulation imaging", and imaging corresponding to the bright area may be referred to as "short accumulation" or "short accumulation imaging".

[0038] The timing control unit 220 controls the operation timing of each of the vertical scanning circuit 210, the DAC 230, the column signal processing unit 260, and the horizontal scanning circuit 270 in synchronization with the vertical synchronization signal VSYNC. The vertical synchronization signal VSYNC is a periodic signal of a predetermined frequency (such as 60 Hz) that indicates the imaging timing.

[0039] The DAC control section 222 controls the characteristics of the first reference signal (first ramp signal) and the second reference signal (second ramp signal) generated by the DAC 230, whose voltages change over time, in accordance with the timing control of the timing control section 220.

[0040] The DAC230 is a digital-to-analog converter that generates a first reference signal and a second reference signal by DA (Digital to Analog) conversion. For example, a sawtooth ramp signal is used as the first and second reference signals. In the case of long accumulation imaging, the DAC230 supplies the first reference signal to the column signal processing unit 260. On the other hand, in the case of short accumulation imaging, the DAC230 supplies the second reference signal to the column signal processing unit 260. The first and second reference signals are signals whose voltages change over time with different slopes, and are used in the column signal processing unit 260 to detect pixel signals with different resolutions. That is, the rate at which the value of the second reference signal changes over time is greater than the rate at which the value of the first reference signal changes over time.

[0041] The voltages of the first and second reference signals may be referred to as the first and second reference voltages Vramp1 and Vramp2, respectively. The first reference voltage Vramp1 is a reference voltage corresponding to long accumulation, and the second reference voltage Vramp2 is a reference voltage corresponding to short accumulation. The slope indicating the time variation of the second reference signal Vramp2 is generated so as to be stronger than the slope indicating the time variation of the first reference voltage Vramp1. That is, the DAC control unit 222 controls the rate of change of the reference voltage with respect to time and the offset voltage of the reference voltage. The DAC control unit 222 and the DAC 230 will be described in detail later.

[0042] The pixel circuit 250 generates an analog pixel signal by photoelectric conversion and supplies it to the column signal processing unit 260. The column signal processing unit 260 performs signal processing such as AD conversion processing and CDS (Correlated Double Sampling) processing on the pixel signal for each column. The column signal processing unit 260 supplies the first image data consisting of the first digital signal after processing and the second image data consisting of the second digital signal to the DSP circuit 120 via the signal line 209. In this embodiment, the pixel circuit 250 generates either one of the first image data or the second image data, and then generates the other. That is, the pixel circuit 250 generates either one of the first pixel signal corresponding to the first digital signal and the second pixel signal corresponding to the second digital signal, and then generates the other.

[0043] The horizontal scanning circuit 270 controls the column signal processing unit 260 to sequentially output the first digital signal and the second digital signal. The solid-state imaging device 200 shown in FIG. 2 may be configured as a single semiconductor chip as a whole, or may be configured with multiple semiconductor chips.

[0044] FIG. 3 is a diagram showing a detailed configuration example of the pixel circuit 250 and the column signal processing unit 260. As shown in FIG. 3, the pixel circuit 250 includes a photoelectric conversion element 251, a transfer transistor 252, a reset transistor 253, an FD unit 254, an amplification transistor 255, and a selection transistor 256. In the pixel array unit 240, vertical signal lines VSL are wired for each column along the vertical direction. Note that the FD unit 254 may be referred to as a floating diffusion layer. Also, its capacitance may be denoted as CFD.

[0045] The photoelectric conversion element 251 photoelectrically converts incident light to generate charges. One end of the photoelectric conversion element 251 is connected to a predetermined voltage VSS (VSS < VDD), and the other end is connected to one end of the transfer transistor 252.

[0046] The transfer transistor 252 has one end connected to the other end of the photoelectric conversion element 251 and the other end connected to the FD unit 254. The transfer transistor 252 transfers charges from the photoelectric conversion element 251 to the FD unit 254 according to the drive signal TG from the vertical scanning circuit 210.

[0047] The reset transistor 253 has one end connected to the power supply line Lvdd, and the other end connected to the FD unit 254. A power supply voltage VDD is supplied to the power supply line Lvdd. The reset transistor 253 extracts charges from the FD unit 254 and initializes them according to the drive signal RST from the vertical scanning circuit 210.

[0048] The FD unit 254 accumulates charges and generates a voltage according to the amount of charges. The amplification transistor 255 amplifies the voltage of the FD unit 254. The amplification transistor 255 forms a constant current source 235 and a source follower circuit connected to one end of the vertical signal line VSL by connecting the source electrode to the vertical signal line VSL via the selection transistor 256.

[0049] The selection transistor 256 outputs the amplified voltage signal as a pixel signal to the column signal processing unit 260 via the vertical signal line VSL in accordance with the drive signal SEL from the vertical scanning circuit 210. Note that the pixel circuit 250 is not limited to the circuit exemplified in the figure as long as it can generate a pixel signal by photoelectric conversion.

[0050] The column signal processing unit 260 includes a comparator 300, a counter 261, and a latch 262. FIG. 3 shows the configuration of the column signal processing unit 260 corresponding to one vertical signal line VSL. Therefore, the configuration of the column signal processing unit 260 in FIG. 2 is arranged for each column. When the number of columns is N (N is an integer), N comparators 300, counters 261, and latches 262 are arranged. Note that the comparators 300, counters 261, and latches 262 according to this embodiment correspond to the AD conversion unit. That is, a plurality of AD conversion units according to this embodiment are configured to correspond to the columns of the pixel circuits 250 (see FIG. 2) arranged in plurality.

[0051] In the case of long accumulation imaging, the comparator 300 compares a first reference signal from the DAC 230 input via the capacitor 302 with a first pixel signal input from the corresponding column via the capacitor 304. On the other hand, in the case of short accumulation imaging, the comparator 300 compares a second reference signal from the DAC 230 input via the capacitor 302 with a second pixel signal input from the corresponding column via the capacitor 304.

[0052] The voltage of the first pixel signal input via the vertical signal line VSL in the case of long accumulation imaging is hereinafter referred to as input voltage Vs1. Meanwhile, the voltage of the second pixel signal input via the vertical signal line VSL in the case of short accumulation imaging is hereinafter referred to as input voltage Vs2. Also, the input voltages Vs1 and Vs2 when the pixel circuit 250 is initialized are hereinafter referred to as "reset levels", and the input voltages Vs1 and Vs2 when the charge is transferred to the FD section 254 are hereinafter referred to as "signal levels".

[0053] The counter 261 performs a count operation in accordance with a counter clock signal from the timing control section 220. The counter 261 counts the count value over a period until the comparison result of the comparator 300 is inverted.

[0054] In the case of long accumulation imaging, the counter 261 performs a first down count from the start of the inclination of the first reference voltage Vramp1 until the comparison result between the first reference voltage Vramp1 and the reset level is inverted. The counter 261 then performs a second down count from the start of the inclination of the first reference voltage Vramp1 until the comparison result between the first reference voltage Vramp1 and the signal level is inverted. This provides a count value that is the difference between the second down count and the first down count. This count value of the difference corresponds to the difference between the reset level and the signal level of the first pixel signal. The counter 261 outputs this count value to the latch 262 as a first digital signal.

[0055] The latch 262 holds the first digital signal and outputs the held first digital signal to the DSP circuit 120 under the control of the horizontal scanning circuit 270.

[0056] On the other hand, in the case of short accumulation imaging, the counter 261 performs a first down count from the start of the inclination of the second reference voltage Vramp2 until the comparison result between the second reference voltage Vramp1 and the reset level is inverted. Then, the counter 261 performs a second down count from the start of the inclination of the second reference voltage Vramp2 until the comparison result between the second reference voltage Vramp2 and the signal level is inverted. This allows a count value of the difference between the second down count and the first down count to be obtained. This count value of the difference corresponds to the difference between the reset level and the signal level of the second pixel signal. The counter 261 outputs this count value to the latch 262 as a second digital signal.

[0057] The latch 262 holds the second digital signal. The latch 262 outputs the held second digital signal to the DSP circuit 120 under the control of the horizontal scanning circuit 270.

[0058] In this way, the comparator 300 and the counter 261 realize AD conversion processing for converting an analog pixel signal into a digital signal. An ADC using these comparators and counters is generally called a single-slope type ADC. Note that the first and second digital signals correspond to the difference between the same reset level and the same signal level, but as described later, the first and second reference signals have different slopes, so that the first and second digital signals have digital values ​​with different resolutions. For example, the first digital signal is 10 bits, and the second digital signal is 8 bits. In addition, the first and second reference signals have different slopes, so that the first and second digital signals have different processing times. That is, it is possible to shorten the time to obtain an image frame of short accumulation compared to the time to obtain an image frame of long accumulation.

[0059] Although the CDS process is realized by counting up and counting down, the present invention is not limited to this configuration. The counter 261 may only count up or count down, and a downstream circuit may execute the CDS process to find the difference.

[0060] 4 is a diagram showing an example of a circuit configuration of the DAC 230. As shown in FIG. 4, the DAC 230 is an example of a current-controlled DAC, and includes a RAMP DAC 231, a PGA-DAC 232, a CLAMP DAC 233, an adder 234, and a current mirror circuit 238. This current-controlled DAC 230 is configured as a ground-referenced DAC based on the ground GND. Note that the RAMP DAC 231 according to this embodiment corresponds to a current source unit, and the CLAMP DAC 233 corresponds to a level adjustment unit. It is also possible to configure the DAC 230 as a power-supply-referenced DAC based on the power supply VDD.

[0061] That is, one end of the reference resistor R is connected to the ground, the other end of the reference resistor R is connected to the output of the lamp DAC 232 and the output of the clamp DAC 233, and the connection point forms an output node N234. The reference resistor R and the output node N234 form an adder 234. The current mirror circuit 238 supplies currents to the PGA-DAC 232 and the clamp DAC 233 at a predetermined ratio. The reference resistor R according to this embodiment corresponds to the first resistor.

[0062] Fig. 5 is a diagram showing a configuration example of the ramp DAC 231. The ramp DAC 231 can change the current supplied to the reference resistor R over time. As shown in Fig. 5, the ramp DAC 231 has a plurality of current sources 236 and a plurality of switches 237. The current sources 236 and the switches 237 are connected in series, and one end is connected to the output node N234. The plurality of switches 237 are selectively turned on (connected state) or off (disconnected state) in response to a control signal from the DAC control unit 222.

[0063] The PGA-DAC 232 controls the output currents of the multiple current sources 236 in response to a control signal from the DAC control unit 222. The clamp DAC 233 switches the output current to be output to the reference resistor R in response to a control signal from the DAC control unit 222. The clamp DAC 233 can adjust the potential of the output node N234 by switching the output current to be output to the reference resistor R.

[0064] An example of down-counting operation will be described with reference to Fig. 6 while referring to Fig. 3. Fig. 6 is a diagram showing a schematic example of a down-counting operation of counter 261. The voltage drop ΔV of reference voltages Vramp1 and Vramp2 varies depending on the magnitude of current I flowing through resistor R. Reference voltages Vramp1 and Vramp2 correspond to the potential of output node N234. When current I is at its maximum value, reference voltages Vramp1 and Vramp2 are at their maximum values. In other words, the potential of output node N234 is at its maximum value when current I is at its maximum value.

[0065] This current I is a current obtained by adding the output current I1 of the clamp DAC 233 and the output current I2 of the lamp DAC 232. As shown in equation (1), the output current I2 is the added value of the current im. The current im (m=1, or 2) is a current output by the current source 236. The current i1 when m=1 is a current in the case of long accumulation imaging, and the current i2 when m=2 is a current in the case of short accumulation imaging. The currents i1 and i2 have a relationship of i2>i1.

number

[0066] The output current I2 of the lamp DAC 231 decreases as the number of the switches 237 (see FIG. 5) that are turned off increases, and becomes 0 when all the switches 237 are turned off. In other words, the output current I2 becomes a maximum current when all the switches 237 are turned on.

[0067] In addition, the maximum values ​​of the reference voltages Vramp1 and Vramp2 can be adjusted depending on the magnitude of the output current I1 of the clamp DAC 233. That is, the clamp DAC 233 adjusts and controls the voltage levels of the reference voltages Vramp1 and Vramp2. As a result, the maximum values ​​of the reference voltages Vramp1 and Vramp2 are controlled to be the same.

[0068] The DAC control unit 222 turns on all the switches 237 in the initial state, and sequentially turns off the switches 237 in synchronization with the counter clock signal, whereby the reference voltages Vramp1 and Vramp2 drop by a voltage drop ΔV=im×R.

[0069] For example, if the ratio of the current i1 in the case of long accumulation imaging and the current i2 in the case of short accumulation imaging is 8:10, the ratio of the rate of decrease of the reference voltage Vramp1 with respect to time and the ratio of the rate of decrease of the reference voltage Vramp2 with respect to time are 8:10.

[0070] As shown in FIG. 6, the reference voltages Vramp1 and 2 decrease with an increase in the counter clock. Then, the reference voltage Vramp1 and the signal level are inverted when the counter clock is 9. On the other hand, the reference voltage Vramp2 and the signal level are inverted when the counter clock is 7. As can be seen from these, as the ratio between the current i1 in the case of long accumulation imaging and the current i2 in the case of short accumulation imaging increases, the speed until inversion in the case of short accumulation imaging becomes faster than the speed until inversion in the case of long accumulation imaging. On the other hand, the AD resolution of the DAC230 in the case of short accumulation imaging decreases compared to the AD resolution of the DAC230 in the case of long accumulation imaging.

[0071] Next, the current output operation of the ramp DAC 231 and the drop in the reference voltage Vramp will be explained in chronological order with reference to Fig. 7 to Fig. 12. Fig. 7 is a diagram showing the state of the ramp DAC 231 when the reference voltage Vramp is at its highest value. The current flowing through the resistor R when it is at its highest value is I. Fig. 8 is a diagram showing the voltage V of the output node N234 in the state of Fig. 7. The voltage V at this time is I×R. The current i is the output current of the current source 236.

[0072] Fig. 9 is a diagram showing a state in which one switch 237 is turned off. The current flowing through resistor R at this time is Ii. Therefore, the voltage of output node N234 is V = (Ii) x R. Fig. 10 is a diagram showing voltage V of output node N234 in the state of Fig. 9. At this time, voltage V = (Ii) x R, and reference voltage Vramp is lower by the differential voltage i x R than in the state of Fig. 8.

[0073] FIG. 11 is a diagram showing a state in which n switches 237 are turned off. The current flowing through resistor R at this time is In×i. Therefore, the voltage of output node N234 is V=(I-ni)×R. FIG. 12 is a diagram showing voltage V of output node N234 in the state of FIG. 11. At this time, voltage V=(In×i)×R, and reference voltage Vramp is lower by the differential voltage (n×i)×R than in the state of FIG. 8. For example, when n is 6, reference voltage Vramp is lower by the differential voltage (6×i)×R.

[0074] Here, the imaging operation for one frame during long-term accumulation imaging will be described using Fig. 13 with reference to Figs. 2 and 3. Fig. 13 is a time chart showing an example of imaging operation for one frame during long-term accumulation imaging. As shown in Fig. 13, from the top, the horizontal synchronization signal XHS of the timing control unit 220, the drive signal TRG of the transfer transistor 252, the drive signal RST of the reset transistor 253, the AutoZero signal of the timing control unit 220, the input voltage Vs1 which is the voltage of the first pixel signal, and the first reference voltage Vramp1 which is the voltage of the first reference signal are shown.

[0075] At time t0a, the horizontal synchronization signal XHS of the timing control unit 220 becomes high level, and control of long-term accumulation imaging is started. Next, at time t1a, the drive signals TRG and RST become high level, the transfer transistor 252 and the reset transistor 253 become conductive, and the accumulated charges of the photoelectric conversion element 251 and the FD unit 254 are discharged. After the drive signal TRG becomes low level, the drive signal RST becomes low level, and exposure by long-term accumulation is started from time tsl.

[0076] Next, at time ts, the driving signal RST goes high again, and the accumulated charge in the FD section 254 is discharged. At the same time, the auto-zero signal goes high, and the timing control section 220 performs a so-called AZ (AutoZero) operation to match the first reference voltage Vramp1 with the reference of the vertical driving line VSL. Then, the auto-zero signal goes low, and the DAC 230 starts generating the first reference signal.

[0077] The counter 261 generates a first down-count value P' over a period from time t2a when the ramp of the first reference voltage Vramp1 starts to time t3a when the comparison result between the first reference voltage Vramp1 and the reset level of the input voltage Vs1 is inverted.

[0078] Next, at time t4a, the driving signal TRG becomes high level, and the accumulated charge in the long-term accumulation of the photoelectric conversion element 251 is transferred to the FD section 254. The counter 261 generates a second down-count value D' over the period from time t5a when the ramp of the first reference voltage Vramp1 starts to time t6a when the comparison result between the first reference voltage Vramp1 and the signal level of the input voltage Vs1 is inverted. The counter 261 outputs a count value, which is the difference between the first down-count value P' and the second down-count value D', to the latch 262 as a first digital signal.

[0079] Next, at time t7a, the DAC 230 stops the voltage drop of the first reference signal. Then, at time t8a, the horizontal synchronization signal XHS of the timing control section 220 goes to low level, and the control of the long-term accumulation imaging ends.

[0080] Here, the imaging operation for one frame during short accumulation imaging will be described using FIG. 14 with reference to FIGS. 2 and 3. FIG. 14 is a time chart showing an example of imaging operation for one frame during short accumulation imaging. As shown in FIG. 14, from the top, the horizontal synchronization signal XHS of the timing control unit 220, the drive signal TRG of the transfer transistor 252, the drive signal RST of the reset transistor 253, the autozero (AutoZero) signal of the timing control unit 220, the input voltage Vs2 which is the voltage of the second pixel signal, and the second reference voltage Vramp2 which is the voltage of the second reference signal are shown. As described above, the slopes of the second reference voltage Vramp2 and the first reference voltage Vramp1 are different during short accumulation imaging and during long accumulation imaging. Note that, in order to simplify the comparative description, an example in which the charge amount generated by the photoelectric conversion element 251 during short accumulation imaging and during long accumulation imaging is the same will be described. That is, the input voltage Vs2 has the same shape as the input voltage Vs1.

[0081] At time t0b, the horizontal synchronization signal XHS becomes high level, and control of short-time accumulation imaging is started. Next, at time t1b, the drive signal TRG and the drive signal RST become high level, the transfer transistor 252 and the reset transistor 253 become conductive, and the accumulated charges of the photoelectric conversion element 251 and the FD section 254 are discharged. After the drive signal TRG becomes low level, the drive signal RST becomes low level, and imaging with short-time accumulation is started from time tss.

[0082] Next, at time ts, the driving signal RST goes high again, and the accumulated charge in the FD section 254 is discharged. At the same time, the auto-zero signal goes high, and the timing control section 220 performs a so-called AZ (AutoZero) operation to align the second reference voltage Vramp2 with the reference of the vertical driving line VSL. Then, the auto-zero signal goes low, and the DAC 230 starts generating the second reference signal.

[0083] The counter 261 generates the first down-count value P over the period from time t2b when the ramp of the second reference voltage Vramp2 starts to time t3b when the comparison result between the second reference voltage Vramp2 and the reset level of the input voltage Vs2 is inverted.

[0084] Next, at time t4b, the drive signal TRG goes high, and the accumulated charge in the short-term accumulation of the photoelectric conversion element 251 is transferred to the FD section 254. The counter 261 generates a second down-count value D over a period from time t5b when the ramp of the second reference voltage Vramp2 starts to time t6b when the comparison result between the second reference voltage Vramp2 and the signal level of the input voltage Vs2 is inverted. The counter 261 outputs a count value, which is the difference between the first down-count value P and the second down-count value D, to the latch 262 as a second digital signal.

[0085] Next, at time t7b, the DAC 230 stops the voltage drop of the second reference signal. Then, at time t8b, the horizontal synchronization signal XHS of the timing control section 220 goes to the low level, and the control of short-time accumulation imaging ends.

[0086] 15 is a timing chart in which the long accumulation imaging operation and the short accumulation imaging operation are superimposed. From the top, there are shown an input voltage Vs1 which is the voltage of the first pixel signal, a first reference voltage Vramp1 which is the voltage of the first reference signal, an input voltage Vs2 which is the voltage of the second pixel signal, and a second reference voltage Vramp2 which is the voltage of the second reference signal.

[0087] 15, the period from time t2a when the first reference voltage Vramp1 starts its initial voltage drop to time t7a when the generation of the first reference voltage Vramp1 ends is defined as a first period. Meanwhile, the period from time t2b when the second reference voltage Vramp2 starts its initial voltage drop to time t7b when the generation of the first reference voltage Vramp1 ends is defined as a second period. It can be seen that the first period and the second period are the difference D12 between the addition time of the first down-count value P' and the second down-count value D' and the addition time of the first down-count value P and the second down-count value D.

[0088] FIG. 16 is a diagram showing a schematic diagram of an analog-digital conversion period in one frame of long accumulation data and short accumulation data. The horizontal axis indicates time, and the vertical axis indicates the position of a row in the vertical direction of the pixel array section 240. Line L10 indicates the corresponding relationship between the row in the pixel array section 240 (see FIG. 1) in long accumulation imaging and the readout time. Similarly, line L20 indicates the corresponding relationship between the row in the pixel array section 240 (see FIG. 1) in short accumulation imaging and the readout time. In short accumulation imaging, the time of analog-digital conversion in row units is shortened compared to long accumulation imaging. Thereby, the analog-digital conversion period in one frame unit of short accumulation imaging is shortened compared to the analog-digital conversion period in one frame unit of long accumulation imaging.

[0089] 17 is a diagram showing a schematic processing order of the HDR image acquisition process. From the top, the exposure time, the data conversion period, and the data transfer period are shown. When the short accumulation exposure is completed, the solid-state imaging element 200 (see FIGS. 1 and 2) performs conversion to short accumulation data and image-transfers the short accumulation data to the DSP circuit 120 via the signal line 209. Similarly, when the long accumulation exposure is completed, the solid-state imaging element 200 (see FIGS. 1 and 2) performs conversion to long accumulation data and image-transfers the long accumulation data to the DSP circuit 120 via the signal line 209. Then, the long accumulation data and the short accumulation data are stored in the storage unit 170 as the first image data and the second image data, respectively.

[0090] 18 is a diagram showing a schematic diagram of the concept of image synthesis of long accumulation data and short accumulation data. The horizontal axis indicates the brightness of the subject, and the vertical axis indicates the magnitude of the analog image signal generated by each pixel circuit 250 (see FIG. 3). That is, it corresponds to the signal levels of the input voltages Vs1 and Vs2 when the charge is transferred to the FD section 254 (see FIG. 3). In this way, even if a subject of the same brightness is imaged, the signal levels of the input voltages Vs1 and Vs2 differ due to the difference in exposure time.

[0091] For this reason, the processing unit 180 (see FIG. 1) multiplies the short accumulation data by the gain difference caused by the difference in exposure time as G1. Furthermore, the short accumulation data is further multiplied by the difference in data resolution (number of bits) caused by the difference between the slope of the first reference voltage Vramp1 and the slope of the second reference voltage Vramp2 as B1. Thereby, the long accumulation data and the short accumulation data are made to correspond to each other so as to have the same value for the same brightness.

[0092] As shown in FIG. 18, the long accumulation data is saturated when, for example, the signal level of the input voltage Vs1 exceeds a value corresponding to 1023. For this reason, the processing unit 180 (see FIG. 1) sets the weight w1 of the short accumulation data to 1, the weight w2 of the long accumulation data to 1-w1, and sets w1 to 1 on the high illuminance side of the line L40, and adds the long accumulation data and the short accumulation data. On the other hand, on the low illuminance side of the line L40, the weight w1 is decreased from 1 as the illuminance becomes lower. By performing such a synthesis process, HDR (High Dynamic Range) image data is generated. The resolution of the image data on the high illuminance side shows a tendency to be lower than the resolution of the image data on the low illuminance side. However, since the visual differentiation ability of humans becomes lower as the illuminance becomes higher, the influence on the image to be viewed is suppressed.

[0093] As described above, according to this embodiment, the first image data of the long accumulation imaging and the second image data of the short accumulation imaging are added with a predetermined weighting to generate the composite processing image data. This makes it possible to expand the range of the measurable high illuminance region of the composite processing image data, and to expand the dynamic range. In this case, the second reference voltage Vramp2, which has a faster rate of decrease with respect to time, is used when generating the second image data of the short accumulation imaging, so that the analog-to-digital conversion can be accelerated. In addition, the second image data of the short accumulation imaging shows a tendency to have a lower image data resolution (number of bits) than the first image data of the long accumulation imaging, but since the visual differentiation ability of humans becomes lower as the illuminance increases, the influence on the composite processing image that is visually recognized is suppressed.

[0094] Second embodiment The imaging device 100 according to the second embodiment differs from the imaging device 100 according to the first embodiment in that the pixel circuit 250A has a plurality of pixels with different light receiving sensitivities. The differences from the imaging device 100 according to the first embodiment will be described below.

[0095] 19 is a diagram showing an example of the configuration of a pixel circuit 250A according to the second embodiment. The same components as those in the pixel circuit 250 according to the first embodiment are given the same reference numbers, and the description thereof may be omitted.

[0096] 19, the pixel circuit 250A is configured to include a second photoelectric conversion element 103, a second transfer transistor 104, a third transfer transistor 105, a charge accumulation unit 106, a photoelectric conversion element 251, a transfer transistor 252, a reset transistor 253, an FD unit 254, an amplification transistor 255, and a selection transistor 256. Each transistor of the pixel circuit 250A is, for example, an NMOS transistor.

[0097] Furthermore, the vertical scanning circuit 210 (see FIG. 2) supplies various drive signals TRG, TGS, FCG, RST, and SEL to the pixel circuit 250A. Because each transistor of the pixel circuit 250A is an NMOS transistor, these drive signals TRG, TGS, FCG, RST, and SEL are signals that are in an active state when they are at a high level (e.g., a power supply voltage VDD) and in an inactive state when they are at a low level (e.g., a negative potential).

[0098] The photoelectric conversion element 251 is, for example, a PN junction photodiode. The photoelectric conversion element 251 generates and accumulates electric charges according to the amount of light received.

[0099] The transfer transistor 252 is connected between the photoelectric conversion element 251 and the FD section 254. A drive signal TRG is applied to a gate electrode of the transfer transistor 252. When the drive signal TRG becomes active, the transfer transistor 252 becomes conductive, and the charge accumulated in the photoelectric conversion element 251 is transferred to the FD section 254 via the transfer transistor 252.

[0100] The second photoelectric conversion element 103 is, for example, a PN junction photodiode, similar to the photoelectric conversion element 251. The second photoelectric conversion element 103 generates and accumulates electric charges according to the amount of light received.

[0101] Comparing the photoelectric conversion element 251 and the second photoelectric conversion element 103, the photoelectric conversion element 251 has a larger light receiving surface area and higher sensitivity, while the second photoelectric conversion element 103 has a smaller light receiving surface area and lower sensitivity.

[0102] The second transfer transistor 104 is connected between the charge accumulation unit 106 and the FD unit 254. A drive signal FCG is applied to the gate electrode of the second transfer transistor 104. When the drive signal FCG becomes active, the second transfer transistor 104 becomes conductive, and the potentials of the charge accumulation unit 106 and the FD unit 254 are averaged.

[0103] The third transfer transistor 105 is connected between the second photoelectric conversion element 103 and the charge accumulation unit 106. A drive signal TGS is applied to a gate electrode of the third transfer transistor 105. When the drive signal TGS becomes active, the third transfer transistor 105 becomes conductive, and the charge accumulated in the second photoelectric conversion element 103 is transferred via the third transfer transistor 105 to the charge accumulation unit 106 or to a region where the potentials of the charge accumulation unit 106 and the FD unit 254 are averaged.

[0104] In addition, the potential is slightly deeper below the gate electrode of the third transfer transistor 105, and an overflow path is formed that transfers charge that exceeds the saturation charge amount of the second photoelectric conversion element 103 and overflows from the second photoelectric conversion element 103 to the charge accumulation section 106. Note that hereinafter, the overflow path formed below the gate electrode of the third transfer transistor 105 will be simply referred to as the overflow path of the third transfer transistor 105.

[0105] The charge accumulation unit 106 is, for example, a capacitor, and is connected between the second transfer transistor 104 and the third transfer transistor 105. The counter electrode of the charge accumulation unit 106 is connected between a power supply VDD that supplies a power supply voltage VDD. The charge accumulation unit 106 accumulates the charge transferred from the second photoelectric conversion element 103.

[0106] The reset transistor 253 is connected between a power supply VDD and the FD unit 254. A drive signal RST is applied to the gate electrode of the reset transistor 253. When the drive signal RST becomes active, the reset transistor 253 becomes conductive, and the potential of the FD unit 254 is reset to the level of the power supply voltage VDD. The FD unit 254 converts the charge into a voltage signal and outputs the voltage signal.

[0107] The amplifying transistor 255 has a gate electrode connected to the FD section 254 and a drain electrode connected to a power supply VDD, and serves as an input section of a read circuit that reads out the charges held in the FD section 254, that is, a so-called source follower circuit. That is, the amplifying transistor 255 has a source electrode connected to the vertical signal line VSL via the selection transistor 256, and thereby constitutes a source follower circuit together with the constant current source 235 connected to one end of the vertical signal line VSL.

[0108] The selection transistor 256 is connected between the source electrode of the amplification transistor 255 and the vertical signal line VSL. A drive signal SEL is applied to the gate electrode of the selection transistor 256. When the drive signal SEL becomes active, the selection transistor 256 becomes conductive and the pixel circuit 250A becomes selected. As a result, the pixel signal output from the amplification transistor 255 is output to the vertical signal line VSL via the selection transistor 256.

[0109] In the following description, when each drive signal becomes active, it is also referred to as the drive signal being turned on, and when each drive signal becomes inactive, it is also referred to as the drive signal being turned off.

[0110] {Operation of pixel circuit 250A} Next, the operation of the pixel circuit 250A will be described with reference to the timing charts of Fig. 20 and Fig. 21. Fig. 20 is a timing chart showing an example of a storage operation of the pixel circuit 250A. Fig. 21 is a timing chart showing an example of a readout operation of the pixel circuit 250A.

[0111] (Operation of pixel circuit 250A at the start of exposure) First, the operation of the pixel circuit 250A at the start of exposure will be described with reference to the timing chart of Fig. 20. This process is performed in a predetermined scanning order for each pixel row or for each set of multiple pixel rows in the pixel array section 240 (see Fig. 2). Note that Fig. 20 shows a timing chart of the horizontal synchronization signal XHS and the drive signals SEL, RST, TGS, FCG, and TRG.

[0112] First, at time t1, the horizontal synchronization signal XHS is input, and the exposure process of the pixel circuit 250A starts. Next, at time t2, the drive signal RST is turned on, and the reset transistor 253 is turned on. This resets the potential of the FD section 254 to the level of the power supply voltage VDD.

[0113] Next, at time t3, the drive signals TRG, FCG, and TGS are turned on, and the transfer transistor 252, the second transfer transistor 104, and the third transfer transistor 105 are turned on. This averages the potentials of the charge storage section 106 and the FD section 254. Also, the charge accumulated in the photoelectric conversion element 251 is transferred to the combined region via the transfer transistor 252, and the charge accumulated in the second photoelectric conversion element 103 is transferred to the combined region via the third transfer transistor 105. Then, the combined region is reset.

[0114] Next, at time t4, the drive signals TRG and TGS are turned off, and the transfer transistor 252 and the third transfer transistor 105 are turned off. This starts accumulation of charges in the photoelectric conversion element 251 and the second photoelectric conversion element 103, and the exposure period begins.

[0115] Next, at time t5, the drive signal RST is turned off, and the reset transistor 253 is turned off. Next, at time t6, the drive signal FCG is turned off, and the second transfer transistor 104 is turned off. As a result, the charge accumulation unit 106 starts accumulating the charge that overflows from the second photoelectric conversion element 103 and is transferred via the overflow path of the third transfer transistor 105. Then, at time t7, the horizontal synchronization signal XHS is input.

[0116] (Readout Operation of Pixel Circuit 250A) Next, the operation of the pixel circuit 250A when reading out pixel signals will be described with reference to the timing chart of Fig. 21. This process is performed, for example, for each pixel row of the pixel array section 240, or for each set of multiple pixel rows, in a predetermined scanning order a predetermined time after the process of Fig. 20 is performed. Note that Fig. 20 shows a timing chart of the horizontal synchronization signal XHS, and the drive signals SEL, RST, TGS, FCG, and TRG. At this time,

[0117] First, at time t21, the horizontal synchronization signal XHS is input, and the readout period of the pixel circuit 250A begins. Next, at time t22, the drive signal SEL is turned on, and the selection transistor 256 is turned on. This causes the pixel circuit 250A to enter a selected state.

[0118] Next, at time t23, the drive signal RST is turned on, turning on the reset transistor 253. This resets the potential of the FD section 254 to the level of the power supply voltage VDD.

[0119] Next, at time t24, the drive signal RST is turned off, and the reset transistor 253 is turned off. Next, at time t25, the drive signals FCG and TGS are turned on, and the second transfer transistor 104 and the third transfer transistor 105 are turned on. As a result, the potentials of the charge accumulation unit 106 and the FD unit 254 are averaged, and the charge accumulated in the second photoelectric conversion element 103 is transferred to the combined region. As a result, the charge accumulated in the second photoelectric conversion element 103 and the charge accumulation unit 106 during the exposure period is accumulated in the combined region.

[0120] At this time t25, the readout of the pixel signal starts, and the exposure period ends. Next, at time t26, the drive signal TGS is turned off, and the third transfer transistor 105 is turned off. This stops the transfer of charge from the second photoelectric conversion element 103.

[0121] Next, at time ta between time t26 and time t27, a signal SL based on the electric potential of the area where the electric potentials of the charge storage unit 106 and the FD unit 254 are averaged is output to the vertical signal line VSL via the amplification transistor 255 and the selection transistor 256. The signal SL is a signal based on the electric charge generated by the second photoelectric conversion element 103 during the exposure period and accumulated in the second photoelectric conversion element 103 and the charge storage unit 106. The signal SL is also a signal based on the electric potential of the combined area in a state where the electric charge accumulated in the second photoelectric conversion element 103 and the charge storage unit 106 during the exposure period is accumulated in the area where the electric potentials of the charge storage unit 106 and the FD unit 254 are averaged. Therefore, the capacity for charge-voltage conversion of the electric charge when reading out the signal SL is the combined capacity of the charge storage unit 106 and the FD unit 254. Hereinafter, the signal SL is also referred to as a low-sensitivity data signal SL.

[0122] The low-sensitivity data signal SL is input to the column signal processing unit 260 (see FIG. 3) and is converted from analog to digital. At this time, the DAC 230 supplies a second reference signal equivalent to the period from t4b to t8b in FIG. 14 to the column signal processing unit 260. As a result, for example, an 8-bit second downcount D is acquired.

[0123] The slope of the second reference voltage Vramp2 in the second reference signal is steeper than the slope of the first reference voltage Vramp1 in the first reference signal, which will be described later. Therefore, the generation time of the second downcount D of the low-sensitivity data signal SL is shorter than the generation time of the second downcount D' of the high-sensitivity data signal SH, which will be described later. As a result, the period SL is shorter than the period SH.

[0124] Next, at time t27, the drive signal RST is turned on, and the reset transistor 253 is turned on. This resets the area where the potentials of the charge storage section 106 and the FD section 254 are averaged.

[0125] Next, at time t28, the selection signal SEL is turned off, turning off the selection transistor 256. This causes the pixel circuit 250A to enter a non-selected state.

[0126] Next, at time t29, the drive signal RST is turned off, turning off the reset transistor 253. Next, at time t30, the selection signal SEL is turned on, turning on the selection transistor 256. This causes the pixel circuit 250A to enter a selected state.

[0127] Next, at time tb between time t30 and time t31, a signal NL based on the potential of the region where the potentials of the charge accumulation unit 106 and the FD unit 254 are averaged is output to the vertical signal line VSL via the amplification transistor 255 and the selection transistor 256. This signal NL is a signal based on the potential of the combined region in a state where the region where the potentials of the charge accumulation unit 106 and the FD unit 254 are averaged is reset. Hereinafter, the signal NL is also referred to as a low-sensitivity reset signal NL.

[0128] The low-sensitivity reset signal NL is input to the column signal processing unit 260 (see FIG. 3) and is analog-to-digital converted. At this time, the DAC 230 supplies a second reference signal equivalent to the period ts1 to t4b in FIG. 14 to the column signal processing unit 260. As a result, a first down-count P of, for example, 8 bits is acquired. The counter 261 (see FIG. 3) outputs a count value, which is the difference between the first down-count value P and the second down-count value D, to the latch 262 as a second digital signal. Then, the latch 262 outputs the second digital signal to the DSP circuit 120.

[0129] The slope of the second reference voltage Vramp2 in the second reference signal is steeper than the slope of the first reference voltage Vramp1 in the first reference signal, which will be described later. Therefore, the generation time of the first downcount P of the low-sensitivity reset signal NL is shorter than the generation time of the first downcount P' of the high-sensitivity reset signal NH, which will be described later. As a result, the period NL is smaller than the period NH.

[0130] Next, at time t31, the drive signal FCG is turned off, and the second transfer transistor 104 is turned off. Next, at time tc between time t31 and time t32, a signal NH based on the potential of the FD section 254 is output to the vertical signal line VSL via the amplification transistor 255 and the selection transistor 256. The signal NH is a signal based on the potential of the FD section 254 in a reset state. Note that hereinafter, the signal NH is also referred to as a high-sensitivity reset signal NH.

[0131] The high-sensitivity reset signal NH is input to the column signal processing unit 260 (see FIG. 3) and is converted from analog to digital. At this time, the DAC 230 supplies a second reference signal equivalent to the period from ts1 to t4a in FIG. 13 to the column signal processing unit 260. As a result, a first downcount P' of, for example, 10 bits is acquired.

[0132] Next, at time t32, the drive signal TRG is turned on, and the transfer transistor 252 is turned on. As a result, the charge generated and accumulated in the photoelectric conversion element 251 during the exposure period is transferred to the FD section 254 via the transfer transistor 252.

[0133] Next, at time t33, the drive signal TRG is turned off, and the transfer transistor 252 is turned off. This causes the transfer of charges from the photoelectric conversion element 251 to the FD section 254 to stop.

[0134] Next, at time td between time t33 and time t34, a signal SH based on the potential of the FD section 254 is output to the vertical signal line VSL via the amplification transistor 255 and the selection transistor 256. The signal SH is a signal based on the charge generated and accumulated in the photoelectric conversion element 251 during the exposure period. The signal SH is also a signal based on the potential of the FD section 254 in a state in which the charge accumulated in the photoelectric conversion element 251 during the exposure period is accumulated in the FD section 254. Therefore, the capacity for charge-to-voltage conversion during reading of the signal SH is the capacity of the FD section 254, which is smaller than that during reading of the low-sensitivity data signal SH at time ta. Hereinafter, the signal SH is also referred to as a high-sensitivity data signal SH.

[0135] The high-sensitivity data signal SH is input to the column signal processing unit 260 (see FIG. 3) and is analog-to-digital converted. At this time, the DAC 230 supplies the column signal processing unit 260 with a second reference signal equivalent to the period from t4b to t8b in FIG. 14. As a result, a second down count D' of, for example, 10 bits is acquired. The counter 261 (see FIG. 3) outputs a count value, which is the difference between the first down count value P' and the first down count value D', as a first digital signal to the latch 262. Then, the latch 262 outputs the first digital signal to the DSP circuit 120.

[0136] Next, at time t34, the selection signal SEL is turned off, and the selection transistor 256 is turned off. This causes the pixel circuit 250A to enter a non-selected state. Next, at time t35, the horizontal synchronization signal XHS is input, and the readout period of the pixel signal of the pixel circuit 250A ends.

[0137] The first digital signal is high sensitivity data and corresponds to long accumulation data, while the second digital signal is low sensitivity data and corresponds to short accumulation data.

[0138] The processing unit 180 (see FIG. 1) multiplies the low-sensitivity data by a gain difference caused by the difference in sensitivity as G2. Furthermore, the low-sensitivity data is further multiplied by a difference in data resolution (number of bits) caused by the difference between the slope of the first reference voltage Vramp1 and the slope of the second reference voltage Vramp2 as B1. This allows the high-sensitivity data and the low-sensitivity data to correspond to each other so that they have the same value for the same brightness.

[0139] Moreover, the processing unit 180 (see FIG. 1) adds the low-sensitivity data and the high-sensitivity data, setting the weight w1 for the low-sensitivity data and the weight w2 for the high-sensitivity data (w2=1-w1). In this case, the weight w1 decreases as the illuminance decreases. By performing such a synthesis process, HDR (High Dynamic Range) image data is generated. The resolution (number of bits) of image data on the high-illuminance side tends to be lower than the resolution of image data on the low-illuminance side. However, since the human visual differentiation ability decreases as the illuminance increases, the effect on the visually recognized image is suppressed.

[0140] As described above, according to this embodiment, the composite processed image data is generated by adding the low sensitivity image data and the high sensitivity image data with a predetermined weighting. This makes it possible to expand the range of the high illuminance region that can be measured in the composite processed image data, and to expand the dynamic range. In this case, the second reference voltage Vramp2, which decreases faster with time, is used when generating the low sensitivity image data, so that the analog-to-digital conversion can be accelerated. In addition, the low sensitivity image data tends to have a lower image resolution (number of bits) than the high sensitivity image data, but since the human visual differentiation ability decreases as the illuminance increases, the effect on the composite processed image that is visually recognized is suppressed.

[0141] Third embodiment The imaging device 100 according to the third embodiment differs from the imaging device 100 according to the first embodiment in that it has a storage transistor 258 and a storage capacitance element 259. The differences from the imaging device 100 according to the first embodiment will be described below.

[0142] FIG. 22 is a diagram showing a detailed configuration example of a pixel circuit 250B and a column signal processing unit 260 according to the third embodiment. The same components as those in the pixel circuit 250 according to the first embodiment are denoted by the same reference numerals, and the description thereof may be omitted. As shown in FIG. 22, the pixel circuit 250B includes a photoelectric conversion element 251, a transfer transistor 252, a reset transistor 253, an FD unit 254, an amplification transistor 255, a selection transistor 256, a storage transistor 258, and a storage capacitance element 259. That is, the pixel circuit 250B differs from the pixel circuit 250 according to the first embodiment in that it further includes the storage transistor 258 and the storage capacitance element 259.

[0143] One end of the storage transistor 258 is connected to the FD section 254, and the other end is connected to the storage capacitance element 259. When the storage transistor 258 is on (connected state), it connects the FD section 254 and the storage capacitance element 259 in parallel in accordance with the drive signal TRS from the vertical scanning circuit 210. This makes it possible to switch the storage capacitance of the photoelectric charge generated by the photoelectric conversion element 251 between two stages when the storage transistor 258 is on (connected state) and when it is off (disconnected state). Note that the FD section 254 according to this embodiment corresponds to a first charge storage section, and the storage capacitance element 259 corresponds to a second charge storage section.

[0144] The FD section 254 has a capacitance CFD, and the storage capacitance element 259 has a capacitance CS. In addition, the photoelectric conversion element 251 forms a capacitance CPD having a lower potential than the capacitance CFD and the capacitance CS.

[0145] Fig. 23 is a timing chart showing the drive signals RST, TRG, and TRS at two levels. As shown in Fig. 23, at time T0b when a new field starts, RST is turned on with TRG turned off and TRS turned on, to drain and reset all the photoelectric charges generated in the previous field.

[0146] The accumulation period (approximately equivalent to the video period) TPD for the capacitance CPD starts from the point in time when TRG is turned off just before time T0b, and accumulation of photocharges begins in the capacitance CPD. Note that immediately after time T0b, TRG is at the (+α) level.

[0147] Next, at time T1b, a certain time after the start of the video time, RST is turned off. At this time, since TRS is on, the capacitance CFD and capacitance CS are in a coupled state, and so-called kTC noise accompanying the reset operation occurs in capacitance CFD + capacitance CS immediately after reset. Here, the reset level signal of capacitance CFD + capacitance CS is read out as noise N2.

[0148] The method that can improve the S / N ratio the most is to read out the noise N2, store it in a frame memory (storage means) described later, and use the noise N2 when generating an image signal, but in the case of oversaturation, the noise N2 is sufficiently small compared to the pre-saturation charge (low illuminance signal) + oversaturation charge (high illuminance signal), so the noise N1 described later may be used instead of the noise N2. Also, the noise N2 of the next frame may be used instead of the noise N2 of the current frame.

[0149] At time T1b, the storage capacitor element storage period TCS for the capacitance CS starts, and the photocharges overflowing from the photoelectric conversion element 251 start to be stored in the capacitance CS.

[0150] In this way, when the amount of photocharge is less than the amount that saturates capacitance CPD, photocharge accumulates only in capacitance CPD, and when the amount of photocharge is more than the amount that saturates capacitance CPD, photocharge accumulates not only in capacitance CPD but also in capacitance CFD and capacitance CS.

[0151] Next, TRG is turned off from the (+α) level, and at the end of the storage capacitance element storage period TCS, TRS is turned off to divide the potential of capacitance CFD and capacitance CS. This ends the storage in capacitance CS. Next, RST is turned on to drain and reset the photocharge in capacitance CFD.

[0152] Next, at time T2b, immediately after RST is turned off and the reset is completed, kTC noise is newly generated in the capacitive CFD. Here, the reset level signal of this capacitive CFD is read out as noise N1.

[0153] Next, TRG is turned on to transfer the pre-saturation charge QB in the capacitance CPD to the capacitance CFD. Here, the potential of the capacitance CPD is shallower than that of the capacitance CFD, and the level of the transfer transistor 252 is deeper than that of the capacitance CPD, so complete charge transfer can be achieved in which all of the pre-saturation charge QB in the capacitance CPD is transferred to the capacitance CFD. Here, TRG is turned off again at time T3b, and the pre-saturation charge signal S1 is read out from the pre-saturation charge QB transferred to the capacitance CFD. However, since capacitance CFD noise is present here, what is actually read out is S1+N1.

[0154] Next, by turning on TRS and then turning on TRG, the potentials of the capacitances CFD and CS are coupled, and the pre-saturated charge QB in the capacitance CFD and the supersaturated charge QA in the capacitance CS are mixed. Here, at time T4b, TRG is turned off again, and the sum signal of the pre-saturated charge signal S1 and the supersaturated charge signal S2 is read out from the pre-saturated charge QB + the supersaturated charge QA that spreads to the capacitances CFD and CS. However, since the capacitances CFD and CS have noise and are read out from the charge that spreads to the capacitances CFD and CS, what is actually read out is S1'+S2'+N2) (S1' and S2' are the values ​​of S1 and S2 that are modulated by the capacitance ratio of the capacitances CFD and CS, respectively).

[0155] Next, as described above, with TRG turned off and TRS turned on, RST is turned on to drain all the photoelectric charges generated in this field, reset, and move on to the next field.

[0156] Next, the analog-to-digital conversion of the pixel having the above configuration will be described. As described above, the four values ​​of the pre-saturated charge signal (S1)+capacitive CFD noise (N1), capacitive CFD noise (N1), modulated pre-saturated charge signal (S1')+modulated oversaturated charge signal (S2')+capacitive CFD+capacitive CS noise (N2), and capacitive CFD+capacitive CS noise (N2) are output to the column signal processor 260 via the vertical signal line VSL at the respective timings.

[0157] In this embodiment, the first reference voltage Vramp1 (see FIG. 6) is used when taking the difference between the pre-saturated charge signal (S1)+the capacitive CFD noise (N1) and the capacitive CFD noise (N1). On the other hand, the second reference voltage Vramp2 is used when taking the difference between the modulated pre-saturated charge signal (S1')+the modulated oversaturated charge signal (S2')+the capacitive CFD+the capacitive CS noise (N2) and the capacitive CFD+the capacitive CS noise (N2). That is, the first reference voltage Vramp1 (see FIG. 6) is used when taking the difference between the pre-saturated charge signal (S1)+the capacitive CFD noise (N1) stored in the capacitive CFD and the capacitive CFD noise (N1). Also, the second reference voltage Vramp2 is used when taking the difference between the pre-saturated charge signal (S1')+the modulated oversaturated charge signal (S2')+the capacitive CFD+the capacitive CS noise (N2) stored in the capacitive CFD+the capacitive CS and the capacitive CFD+the capacitive CS noise (N2).

[0158] That is, the counter 261 (see FIG. 3) sets the capacitive CFD noise (N1) as a first down count P', sets the pre-saturation charge signal (S1)+capacitive CFD noise (N1) as a second down count D', and outputs a count value that is the difference between the first down count value P' and the second down count value D' as a first digital signal to the latch 262. For example, the first digital signal is 10 bits.

[0159] Similarly, the counter 261 (see FIG. 3) sets the capacitive CFD+capacitive CS noise (N2) as a first down count P, sets the pre-saturated charge signal (S1)+capacitive CFD noise (N1) as a second down count D, and outputs a count value that is the difference between the first down count value P and the second down count value D as a second digital signal to the latch 262. For example, the second digital signal is 8 bits. That is, the conversion time of the analog conversion process of the second digital signal is shorter than the conversion time of the analog conversion process of the first digital signal.

[0160] An example of processing by the processing unit 180 (see FIG. 1) will be described. As described above, the difference in data resolution (number of bits) caused by the difference between the slope of the first reference voltage Vramp1 and the slope of the second reference voltage Vramp2 is denoted as B1. The processing unit 180 multiplies the second digital signal by B1 to eliminate the difference in resolution.

[0161] S1', S2', α (charge distribution ratio from capacitance CFD to capacitance CFD+capacitor CS), and β (charge distribution ratio from capacitance CS to capacitance CFD+capacitor CS) are expressed by the following formulas.

[0162] S1' = S1 × α (2) S2' = S2 × α × β (3) α=Capacity CFD / (Capacity CFD+Capacity CS) (4) β=Capacity CS / (Capacity CFD+Capacity CS) (5)

[0163] The processing unit 180 calculates α and β from the capacitance CFD and capacitance CS using the above formulas (4) and (5), and substitutes the calculated values ​​into the above formulas (2) and (3) to restore S1+S2 and adjust the gain to the same as S1 obtained separately. Note that S1 corresponds to the high sensitivity side (dark region), and S1+S2 corresponds to the low sensitivity side (light region).

[0164] Then, the processing unit 180 selects either S1 or S1+S2 as the final output. For example, if S1 is equal to or less than a predetermined value (a preset reference potential V0), the processing unit 180 outputs S1, otherwise it outputs S1+S2. The reference potential V0 is a potential before saturation according to the capacity of the photoelectric conversion element 251.

[0165] As described above, in the imaging device 100 of this embodiment, two signals, the pre-saturated charge signal (S1) and the sum (S1+S2) of the pre-saturated charge signal and the oversaturated charge signal, are obtained per pixel circuit 250B, and it is possible to select either S1 or S1+S2 by determining whether the photoelectric conversion element 251 (capacitor CPD) is actually saturated or close to it. This makes it possible to expand the range of the high illuminance region that can be measured, and to expand the dynamic range. In this case, the second reference voltage Vramp2 is used when taking the difference between the modulated pre-saturated charge signal (S1')+modulated oversaturated charge signal (S2')+capacitor CFD+capacitor CS noise (N2) and capacitor CFD+capacitor CS noise (N2), so that the analog-to-digital conversion can be performed at a high speed.

[0166] <<Application Examples>> The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, a construction machine, an agricultural machine (tractor), etc.

[0167] 24 is a block diagram showing a schematic configuration example of a vehicle control system 7000 which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 24, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

[0168] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various control target devices. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle by wired communication or wireless communication. In FIG. 24, a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690 are illustrated as the functional configuration of the integrated control unit 7600. Other control units also include a microcomputer, a communication I / F, a storage unit, and the like.

[0169] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle according to various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle. The drivetrain control unit 7100 may also function as a control device such as an ABS (Antilock Brake System) or an ESC (Electronic Stability Control).

[0170] A vehicle state detection unit 7110 is connected to the drive system control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of an accelerator pedal, the amount of operation of a brake pedal, the steering angle of a steering wheel, the engine rotation speed, or the rotation speed of wheels, for example. The drive system control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0171] The body system control unit 7200 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 7200. The body system control unit 7200 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0172] The battery control unit 7300 controls the secondary battery 7310, which is a power supply source for the drive motor, according to various programs. For example, information such as battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs calculation processing using these signals, and controls the temperature regulation of the secondary battery 7310 or controls a cooling device or the like equipped in the battery device.

[0173] The outside-vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside-vehicle information detection unit 7420 is connected to the outside-vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside-vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, and the like around the vehicle equipped with the vehicle control system 7000.

[0174] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may be provided as independent sensors or devices, or may be provided as a device in which multiple sensors or devices are integrated.

[0175] Here, FIG. 25 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield in the vehicle interior of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0176] 25 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, image data captured by the imaging units 7910, 7912, 7914, and 7916 are superimposed to obtain an overhead image of the vehicle 7900.

[0177] The outside information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, and corners of the vehicle 7900 and on the upper part of the windshield inside the vehicle cabin may be, for example, ultrasonic sensors or radar devices. The outside information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and on the upper part of the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, and the like.

[0178] Returning to FIG. 24, the description will be continued. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle, and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the outside-vehicle information detection unit 7420 connected thereto. When the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves, and receives information on the received reflected waves. The outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface, based on the received information. The outside-vehicle information detection unit 7400 may perform environment recognition processing for recognizing rainfall, fog, road surface conditions, and the like, based on the received information. The outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

[0179] Furthermore, the outside vehicle information detection unit 7400 may perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface, based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or position adjustment on the received image data, and may generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410. The outside vehicle information detection unit 7400 may perform viewpoint conversion processing using image data captured by different imaging units 7410.

[0180] The in-vehicle information detection unit 7500 detects information inside the vehicle. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the bioinformation of the driver, or a microphone that collects sound in the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the bioinformation of a passenger sitting in the seat or a driver gripping the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected sound signal.

[0181] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. The input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by a device that can be operated by an occupant to input, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the occupant can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the occupant may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the occupant using the above-mentioned input unit 7800 and outputs the input signal to the integrated control unit 7600. A passenger or the like operates the input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0182] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device, etc.

[0183] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication between various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (registered trademark), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-Advanced (LTE-A), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or a carrier-specific network) via, for example, a base station or an access point. The general-purpose communication I / F 7620 may also connect to a terminal (e.g., a driver's, pedestrian's, or store's terminal, or a Machine Type Communication (MTC) terminal) present in the vicinity of the vehicle using, for example, a Peer To Peer (P2P) technology.

[0184] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in a vehicle. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE802.11p and a higher layer IEEE1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0185] The positioning unit 7640 performs positioning by receiving, for example, a GNSS signal from a Global Navigation Satellite System (GNSS) satellite (for example, a GPS signal from a Global Positioning System (GPS) satellite) and generates position information including the latitude, longitude, and altitude of the vehicle. The positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone having a positioning function.

[0186] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closure, required time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0187] The in-vehicle device I / F 7660 is a communication interface that mediates a connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). In addition, the in-vehicle device I / F 7660 may establish a wired connection such as a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI (registered trademark), or a Mobile High-definition Link (MHL)) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0188] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0189] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate a control target value of a driving force generating device, a steering mechanism, or a braking device based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc., based on information acquired about the vehicle's surroundings.

[0190] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including peripheral information of the current position of the vehicle, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. The microcomputer 7610 may also predict dangers such as vehicle collisions, the approach of pedestrians, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0191] The audio / image output unit 7670 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of FIG. 24, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may be other devices such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp, other than these devices. When the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, image, table, graph, etc. When the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced audio data or acoustic data into an analog signal and audibly outputs it.

[0192] In the example shown in FIG. 24, at least two control units connected via the communication network 7010 may be integrated into one control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by any control unit may be provided by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, a predetermined arithmetic processing may be performed by any control unit. Similarly, a sensor or device connected to any control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0193] A computer program for implementing each function of the imaging device 100 according to the present embodiment described with reference to FIG. 1 can be implemented in any control unit or the like. A computer-readable recording medium in which such a computer program is stored can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. The computer program may also be distributed, for example, via a network, without using a recording medium.

[0194] In the vehicle control system 7000 described above, the imaging device 100 according to the present embodiment described with reference to Fig. 1 can be applied to the integrated control unit 7600 of the application example shown in Fig. 24. For example, the imaging device 100 can be used in the imaging section 7410.

[0195] In addition, at least some of the components of the imaging device 100 of this embodiment described using Figure 1 may be realized in a module for the integrated control unit 7600 shown in Figure 24 (e.g., an integrated circuit module configured on a single die).

[0196] The present technology can be configured as follows.

[0197] (1) a pixel circuit that generates a first pixel signal corresponding to a first illuminance and a second pixel signal corresponding to a second illuminance higher than the first illuminance; an AD conversion unit that performs AD conversion on the first pixel signal and the second pixel signal to generate a first digital signal and a second digital signal; a processing unit that performs a synthesis process on the first digital signal and the second digital signal to generate a synthesis image, an AD conversion unit that converts the second pixel signal into a signal having a lower resolution than the first pixel signal;

[0198] (2) The imaging device according to (1), wherein the pixel circuit sets a second exposure time when generating the second pixel signal to be shorter than a first exposure time when generating the first pixel signal.

[0199] (3) the AD conversion unit generates the first digital signal based on a comparison between the first pixel signal and a first reference signal, and generates the second digital signal based on a comparison between the second pixel signal and a second reference signal; The imaging device according to (1), wherein the first reference signal and the second reference signal change over time with different slopes.

[0200] (4) The imaging device according to (3), wherein a rate at which the value of the second reference signal changes over time is greater than a rate at which the value of the first reference signal changes over time.

[0201] (5) The imaging device according to (4), further comprising a reference signal generating unit that generates the first reference signal and the second reference signal.

[0202] (6) The reference signal generation unit The first resistor, a current source section capable of varying a current supplied to the first resistor over time; an adder that generates the first reference signal and the second reference signal based on a potential of the first resistor by making different a rate of change of the current over time; The imaging device according to (5) above,

[0203] (7) the current source section connects a plurality of current sources to the first resistor via a corresponding plurality of switching elements; The imaging device according to (6), wherein the current supplied to the first resistor is changed over time by switching the switching elements between a connected state and a disconnected state.

[0204] (8) The imaging device according to (7), wherein a potential at one end of the first resistor corresponds to the first reference signal or the second reference signal, and the other end of the first resistor is connected to a predetermined low potential.

[0205] (9) The imaging device described in (6), wherein the current source unit reduces the second current supplied to the first resistor over time when generating the second reference signal by a rate greater than the rate at which the first current supplied to the first resistor over time is reduced when generating the first reference signal.

[0206] (10) The imaging device according to (6), further comprising a level adjustment unit capable of supplying further current to the first resistor.

[0207] (11) The AD conversion unit is a comparator that compares a readout signal potential of the pixel circuit with the first reference signal or the second reference signal, and inverts an output level when the readout signal potential matches a level of the first reference signal or the second reference signal; a counter whose operation is controlled by an output of the comparator and which counts a comparison time between the read signal potential and the first reference signal or the second reference signal; The imaging device according to (3) above.

[0208] (12) The imaging device according to (11), wherein the AD conversion unit generates a difference between a first downcount of the counter at a reset potential and a second downcount of the counter at the readout signal potential as a digital signal.

[0209] (13) The imaging device according to any one of (1) to (12), wherein the processing unit adjusts resolution of the first digital signal and the second digital signal to generate the composite image.

[0210] (14) The imaging device described in any one of (1) to (12), wherein the processing unit adjusts the resolution of the first digital signal and the second digital signal, and adds the first digital signal and the second digital signal at a ratio corresponding to the value of the adjusted digital signal, thereby generating the composite image having an expanded dynamic range.

[0211] (15) the pixel circuit includes a first photoelectric conversion element having a first sensitivity corresponding to the first illuminance, and a second photoelectric conversion element having a second sensitivity corresponding to the second illuminance and lower than the first sensitivity; The imaging device according to any one of (1) to (14), wherein the first photoelectric conversion element generates the first pixel signal, and the second photoelectric conversion element generates the second pixel signal.

[0212] (16) The pixel circuit includes: a photoelectric conversion element that receives light and generates and accumulates photocharges; a transfer transistor for transferring photocharges; a first charge accumulation unit connected to the photoelectric conversion element via the transfer transistor; A second charge storage section; a storage transistor having one end connected to the first charge storage unit and the other end connected to the second charge storage unit; An imaging device described in any one of (1) to (14), wherein the second pixel signal is based on the photocharges accumulated in the first charge accumulation unit, and the first pixel signal is based on the photocharges accumulated in the first charge accumulation unit and the second charge accumulation unit.

[0213] (17) The imaging device according to any one of (1) to (16), wherein the pixel circuit generates one of the first pixel signal and the second pixel signal, and then generates the other.

[0214] (18) The pixel circuits are arranged in a matrix, An optical system that forms an optical image on the pixel circuits arranged in a plurality of locations, The imaging device according to any one of (1) to (16) further comprises:

[0215] (19) The imaging device according to (17), wherein the AD conversion unit is configured in plurality to correspond to a plurality of columns of the pixel circuits.

[0216] (20) The first resistor, a current source section capable of varying a current supplied to the first resistor over time; an adder that generates a first reference signal and a second reference signal that change over time with different slopes based on a potential of the first resistor by varying a rate of change of the current over time; A reference signal generating device comprising:

[0217] The aspects of the present disclosure are not limited to the above-mentioned individual embodiments, but include various modifications that may be conceived by a person skilled in the art, and the effects of the present disclosure are not limited to the above-mentioned contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and intent of the present disclosure derived from the contents defined in the claims and their equivalents. [Explanation of symbols]

[0218] 100: imaging device, 103: second photoelectric conversion element, 110: optical system, 106: charge accumulation section, 215: reference signal generation section, 222: DAC control section, 230: DAC, 231: lamp DAC 231, 232: PGA-DAC, 233: clamp DAC, 234: addition section 234, 250: pixel circuit, 251: photoelectric conversion element, 252: transfer transistor, 254: FD section (floating diffusion layer), 258: storage transistor, 259: storage capacitance element, 260: column signal processing section.

Claims

1. a pixel circuit that generates a first pixel signal corresponding to a first illuminance and a second pixel signal corresponding to a second illuminance higher than the first illuminance; an AD conversion unit that performs AD conversion on the first pixel signal and the second pixel signal to generate a first digital signal and a second digital signal; a processing unit that performs a synthesis process on the first digital signal and the second digital signal to generate a synthesis image, An imaging device in which the AD conversion unit has a lower resolution for the second pixel signal than a resolution for the first pixel signal.

2. The imaging device according to claim 1 , wherein the pixel circuit sets a second exposure time when generating the second pixel signal to be shorter than a first exposure time when generating the first pixel signal.

3. the AD conversion unit generates the first digital signal based on a comparison between the first pixel signal and a first reference signal, and generates the second digital signal based on a comparison between the second pixel signal and a second reference signal; The imaging device according to claim 1 , wherein the first reference signal and the second reference signal change over time with different slopes.

4. The imaging device according to claim 3 , wherein a rate at which the value of the second reference signal changes over time is greater than a rate at which the value of the first reference signal changes over time.

5. The imaging device according to claim 4 , further comprising a reference signal generating unit that generates the first reference signal and the second reference signal.

6. The reference signal generation unit A first resistor; a current source section capable of varying a current supplied to the first resistor over time; an adder that generates the first reference signal and the second reference signal based on a potential of the first resistor by making different a rate of change of the current over time; The imaging device according to claim 5 , further comprising:

7. the current source section connects a plurality of current sources to the first resistor via a corresponding plurality of switching elements; The imaging device according to claim 6 , wherein the current supplied to the first resistor is changed over time by switching the switching elements between a connected state and a disconnected state.

8. The imaging device according to claim 7 , wherein a potential at one end of the first resistor corresponds to the first reference signal or the second reference signal, and the other end of the first resistor is connected to a predetermined low potential.

9. 7. The imaging device of claim 6, wherein the current source unit reduces the second current supplied to the first resistor over time when generating the second reference signal by a rate greater than the rate at which the first current supplied to the first resistor over time when generating the first reference signal.

10. The imaging device according to claim 6 , further comprising a level adjustment section capable of supplying further current to the first resistor.

11. The AD conversion unit includes: a comparator that compares a readout signal potential of the pixel circuit with the first reference signal or the second reference signal, and inverts an output level when the readout signal potential matches a level of the first reference signal or the second reference signal; a counter whose operation is controlled by an output of the comparator and which counts a comparison time between the read signal potential and the first reference signal or the second reference signal; The imaging device according to claim 3 .

12. The imaging device according to claim 11 , wherein the AD conversion section generates, as a digital signal, a difference between a first downcount of the counter at a reset potential and a second downcount of the counter at the read signal potential.

13. The imaging device according to claim 1 , wherein the processing unit generates the composite image by adjusting resolutions of the first digital signal and the second digital signal.

14. 2. The imaging device according to claim 1, wherein the processing unit adjusts resolution of the first digital signal and the second digital signal, and adds the first digital signal and the second digital signal at a ratio according to a value of the adjusted digital signal to generate the composite image having an expanded dynamic range.

15. the pixel circuit includes a first photoelectric conversion element having a first sensitivity corresponding to the first illuminance, and a second photoelectric conversion element having a second sensitivity corresponding to the second illuminance and lower than the first sensitivity; The imaging device according to claim 1 , wherein the first photoelectric conversion element generates the first pixel signal, and the second photoelectric conversion element generates the second pixel signal.

16. The pixel circuit includes: a photoelectric conversion element that receives light and generates and accumulates photocharges; a transfer transistor for transferring photocharges; a first charge accumulation unit connected to the photoelectric conversion element via the transfer transistor; A second charge storage unit; a storage transistor having one end connected to the first charge storage unit and the other end connected to the second charge storage unit; 2 . The imaging device according to claim 1 , wherein the second pixel signal is based on the photocharges accumulated in the first charge accumulation unit, and the first pixel signal is based on the photocharges accumulated in the first charge accumulation unit and the second charge accumulation unit.

17. The imaging device according to claim 1 , wherein the pixel circuit generates one of the first pixel signal and the second pixel signal, and then generates the other of the first pixel signal and the second pixel signal.

18. The pixel circuits are arranged in a matrix, An optical system that forms an optical image on the pixel circuits arranged in a plurality of locations, The imaging device according to claim 1 , further comprising:

19. The imaging device according to claim 17 , wherein the AD conversion unit is configured in a plurality of units corresponding to the plurality of columns of the pixel circuits.

20. A first resistor; a current source section capable of varying a current supplied to the first resistor over time; an adder that generates a first reference signal and a second reference signal that change over time with different slopes based on a potential of the first resistor by varying a rate of change of the current over time; A reference signal generating device comprising:

Images