Image sensor, imaging device, and image sensor control method

Abstract

The present invention improves the performance of an image sensor in which a column amplifier is disposed.　The column amplifier outputs an output signal by executing any one among sample-and-hold of a pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted average processing of the pixel signal. An analog-to-digital converter converts the output signal into a digital signal. A determination control circuit controls the column amplifier to execute the sample-and-hold or amplification of the pixel signal when the illuminance is lower than a prescribed value, and controls the column amplifier to execute the attenuation or weighted average processing of the pixel signal when the illuminance is higher than the prescribed value.

Description

Image sensor, imaging device, and control method for image sensor

[0001] This technology relates to image sensors. More specifically, it describes an image sensor with an ADC (Analog to Digital Converter) arranged in each column, an imaging device, and a control method for the image sensor.

[0002] Conventionally, in column ADC type image sensors, a column amplifier is sometimes placed before the ADC for each column. For example, an image sensor has been proposed that determines whether the illuminance is high or low for each column and applies an offset to the input of the column amplifier based on the determination result (see, for example, Patent Document 1). Furthermore, this image sensor uses a CDS (Correlated Double Sampling) method in which the signal level is sampled after the reset level.

[0003] Japanese Patent Publication No. 2016-5054

[0004] In the conventional technology described above, an offset is applied to the input of the column amplifier based on the illuminance determination result to prevent a decrease in dynamic range caused by saturation of the output signal of the column amplifier. However, since the Double Data Sampling (DDS) method, which samples the reset level after the signal level, is sometimes used, the above configuration cannot perform proper control when the DDS method and CDS method are mixed. As a result, the image sensor described above may experience a decrease in performance such as image quality and readout speed.

[0005] This technology was developed in light of these circumstances and aims to improve the performance of image sensors equipped with column amplifiers.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor and control method comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal; an analog-to-digital converter that converts the output signal into a digital signal; and a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. This results in an improvement in the performance of the image sensor.

[0007] Furthermore, in this first aspect, the system may further include a data recovery unit that, when amplification of the pixel signal is performed, multiplies the digital signal by the reciprocal of the amplification factor, and when attenuation or weighted averaging of the pixel signal is performed, multiplies the digital signal by the reciprocal of the attenuation factor. This results in the data being restored to its state before amplification or attenuation.

[0008] Furthermore, in this first aspect, the pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel, and the pixel may output the reset level and the signal level in a predetermined order. This results in the reset level and the signal level being input to the column amplifier in sequence.

[0009] Furthermore, in this first aspect, the pixel may output the signal level after the reset level within a predetermined CDS period, and within the CDS period, the determination control circuit may perform a sample-and-hold operation of the reset level, amplify the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and attenuate the difference if the illuminance is higher than the predetermined value. This results in improved image quality and readout speed.

[0010] Furthermore, in this first aspect, the pixel may output the reset level after the signal level within a predetermined DDS period, and within the DDS period, the determination control circuit may perform a sample-and-hold operation on the signal level, perform a sample-and-hold operation on the reset level if the illuminance is lower than the predetermined value, and perform a weighted average operation on the reset level and the signal level if the illuminance is higher than the predetermined value. This results in an improved readout speed.

[0011] Furthermore, in this first aspect, the column amplifier may include an operational amplifier that outputs the output signal, first and second capacitive elements, each having one end connected to the input terminal of the operational amplifier, a predetermined number of first switches that open and close the path between the other end of the first capacitive element and the input terminal of the column amplifier, a predetermined number of second switches that open and close the path between the other end of the second capacitive element and the input terminal of the column amplifier, a predetermined number of third switches that open and close the path between the other end of the first capacitive element and the output terminal of the column amplifier, and a predetermined number of fourth switches that open and close the path between the other end of the first capacitive element and the output terminal. This results in the pixel signal being increased or decreased by controlling the switches.

[0012] Furthermore, in this first aspect, the determination control circuit may include a determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illuminance determination result, and a control circuit that controls the first switch, second switch, third switch, and fourth switch based on the determination result. This results in the illuminance being determined within the determination control circuit.

[0013] Also, in this first aspect, the pixel outputs the signal level next after the reset level within a predetermined CDS period. During the CDS period, when the reset level is output, the control circuit controls the first switch and the second switch to be in an on state for a predetermined period. When the illuminance is lower than the predetermined value, the control circuit controls the first switch and the third switch to be in an on state for a predetermined period. When the illuminance is higher than the predetermined value, the control circuit may control the second switch and the fourth switch to be in an on state for a predetermined period. This brings about the effect of increasing or decreasing the difference between the reset level and the signal level.

[0014] Also, in this first aspect, the pixel outputs the reset level next after the signal level within a predetermined DDS period. During the DDS period, when the reset level is output, the control circuit controls the first switch and the second switch to be in an on state for a predetermined period. When the illuminance is lower than the predetermined value, after the control circuit controls the first switch and the second switch to be in an on state for a predetermined period, the control circuit controls the third switch and the fourth switch to be in an on state for a predetermined period. When the illuminance is higher than the predetermined value, after the control circuit controls the second switch to be in an on state for a predetermined period, the control circuit controls the third switch and the fourth switch to be in an on state for a predetermined period. This brings about the effect of performing sample and hold of the reset level or weighted average processing of the reset level and the signal level.

[0015] Also, in this first aspect, the pixel signal may include first and second pixel signals having different exposure period lengths, and the determination threshold value may include a first determination threshold value corresponding to the first pixel signal and a second determination threshold value corresponding to the second pixel signal. This brings about the effect of being able to set three or more exposure periods with different lengths regardless of the pixel circuit.

[0016] Furthermore, the second aspect of this technology is an imaging device comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of pixel signals from pixels, amplification of the pixel signals, attenuation of the pixel signals, or weighted averaging of the pixel signals; an analog-to-digital converter that converts the output signal into a digital signal; a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signals when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signals when the illuminance is higher than the predetermined value; and an image processing unit that processes an image formed by arranging the digital signals. This results in an improvement in the performance of the imaging device.

[0017] This is a block diagram showing an example configuration of an imaging device in the first embodiment of this technology. This is a diagram showing an example of a stacked structure of an image sensor in the first embodiment of this technology. This is a block diagram showing an example configuration of an image sensor in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a pixel in the first embodiment of this technology. This is a block diagram showing an example configuration of a column signal processing unit in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a column amplifier in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a comparison unit in the first embodiment of this technology. This is a circuit diagram showing an example configuration of a determination control circuit in the first embodiment of this technology. This is a timing chart showing an example of a pixel driving method in the first embodiment of this technology. This is a timing chart showing an example of fluctuations in pixel signals and determination thresholds in the first embodiment of this technology. This is a timing chart showing an example of the operation of the timing control unit and determination control circuit during the CDS period in the first embodiment of this technology. This is a timing chart showing an example of the operation of the timing control unit and determination control circuit during the DDS period in the first embodiment of this technology. This is a diagram showing an example of the state of the column amplifier and comparison unit during the P phase period and the determination period of the CDS period in the first embodiment of this technology. This figure shows an example of the state of the column amplifier and comparator during the AD (Analog to Digital) conversion period within the CDS period in the first embodiment of this technology. This figure shows an example of the state of the column amplifier and comparator during the D-phase period and the determination period within the DDS period in the first embodiment of this technology. This figure shows an example of the state of the column amplifier and comparator during low illumination during the P-phase sampling period and the AD conversion period within the DDS period in the first embodiment of this technology. This figure shows an example of the state of the column amplifier and comparator during high illumination during the P-phase sampling period and the AD conversion period within the DDS period in the first embodiment of this technology. This figure illustrates the effect of pixel signal attenuation in the first embodiment of this technology. This figure shows an example of the input / output characteristics of the column amplifier and ADC in the first embodiment of this technology. This flowchart shows an example of the operation of the image sensor in the first embodiment of this technology.A flowchart showing an example of CDS processing in the first embodiment of the present technology. A flowchart showing an example of DDS processing in the first embodiment of the present technology. A timing chart showing the pixel driving method and the variation of the determination threshold in the second embodiment of the present technology. A block diagram showing an example of the schematic configuration of a vehicle control system. An explanatory diagram showing an example of the installation positions of an external vehicle information detection unit and an imaging unit.

[0018] Hereinafter, embodiments for implementing the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order. 1. First embodiment (example of controlling a column amplifier according to illuminance) 2. Second embodiment (example of changing a determination threshold according to exposure time and controlling a column amplifier according to illuminance) 3. Application example to a moving body

[0019] <1. First embodiment> [Example of configuration of imaging device] FIG. 1 is a block diagram showing an example of the configuration of an imaging device 100 in the first embodiment of the present technology. This imaging device 100 is a device for imaging image data, and includes an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. Further, the imaging device 100 includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. As the imaging device 100, a camera mounted on a smartphone, an in-vehicle camera, etc. are assumed.

[0020] The optical unit 110 collects light from a subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion. This image sensor 200 supplies the generated image data to the DSP circuit 120 via a signal line 209.

[0021] The DSP circuit 120 executes predetermined signal processing on the image data. This DSP circuit 120 outputs the processed image data to the frame memory 160 etc. via the bus 150.

[0022] The display unit 130 displays image data. The display unit 130 could be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The operation unit 140 generates operation signals according to user input.

[0023] Bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with each other.

[0024] The frame memory 160 stores image data. The storage unit 170 stores various data, including image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and other components.

[0025] Figure 2 shows an example of a stacked structure of an image sensor 200 in a first embodiment of this technology. This image sensor 200 comprises a circuit chip 202 and a light-receiving chip 201 stacked on the circuit chip 202. These chips are electrically connected via connection points such as vias. In addition to vias, connections can also be made by Cu-Cu junctions or bumps.

[0026] [Example of Image Sensor Configuration] Figure 3 is a block diagram showing an example configuration of an image sensor 200 in the first embodiment of this technology. This image sensor 200 includes a vertical scanning circuit 210, a timing control unit 220, a DAC (Digital to Analog Converter) 230, a pixel array unit 240, a column signal processing unit 260, a horizontal scanning circuit 270, and an image processing unit 280. Multiple pixels 250 are arranged in a two-dimensional grid in the pixel array unit 240.

[0027] The pixel array section 240 is located on, for example, the light-receiving chip 201, and the remaining circuits are located on the circuit chip 202. Note that the circuits located on each chip are not limited to those illustrated in the figure.

[0028] The vertical scanning circuit 210 sequentially selects and drives rows within the pixel array section 240.

[0029] The timing control unit 220 controls the operating timing of the vertical scanning circuit 210, DAC 230, column signal processing unit 260, and horizontal scanning circuit 270 in synchronization with the vertical synchronization signal VSYNC.

[0030] The DAC230 generates a sawtooth-shaped ramp signal and supplies it to the column signal processing unit 260 as a reference signal.

[0031] Each pixel 250 generates an analog pixel signal by photoelectric conversion in accordance with the control of the vertical scanning circuit 210. Each pixel 250 in each column outputs a pixel signal to the column signal processing unit 260 via a vertical signal line (not shown).

[0032] The column signal processing unit 260 has a column amplifier and an ADC (not shown) for each column of pixels 250. Each ADC converts the pixel signal of the corresponding column into a digital signal and outputs it to the image processing unit 280 according to the control of the horizontal scanning circuit 270.

[0033] The horizontal scanning circuit 270 controls the column signal processing unit 260 to output digital signals sequentially.

[0034] The image processing unit 280 performs various image processing operations on image data, which is an array of digital signals. The image processing unit 280 supplies the processed image data to the DSP circuit 120 via the signal line 209.

[0035] [Example of Pixel Configuration] Figure 4 is a circuit diagram showing an example of the configuration of a pixel 250 in the first embodiment of this technology. This pixel 250 includes a photoelectric conversion unit 251, a transfer transistor 252, a reset transistor 253, a first FD (Floating Diffusion) unit 257, an amplification transistor 258, and a selection transistor 259. The pixel 250 further includes an interwiring capacitance 254, a second FD unit 255, and a connection transistor 256.

[0036] The photoelectric conversion unit 251 has its anode terminal grounded and its cathode terminal connected to the first FD unit 257 via a transfer transistor 252. The photoelectric conversion unit 251 receives light irradiated onto the light-receiving surface of the image sensor 200 and converts it into an electric charge corresponding to the amount of light.

[0037] The transfer transistor 252 is positioned to connect the photoelectric conversion unit 251 and the first FD unit 257. The transfer transistor 252 is driven according to the transfer signal TG from the vertical scanning circuit 210 and transfers the charge photoelectrically converted in the photoelectric conversion unit 251 to the first FD unit 257.

[0038] The first FD unit 257 stores the charge transferred from the photoelectric conversion unit 251 via the transfer transistor 252 in order to convert it into a pixel signal.

[0039] The connecting transistor 256 is positioned to connect the first FD section 257 and the second FD section 255. The connecting transistor 256 is driven according to the connection signal FDG from the vertical scanning circuit 210, and switches the connection between the first FD section 257 and the second FD section 255 on and off.

[0040] The second FD section 255 is connected to the first FD section 257 when the connecting transistor 256 is turned on, and together with the first FD section 257, it accumulates charge.

[0041] The inter-wiring capacitance 254 is composed of a MIM (Metal-Insulator-Metal) capacitor provided between the wires and is positioned to connect the second FD section 255 and the signal wiring MIMVDD. The inter-wiring capacitance 254 accumulates charge in the same manner as the second FD section 255.

[0042] The reset transistor 253 is positioned to connect the second FD section 255 and the power supply wiring VDD. The reset transistor 253 is driven according to the reset signal RST from the vertical scanning circuit 210. When the connecting transistor 256 and the reset transistor 253 are turned on, the charge accumulated in the first FD section 257, the second FD section 255, and the interwiring capacitance 254 is discharged to the power supply wiring VDD, and the charge is reset.

[0043] The amplification transistor 258 is positioned so that the first FD section 257 is connected to its gate electrode, and the power supply wiring VDD and the selection transistor 259 are connected. The amplification transistor 258 converts the charge stored in the first FD section 257 alone, or in the first FD section 257, the second FD section 255, and the interwiring capacitance 254, into a pixel signal at a level corresponding to the charge in each capacitance.

[0044] The selection transistor 259 is positioned to connect the amplification transistor 258 and the vertical signal line 249. The selection transistor 259 is driven according to the selection signal SEL from the vertical scanning circuit 210, and while the selection transistor 259 is ON, the pixel signal SIG from the amplification transistor 258 is output to the vertical signal line 249.

[0045] The level of the pixel signal SIG when the reset transistor 253 initializes the pixel 250 is referred to as the "reset level" or "P-phase level." The level of the pixel signal SIG when the transfer transistor 252 transfers charge within the pixel is referred to as the "signal level" or "D-phase level."

[0046] For example, an nMOS (n-channel Metal Oxide Semiconductor) transistor is used as the transistor within the pixel 250. In the circuit configuration illustrated in the figure, the FD gain, which is the conversion efficiency when converting charge to voltage, is controlled in two stages. The FD gain when the connected transistor 256 is ON is lower than when the connected transistor 256 is OFF; the former is called "low gain" and the latter is called "high gain".

[0047] The image sensor 200 exposes each of the pixels 250 for a predetermined exposure period and reads out the pixel signals in both low-gain and high-gain conditions, thereby obtaining pixel signals similar to those obtained when the pixels are exposed sequentially for two different exposure periods. These analog pixel signals are converted into digital signals by the column signal processing unit 260. The subsequent image processing unit 280 expands the dynamic range by performing HDR synthesis on the digital signals from the low-gain and high-gain conditions.

[0048] [Example of Column Signal Processing Unit Configuration] Figure 5 is a block diagram showing an example configuration of the column signal processing unit 260 in the first embodiment of this technology. This column signal processing unit 260 includes a determination control circuit 310, a column amplifier 400, an ADC 261, and a data recovery unit 340 for each column.

[0049] The column amplifier 400 performs one of the following processes on the pixel signal SIG from the corresponding column: sample-and-hold, amplification, attenuation, or weighted averaging. The column amplifier 400 outputs the analog pixel signal, after performing the sample-and-hold process, as an output signal to the corresponding ADC 261.

[0050] The determination control circuit 310 determines whether the illuminance is high or low, based on the pixel signal SIG of the corresponding column, and controls the column amplifier 400 based on the determination result CMP. If the illuminance is low, below the predetermined value, the determination control circuit 310 controls the column amplifier 400 to perform sample-and-hold or amplification of the pixel signal SIG. On the other hand, if the illuminance is high, the determination control circuit 310 controls the column amplifier 400 to perform attenuation or weighted averaging of the pixel signal SIG. The determination control circuit 310 also supplies the determination result CMP to the data recovery unit 340.

[0051] The ADC261 performs AD conversion of the output signal of the column amplifier 400 into a digital signal CNT. This ADC261 is a single-slope type ADC equipped with a comparison unit 320 and a counter 330.

[0052] The comparison unit 320 compares the reference signal RMP from the DAC 230 with the output signal of the column amplifier 400 and outputs the comparison result VCO to the counter 330.

[0053] The counter 330 counts the count value over the period until the comparison result VCO reverses. This counter 330 supplies a digital signal CNT indicating the count value to the data recovery unit 340. The value of this digital signal CNT will be different from the value when the pixel signal is amplified or attenuated by the column amplifier 400 compared to when these actions are not performed.

[0054] The data restoration unit 340 restores the value of the digital signal CNT to the value obtained when amplification or other processing is not performed, based on the judgment result CMP. When the column amplifier 400 performs amplification, the data restoration unit 340 restores the digital signal CNT by multiplying it by the reciprocal of the amplification factor. On the other hand, when the column amplifier 400 performs attenuation or weighted averaging, the data restoration unit 340 restores the digital signal CNT by multiplying it by the reciprocal of the attenuation factor. The data restoration unit 340 outputs the restored digital signal as Dout to the image processing unit 280.

[0055] [Example of Column Amplifier Configuration] Figure 6 is a circuit diagram showing an example configuration of a column amplifier 400 in the first embodiment of this technology. This column amplifier 400 comprises an attenuation / amplification unit 410, an auto-zero switch 430, and an operational amplifier 440. The attenuation / amplification unit 410 comprises N (N is an integer) capacitive elements 421, N switches 411, and N switches 413. Furthermore, the attenuation / amplification unit 410 comprises M (M is an integer) capacitive elements 422, M switches 412, and M switches 414. N may be a different value from or the same value as M.

[0056] Furthermore, the input terminal 401 of the column amplifier 400 is connected to the vertical signal line 249 of the corresponding column. The output terminal 402 of the column amplifier 400 is connected to the comparison unit 320 of the corresponding column.

[0057] Each of the N capacitive elements 421 and each of the M capacitive elements 422 are connected in common to the inverting input terminal (-) of the operational amplifier 440. Note that the capacitive elements 421 and 422 are examples of the first and second capacitive elements described in the claims.

[0058] The nth switch 411 (where n is an integer from 1 to N) opens and closes the path between the other end of the nth capacitive element 421 and the input terminal 401, according to the control signal Φsn from the determination control circuit 310. Note that switch 411 is an example of the first switch described in the claims.

[0059] The m-th switch 412 (where m is an integer from 1 to M) opens and closes the path between the other end of the m-th capacitive element 422 and the input terminal 401, according to the control signal Φsm from the determination control circuit 310. Note that switch 412 is an example of the second switch described in the claims.

[0060] The nth switch 413 opens and closes the path between the other end of the nth capacitive element 421 and the output terminal of the operational amplifier 440, according to the control signal Φhn from the determination control circuit 310. Note that switch 413 is an example of the third switch described in the claims.

[0061] The m-th switch 414 opens and closes the path between the other end of the m-th capacitive element 422 and the output terminal of the operational amplifier 440, according to the control signal Φhm from the determination control circuit 310. Note that switch 414 is an example of the fourth switch described in the claims.

[0062] The auto-zero switch 430 opens and closes the path between the inverting input terminal (-) and the output terminal of the operational amplifier 440 according to the auto-zero signal AZsh from the timing control unit 220.

[0063] The non-inverting input terminal (+) of the operational amplifier 440 is connected to ground potential, and its output terminal is connected to the output terminal 402 of the column amplifier 400. This operational amplifier 440 outputs the output signal SHOUT to the comparison unit 320.

[0064] The determination control circuit 310, based on the control signal Φsn, selects n of the N switches 411.０ (n ０ This switches (1 to N integers) on and off. The remaining switches 411 are controlled to be in the off state. When the nth switch 411 is switched on or off, the control signal Φhn also switches the nth switch 413 on or off.

[0065] Furthermore, the determination control circuit 310, based on the control signal Φsm, selects m of the M switches 412. ０ (m ０ This switches (1 to M integers) on and off. The remaining switches 412 are controlled to be in the off state. When the mth switch 412 is switched on or off, the control signal Φhm also switches the mth switch 414 on or off.

[0066] The control method for the switch shown in the figure will be described later. Also, m ０ han ０ These m ０ and n ０ The value is pre-set and stored in a register or similar location.

[0067] [Example of Comparison Unit Configuration] Figure 7 is a circuit diagram showing an example of the configuration of the comparison unit 320 in the first embodiment of this technology. This comparison unit 320 comprises capacitive elements 321 and 322, a comparator 323, and auto-zero switches 324 and 325.

[0068] The output signal SHOUT from the column amplifier 400 is input to the inverting input terminal (-) of comparator 323 via capacitive element 321. The reference signal RMP from DAC 230 is input to the non-inverting input terminal (+) of comparator 323 via capacitive element 322. Comparator 323 compares these signals and outputs the comparison result VCO to counter 330.

[0069] The auto-zero switch 324 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0070] The auto-zero switch 325 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0071] [Example of configuration of the determination control circuit] Figure 8 is a circuit diagram showing an example of the configuration of the determination control circuit 310 in the first embodiment of this technology. This determination control circuit 310 comprises a determination circuit 311 and a control circuit 317.

[0072] The determination circuit 311 determines whether the illuminance is high, which is higher than a predetermined value, based on the pixel signal SIG of the corresponding column. This determination circuit 311 comprises capacitive elements 312 and 313, a comparator 314, and auto-zero switches 315 and 316.

[0073] The inverting input terminal (-) of comparator 314 is connected to the vertical signal line 249 via capacitive element 312, and the pixel signal SIG is input to this terminal. The non-inverting input terminal (+) of comparator 314 is input to the judgment threshold Th from the timing control unit 220 via capacitive element 313. Comparator 324 compares the level of the pixel signal SIG with the judgment threshold Th and outputs the comparison result as the illuminance judgment result CMP to the control circuit 317 and the data restoration unit 340.

[0074] The auto-zero switch 315 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0075] The auto-zero switch 316 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0076] The control circuit 317 controls the column amplifier 400 using control signals Φsm, Φsn, Φhm, and Φhn, based on the determination result CMP and the control signals Ctrl and MODE from the timing control unit 220. The control signal Ctrl is a signal that indicates the timing for controlling the switch. The mode signal MODE is a signal that indicates either the CDS method or the DDS method.

[0077] [Example of Image Sensor Operation] Figure 9 is a timing chart showing an example of a method for driving pixels 250 in the first embodiment of this technology. In the figure, a is a timing chart showing an example of a method for driving all rows. In the figure, b is a timing chart showing an example of a method for driving the first row.

[0078] In the figure, the vertical axis of 'a' represents the row address, and the horizontal axis represents time. For example, from timing T1 onward, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. The solid diagonal lines in the figure indicate the exposure start timing for each row.

[0079] Then, from timing T2 onward, the vertical scanning circuit 210 drives the rows sequentially to complete the exposure and outputs the pixel signal. The dashed lines in the figure indicate the timing of the end of exposure and readout for each row. The period from timing T1 to T2 in the figure a indicates the exposure period for the first row.

[0080] As illustrated in figure b, during the pulse period from timing T0 to T1, the vertical scanning circuit 210 raises the connection signal FDG, the transfer signal TG, and the reset signal RST to high levels. This initiates exposure.

[0081] Then, during the CDS period from exposure end timing T2 to timing T5, the vertical scanning circuit 210 drives the pixels 250 using the CDS method. This CDS period includes a P-phase period from timing T2 to timing T3 and a D-phase period from timing T4 to timing T5.

[0082] During the P-phase period from timing T2 to T3, the vertical scanning circuit 210 raises the selection signal SEL to a high level. During this period, the column amplifier 400 samples the reset level (in other words, the P-phase level).

[0083] Then, during the pulse period from timing T3 to T4, the vertical scanning circuit 210 raises the transfer signal TG to a high level. This causes charge to be transferred within the pixel 250, generating a signal level (in other words, a D-phase level). Then, at timing T4, the vertical scanning circuit 210 raises the selection signal SEL to a high level.

[0084] Furthermore, the vertical scanning circuit 210 keeps the connection signal FDG at a low level during the CDS period from timing T2 to timing T5. As a result, the P-phase level and D-phase level are generated at high gain.

[0085] Then, the vertical scanning circuit 210 raises the connection signal FDG to a high level at timing T5, and raises the transfer signal TG to a high level within the pulse period from timing T5 to timing T6. This generates a D-phase level with low gain.

[0086] At timing T6, the vertical scanning circuit 210 changes the mode signal MODE from a low level to a high level. The mode signal MODE is controlled to a low level when using the CDS method and to a high level when using the DDS method.

[0087] During the DDS period from timing T6 to timing T9, the vertical scanning circuit 210 drives the pixels 250 using the DDS method. Since the connection signal FDG is at a high level during the DDS period, the P-phase level and D-phase level during this period are generated at low gain. The DDS period also includes the D-phase period from timing T6 to timing T7 and the P-phase period from timing T8 to timing T9. The D-phase level is sampled during this D-phase period. At timing T7, the vertical scanning circuit 210 sets the selection signal SEL to a low level.

[0088] Then, during the pulse period from timing T7 to T8, the vertical scanning circuit 210 sets the reset signal RST to a high level. This initializes the pixel 250 and generates a reset level (P-phase level). During the P-phase period from timing T8 to T9, the vertical scanning circuit 210 sets the selection signal SEL to a high level.

[0089] Furthermore, the vertical scanning circuit 210 switches the potential of the signal wiring MIMVDD from a low level (for example, an intermediate voltage lower than the power wiring VDD) to a high level (for example, the same voltage as the power wiring VDD) before exposure begins. After exposure begins, the potential of the signal wiring MIMVDD is switched from a high level to a low level. Subsequently, at pixel 250, the potential of the signal wiring MIMVDD is switched from a low level to a high level before the CDS period. By controlling the potential of the signal wiring MIMVDD in this way, the increase in noise during the exposure period can be suppressed. This improves image quality.

[0090] Although the image sensor 200 uses both the CDS method and the DDS method when performing HDR synthesis, it is not limited to this configuration. The image sensor 200 can also perform imaging without HDR synthesis using only one of the CDS method or the DDS method.

[0091] Figure 10 is a timing chart showing an example of the variation of the pixel signal SIG and the judgment threshold Th in the first embodiment of this technology. In the figure, a shows the variation of the pixel signal SIG and the judgment threshold Th in the case of low illumination. Also, in the figure, b shows the variation of the pixel signal SIG and the judgment threshold Th in the case of high illumination.

[0092] As illustrated in figures a and b, the timing control unit 220 sets the determination threshold Th to Th0 over the P-phase period within the CDS period. This Th0 is set to a value slightly lower than the P-phase level, for example.

[0093] Then, at the timing T3 in which the charge is transferred, the timing control unit 220 sets the determination threshold Th to Th1, which is lower than Th0.

[0094] Then, at timing T5, the start of the D-phase period within the DDS period, the timing control unit 220 sets the determination threshold Th to Th2. This Th2 is set to a value lower than the D-phase level expected at high illuminance.

[0095] Then, at the timing T7 in which the pixels 250 within the DDS period are initialized, the timing control unit 220 sets the determination threshold Th to Th3, which is higher than Th2.

[0096] Furthermore, as illustrated in figure a, when the illumination is low, the pixel signal SIG is higher than the judgment threshold in all periods. Therefore, the judgment result of the judgment control circuit 310 is not inverted.

[0097] On the other hand, as illustrated in figure b, when the illuminance is high, the level of the pixel signal SIG (here, the D-phase level) becomes lower than the judgment threshold Th1 during the transfer period from timing T3 to T4, and the judgment result of the judgment control circuit 310 is reversed. Also, during the initialization period from timing T7 to T8, the level of the pixel signal SIG (here, the P-phase level) becomes higher than the judgment threshold Th3, and the judgment result of the judgment control circuit 310 is reversed.

[0098] As illustrated in figures a and b, the determination control circuit 310 can determine whether the illuminance is higher than a predetermined value based on the determination threshold Th. Th_cds, which is the difference between determination thresholds Th0 and Th1, corresponds to a relative threshold that is compared with the difference between the P-phase level and the D-phase level during the CDS period. Th_dds, which is the difference between determination thresholds Th2 and Th3, corresponds to a relative threshold that is compared with the difference between the P-phase level and the D-phase level during the DDS period.

[0099] Figure 11 is a timing chart showing an example of the operation of the timing control unit 220 and the determination control circuit 310 during the CDS period in the first embodiment of this technology.

[0100] During the CDS period, within the P-phase period from timing T2 to T3, the timing control unit 220 sets the auto-zero signals AZcmp and AZsh to high level, and causes the comparators of the column amplifier 400 and the determination control circuit 310 to perform auto-zero operation.

[0101] Also, within the P-phase period, the determination control circuit 310 sets m ０ control signals Φsm and n ０ control signals Φsn to high level. As a result, the P-phase level Vp is sampled and held when the P-phase period elapses.

[0102] Then, at timing T3, the timing control unit 220 changes the determination threshold Th from Th0 to Th1.

[0103] Also, within the determination period from timing T3 to timing T4', charges are transferred, and the level of the pixel signal SIG changes from the P-phase level Vp to the D-phase level Vd. During this determination period, when the illuminance is lower than a predetermined value, the determination result CMP reverses from high level to low level. On the other hand, when the illuminance is higher than the predetermined value, the determination result CMP remains at high level. The dashed-dotted line in the figure shows the determination result when the illuminance is lower than the predetermined value.

[0104] During the AD conversion period after timing T4', when the illuminance is higher than a predetermined value, the determination control circuit 310 sets n ０ control signals Φsn and m ０ control signals Φhm to high level. As a result, the difference between Vp and Vd is attenuated.

[0105] On the other hand, when the illuminance is lower than the predetermined value, the determination control circuit 310 sets m ０ control signals Φsm and n ０ control signals Φhn to high level. As a result, the difference between Vp and Vd is amplified. The dashed-dotted line in the figure shows the control signals when the illuminance is lower than the predetermined value. Also, the difference after attenuation or amplification is AD-converted.

[0106] FIG. 12 is a timing chart showing an example of the operations of the timing control unit 220 and the determination control circuit 310 during the DDS period in the first embodiment of the present technology.

[0107] During the DDS period, within the D phase period from timing T6 to T7, the timing control unit 220 raises the auto-zero signals AZcmp and AZsh to high levels, causing the comparators of the column amplifier 400 and the determination control circuit 310 to perform auto-zero operations.

[0108] Furthermore, during the D phase period, the determination control circuit 310 will ０ The control signals Φsm and n ０ The control signals Φsn are set to a high level. As a result, the D-phase level Vd is sampled and held when the D-phase period has elapsed.

[0109] Then, at timing T7, the timing control unit 220 changes the judgment threshold Th from Th2 to Th3.

[0110] Furthermore, during the judgment period from timing T7 to timing T8', pixel 250 is initialized, and the level of the pixel signal SIG changes from the D-phase level Vd to the P-phase level Vp. During this judgment period, if the illuminance is higher than a predetermined value, the judgment result CMP inverts from a high level to a low level. On the other hand, if the illuminance is lower than the predetermined value, the judgment result CMP remains at a high level. The dashed line in the figure shows the judgment result when the illuminance is lower than the predetermined value. Also, during the judgment period, the D-phase level is converted using AD conversion.

[0111] In the case of low light, the determination control circuit 310 operates during the P-phase sampling period from timing T8' to timing T81. ０ The control signals Φsm and n ０ The control signal Φsn is set to a high level. As a result, the P-phase level is sampled and held after the P-phase sampling period has elapsed. Then, within the AD conversion period after timing T81, the P-phase level is converted using AD conversion.

[0112] On the other hand, in the case of high illuminance, the determination control circuit 310 during the P-phase sampling period is m ０ The control signal Φsm is set to a high level. Then, within the AD conversion period after timing T81, the determination control circuit 310 is set to m ０ The individual control signals Φhm and n ０The individual control signals Φhn are set to a high level. This causes a weighted average of Vp and Vd to be performed, and that value is then converted using AD conversion.

[0113] Next, referring to Figures 13 to 17, the state of the column amplifier 400 during the CDS period and the DDS period will be explained.

[0114] Figure 13 shows an example of the state of the column amplifier 400 and the comparison unit 320 during the P-phase period and the determination period of the CDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the P-phase period, and b shows an example of the state during the determination period.

[0115] out of N items ０ 411 switches and m of the M switches ０ 412 switches and n out of N switches ０ 413 switches and m of the M switches ０ The switches 414 are to be turned on and off. However, for convenience of description, in figures a and b, only one of the switches 411, 412, 413, and 414 to be controlled is shown, and the rest are omitted. Similarly, only one of the N capacitive elements 421 and M capacitive elements 422 is shown, and the rest are omitted. The same applies to Figures 14 to 17.

[0116] As illustrated in figure a, during the P-phase period, the timing control unit 220 controls the auto-zero switch 430 to the ON state, and the determination control circuit 310 controls switches 411 and 412 to the ON state. As a result, the P-phase level Vp is sampled by the capacitive elements 421 and 422.

[0117] Then, as illustrated in figure b, the timing control unit 220 controls the auto-zero switches 324 and 325 to the ON state during the determination period. During this period, it is determined whether or not the illuminance is higher than a predetermined value.

[0118] Figure 14 shows an example of the state of the column amplifier 400 and the comparison unit 320 during the AD conversion period of the CDS period in the first embodiment of this technology. Figure a shows an example of the state when the illuminance is lower than a predetermined value, and figure b shows an example of the state when the illuminance is higher than a predetermined value.

[0119] As illustrated in figure a, when the illuminance is low, the determination control circuit 310 turns on switches 412 and 413. The level of the output signal SHOUT from the column amplifier 400 at this time is expressed by the following formula: SHOUT = (Vp - Vd) × m ０ / n ０ ...Formula 1

[0120] As mentioned above, m ０ n ０ Since a larger value is set, the output signal SHOUT from Equation 1 becomes the amplified difference between Vp and Vd. This amplification reduces the noise in the signal from the comparison unit 320 onward.

[0121] On the other hand, as illustrated in figure b, when the illuminance is high, the determination control circuit 310 turns on switches 411 and 414. The level of the output signal SHOUT from the column amplifier 400 at this time is expressed by the following formula: SHOUT = (Vp - Vd) × n ０ / m ０ ...Formula 2

[0122] I understand ０ n ０ Because a larger value is set, the output signal SHOUT from Equation 2 becomes the value obtained by attenuating the difference between Vp and Vd. This attenuation shortens the AD conversion time (in other words, improves the read speed) and reduces power consumption.

[0123] Furthermore, as illustrated in figures a and b, the difference between Vp and Vd is sampled in the capacitive elements 421 and 422, thereby realizing analog CDS processing. This analog CDS processing eliminates the need for P-phase level AD conversion.

[0124] Figure 15 shows an example of the state of the column amplifier 400 and comparison unit 320 during the D-phase period and the determination period of the DDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the D-phase period, and b shows an example of the state during the determination period.

[0125] As illustrated in figure a, during the D-phase period, the timing control unit 220 controls the auto-zero switch 430 to the ON state, and the determination control circuit 310 controls switches 411 and 412 to the ON state. As a result, the D-phase level Vd is sampled by the capacitive elements 421 and 422.

[0126] Then, as illustrated in figure b, the timing control unit 220 controls switches 413 and 414 to the ON state during the determination period. During this period, it is determined whether the illuminance is higher than a predetermined value. Also, the D-phase level is converted using AD conversion. During this AD conversion, the counter 330 performs an up-count.

[0127] Figure 16 shows an example of the state of the column amplifier 400 and the comparison unit 320 at low light levels during the P-phase sampling period and the AD conversion period of the DDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the P-phase sampling period, and b shows an example of the state during the AD conversion period.

[0128] When the illuminance is low, as illustrated in figure a, the timing control unit 220 controls the auto-zero switch 430 to the ON state, and the determination control circuit 310 controls switches 411 and 412 to the ON state during the P-phase sampling period. As a result, the P-phase level Vp is sampled by the capacitive elements 421 and 422.

[0129] Then, as illustrated in figure b, the determination control circuit 310 controls switches 413 and 414 to the ON state during the AD conversion period. As a result, the P-phase level Vp is held and AD conversion is performed. During this AD conversion, the counter 330 performs a down count.

[0130] Since the counter 330 performs an up-count when the D-phase level Vd is converted and a down-count when the P-phase level Vd is converted, its count value becomes the difference between Vp and Vd, and digital CDS processing is realized.

[0131] Furthermore, the counter 330 can be configured to only perform increment counting. In this case, a calculation circuit that performs digital CDS processing is inserted between the counter 330 and the data recovery unit 340.

[0132] Figure 17 shows an example of the state of the column amplifier 400 and the comparison unit 320 at high illuminance during the P-phase sampling period and the AD conversion period of the DDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the P-phase sampling period, and b shows an example of the state during the AD conversion period.

[0133] When the illuminance is high, as illustrated in figure a, the timing control unit 220 controls the auto-zero switch 430 to the ON state during the P-phase sampling period, and the determination control circuit 310 controls only the switch 412 to the ON state. The amount of charge Q held in the capacitive elements 421 and 422 at this time is expressed by the following equation: Q = n ０ C×(Vd-Vm)+m ０ C × (Vp - Vm) ... Equation 3 In the above equation, C represents the capacitance values ​​of capacitive elements 421 and 422, respectively.

[0134] Then, as illustrated in figure b, the determination control circuit 310 controls switches 413 and 414 to the ON state during the AD conversion period. The amount of charge Q held in capacitive elements 421 and 422 at this time is expressed by the following formula: Q = (SHOUT - Vm) × (n ０ +m ０ )×C...Formula 4

[0135] From equations 3 and 4, the following equation is obtained: SHOUT = (n ０ Vd + m ０ Vp) / (n ０ +m ０ )...Equation 5

[0136] From Equation 5, the output signal SHOUT of the column amplifier 400 is a weighted average of Vp and Vd. This weighted average is then converted using AD conversion. During this AD conversion, the counter 330 performs a down-count.

[0137] Since the counter 330 performs an upcount when converting the D-phase level Vd and a downcount when converting the weighted average, its count value is the difference between Vd and the weighted average, and is expressed by the following formula: Vd - (n ０ Vd + m ０ Vp) / (n ０ +m ０ ) = m ０ (Vp-Vd) / (m ０ +n ０ )...Equation 6

[0138] From Equation 6, the difference in the digital signals is the attenuated net pixel signal, which is the difference between Vp and Vd. This attenuation shortens the AD conversion time and reduces power consumption.

[0139] Figure 18 is a diagram illustrating the effect of pixel signal attenuation in the first embodiment of this technology. It is assumed that the illuminance is determined to be high during the determination period from timing T3 to T4'. In this case, the column amplifier 400 raises the D-phase level to Vd' during the D-phase period from timing T4' onward. As a result, the difference between the P-phase level and the D-phase level is attenuated. The D-phase level after attenuation is denoted as Vd'. The dashed line in the figure shows the trajectory of the pixel signal SIG when there is no attenuation. SHOUT, which is the difference between Vp and Vd', is expressed by Equation 2.

[0140] During the D phase, the reference signal RMP decreases and falls below Vd' at timing T41, causing the comparison result to invert and the AD conversion to end. On the other hand, if there is no attenuation, the reference signal RMP falls below Vd at timing T42, after timing T41, and the AD conversion ends.

[0141] As illustrated in the figure, the voltage range is reduced compared to the case without attenuation due to the attenuation of the difference between the P-phase level and the D-phase level. As a result, the AD conversion time is shortened, and power consumption can be reduced.

[0142] Figure 19 shows an example of the input / output characteristics of the column amplifier 400 and ADC 261 in the first embodiment of this technology. The horizontal axis in the figure represents the net pixel signal SIG', which is the difference between Vp and Vd, and this signal is input to the column amplifier 400. The vertical axis in the figure represents the digital signal CNT output from the ADC 261.

[0143] Furthermore, the thick line shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is less than or equal to the threshold Th_cds, in other words, when the illuminance is low. From Equation 1, the amplification factor m ０ / n ０ The difference is amplified. Therefore, the subsequent data recovery unit 340 uses the reciprocal of the amplification factor n. ０ / m ０ By multiplying this by the digital signal CNT, the data without amplification is restored. The thin line in the figure shows the restored input / output characteristics when the illumination is low.

[0144] The dashed line shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is higher than the threshold Th_cds, in other words, when the illuminance is high. From Equation 1, the amplification factor n ０ / m ０ The difference is attenuated. Therefore, the subsequent data recovery unit 340 calculates the reciprocal of the attenuation rate m ０ / n ０ By multiplying this by the digital signal CNT, the data is restored as if there were no attenuation. The thick dotted line in the figure shows the restored input / output characteristics when the illuminance is high.

[0145] Furthermore, during the DDS period, the attenuation rate m is calculated using Equation 6. ０ / (m ０ +n ０ The difference is attenuated by (m). Therefore, the subsequent data recovery unit 340 calculates the reciprocal of the attenuation rate (m ０ +n ０ ) / m ０ By multiplying this by the digital signal CNT, the data as it would be without attenuation is restored.

[0146] The subsequent image processing unit 280 performs HDR synthesis on the digital signal restored during the CDS period and the digital signal restored during the DDS period. The former is the signal when the FD gain is low gain, and the latter is the signal when the FD gain is high gain.

[0147] Figure 20 is a flowchart illustrating an example of the operation of the image sensor 200 in a first embodiment of this technology. This operation is initiated, for example, when a predetermined application for capturing HDR images is executed.

[0148] The vertical scanning circuit 210 selects a row (step S901) and executes CDS processing at the end of exposure (step S910). Then, the data recovery unit 340 recovers the digital signal after CDS processing (step S902).

[0149] Then, the vertical scanning circuit 210 performs DDS processing (step S920), and the data recovery unit 340 recovers the digital signal after DDS processing (step S903).

[0150] The vertical scanning circuit 210 determines whether all rows have been selected (step S904). If there are any rows that have not been selected (step S904: No), the image sensor 200 executes steps S901 onwards.

[0151] On the other hand, if all rows are selected (step S904: Yes), the image processing unit 280 performs HDR synthesis on the digital signal after restoration during CDS processing and the digital signal after restoration during DDS processing (step S905). After step S905, the operation for capturing the HDR image is completed.

[0152] When capturing multiple HDR images in succession, the processes from steps S901 to S905 are repeatedly executed in synchronization with the vertical synchronization signal.

[0153] Figure 21 is a flowchart showing an example of CDS processing in the first embodiment of this technology. The column amplifier 400 samples the P-phase level (step S911), and the determination control circuit 310 determines whether the illuminance is high illuminance, which is higher than a predetermined value (step S912).

[0154] If the illumination is low (step S913: No), the column amplifier 400 amplifies the difference between the P-phase level and the D-phase level (step S914). On the other hand, if the illumination is high (step S913: Yes), the column amplifier 400 attenuates the difference between the P-phase level and the D-phase level (step S915). After step S914 or S915, the ADC 261 performs AD conversion on the output signal of the column amplifier 400 (step S916), and the CDS processing is completed.

[0155] Figure 22 is a flowchart showing an example of CDS processing in the first embodiment of this technology. The column amplifier 400 samples the D-phase level (step S921). The determination control circuit 310 then determines whether the illuminance is high illuminance, which is higher than a predetermined value, and the ADC 261 performs AD conversion on the D-phase level (step S922).

[0156] If the illuminance is low (step S923: No), the column amplifier 400 samples the P-phase level (step S924). On the other hand, if the illuminance is high (step S923: Yes), the column amplifier 400 performs a weighted averaging process on the D-phase level and the P-phase level (step S925). After step S924 or S925, the ADC 261 performs AD conversion on the output signal of the column amplifier 400 (step S926), and the DDS processing is completed.

[0157] Thus, according to the first embodiment of this technology, the column amplifier 400 amplifies or attenuates the pixel signal depending on whether the illumination is high or low, so even when CDS and DDS systems are mixed, improvements in image quality and readout speed can be achieved.

[0158] <2. Second Embodiment> In the first embodiment described above, the image sensor 200 read out two pixel signals with different FD gains using the CDS method and the DDS method to perform HDR synthesis, but the embodiment is not limited to this configuration. The image sensor 200 in this second embodiment differs from the first embodiment in that it generates multiple pixel signals with different exposure periods and reads them out using the CDS method to perform HDR synthesis.

[0159] Figure 23 is a timing chart showing the driving method of the pixel 250 and the variation of the judgment threshold Th in a second embodiment of the present technology. In the figure, a shows the driving method of the pixel 250, and b shows the variation of the judgment threshold Th.

[0160] In the figure, the vertical axis of 'a' represents the row address, and the horizontal axis represents time. For example, from timing T21 onward, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. The solid diagonal lines in the figure indicate the exposure start timing for each row.

[0161] Then, from timing T22 onward, the vertical scanning circuit 210 drives the rows sequentially to complete the exposure and outputs the pixel signal. The dashed lines in the figure indicate the timing of the end of exposure and readout for each row. However, the FD gain is not switched when reading out the pixel signal, and only the CDS method is used.

[0162] Then, from timing T23 onwards, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. Then, from timing T24 onwards, the vertical scanning circuit 210 drives the rows sequentially to end exposure and outputs a pixel signal. When reading out the pixel signal, the FD gain is not switched, and only the CDS method is used. Also, the length of the second exposure period is set to a different value from the first (for example, a shorter value than the first). The period from timing T21 to T22 corresponds to the longer exposure period EX_L for the first row, and the period from timing T23 to T24 corresponds to the shorter exposure period EX_S for the first row.

[0163] The image sensor 200 has two exposure periods of different lengths set, but it is also possible to set three or more exposure periods of different lengths. Furthermore, the pixel signal for exposure period EX_L is an example of the first pixel signal described in the claims, and the pixel signal for exposure period EX_S is an example of the second pixel signal described in the claims.

[0164] Furthermore, the period from timing T31 to T33 in the figure corresponds to the CDS period when reading out the pixel signal with exposure period EX_L in a predetermined row. The period from timing T34 to T36 corresponds to the CDS period when reading out the pixel signal with exposure period EX_S in a predetermined row.

[0165] The column amplifier 400 samples the P-phase level of the exposure period EX_L during the P-phase period from timing T31 to T32. At timing T32, the timing control unit 220 switches the judgment threshold Th from Th0 to Th1. At timing T33, the timing control unit 220 returns the judgment threshold Th to Th0.

[0166] The column amplifier 400 then samples the P-phase level of the exposure period EX_S during the P-phase period from timing T34 to T35. At timing T35, the timing control unit 220 switches the determination threshold Th from Th0 to Th2. This determination threshold Th2 is set to a higher value than Th1. The determination thresholds Th1 and Th2 are examples of the first and second determination thresholds described in the claims.

[0167] Furthermore, the ADC 261 performs AD conversion on the output signal of the column amplifier 400, and the image processing unit 280 performs HDR synthesis on two digital signals with different exposure times.

[0168] When reading out two pixel signals using the CDS method and the DDS method, the column amplifier 400 cannot perform amplification in the DDS method. On the other hand, in the control shown in examples a and b in the figure, only the CDS method is used, so the column amplifier 400 can amplify both of the two pixel signals to be combined.

[0169] Furthermore, switching the FD gain in three or more stages requires adding inter-wiring capacitance or transistors within the pixel 250. On the other hand, in the control exemplified in a and b of the figure, the image sensor 200 can set three or more exposure periods of different lengths without changing the circuit of the pixel 250. However, it should be noted that the frame rate will decrease compared to when the FD gain is switched.

[0170] The control systems illustrated in a and b in the figure and the control system of the first embodiment can be switched depending on the situation.

[0171] Thus, according to the second embodiment of this technology, since two pixel signals with different exposure times are read out using the CDS method, the column amplifier 400 can amplify both of them. Furthermore, the image sensor 200 can set three or more exposure periods of different lengths without changing the circuit of the pixel 250.

[0172] <3. Examples of Application to Mobile Devices> The technology disclosed herein (the technology) can be applied to various products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0173] Figure 24 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0174] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 24, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0175] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0176] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0177] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0178] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0179] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0180] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0181] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0182] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0183] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 24, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0184] Figure 25 shows an example of the installation position of the imaging unit 12031.

[0185] In Figure 25, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0186] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0187] Figure 25 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0188] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0189] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0190] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0191] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0192] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to, for example, the imaging unit 12031 of the configuration described above. Specifically, the imaging device 100 in Figure 1 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, it is possible to improve the image quality and readout speed, thereby improving the performance of the system.

[0193] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0194] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0195] Furthermore, this technology can also take the following configurations: (1) An image sensor comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of a pixel signal from a pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal; an analog-to-digital converter that converts the output signal into a digital signal; and a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. (2) The image sensor according to (1), further comprising a data recovery unit that performs a process of multiplying the digital signal by the reciprocal of the amplification factor when amplification of the pixel signal is performed, and a process of multiplying the digital signal by the reciprocal of the attenuation factor when attenuation or weighted averaging of the pixel signal is performed. (3) The image sensor according to (1) or (2), wherein the pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel, and the pixel outputs the reset level and the signal level in a predetermined order. (4) The image sensor according to (3), wherein the pixel outputs the signal level after the reset level within a predetermined CDS period, and the determination control circuit performs a sample-and-hold operation of the reset level within the CDS period, performs amplification of the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and performs attenuation of the difference if the illuminance is higher than the predetermined value. (5) The image sensor according to (3) or (4), wherein the pixel outputs the reset level after the signal level within a predetermined DDS period, and the determination control circuit performs a sample-and-hold operation of the signal level within the DDS period, performs a sample-and-hold operation of the reset level if the illuminance is lower than the predetermined value, and performs weighted averaging of the reset level and the signal level if the illuminance is higher than the predetermined value.(6) The image sensor according to any one of (3) to (5), wherein the column amplifier comprises an operational amplifier that outputs the output signal, first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, a predetermined number of first switches that open and close a path between the other end of the first capacitive element and the input terminal of the column amplifier, a predetermined number of second switches that open and close a path between the other end of the second capacitive element and the input terminal of the column amplifier, a predetermined number of third switches that open and close a path between the other end of the first capacitive element and the output terminal of the column amplifier, and a predetermined number of fourth switches that open and close a path between the other end of the first capacitive element and the output terminal. (7) The image sensor according to (6), wherein the determination control circuit comprises a determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illuminance determination result, and a control circuit that controls the first switch, second switch, third switch and fourth switch based on the determination result. (8) The image sensor according to (7), wherein the pixel outputs the signal level after the reset level within a predetermined CDS period, and the control circuit controls the first switch and the second switch to be ON for a predetermined period if the reset level is output within the CDS period, controls the first switch and the third switch to be ON for a predetermined period if the illuminance is lower than the predetermined value, and controls the second switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than the predetermined value. (9) The image sensor according to (7) or (8), wherein the pixel outputs the reset level after the signal level within a predetermined DDS period, and within the DDS period, the control circuit controls the first switch and the second switch to be ON for a predetermined period if the reset level is output, controls the first switch and the second switch to be ON for a predetermined period if the illuminance is lower than the predetermined value, and then controls the third switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than the predetermined value, and then controls the third switch and the fourth switch to be ON for a predetermined period.(10) The image sensor according to any one of (1) to (9), wherein the pixel signal includes first and second pixel signals having different exposure periods, and the determination threshold includes a first determination threshold corresponding to the first pixel signal and a second determination threshold corresponding to the second pixel signal. (11) An imaging device comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of pixel signals from pixels, amplification of the pixel signals, attenuation of the pixel signals, or weighted averaging of the pixel signals; an analog-to-digital converter that converts the output signal into a digital signal; a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signals when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signals when the illuminance is higher than the predetermined value; and an image processing unit that processes an image of the digital signals arranged in sequence. (12) An image sensor control method comprising: an output procedure in which a column amplifier outputs an output signal by performing one of the following: sampling and holding of a pixel signal from a pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal; an analog-to-digital converter performing an analog-to-digital conversion procedure in which an analog-to-digital converter converts the output signal into a digital signal; and a determination control procedure in which a determination control circuit controls the column amplifier to perform sampling and holding or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value.

[0196] 100 Imaging device 110 Optical unit 120 DSP circuit 130 Display unit 140 Operation unit 150 Bus 160 Frame memory 170 Storage unit 180 Power supply unit 200 Image sensor 201 Light receiving chip 202 Circuit chip 210 Vertical scanning circuit 220 Timing control unit 230 DAC 240 Pixel array unit 250 Pixel 251 Photoelectric conversion unit 252 Transfer transistor 253 Reset transistor 254 Interwiring capacitance 255 Second FD unit 256 Connection transistor 257 First FD unit 258 Amplification transistor 259 Selection transistor 260 Column signal processing unit 261 ADC 270 Horizontal scanning circuit 280 Image processing unit 310 Judgment control circuit 311 Judgment circuit 312, 313, 321, 322, 421, 422 Capacitive elements 314, 323 Comparators 315, 316, 324, 325, 430 Auto-zero switches 317 Control circuit 320 Comparison unit 330 Counter 340 Data recovery unit 400 Column amplifier 410 Attenuation / amplification unit 411, 412, 413, 414 Switches 440 Operational amplifier 12031 Imaging unit

Claims

1. An image sensor comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of pixel signals from pixels, amplification of the pixel signals, attenuation of the pixel signals, or weighted averaging of the pixel signals; an analog-to-digital converter that converts the output signal into a digital signal; and a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signals when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signals when the illuminance is higher than the predetermined value.

2. The image sensor according to claim 1, further comprising a data recovery unit that performs the following: multiplying the digital signal by the reciprocal of the amplification factor when amplification of the pixel signal is performed; and multiplying the digital signal by the reciprocal of the attenuation rate when attenuation or weighted averaging of the pixel signal is performed.

3. The image sensor according to claim 1, wherein the pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel, and the pixel outputs the reset level and the signal level in a predetermined order.

4. The image sensor according to claim 3, wherein the pixel outputs the signal level after the reset level within a predetermined CDS (Correlated Double Sampling) period, and within the CDS period, the determination control circuit causes the reset level to perform a sample-and-hold operation, amplifies the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and attenuates the difference if the illuminance is higher than the predetermined value.

5. The image sensor according to claim 3, wherein the pixel outputs the reset level after the signal level within a predetermined DDS (Double Data Sampling) period, and within the DDS period, the determination control circuit causes the signal level to perform a sample-and-hold operation, and if the illuminance is lower than the predetermined value, it causes the reset level to perform a sample-and-hold operation, and if the illuminance is higher than the predetermined value, it causes a weighted average operation of the reset level and the signal level.

6. The image sensor according to claim 3, wherein the column amplifier comprises an operational amplifier that outputs the output signal; first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier; a predetermined number of first switches that open and close a path between the other end of the first capacitive element and the input terminal of the column amplifier; a predetermined number of second switches that open and close a path between the other end of the second capacitive element and the input terminal of the column amplifier; a predetermined number of third switches that open and close a path between the other end of the first capacitive element and the output terminal of the column amplifier; and a predetermined number of fourth switches that open and close a path between the other end of the first capacitive element and the output terminal.

7. The image sensor according to claim 6, wherein the determination control circuit comprises a determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illumination determination result, and a control circuit that controls the first switch, second switch, third switch, and fourth switch based on the determination result.

8. The image sensor according to claim 7, wherein the pixel outputs the signal level after the reset level within a predetermined CDS period, and within the CDS period, the control circuit controls the first switch and the second switch to be ON for a predetermined period if the reset level is output, controls the first switch and the third switch to be ON for a predetermined period if the illuminance is lower than a predetermined value, and controls the second switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than a predetermined value.

9. The image sensor according to claim 7, wherein the pixel outputs the reset level after the signal level within a predetermined DDS period, and within the DDS period, the control circuit controls the first switch and the second switch to be ON for a predetermined period if the reset level is output, controls the first switch and the second switch to be ON for a predetermined period if the illuminance is lower than the predetermined value, and then controls the third switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than the predetermined value, and then controls the second switch to be ON for a predetermined period, and then controls the third switch and the fourth switch to be ON for a predetermined period.

10. The image sensor according to claim 1, wherein the pixel signal includes first and second pixel signals having different exposure periods, and the determination threshold includes a first determination threshold corresponding to the first pixel signal and a second determination threshold corresponding to the second pixel signal.

11. An imaging device comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of pixel signals from pixels, amplification of the pixel signals, attenuation of the pixel signals, or weighted averaging of the pixel signals; an analog-to-digital converter that converts the output signal into a digital signal; a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signals when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signals when the illuminance is higher than the predetermined value; and an image processing unit that processes an image obtained by arranging the digital signals.

12. An image sensor control method comprising: an output procedure in which a column amplifier outputs an output signal by performing one of the following: sampling and holding of a pixel signal from a pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal; an analog-to-digital converter performing an analog-to-digital conversion procedure in which an analog-to-digital converter converts the output signal into a digital signal; and a determination control procedure in which a determination control circuit controls the column amplifier to perform sampling and holding or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value.

Images