Image sensor, imaging device, and control method for image sensor

Abstract

To improve performance in an image sensor equipped with a column amplifier. [Solution] The column amplifier outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal. The analog-to-digital converter converts the output signal into a digital signal. The decision control circuit controls the column amplifier to perform sampling and holding or amplification of the pixel signal if the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal if the illuminance is higher than a predetermined value.

Description

Technical Field

[0001] The present technology relates to an image sensor. More specifically, it relates to an image sensor in which an ADC (Analog to Digital Converter) is arranged for each column, an imaging device, and a method for controlling the image sensor.

Background Art

[0002] Conventionally, in a column ADC type image sensor, a column amplifier may be arranged in front of the ADC for each column. For example, an image sensor has been proposed that determines whether the illuminance is high or not for each column and gives an offset to the input of the column amplifier based on the determination result (see, for example, Patent Document 1). Further, this image sensor uses a CDS (Correlated Double Sampling) method that samples the signal level after the reset level.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-mentioned conventional technology, by giving an offset to the input of the column amplifier based on the determination result of the illuminance, it is intended to prevent a decrease in the dynamic range due to saturation of the output signal of the column amplifier. However, a DDS (Double Data Sampling) method that samples the reset level after the signal level may also be used, and in the case where the DDS method and the CDS method coexist, the above-mentioned configuration cannot perform appropriate control. As a result, in the above-mentioned image sensor, there is a risk that performance such as image quality and readout speed may deteriorate. [[ID=Ⳍ]]

[0005] This technology was developed in light of these circumstances and aims to improve performance in image sensors equipped with column amplifiers. [Means for solving the problem]

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor and control method comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal; an analog-to-digital converter that converts the output signal into a digital signal; and a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. This results in an improvement in the performance of the image sensor.

[0007] Furthermore, in this first aspect, the system may further include a data recovery unit that, when amplification of the pixel signal is performed, multiplies the digital signal by the reciprocal of the amplification factor, and when attenuation or weighted averaging of the pixel signal is performed, multiplies the digital signal by the reciprocal of the attenuation factor. This results in the data being restored to its state before amplification or attenuation.

[0008] Furthermore, in this first aspect, the pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel, and the pixel may output the reset level and the signal level in a predetermined order. This results in the reset level and the signal level being input to the column amplifier in sequence.

[0009] Furthermore, in this first aspect, the pixel may output the signal level after the reset level within a predetermined CDS period, and within the CDS period, the determination control circuit may perform a sample-and-hold operation of the reset level, amplify the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and attenuate the difference if the illuminance is higher than the predetermined value. This results in improved image quality and readout speed.

[0010] Furthermore, in this first aspect, the pixel may output the reset level after the signal level within a predetermined DDS period, and within the DDS period, the determination control circuit may perform a sample-and-hold operation on the signal level, perform a sample-and-hold operation on the reset level if the illuminance is lower than the predetermined value, and perform a weighted average operation on the reset level and the signal level if the illuminance is higher than the predetermined value. This results in an improved readout speed.

[0011] Furthermore, in this first aspect, the column amplifier may include an operational amplifier that outputs the output signal, first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, a predetermined number of first switches that open and close the path between the other end of the first capacitive element and the input terminal of the column amplifier, a predetermined number of second switches that open and close the path between the other end of the second capacitive element and the input terminal of the column amplifier, a predetermined number of third switches that open and close the path between the other end of the first capacitive element and the output terminal of the column amplifier, and a predetermined number of fourth switches that open and close the path between the other end of the first capacitive element and the output terminal. This results in the pixel signal being increased or decreased by controlling the switches.

[0012] Furthermore, in this first aspect, the determination control circuit may include a determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illuminance determination result, and a control circuit that controls the first switch, second switch, third switch, and fourth switch based on the determination result. This results in the illuminance being determined within the determination control circuit.

[0013] Furthermore, in this first aspect, the pixel may output the signal level after the reset level within a predetermined CDS period, and within the CDS period, the control circuit may control the first switch and the second switch to be ON for a predetermined period if the reset level is output, control the first switch and the third switch to be ON for a predetermined period if the illuminance is lower than the predetermined value, and control the second switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than the predetermined value. This results in an effect where the difference between the reset level and the signal level is increased or decreased.

[0014] Furthermore, in this first aspect, the pixel may output the reset level after the signal level within a predetermined DDS period, and within the DDS period, the control circuit may control the first switch and the second switch to be ON for a predetermined period if the reset level is output, control the first switch and the second switch to be ON for a predetermined period if the illuminance is lower than the predetermined value, and then control the third switch and the fourth switch to be ON for a predetermined period if the illuminance is higher than the predetermined value, and then control the second switch to be ON for a predetermined period and then control the third switch and the fourth switch to be ON for a predetermined period. This results in the effect of performing a sample-and-hold operation of the reset level, or a weighted average operation of the reset level and the signal level.

[0015] Furthermore, in this first aspect, the pixel signal may include first and second pixel signals with different exposure period lengths, and the determination threshold may include a first determination threshold corresponding to the first pixel signal and a second determination threshold corresponding to the second pixel signal. This provides the ability to set three or more exposure periods of different lengths, regardless of the pixel circuit.

[0016] Furthermore, the second aspect of this technology is an imaging device comprising: a column amplifier that outputs an output signal by performing one of the following: sampling and holding of pixel signals from pixels, amplification of the pixel signals, attenuation of the pixel signals, or weighted averaging of the pixel signals; an analog-to-digital converter that converts the output signal into a digital signal; a determination control circuit that controls the column amplifier to perform sampling and holding or amplification of the pixel signals when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signals when the illuminance is higher than the predetermined value; and an image processing unit that processes an image formed by arranging the digital signals. This results in an improvement in the performance of the imaging device. [Brief explanation of the drawing]

[0017] [Figure 1] This is a block diagram showing an example configuration of an imaging device in the first embodiment of this technology. [Figure 2] This figure shows an example of a stacked structure of an image sensor in the first embodiment of this technology. [Figure 3] This is a block diagram showing an example configuration of an image sensor in the first embodiment of this technology. [Figure 4] This is a circuit diagram showing an example of a pixel configuration in the first embodiment of this technology. [Figure 5] This is a block diagram showing an example configuration of the column signal processing unit in the first embodiment of this technology. [Figure 6] This is a circuit diagram showing one example configuration of a column amplifier in the first embodiment of this technology. [Figure 7]It is a circuit diagram showing a configuration example of a comparison unit in the first embodiment of the present technology. [Figure 8] It is a circuit diagram showing a configuration example of a determination control circuit in the first embodiment of the present technology. [Figure 9] It is a timing chart showing an example of a pixel driving method in the first embodiment of the present technology. [Figure 10] It is a timing chart showing an example of fluctuations in pixel signals and determination thresholds in the first embodiment of the present technology. [Figure 11] It is a timing chart showing an example of the operations of a timing control unit and a determination control circuit during a CDS period in the first embodiment of the present technology. [Figure 12] It is a timing chart showing an example of the operations of a timing control unit and a determination control circuit during a DDS period in the first embodiment of the present technology. [Figure 13] It is a diagram showing an example of the states of a column amplifier and a comparison unit during a P-phase period and a determination period within a CDS period in the first embodiment of the present technology. [Figure 14] It is a diagram showing an example of the states of a column amplifier and a comparison unit during an AD (Analog to Digital) conversion period within a CDS period in the first embodiment of the present technology. [Figure 15] It is a diagram showing an example of the states of a column amplifier and a comparison unit during a D-phase period and a determination period within a DDS period in the first embodiment of the present technology. [Figure 16] It is a diagram showing an example of the states of a column amplifier and a comparison unit during a P-phase sample period and an AD conversion period at low illuminance within a DDS period in the first embodiment of the present technology. [Figure 17] It is a diagram showing an example of the states of a column amplifier and a comparison unit during a P-phase sample period and an AD conversion period at high illuminance within a DDS period in the first embodiment of the present technology. [Figure 18] It is a diagram for explaining the effect of attenuation of pixel signals in the first embodiment of the present technology. [Figure 19]This figure shows an example of the input / output characteristics of the column amplifier and ADC in the first embodiment of this technology. [Figure 20] This is a flowchart illustrating an example of the operation of the image sensor in the first embodiment of this technology. [Figure 21] This is a flowchart showing an example of CDS processing in the first embodiment of this technology. [Figure 22] This is a flowchart showing an example of DDS processing in the first embodiment of this technology. [Figure 23] This is a timing chart showing the pixel driving method and the fluctuation of the judgment threshold in the second embodiment of this technology. [Figure 24] This block diagram shows an example of a schematic configuration of a vehicle control system. [Figure 25] This is an explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit. [Modes for carrying out the invention]

[0018] The following describes the embodiments for implementing this technology. The description will proceed in the following order. 1. First Embodiment (Example of controlling a column amplifier according to illuminance) 2. Second Embodiment (An example in which the judgment threshold is changed according to the exposure time and the column amplifier is controlled according to the illuminance) 3. Examples of applications to mobile devices

[0019] <1. First Embodiment> [Example of imaging device configuration] Figure 1 is a block diagram showing an example configuration of an imaging device 100 in the first embodiment of this technology. This imaging device 100 is a device for capturing image data and comprises an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. Furthermore, the imaging device 100 comprises a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. Examples of imaging devices 100 include cameras mounted on smartphones and in-vehicle cameras.

[0020] The optical unit 110 collects light from the subject and directs it to the image sensor 200. The image sensor 200 generates image data through photoelectric conversion. The image sensor 200 supplies the generated image data to the DSP circuit 120 via the signal line 209.

[0021] The DSP circuit 120 performs predetermined signal processing on the image data. The DSP circuit 120 outputs the processed image data to a frame memory 160 or the like via the bus 150.

[0022] The display unit 130 displays image data. The display unit 130 could be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The operation unit 140 generates operation signals according to user input.

[0023] Bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with each other.

[0024] The frame memory 160 stores image data. The storage unit 170 stores various data, including image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and other components.

[0025] Figure 2 shows an example of a stacked structure of an image sensor 200 in the first embodiment of this technology. This image sensor 200 comprises a circuit chip 202 and a light-receiving chip 201 stacked on the circuit chip 202. These chips are electrically connected via connection points such as vias. In addition to vias, connections can also be made by Cu-Cu junctions or bumps.

[0026] [Example of image sensor configuration] Figure 3 is a block diagram showing an example configuration of an image sensor 200 in a first embodiment of this technology. This image sensor 200 includes a vertical scanning circuit 210, a timing control unit 220, a DAC (Digital to Analog Converter) 230, a pixel array unit 240, a column signal processing unit 260, a horizontal scanning circuit 270, and an image processing unit 280. Multiple pixels 250 are arranged in a two-dimensional grid in the pixel array unit 240.

[0027] The pixel array section 240 is located on, for example, the light-receiving chip 201, and the remaining circuits are located on the circuit chip 202. Note that the circuits located on each chip are not limited to those illustrated in the figure.

[0028] The vertical scanning circuit 210 sequentially selects and drives rows within the pixel array section 240.

[0029] The timing control unit 220 controls the operating timing of the vertical scanning circuit 210, DAC 230, column signal processing unit 260, and horizontal scanning circuit 270 in synchronization with the vertical synchronization signal VSYNC.

[0030] The DAC230 generates a sawtooth-shaped ramp signal and supplies it to the column signal processing unit 260 as a reference signal.

[0031] Pixel 250 generates an analog pixel signal by photoelectric conversion in accordance with the control of the vertical scanning circuit 210. Each pixel 250 in a column outputs a pixel signal to the column signal processing unit 260 via a vertical signal line (not shown).

[0032] The column signal processing unit 260 has a column amplifier and an ADC (not shown) for each column of pixels 250. Each ADC converts the pixel signal of the corresponding column into a digital signal and outputs it to the image processing unit 280 according to the control of the horizontal scanning circuit 270.

[0033] The horizontal scanning circuit 270 controls the column signal processing unit 260 to output digital signals sequentially.

[0034] The image processing unit 280 performs various image processing operations on image data, which is an array of digital signals. The image processing unit 280 supplies the processed image data to the DSP circuit 120 via the signal line 209.

[0035] [Example of pixel configuration] Figure 4 is a circuit diagram showing one example configuration of a pixel 250 in the first embodiment of this technology. This pixel 250 comprises a photoelectric conversion unit 251, a transfer transistor 252, a reset transistor 253, a first FD (Floating Diffusion) unit 257, an amplification transistor 258, and a selection transistor 259. The pixel 250 further comprises an interwiring capacitance 254, a second FD unit 255, and a connection transistor 256.

[0036] The photoelectric conversion unit 251 has its anode terminal grounded and its cathode terminal connected to the first FD unit 257 via a transfer transistor 252. The photoelectric conversion unit 251 receives light irradiated onto the light-receiving surface of the image sensor 200 and converts it into an electric charge corresponding to the amount of light.

[0037] The transfer transistor 252 is positioned to connect the photoelectric conversion unit 251 and the first FD unit 257. The transfer transistor 252 is driven according to the transfer signal TG from the vertical scanning circuit 210 and transfers the charge photoelectrically converted in the photoelectric conversion unit 251 to the first FD unit 257.

[0038] The first FD unit 257 stores the charge transferred from the photoelectric conversion unit 251 via the transfer transistor 252 in order to convert it into a pixel signal.

[0039] The connecting transistor 256 is positioned to connect the first FD section 257 and the second FD section 255. The connecting transistor 256 is driven according to the connection signal FDG from the vertical scanning circuit 210, and switches the connection between the first FD section 257 and the second FD section 255 on and off.

[0040] The second FD section 255 is connected to the first FD section 257 when the connecting transistor 256 is turned on, and together with the first FD section 257, it accumulates charge.

[0041] The inter-wiring capacitance 254 is composed of a MIM (Metal-Insulator-Metal) structure capacitor placed between the wires and is positioned to connect the second FD section 255 and the signal wiring MIMVDD. The inter-wiring capacitance 254 accumulates charge in the same way as the second FD section 255.

[0042] The reset transistor 253 is positioned to connect the second FD section 255 and the power supply wiring VDD. The reset transistor 253 is driven according to the reset signal RST from the vertical scanning circuit 210. When the connecting transistor 256 and the reset transistor 253 are turned on, the charge accumulated in the first FD section 257, the second FD section 255, and the inter-wiring capacitance 254 is discharged to the power supply wiring VDD, and the charge is reset.

[0043] The amplification transistor 258 is positioned so that the first FD section 257 is connected to its gate electrode, and the power supply wiring VDD is connected to the selection transistor 259. The amplification transistor 258 converts the charge stored in the first FD section 257 alone, or in the first FD section 257, the second FD section 255, and the interwiring capacitance 254, into a pixel signal at a level corresponding to the charge in each capacitance.

[0044] The selection transistor 259 is positioned to connect the amplification transistor 258 and the vertical signal line 249. The selection transistor 259 is driven according to the selection signal SEL from the vertical scanning circuit 210, and while the selection transistor 259 is ON, the pixel signal SIG from the amplification transistor 258 is output to the vertical signal line 249.

[0045] The level of the pixel signal SIG when the reset transistor 253 initializes the pixel 250 is referred to as the "reset level" or "P-phase level." The level of the pixel signal SIG when the transfer transistor 252 transfers charge within the pixel is referred to as the "signal level" or "D-phase level."

[0046] For example, an nMOS (n-channel Metal Oxide Semiconductor) transistor is used as the transistor within pixel 250. In the circuit configuration exemplified in the figure, the FD gain, which is the conversion efficiency when converting charge to voltage, is controlled in two stages. The FD gain when the connected transistor 256 is ON is lower than when the connected transistor 256 is OFF; the former is called "low gain" and the latter is called "high gain".

[0047] The image sensor 200 can obtain pixel signals similar to those obtained when exposing each of the 250 pixels for a predetermined exposure period and reading out the pixel signals at both low and high gain. These analog pixel signals are converted into digital signals by the column signal processing unit 260. The subsequent image processing unit 280 can expand the dynamic range by performing HDR synthesis on the digital signals from the low-gain and high-gain states.

[0048] [Example of a column signal processing unit configuration] Figure 5 is a block diagram showing an example configuration of the column signal processing unit 260 in the first embodiment of this technology. This column signal processing unit 260 includes a determination control circuit 310, a column amplifier 400, an ADC 261, and a data recovery unit 340 for each column.

[0049] The column amplifier 400 performs one of the following operations on the pixel signal SIG from the corresponding column: sample-and-hold, amplification, attenuation, or weighted averaging. The column amplifier 400 outputs the analog pixel signal, after performing sample-and-hold or other operations, as an output signal to the corresponding ADC261.

[0050] The determination control circuit 310 determines whether the illuminance is high or low (high illuminance) based on the pixel signal SIG of the corresponding column, and controls the column amplifier 400 based on the determination result CMP. If the illuminance is low (low illuminance) below the predetermined value, the determination control circuit 310 controls the column amplifier 400 to perform sample-and-hold or amplification of the pixel signal SIG. On the other hand, if the illuminance is high, the determination control circuit 310 controls the column amplifier 400 to perform attenuation or weighted averaging of the pixel signal SIG. The determination control circuit 310 also supplies the determination result CMP to the data recovery unit 340.

[0051] The ADC261 performs AD conversion of the output signal of the column amplifier 400 into a digital signal CNT. This ADC261 is a single-slope type ADC equipped with a comparator 320 and a counter 330.

[0052] The comparison unit 320 compares the reference signal RMP from the DAC230 with the output signal of the column amplifier 400 and outputs the comparison result VCO to the counter 330.

[0053] The counter 330 counts the count value over the period until the comparison result VCO inverts. This counter 330 supplies a digital signal CNT indicating the count value to the data recovery unit 340. The value of this digital signal CNT will be different from the value when the pixel signal is amplified or attenuated by the column amplifier 400 compared to when these actions are not performed.

[0054] The data restoration unit 340 restores the value of the digital signal CNT to the value obtained when amplification or other processing is not performed, based on the judgment result CMP. When the column amplifier 400 performs amplification, the data restoration unit 340 restores the digital signal CNT by multiplying it by the reciprocal of the amplification factor. On the other hand, when the column amplifier 400 performs attenuation or weighted averaging, the data restoration unit 340 restores the digital signal CNT by multiplying it by the reciprocal of the attenuation factor. The data restoration unit 340 outputs the restored digital signal as Dout to the image processing unit 280.

[0055] [Example of a column amplifier configuration] Figure 6 is a circuit diagram showing one example configuration of a column amplifier 400 in the first embodiment of this technology. This column amplifier 400 comprises an attenuation / amplification unit 410, an auto-zero switch 430, and an operational amplifier 440. The attenuation / amplification unit 410 comprises N (N is an integer) capacitive elements 421, N switches 411, and N switches 413. Furthermore, the attenuation / amplification unit 410 comprises M (M is an integer) capacitive elements 422, M switches 412, and M switches 414. N may be a different value from or the same value as M.

[0056] Furthermore, the input terminal 401 of the column amplifier 400 is connected to the vertical signal line 249 of the corresponding column. The output terminal 402 of the column amplifier 400 is connected to the comparison unit 320 of the corresponding column.

[0057] Each of the N capacitive elements 421 and each of the M capacitive elements 422 are connected in common to the inverting input terminal (-) of the operational amplifier 440. Note that the capacitive elements 421 and 422 are examples of the first and second capacitive elements described in the claims.

[0058] The nth (where n is an integer from 1 to N) switch 411 opens and closes the path between the other end of the nth capacitive element 421 and the input terminal 401, according to the control signal Φsn from the determination control circuit 310. Note that switch 411 is an example of the first switch described in the claims.

[0059] The m-th switch 412 (where m is an integer from 1 to M) opens and closes the path between the other end of the m-th capacitive element 422 and the input terminal 401, according to the control signal Φsm from the determination control circuit 310. Note that switch 412 is an example of the second switch described in the claims.

[0060] The nth switch 413 opens and closes the path between the other end of the nth capacitive element 421 and the output terminal of the operational amplifier 440, according to the control signal Φhn from the determination control circuit 310. Note that switch 413 is an example of the third switch described in the claims.

[0061] The m-th switch 414 opens and closes the path between the other end of the m-th capacitive element 422 and the output terminal of the operational amplifier 440, according to the control signal Φhm from the determination control circuit 310. Note that switch 414 is an example of the fourth switch described in the claims.

[0062] The auto-zero switch 430 opens and closes the path between the inverting input terminal (-) and output terminal of the operational amplifier 440 according to the auto-zero signal AZsh from the timing control unit 220.

[0063] The non-inverting input terminal (+) of the operational amplifier 440 is connected to ground potential, and its output terminal is connected to the output terminal 402 of the column amplifier 400. This operational amplifier 440 outputs the output signal SHOUT to the comparison unit 320.

[0064] The decision control circuit 310 uses the control signal Φsn to turn on or off n0 (where n0 is an integer from 1 to N) of the N switches 411. The remaining switches 411 are controlled to be in the off state. When the nth switch 411 is turned on or off, the control signal Φhn also turns on or off the nth switch 413.

[0065] Furthermore, the judgment control circuit 310 uses the control signal Φsm to turn on or off m0 (where m0 is an integer from 1 to M) of the M switches 412. The remaining switches 412 are controlled to be in the off state. When the mth switch 412 is turned on or off, the control signal Φhm also turns on or off the mth switch 414.

[0066] The control method for the switches in the diagram will be described later. Also, m0 is assumed to be greater than n0. These values ​​of m0 and n0 are pre-set and stored in registers or similar.

[0067] [Example of the configuration of the comparison section] Figure 7 is a circuit diagram showing one example configuration of the comparison unit 320 in the first embodiment of this technology. This comparison unit 320 comprises capacitive elements 321 and 322, a comparator 323, and auto-zero switches 324 and 325.

[0068] The output signal SHOUT from the column amplifier 400 is input to the inverting input terminal (-) of comparator 323 via capacitive element 321. The reference signal RMP from DAC230 is input to the non-inverting input terminal (+) of comparator 323 via capacitive element 322. Comparator 323 compares these signals and outputs the comparison result VCO to counter 330.

[0069] The auto-zero switch 324 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0070] The auto-zero switch 325 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0071] [Example of a judgment control circuit configuration] Figure 8 is a circuit diagram showing one example configuration of the determination control circuit 310 in the first embodiment of this technology. This determination control circuit 310 comprises a determination circuit 311 and a control circuit 317.

[0072] The determination circuit 311 determines whether the illuminance is higher than a predetermined value based on the pixel signal SIG of the corresponding column. This determination circuit 311 comprises capacitive elements 312 and 313, a comparator 314, and auto-zero switches 315 and 316.

[0073] The inverting input terminal (-) of comparator 314 is connected to the vertical signal line 249 via capacitive element 312, and the pixel signal SIG is input to this terminal. The non-inverting input terminal (+) of comparator 314 is input to the judgment threshold Th from the timing control unit 220 via capacitive element 313. Comparator 324 compares the level of the pixel signal SIG with the judgment threshold Th and outputs the comparison result as the illuminance judgment result CMP to the control circuit 317 and the data recovery unit 340.

[0074] The auto-zero switch 315 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0075] The auto-zero switch 316 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0076] The control circuit 317 controls the column amplifier 400 using control signals Φsm, Φsn, Φhm, and Φhn, based on the determination result CMP and the control signals Ctrl and MODE from the timing control unit 220. The control signal Ctrl is a signal that indicates the timing for controlling the switch. The mode signal MODE is a signal that indicates either the CDS method or the DDS method.

[0077] [Example of image sensor operation] Figure 9 is a timing chart showing an example of a method for driving pixels 250 in the first embodiment of this technology. In the figure, a is a timing chart showing an example of a method for driving all rows. In the figure, b is a timing chart showing an example of a method for driving the first row.

[0078] In the figure, the vertical axis of 'a' represents the row address, and the horizontal axis represents time. For example, from timing T1 onward, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. The solid diagonal lines in the figure indicate the exposure start timing for each row.

[0079] Then, from timing T2 onward, the vertical scanning circuit 210 drives the rows sequentially to complete the exposure and outputs the pixel signal. The dashed lines in the figure indicate the timing of the end of exposure and readout for each row. The period from timing T1 to T2 in the figure a indicates the exposure period for the first row.

[0080] As illustrated in figure b, during the pulse period from timing T0 to T1, the vertical scanning circuit 210 raises the connection signal FDG, the transfer signal TG, and the reset signal RST to high levels. This initiates exposure.

[0081] Then, during the CDS period from exposure end timing T2 to timing T5, the vertical scanning circuit 210 drives the pixels 250 using the CDS method. This CDS period includes a P-phase period from timing T2 to timing T3 and a D-phase period from timing T4 to timing T5.

[0082] During the P-phase period from timing T2 to T3, the vertical scanning circuit 210 raises the selection signal SEL to a high level. During this period, the column amplifier 400 samples the reset level (in other words, the P-phase level).

[0083] Then, during the pulse period from timing T3 to T4, the vertical scanning circuit 210 raises the transfer signal TG to a high level. This causes charge to be transferred within the pixel 250, generating a signal level (in other words, a D-phase level). Then, at timing T4, the vertical scanning circuit 210 raises the selection signal SEL to a high level.

[0084] Furthermore, the vertical scanning circuit 210 keeps the connection signal FDG at a low level during the CDS period from timing T2 to timing T5. As a result, the P-phase level and D-phase level are generated at high gain.

[0085] Then, the vertical scanning circuit 210 raises the connection signal FDG to a high level at timing T5, and raises the transfer signal TG to a high level within the pulse period from timing T5 to timing T6. This generates a D-phase level with low gain.

[0086] At timing T6, the vertical scanning circuit 210 changes the mode signal MODE from a low level to a high level. The mode signal MODE is controlled to a low level when using the CDS method and to a high level when using the DDS method.

[0087] During the DDS period from timing T6 to timing T9, the vertical scanning circuit 210 drives the pixels 250 using the DDS method. Since the connection signal FDG is at a high level during the DDS period, the P-phase level and D-phase level during this period are generated at low gain. The DDS period also includes the D-phase period from timing T6 to timing T7 and the P-phase period from timing T8 to timing T9. The D-phase level is sampled during this D-phase period. At timing T7, the vertical scanning circuit 210 sets the selection signal SEL to a low level.

[0088] Then, during the pulse period from timing T7 to T8, the vertical scanning circuit 210 sets the reset signal RST to a high level. This initializes the pixel 250 and generates a reset level (P-phase level). During the P-phase period from timing T8 to T9, the vertical scanning circuit 210 sets the selection signal SEL to a high level.

[0089] Furthermore, the vertical scanning circuit 210 switches the potential of the signal wiring MIMVDD from a low level (for example, an intermediate voltage lower than the power wiring VDD) to a high level (for example, the same voltage as the power wiring VDD) before exposure begins. After exposure begins, the potential of the signal wiring MIMVDD is switched from a high level to a low level. Subsequently, at pixel 250, the potential of the signal wiring MIMVDD is switched from a low level to a high level before the CDS period. By controlling the potential of the signal wiring MIMVDD in this way, the increase in noise during the exposure period can be suppressed. This improves image quality.

[0090] While the image sensor 200 uses both the CDS and DDS methods when performing HDR synthesis, it is not limited to this configuration. The image sensor 200 can also perform imaging without HDR synthesis using only one of the CDS or DDS methods.

[0091] Figure 10 is a timing chart showing an example of the variation of the pixel signal SIG and the judgment threshold Th in the first embodiment of this technology. In the figure, a shows the variation of the pixel signal SIG and the judgment threshold Th in the case of low illumination. In the figure, b shows the variation of the pixel signal SIG and the judgment threshold Th in the case of high illumination.

[0092] As illustrated in figures a and b, the timing control unit 220 sets the determination threshold Th to Th0 over the P-phase period within the CDS period. This Th0 is set to a value slightly lower than the P-phase level, for example.

[0093] Then, at the timing T3 in which the charge is transferred, the timing control unit 220 sets the determination threshold Th to Th1, which is lower than Th0.

[0094] Then, at timing T5, the start of the D-phase period within the DDS period, the timing control unit 220 sets the determination threshold Th to Th2. This Th2 is set to a value lower than the D-phase level expected at high illuminance.

[0095] Then, at timing T7 when the pixels 250 within the DDS period are initialized, the timing control unit 220 sets the determination threshold Th to Th3, which is higher than Th2.

[0096] Furthermore, as illustrated in figure a, when the illumination is low, the pixel signal SIG is higher than the judgment threshold in all periods. Therefore, the judgment result of the judgment control circuit 310 is not inverted.

[0097] On the other hand, as illustrated in figure b, when the illuminance is high, the level of the pixel signal SIG (here, the D-phase level) becomes lower than the judgment threshold Th1 during the transfer period from timing T3 to T4, and the judgment result of the judgment control circuit 310 is reversed. Also, during the initialization period from timing T7 to T8, the level of the pixel signal SIG (here, the P-phase level) becomes higher than the judgment threshold Th3, and the judgment result of the judgment control circuit 310 is reversed.

[0098] As illustrated in figures a and b, the determination control circuit 310 can determine whether the illuminance is higher than a predetermined value based on the determination threshold Th. Th_cds, which is the difference between determination thresholds Th0 and Th1, corresponds to a relative threshold that is compared with the difference between the P-phase level and D-phase level during the CDS period. Th_dds, which is the difference between determination thresholds Th2 and Th3, corresponds to a relative threshold that is compared with the difference between the P-phase level and D-phase level during the DDS period.

[0099] Figure 11 is a timing chart showing an example of the operation of the timing control unit 220 and the determination control circuit 310 during the CDS period in the first embodiment of this technology.

[0100] During the P-phase period from timing T2 to T3 within the CDS period, the timing control unit 220 raises the auto-zero signals AZcmp and AZsh to high levels, causing the comparators of the column amplifier 400 and the decision control circuit 310 to perform auto-zero operations.

[0101] Furthermore, during the P-phase period, the decision control circuit 310 raises m0 control signals Φsm and n0 control signals Φsn to high levels. As a result, the P-phase level Vp is sampled and held when the P-phase period elapses.

[0102] Then, at timing T3, the timing control unit 220 changes the judgment threshold Th from Th0 to Th1.

[0103] Furthermore, during the judgment period from timing T3 to timing T4', charge is transferred, and the level of the pixel signal SIG changes from the P-phase level Vp to the D-phase level Vd. During this judgment period, if the illuminance is lower than a predetermined value, the judgment result CMP inverts from a high level to a low level. On the other hand, if the illuminance is higher than a predetermined value, the judgment result CMP remains at a high level. The dashed line in the figure shows the judgment result when the illuminance is lower than a predetermined value.

[0104] During the AD conversion period after timing T4', the determination control circuit 310 raises n0 control signals Φsn and m0 control signals Φhm to high levels if the illuminance is higher than a predetermined value. This attenuates the difference between Vp and Vd.

[0105] On the other hand, if the illuminance is lower than a predetermined value, the detection control circuit 310 raises m0 control signals Φsm and n0 control signals Φhn to high levels. This amplifies the difference between Vp and Vd. The dashed line in the figure shows the control signals when the illuminance is lower than a predetermined value. The difference after attenuation or amplification is then converted using AD conversion.

[0106] Figure 12 is a timing chart showing an example of the operation of the timing control unit 220 and the decision control circuit 310 during the DDS period in the first embodiment of this technology.

[0107] During the DDS period, within the D phase period from timing T6 to T7, the timing control unit 220 raises the auto-zero signals AZcmp and AZsh to high levels, causing the comparators of the column amplifier 400 and the decision control circuit 310 to perform auto-zero operations.

[0108] Furthermore, during the D-phase period, the determination control circuit 310 raises m0 control signals Φsm and n0 control signals Φsn to high levels. As a result, the D-phase level Vd is sampled and held when the D-phase period has elapsed.

[0109] Then, at timing T7, the timing control unit 220 changes the judgment threshold Th from Th2 to Th3.

[0110] Furthermore, during the judgment period from timing T7 to timing T8', pixel 250 is initialized, and the level of the pixel signal SIG changes from the D-phase level Vd to the P-phase level Vp. During this judgment period, if the illuminance is higher than a predetermined value, the judgment result CMP inverts from a high level to a low level. On the other hand, if the illuminance is lower than the predetermined value, the judgment result CMP remains at a high level. The dashed line in the figure shows the judgment result when the illuminance is lower than the predetermined value. Also, during the judgment period, the D-phase level is converted using AD conversion.

[0111] In the case of low illumination, during the P-phase sampling period from timing T8' to timing T81, the determination control circuit 310 raises m0 control signals Φsm and n0 control signals Φsn to high levels. As a result, the P-phase level is sampled and held after the P-phase sampling period has elapsed. Then, during the AD conversion period from timing T81 onward, the P-phase level is converted using AD conversion.

[0112] On the other hand, in the case of high illuminance, the determination control circuit 310 raises m0 control signals Φsm to a high level during the P-phase sampling period. Then, within the AD conversion period after timing T81, the determination control circuit 310 raises m0 control signals Φhm and n0 control signals Φhn to a high level. As a result, a weighted average processing of Vp and Vd is performed, and that value is AD converted.

[0113] Next, referring to Figures 13 to 17, the state of column amplifier 400 during the CDS period and DDS period will be explained.

[0114] Figure 13 shows an example of the state of the column amplifier 400 and comparison unit 320 during the P-phase period and the determination period of the CDS period in the first embodiment of this technology. Figure a shows an example of the state during the P-phase period, and figure b shows an example of the state during the determination period.

[0115] It is assumed that n0 of the N switches 411, m0 of the M switches 412, n0 of the N switches 413, and m0 of the M switches 414 are switched on and off. However, for convenience of description, in figures a and b, only one of the controlled switches 411, 412, 413, and 414 is shown, and the rest are omitted. Similarly, only one of the N capacitive elements 421 and M capacitive elements 422 is shown, and the rest are omitted. The same applies to Figures 14 to 17.

[0116] As illustrated in figure a, during the P-phase period, the timing control unit 220 controls the auto-zero switch 430 to the ON state, and the determination control circuit 310 controls switches 411 and 412 to the ON state. As a result, the P-phase level Vp is sampled by the capacitive elements 421 and 422.

[0117] Then, as illustrated in figure b, the timing control unit 220 controls the auto-zero switches 324 and 325 to the ON state during the determination period. During this period, it is determined whether or not the illuminance is higher than a predetermined value.

[0118] Figure 14 shows an example of the state of the column amplifier 400 and the comparator 320 during the AD conversion period of the CDS period in the first embodiment of this technology. Figure a shows an example of the state when the illuminance is lower than a predetermined value, and figure b shows an example of the state when the illuminance is higher than a predetermined value.

[0119] As illustrated in figure a, when the illuminance is low, the determination control circuit 310 turns on switches 412 and 413. The level of the output signal SHOUT from the column amplifier 400 at this time is expressed by the following formula. SHOUT=(Vp-Vd)×m0 / n0...Equation 1

[0120] As mentioned above, since m0 is set to a value greater than n0, the output signal SHOUT from Equation 1 is the amplified difference between Vp and Vd. This amplification reduces the noise in the signal after the comparison unit 320.

[0121] On the other hand, as illustrated in figure b, when the illuminance is high, the determination control circuit 310 turns on switches 411 and 414. The level of the output signal SHOUT from the column amplifier 400 at this time is expressed by the following formula. SHOUT=(Vp-Vd)×n0 / m0...Equation 2

[0122] Since m0 is set to a value greater than n0, the output signal SHOUT, according to Equation 2, becomes the attenuated value of the difference between Vp and Vd. This attenuation shortens the AD conversion time (in other words, improves the read speed) and reduces power consumption.

[0123] Furthermore, as illustrated in figures a and b, the difference between Vp and Vd is sampled in the capacitive elements 421 and 422, thereby realizing analog CDS processing. This analog CDS processing eliminates the need for P-phase level AD conversion.

[0124] Figure 15 shows an example of the state of the column amplifier 400 and the comparison unit 320 during the D-phase period and the determination period of the DDS period in the first embodiment of this technology. Figure a shows an example of the state during the D-phase period, and figure b shows an example of the state during the determination period.

[0125] As illustrated in figure a, during the D phase period, the timing control unit 220 controls the auto-zero switch 430 to the ON state, and the determination control circuit 310 controls switches 411 and 412 to the ON state. As a result, the D phase level Vd is sampled by the capacitive elements 421 and 422.

[0126] Then, as illustrated in figure b, the timing control unit 220 controls switches 413 and 414 to the ON state during the determination period. During this period, it is determined whether the illuminance is higher than a predetermined value. Also, the D-phase level is converted using AD conversion. During this AD conversion, the counter 330 performs an up-count.

[0127] Figure 16 shows an example of the state of the column amplifier 400 and the comparison unit 320 at low light levels during the P-phase sampling period and the AD conversion period of the DDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the P-phase sampling period, and b shows an example of the state during the AD conversion period.

[0128] When the illuminance is low, as illustrated in figure a, the timing control unit 220 controls the auto-zero switch 430 to the ON state and the determination control circuit 310 controls switches 411 and 412 to the ON state during the P-phase sampling period. As a result, the P-phase level Vp is sampled by the capacitive elements 421 and 422.

[0129] Then, as illustrated in figure b, the determination control circuit 310 controls switches 413 and 414 to the ON state during the AD conversion period. As a result, the P-phase level Vp is held and AD conversion is performed. During this AD conversion, the counter 330 performs a down count.

[0130] Since counter 330 performs up-counting when converting the D-phase level Vd and down-counting when converting the P-phase level Vd, its count value becomes the difference between Vp and Vd, enabling digital CDS processing.

[0131] It is also possible to configure the counter 330 to only perform up-counting. In this case, a calculation circuit that performs digital CDS processing is inserted between the counter 330 and the data recovery unit 340.

[0132] Figure 17 shows an example of the state of the column amplifier 400 and the comparison unit 320 at high illuminance during the P-phase sampling period and the AD conversion period of the DDS period in the first embodiment of this technology. In the figure, a shows an example of the state during the P-phase sampling period, and b shows an example of the state during the AD conversion period.

[0133] When the illuminance is high, as illustrated in figure a, the timing control unit 220 controls the auto-zero switch 430 to the ON state during the P-phase sampling period, and the determination control circuit 310 controls only the switch 412 to the ON state. The amount of charge Q held in the capacitive elements 421 and 422 at this time is expressed by the following equation. Q=n0C×(Vd-Vm)+m0C×(Vp-Vm)...Equation 3 In the above equation, C represents the capacitance values ​​of the capacitive elements 421 and 422, respectively.

[0134] Then, as illustrated in figure b, the decision control circuit 310 controls switches 413 and 414 to the ON state during the AD conversion period. The amount of charge Q held in capacitive elements 421 and 422 at this time is expressed by the following equation. Q=(SHOUT-Vm)×(n0+m0)×C...Equation 4

[0135] From equations 3 and 4, the following equation is obtained. SHOUT=(n0Vd+m0Vp) / (n0+m0)...Equation 5

[0136] From Equation 5, the output signal SHOUT of the column amplifier 400 is a weighted average of Vp and Vd. This weighted average is then converted using AD conversion. During this AD conversion, the counter 330 performs a down-count.

[0137] Since counter 330 performs an upcount when converting the D-phase level Vd and a downcount when converting the weighted average, its count value is the difference between Vd and the weighted average, and is expressed by the following formula. Vd - (n0Vd + m0Vp) / (n0 + m0) =m0(Vp-Vd) / (m0+n0) ...Equation 6

[0138] From Equation 6, the difference in the digital signal is the attenuated net pixel signal, which is the difference between Vp and Vd. This attenuation shortens the AD conversion time and reduces power consumption.

[0139] Figure 18 illustrates the effect of pixel signal attenuation in the first embodiment of this technology. It is assumed that the illuminance is determined to be high during the determination period from timing T3 to T4'. In this case, the column amplifier 400 raises the D-phase level to Vd' during the D-phase period from timing T4' onward. This attenuates the difference between the P-phase level and the D-phase level. The D-phase level after attenuation is denoted as Vd'. The dashed line in the figure shows the trajectory of the pixel signal SIG when there is no attenuation. SHOUT, which is the difference between Vp and Vd', is expressed by Equation 2.

[0140] During the D phase, the reference signal RMP decreases and falls below Vd' at timing T41, causing the comparison result to invert and the AD conversion to end. On the other hand, if there is no attenuation, the reference signal RMP falls below Vd at timing T42, after timing T41, and the AD conversion ends.

[0141] As illustrated in the figure, the voltage range is reduced compared to the case without attenuation due to the attenuation of the difference between the P-phase level and the D-phase level. As a result, the AD conversion time is shortened, and power consumption can be reduced.

[0142] Figure 19 shows an example of the input / output characteristics of the column amplifier 400 and ADC261 in the first embodiment of this technology. The horizontal axis in the figure represents the net pixel signal SIG', which is the difference between Vp and Vd, and this signal is input to the column amplifier 400. The vertical axis in the figure represents the digital signal CNT output from the ADC261.

[0143] Furthermore, the thick line shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is less than or equal to the threshold Th_cds, in other words, when the illuminance is low. From Equation 1, the difference is amplified by an amplification factor m0 / n0. Therefore, the subsequent data restoration unit 340 restores the data as it would be without amplification by multiplying the digital signal CNT by the reciprocal of the amplification factor n0 / m0. The thin line in the same figure shows the input / output characteristics after restoration when the illuminance is low.

[0144] The dashed line shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is higher than the threshold Th_cds, in other words, when the illuminance is high. From Equation 1, the difference is attenuated by an amplification factor of n0 / m0. Therefore, the subsequent data recovery unit 340 recovers the data as if it were not attenuated by multiplying the digital signal CNT by the reciprocal of the attenuation rate, m0 / n0. The thick dotted line in the figure shows the input / output characteristics after recovery when the illuminance is high.

[0145] Furthermore, during the DDS period, the difference is attenuated by an attenuation rate m0 / (m0+n0) according to Equation 6. Therefore, the subsequent data recovery unit 340 recovers the data as it would be without attenuation by multiplying the digital signal CNT by the reciprocal of the attenuation rate, (m0+n0) / m0.

[0146] The subsequent image processing unit 280 performs HDR synthesis of the digital signal restored during the CDS period and the digital signal restored during the DDS period. The former is the signal when the FD gain is low, and the latter is the signal when the FD gain is high.

[0147] Figure 20 is a flowchart illustrating an example of the operation of the image sensor 200 in a first embodiment of this technology. This operation is initiated, for example, when a predetermined application for capturing HDR images is executed.

[0148] The vertical scanning circuit 210 selects a row (step S901) and performs CDS processing at the end of exposure (step S910). Then, the data recovery unit 340 recovers the digital signal after CDS processing (step S902).

[0149] Then, the vertical scanning circuit 210 performs DDS processing (step S920), and the data recovery unit 340 recovers the digital signal after DDS processing (step S903).

[0150] The vertical scanning circuit 210 determines whether all rows have been selected (step S904). If there are any unselected rows (step S904: No), the image sensor 200 executes steps S901 onwards.

[0151] On the other hand, if all rows are selected (step S904: Yes), the image processing unit 280 performs HDR synthesis of the digital signal after restoration during CDS processing and the digital signal after restoration during DDS processing (step S905). After step S905, the operation for capturing the HDR image is completed.

[0152] When capturing multiple HDR images in succession, the processes from steps S901 to S905 are repeatedly executed in synchronization with the vertical synchronization signal.

[0153] Figure 21 is a flowchart showing an example of CDS processing in the first embodiment of this technology. The column amplifier 400 samples the P-phase level (step S911), and the determination control circuit 310 determines whether the illuminance is high illuminance, which is higher than a predetermined value (step S912).

[0154] If the illumination is low (step S913: No), the column amplifier 400 amplifies the difference between the P-phase level and the D-phase level (step S914). On the other hand, if the illumination is high (step S913: Yes), the column amplifier 400 attenuates the difference between the P-phase level and the D-phase level (step S915). After step S914 or S915, the ADC261 performs AD conversion on the output signal of the column amplifier 400 (step S916), and the CDS processing is completed.

[0155] Figure 22 is a flowchart showing an example of CDS processing in the first embodiment of this technology. The column amplifier 400 samples the D-phase level (step S921). The determination control circuit 310 then determines whether the illuminance is high illuminance, which is higher than a predetermined value, and the ADC 261 performs AD conversion on the D-phase level (step S922).

[0156] If the illumination is low (step S923: No), the column amplifier 400 samples the P-phase level (step S924). On the other hand, if the illumination is high (step S923: Yes), the column amplifier 400 performs a weighted averaging process on the D-phase level and the P-phase level (step S925). After step S924 or S925, the ADC 261 performs AD conversion on the output signal of the column amplifier 400 (step S926), and the DDS process is completed.

[0157] Thus, according to the first embodiment of this technology, the column amplifier 400 amplifies or attenuates the pixel signal depending on whether the illumination is high or low, so even when CDS and DDS systems are mixed, improvements in image quality and readout speed can be achieved.

[0158] <2. Second Embodiment> In the first embodiment described above, the image sensor 200 read out two pixel signals with different FD gains using the CDS method and the DDS method to perform HDR synthesis, but the system is not limited to this configuration. The image sensor 200 in this second embodiment differs from the first embodiment in that it generates multiple pixel signals with different exposure periods and reads them out using the CDS method to perform HDR synthesis.

[0159] Figure 23 is a timing chart showing the driving method for pixel 250 and the variation of the judgment threshold Th in a second embodiment of the present technology. In the figure, a shows the driving method for pixel 250, and b shows the variation of the judgment threshold Th.

[0160] In the figure, the vertical axis of 'a' represents the row address, and the horizontal axis represents time. For example, from timing T21 onward, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. The solid diagonal lines in the figure indicate the exposure start timing for each row.

[0161] Then, from timing T22 onward, the vertical scanning circuit 210 drives the rows sequentially to complete the exposure and outputs the pixel signal. The dashed lines in the figure indicate the timing of the end of exposure and readout for each row. However, the FD gain is not switched during the readout of the pixel signal, and only the CDS method is used.

[0162] Then, from timing T23 onwards, the vertical scanning circuit 210 drives the rows sequentially and starts exposure. Then, from timing T24 onwards, the vertical scanning circuit 210 drives the rows sequentially to end exposure and outputs the pixel signal. When reading the pixel signal, the FD gain is not switched, and only the CDS method is used. Also, the length of the second exposure period is set to a different value from the first (for example, a shorter value than the first). The period from timing T21 to T22 corresponds to the longer exposure period EX_L for the first row, and the period from timing T23 to T24 corresponds to the shorter exposure period EX_S for the first row.

[0163] The image sensor 200 has two exposure periods of different lengths set, but it is also possible to set three or more exposure periods of different lengths. Furthermore, the pixel signal for exposure period EX_L is an example of the first pixel signal described in the claims, and the pixel signal for exposure period EX_S is an example of the second pixel signal described in the claims.

[0164] Furthermore, the period from timing T31 to T33 in the figure corresponds to the CDS period when reading out the pixel signal for exposure period EX_L in a predetermined row. The period from timing T34 to T36 corresponds to the CDS period when reading out the pixel signal for exposure period EX_S in a predetermined row.

[0165] The column amplifier 400 samples the P-phase level of the exposure period EX_L during the P-phase period from timing T31 to T32. At timing T32, the timing control unit 220 switches the judgment threshold Th from Th0 to Th1. At timing T33, the timing control unit 220 returns the judgment threshold Th to Th0.

[0166] The column amplifier 400 then samples the P-phase level of the exposure period EX_S during the P-phase period from timing T34 to T35. At timing T35, the timing control unit 220 switches the determination threshold Th from Th0 to Th2. This determination threshold Th2 is set to a higher value than Th1. The determination thresholds Th1 and Th2 are examples of the first and second determination thresholds described in the claims.

[0167] Furthermore, the ADC261 performs AD conversion on the output signal of the column amplifier 400, and the image processing unit 280 performs HDR synthesis on two digital signals with different exposure times.

[0168] When reading out two pixel signals using the CDS method and the DDS method, the column amplifier 400 cannot perform amplification in the DDS method. On the other hand, in the control examples a and b in the figure, only the CDS method is used, so the column amplifier 400 can amplify both of the two pixel signals to be combined.

[0169] Furthermore, switching the FD gain in three or more stages requires adding inter-wiring capacitance or transistors within the pixel 250. On the other hand, in the control exemplified in figures a and b, the image sensor 200 can set three or more exposure periods of different lengths without changing the circuit of the pixel 250. However, it should be noted that the frame rate will decrease compared to when the FD gain is switched.

[0170] The controls exemplified in figures a and b and the controls of the first embodiment can be switched depending on the situation.

[0171] Thus, according to the second embodiment of this technology, since two pixel signals with different exposure times are read out using the CDS method, the column amplifier 400 can amplify both of them. Furthermore, the image sensor 200 can set three or more exposure periods of different lengths without changing the circuit of the pixel 250.

[0172] <3. Examples of applications to mobile devices> The technology disclosed herein (the Technology) can be applied to a variety of products. For example, the Technology disclosed herein may be implemented as a device mounted on any type of mobile vehicle, such as an automobile, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility device, airplane, drone, ship, or robot.

[0173] Figure 24 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0174] The vehicle control system 12000 comprises multiple electronic control units connected via a communication network 12001. In the example shown in Figure 24, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is also shown, consisting of a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0175] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0176] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0177] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0178] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0179] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0180] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking system based on information from inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0181] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0182] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0183] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example in Figure 24, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0184] Figure 25 shows an example of the installation position of the imaging unit 12031.

[0185] In Figure 25, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0186] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0187] Figure 25 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0188] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0189] For example, the microcomputer 12051, based on distance information obtained from imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to vehicle 12100). In particular, it can extract the nearest object on the vehicle 12100's path that is traveling in approximately the same direction as vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0190] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, heavy vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0191] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0192] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to, for example, the imaging unit 12031 of the configuration described above. Specifically, the imaging device 100 in Figure 1 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, it is possible to improve image quality and readout speed, thereby improving the performance of the system.

[0193] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0194] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0195] Furthermore, this technology can also be configured as follows. (1) A column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal, An analog-to-digital converter that converts the output signal into a digital signal, A determination control circuit controls the column amplifier to perform sample-and-hold or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. An image sensor equipped with the following features. (2) The data recovery unit further comprises a process that multiplies the digital signal by the reciprocal of the amplification factor if the pixel signal has been amplified, and a process that multiplies the digital signal by the reciprocal of the attenuation factor if the pixel signal has been attenuated or weighted averaged. The image sensor described in (1) above. (3) The pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel. The aforementioned pixel outputs the reset level and the signal level in a predetermined order. The image sensor described in (1) or (2) above. (4) The pixel outputs the signal level after the reset level within a predetermined CDS period. During the CDS period, the determination control circuit performs a sample-and-hold operation of the reset level, amplifies the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and attenuates the difference if the illuminance is higher than the predetermined value. The image sensor described in (3) above. (5) The pixel outputs the reset level after the signal level within a predetermined DDS period, During the DDS period, the determination control circuit performs a sample-and-hold operation on the signal level. If the illuminance is lower than the predetermined value, it performs a sample-and-hold operation on the reset level. If the illuminance is higher than the predetermined value, it performs a weighted average operation on the reset level and the signal level. The image sensor described in (3) or (4) above. (6) The column amplifier is An operational amplifier that outputs the aforementioned output signal, The first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, A predetermined number of first switches that open and close the path between the other end of the first capacitive element and the input terminal of the column amplifier, A predetermined number of second switches that open and close the path between the other end of the second capacitive element and the input terminal of the column amplifier, A predetermined number of third switches that open and close the path between the other end of the first capacitive element and the output terminal of the column amplifier, A predetermined number of fourth switches that open and close the path between the other end of the first capacitive element and the output terminal, An image sensor according to any one of (3) to (5) above, comprising the above. (7) The determination control circuit is: A determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illuminance determination result, A control circuit that controls the first switch, second switch, third switch, and fourth switch based on the determination result, The image sensor described in (6) above, comprising the above. (8) The pixel outputs the signal level after the reset level within a predetermined CDS period, During the CDS period, the control circuit controls the first and second switches to be ON for a predetermined period if the reset level is output, controls the first and third switches to be ON for a predetermined period if the illuminance is lower than the predetermined value, and controls the second and fourth switches to be ON for a predetermined period if the illuminance is higher than the predetermined value. The image sensor described in (7) above. (9) The pixel outputs the reset level after the signal level within a predetermined DDS period, During the DDS period, the control circuit controls the first and second switches to be ON for a predetermined period if the reset level is output; if the illuminance is lower than the predetermined value, it controls the first and second switches to be ON for a predetermined period, and then controls the third and fourth switches to be ON for a predetermined period; and if the illuminance is higher than the predetermined value, it controls the second switch to be ON for a predetermined period, and then controls the third and fourth switches to be ON for a predetermined period. The image sensor described in (7) or (8) above. (10) The pixel signal includes first and second pixel signals with different exposure periods, The determination threshold includes a first determination threshold corresponding to the first pixel signal and a second determination threshold corresponding to the second pixel signal. An image sensor as described in any of (1) to (9) above. (11) A column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal, An analog-to-digital converter that converts the output signal into a digital signal, A determination control circuit controls the column amplifier to perform sample-and-hold or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. An image processing unit that processes an image in which the aforementioned digital signals are arranged. An imaging device equipped with the following. (12) Output procedure in which the column amplifier outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal, The analog-to-digital converter performs an analog-to-digital conversion procedure in which it converts the output signal into a digital signal, A determination control procedure is provided which, if the illuminance is lower than a predetermined value, the column amplifier is controlled to perform sample-and-hold or amplification of the pixel signal, and if the illuminance is higher than the predetermined value, the column amplifier is controlled to perform attenuation or weighted averaging of the pixel signal. A control method for an image sensor equipped with the following. [Explanation of symbols]

[0196] 100 Imaging device 110 Optics Department 120 DSP circuits 130 Display section 140 Operation section 150 bus 160 frames memory 170 Storage section 180 Power supply section 200 Image Sensors 201 Light-receiving chip 202 Circuit Chips 210 Vertical scanning circuit 220 Timing Control Unit 230 DAC 240-pixel array section 250 pixels 251 Photoelectric conversion unit 252 transfer transistors 253 Reset Transistor 254 Inter-wiring capacitance 255 2nd FD Department 256 connected transistors 257 1st FD Department 258 Amplifying transistors 259 Select Transistors 260-column signal processing unit 261 ADC 270 Horizontal scanning circuit 280 Image Processing Unit 310 Judgment control circuit 311 Judgment circuit 312, 313, 321, 322, 421, 422 capacitive elements 314, 323 Comparators 315, 316, 324, 325, 430 Auto Zero Switch 317 Control circuit 320 Comparison Section 330 counter 340 Data Recovery Unit 400 Column Amplifier 410 Attenuation / Amplification Section 411, 412, 413, 414 switches 440 Op-amps 12031 Imaging Unit

Claims

1. A column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal, An analog-to-digital converter that converts the output signal into a digital signal, A determination control circuit controls the column amplifier to perform sample-and-hold or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. An image sensor equipped with the following features.

2. The system further comprises a data recovery unit that, if amplification of the pixel signal is performed, multiplies the digital signal by the reciprocal of the amplification factor, and if attenuation or weighted averaging of the pixel signal is performed, multiplies the digital signal by the reciprocal of the attenuation factor. The image sensor according to claim 1.

3. The pixel signal includes a reset level when the pixel is initialized and a signal level when charge is transferred within the pixel. The aforementioned pixel outputs the reset level and the signal level in a predetermined order. The image sensor according to claim 1.

4. The aforementioned pixel outputs the signal level after the reset level within a predetermined CDS (Correlated Double Sampling) period. During the CDS period, the determination control circuit performs a sample-and-hold operation of the reset level, amplifies the difference between the reset level and the signal level if the illuminance is lower than the predetermined value, and attenuates the difference if the illuminance is higher than the predetermined value. The image sensor according to claim 3.

5. The aforementioned pixel outputs the reset level after the signal level within a predetermined DDS (Double Data Sampling) period. During the DDS period, the determination control circuit performs a sample-and-hold operation on the signal level. If the illuminance is lower than the predetermined value, it performs a sample-and-hold operation on the reset level. If the illuminance is higher than the predetermined value, it performs a weighted average operation on the reset level and the signal level. The image sensor according to claim 3.

6. The column amplifier described above is An operational amplifier that outputs the aforementioned output signal, The first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, A predetermined number of first switches that open and close the path between the other end of the first capacitive element and the input terminal of the column amplifier, A predetermined number of second switches that open and close the path between the other end of the second capacitive element and the input terminal of the column amplifier, A predetermined number of third switches that open and close the path between the other end of the first capacitive element and the output terminal of the column amplifier, A predetermined number of fourth switches that open and close the path between the other end of the first capacitive element and the output terminal, The image sensor according to claim 3, comprising:

7. The aforementioned determination control circuit is A determination circuit that compares the pixel signal with a predetermined determination threshold and outputs the comparison result as an illuminance determination result, A control circuit that controls the first switch, second switch, third switch, and fourth switch based on the determination result. The image sensor according to claim 6, comprising:

8. The pixel outputs the signal level after the reset level within a predetermined CDS period. During the CDS period, the control circuit controls the first and second switches to be ON for a predetermined period if the reset level is output, controls the first and third switches to be ON for a predetermined period if the illuminance is lower than the predetermined value, and controls the second and fourth switches to be ON for a predetermined period if the illuminance is higher than the predetermined value. The image sensor according to claim 7.

9. The aforementioned pixel outputs the reset level after the signal level within a predetermined DDS period. During the DDS period, the control circuit controls the first and second switches to be ON for a predetermined period if the reset level is output; if the illuminance is lower than the predetermined value, it controls the first and second switches to be ON for a predetermined period, and then controls the third and fourth switches to be ON for a predetermined period; and if the illuminance is higher than the predetermined value, it controls the second switch to be ON for a predetermined period, and then controls the third and fourth switches to be ON for a predetermined period. The image sensor according to claim 7.

10. The aforementioned pixel signal includes first and second pixel signals with different exposure period lengths. The determination threshold includes a first determination threshold corresponding to the first pixel signal and a second determination threshold corresponding to the second pixel signal. The image sensor according to claim 1.

11. A column amplifier that outputs an output signal by performing one of the following: sampling and holding of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal, An analog-to-digital converter that converts the output signal into a digital signal, A determination control circuit controls the column amplifier to perform sample-and-hold or amplification of the pixel signal when the illuminance is lower than a predetermined value, and controls the column amplifier to perform attenuation or weighted averaging of the pixel signal when the illuminance is higher than the predetermined value. An image processing unit that processes an image in which the aforementioned digital signals are arranged. An imaging device equipped with the following.

12. The column amplifier performs one of the following actions to output an output signal: sample and hold of the pixel signal from the pixel, amplification of the pixel signal, attenuation of the pixel signal, or weighted averaging of the pixel signal. The analog-to-digital converter performs an analog-to-digital conversion procedure in which it converts the output signal into a digital signal, A determination control procedure is provided which, if the illuminance is lower than a predetermined value, the column amplifier is controlled to perform sample-and-hold or amplification of the pixel signal, and if the illuminance is higher than the predetermined value, the column amplifier is controlled to perform attenuation or weighted averaging of the pixel signal. A control method for an image sensor equipped with the following.

Images