Image sensor, imaging device, and control method for image sensor

Abstract

This improves the readout speed in column ADC (Analog to Digital Converter) type image sensors. [Solution] The first sample-and-hold circuit samples and holds one of the following levels: the reset level when the pixel is initialized and the signal level when charge is transferred within the pixel, and outputs it. The second sample-and-hold circuit samples and holds the other of the reset level and signal level when the illuminance is lower than a predetermined value, and outputs a weighted average of the two output levels when the illuminance is higher than a predetermined value.

Description

[Technical Field]

[0001] This technology relates to image sensors. More specifically, it describes an image sensor with an ADC (Analog to Digital Converter) arranged in each column, an imaging device, and a control method for the image sensor. [Background technology]

[0002] Conventionally, column ADC systems, in which an ADC is placed in each column, have been widely used in image sensors. For example, an image sensor has been proposed that employs a column ADC system and has an attenuation unit placed before the comparator in the ADC to attenuate the signal at high illumination levels (see, for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2019 / 239670 [Overview of the project] [Problems that the invention aims to solve]

[0004] The conventional technology described above attempts to increase read speed by attenuating the signal at high illumination levels. However, further improving the read speed remains difficult.

[0005] This technology was developed in light of these circumstances and aims to improve the readout speed in column ADC type image sensors. [Means for solving the problem]

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor and control method comprising: a first sample-and-hold circuit that samples and holds and outputs one of the levels of either the reset level when a pixel is initialized or the signal level when charge is transferred within the pixel; and a second sample-and-hold circuit that samples and holds and outputs the other of the reset level and signal level when the illuminance is lower than a predetermined value, and outputs a weighted average of the output levels when the illuminance is higher than the predetermined value. This results in an improved readout speed.

[0007] Furthermore, in this first aspect, the pixel may output the signal level after the reset level within a predetermined CDS (Correlated Double Sampling) period, the first sample-and-hold circuit may sample and hold the reset level and output it, and the second sample-and-hold circuit may sample and hold the signal level and output it if the illuminance is lower than a predetermined value, and output a weighted average of the output reset level and the signal level if the illuminance is higher than the predetermined value. This results in an improved read speed when the reset level is read first.

[0008] Furthermore, in this first aspect, the pixel may output the reset level after the signal level within a predetermined DDS (Double Data Sampling) period, the first sample-and-hold circuit may sample and hold the signal level and output it, and the second sample-and-hold circuit may sample and hold the reset level and output it if the illuminance is lower than a predetermined value, and output a weighted average of the output signal level and the reset level if the illuminance is higher than the predetermined value. This results in an improved readout speed when the signal level is read out first.

[0009] Furthermore, in this first aspect, a selector may be provided that selects one of the output signals of the first sample-and-hold circuit or the second sample-and-hold circuit and outputs it to the ADC. This results in the output signal of the other being read out while one of the first and second sample-and-hold circuits is sampling.

[0010] Furthermore, in this first aspect, each of the first and second sample-and-hold circuits may include an operational amplifier that outputs the output signal to the selector, first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, a first switch that opens and closes the path between the other end of the first capacitive element and the vertical signal line connected to the pixel, a second switch that opens and closes the path between the other end of the second capacitive element and the vertical signal line, a third switch that opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier, and a fourth switch that opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier. This results in the implementation of sample-and-hold and weighted averaging.

[0011] Furthermore, in this first aspect, the second sample-and-hold circuit may further include a fifth switch that opens and closes the path between the output terminal of the first sample-and-hold circuit and the other end of the second capacitive element. This results in the output signal of the first sample-and-hold circuit being sampled by the second sample-and-hold circuit.

[0012] Furthermore, in this first aspect, the system may further include a determination control circuit that determines whether the illuminance is higher than the predetermined value and controls the first and second sample-and-hold circuits based on the determination result. This results in the sample-and-hold circuits being controlled according to the illuminance.

[0013] In addition, a second aspect of the present technology is an imaging device including: a first sample-and-hold circuit that sample-holds and outputs one of a reset level when a pixel is initialized and a signal level when charge is transferred within the pixel; a second sample-and-hold circuit that sample-holds and outputs the other of the reset level and the signal level when the illuminance is lower than a predetermined value, and outputs a weighted average of the one output level and the other level when the illuminance is higher than the predetermined value; and an analog-to-digital converter that converts the output signals of the first and second sample-and-hold circuits into digital signals. This brings about the effect of improving the readout speed of the image sensor in the imaging device.

Brief Description of the Drawings

[0014] [Figure 1] It is a block diagram showing a configuration example of an imaging device in a first embodiment of the present technology. [Figure 2] It is a diagram showing an example of the stacked structure of an image sensor in a first embodiment of the present technology. [Figure 3] It is a block diagram showing a configuration example of an image sensor in a first embodiment of the present technology. [Figure 4] It is a circuit diagram showing a configuration example of a pixel in a first embodiment of the present technology. [Figure 5] It is a block diagram showing a configuration example of a column signal processing unit in a first embodiment of the present technology. [Figure 6] It is a circuit diagram showing a configuration example of a column amplifier in a first embodiment of the present technology. [Figure 7] It is a circuit diagram showing a configuration example of a sample-and-hold circuit when the capacitance value is variable in a first embodiment of the present technology. [Figure 8] It is a circuit diagram showing a configuration example of a sample-and-hold circuit when the capacitance value is variable in a first embodiment of the present technology. [Figure 9] It is a circuit diagram showing a configuration example of a comparison unit in a first embodiment of the present technology. [Figure 10] This is a circuit diagram showing one example of the configuration of a decision control circuit in the first embodiment of this technology. [Figure 11] This is a timing chart showing an example of the variation in pixel signals and judgment thresholds in the first embodiment of this technology. [Figure 12] This figure shows an example of the state of the column amplifier when sampling the P phase in the first embodiment of this technology. [Figure 13] This figure shows an example of the state of the column amplifier when sampling the D phase while converting the P phase at low light levels in the first embodiment of this technology. [Figure 14] This figure shows an example of the state of the column amplifier when the D phase is converted and the next P phase is sampled at low light levels in the first embodiment of this technology. [Figure 15] This figure shows an example of the state of the column amplifier when sampling the P-phase and D-phase while converting the P-phase at high illuminance in the first embodiment of this technology. [Figure 16] This figure shows an example of the state of the column amplifier when sampling the next P-phase while converting the weighted average at high illuminance in the first embodiment of this technology. [Figure 17] This figure illustrates the effect of pixel signal attenuation in the first embodiment of this technology. [Figure 18] This figure shows an example of the input / output characteristics of the column amplifier and ADC in the first embodiment of this technology. [Figure 19] This is a flowchart illustrating an example of the operation of the image sensor in the first embodiment of this technology. [Figure 20] This is a timing chart showing an example of the variation in pixel signals and judgment thresholds in the second embodiment of this technology. [Figure 21] This figure shows an example of the state of the column amplifier when sampling the D phase in the first embodiment of this technology. [Figure 22]This figure shows an example of the state of the column amplifier when sampling the P phase while converting the D phase at low light levels in the first embodiment of this technology. [Figure 23] This figure shows an example of the state of the column amplifier when the P phase is converted and the next D phase is sampled at low light levels in the first embodiment of this technology. [Figure 24] This figure shows an example of the state of the column amplifier when sampling the P phase and D phase while converting the D phase at high illuminance in the first embodiment of this technology. [Figure 25] This figure shows an example of the state of the column amplifier when sampling the next D phase while converting the weighted average at high illuminance in the first embodiment of this technology. [Figure 26] This block diagram shows an example of a schematic configuration of a vehicle control system. [Figure 27] This is an explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit. [Modes for carrying out the invention]

[0015] The following describes the embodiments for implementing this technology. The description will proceed in the following order. 1. First Embodiment (An example in which one of two sample-and-hold circuits outputs to the other) 2. Second Embodiment (An example in which one of two sample-and-hold circuits outputs to the other in a DDS system) 3. Examples of applications to mobile devices

[0016] <1. First Embodiment> [Example configuration of an imaging device] Figure 1 is a block diagram showing an example configuration of an imaging device 100 in the first embodiment of this technology. This imaging device 100 is a device for capturing image data and comprises an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. Furthermore, the imaging device 100 comprises a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. Examples of imaging devices 100 include cameras mounted on smartphones and in-vehicle cameras.

[0017] The optical unit 110 collects light from the subject and directs it to the image sensor 200. The image sensor 200 generates image data through photoelectric conversion. The image sensor 200 supplies the generated image data to the DSP circuit 120 via the signal line 209.

[0018] The DSP circuit 120 performs predetermined signal processing on the image data. The DSP circuit 120 outputs the processed image data to a frame memory 160 or the like via the bus 150.

[0019] The display unit 130 displays image data. The display unit 130 could be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The operation unit 140 generates operation signals according to user input.

[0020] Bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with each other.

[0021] The frame memory 160 stores image data. The storage unit 170 stores various data, including image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and other components.

[0022] Figure 2 shows an example of a stacked structure of an image sensor 200 in the first embodiment of this technology. This image sensor 200 comprises a circuit chip 202 and a light-receiving chip 201 stacked on the circuit chip 202. These chips are electrically connected via connection points such as vias. In addition to vias, connections can also be made by Cu-Cu junctions or bumps.

[0023] [Example of image sensor configuration] Figure 3 is a block diagram showing an example configuration of an image sensor 200 in a first embodiment of this technology. This image sensor 200 includes a vertical scanning circuit 210, a timing control unit 220, a DAC (Digital to Analog Converter) 230, a pixel array unit 240, a column signal processing unit 260, a horizontal scanning circuit 270, and an image processing unit 280. Multiple pixels 250 are arranged in a two-dimensional grid in the pixel array unit 240.

[0024] The pixel array section 240 is located on, for example, the light-receiving chip 201, and the remaining circuits are located on the circuit chip 202. Note that the circuits located on each chip are not limited to those illustrated in the figure.

[0025] The vertical scanning circuit 210 sequentially selects and drives rows within the pixel array section 240.

[0026] The timing control unit 220 controls the operating timing of the vertical scanning circuit 210, DAC 230, column signal processing unit 260, and horizontal scanning circuit 270 in synchronization with the vertical synchronization signal VSYNC.

[0027] The DAC230 generates a sawtooth-shaped ramp signal and supplies it to the column signal processing unit 260 as a reference signal.

[0028] Pixel 250 generates an analog pixel signal by photoelectric conversion in accordance with the control of the vertical scanning circuit 210. Each pixel 250 in a column outputs a pixel signal to the column signal processing unit 260 via a vertical signal line.

[0029] The column signal processing unit 260 has a column amplifier and an ADC (not shown) for each column of pixels 250. Each ADC converts the pixel signal of the corresponding column into a digital signal and outputs it to the image processing unit 280 according to the control of the horizontal scanning circuit 270.

[0030] The horizontal scanning circuit 270 controls the column signal processing unit 260 to output digital signals sequentially.

[0031] The image processing unit 280 performs various image processing operations on image data, which is an array of digital signals. The image processing unit 280 supplies the processed image data to the DSP circuit 120 via the signal line 209.

[0032] [Example of pixel configuration] Figure 4 is a circuit diagram showing one example configuration of a pixel 250 in the first embodiment of this technology. This pixel 250 comprises a photoelectric conversion unit 251, a transfer transistor 252, a reset transistor 253, an FD (Floating Diffusion) 254, an amplification transistor 255, and a selection transistor 256.

[0033] The photoelectric conversion unit 251 has its anode terminal grounded and its cathode terminal connected to the FD254 via the transfer transistor 252. The photoelectric conversion unit 251 receives light irradiated onto the light-receiving surface of the image sensor 200 and converts it into an electric charge corresponding to the amount of light.

[0034] The transfer transistor 252 is positioned to connect the photoelectric conversion unit 251 and the FD254. The transfer transistor 252 is driven according to the transfer signal TG from the vertical scanning circuit 210 and transfers the charge photoelectrically converted in the photoelectric conversion unit 251 to the FD254.

[0035] FD254 stores the charge transferred from the photoelectric conversion unit 251 via the transfer transistor 252 in order to convert it into a pixel signal.

[0036] The reset transistor 253 is positioned to connect FD254 to the power supply wiring VDD. The reset transistor 253 is driven according to the reset signal RST from the vertical scanning circuit 210. When the reset transistor 253 is turned on, the charge accumulated in FD254 is discharged to the power supply wiring VDD, and the charge is reset.

[0037] The amplifying transistor 255 is positioned so that FD254 is connected to its gate electrode, and the power supply wiring VDD is connected to the selection transistor 256. The amplifying transistor 255 then converts the charge stored in FD254 into a pixel signal with a level corresponding to the charge relative to its capacitance.

[0038] The selection transistor 256 is positioned to connect the amplification transistor 255 and the vertical signal line 249. The selection transistor 256 is driven according to the selection signal SEL from the vertical scanning circuit 210, and while the selection transistor 256 is ON, the pixel signal SIG from the amplification transistor 255 is output to the vertical signal line 249.

[0039] The level of the pixel signal SIG when the reset transistor 253 initializes the pixel 250 is referred to as the "reset level" or "P-phase level." The level of the pixel signal SIG when the transfer transistor 252 transfers charge within the pixel is referred to as the "signal level" or "D-phase level."

[0040] The circuit configuration for pixel 250 is not limited to the example shown in the figure, as long as it can generate pixel signals by photoelectric conversion. For example, an additional transistor for switching the FD gain can be added.

[0041] [Example of a column signal processing unit configuration] Figure 5 is a block diagram showing an example configuration of the column signal processing unit 260 in the first embodiment of this technology. This column signal processing unit 260 includes a determination control circuit 310, a column amplifier 400, an ADC 261, and a data recovery unit 340 for each column.

[0042] The column amplifier 400 performs sample-and-hold and weighted averaging on the pixel signals SIG from the corresponding column. The column amplifier 400 outputs the analog pixel signals, after performing sample-and-hold and other operations, as an output signal to the corresponding ADC261.

[0043] The determination control circuit 310 determines whether the illuminance is high or low (high illuminance) based on the pixel signal SIG of the corresponding column, and controls the column amplifier 400 based on the determination result CMP. If the illuminance is low (low illuminance) below the predetermined value, the determination control circuit 310 controls the column amplifier 400 to perform sample-and-hold on the pixel signal SIG. On the other hand, if the illuminance is high, the determination control circuit 310 controls the column amplifier 400 to perform weighted averaging on the pixel signal SIG. The determination control circuit 310 also supplies the determination result CMP to the data recovery unit 340.

[0044] The ADC261 performs analog-to-digital (AD) conversion of the output signal of the column amplifier 400 into a digital signal CNT. This ADC261 is a single-slope type ADC equipped with a comparator 320 and a counter 330.

[0045] The comparison unit 320 compares the reference signal RMP from the DAC230 with the output signal of the column amplifier 400 and outputs the comparison result VCO to the counter 330.

[0046] The counter 330 counts the count value over the period until the comparison result VCO inverts. This counter 330 supplies a digital signal CNT indicating the count value to the data recovery unit 340. The value of this digital signal CNT will be different from the value when the column amplifier 400 is not performed due to the attenuation of the pixel signal.

[0047] The data recovery unit 340 recovers the value of the digital signal CNT to the value obtained when weighted averaging processing is not performed, based on the judgment result CMP. If the column amplifier 400 has performed weighted averaging processing, the data recovery unit 340 recovers the value by multiplying the digital signal CNT by the reciprocal of the attenuation rate. The data recovery unit 340 outputs the recovered digital signal as Dout to the image processing unit 280.

[0048] [Example of a column amplifier configuration] Figure 6 is a circuit diagram showing one example configuration of a column amplifier 400 in the first embodiment of this technology. This column amplifier 400 includes sample-and-hold circuits 410 and 450 and a selector 490.

[0049] Sample-and-hold circuits 410 and 450 perform sample-and-hold and weighted averaging of pixel signals. Sample-and-hold circuit 410 includes switches 411, 412, 413, 414, and 415, capacitive elements 421 and 422, an auto-zero switch 430, and an operational amplifier 440. Sample-and-hold circuit 450 includes switches 451, 452, 453, 454, and 455, capacitive elements 461 and 462, an auto-zero switch 470, and an operational amplifier 480.

[0050] The sample-and-hold circuits 410 and 450 are examples of the first and second sample-and-hold circuits described in the claims.

[0051] In the sample-and-hold circuit 410, one end each of the capacitive elements 421 and 422 is connected to the inverting input terminal (-) of the operational amplifier 440.

[0052] Switch 411 opens and closes the path between the vertical signal line 249 and the other end of the capacitive element 421 in accordance with the control signal S11 from the determination control circuit 310. Switch 412 opens and closes the path between the vertical signal line 249 and the other end of the capacitive element 422 in accordance with the control signal S12 from the determination control circuit 310. Note that switches 411 and 412 are examples of the first and second switches described in the claims.

[0053] Switch 413 opens and closes the path between the other end of the capacitive element 421 and the output terminal of the operational amplifier 440 in accordance with the control signal CL11 from the determination control circuit 310. Switch 414 opens and closes the path between the other end of the capacitive element 422 and the output terminal of the operational amplifier 440 in accordance with the control signal CL12 from the determination control circuit 310. Note that switches 413 and 414 are examples of the third and fourth switches described in the claims.

[0054] Switch 415 opens and closes the path between the output terminal of the operational amplifier 480 (in other words, the output terminal of the sample-and-hold circuit 450) and the other end of the capacitive element 422, according to the control signal S_OUT2 from the decision control circuit 310.

[0055] The auto-zero switch 430 opens and closes the path between the inverting input terminal (-) and output terminal of the operational amplifier 440 according to the auto-zero signal AZ1 from the timing control unit 220.

[0056] The non-inverting input terminal (+) of the operational amplifier 440 is connected to ground potential, and its output terminal is connected to the selector 490 and the sample-and-hold circuit 450. The operational amplifier 440 outputs the output signal SHOUT1 to the selector 490 and the sample-and-hold circuit 450.

[0057] Next, in the sample-and-hold circuit 450, one end each of the capacitive elements 461 and 462 is connected to the inverting input terminal (-) of the operational amplifier 480.

[0058] Switch 451 opens and closes the path between the vertical signal line 249 and the other end of the capacitive element 461 according to the control signal S21 from the determination control circuit 310. Switch 452 opens and closes the path between the vertical signal line 249 and the other end of the capacitive element 462 according to the control signal S22 from the determination control circuit 310.

[0059] Switch 453 opens and closes the path between the other end of the capacitive element 461 and the output terminal of the operational amplifier 480 according to the control signal CL21 from the determination control circuit 310. Switch 454 opens and closes the path between the other end of the capacitive element 462 and the output terminal of the operational amplifier 480 according to the control signal CL22 from the determination control circuit 310.

[0060] Switch 455 opens and closes the path between the output terminal of the operational amplifier 440 (in other words, the output terminal of the sample-and-hold circuit 410) and the other end of the capacitive element 462, according to the control signal S_OUT1 from the determination control circuit 310. Note that switch 455 is an example of the fifth switch described in the claims.

[0061] The auto-zero switch 470 opens and closes the path between the inverting input terminal (-) and output terminal of the operational amplifier 480 according to the auto-zero signal AZ2 from the timing control unit 220.

[0062] The non-inverting input terminal (+) of the operational amplifier 480 is connected to ground potential, and its output terminal is connected to the selector 490 and the sample-and-hold circuit 410. The operational amplifier 480 outputs the output signal SHOUT2 to the selector 490 and the sample-and-hold circuit 410.

[0063] The selector 490 selects one of the output signals SHOUT1 and SHOUT2 according to the selection signal SEL_SH from the timing control unit 220 and outputs it as SHOUT to the comparison unit 320.

[0064] With the circuit configuration shown in the figure, the sample-and-hold circuit 410 samples and holds either the reset level or the signal level and outputs it to the selector 490 and the sample-and-hold circuit 450.

[0065] Furthermore, if the illuminance is lower than a predetermined value, the sample-and-hold circuit 450 samples and holds the other level of the reset level and signal level and outputs it to the selector 490. On the other hand, if the illuminance is higher than a predetermined value, the sample-and-hold circuit 450 samples the level output by the sample-and-hold circuit 410 and the other level, and outputs their weighted average.

[0066] In the first embodiment, each time the vertical scanning circuit 210 drives a row, the pixels within that row generate a reset level, and then a signal level. In this case, the sample-and-hold circuit 410 samples and holds the reset level and outputs it to the selector 490 and the sample-and-hold circuit 450.

[0067] Furthermore, the sample-and-hold circuit 450 samples and holds the signal level when the illuminance is lower than a predetermined value and outputs it to the selector 490. On the other hand, when the illuminance is higher than a predetermined value, the sample-and-hold circuit 450 samples the reset level output by the sample-and-hold circuit 410 and the signal level, and outputs their weighted average.

[0068] Furthermore, the capacitance values ​​of the capacitive elements 421 and 422 in the sample-and-hold circuit 410 are assumed to be fixed values. However, these capacitance values ​​are not limited to fixed values ​​and can be made variable.

[0069] Figure 7 is a circuit diagram showing one example configuration of a sample-and-hold circuit 410 when the capacitance value is variable in the first embodiment of this technology. When the capacitance value is variable, M (where M is an integer) switches 411, 413, and capacitance elements 421 are each arranged in quantities. Also, N (where N is an integer) switches 412, 414, 415, and capacitance elements 422 are each arranged in quantities. These switches and capacitance elements are connected in parallel.

[0070] The determination control circuit 310, using the control signal S11m, turns on or off at least some of the M switches 411 and turns off the remaining switches 411. When the m (where m is an integer from 1 to M) switch 411 is turned on or off, the control signal CL11m also turns on or off the mth switch 413.

[0071] The decision control circuit 310, using the control signal S12n, turns on or off at least some of the N switches 412 and turns off the remaining switches 412. When the nth (where n is an integer from 1 to N)th switch 412 is turned on or off, the control signals CL12n and S_OUT2n also turn on or off the nth switches 414 and 415.

[0072] Furthermore, the capacitance values ​​of the capacitive elements 461 and 462 in the sample-and-hold circuit 450 are assumed to be fixed values. However, these capacitance values ​​are not limited to fixed values ​​and can be made variable.

[0073] Figure 8 is a circuit diagram showing one example configuration of a sample-and-hold circuit 450 when the capacitance value is variable in the first embodiment of this technology. When the capacitance value is variable, M units each of switches 451, 453, and capacitance elements 461 are arranged. Also, N units each of switches 452, 454, 455, and capacitance elements 462 are arranged. These switches and capacitance elements are connected in parallel.

[0074] [Example of the configuration of the comparison section] Figure 9 is a circuit diagram showing one example configuration of the comparison unit 320 in the first embodiment of this technology. This comparison unit 320 comprises capacitive elements 321 and 322, a comparator 323, and auto-zero switches 324 and 325.

[0075] The output signal SHOUT from the column amplifier 400 is input to the inverting input terminal (-) of comparator 323 via capacitive element 321. The reference signal RMP from DAC230 is input to the non-inverting input terminal (+) of comparator 323 via capacitive element 322. Comparator 323 compares these signals and outputs the comparison result VCO to counter 330.

[0076] The auto-zero switch 324 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0077] The auto-zero switch 325 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 323 according to the auto-zero signal AZ_c from the timing control unit 220.

[0078] [Example of a judgment control circuit configuration] Figure 10 is a circuit diagram showing one example configuration of a determination control circuit 310 in the first embodiment of this technology. This determination control circuit 310 comprises a determination circuit 311 and a control circuit 317.

[0079] The determination circuit 311 determines whether the illuminance is higher than a predetermined value based on the pixel signal SIG of the corresponding column. This determination circuit 311 comprises capacitive elements 312 and 313, a comparator 314, and auto-zero switches 315 and 316.

[0080] The inverting input terminal (-) of comparator 314 is connected to the vertical signal line 249 via capacitive element 312, and the pixel signal SIG is input to this terminal. The non-inverting input terminal (+) of comparator 314 is input to the judgment threshold Th from the timing control unit 220 via capacitive element 313. Comparator 314 compares the level of the pixel signal SIG with the judgment threshold Th and outputs the comparison result as the illuminance judgment result CMP to the control circuit 317 and the data recovery unit 340.

[0081] The auto-zero switch 315 opens and closes the path between the inverting input terminal (-) and output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0082] The auto-zero switch 316 opens and closes the path between the non-inverting input terminal (+) and the output terminal of the comparator 314 according to the auto-zero signal AZcmp from the timing control unit 220.

[0083] The control circuit 317 controls the column amplifier 400 using control signals S11, S12, CL11, CL12, S21, S22, CL21, and CL22, based on the determination result CMP and the control signal Ctrl from the timing control unit 220. The control signal Ctrl is a signal that indicates the timing for controlling the switch.

[0084] Figure 11 is a timing chart showing an example of the variation of the pixel signal SIG and the judgment threshold Th in the first embodiment of this technology. In the figure, a shows the variation of the pixel signal SIG and the judgment threshold Th in the case of low illumination. In the figure, b shows the variation of the pixel signal SIG and the judgment threshold Th in the case of high illumination.

[0085] During the CDS period from timing T1 to T4, sampling of one row is performed using the CDS method. During this CDS period, each pixel in the selected row generates a signal level after the reset level.

[0086] As illustrated in figures a and b, the timing control unit 220 sets the determination threshold Th to Th0 over the P-phase period from timing T1 to T2. This Th0 is set to a value slightly lower than the P-phase level, for example.

[0087] Then, at the timing T2 in which the charge is transferred, the timing control unit 220 sets the determination threshold Th to Th1, which is lower than Th0.

[0088] Furthermore, as illustrated in figure a, when the illumination is low, the pixel signal SIG is higher than the judgment threshold in all periods. Therefore, the judgment result of the judgment control circuit 310 is not inverted.

[0089] On the other hand, as illustrated in figure b, when the illuminance is high, the level of the pixel signal SIG (in this case, the D-phase level) becomes lower than the judgment threshold Th1 during the transfer period from timing T2 to T3, and the judgment result of the judgment control circuit 310 is reversed.

[0090] As illustrated in figures a and b, the determination control circuit 310 can determine whether the illuminance is higher than a predetermined value based on the determination threshold Th. The difference between the determination thresholds Th0 and Th1, Th_cds, corresponds to a relative threshold that is compared with the difference between the P-phase level and the D-phase level during the CDS period.

[0091] [Example of image sensor operation] Next, referring to Figures 12 to 16, we will explain the control of the column amplifier 400 from the sampling of one row until the AD conversion of that row is completed.

[0092] As illustrated in Figure 12, during the P-phase period, the timing control unit 220 turns on the auto-zero switch 430 for the duration of the pulse, causing the sample-and-hold circuit 410 to perform the auto-zero operation. The timing control unit 220 also turns on switches 411 and 412, causing the sample-and-hold circuit 410 to sample the reset level (i.e., the P-phase level) Vp.

[0093] Furthermore, the timing control unit 220 turns on switches 453 and 454, causing the sample-and-hold circuit 450 to hold the signal from the previous row. The selector 490 selects the output signal of the sample-and-hold circuit 450, and that signal is then converted using AD conversion.

[0094] Then, the charge is transferred, and within that transfer period, the determination control circuit 310 determines the illuminance. In Figures 13 and 14, it is assumed that the illuminance is determined to be low, below a predetermined value.

[0095] As illustrated in Figure 13, in the case of low light, the timing control unit 220 turns on switches 413 and 414 during the D phase period, causing the sample-and-hold circuit 410 to hold the P phase level Vp. The selector 490 selects the output signal of the sample-and-hold circuit 410 (i.e., the P phase level), and this P phase level Vp is converted using AD conversion. During this AD conversion, the counter 330 performs a down count.

[0096] Furthermore, during the D-phase period, the timing control unit 220 turns on the auto-zero switch 470 for the duration of the pulse, causing the sample-and-hold circuit 450 to perform an auto-zero operation. The timing control unit 220 also turns on switches 451 and 452, causing the sample-and-hold circuit 450 to sample the signal level (i.e., the D-phase level) Vd.

[0097] Then, as illustrated in Figure 14, during the P-phase period of the next row, the timing control unit 220 turns on the auto-zero switch 430 for the duration of the pulse, causing the sample-and-hold circuit 410 to perform the auto-zero operation. The timing control unit 220 also turns on switches 411 and 412, causing the sample-and-hold circuit 410 to sample the P-phase level of the next row.

[0098] The timing control unit 220 turns on switches 453 and 454 during the P-phase period, causing the sample-and-hold circuit 450 to hold the D-phase level Vd. The selector 490 selects the output signal of the sample-and-hold circuit 450 (i.e., the D-phase level), and this D-phase level Vd is converted using AD conversion. During this AD conversion, the counter 330 performs an up-count.

[0099] Since counter 330 performs down-counting when converting to the P-phase level Vp and up-counting when converting to the D-phase level Vd, its count value becomes the difference between Vp and Vd, enabling digital CDS processing.

[0100] Furthermore, the counter 330 can be configured to perform only increments. In this case, a calculation circuit that performs digital CDS processing is inserted between the counter 330 and the data recovery unit 340.

[0101] Next, in Figures 15 and 16, it is assumed that the illuminance is determined to be high, higher than a predetermined value.

[0102] As illustrated in Figure 15, in the case of high illumination, the timing control unit 220 turns on switches 413 and 414 during the D-phase period, causing the sample-and-hold circuit 410 to hold the P-phase level Vp. The selector 490 selects the output signal of the sample-and-hold circuit 410 (i.e., the P-phase level), and this P-phase level Vp is converted using AD conversion. During this AD conversion, the counter 330 performs a down-count.

[0103] Furthermore, during the D-phase period, the timing control unit 220 turns on the auto-zero switch 470 for the duration of the pulse, causing the sample-and-hold circuit 450 to perform an auto-zero operation. The timing control unit 220 also turns on switches 455 and 451, causing the sample-and-hold circuit 450 to sample the P-phase level Vp and D-phase level Vd output from the sample-and-hold circuit 410. The amount of charge Q held by the capacitive elements 461 and 462 at this time is expressed by the following equation. Q=C21 ×(Vd - Vm) + C 22 ×(Vp - Vm) ··· Equation 1 In the above equation, C 21 represents the capacitance value of the capacitor element 461, and C 22 represents the capacitance value of the capacitor element 462. Vm represents the level of the connection node of the capacitor elements 461 and 462.

[0104] Then, as illustrated in FIG. 16, during the P-phase period of the next row, the timing control unit 220 turns on the auto-zero switch 430 over the pulse period to cause the sample-and-hold circuit 410 to perform an auto-zero operation. Also, the timing control unit 220 turns on the switches 411 and 412 to cause the sample-and-hold circuit 410 to sample the P-phase level of the next row.

[0105] Also, during the P-phase period, the timing control unit 220 turns on the switches 453 and 454 to cause the sample-and-hold circuit 450 to hold a signal. The amount of charge Q held in the capacitor elements 461 and 462 at this time is represented by the following equation. Q = (SHOUT2 - Vm)×(C 21 + C 22 ) ··· Equation 2

[0106] From Equation 1 and Equation 2, the following equation is obtained. SHOUT2 = (VpC 22 + VdC 21 ) / (C 21 + C 22 )... Equation 3

[0107] From Equation 3, the output signal SHOUT2 of the sample-and-hold circuit 450 becomes a weighted average of Vp and Vd. This weighted average is AD-converted. During this AD conversion, the counter 330 performs an up-count.

[0108] Since the counter 330 performs a down-count when converting the P-phase level Vp and an up-count when converting the weighted average, its count value becomes the difference between Vp and the weighted average and is represented by the following equation. (VpC 22 +VdC 21 ) / (C 21 +C 22 )-Vp =C 21 (Vd-Vp) / (C 21 +C 22 )...Equation 4

[0109] From Equation 4, the difference in the digital signal is the attenuated net pixel signal, which is the difference between Vp and Vd. This attenuation shortens the AD conversion time and reduces power consumption.

[0110] Furthermore, as illustrated in Figures 12 to 16, while the sample-and-hold circuit 410 is sampling the P-phase level, the sample-and-hold circuit 450 holds the signal (weighted average or D-phase level), and that signal is then converted using AD conversion. By performing sampling and AD conversion in parallel using a pipeline method in this way, the readout speed can be improved.

[0111] In the control systems illustrated in Figures 12 to 16, the sample-and-hold circuit 410 samples the P-phase level during the P-phase period. However, the sample-and-hold circuit 450 can also sample the P-phase level. In this case, the switch 415 is controlled to the ON state when the illumination is high, and the sample-and-hold circuit 410 samples the P-phase level and D-phase level output from the sample-and-hold circuit 450 when the illumination is high.

[0112] Figure 17 illustrates the effect of pixel signal attenuation in the first embodiment of this technology. It is assumed that the illuminance is determined to be high during the judgment period from timing T2 to T3'. In this case, the column amplifier 400 outputs a weighted average value Vd' during the D-phase period from timing T3' onward. This attenuates the difference between the P-phase level and the D-phase level. The dashed line in the figure shows the trajectory of the pixel signal SIG when there is no attenuation. SHOUT, which is the difference between Vp and Vd', is expressed by Equation 4.

[0113] During the D phase, the reference signal RMP decreases and falls below Vd' at timing T31, causing the comparison result to invert and the AD conversion to end. On the other hand, if there is no attenuation, the reference signal RMP falls below Vd at timing T32, after timing T31, and the AD conversion ends.

[0114] As illustrated in the figure, the voltage range is reduced compared to the case without attenuation due to the attenuation of the difference between the P-phase level and the D-phase level. As a result, the AD conversion time is shortened, and power consumption can be reduced.

[0115] Figure 18 shows an example of the input / output characteristics of the column amplifier 400 and ADC261 in the first embodiment of this technology. The horizontal axis in the figure represents the net pixel signal SIG', which is the difference between Vp and Vd, and this signal is input to the column amplifier 400. The vertical axis in the figure represents the digital signal CNT output from the ADC261.

[0116] Furthermore, the solid line trajectory shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is less than or equal to the threshold Th_cds, in other words, when the illuminance is low.

[0117] The dashed line shows the input / output characteristics when the difference between the P-phase level and the D-phase level (SIG') during the CDS period is higher than the threshold Th_cds, in other words, when the illuminance is high. From Equation 4, C 21 / (C 21 +C 22 The difference is attenuated by the attenuation rate of ). Therefore, the subsequent data restoration unit 340 restores the data as it would be without attenuation by multiplying the digital signal CNT by the reciprocal of the attenuation rate. The thick dotted line in the figure shows the input / output characteristics after restoration when the illuminance is high.

[0118] Figure 19 is a flowchart illustrating an example of the operation of the image sensor 200 in a first embodiment of this technology. This operation is initiated, for example, when a predetermined application for capturing an image is executed.

[0119] The vertical scanning circuit 210 selects and drives a row (step S901). During the P-phase period, the column amplifier 400 samples the P-phase level, and the ADC 261 performs AD conversion on the weighted average or D-phase level of the previous row held by the column amplifier 400 (step S902). In the first row, since there is no signal from the previous row, only sampling is performed in step S902.

[0120] Then, if the weighted average was converted using AD conversion in the previous step, data reconstruction is performed on the digital signal after CDS processing (step S903).

[0121] Next, the determination control circuit 310 determines whether the illuminance is high, which is higher than a predetermined value (step S903). If the illuminance is high (step S904: Yes), the column amplifier 400 samples the P-phase level and the D-phase level during the D-phase period, and the ADC 261 performs AD conversion on the P-phase level held by the column amplifier 400 (step S905).

[0122] On the other hand, in the case of low illuminance (step S904: No), the column amplifier 400 samples the D-phase level during the D-phase period, and the ADC 261 performs AD conversion on the P-phase level held by the column amplifier 400 (step S906).

[0123] In the final row, the weighted average or D-phase level held in step S905 or S906 is converted using AD conversion, and step S907 is executed after data recovery.

[0124] After step S905 or S906, the vertical scanning circuit 210 determines whether all rows have been selected (step S907). If there are any unselected rows (step S907: No), the image sensor 200 executes steps S901 onwards.

[0125] On the other hand, if all rows are selected (step S907: Yes), the operation for capturing the image ends.

[0126] When capturing multiple images in succession, the processes from steps S901 to S907 are repeatedly executed in synchronization with the vertical synchronization signal.

[0127] Thus, according to the first embodiment of this technology, while the sample-and-hold circuit 410 is sampling the reset level, the sample-and-hold circuit 450 holds the weighted average or signal level, and that signal is converted using AD conversion. This makes it possible to improve the read speed.

[0128] Furthermore, when the illuminance is high, the sample-and-hold circuit 450 outputs a weighted average of the P-phase level and D-phase level output by the sample-and-hold circuit 410, thereby attenuating the digital signal after CDS processing. This shortens the AD conversion time and reduces power consumption.

[0129] <2. Second Embodiment> In the first embodiment described above, the vertical scanning circuit 210 caused the selected row to generate a signal level after the reset level, but the control is not limited to this. The image sensor 200 in this second embodiment differs from the first embodiment in that the vertical scanning circuit 210 causes the selected row to generate a reset level after the signal level.

[0130] Figure 20 is a timing chart showing an example of the variation of the pixel signal and the judgment threshold in a second embodiment of this technology. In the figure, a shows the variation of the pixel signal SIG and the judgment threshold Th in the case of low illumination. In the figure, b shows the variation of the pixel signal SIG and the judgment threshold Th in the case of high illumination.

[0131] During the DDS period from timing T1 to T4, sampling of one row is performed using the DDS method. During this DDS period, each pixel in the selected row generates a reset level after the signal level.

[0132] As illustrated in figures a and b, the timing control unit 220 sets the determination threshold Th to Th2 over the D-phase period from timing T1 to T2. This Th2 is set to a value higher than the D-phase level expected at high illuminance.

[0133] Then, at the timing T2 in which the pixels 250 within the DDS period are initialized, the timing control unit 220 sets the judgment threshold Th to Th3, which is higher than Th2.

[0134] Furthermore, as illustrated in figure a, when the illumination is low, the pixel signal SIG is higher than the judgment threshold in all periods. Therefore, the judgment result of the judgment control circuit 310 is not inverted.

[0135] On the other hand, as illustrated in figure b, when the illuminance is high, the level of the pixel signal SIG (in this case, the D-phase level) becomes higher than the judgment threshold Th3 during the transfer period from timing T2 to T3, and the judgment result of the judgment control circuit 310 is reversed. Also, timing T3 During the initialization period from to T4, the level of the pixel signal SIG (in this case, the P-phase level) becomes higher than the judgment threshold Th3, and the judgment result of the judgment control circuit 310 is inverted.

[0136] Next, referring to Figures 21 to 25, we will explain the control of the column amplifier 400 from the sampling of one row until the AD conversion of that row is completed.

[0137] As illustrated in Figure 21, during the D-phase period, the timing control unit 220 turns on switches 411 and 412, causing the sample-and-hold circuit 410 to sample the signal level (i.e., the D-phase level) Vd.

[0138] Furthermore, the timing control unit 220 turns on switches 453 and 454, causing the sample-and-hold circuit 450 to hold the signal from the previous row. The selector 490 selects the output signal of the sample-and-hold circuit 450, and that signal is then converted using AD conversion.

[0139] Then, the charge is transferred, and within that transfer period, the determination control circuit 310 determines the illuminance. In Figures 22 and 23, it is assumed that the illuminance is determined to be low, below a predetermined value.

[0140] As illustrated in Figure 22, in the case of low light, the timing control unit 220 turns on switches 413 and 414 during the P-phase period, causing the sample-and-hold circuit 410 to hold the D-phase level Vd. The selector 490 selects the output signal of the sample-and-hold circuit 410 (i.e., the D-phase level), and that D-phase level Vd is converted using AD conversion.

[0141] Furthermore, during the P-phase period, the timing control unit 220 turns on switches 451 and 452, causing the sample-and-hold circuit 450 to sample the reset level (i.e., the P-phase level) Vp.

[0142] Then, as illustrated in Figure 23, during the D-phase period of the next row, the timing control unit 220 turns on switches 411 and 412, causing the sample-and-hold circuit 410 to sample the D-phase level of the next row.

[0143] The timing control unit 220 turns on switches 453 and 454 during the D-phase period, causing the sample-and-hold circuit 450 to hold the P-phase level Vp. The selector 490 selects the output signal of the sample-and-hold circuit 450 (i.e., the P-phase level), and this P-phase level Vp is converted using AD conversion.

[0144] Next, in Figures 24 and 25, it is assumed that the illuminance is determined to be high, higher than a predetermined value.

[0145] As illustrated in Figure 24, in the case of high illumination, the timing control unit 220 turns on switches 413 and 414 during the P-phase period, causing the sample-and-hold circuit 410 to hold the D-phase level Vd. The selector 490 selects the output signal of the sample-and-hold circuit 410 (i.e., the D-phase level), and this D-phase level Vd is converted using AD conversion. During this AD conversion, the counter 330 performs an up-count.

[0146] Furthermore, during the P-phase period, the timing control unit 220 turns on switches 455 and 451, causing the sample-and-hold circuit 450 to sample the D-phase level Vd and P-phase level Vp output from the sample-and-hold circuit 410.

[0147] Then, as illustrated in Figure 25, during the D-phase period of the next row, the timing control unit 220 turns on switches 411 and 412, causing the sample-and-hold circuit 410 to sample the D-phase level of the next row.

[0148] Furthermore, during the D-phase period, the timing control unit 220 turns on switches 453 and 454, causing the sample-and-hold circuit 450 to hold the weighted average. The selector 490 selects the output signal (i.e., the weighted average) of the sample-and-hold circuit 450, and this weighted average is converted using AD conversion. During this AD conversion, the counter 330 performs a down-count.

[0149] Since counter 330 performs an upcount when converting the D-phase level Vd and a downcount when converting the weighted average, its count value is the difference between Vd and the weighted average, and is expressed by the following formula. Vd-(VpC 22 +VdC 21 ) / (C 21 +C 22 ) =C 22 (Vd-Vp) / (C 21 +C 22 )...Equation 5

[0150] From Equation 5, the difference in the digital signals is the attenuated net pixel signal, which is the difference between Vp and Vd.

[0151] Thus, according to the second embodiment of this technology, while the sample-and-hold circuit 410 is sampling the signal level, the sample-and-hold circuit 450 holds the weighted average or reset level, and the signal is AD converted. This makes it possible to improve the readout speed when the signal level is read out first.

[0152] <3. Examples of applications to mobile devices> The technology disclosed herein (the Technology) can be applied to a variety of products. For example, the Technology disclosed herein may be implemented as a device mounted on any type of mobile vehicle, such as an automobile, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility device, airplane, drone, ship, or robot.

[0153] Figure 26 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0154] The vehicle control system 12000 comprises multiple electronic control units connected via a communication network 12001. In the example shown in Figure 26, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0155] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0156] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0157] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0158] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0159] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0160] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking system based on information from inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0161] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0162] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0163] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 26, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0164] Figure 27 shows an example of the installation position of the imaging unit 12031.

[0165] In Figure 27, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0166] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0167] Figure 27 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0168] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0169] For example, the microcomputer 12051, based on distance information obtained from imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to vehicle 12100). In particular, it can extract the nearest object on the vehicle 12100's path that is traveling in approximately the same direction as vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0170] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, heavy vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0171] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0172] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to the imaging unit 12031 of the configuration described above. Specifically, the image sensor 200 in Figure 3 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, it becomes possible to speed up readout and increase the frame rate.

[0173] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0174] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0175] Furthermore, this technology can also be configured as follows. (1) A first sample-and-hold circuit that samples and holds one of the levels of the reset level when the pixel is initialized and the signal level when charge is transferred within the pixel and outputs the result; A second sample-and-hold circuit that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. An image sensor equipped with the following features. (2) The pixel outputs the signal level after the reset level within a predetermined CDS period, The first sample-and-hold circuit samples and holds the reset level and outputs it. The second sample-and-hold circuit samples and holds the signal level and outputs it when the illuminance is lower than a predetermined value, and outputs a weighted average of the output reset level and the signal level when the illuminance is higher than the predetermined value. The image sensor described in (1) above. (3) The pixel outputs the reset level after the signal level within a predetermined DDS period, The first sample-and-hold circuit samples and holds the signal level and outputs it. The second sample-and-hold circuit samples and holds the reset level and outputs it when the illuminance is lower than a predetermined value, and outputs a weighted average of the output signal level and the reset level when the illuminance is higher than the predetermined value. The image sensor described in (1) above. (4) The device further comprises a selector that selects one of the output signals of the first sample-and-hold circuit and the output signal of the second sample-and-hold circuit and outputs it to the ADC. The image sensor described in any of (1) to (3) above. (5) Each of the first and second sample-and-hold circuits is: An operational amplifier that outputs the output signal to the selector, The first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, A first switch opens and closes the path between the other end of the first capacitive element and the vertical signal line connected to the pixel, A second switch opens and closes the path between the other end of the second capacitive element and the vertical signal line, A third switch opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier, A fourth switch opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier. The image sensor described in (4) above, comprising the above. (6) The second sample-and-hold circuit further comprises a fifth switch that opens and closes the path between the output terminal of the first sample-and-hold circuit and the other end of the second capacitive element. The image sensor described in (5) above. (7) The system further comprises a determination control circuit that determines whether the illuminance is higher than the predetermined value and controls the first and second sample-and-hold circuits based on the determination result. The image sensor according to claim 1. (8) A first sample-and-hold circuit that samples and holds one of the levels of the reset level when the pixel is initialized and the signal level when charge is transferred within the pixel and outputs the result; A second sample-and-hold circuit that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. An analog-to-digital converter that converts the output signals of the first and second sample-and-hold circuits into digital signals, An imaging device equipped with the following. (9) A first sample-and-hold procedure that samples and holds one of the levels of the reset level when the pixel is initialized and the signal level when charge is transferred within the pixel and outputs it, A second sample-and-hold procedure that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. A control method for an image sensor equipped with the following. [Explanation of Symbols]

[0176] 100 Imaging device 110 Optics Department 120 DSP circuits 130 Display section 140 Operation section 150 bus 160 frames memory 170 Storage section 180 Power supply section 200 Image Sensors 201 Light-receiving chip 202 Circuit Chips 210 Vertical scanning circuit 220 Timing Control Unit 230 DAC 240-pixel array section 250 pixels 251 Photoelectric conversion unit 252 transfer transistors 253 Reset Transistor 254 FD 255 Amplifying Transistors 256 Selective Transistors 260-column signal processing unit 261 ADC 270 Horizontal scanning circuit 280 Image Processing Unit 310 Judgment control circuit 311 Judgment circuit 312, 313, 321, 322, 421, 422, 461, 462 Capacitive elements 314, 323 Comparators 315, 316, 324, 325, 430, 470 Auto Zero Switch 317 Control circuit 320 Comparison Section 330 counter 340 Data Recovery Unit 400 Column Amplifier 410, 450 Sample-and-Hold Circuit Switches 411-415, 451-455 440, 480 op-amps 490 Selector 12031 Imaging Unit

Claims

1. A first sample-and-hold circuit that samples and holds one of the levels of the reset level when a pixel is initialized and the signal level when charge is transferred within the pixel, and outputs the result. A second sample-and-hold circuit that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. An image sensor equipped with the following features.

2. The aforementioned pixel outputs the signal level after the reset level within a predetermined CDS (Correlated Double Sampling) period. The first sample-and-hold circuit samples and holds the reset level and outputs it. The second sample-and-hold circuit samples and holds the signal level and outputs it when the illuminance is lower than a predetermined value, and outputs a weighted average of the output reset level and the signal level when the illuminance is higher than the predetermined value. The image sensor according to claim 1.

3. The aforementioned pixel outputs the reset level after the signal level within a predetermined DDS (Double Data Sampling) period. The first sample-and-hold circuit samples and holds the signal level and outputs it. The second sample-and-hold circuit samples and holds the reset level and outputs it when the illuminance is lower than a predetermined value, and outputs a weighted average of the output signal level and the reset level when the illuminance is higher than the predetermined value. The image sensor according to claim 1.

4. The system further includes a selector that selects one of the output signals from the first sample-and-hold circuit and the second sample-and-hold circuit and outputs it to an ADC (Analog to Digital Converter). The image sensor according to claim 1.

5. Each of the first and second sample-and-hold circuits is: An operational amplifier that outputs the output signal to the selector, The first and second capacitive elements, each having one end connected in common to the input terminal of the operational amplifier, A first switch opens and closes the path between the other end of the first capacitive element and the vertical signal line connected to the pixel, A second switch opens and closes the path between the other end of the second capacitive element and the vertical signal line, A third switch opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier, A fourth switch opens and closes the path between the other end of the first capacitive element and the output terminal of the operational amplifier. The image sensor according to claim 4, comprising:

6. The second sample-and-hold circuit further includes a fifth switch that opens and closes the path between the output terminal of the first sample-and-hold circuit and the other end of the second capacitive element. The image sensor according to claim 5.

7. The system further comprises a determination control circuit that determines whether the illuminance is higher than the predetermined value and controls the first and second sample-and-hold circuits based on the determination result. The image sensor according to claim 1.

8. A first sample-and-hold circuit that samples and holds one of the levels of the reset level when a pixel is initialized and the signal level when charge is transferred within the pixel, and outputs the result. A second sample-and-hold circuit that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. An analog-to-digital converter that converts the output signals of the first and second sample-and-hold circuits into digital signals, An imaging device equipped with the following.

9. A first sample-and-hold procedure that samples and holds one of the levels of the reset level when a pixel is initialized and the signal level when charge is transferred within the pixel, and outputs the result; A second sample-and-hold procedure that, when the illuminance is lower than a predetermined value, samples and holds the other level of the reset level and the signal level and outputs it, and when the illuminance is higher than the predetermined value, outputs a weighted average of the output level and the other level. A control method for an image sensor equipped with the following.

Images