Image sensor and control method for image sensor

Abstract

The present invention improves a reading speed in an image sensor that switches a conversion gain when converting charges into a voltage.　In the present invention, a threshold voltage of a threshold modulation transistor is modulated by the amount of charges stored in a photodiode. A comparator compares a sense voltage output from the threshold modulation transistor with a predetermined determination threshold within a predetermined determination period, and outputs a comparison result. A control signal generation unit generates a predetermined control signal on the basis of the comparison result. A transfer transistor 312 transfers charges from the photodiode to a floating diffusion (FD) after lapse of the determination period. A switching transistor switches a gain at the time of converting charges into a voltage according to the control signal.

Description

Image sensor and image sensor control method

[0001] The present technology relates to image sensors. More specifically, it relates to an image sensor that switches conversion gain when converting electric charge to voltage, and to a control method for an image sensor.

[0002] Conventionally, in image sensors, an image synthesis technique called HDR (High Dynamic Range) synthesis has been used to achieve a wider dynamic range than normal. For example, an image sensor has been proposed that switches the conversion gain when converting electric charge to voltage between two levels, HCG (Hi Convert Gain) and LCG (Low Convert Gain), in accordance with brightness (see, for example, Patent Document 1). In this image sensor, a column ADC (Analog to Digital Converter) performs AD (Analog to Digital) conversion on a reset level generated with HCG. Then, after transfer to an FD (Floating Diffusion), the conversion gain is switched in accordance with the brightness.

[0003] Japanese Patent Laid-Open No. 2016-092661

[0004] In the above-described conventional technology, HDR synthesis in a single imaging is achieved by switching the conversion gain after transfer to the FD. However, in the above-described conventional technology, the readout speed is lower than in the case where conversion gain is not switched. For example, in the above-described conventional technology, a column ADC AD-converts the reset level generated with HCG, and if the brightness is high after FD transfer, the gain is switched to LCG to AD-convert the reset level and the signal level in sequence. For this reason, a readout time for three AD conversions is required for each row, which causes a problem that the readout speed decreases.

[0005] The present technology was developed in view of such circumstances, and aims to improve the readout speed in an image sensor that switches conversion gain when converting electric charge to voltage.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an image sensor and control method comprising: a threshold modulation transistor whose threshold voltage is modulated by the amount of charge stored in the photodiode; a comparator that compares the sense voltage output from the threshold modulation transistor with a predetermined judgment threshold within a predetermined judgment period and outputs a comparison result; a control signal generation unit that generates a predetermined control signal based on the comparison result; a transfer transistor that transfers the charge from the photodiode to the FD after the judgment period has elapsed; and a switching transistor that switches the gain when converting the charge to voltage according to the control signal. This results in an improvement in read speed.

[0007] Furthermore, in this first aspect, the device may further include a reset transistor for initializing the FD, and the switching transistor may include first and second switching transistors connected in series. This results in the conversion gain being switched in three stages.

[0008] Furthermore, in this first aspect, the first and second switching transistors are connected in series between the reset transistor and the FD, the judgment threshold includes the first and second judgment thresholds, the comparator includes a first comparator that compares the sense voltage with the first judgment threshold and a second comparator that compares the sense voltage with the second judgment threshold, the control signal includes the first and second control signals, and the control signal generation unit may generate the first and second control signals based on the comparison results of the first and second comparators and output them to the gates of the first and second switching transistors. This results in the conversion gain being switched in three stages based on the comparison results of the sense voltage with the first and second judgment thresholds.

[0009] Furthermore, in this first aspect, the system may also be further equipped with a capacitor, the first switching transistor opening and closing the path between the reset transistor and the FD, and the second switching transistor opening and closing the path between the connection node of the first switching transistor and the reset transistor and the capacitor. This results in an expansion of the dynamic range.

[0010] Furthermore, in this first aspect, the judgment threshold includes a first and a second judgment threshold, the comparator includes a first comparator that compares the sense voltage with the first judgment threshold, and a second comparator that compares the sense voltage with the second judgment threshold, the control signal includes a first and a second control signal, and the control signal generation unit may generate the first and a second control signal based on the comparison results of the first and a second comparator and output them to the gates of the first and a second switching transistors. This results in the conversion gain being switched in three stages based on the comparison results of the sense voltage with the first and a second judgment threshold.

[0011] Furthermore, in this first aspect, the switching transistor may further include a third switching transistor connected in parallel with the first switching transistor, and a fourth switching transistor that opens and closes the path between the first switching transistor and the capacitor, and the control signal generation unit may output the control signal to the gates of both the first and second switching transistors. This results in the setting of two of the three conversion gains.

[0012] Furthermore, in this first aspect, the switching transistor further comprises first and second capacitors, and the switching transistor further includes third, fourth, fifth and sixth switching transistors, the first switching transistor opens and closes the path between the reset transistor and the FD, the second switching transistor opens and closes the path between the connection node of the first switching transistor and the reset transistor and the fourth switching transistor, the third switching transistor is connected in parallel with the first switching transistor, the fourth switching transistor is connected in parallel with the second switching transistor, the fifth switching transistor opens and closes the path between one end of the second capacitor and the third and fourth switching transistors, the sixth switching transistor opens and closes the path between one end of the first capacitor and one end of the second capacitor, and the control signal generation unit may output the control signal to the gates of both the first and second switching transistors. This results in the effect of setting three of the four conversion gains.

[0013] Furthermore, in this first aspect, the photodiode, the threshold modulation transistor, and the transfer transistor may be arranged in the first and second pixels, respectively, and the first and second pixels may share the FD. This results in a reduction in the number of elements per pixel.

[0014] Furthermore, in this first aspect, the photodiode, the threshold modulation transistor, and the transfer transistor may be arranged in each of the multiple pixels in the pixel array, and the comparator and the control signal generation unit may be provided in each row in the pixel array. This results in the effect of switching the conversion gain for each row.

[0015] Furthermore, in this first aspect, the photodiode, the threshold modulation transistor, and the transfer transistor may be arranged for each of the multiple pixels in the pixel array, and the comparator and the control signal generation unit may be provided for each pixel in the pixel array or for each area in the pixel array. This results in the effect of switching the conversion gain for each pixel or area.

[0016] Furthermore, in this first aspect, the comparator may compare the sense voltage of the n rows with the determination threshold within the determination period after the exposure period of n rows (where n is an integer) has elapsed. This results in the comparison of the sense voltage and the determination threshold after the exposure period has elapsed.

[0017] Furthermore, in this first aspect, the comparator may compare the sense voltage of the n rows with the determination threshold during the determination period while the n rows (where n is an integer) are being exposed. This results in an improvement in readout speed.

[0018] Furthermore, in this first aspect, the system may further include a DAC (Digital to Analog Converter) that generates either the judgment threshold or a predetermined ramp signal and supplies it to the comparator. This eliminates the need for a selector to choose between the judgment threshold and the ramp signal.

[0019] Furthermore, in this first aspect, the system may further include a DAC that generates a predetermined ramp signal, and a selector that selects one of the judgment threshold and the ramp signal and supplies it to the comparator. This eliminates the need for the DAC to supply the judgment threshold.

[0020] This is a block diagram showing an example configuration of an imaging device in the first embodiment of this technology. This is a block diagram showing an example configuration of an image sensor in the first embodiment of this technology. This is a diagram showing an example of a stacked structure of an image sensor in the first embodiment of this technology. This is a circuit diagram showing an example of a pixel configuration in the first embodiment of this technology. This is a diagram showing an example of a cross-sectional view of a substrate in the first embodiment of this technology. This is a graph showing the relationship between a potential diagram, charge amount, and sense voltage in the first embodiment of this technology. This is a block diagram showing an example configuration of a column signal processing unit in the first embodiment of this technology. This is a block diagram showing an example of a column signal processing unit with a selector added in the first embodiment of this technology. This is a diagram for explaining the control of the conversion gain in the first embodiment of this technology. This is a timing chart showing an example of the operation of an image sensor in the first embodiment of this technology. This is a diagram for explaining the timing of determination in the first embodiment of this technology. This is an example of a potential diagram in the first embodiment of this technology. This is a circuit diagram showing an example of an image sensor in a comparative example. This is a diagram for explaining the readout method in a comparative example. This is a circuit diagram showing an example of a pixel configuration in the second embodiment of this technology. This is a block diagram showing an example of a column signal processing unit in the second embodiment of this technology. This is a circuit diagram showing an example of a pixel configuration in the third embodiment of this technology. This is a timing chart showing an example of the operation of an image sensor in the third embodiment of this technology. This is a circuit diagram showing another example of a pixel in the third embodiment of this technology. This is a circuit diagram showing an example of a pixel configuration in the fourth embodiment of this technology. This is a diagram for explaining the readout method in the fourth embodiment of this technology. This is a circuit diagram showing another example of a pixel in the fourth embodiment of this technology. This is a circuit diagram showing an example of a pixel configuration in the fifth embodiment of this technology. This is a timing chart showing an example of the operation of an image sensor in the fifth embodiment of this technology. This is a diagram for explaining the readout method in the fifth embodiment of this technology. This is a circuit diagram showing another example of a pixel in the fifth embodiment of this technology. This is a circuit diagram showing an example of a FD shared block configuration in the sixth embodiment of this technology.This figure shows an example of the stacked structure and unit circuit of an image sensor in the seventh embodiment of this technology. This block diagram shows an example of a schematic configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation positions of the external information detection unit and the imaging unit.

[0021] The following describes the embodiments for implementing this technology (hereinafter referred to as "embodiments"). The explanation will proceed in the following order: 1. First embodiment (Example of determining brightness before FD transfer) 2. Second embodiment (Example of determining brightness before FD transfer and switching the conversion gain in three stages) 3. Third embodiment (Example of determining brightness before FD transfer and adding capacity) 4. Fourth embodiment (Example of determining brightness before FD transfer and setting two of the three conversion gains) 5. Fifth embodiment (Example of determining brightness before FD transfer and setting three of the four conversion gains) 6. Sixth embodiment (Example of sharing an FD among multiple pixels and determining brightness before FD transfer) 7. Seventh embodiment (Example of determining brightness before FD transfer and performing AD conversion for each pixel or area) 8. Application examples to mobile devices

[0022] <1. First Embodiment> [Example of Imaging Device Configuration] Figure 1 is a block diagram showing an example configuration of an imaging device 100 in the first embodiment of this technology. This imaging device 100 is a device for capturing image data and includes an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. Furthermore, the imaging device 100 includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. Examples of imaging devices 100 include cameras mounted on smartphones and in-vehicle cameras.

[0023] The optical unit 110 collects light from the subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion. The image sensor 200 supplies the generated image data to the DSP circuit 120 via the signal line 209.

[0024] The DSP circuit 120 performs predetermined signal processing on the image data. The DSP circuit 120 outputs the processed image data to a frame memory 160 or the like via the bus 150.

[0025] The display unit 130 displays image data. The display unit 130 could be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The operation unit 140 generates operation signals according to user input.

[0026] Bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with each other.

[0027] The frame memory 160 stores image data. The storage unit 170 stores various data, including image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and other components.

[0028] [Example of Image Sensor Configuration] Figure 2 is a block diagram showing an example configuration of an image sensor 200 in the first embodiment of this technology. This image sensor 200 includes a vertical scanning circuit 211, a timing control unit 212, a DAC 213, a pixel array unit 214, a column signal processing unit 220, a horizontal scanning circuit 215, and an image processing unit 216. Multiple pixels 300 are arranged in a two-dimensional grid in the pixel array unit 214.

[0029] The vertical scanning circuit 211 sequentially selects and drives rows within the pixel array section 214.

[0030] The timing control unit 212 controls the operating timing of the vertical scanning circuit 211, DAC 213, column signal processing unit 220, and horizontal scanning circuit 215 in synchronization with the vertical synchronization signal VSYNC.

[0031] The DAC213 generates a sawtooth-shaped ramp signal and supplies it to the column signal processing unit 220.

[0032] Each pixel 300 generates an analog pixel signal by photoelectric conversion in accordance with the control of the vertical scanning circuit 211. Each pixel 300 in each column outputs a pixel signal to the column signal processing unit 220 via a vertical signal line.

[0033] The column signal processing unit 220 has an ADC (not shown) for each column of pixels 300. This column signal processing unit 220 converts the pixel signals of each column into digital signals in synchronization with the horizontal synchronization signal and outputs them to the image processing unit 216 according to the control of the horizontal scanning circuit 215.

[0034] The horizontal scanning circuit 215 controls the column signal processing unit 220 to output digital signals sequentially.

[0035] The image processing unit 216 performs various image processing operations, such as HDR synthesis, on image data obtained by arranging digital signals. The image processing unit 216 supplies the processed image data to the DSP circuit 120 via the signal line 209. Alternatively, an external circuit of the image sensor 200 (such as the DSP circuit 120) can perform the HDR synthesis and other processing operations instead of the image processing unit 216.

[0036] Figure 3 shows an example of a stacked structure of an image sensor 200 in the first embodiment of this technology. This image sensor 200 comprises a circuit chip 202 and a pixel chip 201 stacked on the circuit chip 202. These chips are electrically connected via connection points such as vias. In addition to vias, connections can also be made by Cu-Cu junctions or bumps.

[0037] The pixel chip 201 is provided with a pixel array section 214. The circuit chip 202 is provided with peripheral circuits 210 and a column signal processing section 220. The peripheral circuits 210 include, for example, a vertical scanning circuit 211, a timing control section 212, a DAC 213, a horizontal scanning circuit 215, and an image processing section 216, as illustrated in Figure 2.

[0038] [Configuration example of pixel] FIG. 4 is a circuit diagram illustrating one configuration example of a pixel 300 according to the first embodiment of the present technology. The pixel 300 includes a photodiode 311, a transfer transistor 312, a reset transistor 313, a switching transistor 321, an FD1, an FD2, an amplification transistor 314, and a selection transistor 315. The pixel 300 further includes a threshold modulation transistor 316 and a selection transistor 317. As a transistor in the pixel 300, for example, an nMOS (n-channel Metal Oxide Semiconductor) transistor is used.

[0039] In addition, the vertical scanning circuit 211 supplies a reset signal RST, a transfer signal TG, a bias voltage SENSE, a selection signal SELjg, and a selection signal SELfd to each of the pixels 300 in a row via signal lines wired in the horizontal direction.

[0040] In addition, in the pixel array section 214, a vertical control line 308 and a vertical signal line 309 are wired in the vertical direction for each column. Each of the pixels 300 in the column supplies a pixel signal SIG to the column signal processing section 220 via the vertical signal line 309. In addition, the column signal processing section 220 supplies a control signal FDGV to each of the pixels 300 in the column via the vertical control line 308.

[0041] The photodiode 311 converts incident light into electric charge. The transfer transistor 312 transfers electric charge (such as electrons) from the photodiode 311 to the FD1 in accordance with the transfer signal TG.

[0042] The reset transistor 313 initializes the FD1 and the FD2 in accordance with the reset signal RST.

[0043] The switching transistor 321 switches the conversion gain when converting electric charge into voltage in accordance with the control signal FDGV. The switching transistor 321 is inserted between the reset transistor 313 and the FD1, and FD2 is provided at the connection node of the reset transistor 313 and the switching transistor 321. When the switching transistor 321 is in an on state, FD1 is coupled to FD2, so the conversion gain is lower than that when the switching transistor 321 is in an off state. Let the former conversion gain be LCG and the latter conversion gain be HCG.

[0044] The amplification transistor 314 constitutes a source follower circuit, and outputs a voltage corresponding to the voltage of FD1. The selection transistor 315 outputs the voltage from the amplification transistor 314 to the vertical signal line 309 in accordance with the selection signal SELfd.

[0045] The threshold modulation transistor 316 is a transistor whose threshold voltage is modulated according to the amount of charge accumulated in the photodiode 311. The voltage output from the source of this threshold modulation transistor 316 is defined as the sense voltage Sout. With this threshold modulation transistor 316, the level of the photodiode 311 before transfer to FD1 can be read out non-destructively.

[0046] The selection transistor 317 outputs the sense voltage Sout to the vertical signal line 309 in accordance with the selection signal SELjg.

[0047] The selection transistors 315 and 317 are exclusively controlled. The voltage of the vertical signal line 309 when the selection transistor 317 is in the on state (that is, the voltage of the pixel signal SIG) corresponds to the sense voltage Sout.

[0048] FIG. 5 is a diagram showing an example of a cross-sectional view of the substrate 400 according to the first embodiment of the present technology. A P-type substrate is used as the substrate 400. Various transistors are formed on the front surface of the substrate 400, and incident light is irradiated onto the back surface opposite to the front surface. An N layer 431 and an N － layer 432 are formed in the substrate 400. A portion formed by these layers and the P layer on the back side functions as the photodiode 311.

[0049] With the surface side facing upwards, the upper part of the N layer 431 on the surface is N ＋ Layers 421 and 422 are formed, and N is located near them. ＋ Layer 423 is formed. ＋ A gate electrode 411 is formed between layers 421 and 422, and these function as threshold modulation transistors 316. ＋ A gate electrode 412 is formed between layers 422 and 423, and these function as transfer transistors 312.

[0050] With the above structure, a potential corresponding to the amount of charge accumulated in the photodiode 311 is applied to the back gate of the threshold modulation transistor 316, and the threshold voltage of the threshold modulation transistor 316 is modulated according to that potential.

[0051] Figure 6 is a graph showing the relationship between the potential diagram, charge amount, and sense voltage in the first embodiment of this technology. Figure a shows an example of the potential diagram of pixel 300 when the charge amount of the photodiode 311 is initialized (in other words, at the start of exposure). Figure b shows an example of the potential diagram of pixel 300 during exposure. Figure c shows an example of the relationship between the sense voltage Sout and the charge amount.

[0052] In the figure, "PD" in a and b indicates the potential of the photodiode 311. "SENS" indicates a constant bias voltage applied to the gate of the threshold modulation transistor 316. "Sout" indicates the sense voltage output from the source of the threshold modulation transistor 316. In the figure, the dotted line in b indicates the potential at the start of exposure. In the figure, the vertical axis in c indicates the sense voltage Sout, and the horizontal axis indicates the potential and charge amount of the photodiode 311.

[0053] As illustrated in figures a and b, charge accumulates in the photodiode 311 during the exposure period, and the sense voltage Sout fluctuates according to the amount of charge. As illustrated in figure c, when electrons are accumulated as charge, the greater the amount of charge (in other words, the brighter the image), the lower the potential of the photodiode 311. The lower the potential of the photodiode 311, the lower the sense voltage Sout becomes due to the modulation of the threshold voltage.

[0054] [Example of Column Signal Processing Unit Configuration] Figure 7 is a block diagram showing an example configuration of the column signal processing unit 220 in the first embodiment of this technology. This column signal processing unit 220 comprises a plurality of column control signal generation units 221, a plurality of ADCs 230, and a plurality of latch circuits 222. One column control signal generation unit 221, one ADC 230, and one latch circuit 222 are arranged for each column.

[0055] Furthermore, the DAC213 generates either a ramp signal RMP or a judgment threshold Vth and supplies it to the ADC230 for each column. The judgment threshold Vth is supplied during the judgment period at the end of exposure, and the ramp signal RMP is supplied during the readout period after the judgment period has elapsed.

[0056] The ADC230 converts the analog pixel signal SIG into a digital signal. This ADC230 is, for example, a single-slope type ADC and includes a comparator 231 and a counter 232.

[0057] The comparator 231 compares the pixel signal SIG with the ramp signal RMP or the judgment threshold Vth. The inverting input terminal (-) of the comparator 231 is connected to the vertical signal line 309 and the pixel signal SIG is input to it. The non-inverting input terminal (+) of the comparator 231 is connected to the DAC 213 and the ramp signal RMP or the judgment threshold Vth is input to it. The comparator 231 supplies the comparison result VCO to the column control signal generation unit 221 and the counter 232.

[0058] Counter 232 counts the count value over the period until the comparison result VCO inverts. This counter 232 causes the latch circuit 222 to hold the digital signal Dout, which indicates the count value.

[0059] The latch circuit 222 outputs the digital signal Dout to the image processing unit 216 in accordance with the control of the horizontal scanning circuit 215.

[0060] The column control signal generation unit 221 generates a control signal FDGV based on the comparison result VCO and outputs it to the corresponding column via the vertical control line 308.

[0061] The DAC213 generates either the ramp signal RMP or the judgment threshold Vth, but the configuration is not limited to this.

[0062] For example, as illustrated in Figure 8, the DAC 213 can generate only the ramp signal RMP, and a selector 223 can be added to each column, with the selector 223 selecting either a fixed determination threshold Vth or the ramp signal RMP and supplying it to the comparator 231.

[0063] Figure 9 is a diagram illustrating the control of the conversion gain in the first embodiment of this technology. The threshold voltage of the threshold modulation transistor 316 is modulated according to the amount of charge accumulated in the photodiode 311. The selection transistor 317 outputs the sense voltage Sout output from the threshold modulation transistor 316 to the vertical signal line 309 during the determination period at the end of exposure.

[0064] Furthermore, the comparator 231 compares the determination threshold Vth with the voltage of the pixel signal SIG (i.e., the sense voltage Sout) within the determination period. The comparison result VCO of the comparator 231 indicates the brightness determination result. The column control signal generation unit 221 generates a control signal FDGV based on the comparison result VCO and supplies it to the gate of the switching transistor 321.

[0065] After the judgment period has elapsed, the transfer transistor 312 transfers charge from the photodiode 311 to FD1. The switching transistor 321 also switches the conversion gain according to the control signal FDGV.

[0066] As described above, the image sensor 200 can non-destructively read out the sense voltage Vout corresponding to the amount of charge stored in the photodiode 311 before transferring it to FD1, and compare it with the judgment threshold Vth.

[0067] [Example of Image Sensor Operation] Figure 10 is a timing chart showing an example of the operation of the image sensor 200 in the first embodiment of this technology.

[0068] The vertical scanning circuit 211 sequentially selects and exposes rows using a rolling shutter method, and the ADC 230 in the column signal processing unit 220 performs AD conversion of the pixel signal when exposure is complete. The figure shows the operation of one row of the image sensor 200.

[0069] The vertical scanning circuit 211 sets the reset signal RST to a high level during the pulse period from timing T1 to T3, and the transfer signal TG to a high level during the pulse period from timing T2 to T3. In addition, the column control signal generation unit 221 sets the control signal FDGV to a high level during the pulse period from timing T2 to T3. This initiates exposure of the selected row.

[0070] Then, the vertical scanning circuit 211 raises the reset signal RST to a high level during the pulse period from timing T4 to T5, and the column control signal generation unit 221 raises the control signal FDGV to a high level during the same period. As a result, exposure ends, FD is initialized, and the reset level is generated. This reset level is also called the P-phase level.

[0071] Then, the vertical scanning circuit 211 raises the selection signal SELjg to a high level during the determination period from timing T5 to T6. During this period, the voltage of the pixel signal SIG (i.e., the voltage of the vertical signal line) corresponds to the sense voltage Sout from the threshold modulation transistor 316.

[0072] Furthermore, the DAC 213 (not shown) generates a determination threshold Vth within the determination period, and the comparator 231 (not shown) compares the sense voltage Sout with the determination threshold Vth. As the amount of charge increases (in other words, the brightness increases), the sense voltage Sout decreases, so the comparator 231 can determine whether the incident light is bright or not by comparing it with the determination threshold Vth.

[0073] If the sense voltage Sout is lower than the determination threshold Vth (i.e., bright), the column control signal generation unit 221 (not shown) raises the control signal FDGH to a high level during the period from timing T7 to T10. This sets LCG. The dashed line in the figure indicates the level when it is bright.

[0074] On the other hand, if the sense voltage Sout is greater than or equal to the judgment threshold Vth (i.e., it is dark), the column control signal generation unit 221 (not shown) lowers the control signal FDGH to a low level during the period from timing T7 to T10. This sets HCG.

[0075] Furthermore, the vertical scanning circuit 211 sets the selection signal SELfd to a high level at timing T6, and sets the transfer signal TG to a high level during the pulse period from timing T8 to T9. As a result, charge is transferred to FD, and a signal level corresponding to the amount of accumulated charge is generated. This signal level is also called the D-phase level.

[0076] The ADC320 performs AD conversion on the reset level (i.e., the P-phase level) of the pixel signal SIG during the period from timing T7 to T8, and performs AD conversion on the signal level (i.e., the D-phase level) of the pixel signal SIG during the period from timing T9 to T10.

[0077] The column signal processing unit 220 (not shown) performs CDS (Correlated Double Sampling) processing to determine the difference between the P-phase level and the D-phase level. Then, the subsequent image processing unit 216 (not shown) or DSP circuit 120 (not shown) performs HDR synthesis on the digital signal after CDS processing.

[0078] Timing T5 to T10 corresponds to a 1H period, which is the period of the horizontal synchronization signal. This 1H period includes the determination period from timing T5 to T6 and the readout period from timing T6 to T10.

[0079] As illustrated in the figure, the comparator 231 determines the brightness during the judgment period before transfer to the FD, and the conversion gain is switched according to the judgment result. Then, during the read period, the ADC 230 sequentially performs AD conversion on the P-phase level and D-phase level generated by the HCG or LCG.

[0080] In this figure, the comparator 231 performs the judgment on n rows within the judgment period after the exposure period for n rows (where n is an integer) has elapsed, but the configuration is not limited to this.

[0081] For example, as illustrated in Figure 11, the comparator 231 can perform the n-row determination during the exposure of n rows. In this case, the n-row determination is performed during idle time within the 1H period of n-1 rows (for example, the transfer period to the FD). This makes it possible to perform the determination operation without degradation of the 1H period.

[0082] Figure 12 is an example of a potential diagram in the first embodiment of this technology. In the figure, a shows an example of a potential diagram of a pixel before transfer in the dark case, and b shows an example of a potential diagram of a pixel after transfer in the dark case. In the figure, c shows an example of a potential diagram of a pixel before transfer in the bright case, and d shows an example of a potential diagram of a pixel after transfer in the bright case.

[0083] As illustrated in figure a, in the case of darkness, the sense voltage Vout from the threshold modulation transistor becomes greater than or equal to the judgment threshold Vth.

[0084] In this case, the column control signal generation unit 221 (not shown) sets the control signal FDGV to a low level and the conversion gain to HCG, as illustrated in figure b. Then, charge is transferred to FD1, and the level generated by HCG is read out.

[0085] On the other hand, in bright conditions, as illustrated in figure c, the sense voltage Vout from the threshold modulation transistor becomes lower than the judgment threshold Vth.

[0086] In this case, the column control signal generation unit 221 (not shown) sets the control signal FDGV to a high level and the conversion gain to LCG, as illustrated in d in the figure. Then, charge is transferred to FD1 and FD2, and the levels generated by LCG are read out.

[0087] Here, we will consider a configuration in which brightness is determined after transfer to the floppy disk (FD1 or FD2) as a comparative example.

[0088] Figure 13 is a circuit diagram showing an example of an image sensor in a comparative example. In the comparative example, the threshold modulation transistor 316 and the selection transistor 317 are not provided. Therefore, in the comparative example, the level cannot be read non-destructively before charge transfer to the FD. Thus, the comparator 231 compares the pixel signal SIG with the determination threshold Vth after charge transfer from the photodiode 311 to the FD to determine the brightness.

[0089] Figure 14 is a diagram illustrating the reading method in a comparative example. Figure a shows an example of the operation of the ADC230 in dark conditions, and figure b shows an example of the operation of the ADC230 in bright conditions.

[0090] Between timings T10 and T11, the ADC230 performs AD conversion on the reset level (i.e., the P-phase level) generated by the HCG. Then, between timings T11 and T12, the charge is transferred to the FD. The brightness is then determined, and if it is dark, the conversion gain remains with the HCG, and the ADC230 performs AD conversion on the signal level (i.e., the D-phase level) between timings T12 and T13.

[0091] On the other hand, as illustrated in figure b, if it is bright, the system switches to LCG after timing T12. The ADC230 then performs AD conversion of the D phase level during the period from timing T12 to T13. Subsequently, the P phase level is generated by LCG, and the ADC230 performs AD conversion of the P phase level during the period from timing T13 to T14. The period from timing T10 to T14 corresponds to the 1H period.

[0092] As mentioned above, in the comparative example where the comparison is performed after transferring to a floppy disk, it is necessary to set the 1H period for each line when performing AD conversion three times.

[0093] In contrast, in the first embodiment, since the comparison is performed before transferring to the FD, as mentioned above, only two AD conversions are required within the 1H period, and the read speed can be improved compared to the comparative example.

[0094] Furthermore, the comparative image sensor uses CDS drive, which reads out the P-phase followed by the D-phase in bright conditions, whereas it uses DDS (Double Data Sampling) drive, which reads out the D-phase followed by the P-phase in dark conditions. Since DDS drive generates more noise than CDS drive, there is a risk of image quality degradation.

[0095] On the other hand, the image sensor 200 of the first embodiment can perform CDS driving regardless of brightness, and therefore can reduce noise and improve image quality compared to the comparative example.

[0096] Thus, according to the first embodiment of this technology, the comparator 231 compares the sense voltage from the threshold modulation transistor 316 with the determination threshold during the determination period before transfer to the FD, thereby improving the read speed and image quality compared to the comparative example.

[0097] <2. Second Embodiment> In the first embodiment described above, the image sensor 200 switched the conversion gain in two stages, HCG and LCG, but it may also switch in three or more stages. The image sensor 200 in this second embodiment differs from the first embodiment in that it switches the conversion gain in three stages.

[0098] Figure 15 is a circuit diagram showing an example configuration of a pixel 300 in a second embodiment of this technology. The pixel 300 in the second embodiment differs from that in the first embodiment in that it further includes a switching transistor 322 and an FD3. In addition, in the second embodiment, vertical control lines 307 are further wired to each row of the pixel array 214.

[0099] The switching transistor 322 is inserted between the reset transistor 313 and the switching transistor 321. The control signal FCGV is input to the gate of the switching transistor 322 via the vertical control line 307. An FD3 is provided at the connection node between the reset transistor 313 and the switching transistor 322. Note that the switching transistors 321 and 322 are examples of the first and second switching transistors described in the claims.

[0100] When both switching transistors 321 and 322 are ON, the conversion gain is lower than when switching transistor 321 is ON and switching transistor 322 is OFF. The conversion gain in the former case is denoted as LCG (Low Convert Gain), and the conversion gain in the latter case is denoted as MCG (Middle Convert Gain). Furthermore, when both switching transistors 321 and 322 are OFF, the conversion gain is higher than MCG, and this conversion gain is denoted as HCG (High Convert Gain).

[0101] Note that the control signals FDGV and FCGV are examples of the first and second control signals described in the claims.

[0102] Figure 16 is a block diagram showing an example configuration of the column signal processing unit 220 in a second embodiment of the present technology. The column signal processing unit 220 in this second embodiment differs from the first embodiment in that a comparator 224 is further arranged for each column. In the second embodiment, the comparator 231 supplies the comparison result as VCO1 to the column control signal generation unit 221. In the second embodiment, the DAC 213 generates either a ramp signal RMP or a determination threshold Vth1.

[0103] A fixed judgment threshold Vth2 is input to the non-inverting input terminal (+) of comparator 224, and the inverting input terminal (-) is connected to the vertical signal line 309. Comparator 224 compares the pixel signal SIG with the judgment threshold Vth2 and supplies the comparison result as VCO2 to the column control signal generation unit 221. Comparators 231 and 224 are examples of the first and second comparators described in the claims.

[0104] The column control signal generation unit 221 generates control signals FDGV and FCGV based on the comparison results VCO1 and VCO2. For example, the judgment threshold Vth2 is set to a value lower than Vth1. If the voltage of the pixel signal SIG during the judgment period (i.e., the sense voltage Sout) is lower than the judgment threshold Vth2, it is the brightest, so the column control signal generation unit 221 sets the control signals FDGV and FCGV to high levels to make it LCG. If the sense voltage Sout is equal to or greater than the judgment threshold Vth1, it is the darkest, so the column control signal generation unit 221 sets the control signals FDGV and FCGV to low levels to make it HCG. If the sense voltage Sout is equal to or greater than the judgment threshold Vth2 and lower than the judgment threshold Vth1, the column control signal generation unit 221 sets the control signal FDGV to high levels and FCGV to low levels to make it MCG.

[0105] Although the conversion gain is switched in three stages as illustrated in Figures 15 and 16, it is also possible to add switching transistors and comparators to switch in four or more stages. Furthermore, as illustrated in Figure 8 in the second embodiment, the DAC 213 can generate only the ramp signal RMP, and a selector 223 can be added to each column so that the selector 223 selects either the judgment threshold Vth1 or the ramp signal RMP.

[0106] Thus, according to the second embodiment of this technology, the conversion gain can be switched in three stages because a switching transistor 322 and a comparator 224 have been added.

[0107] <3. Third Embodiment> In the second embodiment described above, the conversion gain was switched in three stages, but it is preferable to further expand the dynamic range. The image sensor 200 in this third embodiment differs from that in the second embodiment in that it has additional capacitance.

[0108] Figure 17 is a circuit diagram showing one example configuration of a pixel 300 in a third embodiment of the present technology. The pixel 300 in the third embodiment differs from that in the first embodiment in that it further comprises a capacitor 331 and an OFG transistor 318.

[0109] One end of the capacitor 331 is connected to the MIMVDD voltage supplied from the vertical scanning circuit 211. This capacitor 331 is also called LOFIC (Lateral Overflow Integration Capacitor). The addition of this capacitor 331 (LOFIC) expands the dynamic range compared to the second embodiment.

[0110] In the third embodiment, the switching transistor 322 opens and closes the path between the connection node of the reset transistor 313 and the switching transistor 321 and the other end of the capacitor 331 according to the control signal FCGV.

[0111] The OFG transistor 318 opens and closes the path between the connection node of the photodiode 311 and the transfer transistor 312 and the other end of the capacitor 331 according to the control signal OFG. Charge overflowing from the photodiode 311 is stored in the capacitor 331 via the OFG transistor 318.

[0112] Furthermore, in the column signal processing unit 220 (not shown) of the third embodiment, two comparators are arranged for each column, similar to the second embodiment.

[0113] Figure 18 is a timing chart showing an example of the operation of the image sensor 200 in a third embodiment of this technology.

[0114] The vertical scanning circuit 211 sets the transfer signal TG to a high level during the pulse period from timing T1 to T3, and sets the reset signal RST and voltage MIMVDD to a high level during the pulse period from timing T2 to T3. In addition, the column control signal generation unit 221 sets the control signals FCGV and FDGV to a high level during the pulse period from timing T1 to T3. Exposure is initiated by these controls.

[0115] Then, the vertical scanning circuit 211 raises the reset signal RST to a high level during the pulse period from timing T4 to T6, and the column control signal generation unit 221 raises the control signal FDGV to a high level during the pulse period from timing T4 to T5. As a result, exposure ends and the reset level (P-phase level) is generated.

[0116] The vertical scanning circuit 211 then sets the selection signal SELjg to a high level during the judgment period from timing T6 to T7. Additionally, the vertical scanning circuit 211 sets the voltage MIMVDD to a high level at timing T7, when reading begins.

[0117] Then, the vertical scanning circuit 211 sets the selection signal SELfd to a high level during the period from timing T7 to T8, and the column control signal generation unit 221 generates control signals FDGV and FCGV according to the brightness. When it is brightest, the column control signal generation unit 221 sets LCG by setting both control signals FDGV and FCGV to a high level, and when it is darkest, the column control signal generation unit 221 sets HCG by setting both control signals FDGV and FCGV to a low level. When the brightness is intermediate, the column control signal generation unit 221 sets MCG by setting control signal FDGV to a high level and FCGV to a low level. The ADC 230 also performs AD conversion on the P-phase level generated by LCG, MCG, or HCG.

[0118] Then, the vertical scanning circuit 211 raises the transfer signal TG to a high level during the transfer period from timing T9 to T10. As a result, charge is transferred and the signal level (D phase level) is optimized.

[0119] The vertical scanning circuit 211 raises the selection signal SELfd to a high level during the period from timing T10 to T11, and the column control signal generation unit 221 generates control signals FDGV and FCGV based on the brightness determination result. The ADC 230 performs AD conversion on the D-phase level generated by either LCG, MCG, or HCG.

[0120] Furthermore, as illustrated in Figure 19a, the OFG transistor 318 can be reduced. Also, as illustrated in Figure 19b, one end of the capacitor 331 can be connected to the power supply voltage VDD.

[0121] Thus, according to the third embodiment of this technology, the dynamic range can be expanded by adding this capacitance 331 (LOFIC).

[0122] <4. Fourth Embodiment> In the third embodiment described above, only one of the three conversion gains, LCG, MCG, and HCG, was set, but the configuration is not limited to this. The image sensor 200 in this fourth embodiment differs from the third embodiment in that two of the three conversion gains are set.

[0123] Figure 20 is a circuit diagram showing one example configuration of a pixel 300 in a fourth embodiment of this technology. The pixel 300 in the fourth embodiment differs from that in the third embodiment in that it further includes switching transistors 323 and 324. In addition, in the fourth embodiment, the pixel array section 214 does not have vertical control lines 307 wired to it, but two lines, a vertical control line 308 and a vertical signal line 309, are wired to each row.

[0124] The gates of switching transistors 321 and 322 are connected in common to the vertical control line 308, and the control signal FDGV is input to their gates.

[0125] The switching transistor 323 is connected in parallel with the switching transistor 321, and the control signal FDGH from the vertical scanning circuit 211 is input to its gate.

[0126] The switching transistor 324 is inserted between the capacitor 331 and the switching transistor 322, and the control signal FCGH from the vertical scanning circuit 211 is input to its gate.

[0127] Note that switching transistors 323 and 324 are examples of the third and fourth switching transistors described in the claims.

[0128] Furthermore, in the fourth embodiment, the column signal processing unit 220 (not shown) has only one comparator per column, similar to the first embodiment.

[0129] Figure 21 is a diagram illustrating the reading method in the fourth embodiment of this technology. In the figure, a shows the reading method when it is determined to be dark within the judgment period, and b shows the reading method when it is determined to be bright within the judgment period.

[0130] If darkness is detected, the column control signal generation unit 221 (not shown), as illustrated in figure a, sets the control signal FDGV to a low level. The vertical scanning circuit 211 (not shown) also sets the control signals FDGH and FCGH to low levels. As a result, HCG is set, and the ADC 230 (not shown) performs AD conversion with the P-phase level set to P1.

[0131] After the conversion of P1, the vertical scanning circuit 211 raises the control signals FDGH and FCGH to high levels. This sets MCG, and the ADC 230 performs AD conversion with the P-phase level as P2.

[0132] After the conversion of P2, the vertical scanning circuit 211 transfers the charge using the transfer signal TG. Then, the ADC 230 performs AD conversion with the D phase level as D2.

[0133] After the D2 conversion, the vertical scanning circuit 211 sets the control signals FDGH and FCGH to a low level. This sets HCG, and the ADC 230 performs AD conversion with the D-phase level as D1.

[0134] The column signal processing unit 220 (not shown) calculates the difference between P1 and D1, and the difference between P2 and D2, during the CDS processing. This generates an HCG pixel signal and an MCG pixel signal for each pixel.

[0135] If it is determined that the area is bright, the column control signal generation unit 221 (not shown) raises the control signal FDGV to a high level, as illustrated in figure b. The control method for the control signals FDGH and FCGH is the same as in figure a. As a result, when the area is bright, the pixel signals for MCG and LCG are generated.

[0136] Furthermore, as illustrated in Figure 22a, the OFG transistor 318 can be reduced. Also, as illustrated in Figure 22b, one end of the capacitor 331 can be connected to the power supply voltage VDD.

[0137] Thus, according to the fourth embodiment of this technology, by adding switching transistors 323 and 324, it is possible to set two of the three conversion gains.

[0138] <5. Fifth Embodiment> In the fourth embodiment described above, the image sensor 200 had two of the three conversion gains set, but the conversion gains may have four or more levels. The image sensor 200 in this fifth embodiment differs from the fourth embodiment in that it has three of the four conversion gains set.

[0139] Figure 23 is a circuit diagram showing one example configuration of a pixel 300 in a fifth embodiment of the present technology. The pixel 300 in this fifth embodiment differs from that in the fourth embodiment in that it further comprises a capacitor 332 and switching transistors 325 and 326.

[0140] One end of capacitor 332 is connected to the MIMVDD voltage from the vertical scanning circuit 211. Capacitor 332 is also called LOFIC. Capacitors 331 and 332 are examples of the first and second capacitors described in the claims.

[0141] In the fifth embodiment, the switching transistor 324 is connected in parallel with the switching transistor 323, and the control signal FCG2 from the vertical scanning circuit 211 is input to the gate of the switching transistor 324.

[0142] The switching transistor 325 opens and closes the path between one end of the capacitor 331 and the switching transistors 322 and 324 according to the control signal FCGH from the vertical scanning circuit 211.

[0143] The switching transistor 326 opens and closes the path between one end of capacitor 331 and one end of capacitor 332 according to the control signal FCGH2 from the vertical scanning circuit 211. Note that switching transistors 325 and 326 are examples of the fifth and sixth switching transistors described in the claims.

[0144] When FD1, FD2, and capacitors 331 and 332 are all coupled, the conversion gain becomes lower than LCG, and this conversion gain is defined as ULCG (Ultra Low Convert Gain).

[0145] Figure 24 is a timing chart showing an example of the operation of the image sensor 200 in the fifth embodiment of this technology.

[0146] At the timing T1 when exposure ends, the vertical scanning circuit 211 changes the reset signal RST from a high level to a low level.

[0147] Then, the vertical scanning circuit 211 raises the selection signal SELjg to a high level within the judgment period from timing T1 to T2.

[0148] The vertical scanning circuit 211 sets the selection signal SELfd and the voltage MIMVDD to a high level at timing T2 when reading begins. The column control signal generation unit 221 also sets the control signal FDGV to a high level or a low level at timing T2 according to the brightness.

[0149] Then, the ADC230 performs AD conversion with the reset level (i.e., the P-phase level) set to P1 during the period from timing T2 to T3.

[0150] Then, the vertical scanning circuit 211 raises the transfer signal TG to a high level during the pulse period from timing T3 to T4. This generates the signal level (i.e., the D-phase level).

[0151] Then, the ADC230 performs AD conversion to set the D-phase level to D1 within the period from timing T4 to T5.

[0152] Then, the vertical scanning circuit 211 changes the control signal FDGH from a low level to a high level at timing T5, and changes the transfer signal TG to a high level for the duration of the pulse from timing T6. Then, the vertical scanning circuit 211 changes the control signal FCGH from a low level to a high level at timing T7.

[0153] Then, the ADC230 performs AD conversion to set the D-phase level to D2 within the period from timing T7 to T8.

[0154] Then, the vertical scanning circuit 211 changes the control signal FCGH2 from a low level to a high level at timing T8, and the ADC 230 performs AD conversion to set the D-phase level to D3 within the period from timing T8 to T9.

[0155] Then, the vertical scanning circuit 211 raises the reset signal RST to a high level during the pulse period from timing T9 to T10. This generates the P-phase level.

[0156] Then, the ADC230 performs AD conversion to set the P-phase level to P3 within the period from timing T10 to T11.

[0157] Then, the vertical scanning circuit 211 lowers the control signal FCGH2 to a low level at timing T11, and then lowers the control signal FCGH to a low level at timing T12.

[0158] Then, the ADC 230 performs AD conversion to set the P-phase level to P2 within the period from timing T12 to T13. Then, the vertical scanning circuit 211 sets the control signal FDGH to a low level at timing T13.

[0159] Furthermore, the column signal processing unit 220 (not shown) calculates the difference between P1 and D1, the difference between P2 and D2, and the difference between P3 and D3 during the CDS processing.

[0160] Figure 25 is a diagram illustrating the reading method in the fifth embodiment of this technology. In the figure, a shows the reading method when it is determined to be dark within the judgment period, and b shows the reading method when it is determined to be bright within the judgment period.

[0161] If darkness is detected, the column control signal generation unit 221 (not shown), as illustrated in figure a, sets the control signal FDGV to a low level. The vertical scanning circuit 211 (not shown) also sets the control signals FDGH, FCGH, and FCGH2 to a low level. As a result, HCG is set, and the ADC 230 (not shown) performs AD conversion with the reset level (i.e., the P-phase level) set to P1.

[0162] After the conversion of P1, the vertical scanning circuit 211 transfers the charge using the transfer signal TG. Then, the ADC 230 performs AD conversion with the signal level (i.e., the D-phase level) as D1.

[0163] After the conversion of D1, the vertical scanning circuit 211 raises the control signals FDGH and FCGH to high levels. This sets MCG, and the ADC 230 performs AD conversion with the D-phase level as D2.

[0164] After the D2 conversion, the vertical scanning circuit 211 raises the control signal FCGH2 to a high level. This sets ULCG, and the ADC 230 performs AD conversion with the D-phase level set to D3.

[0165] After the D3 conversion, the vertical scanning circuit 211 initializes the FD with the reset signal RST. This generates a P-phase level, and the ADC 230 performs AD conversion using the P-phase level as P3.

[0166] After the conversion of P3, the control signal FCGH2 is set to a low level. This sets the MCG, and the ADC230 performs AD conversion with the P-phase level as P2. With the above control, if it is dark, the pixel signals for HCG, MCG, and ULCG are generated.

[0167] If it is determined that it is bright, the column control signal generation unit 221 (not shown) raises the control signal FDGV to a high level, as illustrated in figure b. The control method for the control signals FDGH, FCGH, and FCG2 is the same as in figure a. As a result, if it is bright, the pixel signals for MCG, LCG, and ULCG are generated.

[0168] Furthermore, as illustrated in Figure 26a, the OFG transistor 318 can be reduced. Also, as illustrated in Figure 26b, one end of capacitors 331 and 332 can be connected to the power supply voltage VDD.

[0169] Thus, according to the fifth embodiment of this technology, since a capacitor 332 and switching transistors 325 and 326 are added, three of the four conversion gains can be set.

[0170] <6. Sixth Embodiment> In the first embodiment described above, each pixel was provided with an FD (FD1 and FD2), but multiple pixels can also share an FD. The image sensor 200 in this sixth embodiment differs from the first embodiment in that multiple pixels share an FD.

[0171] Figure 27 is a circuit diagram showing one configuration example of an FD sharing block 305 in the sixth embodiment of this technology. In the sixth embodiment, the pixel array section 214 (not shown) is divided by a plurality of FD sharing blocks 305. Two pixels are arranged in each of the FD sharing blocks 305. These two pixels share one FD (FD1 or FD2).

[0172] For example, the FD shared block 305 includes photodiodes 311-1 and 311-2, transfer transistors 312-1 and 312-2, threshold modulation transistors 316-1 and 316-2, and selection transistors 317-1 and 317-2. Furthermore, the FD shared block 305 includes a reset transistor 313, a switching transistor 321, FD1, FD2, an amplification transistor 314, and a selection transistor 315.

[0173] Transfer transistor 312-1 transfers charge from photodiode 311-1 to FD1 according to transfer signal TG1. Transfer transistor 312-2 transfers charge from photodiode 311-2 to FD1 according to transfer signal TG2.

[0174] The threshold voltage of the threshold modulation transistor 316-1 is modulated by the amount of charge accumulated in the photodiode 311-1. The threshold voltage of the threshold modulation transistor 316-2 is modulated by the amount of charge accumulated in the photodiode 311-2.

[0175] The selection transistor 317-1 outputs the sense voltage Sout1 from the threshold modulation transistor 316-1 to the vertical signal line 309 according to the selection signal SELjg1. The selection transistor 317-2 outputs the sense voltage Sout2 from the threshold modulation transistor 316-2 to the vertical signal line 309 according to the selection signal SELjg2.

[0176] The connection configuration of elements from FD1 onwards is the same as in the first embodiment. Although two pixels share the FD (FD1 and FD2), the number of pixels that share the FD is not limited to two. For example, four pixels or eight pixels can also share the FD.

[0177] By sharing the floppy disk (FD) among multiple pixels, the number of elements per pixel can be reduced.

[0178] Furthermore, the second embodiment can be applied to each of the second, third, fourth, and fifth embodiments.

[0179] Thus, according to the sixth embodiment of this technology, since the FD is shared among multiple pixels, the number of elements per pixel can be reduced.

[0180] <7. Seventh Embodiment> In the first embodiment described above, an ADC 230 was provided for each row, but the configuration is not limited to this. The image sensor 200 in this seventh embodiment differs from the first embodiment in that an ADC 230 is provided for each pixel or area.

[0181] Figure 28 shows an example of the stacked structure of the image sensor 200 and the unit circuit 226 in the seventh embodiment of the present technology. In the figure, a shows an example of the stacked structure of the image sensor 200, and b shows an example of the unit circuit 226.

[0182] As illustrated in figure a, in the seventh embodiment, the circuit chip 202 is provided with a signal processing unit 225 instead of a column signal processing unit 220. A unit circuit 226 is arranged in the signal processing unit 225 for each pixel.

[0183] As illustrated in figure b, the unit circuit 226 includes a control signal generation unit 227, an ADC 230, and a latch circuit 222. The control signal generation unit 227 generates a control signal FDGV based on the comparison result VCO from the ADC 230 and supplies it to the corresponding pixel 300. By providing an ADC 230 for each pixel, exposure can be performed using a global shutter method and all pixels can be read out simultaneously.

[0184] Furthermore, the pixel array section 214 can be divided into multiple areas, and a unit circuit 226 can be provided for each area.

[0185] Furthermore, the seventh embodiment can be applied to each of the second, third, fourth, fifth, and sixth embodiments.

[0186] Thus, according to the seventh embodiment of this technology, since an ADC 230 is provided for each pixel, a global shutter system can be used.

[0187] <8. Examples of Application to Mobile Devices> The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as an automobile, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility device, airplane, drone, ship, or robot.

[0188] Figure 29 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0189] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 29, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0190] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0191] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0192] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0193] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0194] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0195] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0196] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0197] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0198] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 29, the output devices include an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0199] Figure 30 shows an example of the installation position of the imaging unit 12031.

[0200] In Figure 30, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0201] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0202] Figure 30 shows an example of the imaging ranges of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0203] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0204] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0205] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0206] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0207] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to the imaging unit 12031 of the configuration described above. Specifically, the imaging device 100 in Figure 1 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, the readout speed is improved, making it possible to increase the frame rate.

[0208] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0209] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0210] Furthermore, this technology can also take the following configurations: (1) An image sensor comprising: a threshold modulation transistor whose threshold voltage is modulated by the amount of charge accumulated in a photodiode; a comparator which compares a sense voltage output from the threshold modulation transistor with a predetermined judgment threshold within a predetermined judgment period and outputs a comparison result; a control signal generation unit which generates a predetermined control signal based on the comparison result; a transfer transistor which transfers the charge from the photodiode to the FD after the judgment period has elapsed; and a switching transistor which switches the gain when converting the charge to a voltage according to the control signal. (2) The image sensor according to (1), further comprising a reset transistor for initializing the FD, wherein the switching transistor includes first and second switching transistors connected in series. (3) The image sensor according to (2), wherein the first and second switching transistors are connected in series between the reset transistor and the FD, the determination threshold includes the first and second determination thresholds, the comparator includes a first comparator that compares the sense voltage with the first determination threshold and a second comparator that compares the sense voltage with the second determination threshold, the control signal includes the first and second control signals, and the control signal generation unit generates the first and second control signals based on the comparison results of the first and second comparators and outputs them to the respective gates of the first and second switching transistors. (4) The image sensor according to (2), further comprising a capacitor, wherein the first switching transistor opens and closes the path between the reset transistor and the FD, and the second switching transistor opens and closes the path between the connection node of the first switching transistor and the reset transistor and the capacitor.(5) The image sensor according to (4), wherein the judgment threshold includes a first and a second judgment threshold, the comparator includes a first comparator that compares the sense voltage with the first judgment threshold, and a second comparator that compares the sense voltage with the second judgment threshold, the control signal includes a first and a second control signal, and the control signal generation unit generates the first and a second control signal based on the comparison results of the first and a second comparator and outputs them to the gates of the first and a second switching transistor, respectively. (6) The image sensor according to (4), wherein the switching transistor further includes a third switching transistor connected in parallel with the first switching transistor, and a fourth switching transistor that opens and closes a path between the first switching transistor and the capacitor, and the control signal generation unit outputs the control signal to the gates of both the first and a second switching transistor. (7) The image sensor according to (2), further comprising first and second capacitors, wherein the switching transistor further includes third, fourth, fifth and sixth switching transistors, the first switching transistor opens and closes a path between the reset transistor and the FD, the second switching transistor opens and closes a path between the connection node of the first switching transistor and the reset transistor and the fourth switching transistor, the third switching transistor is connected in parallel with the first switching transistor, the fourth switching transistor is connected in parallel with the second switching transistor, the fifth switching transistor opens and closes a path between one end of the second capacitor and the third and fourth switching transistors, the sixth switching transistor opens and closes a path between one end of the first capacitor and one end of the second capacitor, and the control signal generation unit outputs the control signal to the gates of both the first and second switching transistors. (8) The photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of the first and second pixels, and the first and second pixels share the FD, as described in any of (1) to (7) above.(9) The image sensor according to any one of (1) to (8), wherein the photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of a plurality of pixels in the pixel array, and the comparator and the control signal generation unit are provided for each row in the pixel array. (10) The image sensor according to any one of (1) to (8), wherein the photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of a plurality of pixels in the pixel array, and the comparator and the control signal generation unit are provided for each pixel in the pixel array, or for each area in the pixel array. (11) The image sensor according to any one of (1) to (10), wherein the comparator compares the sense voltage of the n rows with the determination threshold during the determination period after the exposure period of n rows (where n is an integer) has elapsed. (12) The image sensor according to any one of (1) to (10), wherein the comparator compares the sense voltage of the n rows with the determination threshold during the determination period while the exposure of n rows (where n is an integer) is in progress. (13) An image sensor according to any one of (1) to (12), further comprising a DAC that generates either the judgment threshold or a predetermined ramp signal and supplies it to the comparator. (14) An image sensor according to any one of (1) to (12), further comprising a DAC that generates a predetermined ramp signal, and a selector that selects either the judgment threshold or the ramp signal and supplies it to the comparator. (15) A control method for an image sensor, comprising: a comparison procedure that compares a sense voltage output from a threshold modulation transistor, whose threshold voltage is modulated by the amount of charge accumulated in a photodiode, with a predetermined judgment threshold within a predetermined judgment period and outputs a comparison result; a control signal generation procedure that generates a predetermined control signal based on the comparison result; a transfer procedure in which a transfer transistor transfers the charge from the photodiode to the FD after the judgment period has elapsed; and a switching procedure in which a switching transistor switches the gain when converting the charge to a voltage according to the control signal.

[0211] 100 Imaging device 110 Optical unit 120 DSP circuit 130 Display unit 140 Operation unit 150 Bus 160 Frame memory 170 Storage unit 180 Power supply unit 200 Image sensor 201 Pixel chip 202 Circuit chip 210 Peripheral circuit 211 Vertical scanning circuit 212 Timing control unit 213 DAC 214 Pixel array unit 215 Horizontal scanning circuit 216 Image processing unit 220 Column signal processing unit 221 Column control signal generation unit 222 Latch circuit 223 Selector 224, 231 Comparator 225 Signal processing unit 226 Unit circuit 227 Control signal generation unit 230 ADC 232 Counter 300 Pixel 305 FD shared block 311, 311-1, 311-2 Photodiode 312, 312-1, 312-2 Transfer transistors 313 Reset transistor 314 Amplifier transistor 315, 317, 317-1, 317-2 Selection transistors 316, 316-1, 316-2 Threshold modulation transistors 318 OFG transistors 321-326 Switching transistors 331, 332 Capacitors 400 Substrate 411, 412 Gate electrodes 421, 422, 423 N ＋ Layer 431 N Layer 432 N － Layer 12031 Imaging section

Claims

1. An image sensor comprising: a threshold modulation transistor whose threshold voltage is modulated by the amount of charge accumulated in a photodiode; a comparator that compares a sense voltage output from the threshold modulation transistor with a predetermined judgment threshold within a predetermined judgment period and outputs a comparison result; a control signal generation unit that generates a predetermined control signal based on the comparison result; a transfer transistor that transfers the charge from the photodiode to an FD (Floating Diffusion) after the judgment period has elapsed; and a switching transistor that switches the gain when converting the charge to a voltage according to the control signal.

2. The image sensor according to claim 1, further comprising a reset transistor for initializing the FD, wherein the switching transistor includes first and second switching transistors connected in series.

3. The image sensor according to claim 2, wherein the first and second switching transistors are connected in series between the reset transistor and the FD, the determination threshold includes the first and second determination thresholds, the comparator includes a first comparator that compares the sense voltage with the first determination threshold and a second comparator that compares the sense voltage with the second determination threshold, the control signal includes the first and second control signals, and the control signal generation unit generates the first and second control signals based on the comparison results of the first and second comparators and outputs them to the respective gates of the first and second switching transistors.

4. The image sensor according to claim 2, further comprising a capacitance, wherein the first switching transistor opens and closes the path between the reset transistor and the FD, and the second switching transistor opens and closes the path between the connection node of the first switching transistor and the reset transistor and the capacitance.

5. The image sensor according to claim 4, wherein the determination threshold includes a first and a second determination threshold, the comparator includes a first comparator that compares the sense voltage with the first determination threshold, and a second comparator that compares the sense voltage with the second determination threshold, the control signal includes a first and a second control signal, and the control signal generation unit generates the first and a second control signal based on the comparison results of the first and a second comparator and outputs them to the gates of the first and a second switching transistors, respectively.

6. The image sensor according to claim 4, wherein the switching transistor further includes a third switching transistor connected in parallel with the first switching transistor and a fourth switching transistor that opens and closes a path between the first switching transistor and the capacitor, and the control signal generation unit outputs the control signal to the gates of both the first and second switching transistors.

7. The image sensor according to claim 2, further comprising first and second capacitors, wherein the switching transistor further includes third, fourth, fifth and sixth switching transistors, the first switching transistor opens and closes a path between the reset transistor and the FD, the second switching transistor opens and closes a path between the connection node of the first switching transistor and the reset transistor and the fourth switching transistor, the third switching transistor is connected in parallel with the first switching transistor, the fourth switching transistor is connected in parallel with the second switching transistor, the fifth switching transistor opens and closes a path between one end of the second capacitor and the third and fourth switching transistors, the sixth switching transistor opens and closes a path between one end of the first capacitor and one end of the second capacitor, and the control signal generation unit outputs the control signal to the gates of both the first and second switching transistors.

8. The image sensor according to claim 1, wherein the photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of the first and second pixels, and the first and second pixels share the FD.

9. The image sensor according to claim 1, wherein the photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of a plurality of pixels in the pixel array, and the comparator and the control signal generation unit are provided in each row in the pixel array.

10. The image sensor according to claim 1, wherein the photodiode, the threshold modulation transistor, and the transfer transistor are arranged in each of a plurality of pixels in the pixel array, and the comparator and the control signal generation unit are provided for each pixel in the pixel array or for each area in the pixel array.

11. The image sensor according to claim 1, wherein the comparator compares the sense voltage of the n rows with the determination threshold within the determination period after the exposure period of n rows (where n is an integer) has elapsed.

12. The image sensor according to claim 1, wherein the comparator compares the sense voltage of the n rows with the determination threshold during the determination period while the n rows are being exposed.

13. The image sensor according to claim 1, further comprising a DAC that generates either the determination threshold or a predetermined lamp signal and supplies it to the comparator.

14. The image sensor according to claim 1, further comprising a DAC (Digital to Analog Converter) that generates a predetermined lamp signal, and a selector that selects one of the determination threshold and the lamp signal and supplies it to the comparator.

15. A control method for an image sensor comprising: a comparison procedure for comparing a sense voltage output from a threshold modulation transistor, whose threshold voltage is modulated by the amount of charge accumulated in a photodiode, with a predetermined judgment threshold within a predetermined judgment period and outputting a comparison result; a control signal generation procedure for generating a predetermined control signal based on the comparison result; a transfer procedure for a transfer transistor to transfer the charge from the photodiode to the FD after the judgment period has elapsed; and a switching procedure for a switching transistor to switch the gain when converting the charge to a voltage according to the control signal.

Images