Imaging device

Abstract

The present invention enables an amount of charge accumulated in a photoelectric conversion unit to be detected without performing destructive readout.　This imaging device comprises: a photoelectric conversion unit that is provided in a pixel; a transfer transistor that transfers charges accumulated in the photoelectric conversion unit to a floating diffusion; an amplification transistor that outputs a pixel signal corresponding to the charges accumulated in the floating diffusion; a first selection transistor that selects the output from the amplification transistor; a threshold modulation transistor having a threshold that is modulated on the basis of the charges accumulated in the photoelectric conversion unit; and a second selection transistor that selects the output from the threshold modulation transistor.

Description

Imaging device

[0001] This technology relates to an imaging device. More specifically, this technology relates to an imaging device capable of reading signals from pixels based on the detection result of the amount of charge accumulated in the photoelectric conversion unit.

[0002] In imaging devices, there are techniques to achieve HDR (High Dynamic Range) regardless of the storage capacitance that stores the charge in the photoelectric conversion unit. For example, a technique has been disclosed in which a pixel is provided with a counting unit that counts and outputs a first signal that is output when a first node matches a predetermined potential, and a readout circuit that reads out the potential of the first node (see, for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2023-183375

[0004] However, in the conventional technology described above, it was necessary to transfer the charge accumulated in the photoelectric conversion section to a floating diffusion and perform destructive readout, which could lead to a decrease in signal strength.

[0005] This technology was developed in light of these circumstances, and its purpose is to enable the detection of the amount of charge accumulated in the photoelectric conversion unit without performing destructive readout.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an imaging device comprising: a photoelectric conversion unit provided in a pixel; a transfer transistor that transfers the charge accumulated in the photoelectric conversion unit to a floating diffusion; an amplification transistor that outputs a pixel signal corresponding to the charge accumulated in the floating diffusion; a first selection transistor that selects the output from the amplification transistor; a threshold modulation transistor whose threshold is modulated based on the charge accumulated in the photoelectric conversion unit; and a second selection transistor that selects the output from the threshold modulation transistor. This provides the effect of enabling the determination of the amount of charge accumulated in the photoelectric conversion unit based on non-destructive readout.

[0007] Furthermore, in the first aspect, the system may include signal lines for transmitting the output from the amplification transistor and the output from the threshold modulation transistor. This allows for the determination of the output levels of the amplification transistor and the threshold modulation transistor.

[0008] Furthermore, in the first aspect, the photoelectric conversion unit, the transfer transistor, the threshold modulation transistor, and the second selection transistor may be provided for each pixel, while the amplification transistor and the first selection transistor may be shared by multiple pixels. This has the effect of enabling the determination of the amount of charge accumulated for each pixel while suppressing an increase in the circuit size of the pixels.

[0009] Furthermore, in the first aspect, the system may further include a first determination unit that, based on the determination result of the output level of the threshold modulation transistor, instructs the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion. This results in the reading of a pixel signal from each pixel according to the amount of charge accumulated in each pixel.

[0010] Furthermore, in the first aspect, the reading of the output level of the threshold modulation transistor may be non-destructive. This has the effect of making it possible to determine the amount of charge accumulated for each pixel without reducing the amount of charge accumulated for each pixel.

[0011] Furthermore, in the first aspect, the output level of the threshold modulation transistor may be read out multiple times during the charge accumulation period. This results in the pixel signal being read out multiple times from each pixel, depending on the amount of charge accumulated for each pixel.

[0012] Furthermore, in the first aspect, the first determination unit may instruct the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion if the output level of the threshold modulation transistor is at a threshold level below the saturation level. This results in the pixel signal being read out multiple times in each frame from each pixel according to the amount of charge accumulated in each pixel, without causing the charge accumulated in the pixels to overflow.

[0013] Furthermore, in the first aspect, the threshold level may be set to a potential lower than or equal to the depletion voltage of the photoelectric conversion unit. This absorbs the offset of each pixel and variations in the maximum storage capacity, while ensuring that a pixel signal is read out from each pixel according to the amount of charge stored in each pixel.

[0014] Furthermore, in the first aspect, the system may further include a switching transistor that switches the conversion efficiency of the pixels, which is set to read out the pixel signal according to the charge accumulated in the floating diffusion. This results in an expansion of the dynamic range based on the switching of the conversion efficiency of each pixel.

[0015] Furthermore, in the first aspect, the system may further include a comparator that determines the output level of the threshold modulation transistor and the output level of the amplification transistor, and a counter that performs a counting operation based on the determination result of the output level of the threshold modulation transistor or the output level of the amplification transistor. This provides the functionality to output a pixel signal from a pixel based on the output level of the threshold modulation transistor, or to output a pixel signal from a pixel based on the difference in the output levels of the threshold modulation transistors.

[0016] Furthermore, in the first aspect, the first determination unit may set the comparator and the counter to non-operation if the output level of the threshold modulation transistor is below the saturation level. This reduces power consumption when reading signals from pixels, while ensuring that pixel signals are read from each pixel according to the amount of charge accumulated for each pixel.

[0017] Furthermore, in the first aspect, the system may further include a storage unit that stores the pixel signals read out based on the determination result of the output level of the threshold modulation transistor, and an adder that adds the pixel signals stored in the storage unit for each pixel. This results in the effect that the pixel signals read out from each pixel are accumulated for each frame according to the amount of charge accumulated for each pixel.

[0018] Furthermore, in the first aspect, the device comprises a pixel array section in which the pixels are arranged in a matrix in the row direction and column direction, a vertical scanning circuit for scanning the pixels in the column direction, a vertical signal line for transmitting the output level of the threshold modulation transistor and the output level of the amplification transistor in the column direction, and a horizontal control line for driving the pixels in the row direction, wherein the vertical scanning circuit may drive the transfer transistor, the first selection transistor and the second selection transistor via the horizontal control line. This results in the output level of the amplification transistor and the output level of the threshold modulation transistor being read out row by row in the column direction.

[0019] Furthermore, in the first aspect, the transfer transistor may include a first transfer transistor that transfers the charge accumulated in the photoelectric conversion unit row by row, and a second transfer transistor that transfers the charge accumulated in the photoelectric conversion unit column by column. This results in the operation of reading out a pixel signal from each pixel according to the amount of charge accumulated in each pixel arranged in a matrix in the row direction and column direction.

[0020] Furthermore, in the first aspect, the system may further include a second determination unit that, based on the determination result of the difference in the output levels of the threshold modulation transistor, instructs the vertical scanning circuit to read out a pixel signal corresponding to the charge accumulated in the floating diffusion. This results in the operation of reading out a pixel signal from each pixel according to the difference in the amount of charge accumulated in each pixel.

[0021] Furthermore, in the first aspect, the difference in the output levels of the threshold modulation transistor may also be the difference between frames. This results in the effect that a pixel signal is read out from each pixel according to the temporal difference in the amount of charge accumulated for each pixel.

[0022] Furthermore, in the first aspect, the difference in the output levels of the threshold modulation transistor may also be the difference between adjacent pixels. This results in the effect that a pixel signal is read out from each pixel according to the spatial difference in the amount of charge accumulated for each pixel.

[0023] Furthermore, in the first aspect, the second determination unit may instruct the vertical scanning circuit to read out pixels having the difference in output levels and pixels adjacent to those pixels. This results in the reading of pixel signals from pixels selected according to the difference in the amount of charge accumulated for each pixel.

[0024] Furthermore, in the first aspect, the second determination unit may set the processing of pixel signals transmitted via the vertical signal line to non-operation for columns other than the column to which the pixel having the output level difference and the pixel adjacent to the pixel belong. This reduces power consumption when reading signals from pixels, while ensuring that pixel signals are read from pixels selected according to the difference in the amount of charge accumulated for each pixel.

[0025] Furthermore, the second aspect is an imaging device comprising a photoelectric conversion unit provided in a pixel, a sense unit provided in the pixel that outputs a sense signal that follows the PD potential corresponding to the charge accumulated in the photoelectric conversion unit, and an output unit that outputs a pixel signal that follows the FD potential of the floating diffusion corresponding to the charge transferred from the photoelectric conversion unit, based on the PD potential detected by the sense unit. This provides the effect of enabling the determination of the amount of charge accumulated in the photoelectric conversion unit based on non-destructive readout.

[0026] This is a block diagram showing an example configuration of an imaging device according to the first embodiment. This is a block diagram showing an example configuration of a solid-state imaging device according to the first embodiment. This is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the first embodiment. This is a block diagram showing an example configuration of an AD conversion unit according to the first embodiment. This is a flowchart showing an example of operation of the imaging device according to the first embodiment. This is a timing chart showing an example of operation of the imaging device according to the first embodiment when saturation state is detected. This is a diagram showing the relationship between exposure time and signal level for each illuminance level of the imaging device according to the first embodiment. This is a diagram showing an example of the waveform when a low-illuminance pixel is operating according to the first embodiment. This is a diagram showing an example of the waveform when a medium-illuminance pixel is operating according to the first embodiment. This is a diagram showing an example of the waveform when a high-illuminance pixel is operating according to the first embodiment. This is a plan view showing an example of the layout of a pixel according to the first embodiment. This is a cross-sectional view showing an example configuration of the sense unit of a pixel according to the first embodiment. This is a diagram showing an example of the potential of the sense unit of a pixel according to the first embodiment. This is a cross-sectional view showing an example configuration of the transfer unit of a pixel according to the first embodiment. This is a diagram showing an example of the potential of the transfer unit of a pixel according to the first embodiment. This is a diagram showing the relationship between the light intensity of a pixel and the PD potential and FD potential according to the first embodiment. This is a diagram showing an example of the PD potential for each pixel according to the first embodiment. This figure shows an example of the circuit configuration of a dummy pixel provided in the solid-state imaging device according to the first embodiment. This is a block diagram showing an example of the configuration of the solid-state imaging device according to the second embodiment. This figure shows an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the second embodiment. This is a flowchart showing an example of the operation of the imaging device according to the second embodiment. This is a timing chart showing an example of the operation of the imaging device according to the second embodiment when an event is detected. This figure shows an example of the PD potential for each pixel in adjacent rows according to the second embodiment. This is a block diagram showing an example of the configuration of the solid-state imaging device according to the third embodiment. This is a flowchart showing an example of the operation of the imaging device according to the third embodiment. This is a block diagram showing an example of the configuration of the solid-state imaging device according to the fourth embodiment. This is a block diagram showing an example of the configuration of the AD conversion unit according to the fourth embodiment. This is a flowchart showing an example of the operation of the imaging device according to the fourth embodiment.This is a timing chart showing an example of operation when detecting a lateral edge of the imaging device according to the fourth embodiment. This is a timing chart showing an example of operation when detecting a vertical edge of the imaging device according to the fourth embodiment. This is a diagram showing an example of the PD potential for each pixel in adjacent frames according to the fourth embodiment. This is a diagram showing an example of pixel connection in the solid-state imaging device according to the fourth embodiment. This is a diagram showing an example of two-pixel sharing in the solid-state imaging device according to the fourth embodiment. This is a diagram showing an example of four-pixel sharing in the solid-state imaging device according to the fourth embodiment. This is a block diagram showing an example of the configuration of the solid-state imaging device according to the fifth embodiment. This is a flowchart showing an example of operation of the imaging device according to the fifth embodiment. This is a diagram showing an example of the waveform during pixel operation according to the fifth embodiment. This is a diagram showing an example of pixel connection in the solid-state imaging device according to the fifth embodiment. This is a diagram showing an example of two-pixel sharing in the solid-state imaging device according to the fifth embodiment. This is a diagram showing an example of four-pixel sharing in the solid-state imaging device according to the fifth embodiment. This is a perspective view showing an example of stacking in the solid-state imaging device according to the sixth embodiment. This is a block diagram showing a schematic configuration example of a vehicle control system. This is an explanatory diagram showing an example of the installation position of the imaging unit.

[0027] The following describes the embodiments for implementing this technology (hereinafter referred to as "embodiments"). The description will proceed in the following order: 1. First embodiment (an example of reading out the FD potential based on the sensing result of the PD potential, and adding and storing the pixel signals based on the FD potential) 2. Second embodiment (an example of reading out the pixel signal based on the difference in PD potential between frames) 3. Third embodiment (an example of reading out the pixel signal from the pixel with the difference based on the difference in PD potential between frames) 4. Fourth embodiment (an example of reading out the pixel signal based on the difference in PD potential between adjacent pixels) 5. Fifth embodiment (an example of reading out the pixel signal from the pixel with the difference based on the difference in PD potential between adjacent pixels) 6. Sixth embodiment (an example of stacking pixel arrays) 7. Application examples to mobile devices

[0028] <1. First Embodiment> Figure 1 is a block diagram showing an example of the configuration of an imaging device according to the first embodiment.

[0029] In the figure, the imaging device 100 comprises an optical system 101, a solid-state imager 102, an imaging control unit 103, an image processing unit 104, a storage unit 105, a display unit 106, and an operation unit 107. The imaging device 100 also includes a drive control unit 109. The imaging control unit 103, image processing unit 104, storage unit 105, display unit 106, operation unit 107, and drive control unit 109 are connected to each other via a bus 108. The imaging device 100 may be used as a standalone unit, incorporated into a mobile terminal such as a smartphone, incorporated into an authentication device or monitoring device, or incorporated into a vehicle or drone.

[0030] The optical system 101 directs light from the subject into the solid-state imaging device 102 and forms an optical image on the light-receiving surface of the solid-state imaging device 102. The optical system 101 may include, for example, a focus lens, a zoom lens, and an aperture. The optical system 101 may also include multiple lenses, such as a wide-angle lens, a standard lens, and a telephoto lens.

[0031] The solid-state imaging device 102 converts the optical image formed on the light-receiving surface into an electrical signal for each pixel, and outputs the electrical signal digitized. At this time, the solid-state imaging device 102 can output pixel signals by switching the conversion efficiency of the pixels. For example, the solid-state imaging device 102 may switch the conversion efficiency of the pixels in two stages or in three stages. The solid-state imaging device 102 may also be equipped with multiple floating diffusions to which charge is transferred from the pixels. The solid-state imaging device 102 can also sense the PD potential corresponding to the charge accumulated in the photoelectric conversion unit. The PD potential is a potential corresponding to the amount of charge accumulated in the photoelectric conversion unit provided in each pixel. The solid-state imaging device 102 can then read the FD potential from the floating diffusion based on the PD potential sensing result, and store the pixel signal based on the FD potential. The FD potential is a potential corresponding to the amount of charge transferred to the floating diffusion provided in each pixel. In this case, the solid-state imaging device 102 can output a pixel signal that tracks the FD potential of a pixel when the PD potential of that pixel reaches a threshold level below the saturation level according to the illuminance of that pixel in each frame. Here, the solid-state imaging device 102 can read out the pixel signal from each pixel multiple times according to the illuminance of that pixel in each frame, and store the sum of these pixel signals. The solid-state imaging device 102 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The CMOS image sensor may be a back-illuminated image sensor or a front-illuminated image sensor.

[0032] The imaging control unit 103 controls imaging by the solid-state imaging device 102 based on commands from the operation unit 107. At this time, the imaging control unit 103 can control the exposure time, exposure amount, and imaging timing of the solid-state imaging device 102.

[0033] The image processing unit 104 performs image processing based on the output from the solid-state imaging device 102. Image processing includes, for example, gamma correction, white balance processing, sharpness processing, and gradation conversion processing. The image processing unit 104 may also include a processor that performs processing based on software. The image processing unit 104 includes an HDR processing unit 104A.

[0034] The HDR processing unit 104A performs HDR processing based on the pixel signals read from the solid-state imaging device 102. For example, the HDR processing unit 104A can generate an HDR image by combining the pixel signals read from low-light pixels, medium-light pixels, and high-light pixels. In this case, the HDR processing unit 104A can generate an HDR image based, for example, on the sum of the pixel signals read multiple times in each frame for medium-light pixels and high-light pixels.

[0035] The storage unit 105 stores images captured by the solid-state imaging device 102, as well as imaging parameters of the solid-state imaging device 102. The storage unit 105 can also store programs that operate the imaging device 100 based on software. The storage unit 105 may include ROM (Read Only Memory), RAM (Random Access Memory), and a memory card.

[0036] The display unit 106 displays captured images and various information to support the imaging operation. The display unit 106 may be a liquid crystal display, an organic EL (Electro Luminescence) display, or a micro LED display.

[0037] The operation unit 107 provides a user interface for operating the imaging device 100. The operation unit 107 may include, for example, buttons, dials, and switches provided on the imaging device 100. The operation unit 107 may be configured as a touch panel together with the display unit 106.

[0038] The drive control unit 109 controls the drive of the optical system 101 based on the pixel signals read from the solid-state imaging device 102 and the operation information operated by the operation unit 107. For example, the drive control unit 109 can perform manual focus or control the zoom magnification based on the operation information operated by the operation unit 107.

[0039] Note that depending on the form of the imaging device 100, some of the above functions may not be necessary, or conversely, it may further have functions not disclosed.

[0040] FIG. 2 is a block diagram showing a configuration example of a solid-state imaging device according to the first embodiment.

[0041] In the figure, the solid-state imaging device 102 includes a pixel array unit 111, a vertical scanning circuit 112, a column readout circuit 113, a column signal processing unit 114, a horizontal scanning circuit 115, a pixel signal storage unit 116, an addition unit 117, a processing unit 118, a vertical drive circuit 119, and a reference signal generation unit 130.

[0042] The pixel array unit 111 includes a plurality of pixels 120. The pixels 120 are arranged in a matrix along the row direction (also referred to as the horizontal direction) and the column direction (also referred to as the vertical direction). The pixels 120 may share 8 pixels, or may share 4 pixels, or may share 2 pixels, or may be composed of a single pixel. The pixel 120 can form a source follower with the column readout circuit 113 when reading a signal. Each pixel 120 is connected to a horizontal control line 131 in the row direction, and is connected to a vertical signal line 132 and a vertical control line 133 in the column direction. The horizontal control line 131 drives each pixel 120 horizontally when reading a signal from each pixel 120. The vertical signal line 132 transmits a potential based on the current flowing when reading a signal from the pixel 120 vertically to the column signal processing unit 114. The vertical control line 133 drives each pixel 120 vertically when reading a pixel signal from each pixel 120.

[0043] Pixel 120 may be configured as a Bayer array or as a quad Bayer array. The light received by pixel 120 may be visible light, near-infrared light (NIR), short-wavelength infrared light (SWIR), ultraviolet light, or X-rays, etc.

[0044] The vertical scanning circuit 112 scans the pixels 120 to be read out vertically row by row via the horizontal control lines 131. The vertical scanning circuit 112 may be configured using vertical registers. The vertical scanning circuit 112 may include an address decoder, or it may include a driver that drives the horizontal control lines 131 selected via the address decoder row by row.

[0045] The column readout circuit 113 can configure a source follower with each pixel 120 when reading a signal from the pixel 120. The signal from the pixel 120 may include a sense signal that detects the amount of charge accumulated in the pixel 120 and a pixel signal read from the pixel 120. At this time, the column readout circuit 113 can change the potential of the vertical signal line 132 based on the charge held in the pixel 120. The charge held in the pixel 120 includes the charge accumulated in the photoelectric conversion unit and the charge transferred to the floating diffusion.

[0046] The column signal processing unit 114 processes signals transmitted vertically from the pixels 120. For example, the column signal processing unit 114 can perform correlated double sampling (CDS) based on signals transmitted vertically from the pixels 120. Furthermore, the column signal processing unit 114 can perform analog-to-digital (AD) conversion based on signals transmitted vertically from each pixel 120 and output a pixel signal.

[0047] The column signal processing unit 114 includes a column ADC unit 114A. The column ADC unit 114A can perform AD conversion processing in parallel for each column. At this time, the column ADC unit 114A can perform AD conversion for each column based on the comparison result between the pixel signal read from the pixel 120 and the reference signal.

[0048] The column ADC unit 114A includes a determination unit 114B. Based on the PD potential, the determination unit 114B instructs the reading of a pixel signal corresponding to the FD potential. At this time, if the FD potential is at a threshold level below the saturation level, the determination unit 114B can instruct the vertical scanning circuit 112 to drive the pixel 120 horizontally, instruct the vertical drive circuit 119 to drive the pixel 120 vertically, and instruct the column ADC unit 114A to perform AD conversion.

[0049] The horizontal scanning circuit 115 scans the pixels 120 to be read out in the row direction. The horizontal scanning circuit 115 may be configured using a horizontal register.

[0050] The pixel signal storage unit 116 stores the AD conversion value of the pixel signal read from each pixel 120. The pixel signal storage unit 116 also stores the sum of the AD conversion values ​​of the pixel signals for each pixel 120 for each frame. For example, the pixel signal storage unit 116 can store the AD conversion value of the pixel signal read only once in each frame for each low-light pixel, or the sum of the AD conversion values ​​of the pixel signals read multiple times in each frame for medium-light and high-light pixels.

[0051] The adder 117 adds the AD conversion values ​​of the pixel signals for each pixel 120 in each frame. For example, the adder 117 can add the AD conversion values ​​of the pixel signals that have been read multiple times in each frame for medium-illumination pixels and high-illumination pixels.

[0052] The processing unit 118 stores the sum of the AD conversion values ​​of the pixel signals output from the adder 117 in the pixel signal storage unit 116 for each pixel 120. The processing unit 118 may be a processor such as a CPU (Central Processing Unit) or it may be composed of logic circuits.

[0053] The vertical drive circuit 119 drives the pixels 120 to be read out column by column via the vertical control line 133.

[0054] The reference signal generation unit 130 supplies a reference signal REF, which is compared with the pixel signal, to the column ADC unit 114A. The reference signal REF may include a ramp wave. The ramp wave can be set for each P-phase and D-phase. The reference signal generation unit 130 also supplies a reference signal STL, which is compared with the sense signal, to the column ADC unit 114A. This reference signal STL can be set to a threshold level below the saturation level of each photodiode PD. This threshold level can be set to a potential below the effect of variations in the depletion voltage of each photodiode PD.

[0055] Figure 3 shows an example of a pixel circuit configuration provided in a solid-state imaging device according to the first embodiment.

[0056] In the figure, pixel 120 includes a photodiode PD, transfer transistors TGV and TGH, reset transistor 121, amplification transistor 122, selection transistor 123, and switching transistor 124. Pixel 120 also includes a threshold modulation transistor 125, selection transistor 126, and floating diffusion transistors FD1 and FD2. The transfer transistors TGV and TGH, reset transistor 121, amplification transistor 122, selection transistors 123 and 126, and switching transistor 124 may be MOS (Metal Oxide Semiconductor) transistors. The threshold modulation transistor 125 and selection transistor 126 are examples of the sense unit described in the claims. The amplification transistor 122 and selection transistor 123 are examples of the output unit described in the claims.

[0057] The photodiode PD performs photoelectric conversion and stores the photoelectrically converted charge. The photodiode PD is an example of the photoelectric conversion unit described in the claims. Transfer transistors TGV and TGH transfer the charge stored in the photodiode PD to the floating diffusion FD1. At this time, transfer transistor TGV is used to transfer the charge of the pixel 120 selected in the column direction. Transfer transistor TGH is used to transfer the charge of the pixel 120 selected in the row direction. Reset transistor 121 resets the floating diffusion FD1 and FD2. At this time, the floating diffusion FD2 can be placed between the reset transistor 121 and the switching transistor 124. Amplifying transistor 122 outputs a signal that follows the potential corresponding to the amount of charge transferred to the floating diffusion FD1 and FD2. The threshold modulation transistor 125 modulates the threshold based on the amount of charge stored in the photodiode PD. A portion of the channel region of the threshold modulation transistor 125 may be positioned over the charge storage region of the photodiode PD. In this case, the threshold modulation transistor 125 can detect the PD potential of the photodiode PD without transferring the charge stored in the photodiode PD to the floating diffusion FD1, thereby achieving non-destructive readout. The selection transistor 123 selects the output of the amplification transistor 122. The selection transistor 126 selects the output of the threshold modulation transistor 125.

[0058] The switching transistor 124 switches the conversion efficiency of the amplification transistor 122. At this time, the switching transistor 124 can switch the capacitance added to the gate of the amplification transistor 122. For example, the switching transistor 124 can set the conversion efficiency of the pixel 120 to LCG (Low Conversion Gain) by adding the capacitance of floating diffusion FD2 to floating diffusion FD1. Alternatively, the switching transistor 124 can set the conversion efficiency of the pixel 120 to HCG (High Conversion Gain) by disconnecting floating diffusion FD2 from floating diffusion FD1.

[0059] The transfer transistors TGV and TGH are connected in series between the cathode of the photodiode PD and the floating diffusion FD1. The amplification transistor 122 and the selection transistor 123 are connected in series. The threshold modulation transistor 125 and the selection transistor 126 are connected in series. The drains of the amplification transistor 122 and the threshold modulation transistor 125 are connected to the power supply potential VDD. The gate of the amplification transistor 122 is connected to the floating diffusion FD1. The gate of the threshold modulation transistor 125 is connected to the cathode of the photodiode PD. The sources of each selection transistor 123 and 126 are connected to the vertical signal line 132.

[0060] The reset transistor 121 is connected between the floating diffusion FD2 and the power supply potential VDD. The switching transistor 124 is connected between the floating diffusion FD1 and FD2.

[0061] The transfer signal TGLV is applied to the gate of the transfer transistor TGV. The transfer signal TGLH is applied to the gate of the transfer transistor TGH. The reset signal RST is applied to the gate of the reset transistor 121. The selection signal SEL1 is applied to the gate of the selection transistor 123. The selection signal SEL2 is applied to the gate of the selection transistor 126. The switching signal FDG is applied to the gate of the switching transistor 124. The transfer signal TGLH, the reset signal RST, the selection signals SEL1, SEL2, and the switching signal FDG can be transmitted to the pixel 120 via the horizontal control line 131. The transfer signal TGLV can be transmitted to the pixel 120 via the vertical control line 133.

[0062] Figure 4 is a block diagram showing an example of the configuration of the AD conversion unit according to the first embodiment.

[0063] In the figure, a constant current source 142 is connected to the vertical signal line 132 for each column. Each constant current source 142 can be supplied with a reference current generated by the reference current generation circuit 141. When sensing the amount of charge accumulated in each pixel 120, the constant current source 142 can form a source follower with the threshold modulation transistor 125 via the vertical signal line 132. When reading out the pixel signal from each pixel 120, the constant current source 142 can form a source follower with the amplification transistor 122 via the vertical signal line 132. The selector 143 can switch the current generated by the constant current source 142.

[0064] The column ADC unit 114A includes a comparator CP, a counter CN, and a determination unit 114B for each column.

[0065] The comparator CP compares the sense signal transmitted via the vertical signal line 132 when sensing the charge amount accumulated in each pixel 120 with the reference signal STL. The comparator CP also compares the pixel signal transmitted via the vertical signal line 132 when reading the pixel signal from each pixel 120 with the reference signal REF. The non-inverting input of the comparator CP is connected to the vertical signal line 132, and the inverting input of the comparator CP is connected to the reference signal generation unit 130.

[0066] The determination unit 114B instructs the reading of the pixel signal from the pixel 120 based on the comparison result between the sense signal and the reference signal STL at the comparator CP. At this time, the determination unit 114B can instruct the comparator CP to perform AD conversion or instruct the selector 143 to switch the current from the constant current source 142.

[0067] The counter CN performs a counting operation based on the output timing of the comparison result of the comparator CP. Then, based on the count value generated by the counting operation, the counter CN digitizes the pixel signal transmitted via the vertical signal line 132 and outputs it.

[0068] Figure 5 is a flowchart showing an example of operation of an imaging device according to the first embodiment. In this figure, an example is shown in which the pixel array 111 includes both low-light pixels and high-light pixels in a given frame.

[0069] In the figure, the vertical scanning circuit 112 drives each pixel 120 row by row and reads a sense signal from each pixel 120 (S101). The sense signal can be read out non-destructively from each pixel 120.

[0070] Next, the determination unit 114B determines whether the charge accumulated in the photodiode PD of the pixel 120 is saturated based on the sense signal read from the pixel 120 (S102).

[0071] Next, the vertical scanning circuit 112 and the vertical drive circuit 119 select a pixel 120 whose charge accumulated in the photodiode PD is saturated, based on the determination result of the determination unit 114B (S103). This selection of pixel 120 can be performed based on XY addressing. In XY addressing, the vertical scanning circuit 112 can select a row containing the pixel 120 to be read, and the vertical drive circuit 119 can select a column containing the pixel 120 to be read. In the figure, an example is shown where the pixel 120 to be read is a high-illumination pixel, and the pixel 120 not to be read is a low-illumination pixel.

[0072] Next, the vertical scanning circuit 112 turns off the transfer transistor TGH for the row containing the low-light pixels, and the vertical drive circuit 119 turns off the transfer transistor TGV for the column containing the low-light pixels (S104).

[0073] Furthermore, the vertical scanning circuit 112 turns on the transfer transistor TGH of the row containing the high-illuminance pixels, and the vertical drive circuit 119 turns on the transfer transistor TGV of the column containing the high-illuminance pixels (S105).

[0074] Next, the pixel signal is read from the high-intensity pixel via the vertical signal line 132 (S106). At this time, the amplification transistor 122 of the high-intensity pixel can form a source follower with the constant current source 142 via the vertical signal line 132.

[0075] Next, the determination unit 114B deactivates the AD conversion of the column containing the low-illumination pixels (S107) and activates the AD conversion of the column containing the high-illumination pixels (S108). At this time, the determination unit 114B can deactivate the comparator CP and counter CN of the column containing the low-illumination pixels and activate the comparator CP and counter CN of the column containing the high-illumination pixels.

[0076] Next, the pixel signal storage unit 116 stores the AD conversion value of the pixel signal read from the high-illumination pixel. Then, the adder 117 adds the AD conversion values ​​of the pixel signals of the high-illumination pixels that have been read multiple times in each frame for each high-illumination pixel (S109). Then, the processing unit 118 stores the sum of the AD conversion values ​​of the pixel signals of the high-illumination pixels in the pixel signal storage unit 116 for each high-illumination pixel.

[0077] Figure 6 is a timing chart showing an example of operation when a saturation state is detected in the imaging device according to the first embodiment. In this figure, an example is shown in which the pixel array 111 contains low-light pixels, medium-light pixels, and high-light pixels in a given frame. In this figure, an example is shown in which the first accumulation is performed in a single frame for low-light pixels, and the first accumulation is performed in a single frame, after which the second accumulation is performed after the first accumulation reaches the threshold level. In an example in which the fourth accumulation is performed in a single frame for high-light pixels, after the first to third accumulations, each of which reach the threshold level.

[0078] In the figure, for low-light pixels, medium-light pixels, and high-light pixels, charge accumulation for each frame begins after a PD / FD reset is performed. During the PD / FD reset, the transfer transistors TGH, TGV and reset transistor 121 are turned on, and the photodiode PD and floating diffusion diodes FD1 and FD2 are reset.

[0079] Then, PD reading and FD reset are performed multiple times within one frame. During PD reading, the selection transistor 126 is turned on, and the sense signal detected by the threshold modulation transistor 125 is read. At this time, the sense signal is input to the determination unit 114B via the vertical signal line 132. The determination unit 114B then determines whether the charge accumulated in the photodiode PD has reached a threshold level.

[0080] Next, for high-illumination pixels, if it is determined that the charge accumulated in the photodiode PD has reached a threshold level based on the first PD readout, an FD readout is performed. During the FD readout, the transfer transistors TGV, TGH and selection transistor 123 are turned on, the charge accumulated in the photodiode PD is transferred to the floating diffusion FD1, and the pixel signal detected by the amplification transistor 122 is read out. After that, the second charge accumulation for the high-illumination pixels begins. For medium-illumination pixels and low-illumination pixels, the first accumulation continues without transferring the charge accumulated in the photodiode PD to the floating diffusion FD1.

[0081] Next, for high-illumination and medium-illumination pixels, if it is determined that the charge accumulated in the photodiode PD has reached a threshold level based on the second PD readout, an FD readout is performed. Subsequently, the third charge accumulation for the high-illumination pixels begins, and the second charge accumulation for the medium-illumination pixels begins. For low-illumination pixels, the first accumulation continues without the charge accumulated in the photodiode PD being transferred to the floating diffusion FD1.

[0082] Next, for high-illumination pixels, if it is determined that the charge accumulated in the photodiode PD has reached a threshold level based on the third PD readout, an FD readout is performed. Subsequently, the fourth charge accumulation for the high-illumination pixels begins. For medium-illumination pixels, the second accumulation continues without the charge accumulated in the photodiode PD being transferred to the floating diffusion FD1. For low-illumination pixels, the first accumulation continues without the charge accumulated in the photodiode PD being transferred to the floating diffusion FD1.

[0083] Next, for the high-illumination, medium-illumination, and low-illumination pixels, FD (floppy disk) readout is performed after the FD reset. After that, a PD / FD reset is performed.

[0084] Figure 7 shows the relationship between exposure time and signal level for each illuminance level of the imaging device according to the first embodiment. In this figure, the relationship between exposure time and signal level is shown for low-illumination pixels, medium-illumination pixels, and high-illumination pixels in Figure 6.

[0085] In the figure, for high-illumination and medium-illumination pixels, the pixel signal is read out before the charge accumulated in the photodiode PD saturates. At this time, the pixel signals read out before the charge accumulated in the photodiode PD saturates are added and stored, thereby expanding the dynamic range of the high-illumination and medium-illumination pixels. Thus, even when the illuminance of the high-illumination and medium-illumination pixels is high, charge can be constantly accumulated in the high-illumination and medium-illumination pixels without causing overflow, enabling HDR while achieving LMF (LED Flicker Mitigation).

[0086] Figure 8 shows an example of the waveform during operation of a low-light pixel according to the first embodiment. In this figure, an example is shown in which the conversion efficiency of the low-light pixel is switched between two stages.

[0087] In the figure, the selection signals SEL1 and SEL2, the reset signal RST, the switching signal FDG, and the transfer signals TGLV and TGLH rise before the accumulation of low-light pixels begins. At this time, the selection transistors 123 and 126, the reset transistor 121, the switching transistor 124, and the transfer transistors TGH and TGV are turned on, and the photodiode PD and floating diffusion FD1 and FD2 are reset. Also, the conversion efficiency of the low-light pixels is set to LCG.

[0088] Next, the selection signals SEL1 and SEL2, the reset signal RST, and the transfer signals TGLV and TGLH fall. At this time, the selection transistors 123 and 126, the reset transistor 121, and the transfer transistors TGH and TGV are turned off, and the first accumulation of low-light pixels begins.

[0089] Next, during the first accumulation period of the low-light pixel, the selection signal SEL2 rises. At this time, the selection transistor 126 turns on, and PD readout 1 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on, and the floating diffusion FD1 and FD2 are reset. After that, the reset signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the low-light pixel is not at the saturation level, the first accumulation is continued.

[0090] Next, the selection signal SEL2 rises during the first accumulation period of the low-light pixel. At this time, the selection transistor 126 turns on and PD readout 2 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on and the floating diffusion FD1 and FD2 are reset. After that, the set signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the low-light pixel is not at the saturation level, the first accumulation is continued.

[0091] Next, the selection signal SEL2 rises during the first accumulation period of the low-light pixel. At this time, the selection transistor 126 turns on and PD readout 3 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on and the floating diffusion FD1 and FD2 are reset. After that, the set signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the low-light pixel is not at the saturation level, the first accumulation is continued.

[0092] Next, when the first storage period ends, the reset signal RST rises. At this time, the reset transistor 121 is turned on, and the photodiode PD and floating diffusions FD1 and FD2 are reset. Then, the reset signal RST and the switching signal FDG fall, and the reset transistor 121 and the switching transistor 124 are turned off. At this time, the conversion efficiency of the low-light pixels is set to HCG, and HCGP phase readout is performed.

[0093] Next, the switching signal FDG rises, and the switching transistor 124 is turned on. At this time, the conversion efficiency of the low-light pixels is set to LCG, and LCGP phase readout is performed.

[0094] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the low-light pixels is transferred to the floating diffusion FD1, and LCGD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off.

[0095] Next, the switching signal FDG falls, and the switching transistor 124 is turned off. At this time, the conversion efficiency of the low-light pixels is set to HCG, and HCGD phase readout is performed.

[0096] Figure 9 shows an example of the waveform during operation of a medium-illuminance pixel according to the first embodiment. In this figure, an example is shown in which the conversion efficiency of the medium-illuminance pixel is switched between two stages.

[0097] In the figure, the selection signals SEL1 and SEL2, the reset signal RST, the switching signal FDG, and the transfer signals TGLV and TGLH rise before the accumulation of the medium-intensity pixels begins. At this time, the selection transistors 123 and 126, the reset transistor 121, the switching transistor 124, and the transfer transistors TGH and TGV are turned on, and the photodiode PD and floating diffusion FD1 and FD2 are reset. Also, the conversion efficiency of the medium-intensity pixels is set to LCG.

[0098] Next, the selection signals SEL1 and SEL2, the reset signal RST, and the transfer signals TGLV and TGLH fall. At this time, the selection transistors 123 and 126, the reset transistor 121, and the transfer transistors TGH and TGV are turned off, and the first accumulation of the medium-light pixels begins.

[0099] Next, during the first accumulation period of the medium-intensity pixel, the selection signal SEL2 rises. At this time, the selection transistor 126 turns on, and PD readout 1 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on, and the floating diffusion FD1 and FD2 are reset. After that, the reset signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the medium-intensity pixel is not at the saturation level, the first accumulation is continued.

[0100] Next, during the first accumulation period of the medium-illumination pixel, the selection signal SEL2 rises. At this time, the selection transistor 126 turns on, and PD readout 2 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on, and the floating diffusion FD1 and FD2 are reset. After that, the set signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the medium-illumination pixel has reached a threshold level, FDP phase readout is performed.

[0101] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the medium-light pixels is transferred to the floating diffusion FD1, and FDD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off. At this time, the second accumulation of charge in the medium-light pixels begins.

[0102] Next, the selection signal SEL2 rises during the second accumulation period of the medium-illumination pixel. At this time, the selection transistor 126 turns on, and PD readout 3 is performed. If it is determined that the charge accumulated in the photodiode PD of the medium-illumination pixel is not at the saturation level, the second accumulation continues.

[0103] Next, when the second storage period ends, the reset signal RST rises. At this time, the reset transistor 121 is turned on, and the photodiode PD and floating diffusions FD1 and FD2 are reset. Then, the reset signal RST and the switching signal FDG fall, and the reset transistor 121 and the switching transistor 124 are turned off. At this time, the conversion efficiency of the medium-illumination pixels is set to HCG, and HCGP phase readout is performed.

[0104] Next, the switching signal FDG rises, and the switching transistor 124 is turned on. At this time, the conversion efficiency of the medium-illumination pixels is set to LCG, and LCGP phase readout is performed.

[0105] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the medium-intensity pixels is transferred to the floating diffusion FD1, and LCGD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off.

[0106] Next, the switching signal FDG falls, and the switching transistor 124 is turned off. At this time, the conversion efficiency of the medium-intensity pixels is set to HCG, and HCGD phase readout is performed.

[0107] Figure 10 shows an example of a waveform during operation of a high-intensity pixel according to the first embodiment. In this figure, an example is shown in which the conversion efficiency of the high-intensity pixel is switched between two stages.

[0108] In the figure, the selection signals SEL1 and SEL2, the reset signal RST, the switching signal FDG, and the transfer signals TGLV and TGLH rise before the accumulation of high-intensity pixels begins. At this time, the selection transistors 123 and 126, the reset transistor 121, the switching transistor 124, and the transfer transistors TGH and TGV are turned on, and the photodiode PD and floating diffusion FD1 and FD2 are reset. Also, the conversion efficiency of the high-intensity pixels is set to LCG.

[0109] Next, the selection signals SEL1 and SEL2, the reset signal RST, and the transfer signals TGLV and TGLH fall. At this time, the selection transistors 123 and 126, the reset transistor 121, and the transfer transistors TGH and TGV are turned off, and the first accumulation of high-intensity pixels begins.

[0110] Next, the selection signal SEL2 rises during the first accumulation period of the high-intensity pixel. At this time, the selection transistor 126 turns on and PD readout 1 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on and the floating diffusion FD1 and FD2 are reset. After that, the reset signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the high-intensity pixel has reached a threshold level, FDP phase readout is performed.

[0111] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the medium-intensity pixels is transferred to the floating diffusion FD1, and FDD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off. At this time, the second accumulation of charge in the high-intensity pixels begins.

[0112] Next, during the second accumulation period of the high-intensity pixel, the selection signal SEL2 rises. At this time, the selection transistor 126 turns on, and PD readout 2 is performed. Then, after the selection signal SEL2 falls, the selection signal SEL1 and the reset signal RST rise. At this time, the selection transistor 123 and the reset transistor 121 turn on, and the floating diffusion FD1 and FD2 are reset. After that, the set signal RST falls, and the reset transistor 121 turns off. At this point, if it is determined that the charge accumulated in the photodiode PD of the high-intensity pixel has reached a threshold level, FDP phase readout is performed.

[0113] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the medium-intensity pixels is transferred to the floating diffusion FD1, and FDD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off. At this time, the third accumulation of charge in the high-intensity pixels begins.

[0114] Next, the selection signal SEL2 rises during the third accumulation period of the high-illumination pixel. At this time, the selection transistor 126 turns on, and PD readout 3 is performed. If it is determined that the charge accumulated in the photodiode PD of the high-illumination pixel has reached a threshold level, FDP phase readout is performed.

[0115] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the medium-intensity pixels is transferred to the floating diffusion FD1, and FDD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off. At this time, the fourth accumulation of charge in the high-intensity pixels begins.

[0116] Next, when the fourth storage period ends, the reset signal RST rises. At this time, the reset transistor 121 is turned on, and the photodiode PD and floating diffusions FD1 and FD2 are reset. Then, the reset signal RST and the switching signal FDG fall, and the reset transistor 121 and the switching transistor 124 are turned off. At this time, the conversion efficiency of the high-intensity pixels is set to HCG, and HCGP phase readout is performed.

[0117] Next, the switching signal FDG rises, and the switching transistor 124 is turned on. At this time, the conversion efficiency of the high-intensity pixels is set to LCG, and LCGP phase readout is performed.

[0118] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in the high-intensity pixels is transferred to the floating diffusion FD1, and LCGD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off.

[0119] Next, the switching signal FDG falls, and the switching transistor 124 is turned off. At this time, the conversion efficiency of the high-intensity pixels is set to HCG, and HCGD phase readout is performed.

[0120] Figure 11 is a plan view showing an example of a pixel layout according to the first embodiment.

[0121] In the figure, an element isolation region ISA is formed on the semiconductor substrate SUB to isolate the active region AK from the element. The material of the semiconductor substrate SUB may be Si, GaAs, SiC, GaN, InGaAs, or InP, etc. The element isolation region ISA may be STI (Shallow Trench Isolation). The active region AK can be used to form the channel region, source / drain layer, and contacts of a photodiode PD or pixel transistor.

[0122] On the semiconductor substrate SUB, gate electrodes G1 to G4 and G11 to G14 are formed at positions that cross the active region AK. Gate electrode G1 can be used in the threshold modulation transistor 125. Gate electrode G2 can be used in the selection transistor 126. Gate electrode G3 can be used in the amplification transistor 122. Gate electrode G4 can be used in the selection transistor 123. Gate electrode G11 can be used in the reset transistor 121. Gate electrode G12 can be used in the switching transistor 124. Gate electrode G13 can be used in the transfer transistor TGH. Gate electrode G14 can be used in the transfer transistor TGV.

[0123] The gate electrodes G1 and G2 can be placed adjacent to each other. Furthermore, gate electrode G1 can be placed adjacent to the photodiode PD. The gate electrodes G3 and G4 can be placed adjacent to each other. The gate electrodes G11 and G12 can be placed adjacent to each other. The gate electrodes G13 and G14 can be placed adjacent to each other. A floating diffusion FD1 is formed between the channel regions beneath the gate electrodes G12 and G13.

[0124] Figure 12 is a cross-sectional view showing an example of the configuration of the pixel sense section according to the first embodiment. Note that the figure shows the configuration cut along the line A1-A2 in Figure 11.

[0125] In the figure, a gate electrode G1 is formed on the semiconductor substrate SUB via a gate insulating film GZ. In the semiconductor substrate SUB, on one side of the channel region below the gate electrode G1, N ＋ A type impurity diffusion layer DF5 is formed. ＋ The impurity diffusion layer DF5 may be in contact with the device isolation region ISA. Also, in the semiconductor substrate SUB, N is located under the gate electrode G1. - A type impurity diffusion layer DF3 is formed. - The impurity diffusion layer DF3 can be placed in the channel region of the threshold modulation transistor 125.

[0126] Furthermore, the semiconductor substrate SUB has N - A type impurity diffusion layer DF1 is formed. - The N-type impurity diffusion layer DF1 can be embedded in the semiconductor substrate SUB. Furthermore, an N-type impurity diffusion layer DF2 is formed on the semiconductor substrate SUB. The N-type impurity diffusion layer DF2 is formed on the semiconductor substrate SUB and N - It can be placed across the type impurity diffusion layer DF1. - The N-type impurity diffusion layer DF1 and the N-type impurity diffusion layer DF2 can be used in the photodiode PD. In this case, a portion of the charge storage region of the photodiode PD can be placed below the channel region of the threshold modulation transistor 125. This allows the threshold of the threshold modulation transistor 125 to be modulated based on the charge stored in the photodiode PD.

[0127] Figure 13 shows an example of the potential of the sense portion of a pixel according to the first embodiment. In this figure, an example of the potential of the photodiode PD and the threshold modulation transistor 125 is shown.

[0128] In the figure, the PD potential VPD of the photodiode PD is applied to the gate electrode G1 of the threshold modulation transistor 125. At this time, the potential PT of the channel region of the threshold modulation transistor 125 changes according to the charge EL accumulated in the photodiode PD. Therefore, the threshold voltage Vth of the threshold modulation transistor 125 changes according to the charge EL accumulated in the photodiode PD. At this time, based on the source follower operation of the threshold modulation transistor 125, the source potential Vf of the threshold modulation transistor 125 follows the threshold voltage Vth of the threshold modulation transistor 125. Therefore, by monitoring the source potential Vf of the threshold modulation transistor 125, the charge EL accumulated in the photodiode PD can be detected.

[0129] FIG. 14 is a cross-sectional view showing a configuration example of the transfer portion of the pixel according to the first embodiment. In the figure, the configuration cut along the line B1 - B2 in FIG. 11 is shown.

[0130] In the figure, gate electrodes G11 to G14 are formed separately on the semiconductor substrate SUB via a gate insulating film GZ.

[0131] In the semiconductor substrate SUB, N ＋ type impurity diffusion layers DF4 are formed on both sides of the gate electrode G11 and between the gate electrodes G12 and G13, and the N ＋ type impurity diffusion layer DF4 on one side of the gate electrode G11 is connected to the power supply potential VDD. Also, in the semiconductor substrate SUB, N-type impurity diffusion layers DF6 are formed under each of the gate electrodes G11 to G13.

[0132] N - A part of the type impurity diffusion layer DF1 reaches the surface of the semiconductor substrate SUB under the gate electrode G14. Also, a part of the N-type impurity diffusion layer DF2 reaches the surface of the semiconductor substrate SUB under the gate electrode G14.

[0133] FIG. 15 is a diagram showing an example of the potential of the transfer portion of the pixel according to the first embodiment.

[0134] In figure a, when the conversion efficiency is set to LCG, the switching transistor 124 turns on, and the floating diffusion transistors FD1 and FD2 are connected to each other. When pixel 120 is not selected, the transfer transistors TGV and TGH turn off, and the charge EL accumulated in the photodiode PD is not transferred to the floating diffusion transistors FD1 and FD2.

[0135] In figure b, when pixel 120 is selected, the transfer transistors TGV and TGH are turned on, and the charge EL accumulated in the photodiode PD is transferred to the floating diffusion diodes FD1 and FD2.

[0136] Figure 16 is a diagram showing the relationship between the light intensity of a pixel and the PD potential and FD potential according to the first embodiment.

[0137] In the figure, the PD potential (threshold modulation output) decreases as the light intensity increases, and becomes constant when it reaches the saturation level LV3. At this time, the output of the threshold modulation transistor 125 follows the PD potential, decreasing from the starting level LV1 as the light intensity increases, and becoming constant when it reaches the saturation level LV2.

[0138] Figure 17 shows an example of the PD potential for each pixel according to the first embodiment.

[0139] In the figure, the maximum storage capacity CM of the photodiode PD varies for each pixel 120. Also, the offset capacity CF of the photodiode PD varies for each pixel 120. In this case, the threshold level TH1 used to determine the saturation level of the photodiode PD can be set to be below the influence of the variations in the maximum storage capacity CM and offset capacity CF of the photodiode PD. This allows for the determination of whether the stored charge CS of the photodiode PD is saturated while eliminating the influence of the variations in the maximum storage capacity CM and offset capacity CF of the photodiode PD.

[0140] Figure 18 shows an example of a circuit configuration for a dummy pixel provided in a solid-state imaging device according to the first embodiment.

[0141] In the figure, a current source 134 is connected to the vertical signal line 132 for each column. The current source 134 may also be a constant current source 142. In this case, the amplifying transistor 122 and the threshold modulation transistor 125 can form a source follower with the current source 134 via the vertical signal line 132.

[0142] Furthermore, a dummy source follower circuit 151 is connected to the vertical signal line 132. The dummy source follower circuit 151 includes a source follower transistor 152 and a selection transistor 153.

[0143] The source follower transistor 152 and the selection transistor 153 are connected in series. The drain of the source follower transistor 152 is connected to the power supply potential VDD. The source of the selection transistor 153 is connected to the vertical signal line 132. A reference voltage VRF is applied to the gate of the source follower transistor 152. A selection signal SEL3 is applied to the gate of the selection transistor 153.

[0144] When the selection signal SEL3 is turned on, the source follower transistor 152 can form a source follower with the current source 134 via the vertical signal line 132. At this time, the determination unit 114B can obtain a reference voltage VRF from the dummy source follower circuit 151 and use the reference voltage VRF to determine the saturation level.

[0145] As described above, in the first embodiment, the FD potential is read out based on the sensing result of the PD potential, and the pixel signal based on the FD potential is added and stored. This makes it possible to transfer the accumulated charge of the photodiode PD to the floating diffusion FD1 based on the determination result of the PD potential based on non-destructive readout, read out the pixel signal, and integrate that pixel signal. Therefore, it becomes unnecessary to perform destructive readout to determine the saturation of the charge accumulated in the pixel 120, and it becomes possible to realize HDR while suppressing the decrease in signal amount.

[0146] <2. Second Embodiment> In the first embodiment described above, the FD potential was read out based on the sensing result of the PD potential, and the pixel signal based on the FD potential was added and stored. In this second embodiment, the pixel signal read out based on the difference in PD potential between frames is used for event determination.

[0147] Figure 19 is a block diagram showing an example configuration of a solid-state imaging device according to the second embodiment.

[0148] In the figure, the solid-state imaging device 202 includes a pixel array unit 211, a column signal processing unit 214, a processing unit 218, and a reference signal generation unit 230, instead of the pixel array unit 111, column signal processing unit 114, processing unit 118, and reference signal generation unit 130 of the first embodiment described above. Furthermore, the solid-state imaging device 202 has a sense signal storage unit 216 added to the solid-state imaging device 102 of the first embodiment described above. In addition, the addition unit 117 and vertical drive circuit 119 are removed from the solid-state imaging device 102 of the first embodiment described above. In this case, the solid-state imaging device 202 may be used as an EVS (Event-based Vision Sensor). The other configurations of the solid-state imaging device 202 are the same as those of the solid-state imaging device 102 of the first embodiment described above.

[0149] The pixel array section 211 comprises a plurality of pixels 220. The pixels 220 are arranged in a matrix along the row and column directions. The pixels 220 may be shared by 8 pixels, 4 pixels, 2 pixels, or consist of a single pixel. The pixels 220 can form a source follower with the column readout circuit 113 when reading a signal. Each pixel 220 is connected to a horizontal control line 131 in the row direction and to a vertical signal line 132 in the column direction.

[0150] The column signal processing unit 214 processes signals transmitted vertically from the pixels 220. For example, the column signal processing unit 214 can perform CDS processing based on signals transmitted vertically from the pixels 220. Furthermore, the column signal processing unit 214 can perform AD conversion processing based on signals transmitted vertically from each pixel 220 and output a pixel signal.

[0151] The column signal processing unit 214 includes a column ADC unit 214A. The column ADC unit 214A can perform AD conversion processing in parallel for each column. At this time, the column ADC unit 214A can perform AD conversion of the pixel signal for each column based on the comparison result between the pixel signal read from the pixel 220 and the reference signal REF. In addition, the column ADC unit 214A can perform AD conversion of the sense signal for each column based on the comparison result between the sense signal read from the pixel 220 and the reference signal RES.

[0152] The sense signal storage unit 216 stores the AD conversion value of the sense signal read from each pixel 220. The sense signal storage unit 216 includes a determination unit 216A. The determination unit 216A instructs the reading of the pixel signal corresponding to the FD potential based on the difference in PD potential between frames. At this time, if there is a difference in PD potential between frames, the determination unit 216A can instruct the vertical scanning circuit 112 to drive the pixel 220 horizontally and instruct the column ADC unit 214A to perform an AD conversion operation.

[0153] The processing unit 218 performs output processing of the AD conversion value of the pixel signal stored in the pixel signal storage unit 116.

[0154] The reference signal generation unit 230 supplies a reference signal REF, which is compared with the pixel signal, and a reference signal RES, which is compared with the sense signal, to the column ADC unit 214A. Each of the reference signals REF and RES may include a ramp wave.

[0155] Figure 20 shows an example of a pixel circuit configuration provided in a solid-state imaging device according to the second embodiment.

[0156] In the figure, pixel 220 is the same as pixel 120 in the first embodiment described above, but with the transfer transistor TGV removed. In this case, the transfer transistor TGH is connected between the photodiode PD and the floating diffusion FD1. The other configurations of pixel 220 are the same as those of pixel 120 in the first embodiment described above.

[0157] Figure 21 is a flowchart showing an example of operation of the imaging device according to the second embodiment.

[0158] In the figure, the vertical scanning circuit 112 drives the pixels 220 row by row and reads out the sense signal from the pixels 220 (S201).

[0159] Next, the column ADC unit 214A performs AD conversion on the sense signal read from the pixel 220 and stores it in the sense signal storage unit 216 (S202).

[0160] Next, the determination unit 216A reads the AD conversion values ​​of the sense signals for multiple frames from the sense signal storage unit 216 and determines whether there is a difference between frames in the sense signals (S203).

[0161] Then, if there is no difference in the sense signals between frames, the determination unit 216A instructs not to read out the pixel signal from the pixel 220 in that frame (S204).

[0162] On the other hand, if there is a difference in the sense signal between frames, the determination unit 216A instructs that the pixel signal be read from the pixel 220 in that frame (S205).

[0163] Next, the vertical scanning circuit 112 turns on the transfer transistor TGH of the pixel 220 of the frame instructed to read out the pixel signal, and reads out the pixel signal via the vertical signal line 132 (S206). At this time, the amplification transistor 122 of the pixel 220 of the frame instructed to read out the pixel signal can form a source follower with the constant current source 142 via the vertical signal line 132.

[0164] Next, the column ADC unit 214A performs AD conversion on the pixel signal read from the pixel 220 (S207) and outputs it via the processing unit 218.

[0165] Figure 22 is a timing chart showing an example of the operation of the imaging device according to the second embodiment when an event is detected.

[0166] In the figure, after the PD / FD reset is performed, charge accumulation in each frame begins. During the PD / FD reset, the transfer transistor TGH and the reset transistor 121 are turned on, and the photodiode PD and floating diffusions FD1 and FD2 are reset.

[0167] Then, a PD readout is performed in each frame, and it is determined whether there is a difference in the sense signal between frames, thereby determining whether there is a moving object in that frame. If it is determined that there is a moving object, the pixel signal is read out from pixel 220, and then a PD / FD reset is performed. In this case, in that frame, an FD reset and an FD readout are performed sequentially between the PD readout and the PD / FD reset. On the other hand, if it is determined that there is no moving object, the reading of the pixel signal from pixel 220 is skipped. In this case, in that frame, an FD reset and an FD readout are not performed between the PD readout and the PD / FD reset.

[0168] Figure 23 shows an example of the PD potential for each pixel in adjacent rows according to the second embodiment. In the figure, a represents the PD potential for each pixel in a frame without motion, b represents the PD potential for each pixel in a frame with motion, and c represents the difference in PD potential between frames when motion is present.

[0169] In the figure, the maximum storable capacitance CM and offset capacitance CF of the photodiode PD vary from row to row. In this case, by taking the difference in PD potential between frames, it is possible to detect the accumulated charge CS of the photodiode PD while canceling out the maximum storable capacitance CM and offset capacitance CF of the photodiode PD.

[0170] Thus, in the second embodiment described above, the pixel signal read out based on the difference in PD potential between frames is used for event determination. This allows the accumulated charge of the photodiode PD to be transferred to the floating diffusion FD1 based on the determination result of the PD potential based on non-destructive readout, the pixel signal to be read out, and event determination to be performed. Therefore, it is not necessary to perform destructive readout to determine the difference in PD potential between frames, and it becomes possible to realize EVS while suppressing the decrease in signal intensity.

[0171] <3. Third Embodiment> In the second embodiment described above, the pixel signal read out based on the difference in PD potential between frames was used for event determination. In this third embodiment, based on the difference in PD potential between frames, the pixel signal read out from the pixel 120 that has that difference is used for event determination.

[0172] Figure 24 is a block diagram showing an example configuration of a solid-state imaging device according to the third embodiment.

[0173] In the figure, this solid-state imaging device 302 includes a pixel array unit 111 and a sense signal storage unit 316 instead of the pixel array unit 211 and sense signal storage unit 216 of the second embodiment described above. Furthermore, this solid-state imaging device 302 has a vertical drive circuit 119 added to the solid-state imaging device 202 of the second embodiment described above. In this case, the solid-state imaging device 302 may be used for EVS. The other configurations of this solid-state imaging device 302 are the same as those of the solid-state imaging device 202 of the second embodiment described above.

[0174] The sense signal storage unit 316 stores the AD conversion value of the sense signal read from each pixel 120. The sense signal storage unit 316 includes a determination unit 316A. Based on the difference in PD potential between frames, the determination unit 316A instructs the reading of the pixel signal corresponding to the FD potential of the pixel 120 that has a difference. At this time, if there is a difference in PD potential between frames, the determination unit 316A can instruct the vertical scanning circuit 112 to drive the pixel 120 with the difference in the horizontal direction, instruct the vertical drive circuit 119 to drive the pixel 120 with the difference in the vertical direction, and instruct the column ADC unit 114A to perform an AD conversion operation.

[0175] Figure 25 is a flowchart showing an example of operation of the imaging device according to the third embodiment.

[0176] In the figure, the vertical scanning circuit 112 drives the pixels 120 row by row and reads out the sense signal from the pixels 120 (S301).

[0177] Next, the column ADC unit 214A performs AD conversion on the sense signal read from the pixel 120 and stores it in the sense signal storage unit 316 (S302).

[0178] Next, the determination unit 316A reads the AD conversion values ​​of the sense signals for multiple frames from the sense signal storage unit 316 and determines whether there is an inter-frame difference in the sense signals (S303).

[0179] Next, the vertical scanning circuit 112 and the vertical driving circuit 119 select a pixel 120 with an inter-frame difference in the sense signal based on the determination result of the determination unit 316A (S304). This selection of pixel 120 can be performed based on XY address specification. The determination unit 316A may also select the pixel 120 with the difference and its adjacent pixels.

[0180] Next, the vertical scanning circuit 112 turns off the transfer transistor TGH of the row containing the pixel 120 that has no difference in sense signals, and the vertical driving circuit 119 turns off the transfer transistor TGV of the column containing the pixel 120 that has no difference in sense signals (S305).

[0181] Furthermore, the vertical scanning circuit 112 turns on the transfer transistor TGH of the row containing the pixel 120 with the difference in sense signals, and the vertical driving circuit 119 turns on the transfer transistor TGV of the column containing the pixel 120 with the difference in sense signals (S306).

[0182] Next, the column ADC unit 214A performs AD conversion on the pixel signal read from the pixel 120 (S307) and outputs it via the processing unit 218.

[0183] Thus, in the third embodiment described above, the pixel signal read from the pixel 120 with a difference in PD potential is used for event determination based on the difference in PD potential between frames. This allows the accumulated charge of the pixel 120 with the difference to be transferred to the floating diffusion FD1 to read the pixel signal and perform event determination based on the determination result of the PD potential based on non-destructive readout. Therefore, destructive readout is not required to determine the difference in PD potential between frames, and pixels 120 without a difference can be set to non-operational, making it possible to realize EVS while suppressing a decrease in signal amount and an increase in power consumption.

[0184] <4. Fourth Embodiment> In the first embodiment described above, the FD potential was read out based on the sensing result of the PD potential, and the pixel signal based on the FD potential was added and stored. In this fourth embodiment, the pixel signal read out based on the difference in PD potential between adjacent pixels is used for edge determination. Note that if the pixel is a color pixel such as RGB, adjacent pixels may be adjacent pixels of the same color.

[0185] Figure 26 is a block diagram showing an example configuration of a solid-state imaging device according to the fourth embodiment.

[0186] In the figure, the solid-state imaging device 402 includes a pixel array unit 211, a column signal processing unit 414, and a reference signal generation unit 430, instead of the pixel array unit 111, column signal processing unit 114, and reference signal generation unit 130 of the first embodiment described above. Furthermore, the vertical drive circuit 119 is removed from the solid-state imaging device 102 of the first embodiment described above. In this case, the solid-state imaging device 402 may also be used for edge detection. The other configurations of the solid-state imaging device 402 are the same as those of the solid-state imaging device 102 of the first embodiment described above.

[0187] The column signal processing unit 414 processes signals transmitted vertically from the pixels 220. The column signal processing unit 414 includes a column ADC unit 414A. The column ADC unit 414A can perform AD conversion processing in parallel for each column. At this time, the column ADC unit 414A can perform AD conversion for each column based on the comparison result between the pixel signal read from the pixels 220 and the reference signal REF.

[0188] The column ADC unit 414A includes a determination unit 414B. The determination unit 414B instructs the reading of a pixel signal corresponding to the FD potential based on the difference in PD potential between adjacent pixels. At this time, if there is a difference in PD potential between adjacent pixels, the determination unit 414B can instruct the vertical scanning circuit 112 to drive the pixel 220 horizontally and instruct the column ADC unit 414A to perform AD conversion.

[0189] The reference signal generation unit 430 supplies a reference signal REF, which is compared with the pixel signal, to the column ADC unit 414A.

[0190] Figure 27 is a block diagram showing an example of the configuration of an AD conversion unit according to the fourth embodiment. In this figure, an example is shown in which pixels 220 are arranged in a 2x2 grid.

[0191] In the figure, the determination unit 414B can detect horizontal edges based on sense signals read from adjacent pixels in the column direction, or detect vertical edges based on sense signals read from adjacent pixels in the row direction. When detecting horizontal edges, comparison areas TVP1 and TVP2 can be set for each column. When detecting vertical edges, comparison areas THP1 and THP2 can be set for each row. Each comparison area may include a reference pixel and a target pixel.

[0192] The column ADC section 414A includes comparators CP1 and CP2 and a switching section KB. Comparators CP1 and CP2 can be provided for each column.

[0193] Each comparator CP1 and CP2 compares the pixel signal read from the pixel 220 with the reference signal REF column by column. Furthermore, when detecting a lateral edge, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 220 in the column direction for comparison areas TVP1 and TVP2. Additionally, when detecting a vertical edge, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 220 in the row direction for comparison areas THP1 and THP2.

[0194] Furthermore, each comparator CP1 and CP2 receives an auto-zero signal AZ. The auto-zero signal AZ activates the auto-zero operation during the auto-zero period. At this time, in comparator CP1, capacitor CA1 is connected to the non-inverting input terminal and capacitor CB1 is connected to the inverting input terminal. In comparator CP2, capacitor CA2 is connected to the non-inverting input terminal and capacitor CB2 is connected to the inverting input terminal. Then, in auto-zero operation, the charge accumulated in each capacitor CA1 and CB1 is controlled so that the non-inverting and inverting inputs of comparator CP1 are balanced, and the charge accumulated in each capacitor CA2 and CB2 is controlled so that the non-inverting and inverting inputs of comparator CP2 are balanced.

[0195] The switching unit KB switches the connection of the vertical signal lines 132 to each comparator CP1 and CP2, and the input of the reference signal REF. At this time, when performing AD conversion of the pixel signal read from the pixel 220, the switching unit KB connects the inverting input terminals of each comparator CP1 and CP2 to the vertical signal lines 132 for each column, and inputs the reference signal REF to the non-inverting input terminals of each comparator CP1 and CP2.

[0196] Furthermore, when detecting a lateral edge, the switching unit KB switches the connection of the vertical signal line 132 to the inverting input terminal and non-inverting input terminal of each comparator CP1 and CP2 for each column. At this time, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 220 in the column direction for each comparison area TVP1 and TVP2, and outputs the difference in the sense signals of the comparison areas TVP1 and TVP2. Here, the charge corresponding to the sense signal read from the first row is accumulated in each capacitor CA1 and CA2 for each column, and the sense signal read from the next row can be input to each comparator CP1 and CP2 via each capacitor CB1 and CB2.

[0197] Furthermore, when detecting a vertical edge, the switching unit KB connects the vertical signal lines 132 connected to adjacent pixels 220 in the row direction to the inverting input terminal and non-inverting input terminal of each comparator CP1 and CP2. At this time, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 220 in the row direction for each comparison area THP1 and THP2, and outputs the difference in the sense signals for each comparison area THP1 and THP2.

[0198] Figure 28 is a flowchart showing an example of operation of the imaging device according to the fourth embodiment.

[0199] In the figure, the vertical scanning circuit 112 drives the pixels 220 row by row and reads out the sense signal from the pixels 220 (S401).

[0200] Next, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 220 and outputs the difference in the sense signals. Then, the determination unit 414B determines whether there is a difference in the sense signals read from adjacent pixels 220 (S402).

[0201] Then, if there is no difference in the sense signals read from adjacent pixels 220, the determination unit 414B instructs not to read a pixel signal from that pixel 220 (S403).

[0202] On the other hand, if there is a difference in the sense signals read from adjacent pixels 220, the determination unit 414B instructs to read a pixel signal from that pixel 220 (S404). At this time, the determination unit 414B may also instruct to read pixel signals from the pixel 220 with the difference in sense signals and any number of pixels adjacent to that pixel 220.

[0203] Next, the vertical scanning circuit 112 turns on the transfer transistor TGH of the pixel 220 that has been instructed to read out the pixel signal, and reads out the pixel signal via the vertical signal line 132 (S405).

[0204] Next, the column ADC unit 414A performs AD conversion on the pixel signal read from the pixel 220 (S406) and outputs it via the processing unit 218.

[0205] Figure 29 is a timing chart showing an example of operation of the imaging device according to the fourth embodiment when detecting a lateral edge.

[0206] In the figure, after the PD / FD reset is performed, charge accumulation for each frame begins for every 220 pixels.

[0207] Then, PD readings of the target pixel and reference pixel of adjacent rows are performed column by column, and it is determined whether there is a difference in the sense signals of the target pixel and the reference pixel to determine whether there is a lateral edge. Here, the target pixel and reference pixel refer to pixels that are adjacent to each other in the column direction. If it is determined that there is a difference in the sense signals of the target pixel and the reference pixel, the pixel signal is read from the target pixel, and then a PD / FD reset is performed. At this time, in the row containing that target pixel, an FD reset and an FD read are performed sequentially between the PD read and the PD / FD reset. On the other hand, if it is determined that there is no difference in the sense signals of the target pixel and the reference pixel, the reading of the pixel signal from that target pixel is skipped. At this time, in the row containing that target pixel, no FD read is performed between the PD read and the PD / FD reset.

[0208] Figure 30 is a timing chart showing an example of operation of the imaging device according to the fourth embodiment when detecting a vertical edge.

[0209] In the figure, after the PD / FD reset is performed, charge accumulation for each frame begins for every 220 pixels.

[0210] Then, PD reading of the target pixel and reference pixel in the same row is performed column by column, and it is determined whether there is a difference in the sense signals of the target pixel and the reference pixel to determine whether there is a vertical edge. Here, the target pixel and reference pixel refer to pixels that are adjacent to each other in the row direction. If it is determined that there is a difference in the sense signals of the target pixel and the reference pixel, the pixel signal is read from the target pixel, and then a PD / FD reset is performed. At this time, in the row containing that target pixel, an FD reset and an FD read are performed sequentially between the PD read and the PD / FD reset. On the other hand, if it is determined that there is no difference in the sense signals of the target pixel and the reference pixel, the reading of the pixel signal from the row containing that target pixel is skipped. At this time, in the row containing that target pixel, no FD read is performed between the PD read and the PD / FD reset.

[0211] Figure 31 shows an example of the PD potential for each pixel in an adjacent frame according to the fourth embodiment. In the figure, a represents the PD potential for each reference pixel, b represents the PD potential for each target pixel, and c represents the difference between the PD potentials of the target pixel and the reference pixel.

[0212] In the figure, the maximum storage capacity CM of the photodiode PD varies for each of the 220 pixels. Also, the offset capacitance CF of the photodiode PD varies for each of the 220 pixels. In this case, the threshold level TH2 for determining the difference SCS of the sense signals between adjacent pixels can be set to be lower than the influence of the variation in the difference SCF of the offset capacitance CF. This makes it possible to determine the difference SCS of the sense signals between adjacent pixels while eliminating the influence of the variation in the offset capacitance CF of the photodiode PD.

[0213] Figure 32 shows an example of pixel connection in a solid-state imaging device according to the fourth embodiment.

[0214] In the figure, each pixel 220 may be individually connected to a vertical signal line 132. In this case, a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124 are provided for each pixel 220.

[0215] Figure 33 shows an example of two-pixel sharing in a solid-state imaging device according to the fourth embodiment.

[0216] In the figure, pixels adjacent to each other in the column direction may share a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. In this case, pixels adjacent to each other in the column direction can constitute a cell 221. Cell 221 comprises a photodiode PDA, PDB, a transfer transistor TGA, TGB, a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. Cell 221 also comprises threshold modulation transistors 125A, 125B, selection transistors 126A, 126B, and floating diffusion transistors FD1, FD2.

[0217] Each photodiode PDA and PDB performs photoelectric conversion and stores the converted charge. Each transfer transistor TGA and TGB transfers the charge stored in each photodiode PDA and PDB to the floating diffusion FD1, respectively. Each threshold modulation transistor 125A and 125B modulates the threshold based on the charge stored in each photodiode PDA and PDB. Each selection transistor 126A and 126B selects the output of each threshold modulation transistor 125A and 125B, respectively.

[0218] The transfer transistor TGA is connected in series between the cathode of the photodiode PDA and the floating diffusion FD1. The transfer transistor TGB is connected in series between the cathode of the photodiode PDB and the floating diffusion FD1. The threshold modulation transistor 125A and the selection transistor 126A are connected in series. The threshold modulation transistor 125B and the selection transistor 126B are connected in series. The drains of each threshold modulation transistor 125A and 125B are connected to the power supply potential VDD. The gate of the threshold modulation transistor 125A is connected to the cathode of the photodiode PDA. The gate of the threshold modulation transistor 125B is connected to the cathode of the photodiode PDB. The sources of each selection transistor 126A and 126B are connected to the vertical signal line 132.

[0219] The transfer signal TGLA is applied to the gate of the transfer transistor TGA. The transfer signal TGLB is applied to the gate of the transfer transistor TGB. The selection signal SELA2 is applied to the gate of the selection transistor 126A. The selection signal SELB2 is applied to the gate of the selection transistor 126B.

[0220] Figure 34 shows an example of four-pixel sharing in a solid-state imaging device according to the fourth embodiment.

[0221] In the figure, pixels adjacent to each other in the row direction and column direction may share a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. In this case, pixels adjacent to each other in the row direction and column direction can constitute a cell 222. The cell 222 comprises a photodiode PDA to PDD, a transfer transistor TGA to TGD, a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. The cell 222 also comprises threshold modulation transistors 125A to 125D, selection transistors 126A to 126D, and floating diffusion transistors FD1 and FD2.

[0222] Each photodiode PDA to PDD performs photoelectric conversion and stores the converted charge. Each transfer transistor TGA to TGD transfers the charge stored in each photodiode PDA to the floating diffusion FD1. Each threshold modulation transistor 125A to 125D modulates the threshold based on the charge stored in each photodiode PDA to PDD. Each selection transistor 126A to 126D selects the output of each threshold modulation transistor 125A to 125D.

[0223] Transfer transistor TGA is connected in series between the cathode of photodiode PDA and floating diffusion FD1. Transfer transistor TGB is connected in series between the cathode of photodiode PDB and floating diffusion FD1. Transfer transistor TGC is connected in series between the cathode of photodiode PDC and floating diffusion FD1. Transfer transistor TGD is connected in series between the cathode of photodiode PDD and floating diffusion FD1. Threshold modulation transistor 125A and selection transistor 126A are connected in series. Threshold modulation transistor 125B and selection transistor 126B are connected in series. Threshold modulation transistor 125C and selection transistor 126C are connected in series. Threshold modulation transistor 125D and selection transistor 126D are connected in series. The drains of each threshold modulation transistor 125A to 125D are connected to the power supply potential VDD. The gate of threshold modulation transistor 125A is connected to the cathode of photodiode PDA. The gate of threshold modulation transistor 125B is connected to the cathode of photodiode PDB. The gate of threshold modulation transistor 125C is connected to the cathode of photodiode PDC. The gate of threshold modulation transistor 125D is connected to the cathode of photodiode PDD. The sources of each selection transistor 126A to 126D are connected to the vertical signal line 132.

[0224] The transfer signal TGLA is applied to the gates of each transfer transistor TGA and TGC. The transfer signal TGLB is applied to the gates of each transfer transistor TGB and TGD. The selection signal SELA2 is applied to the gates of each selection transistor 126A and 126C. The selection signal SELB2 is applied to the gates of each selection transistor 126B and 126D.

[0225] As described above, in the fourth embodiment, the pixel signal read out based on the difference in PD potential between adjacent pixels is used for edge detection. This allows the accumulated charge of the photodiode PD to be transferred to the floating diffusion FD1 based on the PD potential determination result based on non-destructive readout, the pixel signal to be read out, and edge detection to be performed. Therefore, it is not necessary to perform destructive readout to determine the difference in PD potential between adjacent pixels, and edge detection can be achieved while suppressing a decrease in signal intensity.

[0226] <5. Fifth Embodiment> In the fourth embodiment described above, the pixel signal read out based on the difference in PD potential between adjacent pixels was used for edge determination. In this fifth embodiment, based on the difference in PD potential between adjacent pixels, the pixel signal read out from the pixel 120 that has that difference is used for edge determination.

[0227] Figure 35 is a block diagram showing an example configuration of a solid-state imaging device according to the fifth embodiment.

[0228] In the figure, the solid-state imaging device 502 includes a column signal processing unit 514 and a reference signal generation unit 430 instead of the column signal processing unit 114 and reference signal generation unit 130 of the first embodiment described above. The other configurations of the solid-state imaging device 502 are the same as those of the solid-state imaging device 102 of the first embodiment described above.

[0229] The column signal processing unit 514 processes signals transmitted vertically from the pixel 120. The column signal processing unit 514 includes a column ADC unit 514A. The column ADC unit 514A can perform AD conversion processing in parallel for each column. At this time, the column ADC unit 514A can perform AD conversion for each column based on the comparison result between the pixel signal read from the pixel 120 and the reference signal REF.

[0230] The column ADC unit 514A includes a determination unit 514B. Based on the difference in PD potential between adjacent pixels, the determination unit 514B instructs the reading of a pixel signal corresponding to the FD potential of the pixel 120 that has the difference. At this time, if there is a difference FD in the PD potential between adjacent pixels, the determination unit 514B can instruct the vertical scanning circuit 112 to drive the pixel 120 with the difference in the horizontal direction, instruct the vertical drive circuit 119 to drive the pixel 120 with the difference in the vertical direction, and instruct the column ADC unit 514A to perform AD conversion.

[0231] Figure 36 is a flowchart showing an example of operation of the imaging device according to the fifth embodiment.

[0232] In the figure, the vertical scanning circuit 112 drives each pixel 120 row by row and reads out the sense signal from the pixel 120 (S501).

[0233] Next, each comparator CP1 and CP2 compares the sense signals read from adjacent pixels 120 and outputs the difference in the sense signals. Then, the determination unit 514B determines whether there is a difference in the sense signals read from adjacent pixels 120 (S502).

[0234] Next, the vertical scanning circuit 112 and the vertical driving circuit 119 select a pixel 120 with a difference in sense signals based on the determination result of the determination unit 514B (S503). This selection of pixel 120 can be performed based on XY address specification. The determination unit 514B may also select the pixel 120 with the difference and its adjacent pixels.

[0235] Next, the vertical scanning circuit 112 turns off the transfer transistor TGH for pixels 120 that have no difference in sense signals, and the vertical driving circuit 119 turns off the transfer transistor TGV for pixels 120 that have no difference in sense signals (S504).

[0236] Furthermore, the vertical scanning circuit 112 turns on the transfer transistor TGH of the pixel 120 that has a difference in sense signals, and the vertical driving circuit 119 turns on the transfer transistor TGV of the pixel 120 that has a difference in sense signals (S505).

[0237] Next, the column ADC unit 214A performs AD conversion on the pixel signal (S506) and outputs it via the processing unit 218.

[0238] Figure 37 shows an example of a waveform during pixel operation according to the fifth embodiment. In this figure, an example of detecting a lateral edge based on the reading of sense signals from the reference pixel and target pixel in the comparison area TVP1 of Figure 27 is shown. Furthermore, the reference pixel is input with selection signal SELA2, and the target pixel is input with selection signal SELB2.

[0239] In the figure, before storage begins, the selection signals SEL1, SELA2, SELB2, the reset signal RST, the switching signal FDG, and the transfer signals TGLV, TGLH rise. At this time, in the target pixel, the selection transistors 123, 126, the reset transistor 121, the switching transistor 124, and the transfer transistors TGH, TGV are turned on, and the photodiode PD and floating diffusion FD1, FD2 are reset. Also, the conversion efficiency is set to LCG.

[0240] Next, when the storage period ends, a difference determination is performed on the sense signals read from adjacent pixels. In the difference determination, the selection signal SELA1 of the reference pixel rises, and the selection transistor 126 of the reference pixel turns on. At this time, a sense signal is read from the reference pixel, and based on that sense signal, the potential of either capacitor CA1 or CB1 is clamped. Next, after the selection signal SELA1 of the reference pixel falls, the selection signal SELB1 of the target pixel rises. At this time, the sense signal read from the target pixel is input to comparator CP1 via the other of capacitor CA1 or CB1, and the difference between the sense signals read from the reference pixel and the target pixel in the comparison area TVP1 is detected.

[0241] Next, the selection signal SEL1 rises, and the selection transistor 123 turns on. Furthermore, the reset signal RST rises, and the reset transistor 121 turns on. At this time, the floating diffusions FD1 and FD2 are reset. Then, the reset signal RST falls, and the reset transistor 121 turns off. Also, the switching signal FDG falls, and the switching transistor 124 turns off. At this time, the conversion efficiency is set to HCG, and HCGP phase readout is performed.

[0242] Next, the switching signal FDG rises, and the switching transistor 124 is turned on. At this time, the conversion efficiency is set to LCG, and LCGP phase readout is performed.

[0243] Next, the transfer signals TGLV and TGLH rise, and the transfer transistors TGV and TGH turn on. At this time, the charge accumulated in pixel 120 is transferred to the floating diffusion FD1, and LCGD phase readout is performed. Then, the transfer signals TGLV and TGLH fall, and the transfer transistors TGV and TGH turn off.

[0244] Next, the switching signal FDG falls, and the switching transistor 124 is turned off. At this time, the conversion efficiency of the pixel 120 is set to HCG, and HCGD phase readout is performed.

[0245] Figure 38 shows an example of pixel connection in a solid-state imaging device according to the fifth embodiment.

[0246] In the figure, each pixel 120 may be individually connected to a vertical signal line 132. In this case, a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124 are provided for each pixel 120.

[0247] Figure 39 shows an example of two-pixel sharing in a solid-state imaging device according to the fifth embodiment.

[0248] In the figure, pixels adjacent to each other in the column direction may share a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. In this case, pixels adjacent to each other in the column direction can constitute a cell 321. Cell 321 is equipped with transfer transistors TGVA, TGVB, TGHA, and TGHB instead of the transfer transistors TGA and TGB of cell 221 in the fourth embodiment described above. The other configurations of cell 321 are the same as those of cell 221 in the fourth embodiment described above.

[0249] Transfer transistors TGVA and TGHA are connected in series between the cathode of photodiode PDA and floating diffusion FD1. Transfer transistors TGVB and TGHB are connected in series between the cathode of photodiode PDB and floating diffusion FD1.

[0250] The transfer signal TGLVA is applied to the gate of transfer transistor TGVA. The transfer signal TGLHA is applied to the gate of transfer transistor TGHA. The transfer signal TGLVB is applied to the gate of transfer transistor TGVB. The transfer signal TGLHB is applied to the gate of transfer transistor TGHB.

[0251] Figure 40 shows an example of four-pixel sharing in a solid-state imaging device according to the fifth embodiment.

[0252] In the figure, pixels adjacent to each other in the row direction and column direction may share a reset transistor 121, an amplification transistor 122, a selection transistor 123, and a switching transistor 124. In this case, pixels adjacent to each other in the row direction and column direction can constitute a cell 322. Cell 322 includes transfer transistors TGVA to TGVD and TGHA to TGHD, instead of transfer transistors TGA to TGD in cell 222 of the fourth embodiment described above. The other configurations of cell 322 are the same as those of cell 222 of the fourth embodiment described above.

[0253] Transfer transistors TGVA and TGHA are connected in series between the cathode of photodiode PDA and floating diffusion FD1. Transfer transistors TGVB and TGHB are connected in series between the cathode of photodiode PDB and floating diffusion FD1. Transfer transistors TGVC and TGHC are connected in series between the cathode of photodiode PDC and floating diffusion FD1. Transfer transistors TGVD and TGHD are connected in series between the cathode of photodiode PDD and floating diffusion FD1.

[0254] The transfer signal TGLVA is applied to the gates of transfer transistors TGVA and TGVC. The transfer signal TGLHA is applied to the gates of transfer transistors TGHA and TGHC. The transfer signal TGLVB is applied to the gates of transfer transistors TGVB and TGVD. The transfer signal TGLHB is applied to the gates of transfer transistors TGHB and TGHD.

[0255] Thus, in the fifth embodiment described above, the pixel signal read from the pixel 120 with a difference in PD potential is used for edge detection based on the difference in PD potential between adjacent pixels. This allows the accumulated charge of the pixel 120 with the difference to be transferred to the floating diffusion FD1 based on the PD potential determination result based on non-destructive readout, the pixel signal to be read out, and edge detection to be performed. Therefore, destructive readout is not required to determine the difference in PD potential between adjacent pixels, and pixels 120 without a difference can be set to non-operational, making it possible to achieve edge detection while suppressing a decrease in signal amount and an increase in power consumption.

[0256] <6. Sixth Embodiment> In the first embodiment described above, the pixel conversion efficiency was switched based on the determination result of the readout level of the pixel signal. In this sixth embodiment, semiconductor chips, each provided with a pixel array section in which pixels are arranged in a matrix, are stacked.

[0257] Figure 41 is a perspective view showing an example of stacking of pixel arrays according to the sixth embodiment.

[0258] In the figure, the solid-state imaging device comprises semiconductor chips 921 and 922. Semiconductor chip 922 is stacked on semiconductor chip 921.

[0259] A pixel array section 923 is formed on the semiconductor chip 922. Pixels 931 are arranged in a matrix in the row and column directions within the pixel array section 923. The pixels 931 may be the pixels 120 and 220 of the above-described embodiment, or they may be provided on the cells 221, 222, 321, and 322 of the above-described embodiment. Pad electrodes 932 and via electrodes 933 are formed around the pixel array section 923. The via electrodes 933 penetrate the semiconductor chip 922 and can electrically connect the semiconductor chips 921 and 922 to each other.

[0260] A peripheral circuit 924 is formed on the semiconductor chip 921. The peripheral circuit 924 includes a column readout circuit 925, a column ADC 926, a communication interface 927, and an oscillation circuit 928. The column readout circuit 925 and the column ADC 926 may be formed to correspond to positions on both sides of the pixel array section 923 in the column direction. The peripheral circuit 924 may be provided with the determination units 114B, 216A, 316A, 414B, and 514B of the above-described embodiment, or it may be provided with a pixel signal storage unit 116, an adder 117, and a processing unit 118, or it may be provided with a sense signal storage unit 216.

[0261] The semiconductor chips 921 and 922 may be directly bonded. Hybrid bonding can be used for the direct bonding of the semiconductor chips 921 and 922. In this case, the semiconductor chips 921 and 922 may be electrically connected based on Cu-Cu connections. The semiconductor substrate material used for the semiconductor chips 921 and 922 may be Si, InGaAs, or InP.

[0262] Thus, in the sixth embodiment described above, the semiconductor chip 922 on which the pixel array 923 is formed is stacked on the semiconductor chip 921 on which the peripheral circuit 924 is formed. This makes it possible to increase the sensitivity of the solid-state imaging device while suppressing an increase in the mounting area of ​​the semiconductor chip on which the solid-state imaging device is formed.

[0263] <7. Examples of Application to Mobile Devices> The technology disclosed herein (the technology) can be applied to various products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0264] Figure 42 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0265] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 42, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0266] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0267] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0268] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0269] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0270] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0271] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0272] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0273] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0274] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 42, the output devices include an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0275] Figure 43 shows an example of the installation position of the imaging unit 12031.

[0276] In Figure 43, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0277] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0278] Figure 43 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0279] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0280] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, etc., that drives autonomously without driver operation, can be performed.

[0281] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0282] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0283] The above describes an example of a vehicle control system to which the technology of this disclosure may be applied. The technology of this disclosure can be applied to the imaging unit 12031 of the configuration described above. Specifically, for example, each imaging device of the above embodiment can be applied to the imaging unit 12031. By applying the technology of this disclosure to the vehicle control system 12000, it becomes possible to achieve HDR, event detection, and edge detection while suppressing a decrease in the signal amount of the imaging unit 12031.

[0284] The embodiments described above are merely examples for realizing the present technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of the present technology bearing the same name. However, the present technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology. Furthermore, the effects described herein are merely examples and are not limiting, and other effects may also exist.

[0285] Furthermore, this technology can also take the following configurations: (1) An imaging device comprising: a photoelectric conversion unit provided in a pixel; a transfer transistor for transferring the charge accumulated in the photoelectric conversion unit to a floating diffusion; an amplification transistor for outputting a pixel signal corresponding to the charge accumulated in the floating diffusion; a first selection transistor for selecting the output from the amplification transistor; a threshold modulation transistor whose threshold is modulated based on the charge accumulated in the photoelectric conversion unit; and a second selection transistor for selecting the output from the threshold modulation transistor. (2) The imaging device according to (1), further comprising signal lines for transmitting the output from the amplification transistor and the output from the threshold modulation transistor. (3) The imaging device according to (1) or (2), wherein the photoelectric conversion unit, the transfer transistor, the threshold modulation transistor and the second selection transistor are provided for each pixel, and the amplification transistor and the first selection transistor are shared by a plurality of pixels. (4) The imaging device according to any one of (1) to (3), further comprising a first determination unit for instructing the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion based on the determination result of the output level of the threshold modulation transistor. (5) The imaging apparatus according to (4), wherein the reading of the output level of the threshold modulation transistor is performed by non-destructive reading. (6) The imaging apparatus according to (4) or (5), wherein the reading of the output level of the threshold modulation transistor is performed multiple times during the charge accumulation period. (7) The imaging apparatus according to any one of (4) to (6), wherein the first determination unit instructs the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion when the output level of the threshold modulation transistor is at a threshold level less than the saturation level. (8) The imaging apparatus according to any one of (1) to (7), wherein the threshold level is set to a potential below the depletion voltage of the photoelectric conversion unit. (9) The imaging apparatus according to any one of (1) to (8), further comprising a switching transistor for switching the conversion efficiency of the pixel set for reading the pixel signal corresponding to the charge accumulated in the floating diffusion.(10) The imaging apparatus according to (7), further comprising a comparator for determining the output level of the threshold modulation transistor and the output level of the amplification transistor, and a counter for performing a count operation based on the determination result of the output level of the threshold modulation transistor or the output level of the amplification transistor. (11) The imaging apparatus according to (10), wherein the first determination unit sets the comparator and the counter to non-operation when the output level of the threshold modulation transistor is less than the saturation level. (12) The imaging apparatus according to any one of (1) to (11), further comprising a storage unit for storing pixel signals read out based on the determination result of the output level of the threshold modulation transistor, and an adder for adding the pixel signals stored in the storage unit for each pixel. (13) The imaging apparatus according to any one of (1) to (12), comprising: a pixel array section in which the pixels are arranged in a matrix in the row direction and the column direction; a vertical scanning circuit for scanning the pixels in the column direction; a vertical signal line for transmitting the output level of the threshold modulation transistor and the output level of the amplification transistor in the column direction; and a horizontal control line for driving the pixels in the row direction, wherein the vertical scanning circuit drives the transfer transistor, the first selection transistor and the second selection transistor via the horizontal control line. (14) The imaging apparatus according to (13), wherein the transfer transistor comprises: a first transfer transistor for transferring the charge accumulated in the photoelectric conversion section row by row; and a second transfer transistor for transferring the charge accumulated in the photoelectric conversion section column by column. (15) The imaging apparatus according to (13) or (14), further comprising: a second determination section for instructing the vertical scanning circuit to read out a pixel signal corresponding to the charge accumulated in the floating diffusion based on the determination result of the difference in the output levels of the threshold modulation transistor. (16) The imaging apparatus according to (15), wherein the difference in the output levels of the threshold modulation transistor is the difference between frames. (17) The imaging apparatus according to (15), wherein the difference in the output levels of the threshold modulation transistor is the difference between adjacent pixels.(18) The imaging apparatus according to (17), wherein the second determination unit instructs the vertical scanning circuit to read out a pixel having the output level difference and a pixel adjacent to the pixel. (19) The imaging apparatus according to (18), wherein the second determination unit sets the processing of pixel signals transmitted via the vertical signal line to non-operation for columns other than the column to which the pixel having the output level difference and the pixel adjacent to the pixel belong. (20) The imaging apparatus comprising: a photoelectric conversion unit provided in a pixel; a sense unit provided in the pixel that outputs a sense signal that follows a PD potential corresponding to the charge accumulated in the photoelectric conversion unit; and an output unit that outputs a pixel signal that follows a floating diffusion FD potential corresponding to the charge transferred from the photoelectric conversion unit, based on the PD potential detected by the sense unit.

[0286] 100 Imaging device 101 Optical system 102 Solid-state imaging device 103 Imaging control unit 104 Image processing unit 105 Storage unit 106 Display unit 107 Operation unit 108 Bus 111 Pixel array unit 112 Vertical scanning circuit 113 Column reading circuit 114 Column signal processing unit 115 Horizontal scanning circuit 116 Pixel signal storage unit 117 Addition unit 118 Processing unit 119 Horizontal drive circuit 120 Pixel 130 Reference signal generation unit 131 Horizontal control line 132 Vertical signal line 133 Vertical control line PD Photodiode FD1, FD2 Floating diffusion TGH, TGV Transfer transistor 121 Reset transistor 122 Amplifier transistor 123, 126 Selection transistor 124 Switching transistor 125 Threshold modulation transistor

Claims

1. An imaging device comprising: a photoelectric conversion unit provided in a pixel; a transfer transistor for transferring the charge accumulated in the photoelectric conversion unit to a floating diffusion; an amplification transistor for outputting a pixel signal corresponding to the charge accumulated in the floating diffusion; a first selection transistor for selecting the output from the amplification transistor; a threshold modulation transistor whose threshold is modulated based on the charge accumulated in the photoelectric conversion unit; and a second selection transistor for selecting the output from the threshold modulation transistor.

2. The imaging apparatus according to claim 1, further comprising signal lines for transmitting the output from the amplification transistor and the output from the threshold modulation transistor.

3. The imaging apparatus according to claim 1, wherein the photoelectric conversion unit, the transfer transistor, the threshold modulation transistor, and the second selection transistor are provided for each pixel, and the amplification transistor and the first selection transistor are shared by a plurality of pixels.

4. The imaging apparatus according to claim 1, further comprising a first determination unit that instructs the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion based on the determination result of the output level of the threshold modulation transistor.

5. The imaging apparatus according to claim 4, wherein the reading of the output level of the threshold modulation transistor is performed by non-destructive reading.

6. The imaging apparatus according to claim 4, wherein the reading of the output level of the threshold modulation transistor is performed multiple times during the charge accumulation period.

7. The imaging apparatus according to claim 4, wherein the first determination unit instructs the reading of a pixel signal corresponding to the charge accumulated in the floating diffusion when the output level of the threshold modulation transistor is at a threshold level below the saturation level.

8. The imaging apparatus according to claim 1, wherein the threshold level is set to a potential lower than or equal to the depletion voltage of the photoelectric conversion unit.

9. The imaging apparatus according to claim 1, further comprising a switching transistor that switches the conversion efficiency of the pixel, which is set to read out a pixel signal corresponding to the charge accumulated in the floating diffusion.

10. The imaging apparatus according to claim 7, further comprising: a comparator for determining the output level of the threshold modulation transistor and the output level of the amplification transistor; and a counter for performing a count operation based on the determination result of the output level of the threshold modulation transistor or the output level of the amplification transistor.

11. The imaging apparatus according to claim 10, wherein the first determination unit sets the comparator and the counter to non-operation when the output level of the threshold modulation transistor is less than the threshold level.

12. The imaging apparatus according to claim 1, further comprising: a storage unit for storing pixel signals read out based on the determination result of the output level of the threshold modulation transistor; and an adder for adding the pixel signals stored in the storage unit for each pixel.

13. The imaging apparatus according to claim 1, comprising: a pixel array section in which the pixels are arranged in a matrix in the row direction and the column direction; a vertical scanning circuit for scanning the pixels in the column direction; a vertical signal line for transmitting the output level of the threshold modulation transistor and the output level of the amplification transistor in the column direction; and a horizontal control line for driving the pixels in the row direction, wherein the vertical scanning circuit drives the transfer transistor, the first selection transistor and the second selection transistor via the horizontal control line.

14. The imaging apparatus according to claim 13, wherein the transfer transistor comprises a first transfer transistor that transfers the charge accumulated in the photoelectric conversion unit row by row, and a second transfer transistor that transfers the charge accumulated in the photoelectric conversion unit column by column.

15. The imaging apparatus according to claim 13, further comprising a second determination unit that instructs the vertical scanning circuit to read out a pixel signal corresponding to the charge accumulated in the floating diffusion based on the determination result of the difference in the output levels of the threshold modulation transistor.

16. The imaging apparatus according to claim 15, wherein the difference in the output levels of the threshold modulation transistor is the difference between frames.

17. The imaging apparatus according to claim 15, wherein the difference in the output levels of the threshold modulation transistor is the difference between adjacent pixels.

18. The imaging apparatus according to claim 17, wherein the second determination unit instructs the vertical scanning circuit to read out a pixel having the difference in output level and a pixel adjacent to the pixel.

19. The imaging apparatus according to claim 18, wherein the second determination unit sets the processing of pixel signals transmitted via the vertical signal line to non-operation for columns other than the column to which the pixel having the difference in output level and the pixel adjacent to the pixel belong.

20. An imaging device comprising: a photoelectric conversion unit provided in a pixel; a sense unit provided in the pixel and outputting a sense signal that follows the PD potential corresponding to the charge accumulated in the photoelectric conversion unit; and an output unit that outputs a pixel signal that follows the FD potential of a floating diffusion corresponding to the charge transferred from the photoelectric conversion unit, based on the PD potential detected by the sense unit.

Images