Imaging device and imaging method

Abstract

The present invention improves flexibility in frame rate when acquiring difference information.　This imaging device comprises: a photoelectric conversion unit that outputs a photoelectric conversion signal corresponding to charges accumulated by photoelectric conversion; and a difference information generation unit that generates difference information based on a difference value of the photoelectric conversion signal when the charges of the photoelectric conversion unit are accumulated. The photoelectric conversion unit may not be reset between the outputs of the photoelectric conversion signals used for generating the difference value. The difference information generation unit may include a subtractor that calculates the difference between the photoelectric conversion signals according to the accumulation of the charges of the photoelectric conversion unit.

Description

Imaging device and imaging method

[0001] The present technology relates to an imaging device and an imaging method, and more particularly, to an imaging device and an imaging method capable of acquiring difference information.

[0002] There is a technology for imaging devices that generates an event image, an illuminance change map, and a captured image using the same pixel. For example, a technology has been disclosed in which a detection unit that detects whether a change in photocurrent corresponding to the amount of charge exceeds a predetermined threshold is connected to a charge accumulation unit to cause a photocurrent to flow (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-88724

[0004] However, in the above-mentioned conventional technology, there is a risk that the frame rate of event detection may be limited by the resetting of the charge storage unit.

[0005] This technology was developed in light of these circumstances, and aims to improve the flexibility of the frame rate when acquiring difference information.

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof is an imaging device including a photoelectric conversion unit that outputs a photoelectric conversion signal corresponding to charge accumulated by photoelectric conversion, and a difference information generation unit that generates difference information based on a difference value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated. This brings about an effect that the difference information can be obtained without obtaining a grayscale signal.

[0007] In the first aspect, the photoelectric conversion unit may not be reset between outputs of the photoelectric conversion signal used to generate the difference value, thereby providing an effect that the photoelectric conversion signal at the time of accumulating charge in the photoelectric conversion unit is output.

[0008] In the first aspect, the difference information generation unit may include a subtractor that generates a difference value of the photoelectric conversion signal, thereby producing an effect that a difference value when charge of the photoelectric conversion unit is accumulated is generated.

[0009] In the first aspect, a sample-and-hold circuit may be provided that samples and holds a difference value of the photoelectric conversion signal, thereby providing an effect that the difference value when the charge of the photoelectric conversion unit is accumulated is discretized.

[0010] In the first aspect, an AD (Analog to Digital) converter may be provided that AD converts a difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit, thereby producing an effect that a luminance value is generated as difference information based on a difference value when charges of the photoelectric conversion unit are accumulated.

[0011] In the first aspect, an event determination circuit may be provided that determines an event based on a difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit, thereby producing an effect that an event determination result is generated as difference information when charge of the photoelectric conversion unit is accumulated.

[0012] In the first aspect, an edge determination circuit may be provided that determines an edge based on a difference value of the photoelectric conversion signal sampled and held by the sample-and-hold circuit, thereby producing an edge determination result as difference information based on a difference value when charge of the photoelectric conversion unit is accumulated.

[0013] In the first aspect, a selection switch may be provided for selecting an output of the difference information, thereby providing an effect that the brightness value, the event determination result, or the edge determination result can be selected as the difference information.

[0014] In the first aspect, the photoelectric conversion unit may include a photodiode and a source follower circuit having a gate connected to a cathode of the photodiode, thereby providing an effect of outputting a photoelectric conversion signal when charge is accumulated in the photoelectric conversion unit.

[0015] In the first aspect, the photoelectric conversion unit may include a grayscale information generation unit that generates a grayscale signal according to the charge accumulated in the photodiode, thereby achieving an effect of simultaneously acquiring difference information and the grayscale signal from the same pixel.

[0016] In the first aspect, the frame rate of the difference information and the frame rate of the gradation signal may be different from each other, thereby achieving an effect of obtaining the difference information and the gradation signal from the same pixel while reducing noise in the gradation signal.

[0017] In addition, in the first aspect, an image processing unit may be provided that generates a high dynamic range (HDR) image based on the luminance information generated based on the difference value and the gradation signal, thereby achieving an effect of speeding up the generation of the HDR image.

[0018] In the first aspect, the pixel circuit may further include a cutoff switch connected between the cathode of the photodiode and the gate of the source follower circuit, thereby achieving low noise in the grayscale signal and enabling the difference information and the grayscale signal to be acquired from the same pixel.

[0019] In the first aspect, a changeover switch may be provided for switching from event determination or edge determination based on the difference value to event determination or edge determination based on the gradation signal, thereby achieving an event determination or edge determination without acquiring difference information.

[0020] In the first aspect, the photoelectric conversion unit may include a floating diffusion shared by a plurality of gradation information generation units, thereby suppressing an increase in circuit size and enabling gradation signals to be acquired from a plurality of gradation information generation units.

[0021] In the first aspect, a comparator may be provided that is shared between the comparison of the gradation signal and the reference signal and the comparison of the difference values, thereby achieving AD conversion of the gradation signal and event or edge determination while suppressing an increase in circuit size.

[0022] In addition, in the first aspect, the pixel array unit may include pixels each having the photoelectric conversion unit arranged in a matrix in the row and column directions, and the difference information generation unit may be provided for each column. This provides the effect of realizing event determination or edge determination for a plurality of pixels while suppressing an increase in circuit size.

[0023] In addition, in the first aspect, the device may include a first chip on which the photoelectric conversion unit is formed, and a second chip on which the first chip is stacked and on which the difference information generation unit is formed, thereby achieving the effect of acquiring difference information while suppressing an increase in the planar size of the first chip and the second chip.

[0024] In addition, in the first aspect, the image sensor may include a gradation pixel having a gradation information generating section for generating a gradation signal, and a hybrid pixel having the gradation information generating section and the difference information generating section, thereby achieving an effect of obtaining difference information while increasing the resolution of the gradation image.

[0025] A second aspect of the present invention is an imaging method that outputs a photoelectric conversion signal corresponding to charges accumulated by photoelectric conversion, and generates difference information based on a difference value of the photoelectric conversion signal during accumulation of the charges, thereby obtaining the difference information without obtaining a grayscale signal.

[0026] 1 is a block diagram showing an example of the configuration of an imaging device according to a first embodiment. FIG. 1 is a block diagram showing an example of the configuration of a solid-state imaging device according to the first embodiment. FIG. 2 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the first embodiment. FIG. 3 is a diagram showing an example of a method for generating differential information according to the first embodiment. FIG. 4 is a diagram showing a relationship between illuminance, a cathode voltage of a photodiode, and differential information according to the first embodiment. FIG. 5 is a diagram showing another example of the circuit configuration of a pixel provided in the solid-state imaging device according to the first embodiment. FIG. 6 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the second embodiment. FIG. 7 is a diagram showing a relationship between illuminance, a cathode voltage of a photodiode, and differential information according to the second embodiment. FIG. 8 is a diagram showing another example of the circuit configuration of a pixel provided in the solid-state imaging device according to the second embodiment. FIG. 9 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the third embodiment. FIG. 10 is a diagram showing a relationship between illuminance, a cathode voltage of a photodiode, and differential information according to the third embodiment. FIG. 11 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the fourth embodiment. FIG. 12 is a block diagram showing an example of the configuration of a solid-state imaging device according to the fifth embodiment. FIG. 13 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the fifth embodiment. 13 is a diagram showing an example of a method for generating difference information according to the fifth embodiment. FIG. 14 is a timing chart showing an example of the operation of the solid-state imaging device according to the fifth embodiment. FIG. 15 is a diagram showing another example of the circuit configuration of a pixel provided in the solid-state imaging device according to the fifth embodiment. FIG. 16 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the sixth embodiment. FIG. 17 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the seventh embodiment. FIG. 18 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the eighth embodiment. FIG. 19 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the ninth embodiment. FIG. 19 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the tenth embodiment. FIG. 20 is a cross-sectional view showing an example of the configuration of a pixel provided in the solid-state imaging device according to the eleventh embodiment. FIG. 21 is a diagram showing an example of the circuit configuration of a pixel provided in the solid-state imaging device according to the twelfth embodiment.13 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to a 13th embodiment. FIG. 14 is a cross-sectional view showing an example of a configuration of a pixel provided in a solid-state imaging device according to the 13th embodiment. FIG. 15 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to a 14th embodiment. FIG. 16 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to a 15th embodiment. FIG. 17 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to a 16th embodiment. FIG. 18 is a diagram showing an example of a circuit configuration of a comparator provided in a solid-state imaging device according to the 17th embodiment. FIG. 19 is a diagram showing an example of a pixel array provided in a solid-state imaging device according to an 18th embodiment. FIG. 19 is a perspective view showing an example of a stack of a solid-state imaging device according to a 19th embodiment. FIG. 20 is a block diagram showing a schematic configuration example of a vehicle control system. FIG. 21 is an explanatory diagram showing an example of an installation position of an imaging unit.

[0027] Hereinafter, modes for implementing the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. First embodiment (an example in which a luminance signal is generated as difference information based on a difference value of a photoelectric conversion signal) 2. Second embodiment (an example in which an event signal is generated as difference information based on a difference value of a photoelectric conversion signal) 3. Third embodiment (an example in which an edge signal is generated as difference information based on a difference value of a photoelectric conversion signal) 4. Fourth embodiment (an example in which an event signal and an edge signal are generated as difference information based on a difference value of a photoelectric conversion signal) 5. Fifth embodiment (an example in which a luminance signal is generated as difference information based on a difference value of a photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal) 6. Sixth embodiment (an example in which difference information is generated based on a difference value of a photoelectric conversion signal after AD conversion of the photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal) 7. Seventh embodiment (an example in which an event signal is generated as difference information based on a difference value of a photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal) 8. 8. Eighth Embodiment (an example in which an edge signal is generated as difference information based on a difference value of a photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal) 9. Ninth Embodiment (an example in which an event signal and an edge signal are generated as difference information based on a difference value of a photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal) 10. Tenth Embodiment (an example in which an event determination based on difference information based on a difference value of a photoelectric conversion signal and generation of a gradation signal based on the photoelectric conversion signal are switched) 11. Eleventh Embodiment (an example in which generation of difference information based on a difference value of a photoelectric conversion signal can be blocked when generation of a gradation signal based on the photoelectric conversion signal) 12. Twelfth Embodiment (an example in which an event determination based on difference information based on a difference value of a photoelectric conversion signal and event determination based on a gradation signal are switched) 13. Thirteenth Embodiment (an example in which a photoelectric conversion unit is arranged on an upper chip, and a difference information processing unit and a gradation information processing unit are arranged on a lower chip) 14. Fourteenth Embodiment (an example in which a rotating diffusion is shared by a plurality of pixels provided with a photoelectric conversion unit that generates difference information) 15. Fifteenth embodiment (example in which a difference information processing unit is shared by multiple pixels)16. Sixteenth embodiment (example in which an event / edge determination circuit is provided for each column) 17. Seventeenth embodiment (example in which a comparator is shared by a difference information processing section and a gradation information processing section) 18. Eighteenth embodiment (example in which gradation pixels and hybrid pixels are combined) 19. Nineteenth embodiment (example in which pixel array sections are stacked) 20. Application example to a moving body

[0028] 1. First Embodiment FIG. 1 is a block diagram showing an example of the configuration of an imaging device according to a first embodiment.

[0029] In the figure, the imaging device 100 includes an optical system 101, a solid-state imaging device 102, an imaging control unit 103, an image processing unit 104, a storage unit 105, a display unit 106, and an operation unit 107. The imaging control unit 103, the image processing unit 104, the storage unit 105, the display unit 106, and the operation unit 107 are connected to one another via a bus 108. The imaging device 100 may be used as a standalone device, or may be incorporated into a mobile terminal such as a smartphone, an authentication device, a monitoring device, a vehicle, or a drone.

[0030] The optical system 101 causes light from a subject to be incident on the solid-state imaging device 102, and forms an optical image on the light-receiving surface of the solid-state imaging device 102. The optical system 101 may include, for example, a focus lens, a zoom lens, and an aperture. The optical system 101 may also include multiple lenses, such as a wide-angle lens, a standard lens, and a telephoto lens.

[0031] The solid-state imaging device 102 converts an optical image formed on the light-receiving surface into an electrical signal for each pixel, digitizes the electrical signal, and outputs it. The solid-state imaging device 102 can output a grayscale signal corresponding to the luminance of the incident light, an event signal that detects a temporal change in the luminance of the incident light as an event, or an event signal that detects a spatial change in the luminance of the incident light as an edge. The solid-state imaging device 102 may support CDS (Correlated Double Sampling) readout or DDS readout. Each pixel may include a single photodiode or multiple photodiodes with different sensitivities. The solid-state imaging device 102 is, for example, a complementary metal oxide semiconductor (CMOS) image sensor. The CMOS image sensor may be a back-illuminated image sensor or a front-illuminated image sensor. The solid-state imaging device 102 may also be a lateral overflow integration capacitor (LOFIC) image sensor.

[0032] The imaging control unit 103 controls imaging by the solid-state imaging device 102 based on instructions from the operation unit 107. At this time, the imaging control unit 103 can control the exposure time, exposure amount, imaging timing, etc. of the solid-state imaging device 102.

[0033] The image processing unit 104 performs image processing based on the output from the solid-state imaging device 102. The image processing includes, for example, gamma correction, white balance processing, sharpness processing, and tone conversion processing. The image processing unit 104 may include a processor that executes processing based on software.

[0034] The storage unit 105 stores images captured by the solid-state imaging device 102 and stores imaging parameters of the solid-state imaging device 102. The storage unit 105 can also store a program that operates the imaging device 100 based on software. The storage unit 105 may include a read-only memory (ROM), a random access memory (RAM), and a memory card.

[0035] The display unit 106 displays captured images and various information that supports the image capturing operation, etc. The display unit 106 may be a liquid crystal display or an organic EL (Electro Luminescence) display.

[0036] The operation unit 107 provides a user interface for operating the imaging device 100. The operation unit 107 may include, for example, buttons, dials, and switches provided on the imaging device 100. The operation unit 107 may be configured as a touch panel together with the display unit 106.

[0037] Depending on the configuration of the imaging device 100, some of the above functions may not be present, or conversely, the imaging device 100 may further include functions that are not disclosed.

[0038] FIG. 2 is a block diagram showing an example of the configuration of the solid-state imaging device according to the first embodiment.

[0039] In the figure, the solid-state imaging device 102 includes a pixel array unit 111, a vertical scanning circuit 112, a difference information output circuit 117, and a control circuit 116.

[0040] The pixel array unit 111 includes a plurality of pixels PX. The pixels PX are arranged in a matrix along the row direction (also referred to as the horizontal direction) and the column direction (also referred to as the vertical direction). Each pixel PX includes a photoelectric conversion unit that outputs a photoelectric conversion signal corresponding to the charge accumulated by photoelectric conversion. At this time, each pixel PX can generate difference information Sout based on the difference value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated.

[0041] The difference information Sout may be luminance information based on the difference value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated, or may be an event signal based on the difference value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated, or may be an edge signal based on the difference value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated.

[0042] Each pixel PX is connected to a horizontal drive line HSL for each row and to a vertical signal line VSL for each column. The horizontal drive line HSL drives each pixel PX for each row when reading out a signal from each pixel PX. The vertical signal line VSL transmits the difference information Sout read from the pixel PX to the difference information output circuit 117 for each column.

[0043] Each pixel PX may be a single pixel, a four-pixel shared pixel, or an eight-pixel shared pixel. The pixels PX may be arranged in a Bayer array or a quad-Bayer array. The light received by each pixel PX may be visible light, near infrared light (NIR), short wavelength infrared light (SWIR), ultraviolet light, X-rays, or the like.

[0044] The vertical scanning circuit 112 scans the pixels PX to be read in the column direction. The vertical scanning circuit 112 may be configured to include a vertical register. Here, when reading out signals from each pixel PX, the vertical scanning circuit 112 can drive each pixel PX row by row via a horizontal drive line HSL.

[0045] The difference information output circuit 117 outputs the difference information Sout read from the pixel PX in a predetermined format. For example, the difference information output circuit 117 may output the difference information Sout for each frame.

[0046] The control circuit 116 controls the vertical scanning circuit 112 and the differential information output circuit 117. For example, the control circuit 116 can control the generation timing of the differential information Sout, the scanning timing in the column direction, the scanning timing in the row direction, and the output timing of the differential information output circuit 117.

[0047] FIG. 3 is a diagram showing an example of a circuit configuration of a pixel provided in the solid-state imaging device according to the first embodiment.

[0048] In the figure, the pixel PX includes a photoelectric conversion unit 121 , a subtractor 131 , a sample-and-hold circuit 141 , and an AD converter 151 .

[0049] The photoelectric conversion unit 121 photoelectrically converts incident light of illuminance L and outputs a photoelectric conversion signal VP corresponding to the accumulated charge. The photoelectric conversion unit 121 includes a photodiode PD, a reset transistor 122, an output transistor 123, and a bias transistor 124. The reset transistor 122, the output transistor 123, and the bias transistor 124 may be MOS (Metal Oxide Semiconductor) transistors.

[0050] The cathode of the photodiode PD is connected to the gate of the output transistor 123. The output transistor 123 and the bias transistor 124 are connected in series. A bias voltage Vb is applied to the gate of the bias transistor 124. In this state, the bias transistor 124 can operate as a current source. The output transistor 123 and the bias transistor 124 can form a source follower circuit. A photoelectric conversion signal VP corresponding to the charge accumulated by photoelectric conversion is output from the source of the output transistor 123.

[0051] The reset transistor 122 resets the charge accumulated in the photodiode PD. The reset transistor 122 is connected in series to the photodiode PD. A reset signal RSB is applied to the gate of the reset transistor 122. At this time, the reset transistor 122 can be prevented from being reset during the output of the photoelectric conversion signal VP used to generate the difference value VD.

[0052] The drain of the reset transistor 122 and the drain of the output transistor 123 are connected to the power supply potential VDD. The anode of the photodiode PD and the source of the bias transistor 124 are connected to the ground potential.

[0053] The subtractor 131 generates a difference value VD of the photoelectric conversion signal VP. The subtractor 131 includes a capacitor C1, a capacitor C2, an inverter 132, a buffer 133, and a switch AW. The capacitor C1 is connected between the source of the output transistor 123 and the input of the inverter 132. The capacitor C2 is connected between the output and input of the inverter 132. The buffer 133 is connected after the inverter 132.

[0054] The switch AW is turned on and off based on the switching signal AZ. When the switch AW is turned on, charge corresponding to the current photoelectric conversion signal VP is accumulated in the capacitor C1. When the switch AW is turned off, the photoelectric conversion signal VP obtained when the charge of the photoelectric conversion unit 121 is accumulated is applied to the inverter 132 via the capacitor C1. At this time, a difference value VD of the photoelectric conversion signal VP obtained when the charge of the photoelectric conversion unit 121 is accumulated is output from the inverter 132 via the buffer 133.

[0055] The sample-and-hold circuit 141 samples and holds a difference value VD of the photoelectric conversion signal VP when the charges of the photoelectric conversion unit 121 are accumulated. The sample-and-hold circuit 141 includes a sample-and-hold switch SW1 and a sample-and-hold capacitor C3. The sample-and-hold switch SW1 is connected in series to the output of the buffer 133. The sample-and-hold capacitor C3 is connected in parallel to the sample-and-hold switch SW1.

[0056] The sample-and-hold switch SW1 is turned on and off based on the sample-and-hold signal SH1. When the sample-and-hold switch SW1 is turned on, the difference value VD of the photoelectric conversion signal VP is sampled in the sample-and-hold capacitor C3. When the sample-and-hold switch SW1 is turned off, the difference value VD of the photoelectric conversion signal VP is held in the sample-and-hold capacitor C3.

[0057] The AD converter 151 AD converts the difference value VD sampled and held by the sample and hold circuit 141. The AD converter 151 includes a comparator 152. The difference value VD sampled and held by the sample and hold circuit 141 is applied to a non-inverting input of the comparator 152. A reference signal VRE is applied to an inverting input of the comparator 152. The reference signal VRE may include a ramp wave. The AD converter 151 is an example of a difference information generating unit recited in the claims.

[0058] FIG. 4 is a diagram illustrating an example of a method for generating difference information according to the first embodiment.

[0059] In the figure, the photodiode PD accumulates an electric charge EL obtained by photoelectrically converting incident light of illuminance L. At this time, the electric charge EL accumulated in the photodiode PD is accumulated over a plurality of frames FM1 to FM6, etc. Note that the units of frames FM1 to FM6, etc., can be set to a periodic period. At this time, the reset transistor 122 can prevent the electric charge accumulated in the photodiode PD from being reset between frames FM1 to FM6, etc.

[0060] Here, the subtractor 131 can generate a difference value VD1 between the photoelectric conversion signal VP after resetting the photodiode PD and the photoelectric conversion signal VP of frame FM1. The subtractor 131 can also generate a difference value VD2 between the photoelectric conversion signals VP of each of frames FM1 and FM2, a difference value VD3 between the photoelectric conversion signals VP of each of frames FM2 and FM3, a difference value VD4 between the photoelectric conversion signals VP of each of frames FM3 and FM4, a difference value VD5 between the photoelectric conversion signals VP of each of frames FM4 and FM5, and a difference value VD6 between the photoelectric conversion signals VP of each of frames FM5 and FM6.

[0061] 5 is a diagram showing the relationship between illuminance, the cathode voltage of the photodiode, and the difference information according to the first embodiment, in which the charge accumulation period of the photodiode PD is 20 frames.

[0062] In the figure, the photodiode PD can be reset at the start and end of the charge accumulation period, and the photodiode PD can be prevented from being reset from the start to the end of the charge accumulation period.

[0063] The illuminance L of light incident on the photodiode PD gradually increases and then gradually decreases from the start to the end of the charge accumulation period. At this time, the photoelectric conversion signal VP gradually decreases from the start to the end of the charge accumulation period. Furthermore, by calculating the difference in the photoelectric conversion signal VP between each frame, a difference value VD of the photoelectric conversion signal VP is generated for each frame.

[0064] Fig. 6 is a diagram showing another example of the circuit configuration of the pixel provided in the solid-state imaging device according to the first embodiment. Note that Fig. 6 shows an example in which the pixels PX of Fig. 3 are arrayed across two rows and two columns.

[0065] 6, the pixels PX in FIG. 3 can be arranged in an array. For example, pixels PX1 to PX4 are arranged in an array of two rows and two columns in the pixel array unit 111. In this case, vertical signal lines VSL1 and VSL2 are provided for each column as vertical signal lines VSL.

[0066] In each of the pixels PX1 to PX4, a counter 153, an output transistor 154, and a selection transistor 155 are added to the pixel PX of Fig. 3. The output transistor 154 and the selection transistor 155 can be MOS transistors.

[0067] The counter 153 performs counting based on the output of the comparator 152. At this time, the counter 153 can AD convert the difference value VD of the photoelectric conversion signal VP based on the count value until the output of the comparator 152 is inverted. The AD converted difference value VD can be used as luminance information.

[0068] The counter 153 is connected to the rear stage of the comparator 152. The output transistor 154 outputs the count value of the counter 153. The selection transistor 155 selects the output of the output transistor 154. The output transistor 154 is connected to vertical signal lines VSL1 and VSL2 for each column via the selection transistor 155. The gate of the output transistor 154 is connected to the output of the counter 153. Selection signals K1 to K4 are applied to the gates of the selection transistors 155 of each of the pixels PX1 to PX4, respectively. The selection signals K1 to K4 can select each of the pixels PX1 to PX4 for each row.

[0069] In this way, in the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectrically converted signal VP. This makes it possible to obtain luminance information from the photoelectrically converted signal VP without obtaining a gradation signal from the photoelectrically converted signal VP, thereby improving the flexibility of the frame rate when obtaining the luminance information.

[0070] 2. Second Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this second embodiment, an event signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP.

[0071] FIG. 7 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the second embodiment.

[0072] In the figure, this pixel includes a sample-and-hold circuit 161 and an event determination circuit 171 instead of the sample-and-hold circuit 141 and AD converter 151 of the first embodiment described above. Other configurations of the pixel of the second embodiment are similar to those of the pixel of the first embodiment described above.

[0073] The sample-and-hold circuit 161 samples and holds the difference value VD of the photoelectric conversion signal VP across the past and present when the charge of the photoelectric conversion unit 121 is accumulated. The sample-and-hold circuit 161 includes sample-and-hold switches SW2 and SW3, sample-and-hold capacitors C4 and C5, and buffers 162 and 163. Each of the sample-and-hold switches SW2 and SW3 is connected to the output of the buffer 133. Each of the sample-and-hold capacitors C4 and C5 is connected in parallel to the sample-and-hold switches SW2 and SW3, respectively. The buffer 162 is connected to the connection point between the sample-and-hold switch SW2 and the sample-and-hold capacitor C4. The buffer 163 is connected to the connection point between the sample-and-hold switch SW3 and the sample-and-hold capacitor C5.

[0074] The sample and hold switches SW2 and SW3 are turned on and off based on the sample and hold signals SH2 and SH3, respectively. The timing at which the sample and hold switches SW2 and SW3 are turned on and off can be shifted by a time equivalent to one frame of the differential value VD. When the sample and hold switch SW2 is turned on, the previous differential value VD for one frame is sampled in the sample and hold capacitor C4. When the sample and hold switch SW2 is turned off, the previous differential value VD for one frame is held in the sample and hold capacitor C4. When the sample and hold switch SW3 is turned on, the current differential value VD is sampled in the sample and hold capacitor C5. When the sample and hold switch SW3 is turned off, the current differential value VD is held in the sample and hold capacitor C5. When the event determination circuit 171 detects an event, the sample and hold circuit 161 discards the previous differential value VD and samples and holds a new differential value VD.

[0075] The event determination circuit 171 detects a temporal change in the luminance of incident light as an event by calculating the difference between the current level of the difference value VD and a past difference value VD as a reference level. The event determination circuit 171 may distinguish between an event in which the luminance of incident light increases (positive event) and an event in which the luminance of incident light decreases (negative event). The event determination circuit 171 includes capacitors C11 to C18 and comparators 172 and 163.

[0076] The capacitors C11 and C16 are connected to the output of the buffer 162. The capacitors C12 and C15 are connected to the output of the buffer 163. The capacitors C13, C14, C17, and C18 are connected in parallel to the capacitors C11, C12, C15, and C16, respectively. The non-inverting input of the comparator 172 is connected to the connection point of the capacitors C11 and C13. The inverting input of the comparator 172 is connected to the connection point of the capacitors C12 and C14. The non-inverting input of the comparator 173 is connected to the connection point of the capacitors C15 and C17. The inverting input of the comparator 173 is connected to the connection point of the capacitors C16 and C18. In this case, the comparator 172 can output a positive event as an event signal EV1, and the comparator 173 can output a negative event as an event signal EV2.

[0077] The difference value VD of the past one frame, sampled and held by the sample-and-hold capacitor C4, is voltage-adjusted by the capacitors C11 and C13, and then applied to the non-inverting input of the comparator 172. The current difference value VD sampled and held by the sample-and-hold capacitor C5 is voltage-adjusted by the capacitors C12 and C14, and then applied to the inverting input of the comparator 172. These values ​​are compared by the comparator 172, and if it is determined that the difference value VD is equal to or greater than the threshold, the comparator 172 outputs an event signal EV1.

[0078] Furthermore, the difference value VD of one frame ago sampled and held by the sample-and-hold capacitor C4 is voltage-adjusted by the capacitors C16 and C18 and then applied to the inverting input of the comparator 173. Furthermore, the current difference value VD sampled and held by the sample-and-hold capacitor C5 is voltage-adjusted by the capacitors C15 and C17 and then applied to the non-inverting input of the comparator 173. These values ​​are compared by the comparator 173, and if it is determined that the difference value VD is equal to or greater than the threshold value, the comparator 173 outputs an event signal EV2.

[0079] FIG. 8 is a diagram showing the relationship between illuminance, cathode voltage of a photodiode, and difference information according to the second embodiment.

[0080] In the figure, the event determination circuit 171 outputs an event signal EV1 as a positive event Pos when the difference between the past difference value VD and the current difference value VD increases by a threshold value or more, and outputs an event signal EV2 as a negative event Neg when the difference between the past difference value VD and the current difference value VD decreases by a threshold value or more.

[0081] FIG. 9 is a diagram showing another example of the circuit configuration of a pixel provided in the solid-state imaging device according to the second embodiment.

[0082] 9, this pixel includes an event determination circuit 171' in place of the event determination circuit 171 in Fig. 7. Other configurations of the pixel in Fig. 9 are the same as those of the pixel in Fig. 7.

[0083] The event determination circuit 171' includes capacitors C21 to C36 instead of the capacitors C11 to C18 in FIG. 7. Furthermore, the event determination circuit 171' includes switches SK1 to SK16 in addition to the components of the event determination circuit 171 in FIG. 7. The capacitors C21 to C36 and the switches SK1 to SK16 can be used to change the threshold value during event determination. The remaining configuration of the event determination circuit 171' in FIG. 9 is the same as the configuration of the event determination circuit 171 in FIG. 7.

[0084] Each of the switches SK1 to SK4 and SK13 to SK16 is connected to the output of the buffer 162. Each of the switches SK5 to SK8 and SK9 to SK12 is connected to the output of the buffer 163. Each of the switches SK1 to SK4 and SK13 to SK16 is connected in series to the capacitors C21 to C24 and C33 to C36, respectively. Each of the switches SK5 to SK8 and SK9 to SK12 is connected in series to the capacitors C25 to C28 and C29 to C32, respectively. The non-inverting input of the comparator 172 is connected to the connection point of the capacitors C11 and C13. The non-inverting input of the comparator 172 is connected to the capacitors C21 to C24. The inverting input of the comparator 172 is connected to the capacitors C25 to C28. The non-inverting input of the comparator 173 is connected to the capacitors C29 to C32. The inverting input of the comparator 173 is connected to the capacitors C33 to C36.

[0085] Here, by switching each of the switches SK1 to SK16, it is possible to change the number of parallel capacitances added to the inputs of the comparators 172 and 163, and to change the threshold value for event determination.

[0086] In this way, in the second embodiment described above, the event signals EV1 and EV2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP. This makes it possible to obtain the event signals EV1 and EV2 from the photoelectric conversion signal VP without obtaining a gradation signal from the photoelectric conversion signal VP, thereby improving the flexibility of the frame rate when obtaining the event signals EV1 and EV2.

[0087] 3. Third Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this third embodiment, an edge signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP.

[0088] FIG. 10 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the third embodiment.

[0089] In the figure, this imaging device includes pixels PX1 and PX2 instead of the pixel PX of the first embodiment described above. Each of the pixels PX1 and PX2 includes the photoelectric conversion unit 121 and subtractor 131 of the first embodiment described above. The imaging device also includes a sample-and-hold circuit 181 and an edge determination circuit 191. Other configurations of the imaging device of the third embodiment are similar to those of the imaging device of the first embodiment described above.

[0090] The sample-and-hold circuit 181 samples and holds, at different positions, the difference value VD of the photoelectric conversion signal VP when the charge of the photoelectric conversion unit 121 is accumulated. The sample-and-hold circuit 181 includes sample-and-hold switches SW4 and SW5, sample-and-hold capacitors C6 and C7, and buffers 182 and 183. Each sample-and-hold switch SW4 and SW5 is connected to the output of the buffer 133 of each pixel PX1 and PX2. Each sample-and-hold capacitor C6 and C7 is connected in parallel to the sample-and-hold switches SW4 and SW5. The buffer 182 is connected to the connection point between the sample-and-hold switch SW4 and the sample-and-hold capacitor C6. The buffer 183 is connected to the connection point between the sample-and-hold switch SW5 and the sample-and-hold capacitor C7.

[0091] The sample and hold switches SW4 and SW5 are turned on and off based on the sample and hold signals SH4 and SH5, respectively. At this time, the sample and hold switches SW4 and SW5 may be turned on and off simultaneously. Here, when the sample and hold switch SW4 is turned on, the difference value VD of pixel PX1 is sampled in the sample and hold capacitor C6. Then, when the sample and hold switch SW4 is turned off, the difference value VD of pixel PX1 is held in the sample and hold capacitor C6. Also, when the sample and hold switch SW5 is turned on, the difference value VD of pixel PX2 is sampled in the sample and hold capacitor C7. Then, when the sample and hold switch SW5 is turned off, the difference value VD of pixel PX2 is held in the sample and hold capacitor C7.

[0092] The edge determination circuit 191 detects a spatial change in the luminance of the incident light as an edge by calculating the difference between the levels of the difference values ​​VD of the pixels PX1 and PX2. The event determination circuit 171 may distinguish between an edge where the luminance of the incident light increases in one direction (positive edge) and an edge where the luminance of the incident light decreases in one direction (negative edge). The edge determination circuit 191 includes capacitors C31 to C38 and comparators 192 and 193.

[0093] The capacitors C31 and C36 are connected to the output of the buffer 182. The capacitors C32 and C35 are connected to the output of the buffer 183. The capacitors C13, C32, C35, and C36 are connected in parallel to the capacitors C33, C34, C37, and C38, respectively. The non-inverting input of the comparator 192 is connected to the connection point of the capacitors C31 and C33. The inverting input of the comparator 192 is connected to the connection point of the capacitors C32 and C34. The non-inverting input of the comparator 173 is connected to the connection point of the capacitors C35 and C37. The inverting input of the comparator 173 is connected to the connection point of the capacitors C36 and C38. In this case, the comparator 192 can output a positive edge as the edge signal EG1, and the comparator 193 can output a negative edge as the edge signal EG2.

[0094] The difference value VD of pixel PX1 sampled and held by sample-and-hold capacitor C6 is voltage-adjusted by capacitors C31 and C33, and then applied to the non-inverting input of comparator 192. The difference value VD of pixel PX2 sampled and held by sample-and-hold capacitor C7 is voltage-adjusted by capacitors C32 and C34, and then applied to the inverting input of comparator 192. These values ​​are compared by comparator 192, and if it is determined that the difference value VD is equal to or greater than the threshold, comparator 192 outputs edge signal EG1.

[0095] Furthermore, the difference value VD of pixel PX1 sampled and held by sample-and-hold capacitor C6 is voltage-adjusted by capacitors C36 and C38, and then applied to the inverting input of comparator 193. Furthermore, the difference value VD of pixel PX2 sampled and held by sample-and-hold capacitor C7 is voltage-adjusted by capacitors C35 and C37, and then applied to the non-inverting input of comparator 193. These values ​​are then compared by comparator 193, and if it is determined that the difference value VD is equal to or greater than the threshold, edge signal EG2 is output from comparator 193.

[0096] 11 is a diagram showing the relationship between illuminance, photodiode cathode voltage, and difference information according to the third embodiment, in which the charge accumulation period of the photodiode PD of each of the pixels PX1 and PX2 is 20 frames.

[0097] In the figure, the photodiode PD of each pixel PX1, PX2 can be reset at the start and end of the charge accumulation period of each pixel PX1, PX2. From the start to the end of the charge accumulation period of each pixel PX1, PX2, the photodiode PD of each pixel PX1, PX2 can be prevented from being reset.

[0098] The illuminance L of light incident on the photodiode PD of each pixel PX1, PX2 gradually increases and then gradually decreases from the start to the end of the charge accumulation period. At this time, the photoelectric conversion signal VP of each pixel PX1, PX2 gradually decreases from the start to the end of the charge accumulation period. Furthermore, for each pixel PX1, PX2, the difference between the photoelectric conversion signal VP before and after each frame is calculated, thereby generating a difference value VD of the photoelectric conversion signal VP for each frame.

[0099] The edge determination circuit 191 outputs an edge signal EG1 as a positive event Pos when the difference between the difference values ​​VD of the pixels PX1 and PX2 increases by a threshold value or more, and outputs an edge signal EG2 as a negative event Neg when the difference between the difference values ​​VD of the pixels PX1 and PX2 decreases by a threshold value or more.

[0100] In this way, in the third embodiment described above, the edge signals EG1 and EG2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP. This makes it possible to obtain the edge signals EG1 and EG2 from the photoelectric conversion signal VP without obtaining a gradation signal from the photoelectric conversion signal VP, thereby improving the flexibility of the frame rate when obtaining the edge signals EG1 and EG2.

[0101] 4. Fourth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this fourth embodiment, an event signal and an edge signal are generated as difference information based on the difference value VD of the photoelectric conversion signal VP.

[0102] FIG. 12 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the fourth embodiment.

[0103] In the figure, this pixel has a sample-and-hold circuit 411 and an event / edge determination circuit 431 instead of the sample-and-hold circuit 181 and edge determination circuit 191 of the third embodiment described above. Furthermore, this pixel has a switching circuit 421 added to the pixel of the third embodiment described above. Other than that, the configuration of the pixel of the fourth embodiment is the same as the configuration of the pixel of the third embodiment described above.

[0104] The sample and hold circuit 411 samples and holds the difference value VD of each pixel PX1, PX2 across the past and present. The sample and hold circuit 411 includes sample and hold switches SW11 to SW14, sample and hold capacitors C41 to C44, and buffers 412 to 415. Each sample and hold switch SW11, SW12 is connected to the output of the buffer 133 of pixel PX1. Each sample and hold switch SW13, SW14 is connected to the output of the buffer 133 of pixel PX2. Each sample and hold capacitor C41 to C44 is connected in parallel to the sample and hold switches SW11 to SW14, respectively. The buffer 412 is connected to the connection point between the sample and hold switch SW11 and the sample and hold capacitor C41. The buffer 413 is connected to the connection point between the sample and hold switch SW12 and the sample and hold capacitor C42. The buffer 414 is connected to the connection point between the sample and hold switch SW13 and the sample and hold capacitor C43. The buffer 415 is connected to the connection point between the sample-and-hold switch SW14 and the sample-and-hold capacitor C44.

[0105] The sample-and-hold switches SW11 to SW14 are turned on and off based on the sample-and-hold signals SH11 to SH14, respectively. At this time, the timing at which the sample-and-hold switches SW11 and SW12 are turned on and off can be shifted by a time equivalent to one frame of the difference value VD of pixel PX1. The timing at which the sample-and-hold switches SW13 and SW14 are turned on and off can be shifted by a time equivalent to one frame of the difference value VD of pixel PX2.

[0106] Here, when the sample and hold switch SW11 is turned on, the difference value VD of the pixel PX1 of the past by one frame is sampled in the sample and hold capacitor C41. Then, when the sample and hold switch SW11 is turned off, the difference value VD of the pixel PX1 of the past by one frame is held in the sample and hold capacitor C41. Also, when the sample and hold switch SW12 is turned on, the difference value VD of the current pixel PX1 is sampled in the sample and hold capacitor C42. Then, when the sample and hold switch SW12 is turned off, the difference value VD of the current pixel PX1 is held in the sample and hold capacitor C42. When the sample and hold switch SW13 is turned on, the difference value VD of the pixel PX2 of the past by one frame is sampled in the sample and hold capacitor C43. Then, when the sample and hold switch SW13 is turned off, the difference value VD of the pixel PX2 of the past by one frame is held in the sample and hold capacitor C43. When the sample-and-hold switch SW14 is turned on, the difference value VD of the current pixel PX2 is sampled in the sample-and-hold capacitor C44, and when the sample-and-hold switch SW14 is turned off, the difference value VD of the current pixel PX2 is held in the sample-and-hold capacitor C44.

[0107] The switching circuit 421 switches between event determination and edge determination. The switching circuit 421 includes switches KW1 and KW2. The switch KW1 switches the input to the capacitors C52 and C55 between buffers 412 and 413. The switch KW2 switches the input to the capacitors C62 and C65 between buffers 414 and 415. Here, in the event determination, the switch KW1 is switched to the buffer 413, and the switch KW2 is switched to the buffer 415. In the edge determination, the switch KW1 is switched to the buffer 412, and the switch KW2 is switched to the buffer 414.

[0108] The event edge determination circuit 431 detects a temporal change in the luminance of the incident light for each pixel PX1, PX2 as an event by determining the difference between the level of the difference value VD of each pixel PX1, PX2 at the present time and the past difference value VD of each pixel PX1, PX2 as a reference level. The event edge determination circuit 431 also detects a spatial change in the luminance of the incident light as an edge by determining the difference between the level of the difference value VD of each pixel PX1, PX2 at the past time and the present time. The event edge determination circuit 431 includes capacitors C51 to C58, C61 to C68, and comparators 432 to 435.

[0109] Each of the capacitors C51 and C56 is connected to the output of the buffer 412. Each of the capacitors C52 and C55 is connected to a selector switch KW1. Each of the capacitors C61 and C66 is connected to the output of the buffer 414. Each of the capacitors C62 and C65 is connected to a selector switch KW2. Each of the capacitors C53, C54, C57, C58, C63, C64, C67, and C68 is connected in parallel to each of the capacitors C51, C52, C55, C56, C61, C62, C65, and C66, respectively. The non-inverting input of the comparator 432 is connected to the connection point of the capacitors C51 and C53. The inverting input of the comparator 432 is connected to the connection point of the capacitors C52 and C54. The non-inverting input of the comparator 433 is connected to the connection point of the capacitors C55 and C57. The inverting input of the comparator 433 is connected to the connection point of the capacitors C56 and C58. The non-inverting input of the comparator 434 is connected to the connection point of the capacitors C61 and C63. The inverting input of the comparator 434 is connected to the connection point of the capacitors C62 and C64. The non-inverting input of the comparator 435 is connected to the connection point of the capacitors C65 and C67. The inverting input of the comparator 435 is connected to the connection point of the capacitors C66 and C68.

[0110] In the event determination, the comparator 432 outputs a positive event of pixel PX1 as an event signal EV11, and the comparator 433 outputs a negative event of pixel PX1 as an event signal EV12. The comparator 434 outputs a positive event of pixel PX2 as an event signal EV21, and the comparator 435 outputs a negative event of pixel PX2 as an event signal EV22.

[0111] In the edge determination, the comparator 432 outputs a current positive edge as an edge signal EG11, and the comparator 433 outputs a current negative edge as an edge signal EG12. The comparator 434 outputs a past positive edge as an edge signal EG21, and the comparator 435 outputs a past negative edge as an edge signal EG22.

[0112] As described above, in the fourth embodiment, the event signals EV11, EV12, EV21, EV22 and the edge signals EG11, EG12, EG21, EG22 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP. This makes it possible to obtain the event signals EV11, EV12, EV21, EV22 and the edge signals EG11, EG12, EG21, EG22 from the photoelectric conversion signal VP without obtaining a gradation signal from the photoelectric conversion signal VP, thereby improving the flexibility of the frame rate when obtaining the event signals EV11, EV12, EV21, EV22 and the edge signals EG11, EG12, EG21, EG22.

[0113] 5. Fifth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this fifth embodiment, for the same pixel, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP.

[0114] FIG. 13 is a block diagram showing an example of the configuration of a solid-state imaging device according to the fifth embodiment.

[0115] In the figure, the solid-state imaging device 502 includes a pixel array unit 511 , a vertical scanning circuit 512 , a column readout circuit 113 , a column signal processing unit 114 , a horizontal scanning circuit 115 , a difference information output circuit 117 , and a control circuit 516 .

[0116] The pixel array unit 511 includes a plurality of pixels HPX. Each pixel HPX generates a luminance signal as differential information based on a differential value VD of the photoelectric conversion signal VP, and generates a grayscale signal based on the photoelectric conversion signal VP. Note that the pixel HPX is an example of a hybrid pixel as defined in the claims. The pixels HPX are arranged in a matrix along the row and column directions. Each pixel HPX can form a source follower with the column readout circuit 113 when reading out a grayscale signal. Each pixel HPX is connected to a horizontal drive line HSL for each row and to vertical signal lines VSLA and VSLB for each column. The horizontal drive line HSL drives each pixel HPX for each row when reading out a signal from each pixel HPX. The vertical signal line VSLA transmits the grayscale signal read out from the pixel HPX to the column signal processing unit 114 for each column. The vertical signal line VSLB transmits the difference information Sout read out from the pixel HPX to the difference information output circuit 117 for each column.

[0117] Each pixel HPX may be a single pixel, a four-pixel shared pixel, or an eight-pixel shared pixel. The pixels HPX may be arranged in a Bayer array or a quad-Bayer array. The light received by each pixel HPX may be visible light, near-infrared light, short-wave infrared light, ultraviolet light, X-rays, or the like.

[0118] The vertical scanning circuit 512 scans the pixels HPX to be read in the column direction. The vertical scanning circuit 512 may be configured to include a vertical register. Here, when reading out signals from each pixel HPX, the vertical scanning circuit 512 can drive each pixel HPX row by row via the horizontal drive line HSL.

[0119] The column readout circuit 113 can configure a source follower with each pixel HPX when reading out a grayscale signal from the pixel HPX. At this time, the column readout circuit 113 can change the potential of the vertical signal line VSLA for each column based on the charge held in each pixel HPX.

[0120] The column signal processing unit 114 processes the grayscale signals transmitted in the column direction from each pixel HPX. For example, the column signal processing unit 114 can perform CDS processing based on the signals transmitted in the column direction from each pixel HPX. The column signal processing unit 114 can also perform AD conversion processing based on the signals transmitted in the column direction from each pixel HPX and output the image pickup signal Gout. The column signal processing unit 114 includes a column ADC unit 114A.

[0121] The column ADC unit 114A can perform AD conversion processing in parallel for each column. At this time, the column ADC unit 114A can perform AD conversion for each column based on the comparison result between the gradation signal read from the pixel HPX and the reference signal REF. This AD conversion may be single-slope AD conversion.

[0122] The horizontal scanning circuit 115 scans the pixels HPX to be read in the row direction. The horizontal scanning circuit 115 may be configured to include a horizontal register.

[0123] The control circuit 516 controls the vertical scanning circuit 512, the column readout circuit 113, the column signal processing unit 114, the horizontal scanning circuit 115, and the differential information output circuit 117. For example, the control circuit 516 can control the generation timing of the differential information Sout, the scanning timing in the column direction, the scanning timing in the row direction, the operation timing of the column readout circuit 113, the processing timing of the column signal processing unit 114, and the output timing of the differential information output circuit 117. At this time, the control circuit 516 can coordinate the vertical scanning circuit 112, the column readout circuit 113, the column signal processing unit 114, and the horizontal scanning circuit 115 so that the accumulation operation, the shutter operation, and the read operation are performed for each row in each frame.

[0124] FIG. 14 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the fifth embodiment.

[0125] In the figure, the pixel HPX includes a photoelectric conversion unit 521 instead of the photoelectric conversion unit 121 of the first embodiment described above. The other configuration of the pixel HPX of the fifth embodiment is the same as the configuration of the pixel PX of the first embodiment described above.

[0126] The photoelectric conversion unit 521 is obtained by adding a transfer transistor 522, a reset transistor 523, an amplification transistor 524, a selection transistor 525, and a floating diffusion FD to the photoelectric conversion unit 121 of the first embodiment described above. Furthermore, the photoelectric conversion unit 521 is obtained by removing the reset transistor 122 from the photoelectric conversion unit 121 of the first embodiment described above. Other configurations of the photoelectric conversion unit 521 of the fifth embodiment are the same as those of the photoelectric conversion unit 121 of the first embodiment described above. MOS transistors can be used as the transfer transistor 522, the reset transistor 523, the amplification transistor 524, and the selection transistor 525.

[0127] The amplification transistor 524 and the selection transistor 525 are connected in series. The cathode of the photodiode PD is connected to the floating diffusion FD via the transfer transistor 522. The floating diffusion FD is connected to the power supply potential VDD via the reset transistor 523. The power supply potential VDD is connected to the vertical signal line VSLA via the series circuit of the amplification transistor 524 and the selection transistor 525. The gate of the amplification transistor 524 is connected to the floating diffusion FD.

[0128] Here, the photodiode PD, transfer transistor 522, reset transistor 523, amplification transistor 524, selection transistor 525, and floating diffusion FD can constitute a gradation information generation unit. The photodiode PD, output transistor 123, and bias transistor 124 can constitute a photoelectric conversion signal output unit. In this case, the photodiode PD is shared by the gradation information generation unit and the photoelectric conversion signal output unit.

[0129] A transfer signal TGL is applied to the gate of the transfer transistor 522. A reset signal RST is applied to the gate of the reset transistor 523. A selection signal SEL is applied to the gate of the selection transistor 525. The transfer signal TGL, reset signal RST, and selection signal SEL can be transmitted to each pixel HPX via the horizontal drive line HSL in FIG. 2 .

[0130] When the transfer transistor 522 is turned on, the charge accumulated in the photodiode PD is transferred to the floating diffusion FD. When the selection transistor 525 is turned on, the source potential of the amplification transistor 524 changes depending on the potential of the floating diffusion FD. The source potential of the amplification transistor 524 is applied to the vertical signal line VSLA via the selection transistor 525 and transmitted via the vertical signal line VSLA. When the reset transistor 523 is turned on, the charge accumulated in the floating diffusion FD is discharged.

[0131] Here, the differential information Sout is generated based on the source follower output of the photoelectric conversion signal VP, which is the cathode potential of the photodiode PD. The gradation signal is generated based on the source follower output of the potential VF of the floating diffusion FD, which reflects the photoelectric conversion signal VP. In this case, the frame rate of the gradation signal can be made slower than the frame rate of the differential information Sout, thereby achieving low noise in the gradation signal. Furthermore, the image processing unit 104 can generate a high dynamic range (HDR) image based on the luminance information and gradation signal generated based on the differential value VD.

[0132] FIG. 15 is a diagram illustrating an example of a method for generating difference information according to the fifth embodiment.

[0133] In the figure, the charge EL stored in the photodiode PD is transferred to the floating diffusion FD, and a grayscale signal is generated based on the source follower output of the potential VF of the floating diffusion FD.

[0134] FIG. 16 is a timing chart showing an example of the operation of the solid-state imaging device according to the fifth embodiment.

[0135] In the figure, a reset period H1, a charge accumulation period H2, a P-phase generation period H6, and a D-phase generation period H7 are set in the frame FK for generating the gradation signal. A frame SK for generating differential information is also set in the frame FK for generating the gradation signal. A plurality of frames SK for generating differential information may be set in the frame FK for generating the gradation signal. A differential value sampling period H3, a differential value acquisition period H4, and a subtractor reset period H5 are set in the frame SK for generating differential information. The frame SK for generating differential information and the P-phase generation period H6 can be set in the charge accumulation period H2.

[0136] In the reset period H1, the switching signal AZ, the reset signal RST, and the transfer signal TGL rise, and the subtractor 131, the photodiode PD, and the floating diffusion FD are reset.

[0137] In the charge accumulation period H2, the switching signal AZ, the reset signal RST, and the transfer signal TGL fall, and charge is accumulated in the photodiode PD according to the illuminance L of the incident light.

[0138] During the P-phase generation period H6, the selection signal SEL rises, turning on the selection transistor 525. At this time, the amplification transistor 524 is connected to the vertical signal line VSLA via the selection transistor 525. In addition, the reset signal RST rises, resetting the floating diffusion FD. After the reset signal RST falls, the P-phase level (also referred to as the reset level) of the floating diffusion FD is read out from the pixel HPX based on the source follower operation when it is applied to the vertical signal line VSLA.

[0139] In the D-phase generation period H7, the transfer signal TGL rises, and the charge accumulated in the photodiode PD is transferred to the floating diffusion FD. After the transfer signal TGL falls, the D-phase level (also referred to as the signal level) of the floating diffusion FD is read out from the pixel HPX based on the source follower operation when the D-phase level (also referred to as the signal level) of the floating diffusion FD is applied to the vertical signal line VSLA.

[0140] In the difference value sampling period H3, the sample and hold signal SH1 rises, and the difference value VD of the photoelectric conversion signal VP is sampled by the sample and hold capacitor C3.

[0141] During the difference value acquisition period H4, the sample and hold signal SH1 falls, and the difference value VD of the photoelectric conversion signal VP is held in the sample and hold capacitor C3.

[0142] In the subtractor reset period H5, the switching signal AZ rises and the subtractor 131 is reset.

[0143] Fig. 17 is a diagram showing another example of the circuit configuration of a pixel provided in a solid-state imaging device according to the fifth embodiment. Fig. 17 shows an example in which the pixels HPX of Fig. 14 are arrayed across two rows and two columns.

[0144] 17, the pixels HPX in FIG. 14 can be arranged in an array. For example, in the pixel array section 511, pixels HPX1 to HPX4 are arranged in an array across two rows and two columns. In this case, vertical signal lines VSLA1 and VSLA2 are provided for each column as the vertical signal line VSLA. The vertical signal line VSLB may be shared by two pixels HPX1 to HPX4 adjacent to each other in the row direction. In this case, the pixels HPX1 and HPX2 can be arranged as mirror images of the pixels HPX3 and HPX4.

[0145] Each of the pixels HPX1 to HPX4 has the counter 153, output transistor 154, and selection transistor 155 of FIG. 6 added to the pixel HPX of FIG.

[0146] The sources of the selection transistors 525 of the pixels HPX1 and HPX2 are connected to a vertical signal line VSLA1. The sources of the selection transistors 525 of the pixels HPX3 and HPX4 are connected to a vertical signal line VSLA2. The sources of the selection transistors 155 of the pixels HPX1 to HPX4 are connected to a vertical signal line VSLB.

[0147] Transfer signals TG1 to TG4 are applied to the gates of the transfer transistors 522 of the pixels HPX1 to HPX4, respectively. Reset signals RS1 to RS4 are applied to the gates of the reset transistors 523 of the pixels HPX1 to HPX4, respectively. Select signals SE1 to SE4 are applied to the gates of the select transistors 525 of the pixels HPX1 to HPX4, respectively.

[0148] As described above, in the fifth embodiment, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP. This makes it possible to simultaneously acquire the luminance signal and the gradation signal at different frame rates based on the photoelectric conversion signal VP generated by the same pixel HPX. This makes it possible to reduce noise in the gradation signal and achieve high-precision HDR synthesis while suppressing a decrease in frame rate.

[0149] 6. Sixth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this sixth embodiment, difference information is generated based on the difference value of the photoelectric conversion signal after AD conversion of the photoelectric conversion signal, and a gradation signal is generated based on the photoelectric conversion signal.

[0150] FIG. 18 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the sixth embodiment.

[0151] In the figure, this pixel includes a subtractor 611 instead of the subtractor 131 of the first embodiment described above. Other configurations of the pixel of the sixth embodiment are the same as those of the pixel of the first embodiment described above.

[0152] A sample-and-hold circuit 141 is connected downstream of the photoelectric conversion unit 121. At this time, the sample-and-hold circuit 141 samples and holds the photoelectric conversion signal VP when the charges of the photoelectric conversion unit 121 are accumulated. Furthermore, an AD converter 151 AD-converts the photoelectric conversion signal VP sampled and held by the sample-and-hold circuit 141.

[0153] The subtractor 611 is connected to the subsequent stage of the AD converter 151. The subtractor 611 calculates the difference between the AD converted values ​​of the past and present photoelectric conversion signals VP. The subtractor 611 includes a memory 612 and a differentiator 613.

[0154] The memory 612 stores the AD converted value of the photoelectric conversion signal VP of a past frame. The differentiator 613 generates a difference value VD' between the AD converted value of the photoelectric conversion signal VP of the past frame and the AD converted value of the photoelectric conversion signal VP of the current frame. The difference value VD' can be used as luminance information.

[0155] In this way, in the sixth embodiment described above, the difference value VD' of the AD converted value of the photoelectric conversion signal VP is generated, and a gradation signal is generated based on the photoelectric conversion signal VP. This makes it possible to simultaneously obtain a luminance signal and a gradation signal at different frame rates based on the photoelectric conversion signal VP generated by the same pixel HPX.

[0156] 7. Seventh Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this seventh embodiment, event signals EV1 and EV2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP.

[0157] FIG. 19 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the seventh embodiment.

[0158] In the figure, this pixel includes the sample-and-hold circuit 161 and event determination circuit 171 of the second embodiment described above, instead of the sample-and-hold circuit 141 and AD converter 151 of the fifth embodiment described above. The other configurations of the pixel of the seventh embodiment are the same as those of the pixel of the fifth embodiment described above.

[0159] Here, the event signals EV1 and EV2 are generated based on the source follower output of the photoelectric conversion signal VP, which is the cathode potential of the photodiode PD, and the gradation signal is generated based on the source follower output of the potential VF of the floating diffusion FD, which reflects the photoelectric conversion signal VP.

[0160] As described above, in the seventh embodiment, the event signals EV1 and EV2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP. This makes it possible to simultaneously obtain the event signals EV1 and EV2 and the gradation signal based on the photoelectric conversion signal VP generated by the same pixel.

[0161] 8. Eighth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this eighth embodiment, edge signals EG1 and EG2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP.

[0162] FIG. 20 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the eighth embodiment.

[0163] In the figure, this pixel includes the sample-and-hold circuit 181 and edge determination circuit 191 of the third embodiment described above, instead of the sample-and-hold circuit 141 and AD converter 151 of the fifth embodiment described above. The other configurations of the pixel of the eighth embodiment are the same as those of the pixel of the fifth embodiment described above.

[0164] Here, the edge signals EG1 and EG2 are generated based on the source follower output of the photoelectric conversion signal VP, which is the cathode potential of the photodiode PD, and the gradation signal is generated based on the source follower output of the potential VF of the floating diffusion FD, which reflects the photoelectric conversion signal VP.

[0165] In this way, in the above-described eighth embodiment, the edge signals EG1 and EG2 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP. This makes it possible to simultaneously obtain the edge signals EG1 and EG2 and the gradation signal based on the photoelectric conversion signal VP generated by the same pixel.

[0166] 9. Ninth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this ninth embodiment, event signals EV11, EV12, EV21, EV22 and edge signals EG11, EG12, EG21, EG22 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and a gradation signal is generated based on the photoelectric conversion signal VP.

[0167] FIG. 21 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the ninth embodiment.

[0168] In the figure, this pixel includes the sample-and-hold circuit 411 and event / edge determination circuit 431 of the fourth embodiment described above, instead of the sample-and-hold circuit 141 and AD converter 151 of the fifth embodiment described above. Also, this pixel has a switching circuit 421 added to the pixel of the fifth embodiment described above. Other configurations of the pixel of the ninth embodiment are the same as those of the pixel of the fifth embodiment described above.

[0169] Here, the event signals EV11, EV12, EV21, and EV22 and the edge signals EG11, EG12, EG21, and EG22 are generated based on the source follower output of the photoelectric conversion signal VP, which is the cathode potential of the photodiode PD. The gradation signals are generated based on the source follower output of the potential VF of the floating diffusion FD, which reflects the photoelectric conversion signal VP.

[0170] As described above, in the ninth embodiment, the event signals EV11, EV12, EV21, and EV22 and the edge signals EG11, EG12, EG21, and EG22 are generated as difference information based on the difference value VD of the photoelectric conversion signal VP, and gradation signals are generated based on the photoelectric conversion signal VP. This makes it possible to simultaneously obtain the event signals EV11, EV12, EV21, and EV22, the edge signals EG11, EG12, EG21, and EG22, and the gradation signals based on the photoelectric conversion signal VP generated at the same pixel.

[0171] 10. Tenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this tenth embodiment, an event determination based on the difference information based on the difference value VD of the photoelectric conversion signal VP and generation of a gradation signal based on the photoelectric conversion signal VP are switched.

[0172] FIG. 22 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the tenth embodiment.

[0173] In the figure, this pixel is the same as the pixel of the seventh embodiment except that the sample-and-hold circuit 141 and AD converter 151 of the fifth embodiment are added to the pixel of the seventh embodiment. Also, this pixel is the same as the pixel of the seventh embodiment except that a selection switch KW10 is added to the pixel of the seventh embodiment. Other configurations of the pixel of the tenth embodiment are the same as the configuration of the pixel of the seventh embodiment.

[0174] The selection switch KW10 selects the output of the difference information. At this time, the selection switch KW10 switches the output of the subtractor 131 between the sample-and-hold circuits 141 and 161. Here, when the output of the subtractor 131 is switched to the sample-and-hold circuit 141, a luminance signal and a gradation signal having different frame rates can be simultaneously obtained based on the photoelectric conversion signal VP generated by the same pixel. When the output of the subtractor 131 is switched to the sample-and-hold circuit 161, the event signals EV1, EV2 and a gradation signal can simultaneously be obtained based on the photoelectric conversion signal VP generated by the same pixel.

[0175] In this way, the above-described tenth embodiment switches between event determination based on difference information based on the difference value VD of the photoelectric conversion signal VP and generation of a gradation signal based on the photoelectric conversion signal VP. This makes it possible to simultaneously obtain the event signals EV1, EV2 and the gradation signal, or simultaneously obtain a luminance signal and a gradation signal having different frame rates, based on the photoelectric conversion signal VP generated by the same pixel.

[0176] 11. Eleventh Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this eleventh embodiment, it is possible to block the generation of difference information based on the difference value VD of the photoelectric conversion signal VP when a gradation signal is generated based on the photoelectric conversion signal VP.

[0177] FIG. 23 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the eleventh embodiment.

[0178] In the figure, this pixel includes a photoelectric conversion unit 1121 and a differential information processing unit PS1. The photoelectric conversion unit 1121 is the same as the photoelectric conversion unit 121 of the fifth embodiment described above, except that a cutoff switch CW is added. Other configurations of the photoelectric conversion unit 1121 are the same as the configuration of the photoelectric conversion unit 121 of the fifth embodiment described above.

[0179] The cutoff switch CW is connected between the connection point of the photodiode PD and the transfer transistor 522 and the gate of the output transistor 123. A MOS transistor can be used for the cutoff switch CW. When the cutoff switch CW is turned on, it becomes possible to simultaneously acquire the grayscale signal generated based on the photoelectric conversion signal VP and the difference information based on the difference value VD. When the cutoff switch CW is turned off, it becomes possible to acquire the grayscale signal generated based on the photoelectric conversion signal VP without generating the difference information based on the difference value VD.

[0180] The difference information processing unit PS1 generates difference information based on the difference value VD of the photoelectric conversion signal VP. The difference information processing unit PS1 may include the subtractor 131, sample-and-hold circuit 141, and AD converter 151 of the first embodiment described above, or the subtractor 131, sample-and-hold circuit 161, and event determination circuit 171 of the second embodiment described above. Alternatively, the difference information processing unit PS1 may include the subtractor 131, sample-and-hold circuit 181, and edge determination circuit 191 of the third embodiment described above, or the subtractor 131, sample-and-hold circuit 411, switching circuit 421, and event / edge determination circuit 431 of the fourth embodiment described above.

[0181] 24 is a cross-sectional view showing an example of the configuration of a pixel provided in a solid-state imaging device according to Embodiment 11. Note that Fig. 24 shows an excerpt of a part of the photoelectric conversion unit 1121 shown in Fig. 23.

[0182] 24, an n-type impurity diffusion layer K1 is formed on a semiconductor substrate SUB. The material of the semiconductor substrate SUB may be, for example, Si, InGaAs, or InP. On the n-type impurity diffusion layer K1, a p ＋ Impurity diffusion layer K2 and n ＋ Impurity diffusion layers K4 and K5 are formed. ＋ The impurity diffusion layer K2 is joined to the n-type impurity diffusion layer K1 to form a photodiode PD.

[0183] In addition, the semiconductor substrate SUB has n ＋ Adjacent to the impurity diffusion layer K4, ＋An impurity diffusion layer K3 is formed, and n ＋ A floating diffusion FD is formed adjacent to the impurity diffusion layer K5. ＋ The impurity diffusion layer K 4 is connected to the gate of the output transistor 123 , and the floating diffusion FD is connected to the gate of the transfer transistor 522 .

[0184] n ＋ A gate electrode G1 is formed on a channel region between the impurity diffusion layers K3 and K4 via a gate insulating film Z1. ＋ A gate electrode G2 is formed on a channel region between the impurity diffusion layer K5 and the floating diffusion FD via a gate insulating film Z2.

[0185] The gate electrode G1 can be used as a cutoff switch CW. At this time, a cutoff signal VCF can be applied to the gate electrode G1. The gate electrode G2 can be used as a transfer transistor 522. At this time, a transfer signal TGL can be applied to the gate electrode G2.

[0186] Here, by turning off the cutoff switch CW, ＋ The photodiode PD can be separated from the impurity diffusion layer K3, which allows the contacts made in the photodiode PD to be separated, thereby reducing noise when generating a grayscale signal.

[0187] As described above, in the eleventh embodiment, the cutoff switch CW is connected between the connection point of the photodiode PD and the transfer transistor 522 and the gate of the output transistor 123. This makes it possible to reduce noise in the grayscale signal in a pixel that can generate a grayscale signal and difference information based on the photoelectric conversion signal VP.

[0188] 12. Twelfth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectrically converted signal VP. In this twelfth embodiment, event determination based on the difference value VD of the photoelectrically converted signal VP and event determination based on a gradation signal are switched.

[0189] FIG. 25 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the twelfth embodiment.

[0190] In the figure, this pixel includes a photoelectric conversion unit 1121, a difference information processing unit PS2, and changeover switches SW21 and SW22.

[0191] The selector switch SW21 switches between the source follower output of the photoelectric conversion signal VP from the photoelectric conversion unit 1121 to the differential information processing unit PS2 and the source follower output of the floating diffusion FD. At this time, a first input of the selector switch SW21 is connected to the connection point between the output transistor 123 and the bias transistor 124, and a second input of the selector switch SW21 is connected to the source of the selection transistor 525.

[0192] The changeover switch SW22 switches the connection between the source of the selection transistor 525 and the vertical signal line VSLA. At this time, the changeover switch SW22 is connected between the source of the selection transistor 525 and the vertical signal line VSLA.

[0193] The difference information processing unit PS2 generates difference information based on the difference value VD of the photoelectric conversion signal VP, or generates difference information based on a gradation signal. The difference information processing unit PS1 may include the subtractor 131, sample-and-hold circuit 161, and event determination circuit 171 of the second embodiment described above, or the difference information processing unit PS1 may include the subtractor 131, sample-and-hold circuit 181, and edge determination circuit 191 of the third embodiment described above, or the subtractor 131, sample-and-hold circuit 411, switching circuit 421, and event / edge determination circuit 431 of the fourth embodiment described above. In this case, when generating difference information based on a gradation signal, the difference information processing unit PS2 can skip the subtractor 131 and input the gradation signal to the subsequent stage.

[0194] Here, when generating difference information based on the difference value VD, the cutoff switch CW and the changeover switch SW22 can be turned on, and the changeover switch SW21 can be switched to the connection point between the output transistor 123 and the bias transistor 124. When generating difference information based on a gradation signal, the cutoff switch CW and the changeover switch SW22 can be turned off, and the changeover switch SW21 can be switched to the source of the selection transistor 525.

[0195] In this way, in the twelfth embodiment described above, event determination based on the difference value VD of the photoelectric conversion signal VP and event determination based on the gradation signal are switched between. This makes it possible to perform event determination in pixels that can generate gradation signals and difference information based on the photoelectric conversion signal VP without the need to generate difference information.

[0196] 13. Thirteenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on a difference value VD of a photoelectric conversion signal VP. In this thirteenth embodiment, a photoelectric conversion unit 1121 is arranged on an upper chip, and a difference information processing unit and a gradation information processing unit are arranged on a lower chip.

[0197] FIG. 26 is a diagram showing an example of a circuit configuration of a pixel provided in a solid-state imaging device according to the thirteenth embodiment.

[0198] In the figure, the imaging device includes semiconductor chips CP1 and CP2. The semiconductor chip CP1 is stacked on the semiconductor chip CP2. The semiconductor chips CP1 and CP2 may be directly bonded. Hybrid bonding can be used for directly bonding the semiconductor chips CP1 and CP2. In this case, bonding electrodes CU1 and CU2 can be formed on the bonding surface of the semiconductor chip CP1, and bonding electrodes CD1 and CD2 can be formed on the bonding surface of the semiconductor chip CP2. The bonding electrodes CU1 and CU2 can be disposed opposite the bonding electrodes CD1 and CD2, respectively. Cu can be used as the material for the bonding electrodes CU1, CU2, CD1, and CD2.

[0199] The junction electrode CU1 is connected to the connection point between the output transistor 123 and the bias transistor 124. The junction electrode CD1 is connected to the differential information processing unit PS1. The junction electrode CU2 is connected to the vertical signal line VSLA. The junction electrode CD2 is connected to the gradation information processing unit PS3.

[0200] The semiconductor chip CP1 is formed with a photoelectric conversion unit 1121. The semiconductor chip CP1 may have a plurality of photoelectric conversion units 1121 arranged in an array. In this case, the plurality of photoelectric conversion units 1121 may be arranged in an array over the entire surface of the semiconductor chip CP1.

[0201] The semiconductor chip CP2 is formed with a difference information processing section PS1 and a gradation information processing section PS3. The gradation information processing section PS3 processes gradation signals transmitted via the vertical signal line VSLA. The gradation information processing section PS3 may include a column read circuit 113, a column signal processing section 114, and a horizontal scanning circuit 115.

[0202] 27 is a cross-sectional view showing an example of the configuration of a pixel provided in a solid-state imaging device according to the thirteenth embodiment. Note that Fig. 27 shows an excerpt of a part of the photoelectric conversion unit 1121 shown in Fig. 26.

[0203] 24, gate electrodes G3 to G7 are added to the semiconductor substrate SUB. A channel region is formed under each of the gate electrodes G3 to G7. The channel region under each of the gate electrodes G3 to G7 has a width of n ＋ Separation occurs at the impurity diffusion layer.

[0204] The gate electrode G3 can be used for the output transistor 123. The gate electrode G4 can be used for the bias transistor 124. At this time, a bias voltage Vb can be applied to the gate electrode G4. The gate electrode G5 can be used for the reset transistor 523. At this time, a reset signal RST can be applied to the gate electrode G5. The gate electrode G6 can be used for the amplification transistor 524. The gate electrode G7 can be used for the selection transistor 525. At this time, a selection signal SEL can be applied to the gate electrode G7.

[0205] In this way, in the above-described thirteenth embodiment, the photoelectric conversion unit 1121 is arranged on the semiconductor chip CP1, and the difference information processing unit PS1 and the gradation information processing unit PS3 are arranged on the semiconductor chip CP2, thereby making it possible to improve the sensitivity of the photodiode PD while suppressing an increase in the mounting area of ​​the semiconductor chips CP1 and CP2.

[0206] 14. Fourteenth Embodiment In the first embodiment described above, a luminance signal is generated as differential information based on a differential value VD of a photoelectric conversion signal VP. In this fourteenth embodiment, a floating diffusion FD is shared by a plurality of pixels provided with photoelectric conversion units that generate differential information.

[0207] Fig. 28 is a diagram showing an example of the circuit configuration of pixels provided in a solid-state imaging device according to the fourteenth embodiment. Note that Fig. 28 shows an example of the configuration of an imaging device for two columns.

[0208] In the figure, this imaging device includes semiconductor chips CP3 and CP4. The semiconductor chip CP3 is stacked on the semiconductor chip CP4. The semiconductor chips CP3 and CP4 may be directly bonded. In this case, bonding electrodes U1 to U5 may be formed on the bonding surface of the semiconductor chip CP3, and bonding electrodes D1 to D5 may be formed on the bonding surface of the semiconductor chip CP4. The bonding electrodes U1 to U5 may be disposed opposite the bonding electrodes D1 to D5, respectively. Cu may be used as the material for the bonding electrodes U1 to U5 and D1 to D5.

[0209] Each of the junction electrodes U1 to D4 is connected to a connection point between the output transistor 123 and the bias transistor 124 of each of the pixels HPX11 to HPX14. Each of the junction electrodes D1 to D4 is connected to the differential information processing unit PS1 of each of the pixels HPX11 to HPX14. The junction electrode U5 is connected to the vertical signal line VSLA. The junction electrode D5 is connected to the gradation information processing unit PS3.

[0210] Each of the pixels HPX11 to HPX14 includes a photodiode PD, a transfer transistor 522, a cutoff switch CW, an output transistor 123, a bias transistor 124, a difference information processing section PS1, output transistors TA and TC, and selection transistors TB and TD. At this time, the reset transistor 523, the amplification transistor 524, the selection transistor 525, and the floating diffusion FD are shared by each of the pixels HPX11 to HPX14.

[0211] The semiconductor chip CP3 has a photodiode PD, a transfer transistor 522, a cutoff switch CW, an output transistor 123, and a bias transistor 124 formed for each of the pixels HPX11 to HPX14. The semiconductor chip CP3 also has a reset transistor 523, an amplifier transistor 524, a selection transistor 525, and a floating diffusion FD formed in common for each of the pixels HPX11 to HPX14. At this time, the floating diffusion FD and the vertical signal line VSLA are shared by each of the pixels HPX11 to HPX14.

[0212] The semiconductor chip CP4 has a differential information processing section PS1, output transistors TA and TC, and selection transistors TB and TD formed for each of the pixels HPX11 to HPX14. The differential information processing section PS1 may have a positive output and a negative output. The semiconductor chip CP4 also has a gradation information processing section PS3 formed in common for each of the pixels HPX11 to HPX14. Vertical signal lines VSLB1, VSLB1', VSLB2, and VSLB2' are provided for each column as the vertical signal lines VSLB.

[0213] The output transistor TA and the selection transistor TB are connected in series. The output transistor TC and the selection transistor TD are connected in series. The positive output of the differential information processing unit PS1 is connected to the gate of the output transistor TA. The negative output of the differential information processing unit PS1 is connected to the gate of the output transistor TC. Selection signals X1 to X4 are applied to the gate of the selection transistor TB of each of the pixels HPX11 to HPX14. Selection signals Y1 to Y4 of each of the pixels HPX11 to HPX14 are applied to the gate of the selection transistor TD.

[0214] The vertical signal line VSLB1 is connected to the selection transistors TB of the pixels HPX11 and HPX12, and the vertical signal line VSLB1' is connected to the selection transistors TD of the pixels HPX11 and HPX12. The vertical signal line VSLB2 is connected to the selection transistors TB of the pixels HPX13 and HPX14, and the vertical signal line VSLB2' is connected to the selection transistors TD of the pixels HPX13 and HPX14.

[0215] In this way, in the above-described fourteenth embodiment, the floating diffusion FD is shared by a plurality of pixels HPX11 to HPX14, each of which is provided with a photoelectric conversion unit that generates differential information. This makes it possible to simultaneously generate a gradation signal and differential information for each of the pixels HPX11 to HPX14, while reducing the circuit scale of each of the pixels HPX11 to HPX14.

[0216] 15. Fifteenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this fifteenth embodiment, the event / edge determination circuit 431 is shared by multiple pixels.

[0217] Fig. 29 is a diagram showing an example of the circuit configuration of pixels provided in a solid-state imaging device according to the fifteenth embodiment. Note that Fig. 29 shows an example of the configuration of an imaging device for two columns.

[0218] In the figure, this imaging device includes semiconductor chips CP5 and CP6. The semiconductor chip CP6 is stacked on the semiconductor chip CP5. The semiconductor chips CP5 and CP6 may be directly bonded. In this case, bonding electrodes U11 to U14 may be formed on the bonding surface of the semiconductor chip CP6, and bonding electrodes D11 to D14 may be formed on the bonding surface of the semiconductor chip CP5. The bonding electrodes U11 to U14 may be disposed opposite the bonding electrodes D11 to D14, respectively. Cu may be used as the material for the bonding electrodes U11 to U14 and D11 to D14.

[0219] Each of the junction electrodes U11 to U14 is connected to a connection point between the output transistor 123 and the bias transistor 124 of each of the pixels HPX21 to HPX24. Each of the junction electrodes D11 to D14 is connected to a subtractor 131 of each of the pixels HPX21 to HPX24.

[0220] Each of the pixels HPX21 to HPX24 includes a photodiode PD, a transfer transistor 522, a reset transistor 523, an amplification transistor 524, a selection transistor 525, and a floating diffusion FD. Each of the pixels HPX21 to HPX24 also includes a cutoff switch CW, an output transistor 123, a bias transistor 124, a subtractor 131, a sample-and-hold circuit 411, an output transistor TG, and a selection transistor TH.

[0221] The semiconductor chip CP5 has a subtractor 131, a sample-and-hold circuit 411, an output transistor TG, and a selection transistor TH formed for each of the pixels HPX21 to HPX24. The semiconductor chip CP5 also has a switching circuit 421, an event / edge determination circuit 431, and a bias transistor 535 formed in common for each of the pixels HPX21 to HPX24. At this time, the vertical signal line VSLB is shared by each of the pixels HPX21 to HPX24.

[0222] The output transistor TG and the selection transistor TH are connected in series. The output of the sample-and-hold circuit 411 is connected to the gate of the output transistor TG. Selection signals B1 to B4 are applied to the gates of the selection transistors TH of each of the pixels HPX21 to HPX24. The vertical signal line VSLB is connected to the selection transistors TH of each of the pixels HPX21 to HPX24.

[0223] The switching circuit 421 is connected to the vertical signal line VSLB, and the event edge determination circuit 431 is connected to the subsequent stage of the switching circuit 421. The bias transistor 535 is connected in series to the vertical signal line VSLB. A bias voltage Vb2 is applied to the gate of the bias transistor 535.

[0224] The semiconductor chip CP6 has a photodiode PD, a transfer transistor 522, a cutoff switch CW, an output transistor 123, and a bias transistor 124 formed for each of the pixels HPX21 to HPX24. The semiconductor chip CP6 also has a reset transistor 523, an amplifier transistor 524, a selection transistor 525, and a floating diffusion FD formed for each of the pixels HPX21 to HPX24. At this time, vertical signal lines VSLA1 and VSLA2 are provided for each column as the vertical signal line VSLA.

[0225] In this way, in the above-described fifteenth embodiment, the event / edge determination circuit 431 is shared by multiple pixels. This makes it possible to simultaneously generate gradation signals and difference information for each of the pixels HPX21 to HPX24 while reducing the circuit scale of each of the pixels HPX21 to HPX24.

[0226] In the above-described fifteenth embodiment, an example has been shown in which the event / edge determination circuit 431 of the above-described fourth embodiment is shared by each of the pixels HPX21 to HPX24. In addition to this, the AD converter 151 of the above-described first embodiment may be shared by a plurality of pixels, the event determination circuit 171 of the above-described second embodiment may be shared by a plurality of pixels, or the edge determination circuit 191 of the above-described third embodiment may be shared by a plurality of pixels.

[0227] 16. Sixteenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this sixteenth embodiment, an event edge determination circuit 431 is provided for each column.

[0228] 30 is a diagram showing an example of the circuit configuration of pixels provided in a solid-state imaging device according to the sixteenth embodiment. Note that Fig. 30 shows an example of the configuration of an imaging device for two columns.

[0229] In the figure, the imaging device includes pixels HPX31 and HPX32. Each of the pixels HPX31 and HPX32 includes a photodiode PD, a transfer transistor 522, a reset transistor 523, an amplification transistor 524, a selection transistor 525, and a floating diffusion FD. Each of the pixels HPX31 and HPX32 also includes an output transistor 123, a bias transistor 124, a subtractor 131, a sample-and-hold circuit 411, an output transistor TG, and a selection transistor TH.

[0230] Furthermore, in this imaging device, an event / edge determination circuit 431 and a gradation information processing section PS3 are provided for each column. Vertical signal lines VSLA1, VSLB1, VSLA2, and VSLB2 are provided for each column as vertical signal lines VSLA and VSLB. The vertical signal lines VSLA1 and VSLA2 are connected to the selection transistors 525 of the pixels HPX31 and HPX32. The vertical signal lines VSLB1 and VSLB2 are connected to the selection transistors TH of the pixels HPX31 and HPX32.

[0231] The gradation information processing section PS3 is connected to the vertical signal lines VSLA1 and VSLA2 for each column. The event edge determination circuit 431 is connected to the vertical signal lines VSLB1 and VSLB2 for each column. The bias transistor 535 is connected in series to the vertical signal lines VSLB1 and VSLB2 for each column. The bias transistor 536 is connected in series to the vertical signal lines VSLA1 and VSLA2 for each column. A bias voltage Vb3 is applied to the gate of the bias transistor 536.

[0232] In this way, in the above-described sixteenth embodiment, an event edge determination circuit 431 is provided for each column, which makes it possible to simultaneously generate a grayscale signal and difference information for each pixel HPX31, HPX32 while reducing the circuit scale of each pixel HPX31, HPX32.

[0233] In the sixteenth embodiment, an example has been shown in which the event edge determination circuit 431 of the fourth embodiment is provided for each column. Alternatively, the AD converter 151 of the first embodiment may be provided for each column, the event determination circuit 171 of the second embodiment may be provided for each column, or the edge determination circuit 191 of the third embodiment may be provided for each column.

[0234] 17. Seventeenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectrically converted signal VP. In this seventeenth embodiment, a comparator is shared by the difference information processing section and the gradation information processing section.

[0235] 31 is a diagram showing an example of the circuit configuration of a pixel provided in a solid-state imaging device according to Embodiment 17. Note that Fig. 31 shows an example of the configuration of an imaging device for one column.

[0236] In the figure, this imaging device includes a gradation processing / event / edge determination circuit 601 instead of the event / edge determination circuit 431 and gradation information processing section PS3 of the sixteenth embodiment. Other configurations of the imaging device of the seventeenth embodiment are the same as those of the imaging device of the sixteenth embodiment.

[0237] A gradation processing / event / edge determination circuit 601 is provided for each column. The gradation processing / event / edge determination circuit 601 serves as both the event / edge determination circuit 431 and the gradation information processing unit PS3 in the sixteenth embodiment. The gradation processing / event / edge determination circuit 601 is connected to the vertical signal lines VSLA1 and VSLB1 for each column.

[0238] FIG. 32 is a diagram showing an example of a circuit configuration of a comparator provided in a solid-state imaging device according to the seventeenth embodiment.

[0239] In the figure, the gradation processing / event / edge determination circuit 601 includes changeover switches SK61, SK62, SK71 to SK78, capacitors C71 to C78, ​​and a comparator 701.

[0240] The selector switch SK61 switches between the difference value SB1 and the gradation signal KA used for the event edge determination. The selector switch SK62 switches between the difference value SB2 and the reference signal VRE used for the event edge determination. At this time, the difference values ​​SB1 and SB2 are input from the pixel HPX31 to the gradation processing / event edge determination circuit 601 via the vertical signal line VSLB1. The gradation signal KA is input from the pixel HPX31 to the gradation processing / event edge determination circuit 601 via the vertical signal line VSLA1. Then, for the event edge determination, the selector switch SK61 is switched to input the difference value SB1, and the selector switch SK62 is switched to input the difference value SB2. For the gradation processing, the selector switch SK61 is switched to input the gradation signal KA, and the selector switch SK62 is switched to input the reference signal VRE.

[0241] The changeover switches SK71 to SK78 adjust the attenuation of the signal input to the comparator 701. Here, the changeover switches SK71 to SK74 adjust the number of parallel connections of the capacitors C71 to C74 connected to the inverting input of the comparator 701. The changeover switches SK75 to SK78 adjust the number of parallel connections of the capacitors C75 to C78 connected to the non-inverting input of the comparator 701. At this time, the changeover switches SK71 to SK78 can be switched so that the signal input to the comparator 701 falls within the operating range of the comparator 701.

[0242] The comparator 701 compares the difference values ​​SB1 and SB2 and compares the grayscale signal KA with the reference signal VRE in response to the switching of the changeover switches SK61 and SK62.

[0243] In this way, in the above-described seventeenth embodiment, the comparator 701 is used for both the gradation processing and event / edge determination of the gradation processing / event / edge determination circuit 601. This makes it possible to reduce the number of comparators 701 used for the gradation processing and event / edge determination, while realizing generation of a gradation signal and difference information using one pixel HPX31.

[0244] 18. Eighteenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on the difference value VD of the photoelectric conversion signal VP. In this eighteenth embodiment, grayscale pixels and hybrid pixels are mounted together.

[0245] FIG. 33 is a diagram showing an example of a pixel array provided in a solid-state imaging device according to the eighteenth embodiment.

[0246] In the figure, the imaging device includes a photoelectric conversion unit 1121 and a pixel 1122. The photoelectric conversion unit 112 can form a pixel HPX. In this case, the photoelectric conversion unit 112 is used as a hybrid pixel capable of generating a grayscale signal and difference information based on a photoelectric conversion signal VP. The pixel 1122 is used as a grayscale pixel capable of generating a grayscale signal.

[0247] The pixel 1122 includes a photodiode PD′, a transfer transistor 522, a reset transistor 523, an amplification transistor 524, a selection transistor 525, and a floating diffusion FD. The photodiode PD′ is connected to the floating diffusion FD via the transfer transistor 522.

[0248] The pixels 1122 may be arranged in a quad Bayer array. The quad Bayer array includes red pixels PXR, blue pixels PXB, and green pixels PXGb and PXGr. In this case, some of the pixels 1122 in the quad Bayer array may be replaced with photoelectric conversion units 1121. For example, two of the four blue pixels PXB in the quad Bayer array can be replaced with photoelectric conversion units 1121. This makes it possible to combine grayscale pixels and hybrid pixels while suppressing a decrease in resolution of a colorized grayscale image.

[0249] In this way, in the above-described eighteenth embodiment, grayscale pixels and hybrid pixels are combined, which makes it possible to obtain difference information while increasing the resolution of grayscale images.

[0250] 19. Nineteenth Embodiment In the first embodiment described above, a luminance signal is generated as difference information based on a difference value VD of a photoelectric conversion signal VP. In this nineteenth embodiment, semiconductor chips each having a pixel array portion in which pixels are arranged in a matrix are stacked.

[0251] FIG. 34 is a perspective view showing an example of lamination of a pixel array unit according to the nineteenth embodiment.

[0252] In the figure, the imaging device includes semiconductor chips 921 and 922. The semiconductor chip 922 is stacked on the semiconductor chip 921.

[0253] A pixel array section 923 is formed in the semiconductor chip 922. In the pixel array section 923, pixels 931 are arranged in a matrix in the row and column directions. The pixels 931 may be any of the pixels in the fifth to thirteenth embodiments described above, or may include pixels in the eighteenth embodiment. Pad electrodes 932 and via electrodes 933 are formed around the pixel array section 923. The via electrodes 933 penetrate the semiconductor chip 922 and can electrically connect the semiconductor chips 921 and 922 to each other.

[0254] A semiconductor chip 921 is formed with a difference information processing unit 924, a column readout circuit 925, a column ADC 926, a communication interface 927, a control circuit 928, and a difference information output circuit 929. The difference information processing unit 924 may be the difference information processing unit PS1 of the eleventh embodiment or the difference information processing unit PS2 of the twelfth embodiment.

[0255] The semiconductor chips 921 and 922 may be directly bonded to each other. Hybrid bonding can be used for directly bonding the semiconductor chips 921 and 922. In this case, the semiconductor chips 921 and 922 may be electrically connected based on Cu-Cu bonding. The material of the semiconductor substrate used for the semiconductor chips 921 and 922 may be Si, InGaAs, or InP.

[0256] As described above, in the nineteenth embodiment, the semiconductor chip 922 on which the pixel array unit 923 is formed is stacked on the semiconductor chip 921 on which the difference information processing unit 924 is formed. This makes it possible to increase the sensitivity of the solid-state imaging device while suppressing an increase in the mounting area of ​​the semiconductor chip on which the solid-state imaging device is formed.

[0257] 20. Application Examples to Mobile Bodies The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0258] FIG. 35 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0259] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 35, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (interface) 12053.

[0260] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0261] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0262] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0263] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0264] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0265] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0266] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0267] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0268] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 35, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0269] FIG. 36 is a diagram showing an example of the installation position of the imaging unit 12031.

[0270] In FIG. 36, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0271] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0272] 36 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0273] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0274] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which runs autonomously without relying on driver operation.

[0275] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0276] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0277] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 of the above-described configuration. Specifically, for example, the imaging device according to the above-described embodiment can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the vehicle control system 12000, the imaging unit 12031 can be made more compact while providing more functionality.

[0278] Note that the above-described embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specifying the invention in the claims correspond to each other. Similarly, the matters specifying the invention in the claims and the matters in the embodiment of the present technology with the same title correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist. Furthermore, the effects described in this specification are merely examples and are not limited, and other effects may also be present.

[0279] The present technology may also be configured as follows: (1) An imaging device including a photoelectric conversion unit that outputs a photoelectric conversion signal corresponding to charge accumulated by photoelectric conversion, and a difference information generation unit that generates difference information based on a difference value of the photoelectric conversion signal when charge of the photoelectric conversion unit is accumulated. (2) The imaging device according to (1), in which the photoelectric conversion unit is not reset between outputs of photoelectric conversion signals used to generate the difference value. (3) The imaging device according to (1) or (2), in which the difference information generation unit includes a subtractor that generates a difference value of the photoelectric conversion signal. (4) The imaging device according to (3), in which a sample-and-hold circuit that samples and holds the difference value of the photoelectric conversion signal. (5) The imaging device according to (4), in which an AD (Analog to Digital) converter that AD-converts the difference value of the photoelectric conversion signal sampled and held by the sample-and-hold circuit. (6) The imaging device according to (4) or (5), in which an event determination circuit that determines an event based on the difference value of the photoelectric conversion signal sampled and held by the sample-and-hold circuit. (7) The imaging device according to any one of (4) to (6), further comprising an edge determination circuit that determines an edge based on a difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit. (8) The imaging device according to any one of (1) to (7), further comprising a selection switch that selects an output of the difference information. (9) The imaging device according to any one of (1) to (8), further comprising: a photodiode; and a source follower circuit having a gate connected to the cathode of the photodiode. (10) The imaging device according to (9), further comprising: a gradation information generation unit that generates a gradation signal according to charge accumulated in the photodiode. (11) The imaging device according to (10), further comprising: a frame rate of the difference information and a frame rate of the gradation signal that are different from each other. (12) The imaging device according to (11), further comprising: an image processing unit that generates an HDR (High Dynamic Range) image based on luminance information generated based on the difference value and the gradation signal.(13) The imaging device according to any one of (10) to (12), further comprising a cutoff switch connected between the cathode of the photodiode and the gate of the source follower circuit. (14) The imaging device according to any one of (10) to (13), further comprising a changeover switch that switches event determination or edge determination based on the difference value to event determination or edge determination based on the gradation signal. (15) The imaging device according to any one of (10) to (14), further comprising a floating diffusion shared by a plurality of gradation information generators. (16) The imaging device according to any one of (10) to (15), further comprising a comparator shared for comparing the gradation signal with a reference signal and for comparing the difference values. (17) The imaging device according to any one of (1) to (16), further comprising a pixel array unit in which pixels provided with the photoelectric conversion unit are arranged in a matrix in row and column directions, and the difference information generator is provided for each column. (18) The imaging device according to any one of (1) to (17), comprising: a first chip on which the photoelectric conversion unit is formed; and a second chip on which the first chip is stacked and on which the difference information generation unit is formed. (19) The imaging device according to any one of (1) to (19), comprising: a gradation pixel on which a gradation information generation unit for generating a gradation signal is formed; and a hybrid pixel on which the gradation information generation unit and the difference information generation unit are formed. (20) An imaging method that outputs a photoelectric conversion signal according to charge accumulated by photoelectric conversion, and generates difference information based on a difference value of the photoelectric conversion signal when the charge is accumulated.

[0280] 100 Imaging device 101 Optical system 102 Solid-state imaging device 103 Imaging control unit 104 Image processing unit 105 Memory unit 106 Display unit 107 Operation unit 108 Bus 111 Pixel array unit 112 Vertical scanning circuit 116 Control circuit 117 Difference information output circuit PX Pixel HSL Horizontal drive line VSL Vertical signal line 121 Photoelectric conversion unit PD Photodiode 122 Reset transistor 123 Output transistor 124 Bias transistor 131 Subtractor C1, C2 Capacitor 132 Inverter 133 Buffer 141 Sample and hold circuit SW1 Sample and hold switch C3 Sample and hold capacitor 151 AD converter 152 Comparator

Claims

1. An imaging device comprising: a photoelectric conversion unit that outputs a photoelectric conversion signal corresponding to the charge accumulated by photoelectric conversion; and a differential information generation unit that generates differential information based on the differential value of the photoelectric conversion signal when the charge of the photoelectric conversion unit is accumulated.

2. The imaging device according to claim 1, wherein the photoelectric conversion section is not reset between outputs of the photoelectric conversion signals used to generate the difference values.

3. The imaging device according to claim 1, wherein the difference information generating section includes a subtractor that generates a difference value of the photoelectric conversion signal.

4. The imaging device according to claim 3, further comprising a sample-and-hold circuit that samples and holds the difference value of the photoelectric conversion signal.

5. The imaging device according to claim 4, further comprising an AD (Analog to Digital) converter that AD converts the difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit.

6. The imaging device according to claim 4, further comprising an event determination circuit that determines an event based on a difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit.

7. The imaging device according to claim 4, further comprising an edge determination circuit that determines an edge based on the difference value of the photoelectric conversion signal sampled and held by the sample and hold circuit.

8. The imaging device according to claim 1, further comprising a selection switch for selecting an output of the difference information.

9. The imaging device according to claim 1, wherein the photoelectric conversion section comprises a photodiode and a source follower circuit in which the cathode of the photodiode is connected to the gate.

10. The imaging device according to claim 9, wherein the photoelectric conversion section includes a gradation information generation section that generates a gradation signal according to the charge accumulated in the photodiode.

11. The imaging device according to claim 10, wherein the frame rate of the difference information and the frame rate of the gradation signal are different from each other.

12. The imaging device according to claim 11, further comprising an image processing unit that generates an HDR (High Dynamic Range) image based on the luminance information generated based on the difference value and the gradation signal.

13. The imaging device according to claim 10, further comprising an isolation switch connected between the cathode of the photodiode and the gate of the source follower circuit.

14. The imaging device according to claim 10, further comprising a changeover switch for switching from event determination or edge determination based on the difference value to event determination or edge determination based on the gradation signal.

15. The imaging device according to claim 10, wherein the photoelectric conversion section includes a floating diffusion shared by a plurality of gradation information generation sections.

16. The imaging device according to claim 10, further comprising a comparator shared between the comparison of the gradation signal and the reference signal and the comparison of the difference values.

17. The imaging device according to claim 1, further comprising a pixel array section in which pixels provided with said photoelectric conversion section are arranged in a matrix in the row and column directions, and said difference information generation section is provided for each column.

18. The imaging device according to claim 1, comprising: a first chip on which the photoelectric conversion unit is formed; and a second chip on which the first chip is stacked and on which the difference information generation unit is formed.

19. An imaging device according to claim 1, comprising: a gradation pixel in which a gradation information generating section that generates a gradation signal is formed; and a hybrid pixel in which the gradation information generating section and the difference information generating section are formed.

20. An imaging method that outputs a photoelectric conversion signal corresponding to electric charges accumulated by photoelectric conversion, and generates difference information based on the difference value of the photoelectric conversion signal when the electric charges are accumulated.

Images