Solid-state imaging element, imaging apparatus, and method of controlling solid-state imaging element

Abstract

The present invention improves the accuracy of a solid-state imaging element that performs image recognition by convolution integration. A plurality of pixels, each of which generates and holds an analog pixel signal, are arranged in a two-dimensional grid within a pixel array section. The plurality of integration circuits time-integrate pixel signals held in a predetermined number of vertically arranged pixels among the plurality of pixels, and output integrated signals. The analog-to-digital converter adds the analog integration signals of the plurality of respective integration circuits and converts the analog integration signals into digital signals.

Description

Technical Field

[0001] The present technology relates to a solid-state imaging element. Specifically, the present technology relates to a solid-state imaging element that performs convolution integration, an imaging device, and a method of controlling the solid-state imaging element. Background Art

[0002] In recent years, with the development of artificial intelligence (AI) technology, the accuracy of image recognition has been significantly improved, and the demand for image recognition processing in applications such as factory automation (FA) and monitoring has increased. In particular, for use in edge AI, attention has been paid to providing a circuit that performs convolution integration in a solid-state imaging element to achieve an increase in processing speed and a reduction in power costs. For example, a solid-state imaging element has been proposed, in which an arithmetic circuit including a pair of capacitor elements for each column and a pixel that applies any one of filter coefficients +1, 0, and -1 is provided (for example, see non-patent document 1). The charge of the pixel with a filter coefficient of +1 is accumulated in one of the capacitor element pairs, and the charge of the pixel with a filter coefficient of -1 is accumulated in the other of the capacitor element pair, and the operation circuit performs analog addition of the difference between the currents of these capacitor elements column by column. This analog circuit implements convolution integration.

[0003] Reference List

[0004] Non-patent literature

[0005] Non-patent document 1: Martin Lefebvre, et al., A0.2-to-3.6TOPS / WProgrammableConvolutional Imager SoC with In-Sensor Current-Domain Ternary-Weighted MACOperations for Feature Extraction and Region-of-Interest Detection, 2021ISSCC. Summary of the invention

[0006] Problems to be solved by the present invention

[0007] In the above conventional technology, convolution integration is performed by an analog circuit to increase the processing speed. However, in the above solid-state imaging element, since the filter coefficients are limited to +1, 0, and -1, the accuracy of image recognition may be insufficient. In addition, since the operation circuit performs analog addition of differences column by column, the processing speed becomes slow, and there is a possibility that the accuracy is insufficient when imaging a moving object.

[0008] The present technology has been made in view of such circumstances, and its object is to improve the accuracy of a solid-state imaging element that performs image recognition by convolution integration.

[0009] Solution to the problem

[0010] The present technology has been made to solve the above-mentioned problems, and its first aspect is a solid-state imaging element, including: a pixel array section, in which a plurality of pixels each generating and holding an analog pixel signal are arranged in a two-dimensional grid pattern; a plurality of integration circuits that time-integrate the pixel signal held in each of a predetermined number of pixels arranged in the vertical direction among the plurality of pixels and output the integration signal; and an analog-to-digital converter and a control method thereof, the analog-to-digital converter performing analog addition of the integration signal of each of the plurality of integration circuits to convert the analog-added integration signal into a digital signal. This brings about an effect of improving accuracy when performing image recognition processing.

[0011] Furthermore, in the first aspect, the integration time for each pixel of the integration circuit may include a time according to the absolute value of the filter coefficient corresponding to the pixel. This brings about the effect of performing a product-sum operation of the pixel signal and the filter coefficient.

[0012] Furthermore, in the first aspect, the pixel signal may include a predetermined reset level and a signal level according to the exposure amount, and the pixel may output the reset level and the signal level in a predetermined order when the sign of the filter coefficient is positive, and output the reset level and the signal level in an order opposite to the predetermined order when the sign of the filter coefficient is negative. This brings about the effect that the sign of the filter coefficient is reflected in the sign of the difference between the reset level and the signal level.

[0013] In addition, in the first aspect, the plurality of integration circuits may include a first integration circuit and a second integration circuit, and the analog-to-digital converter may include: a first capacitance element including one end connected to the first integration circuit; a second capacitance element including one end connected to the second integration circuit; a comparator including two input terminals, one of the two input terminals being connected to the other end of each of the first capacitance element and the second capacitance element; and a counter counting the count value for a period of time until the output signal of the comparator is inverted. This brings about the effect of converting a signal obtained by analog addition of the integration signals into a digital signal.

[0014] Furthermore, in the first aspect, the analog-to-digital converter may further include a connection switch that connects the other end of the first capacitance element and the other end of the second capacitance element according to a predetermined switch signal. This brings about an effect that the size of the filter in the horizontal direction is variable.

[0015] Furthermore, in the first aspect, each pixel may include a photodiode in which a charge storage region is embedded in a predetermined semiconductor substrate. This brings about an effect of improving image quality in a dark place.

[0016] Furthermore, in the first aspect, the pixel signal may include a predetermined reset level and a signal level according to the exposure amount, and each pixel may include: a first capacitance element and a second capacitance element; a front-stage circuit that sequentially generates the reset level and the signal level and causes the first capacitance element and the second capacitance element to hold the reset level and the signal level, respectively; a first rear-stage circuit that reads and outputs the reset level held in the first capacitance element; and a second rear-stage circuit that reads and outputs the signal level held in the second capacitance element. This brings about an effect of increasing the reading speed.

[0017] In addition, in the first aspect, the pixel signal may include a predetermined reset level and a signal level according to the exposure amount, and each pixel may include: a first capacitor element and a second capacitor element; a front-stage circuit that sequentially generates the reset level and the signal level and causes the first capacitor element and the second capacitor element to respectively maintain the reset level and the signal level; a selection circuit that sequentially performs control to connect one of the first capacitor element and the second capacitor element to a predetermined rear-stage node, controls to disconnect both the first capacitor element and the second capacitor element from the rear-stage node, and controls to connect the other of the first capacitor element and the second capacitor element to the rear-stage node; a rear-stage reset transistor that initializes the level of the rear-stage node when both the first capacitor element and the second capacitor element are disconnected from the rear-stage node; and a rear-stage circuit that sequentially reads the reset level and the signal level from the first and second capacitor elements via the rear-stage node, and outputs the reset level and the signal level. This brings about an effect of reducing kTC noise.

[0018] In addition, a second aspect of the present technology is an imaging device including: a pixel array section in which a plurality of pixels each generating and holding an analog pixel signal are arranged in a two-dimensional grid pattern; a plurality of integration circuits that time-integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among the plurality of pixels and output an integration signal; an analog-to-digital converter that adds the integration signal of each of the plurality of integration circuits to convert the added integration signal into a digital signal; and an image recognition section that performs a predetermined image recognition process using the digital signal. This brings about an effect of improving the accuracy of the image recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1It is a block diagram showing a configuration example of an imaging device in a first embodiment of the present technology.

[0020] Figure 2 : is a block diagram showing a configuration example of a solid-state imaging element according to a first embodiment of the present technology.

[0021] Figure 3 is a circuit diagram showing a configuration example of a pixel according to a first embodiment of the present technology.

[0022] Figure 4 : is a circuit diagram showing a configuration example of a pixel of a stacked structure according to the first embodiment of the present technology.

[0023] Figure 5 : is a block diagram showing a configuration example of a load MOS circuit block and a column signal processing circuit according to the first embodiment of the present technology.

[0024] Figure 6 : is a circuit diagram showing a configuration example of an integration circuit and an ADC according to the first embodiment of the present technology.

[0025] Figure 7 is an example of an overall view of the solid-state imaging element according to the first embodiment of the present technology.

[0026] Figure 8 : is a block diagram showing a configuration example of a solid-state imaging element and a DSP circuit according to a first embodiment of the present technology.

[0027] Fig. 9 is an example of an overall view of a solid-state imaging element in a comparative example.

[0028] Fig.10 : is a timing chart showing an example of the global shutter operation according to the first embodiment of the present technology.

[0029] Fig.11 : is a timing chart showing an example of the first auto-zero operation and the first integration operation according to the first embodiment of the present technology.

[0030] Fig.12 : is a diagram showing an example of a state of the solid-state imaging element within the first automatic zero adjustment period according to the first embodiment of the present technology.

[0031] Fig.13 : is a diagram illustrating an example of a state of the solid-state imaging element within the first integration period according to the first embodiment of the present technology.

[0032] Fig.14 : is a timing chart showing an example of the second integration operation according to the first embodiment of the present technology.

[0033] Fig.15 : is a diagram illustrating an example of a state of the solid-state imaging element within the second integration period according to the first embodiment of the present technology.

[0034] Fig.16 : is a timing chart showing an example of the third integration operation according to the first embodiment of the present technology.

[0035] Fig.17 : is a diagram illustrating an example of a state of the solid-state imaging element within the third integration period according to the first embodiment of the present technology.

[0036] Fig.18 is a timing chart showing an example of the first read operation according to the first embodiment of the present technology.

[0037] Fig.19 : is a diagram illustrating an example of a state of the solid-state imaging element in the first reading period according to the first embodiment of the present technology.

[0038] Fig. 20 : is a timing chart showing an example of the second auto-zero operation and the fourth integration operation according to the first embodiment of the present technology.

[0039] Fig.21 : is a diagram showing an example of a state of the solid-state imaging element within the second automatic zero adjustment period according to the first embodiment of the present technology.

[0040] Fig. 22 : is a diagram illustrating an example of a state of the solid-state imaging element within the fourth integration period according to the first embodiment of the present technology.

[0041] Fig.23 : is a timing chart showing an example of the fifth integration operation according to the first embodiment of the present technology.

[0042] Fig.24 : is a diagram illustrating an example of a state of the solid-state imaging element within the fifth integration period according to the first embodiment of the present technology.

[0043] Fig.25 : is a timing chart showing an example of the sixth integration operation according to the first embodiment of the present technology.

[0044] Fig.26 : is a diagram illustrating an example of a state of the solid-state imaging element within the sixth integration period according to the first embodiment of the present technology.

[0045] Fig. 27 : is a timing chart showing an example of the second read operation according to the first embodiment of the present technology.

[0046] Fig.28 : is a diagram illustrating an example of a state of the solid-state imaging element in the second reading period according to the first embodiment of the present technology.

[0047] Fig.29 : is a diagram showing an example of a state of a solid-state imaging element when the filter according to the first embodiment of the present technology slides in the horizontal direction.

[0048] Fig.30 is a flowchart showing an example of the operation of the solid-state imaging element according to the first embodiment of the present technology.

[0049] Fig.31 : is a diagram showing a usage example of the solid-state imaging element according to the first embodiment of the present technology.

[0050] Fig.32 : is a circuit diagram showing a configuration example of a column signal processing circuit in a modification example of the first embodiment of the present technology.

[0051] Fig.33 is a circuit diagram showing a configuration example of an ADC according to a second embodiment of the present technology.

[0052] Fig.34 is a diagram showing a configuration example of a photoelectric conversion element according to a third embodiment of the present technology.

[0053] Fig.35 is a circuit diagram showing a configuration example of a pixel according to a fourth embodiment of the present technology.

[0054] Fig.36 : is a circuit diagram showing a configuration example of a column signal processing circuit according to a fourth embodiment of the present technology.

[0055] Fig.37 is a circuit diagram showing a configuration example of a pixel according to a fifth embodiment of the present technology.

[0056] Fig.38 : is a circuit diagram showing a configuration example of a pixel in a first modification example of the fifth embodiment of the present technology.

[0057] Fig.39 : is a timing chart showing an example of the global shutter operation in the first modification example of the fifth embodiment of the present technology.

[0058] Fig.40 : is a diagram showing an example of a stacked structure of a solid-state imaging element in a second modification example of the fifth embodiment of the present technology.

[0059] Fig.41 : is a circuit diagram showing a configuration example of a pixel in a second modification example of the fifth embodiment of the present technology.

[0060] Fig.42 : is a diagram showing an example of a stacked structure of a solid-state imaging element in a third modification example of the fifth embodiment of the present technology.

[0061] Fig.43 is a circuit diagram showing a configuration example of a pixel according to a sixth embodiment of the present technology.

[0062] Fig.44 : is a timing chart showing an example of a global shutter operation according to the sixth embodiment of the present technology.

[0063] Fig.45 is a circuit diagram showing a configuration example of a pixel according to a seventh embodiment of the present technology.

[0064] Fig.46 : is a timing chart showing an example of voltage control according to the seventh embodiment of the present technology.

[0065] Fig.47 is a timing chart illustrating an example of a rolling shutter operation according to the eighth embodiment of the present technology.

[0066] Fig.48 : is a block diagram showing a configuration example of a solid-state imaging element according to a ninth embodiment of the present technology.

[0067] Fig.49 : is a circuit diagram showing a configuration example of a dummy pixel, a regulator, and a switch section according to a ninth embodiment of the present technology.

[0068] Fig.50 : is a timing chart showing an example of the operation of the dummy pixel and the adjuster in the ninth embodiment of the present technology.

[0069] Fig.51 is a circuit diagram showing a configuration example of an effective pixel according to a ninth embodiment of the present technology.

[0070] Fig.52 : is a timing chart showing an example of a global shutter operation according to the ninth embodiment of the present technology.

[0071] Fig.53 It is a diagram for explaining the effects in the ninth embodiment of the present technology.

[0072] Fig.54 is a block diagram showing a schematic configuration example of a vehicle control system.

[0073] Fig.55 : is an explanatory diagram showing an example of the installation position of the imaging section. DETAILED DESCRIPTION

[0074] Hereinafter, a mode for carrying out the present technology (hereinafter, referred to as an embodiment) will be described. The description will be given in the following order.

[0075] 1. First Embodiment (Example in which an Integrating Circuit and ADC Perform Product-Sum Operation)

[0076] 2. Second Embodiment (Example in which the filter size is made variable and the integration circuit and ADC perform product-sum operations)

[0077] 3. Third Embodiment (Example in which an Integrating Circuit and ADC Perform Product-Sum Operation)

[0078] 4. Fourth Embodiment (Example in which a photodiode is embedded and an integration circuit and ADC perform product-sum operations)

[0079] 5. Fifth Embodiment (Example of Making the First Capacitance Element and the Second Capacitance Element Hold Pixel Signals)

[0080] 6. Sixth Embodiment (Example of adding a discharge transistor and causing the first and second capacitive elements to hold pixel signals)

[0081] 7. Seventh Embodiment (Example of Controlling Reset Power Supply Voltage by Making First Capacitance Element and Second Capacitance Element Hold Pixel Signal)

[0082] 8. Eighth Embodiment (Example of Making the First and Second Capacitive Elements Hold Pixel Signals and Performing Rolling Shutter Operation)

[0083] 9. Ninth Embodiment (Example of Reducing Noise and Making the First Capacitance Element and the Second Capacitance Element Hold Pixel Signal)

[0084] 10. Application examples for mobile objects

[0085] <1. First Embodiment>

[0086] [Configuration Example of Imaging Device]

[0087] Figure 1 1 is a block diagram showing a configuration example of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device that captures image data, and includes an imaging lens 110, a solid-state imaging element 200, a recording section 120, and an imaging control section 130. As the imaging device 100, a digital camera and an electronic device (smartphone, personal computer, etc.) having an imaging function are assumed.

[0088] The solid-state imaging element 200 captures image data and performs predetermined processing such as image recognition processing under the control of the imaging control section 130 . The solid-state imaging element 200 supplies the processed data to the recording section 120 via the signal line 209 .

[0089] The imaging lens 110 condenses light and guides the light to the solid-state imaging element 200. The imaging control section 130 controls the solid-state imaging element 200 to capture image data. For example, the imaging control section 130 provides an imaging control signal including a vertical synchronization signal VSYNC to the solid-state imaging element 200 via a signal line 139. The recording section 120 records the processed data.

[0090] Here, the vertical synchronization signal VSYNC is a signal indicating imaging timing, and a period signal of a constant frequency such as 60 Hz is used as the vertical synchronization signal VSYNC.

[0091] It is to be noted that although the imaging device 100 records the processed data, the data may be transmitted to the outside of the imaging device 100. In this case, an external interface for transmitting data is further provided. Alternatively, the imaging device 100 may further display the processing result. In this case, a display section is further provided.

[0092] [Configuration example of solid-state imaging element]

[0093] Figure 2 1 is a block diagram showing a configuration example of a solid-state imaging element 200 according to a first embodiment of the present technology. The solid-state imaging element 200 includes a vertical scanning circuit 211, a pixel array section 220, a timing control circuit 212, a digital-to-analog converter (DAC) 213, a load MOS circuit block 250, and a column signal processing circuit 260. In the pixel array section 220, a plurality of pixels 300 are arranged in a two-dimensional grid pattern. In addition, each circuit in the solid-state imaging element 200 is provided in, for example, a single semiconductor chip.

[0094] Hereinafter, a group of pixels 300 arranged in the horizontal direction is referred to as a “row”, and a group of pixels 300 arranged in a direction perpendicular to the row is referred to as a “column”.

[0095] The timing control circuit 212 controls the operation timing of each of the vertical scanning circuit 211 , the DAC 213 , and the column signal processing circuit 260 in synchronization with the vertical synchronization signal VSYNC from the imaging control section 130 .

[0096] The DAC 213 generates a sawtooth-shaped ramp signal through digital-to-analog (DA) conversion and provides the generated ramp signal to the column signal processing circuit 260 .

[0097] The vertical scanning circuit 211 sequentially selects and drives the rows to output analog pixel signals. Each pixel 300 photoelectrically converts incident light to generate an analog pixel signal. The pixel 300 supplies the pixel signal to the column signal processing circuit 260 via the load MOS circuit block 250.

[0098] In the load MOS circuit block 250, a MOS transistor that supplies a constant current is provided for each column.

[0099] The column signal processing circuit 260 performs signal processing such as analog-to-digital (AD) conversion processing and correlated double sampling (CDS) processing on the pixel signal of each column, and supplies the processed data to the recording section 120 .

[0100] [Pixel configuration example]

[0101] Figure 3 3 is a circuit diagram showing a configuration example of a pixel 300 according to a first embodiment of the present technology. The pixel 300 includes a previous stage circuit 310 , capacitance elements 321 and 322 , a selection circuit 330 , a subsequent stage reset transistor 341 , and a subsequent stage circuit 350 .

[0102] The previous stage circuit 310 includes a photoelectric conversion element 311 , a transfer transistor 312 , a floating diffusion (FD) reset transistor 313 , an FD 314 , a previous stage amplifier transistor 315 , a precharge transistor 318 , and a current source transistor 316 .

[0103] The photoelectric conversion element 311 generates electric charge by photoelectric conversion. The transfer transistor 312 transfers the electric charge from the photoelectric conversion element 311 to the FD 314 according to the transfer signal trg from the vertical scanning circuit 211 .

[0104] The FD reset transistor 313 extracts charge from the FD 314 according to the FD reset signal rst from the vertical scanning circuit 211 to initialize the FD 314. The FD 314 accumulates the charge and generates a voltage corresponding to the amount of charge. The pre-stage amplifier transistor 315 amplifies the level of the voltage of the FD 314 and outputs the amplified voltage to the pre-stage node 320.

[0105] Furthermore, respective sources of the FD reset transistor 313 and the pre-stage amplifier transistor 315 are connected to the power supply voltage VDD.

[0106] The precharge transistor 318 opens and closes the path between the previous node 320 and the current source transistor 316 according to the control signal PC from the vertical scanning circuit 211. For example, just after the reset level is held in the capacitor 321, the vertical scanning circuit 211 opens the precharge transistor 318 by the control signal PC. Therefore, it is possible to prevent charge from being extracted from the capacitors 321 and 322. The current source transistor 316 supplies the current Id1 according to the bias voltage vb.

[0107] The capacitance elements 321 and 322 have their respective one ends commonly connected to the previous-stage node 320 and their respective other ends connected to the selection circuit 330. Note that the capacitance elements 321 and 322 are examples of the first capacitance element and the second capacitance element described in the claims.

[0108] The selection circuit 330 includes a selection transistor 331 and a selection transistor 332. The selection transistor 331 opens and closes the path between the capacitor element 321 and the subsequent node 340 according to the selection signal Φr from the vertical scanning circuit 211. The selection transistor 332 opens and closes the path between the capacitor element 322 and the subsequent node 340 according to the selection signal Φs from the vertical scanning circuit 211.

[0109] The post-stage reset transistor 341 initializes the level of the post-stage node 340 to a predetermined potential Vreg according to a post-stage reset signal rstb from the vertical scanning circuit 211. A potential different from VDD (eg, a potential lower than VDD) is set as the potential Vreg.

[0110] The post-stage circuit 350 includes a post-stage amplifier transistor 351 and a post-stage selection transistor 352. The post-stage amplifier transistor 351 amplifies the level of the post-stage node 340. The post-stage selection transistor 352 outputs the signal of the level amplified by the post-stage amplifier transistor 351 to the vertical signal line 309 as a pixel signal according to the post-stage selection signal selb from the vertical scanning circuit 211.

[0111] Note that, for example, n-channel metal oxide semiconductor (nMOS) transistors are used as various transistors (the transfer transistor 312 and the like) in the pixel 300 .

[0112] The vertical scanning circuit 211 supplies a high-level FD reset signal rst and a high-level transfer signal trg to all pixels at the start of exposure. Thus, the photoelectric conversion element 311 is initialized. Hereinafter, this control is referred to as "PD reset".

[0113] Then, the vertical scanning circuit 211 provides a high-level FD reset signal rst within the pulse period, and simultaneously sets the subsequent reset signal rstb and the selection signal Φr to a high level for all pixels immediately before the end of exposure. Thus, the FD 314 is initialized, and the level corresponding to the FD 314 at this time is held in the capacitor element 321. This control is hereinafter referred to as "FD reset".

[0114] The level of the FD 314 at the time of FD reset and the level corresponding to the level of the FD 314 (the level held in the capacitive element 321 and the level of the vertical signal line 309 ) are hereinafter collectively referred to as “P phase” or “reset level”.

[0115] At the end of exposure, the vertical scanning circuit 211 provides a high-level transfer signal trg in the pulse period, and sets the subsequent reset signal rstb and the selection signal Φs to a high level for all pixels. Therefore, the signal charge corresponding to the exposure amount is transferred to the FD 314, and the level corresponding to the FD 314 at this time is held in the capacitor 322.

[0116] The level of the FD 314 at the time of signal charge transfer and the level corresponding to the level of the FD 314 (the level held in the capacitance element 322 and the level of the vertical signal line 309 ) are hereinafter collectively referred to as “D phase” or “signal level”.

[0117] The exposure control that starts and ends exposure to all pixels at the same time in this way is called a global shutter method. Under this exposure control, the front-stage circuit 310 of all pixels generates a reset level and a signal level in sequence. The reset level is maintained in the capacitor element 321, and the signal level is maintained in the capacitor element 321.

[0118] After the exposure is finished, the vertical scanning circuit 211 sequentially selects one row and sequentially outputs the reset level and the signal level of the row. For example, it is assumed that the reset level and the signal level are output in this order.

[0119] In this case, the vertical scanning circuit 211 provides a high-level selection signal Φr for a predetermined period, and simultaneously sets the FD reset signal rst and the subsequent stage selection signal selb of the selected row to a high level. As a result, the capacitance element 321 is connected to the subsequent stage node 340, and a reset level is output.

[0120] After outputting the reset level, the vertical scanning circuit 211 provides a high-level rear-stage reset signal rstb during the pulse period, while maintaining the FD reset signal rst and the rear-stage selection signal selb of the selected row at a high level. As a result, the level of the rear-stage node 340 is initialized. At this time, the selection transistor 331 and the selection transistor 332 are both in an on state, and the capacitor elements 321 and 322 are disconnected from the rear-stage node 340.

[0121] After initialization of the subsequent node 340, the vertical scanning circuit 211 provides a high-level selection signal Φs for a predetermined period while maintaining the FD reset signal rst and the subsequent selection signal selb of the selected row at a high level. As a result, the other of the capacitive elements 322 is connected to the subsequent node 340 and outputs a signal level.

[0122] When the vertical scanning circuit 211 supplies the selection signal Φr at a high level next to the selection signal Φs at a high level, the pixel 300 can also output the signal level and the reset level in this order.

[0123] Please note that Figure 4 As shown, the circuits in the solid-state imaging element 200 may be dispersedly arranged on the pixel chip 201 and the circuit chip 202 having a stacked structure. In this case, for example, the photoelectric conversion element 311, the transfer transistor 312, the FD reset transistor 313, the FD 314, and the pre-stage amplifier transistor 315 are provided in the pixel chip 201, and the remaining elements and circuits are provided in the circuit chip 202.

[0124] [Configuration Example of Column Signal Processing Circuit]

[0125] Figure 5 : is a block diagram showing a configuration example of the load MOS circuit block 250 and the column signal processing circuit 260 according to the first embodiment of the present technology.

[0126] In the load MOS circuit block 250, a vertical signal line 309 is wired for each column. When the number of columns is M (M is an integer), M vertical signal lines 309 are wired. In addition, a load MOS transistor 251 that supplies a constant current id2 is connected to each of the vertical signal lines 309.

[0127] In the column signal processing circuit 260, a plurality of integration circuits such as integration circuits 270-1, 270-2, and 270-3, a switch circuit 261, a plurality of ADCs such as ADCs 280-1 and 280-2, and an image recognition section 290 are arranged. When the size of the filter to be applied in the horizontal direction is K (K is an integer) columns, each integration circuit is provided for each column, and each ADC is provided for every K columns. In the figure, ADCs are provided every three columns.

[0128] Here, the filter is a filter coefficient array to be multiplied by a pixel signal of each of the plurality of pixels, and is also referred to as a kernel.

[0129] The integration circuit 270-1 time-integrates the individual pixel signals of the J pixels in the corresponding column with the vertical size of the filter applied as J (J is an integer) rows. The integration time of each pixel of the integration circuit 270-1 is the time corresponding to the absolute value of the filter coefficient corresponding to the pixel. For example, the integration time of a pixel with a filter coefficient of "-2" is set to twice the integration time of a pixel with a filter coefficient of "+1". The integration circuit 270-1 provides the integration signal to the switch circuit 261. Note that the configurations of the integration circuits 270-2 and 270-3 are similar to that of the integration circuit 270-1.

[0130] The switching circuit 261 switches the output destination of each integration circuit based on the application range of the filter. Assume that the filter is applied to three columns corresponding to the integration circuits 270-1, 270-2, and 270-3. In this case, for example, the switching circuit 261 sets the output destinations of these integration circuits to the ADC 280-1. Then, in the case where the filter slides one column to the right, the switching circuit 261 switches the output destinations of the integration circuits 270-2 and 270-3 to the ADC 280-2.

[0131] The ADC 280-1 analog-adds the respective integrated signals of the plurality of integrating circuits to convert the signals into digital signals. The ADC 280-1 supplies the digital signals to the image recognition section 290. It should be noted that the configuration of the ADC 280-2 is similar to that of the ADC 280-1.

[0132] The image recognition section 290 performs image recognition processing using a digital signal from each ADC such as the ADC 280 - 1 . The image recognition section 290 outputs the processed data to the recording section 120 .

[0133] [Configuration example of integrating circuit and ADC]

[0134] Figure 6 270 - 1 and ADC 280 - 1 according to the first embodiment of the present technology. The integration circuit 270 - 1 includes an input switch 271 , a resistance element 272 , an operational amplifier 273 , a capacitance element 274 , and an auto-zero switch 275 .

[0135] The input switch 271 opens and closes the path between the vertical signal line 309 of the corresponding column and the resistance element 272 according to the control signal INT from the timing control circuit 212. The resistance element 272 is inserted between the input switch 271 and the inverting input terminal (-) of the operational amplifier 273. In addition, the non-inverting input terminal (+) of the operational amplifier 273 is grounded. The capacitance element 274 is inserted between the connection node of the input switch 271 and the resistance element 272 and the output terminal of the operational amplifier 273.

[0136] The auto-zero switch 275 opens and closes a path between a connection node of the input switch 271 and the resistance element 272 and an output terminal of the operational amplifier 273 in accordance with a control signal AZ1 from the timing control circuit 212 .

[0137] By using the circuit configuration shown in the figure, multiple pixel signals in the corresponding column can be time-integrated. The integration time can be adjusted by controlling the time when the input switch 271 is in the closed state using the control signal INT. The integration time of each pixel is set to the time corresponding to the absolute value of the filter coefficient corresponding to the pixel.

[0138] Assuming that the size of the filter is 3 rows×3 columns, each of the integration circuits 270 - 1 , 270 - 2 , and 270 - 3 temporally integrates pixel signals of three pixels in the corresponding column and provides the integrated signals to the ADC 280 - 1 via the switch circuit 261 .

[0139] ADC 280-1 includes capacitors 281, 282-1, 282-2 and 282-3, comparator 285, auto-zero switch 286 and counter 287. Capacitor 281 is inserted between DAC 213 and the non-inverting input terminal (+) of comparator 285. One end of each capacitor 282-1 to 282-3 is connected to different integration circuits. For example, one end of capacitor 282-1 is connected to integration circuit 270-1 via switch circuit 261, and one end of capacitor 282-2 is connected to integration circuit 270-2 via switch circuit 261. One end of capacitor 282-3 is connected to integration circuit 270-3 via switch circuit 261. In addition, the other end of capacitors 282-1 to 282-3 is connected to the inverting input terminal (-) of comparator 285 in common. The capacitance values ​​of capacitors 282-1 to 282-3 are substantially the same. The capacitance value of the capacitive element 281 substantially matches the sum of the capacitance values ​​of the capacitive elements 282 - 1 to 282 - 3 .

[0140] The comparator 285 compares the ramp signal Rmp from the DAC 213 with a signal obtained by analog-adding the respective integrated signals of the integrating circuits 270 - 1 to 270 - 3 , and outputs the comparison result to the counter 287 .

[0141] The auto-zero switch 286 opens and closes the path between the inverting input terminal (−) and the output terminal of the comparator 285 according to the control signal AZ2 from the timing control circuit 212 .

[0142] The counter 287 counts the count value during the period until the comparison result of the comparator 285 is inverted. The counter 287 supplies a digital signal indicating the count value to the image recognition unit 290. In addition, the counter 287 corresponds to up-counting and down-counting, and the timing control circuit 212 can control whether to perform up-counting or down-counting.

[0143] Figure 7 is an example of an overall view of the solid-state imaging element 200 according to the first embodiment of the present technology.

[0144] Multiple pixels arranged in a two-dimensional grid pattern generate and hold analog pixel signals respectively. Each pixel signal includes a reset level and a signal level corresponding to the exposure amount. Each pixel holds the reset level and the signal level and outputs the reset level and the signal level in sequence.

[0145] Here, the size of the filter is defined as J rows × K columns. In the J rows × K columns to which the filter is applied, the signal to be output first among the reset level and the signal level of the j-th row and the k-th column is represented as S1_[j, k], and the signal to be output next is represented as S2_[j, k]. j is an integer from 0 to J-1, and k is an integer from 0 to K-1. In addition, the integration time according to the absolute value of the filter coefficient corresponding to the j-th row and the k-th column is defined as t_[j, k].

[0146] The integration circuit in the kth column performs time integration of each pixel signal (reset level or signal level) of the J pixels in the column. The ADC 280-1 analog-adds the integration signals of each column to convert the analog-added integration signals into digital signals. The counter in the ADC 280-1 counts up when converting the analog signal corresponding to S1_[j, k], and counts down when converting the analog signal corresponding to S2_[j, k]. As a result, a CDS process is implemented to obtain the difference between the reset level and the signal level (in other words, the net signal level). It should be noted that although the ADC performs both AD conversion and CDS processing, a configuration in which a CDS circuit is added in a subsequent stage, the ADC performs only AD conversion, and the CDS circuit performs CDS processing may be adopted.

[0147] Then, the ADC 280-1 outputs a digital signal indicating the count value. The digital signal Convd is represented by the following convolution integral.

[0148] [Mathematical formula 1]

[0149]

[0150] In addition, when the sign of the corresponding filter coefficient is positive, each of the pixels outputs a reset level and a signal level in a predetermined order, and when the sign is negative, the reset level and the signal level are output in a reverse order. For example, when the filter coefficient is positive, the pixel outputs a signal level and a reset level in this order, and when the filter coefficient is negative, the pixel outputs a reset level and a signal level in this order. As described above, by changing the output order according to the sign of the filter coefficient, the sign of the filter coefficient can be reflected on the sign of the difference after CDS processing.

[0151] In addition, the image recognition unit 290 includes a convolutional neural network (CNN) processing unit 291 and a data comparison unit 292. The CNN processing unit 291 performs CNN processing on the digital signal of each ADC. The CNN includes a plurality of convolutional layers, and the product-sum operation of the first convolutional layer is implemented by an analog circuit including a plurality of integration circuits and a comparator (not shown) in the ADC. The operation of the second and subsequent convolutional layers is performed by the CNN processing unit 291. The CNN processing unit 291 provides the data of the processed feature quantity to the data comparison unit 292.

[0152] The data comparison unit 292 compares the feature quantity obtained by the CNN process with the feature quantity of the recognition target, and outputs a comparison result as a recognition result.

[0153] Note that although the CNN processing section 291 and the data comparison section 292 are arranged in the solid-state imaging element 200, all or part of them may be arranged outside the solid-state imaging element 200. For example, Figure 8 As shown, the CNN processing unit 291 and the data comparison unit 292 can be provided in a digital signal processing (DSP) circuit outside the solid-state imaging element 200 .

[0154] In addition, although the ADC performs AD conversion processing and CDS processing by up-counting and down-counting, the AD conversion processing may be performed by only one of up-counting and down-counting. In this case, the subsequent circuit performs CDS processing.

[0155] Here, a configuration in which a product-sum operation of a convolutional layer is performed after AD conversion is assumed as a comparative example.

[0156] Fig. 9 2 is an example of an overall view of the solid-state imaging element 200 in the comparative example. In the comparative example, an ADC is provided for each column. In addition, the integration circuit 270-1 and the switching circuit 261 are not provided, and a frame memory is further provided. The frame memory holds image data including a digital signal after AD conversion. The CNN processing unit 291 performs CNN processing on the image data in the frame memory. The product-sum operation of the first convolutional layer of the CNN is performed by a digital circuit.

[0157] On the other hand, Figure 7 and Figure 8 In the solid-state imaging element 200 shown in FIG. 1 , since it is not necessary to store image data in a frame memory, power consumption can be reduced compared to the comparative example. Figure 7 and Figure 8In the solid-state imaging element 200 shown, since the product-sum operation of the first convolutional layer is performed in parallel by the integration circuit of each column and the ADC of every three columns, the processing speed can be faster than the comparative example. In addition, since the integration signals of the three columns are added at the same time, the processing speed can be faster than the processing speed in the non-patent document 1 in which the integration signals of the three columns are added one by one. As a result, in particular, the accuracy of image recognition of moving objects can be improved.

[0158] Furthermore, since the filter coefficient is reflected in the integration time, any filter coefficient can be set, and the accuracy of image recognition can be improved, compared with Non-Patent Literature 1 in which the filter coefficient is limited to +1, 0, and -1.

[0159] In addition, Figure 7 and Figure 8 In the solid-state imaging element 200 in the embodiment, since exposure is performed by a global shutter method, rolling shutter distortion is eliminated and surface simultaneity can be maintained. As a result, image recognition can be performed using high-quality image data equivalent to image data of a normal CMOS image sensor (CIS). Since such a solid-state imaging element 200 is advantageous for the recognition of moving objects moving at high speed, it can be used for the following various applications.

[0160] (1) An application in which the solid-state imaging element 200 is mounted on a vehicle-mounted camera and autonomous driving is achieved using the result of image recognition processing.

[0161] (2) Applications in which the solid-state imaging element 200 is mounted on an industrial camera and various inspections are performed using the results of image recognition processing.

[0162] (3) An application in which the solid-state imaging element 200 is mounted on a smart device in a traffic signal or the like in a city, and traffic control or the like is performed using the result of image recognition processing.

[0163] (4) An application in which the solid-state imaging element 200 is mounted on a surveillance camera in a store, and the result of image recognition processing is used to detect shoplifting or analyze the purchasing tendency of customers.

[0164] (5) An application in which the solid-state imaging element 200 is mounted on a camera set up in a stadium or the like, and data analysis on sports is performed using the result of image recognition processing.

[0165] [Operation Example of Solid-State Imaging Element]

[0166] Fig.101 is a timing diagram showing an example of the global shutter operation according to the first embodiment of the present technology. The vertical scanning circuit 211 supplies a high-level FD reset signal rst and a high-level transfer signal trg to all rows (in other words, all pixels) in a period from the time T0 immediately before the start of exposure to the time T1 after the pulse period has passed. Therefore, all pixels are reset by PD, and exposure starts simultaneously in all rows.

[0167] Here, rst_[n] and trg_[n] in the figure indicate signals to pixels of the nth row among N rows. N is an integer indicating the total number of rows, and n is an integer from 1 to N.

[0168] Then, at time T2 immediately before the end of the exposure period, the vertical scanning circuit 211 provides a high-level FD reset signal rst within the pulse period, and simultaneously sets the subsequent reset signal rstb and the selection signal Φr to a high level for all pixels. Thus, all pixels are reset by FD, and the reset level is sampled and held. Here, rstb_[n] and Φr_[n] in the figure indicate signals to the pixels of the nth row.

[0169] At time T3 after time T2, the vertical scanning circuit 211 returns the selection signal Φr to the low level.

[0170] At the moment T4 as the end of exposure, the vertical scanning circuit 211 provides a high-level transfer signal trg during the pulse, and sets the rear-stage reset signal rstb and the selection signal Φs to a high level for all pixels. As a result, the signal level is sampled and held. In addition, the level of the front-stage node 320 is reduced from the reset level (VDD-Vgs) to the signal level (VDD-Vgs-Vsig). Here, VDD represents the power supply voltage, and Vsig represents the net signal level obtained as a result of the CDS process. Vgs represents the gate-source voltage of the front-stage amplifier transistor 315. In addition, Φs_[n] in the figure represents the signal to the pixel of the nth row.

[0171] At time T5 after time T4, the vertical scanning circuit 211 returns the selection signal Φs to the low level.

[0172] In addition, the vertical scanning circuit 211 controls the current source transistor 316 of all rows (all pixels) to supply current id1. Here, idl_[n] in the figure represents the current of the pixel in the nth row. The larger the current id, the larger the IR drop becomes, thereby requiring the current idl to be on the order of a few nanoamperes (nA) to tens of nanoamperes (nA). On the other hand, the load MOS transistors 251 of all columns are in the off state, and the current id2 is not supplied to the vertical signal line 309.

[0173] Fig.11 1 is a timing diagram showing an example of a first auto-zero operation and a first (in other words, zeroth row) integration operation according to a first embodiment of the present technology. a in the accompanying drawings shows an example of a filter. The size of the filter is 3 rows × 3 columns, and the relative rows and columns in the filter are set to the zeroth row, the first row, and the second row, and the zeroth row, the first column, and the second column.

[0174] After exposure by the global shutter method, the solid-state imaging element 200 performs convolution integration while sliding the filter in the horizontal direction or the vertical direction. However, as described above, since the ADC is set every three columns, the solid-state imaging element 200 can perform M / 3 convolution integrations in parallel, where the number of columns is M. Therefore, the number of sliding in the horizontal direction is twice.

[0175] Furthermore, the operation period of the convolution integration for three rows includes a first auto-zero period, an integration period for three rows, a first reading period, a second auto-zero period, an integration period for three rows, and a second reading period.

[0176] b in the figure is a timing chart showing an example of the first auto-zero operation and the first (zeroth row) integration operation.

[0177] In the first auto-zero period, the vertical scanning circuit 211 supplies the high-level control signals AZ1 and AZ2 from the timing T11 to the pulse period.

[0178] The vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent selection signal selb to the zeroth row during the integration period of the zeroth row from time T12 to T14. In the drawing, selb_[n] indicates a signal to a pixel of the nth row.

[0179] At time T15 immediately after the integration time of the zeroth row, the vertical scanning circuit 211 supplies the high-level subsequent-stage reset signal rstb to the zeroth row.

[0180] In addition, the vertical scanning circuit 211 sets one of the selection signals Φr and Φs to a high level during the integration period of the zeroth row and outputs one of a reset level and a signal level for each column. Which of the selection signals Φr and Φs is set to a high level is determined by the sign of the corresponding filter coefficient.

[0181] For example, it is assumed that the filter coefficients in the zeroth row, the zeroth column, the first column, and the second column are "-1", "+1", and "-1". At this time, the selection signal Φr_

[00] of the zeroth row and the zeroth column is controlled to a high level, the selection signal Φs_

[01] of the zeroth row and the first column is controlled to a high level, and the selection signal Φr_

[02] of the zeroth row and the second column is controlled to a high level. Therefore, a reset level is output from the zeroth row and the zeroth column, a signal level is output from the zeroth row and the first column, and a reset level is output from the zeroth row and the second column.

[0182] In addition, after the moment T13 during the integration period of the zeroth row, the timing control circuit 212 provides a control signal INT for time integration of the pixel signal to each column. INT_[m] in the figure indicates the signal of the integration circuit of the mth column (m is 0, 1 or 2). The pulse width of the control signal INT_[m] is a value corresponding to the absolute value of the corresponding filter coefficient. Since the filter coefficients in the zeroth row and the zeroth column, the first column and the second column are "-1", "+1" and "-1", and their absolute values ​​are the same, the pulse widths of the control signals INT_[0], INT_[1] and INT_[2] are set to be the same.

[0183] Fig.12 1 is a diagram showing an example of the state of the solid-state imaging element 200 in the first automatic zero adjustment period according to the first embodiment of the present technology. In the integration circuit of each column, the automatic zero adjustment switch 275 is closed by the control signal AZ1, and the integration circuit is reset. In addition, in each ADC, the automatic zero adjustment switch 286 is closed by the control signal AZ2, and the comparator 285 is reset.

[0184] Fig.13 : is a diagram showing an example of the state of the solid-state imaging element 200 in the first integration period (in the zeroth row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of a pixel to be integrated.

[0185] B in the figure shows an example of the state of the solid-state imaging element 200 in the first integration period (in the zeroth row). In the integration circuit of each column, the input switch 271 is closed during the integration time according to the absolute value of the filter coefficient of the control signal INT. The filter coefficients "-1", "+1" and "-1" in the zeroth row and the zeroth column, the first column and the second column have the same absolute value, and therefore the integration time of each column is the same.

[0186] The integration time of each column is t, and the reset level (P-phase level) output from the pixel in the zeroth row and the zeroth column is P00. The signal level (D-phase level) output from the pixel in the zeroth row and the first column is D01, and the reset level (P-phase level) output from the pixel in the zeroth row and the second column is P02. In this case, the integrated signals of the zeroth column, the first column, and the second column are P00×t, D01×t, and P02×t.

[0187] Fig.14 : is a timing chart showing an example of the second (in other words, first row) integration operation according to the first embodiment of the present technology.

[0188] During the integration period of the first row from time T16 to T17, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent-stage selection signal selb to the first row.

[0189] At time T18 immediately after the integration time of the first row, the vertical scanning circuit 211 supplies the high-level subsequent-stage reset signal rstb to the first row.

[0190] Furthermore, it is assumed that the filter coefficients in the first row and the zeroth column, the first column, and the second column are "+1", "+3", and "+1". At this time, during the integration period of the first row, the selection signal Φs_

[10] of the zeroth column of the first row is controlled to be a high level, the selection signal Φs_

[11] of the first column of the first row is controlled to be a high level, and the selection signal Φs_

[12] of the second column of the first row is controlled to be a high level. Therefore, a signal level is output from each of the zeroth column, the first column, and the second column of the first row.

[0191] In addition, during the integration period of the first row, the timing control circuit 212 provides a control signal INT for time-integrating the pixel signal to each column. Since the filter coefficients in the zeroth row and the zeroth column, the first column, and the second column are "+1", "+3", and "+1", the pulse width of the control signal INT_[1] is set to three times the pulse width of the control signal INT_[0] and the control signal INT_[2].

[0192] Fig.15 : is a diagram showing an example of the state of the solid-state imaging element 200 in the second integration period (first row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of the pixel to be integrated.

[0193] b in the figure shows an example of the state of the solid-state imaging element 200 in the second integration period. In the integration circuit of each column, the input switch 271 is closed during the integration time according to the absolute value of the filter coefficient of the control signal INT. Since the filter coefficients in the first row and the zeroth column, the first column and the second column are "+1", "+3" and "+1", the integration time of the first column is three times the integration time of the zeroth column and the second column.

[0194] The integration time of the zeroth column is t, and the signal level (D-phase level) output from the pixel in the first row and the zeroth column is D10. The signal level (D-phase level) output from the first row and the first column pixel is D11, and the signal level (D-phase level) output from the first row and the second column pixel is D12. In this case, the integrated signal of the zeroth column is P00×t+D10×t, the integrated signal of the first column is D01×t+D11×3t, and the integrated signal of the second column is P02×t+D12×t.

[0195] Fig.16 is a timing chart showing an example of the third (in other words, second row) integration operation according to the first embodiment of the present technology.

[0196] During the integration period of the second row from the timing T19 to T21, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent-stage selection signal selb to the second row.

[0197] At time T22 immediately after the integration time of the second row, the vertical scanning circuit 211 supplies the high-level subsequent-stage reset signal rstb to the second row.

[0198] Furthermore, it is assumed that the filter coefficients in the second row and the zeroth column, the first column, and the second column are "-1", "+1", and "-1". At this time, the selection signal Φr_

[20] of the second row and the zeroth column is controlled to be a high level, the selection signal Φs_

[21] of the second row and the first column is controlled to be a high level, and the selection signal Φr_

[22] of the second row and the second column is controlled to be a high level. Therefore, a reset level is output from the second row and the zeroth column, a signal level is output from the second row and the first column, and a reset level is output from the second row and the second column.

[0199] In addition, after the time T20 in the integration period of the second row, the timing control circuit 212 provides a control signal INT for time-integrating the pixel signal to each column. Since the filter coefficients in the second row and the zeroth column, the first column, and the second column are "-1", "+1", and "-1", and their absolute values ​​are the same, the pulse widths of the control signals INT_[0], INT_[1], and INT_[2] are set to be the same.

[0200] Fig.17: is a diagram showing an example of the state of the solid-state imaging element in the third integration period (second row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of the pixel to be integrated.

[0201] B in the figure shows an example of the state of the solid-state imaging element 200 in the third integration period (in the second row). In the integration circuit of each column, the input switch 271 is closed within the integration time according to the absolute value of the filter coefficient of the control signal INT. The filter coefficients "-1", "+1", and "-1" in the first row and the zeroth column, the first column, and the second column have the same absolute value, and therefore the integration time of each column is the same.

[0202] The integration time of each column is t, and the reset level (P-phase level) output from the pixel in the second row and the zeroth column is P20. The signal level (D-phase level) output from the pixel in the second row and the first column is represented by D21, and the reset level (P-phase level) output from the pixel in the second row and the second column is represented by P22. In this case, the integration signal of the zeroth column is P00×t+D10×t+P20×t. And the integration signal of the first column is D01×t+D11×3t+D21×t, and the integration signal of the second column is P02×t+D12×t+P22×t.

[0203] Fig.18 2 is a timing chart showing an example of the first reading operation according to the first embodiment of the present technology. The ramp signal Rmp gradually fluctuates in the reading period from the timing T23 to T24, and AD conversion is performed.

[0204] Fig.19 : is a diagram showing an example of the state of the solid-state imaging element 200 in the first reading period according to the first embodiment of the present technology. The integration circuits 270-1, 270-2, and 270-3 provide the held integration signal to the ADC 280-1. The ADC 280-1AD converts the signal obtained by analog addition of the integration signal into a digital signal Conv1. The digital signal Conv1 is represented by the following formula.

[0205] Conv1=P00×t+D10×t+P20×t

[0206] +D01×t+D11×3t+D21×t

[0207] +P02×t+D12×t+P22×t...Formula 2

[0208] Fig. 20 is a timing chart showing an example of the second auto-zero operation and the fourth (in other words, zeroth row) integration operation according to the first embodiment of the present technology.

[0209] In the second auto-zero period, the vertical scanning circuit 211 supplies the high-level control signal AZ1 to the n-th row from the timing T25 in the pulse period.

[0210] The vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent-stage selection signal selb to the zeroth row during the integration period of the zeroth row from the timing T26 to T28.

[0211] Furthermore, it is assumed that the filter coefficients in the zeroth row and the zeroth column, the first column, and the second column are "-1", "+1", and "-1". At this time, the selection signal Φs_

[00] of the zeroth row and the zeroth column is controlled to be a high level, the selection signal Φr_

[01] of the zeroth row and the first column is controlled to be a high level, and the selection signal Φs_

[02] of the zeroth row and the second column is controlled to be a high level. Therefore, a signal level is output from the zeroth row and the zeroth column, a reset level is output from the zeroth row and the first column, and a signal level is output from the zeroth row and the second column.

[0212] In addition, after the time T27 during the integration period of the zeroth row, the timing control circuit 212 provides the control signal INT for time integration of the pixel signal to each column. Because the absolute values ​​of the filter coefficients in the zeroth row and the zeroth column, the first column, and the second column are the same, the pulse widths of the control signals INT_[0], INT_[1], and INT_[2] are set to be the same.

[0213] like Fig.11 b and Fig. 20 As shown in, for each pixel, the vertical scanning circuit 211 sets one of the selection signals Φr and Φs to a high level in a first integration time period, and sets the other of Φr and Φs to a high level in a second integration time period after a specific time period. Through this control, the selection circuit 330 of the selected row sequentially performs control of connecting the capacitor element 321 to the subsequent node 340, control of disconnecting the capacitor elements 321 and 322 from the subsequent node 340, and control of connecting the capacitor element 322 to the subsequent node 340. In addition, when the capacitor elements 321 and 322 are disconnected from the subsequent node 340, the subsequent reset transistor 341 of the selected row initializes the level of the subsequent node 340. In this way, by disconnecting the capacitor elements 321 and 322 when driving the subsequent reset transistor 341, kTC noise can be reduced.

[0214] Fig.21 AZ1 is a diagram showing an example of the state of the solid-state imaging element in the second auto-zero period according to the first embodiment of the present technology. In the integration circuit of each column, the auto-zero switch 275 is closed by the control signal AZ1, and the integration circuit is reset.

[0215] Fig. 22 : is a diagram showing an example of the state of the solid-state imaging element in the fourth integration period (in the zeroth row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of the pixel to be integrated.

[0216] B in the figure shows an example of the state of the solid-state imaging element 200 in the fourth integration period (zeroth row). In the integration circuit of each column, the input switch 271 is closed within the integration time according to the absolute value of the filter coefficient of the control signal INT. The filter coefficients "-1", "+1" and "-1" in the zeroth row and the zeroth column, the first column and the second column have the same absolute value, and therefore the integration time of each column is the same.

[0217] The integration time of each column is t, and the signal level (D-phase level) output from the pixel in the zeroth row and the zeroth column is D00. The reset level (P-phase level) output from the pixel in the zeroth row and the first column is P01, and the signal level (D-phase level) output from the pixel in the zeroth row and the second column is D02. In this case, the integrated signals of the zeroth column, the first column, and the second column are D00×t, P01×t, and D02×t.

[0218] Fig.23 is a timing chart showing an example of the fifth (in other words, first row) integration operation according to the first embodiment of the present technology.

[0219] During the integration period of the first row from time T29 to T30, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent-stage selection signal selb to the first row.

[0220] In addition, the filter coefficients in the first row and the zeroth column, the first column, and the second column are "+1", "+3", and "+1". At this time, during the integration period of the first row, the selection signal Φr_

[10] of the zeroth column of the first row is controlled to a high level, the selection signal Φr_

[11] of the first column of the first row is controlled to a high level, and the selection signal Φr_

[12] of the second column of the first row is controlled to a high level. As a result, a reset level is output from each of the zeroth column, the first column, and the second column of the first row.

[0221] In addition, during the integration period of the first row, the timing control circuit 212 provides a control signal INT for time-integrating the pixel signal to each column. Since the filter coefficients in the zeroth row and the zeroth column, the first column, and the second column are "+1", "+3", and "+1", the pulse width of the control signal INT_[1] is set to three times the pulse width of the control signal INT_[0] and the control signal INT_[2].

[0222] Fig.241 is a diagram showing an example of the state of the solid-state imaging element in the fifth integration period (first row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of the pixel to be integrated.

[0223] b in the figure shows an example of the state of the solid-state imaging element 200 in the fifth integration period. In the integration circuit of each column, the input switch 271 is closed during the integration time according to the absolute value of the filter coefficient of the control signal INT. Since the filter coefficients in the first row and the zeroth column, the first column and the second column are "+1", "+3" and "+1", the integration time of the first column is three times the integration time of the zeroth column and the second column.

[0224] The integration time of the zeroth column is t, and the reset level (P-phase level) output from the pixel of the first row and the zeroth column is P10. The reset level (P-phase level) output from the pixel of the first row and the first column is P11, and the reset level (P-phase level) output from the pixel of the first row and the second column is P12. In this case, the integration signal of the zeroth column is D00×t+P10×t, the integration signal of the first column is P01×t+P11×3t, and the integration signal of the second column is D02×t+P12×t.

[0225] Fig.25 : is a timing chart showing an example of the sixth (in other words, second row) integration operation according to the first embodiment of the present technology.

[0226] During the integration period of the second row from time T32 to T34, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the subsequent-stage selection signal selb to the second row.

[0227] Furthermore, it is assumed that the filter coefficients in the second row and the zeroth column, the first column, and the second column are "-1", "+1", and "-1". At this time, the selection signal Φs_

[20] of the second row and the zeroth column is controlled to be a high level, the selection signal Φr_

[21] of the second row and the first column is controlled to be a high level, and the selection signal Φs_

[22] of the second row and the second column is controlled to be a high level. Therefore, a signal level is output from the second row and the zeroth column, a reset level is output from the second row and the first column, and a signal level is output from the second row and the second column.

[0228] In addition, after time T33 within the integration period of the second row, the timing control circuit 212 provides a control signal INT for time-integrating the pixel signal to each column. Since the filter coefficients in the second row and the zeroth column, the first column, and the second column are "-1", "+1", and "-1", and their absolute values ​​are the same, the pulse widths of the control signals INT_[0], INT_[1], and INT_[2] are set to be the same.

[0229] Fig.26 : is a diagram showing an example of the state of the solid-state imaging element in the sixth integration period (second row) according to the first embodiment of the present technology. The portion surrounded by a thick frame in the figure indicates the position of the pixel to be integrated.

[0230] The figure b shows an example of the state of the solid-state imaging element 200 in the sixth integration period. In the integration circuit of each column, the input switch 271 is closed during the integration time according to the absolute value of the filter coefficient of the control signal INT. The filter coefficients "-1", "+1", and "-1" in the first row and the zeroth column, the first column, and the second column have the same absolute value, and therefore the integration time of each column is the same.

[0231] The integration time of each column is t, and the signal level (D phase level) output from the pixel of the second row and the zeroth column is D20. The reset level (P phase level) output from the pixel of the second row and the first column is P21, and the signal level (D phase level) output from the pixel of the second row and the second column is D22. In this case, the integration signal of the zeroth column is D00×t+P10×t+D20×t. In addition, the integration signal of the first column is P01×t+P11×3t+P21×t, and the integration signal of the second column is D02×t+P12×t+D22×t.

[0232] Fig. 27 3 is a timing chart showing an example of the second reading operation according to the first embodiment of the present technology. The ramp signal Rmp gradually fluctuates in the reading period from the timing T35 to T36, and AD conversion is performed.

[0233] Fig.28 2 is a diagram showing an example of the state of the solid-state imaging element 200 in the second reading period according to the first embodiment of the present technology. The integration circuits 270-1, 270-2, and 270-3 provide the held integration signal to the ADC 280-1. The ADC 280-1 converts the signal AD obtained by analog addition of the integration signal into a digital signal Conv2, and calculates Convd which is the difference between the first digital signal Conv1 and the second digital signal Conv2. These are represented by the following formulas.

[0234] Conv2=D00×t+P10×t+D20×t

[0235] +P01×t+P11×3t+P21×t

[0236] +D02×t+P12×t+D22×t...Formula 3

[0237] Convd = Conv1 - Conv2 ... Formula 4

[0238] When J and K are both "3", Formula 4 is equivalent to Formula 1.

[0239] Fig.29 2 is a diagram showing an example of the state of the solid-state imaging element 200 when the filter according to the first embodiment of the present technology slides one column in the horizontal direction. The pixels in the coarse frame in the figure indicate the pixels to which the filter is applied. For example, the switching circuit 261 switches the output destinations of the integration circuits 270-2 and 270-3 to the ADC 280-2. In the case of sliding two columns, the switching circuit 261 switches the output destinations of the integration circuits 270-1 and 270-2 to the ADC 280-1, and switches the output destination of 270-3 to the ADC 280-2. In the case of sliding the filter in the vertical direction, the vertical scanning circuit 211 only needs to change the row addresses of the three rows to be driven.

[0240] As shown in Formulas 1 to 4, by changing the output order of the reset level and the signal level according to the sign of the filter coefficient, the sign of the filter coefficient can be reflected in the sign of the difference of each pixel.

[0241] Fig.30 1 is a flowchart showing an example of the operation of the solid-state imaging element 200 according to the first embodiment of the present technology. For example, when a predetermined application for performing image recognition is executed, the operation is started.

[0242] The vertical scanning circuit 211 exposes all pixels (step S901). Each pixel samples and holds the reset level and signal level (step S902). The vertical scanning circuit 211 selects three rows to be read, and the integration circuit of each column performs time integration on the pixel signals (reset level or signal level) for the three rows (step S903). Then, each ADC performs the first AD conversion (step S904). Then, the integration circuit of each column performs time integration on the pixel signals of the three rows (step S905), and each ADC performs the second AD conversion and differential operation (step S906).

[0243] The solid-state imaging element 200 determines whether the filtering process is completed for all pixels (step S907). If the filtering process is not completed (step S907: No), the solid-state imaging element 200 slides the filter in the horizontal direction or the vertical direction (step S909), and repeats step S903 and subsequent steps.

[0244] When the filtering process is completed (step S907: Yes), the solid-state imaging element 200 performs CNN processing (step S908), and compares the result of the CNN processing with the data of the feature quantity of the recognition object (step S909). After step S909, the solid-state imaging element 200 ends the image recognition process.

[0245] Fig.31 : is a diagram showing an example of the use of the solid-state imaging element 200 according to the first embodiment of the present technology. In the accompanying drawings, a is a diagram showing an example of a general CNN. The CNN includes an input layer, a multi-stage convolution layer, one or more pooling layers, a fully connected layer, and an output layer.

[0246] For example, the sum-of-products operation in the first convolutional layer consists of Figure 7 The integration circuit shown in Figure 1 and the analog circuit of the ADC are used for subsequent processing. Figure 7 The CNN processing unit 291 executes it.

[0247] It should be noted that Fig.31 As shown in b, it can also be used for compressed sensing. In the case of compressed sensing, the solid-state imaging element 200 calculates the observed data y as the product of the random design matrix Φm and the sparse input data x. The pixel signals of all pixels are used as the input data x. Therefore, the size of the filter is equivalent to the size of the image data. In the case where x is sparse, then Φ can be used to recover x from y. The data size of the image data can be reduced by this compressed sensing.

[0248] As described above, according to the first embodiment of the present technology, since the ADC performs analog addition on the time-integrated signal through the integration circuit and converts the time-integrated signal into a digital signal, the filter coefficient is not limited and the processing speed is improved. As a result, the accuracy of image recognition can be improved.

[0249] [Modifications]

[0250] In the first embodiment described above, the solid-state imaging element 200 uses an integration circuit or the like to perform image recognition processing, but normal image data can also be captured without performing convolution integration. The solid-state imaging element 200 according to the modified example of the first embodiment differs from the solid-state imaging element of the second embodiment in that a bypass switch for bypassing the integration circuit is added.

[0251] Fig.321 is a circuit diagram showing a configuration example of a column signal processing circuit 260 in a modification of the first embodiment of the present technology. The column signal processing circuit 260 according to the modification of the first embodiment is different from the column signal processing circuit 260 of the first embodiment in that it further includes bypass switches 262-1 to 262-3 and bypass switches 263-1 to 263-3. Moreover, in the modification of the second embodiment, an ADC is provided for each column.

[0252] The bypass switches 262 - 1 to 262 - 3 and the bypass switches 263 - 1 to 263 - 3 bypass the integration circuits 270 - 1 to 270 - 3 under the control of the timing control circuit 212 .

[0253] For example, an image recognition mode in which image recognition processing is performed or a normal mode in which image recognition processing is not performed is set in the solid-state imaging element 200. In the case where the normal mode is set, the bypass switch bypasses the integration circuit.

[0254] In addition, in the case of setting the normal mode, the switching circuit 261 connects the bypass switch of each column and the ADC of the column. For example, the bypass switch 263-1 and the ADC 280-1 are connected, and the bypass switch 263-2 and the ADC 280-2 are connected (not shown).

[0255] As described above, in the modification of the first embodiment of the present technology, since a bypass switch that bypasses the integration circuit is further provided, normal image data can be captured without performing convolution integration.

[0256] <2. Second Embodiment>

[0257] In the first embodiment described above, the size of the filter in the horizontal direction is fixed, but the present technology is not limited to this configuration. The solid-state imaging element 200 of the second embodiment is different from that of the first embodiment in that the size of the filter in the horizontal direction is variable.

[0258] Fig.33 280-1 according to the second embodiment of the present technology. The ADC 280-1 of the second embodiment is different from the ADC 280-1 of the first embodiment in that it includes connection switches 283-1 to 283-(K-1), where K is the maximum value of the size of the filter in the horizontal direction.

[0259] Furthermore, instead of the capacitance elements 282 - 1 to 282 - 3 , capacitance elements 282 - 1 to 282 -K are provided. One end of each capacitance element 282 - 1 to 282 -K is connected to a corresponding integration circuit via a switch circuit 261 , and the other end is commonly connected to an inverting input terminal (−) of a comparator 285 .

[0260] The connection switch 283-k (k is an integer from 1 to K-1) connects one end of the capacitance element 282-k on the comparator 285 side and one end of the capacitance element 282-(k+1) on the comparator 285 side according to the switching signal SW_k. For example, the switching signal SW_k is provided from the timing control circuit 212. Among the capacitance elements 282-1 to 282-K, two adjacent capacitance elements are examples of the first capacitance element and the second capacitance element in the claims.

[0261] By controlling the number of switches to be closed among the connection switches 283 - 1 to 283 -(K- 1 ), the size of the filter in the horizontal direction can be changed.

[0262] In addition, when the number of columns is M and the minimum size of the filter in the horizontal direction is R (R is an integer), M / RADC is set. For example, when the minimum size is "2", ADC is set every two columns.

[0263] Furthermore, the modified example of the first embodiment may be applied to the second embodiment.

[0264] As described above, according to the second embodiment of the present technology, since the connection switch 283 - k that connects one ends of two adjacent capacitive elements is provided, the size of the filter in the horizontal direction can be made variable.

[0265] <3. Third Embodiment>

[0266] In the first embodiment described above, the photoelectric conversion element 311 generates and accumulates charge, but the image quality in the dark may be insufficient. The solid-state imaging element 200 in the third embodiment is different from that in the first embodiment in that an embedded photodiode is used as the photoelectric conversion element 311.

[0267] Fig.34 is a diagram showing an example of the configuration of a photoelectric conversion element 311 according to a third embodiment of the present technology. An n-type region of a charge storage portion 502 as a photodiode (photoelectric conversion element 311) is embedded in a substrate 511. The interface between the silicon and silicon oxide films is covered with a high-concentration p-layer 503, and white spots and dark currents can be suppressed by separating it in a Sip-type region 503 having many crystal defects. However, when the P-type concentration of the peripheral portion 514 of the TG electrode 506 is too high, signal readout becomes difficult, so it is necessary to reduce the P-type concentration only in the region of 514. In this case, by providing a negative bias (e.g., -1.2V) when the TG electrode 506 is disconnected, the entire surface can be filled with holes, and the dark current can be reduced.

[0268] As shown in the figure, by using a photodiode in which a charge storage region is embedded in a semiconductor substrate (substrate 511), dark current can be suppressed and image quality in a dark place can be improved.

[0269] Note that in Non-Patent Document 1, since the photoelectric conversion current of the photodiode is used and the embedding structure shown in the figure cannot be adopted, it is difficult to improve the image quality in a dark place.

[0270] Furthermore, modifications of the first embodiment and the second embodiment may be applied to the third embodiment.

[0271] As described above, according to the third embodiment of the present technology, since a photodiode in which a charge storage region is embedded in a semiconductor substrate (substrate 511 ) is used, image quality in a dark place can be improved.

[0272] <4. Fourth Embodiment>

[0273] In the first embodiment described above, a system of a post-stage circuit is provided for each pixel, but in this configuration, the reading speed may be insufficient. The solid-state imaging element 200 in the fourth embodiment is different from the solid-state imaging element 200 in the first embodiment in that two systems of post-stage circuits are provided for each pixel.

[0274] Fig.35 350 - s is a circuit diagram showing a configuration example of a pixel 300 according to a fourth embodiment of the present technology. The pixel 300 of the fourth embodiment includes subsequent-stage circuits 350 - r and 350 - s instead of the subsequent-stage circuit 350 .

[0275] The selection transistor 331 opens and closes a path between the capacitance element 321 and the subsequent-stage circuit 350 - r , and the selection transistor 332 opens and closes a path between the capacitance element 322 and the subsequent-stage circuit 350 - s .

[0276] The post-stage circuit 350-r includes a post-stage amplifier transistor 351-r and a post-stage selection transistor 352-r, and the post-stage circuit 350-s includes a post-stage amplifier transistor 351-s and a post-stage selection transistor 352-s. In addition, two vertical signal lines are wired for each column. The post-stage circuit 350-r outputs a reset level to the vertical signal line 309-r, and the post-stage circuit 350-s outputs a signal level to the vertical signal line 309-s. The reset level and the signal level are output simultaneously.

[0277] Fig.362 is a circuit diagram showing a configuration example of a column signal processing circuit 260 according to a fourth embodiment of the present technology. The number of each integration circuit and ADC is twice the number of the first embodiment. For example, assuming that the size of the filter is 3 rows x 3 columns, six integration circuits (e.g., integration circuits 270-1 to 270-6) and two ADCs (e.g., ADC280-1 and 280-2) are set for every three columns. In addition, a CDS circuit 289 is further set for every three columns.

[0278] In addition, switch circuits 261-1 and 261-2 are provided instead of the switch circuit 261. The switch circuit 261-1 switches the connection destination of the vertical signal line, and the switch circuit 261-2 switches the connection destination of the integration circuit.

[0279] For example, assume that the switching circuit 261-2 connects the output destinations of the integrating circuits 270-1, 270-2, and 270-3 to the ADC 280-1, and connects the output destinations of the integrating circuits 270-4, 270-5, and 270-6 to the ADC 280-2. In addition, these ADCs do not perform CDS processing and perform only AD conversion.

[0280] The CDS circuit 289 subtracts the digital signal output from the ADC 280 - 2 from the digital signal output from the ADC 280 - 1 , and supplies the subtracted digital signal to the image recognition section 290 .

[0281] The switching circuit 261-1 switches the connection destination of the vertical signal line according to the sign of the filter coefficient of the pixel. For example, focusing on a certain row in the filter, it is assumed that the filter coefficient of the first column on the left of the row is -1, the filter coefficient of the second column is +1, and the filter coefficient of the third column is -1. In this case, the switch circuit 261-1 connects the vertical signal line (such as 309-r) that transmits the reset level of the first column to the integration circuit 270-1, and connects the vertical signal line (such as 309-s) that transmits the signal level of the column to the integration circuit 270-4.

[0282] Furthermore, the switch circuit 261-1 connects the vertical signal line transmitting the reset level of the second column to the integration circuit 270-5, and connects the vertical signal line transmitting the signal level of the column to the integration circuit 270-2. The switch circuit 261-1 connects the vertical signal line transmitting the reset level of the third column to the integration circuit 270-3, and connects the vertical signal line transmitting the signal level of the column to the integration circuit 270-6.

[0283] With the configuration shown in the drawing, AD conversion of the reset level and AD conversion of the signal level of each pixel can be performed in parallel. Therefore, the reading speed can be increased.

[0284] Furthermore, modifications of the first embodiment and the second and third embodiments can also be applied to the fourth embodiment.

[0285] As described above, according to the fourth embodiment of the present technology, the subsequent circuits 350 - r and 350 - s simultaneously output the reset level and the signal level, and the ADCs 280 - 1 and 280 - 2 simultaneously perform their AD conversions, making it possible to increase the reading speed.

[0286] <5. Fifth embodiment>

[0287] In the first embodiment described above, the precharge transistor 318 is provided in the pixel 300, but the transistor can be reduced. The solid-state imaging element 200 in the fifth embodiment is different from the solid-state imaging element 200 in the first embodiment in that the precharge transistor 318 is restored.

[0288] Fig.37 is a circuit diagram showing a configuration example of a pixel 300 according to a fifth embodiment of the present technology. The pixel 300 of the fifth embodiment is different from the pixel 300 of the first embodiment in that the precharge transistor 318 is reduced. The vertical scanning circuit 211 only needs to turn off the current source transistor 316 instead of the precharge transistor 318 by controlling the gate voltage to prevent charge loss.

[0289] Furthermore, the modified example of the first embodiment and the second to fourth embodiments can be applied to the fifth embodiment.

[0290] As described above, according to the fourth embodiment of the present technology, since the precharge transistor 318 is reduced, the circuit scale of the solid-state imaging element 200 can be reduced.

[0291] [First Modification]

[0292] In the fifth embodiment described above, the front-stage circuit 310 reads a signal while being connected to the front-stage node 320, but with this configuration, the noise from the front-stage node 320 cannot be cut off when reading. The pixel 300 according to the first variant of the fifth embodiment is different from the pixel of the first embodiment in that a transistor is inserted between the front-stage circuit 310 and the front-stage node 320.

[0293] Fig.38 : is a circuit diagram showing a configuration example of a pixel 300 in a first variant of the fifth embodiment of the present technology. The pixel 300 according to the first variant of the fifth embodiment is different from the pixel 300 of the fifth embodiment in that it further includes a front-stage reset transistor 323 and a front-stage selection transistor 324. In addition, the power supply voltage of the front-stage circuit 310 and the rear-stage circuit 350 of the first variant of the fifth embodiment is VDD1.

[0294] The previous stage reset transistor 323 initializes the level of the previous stage node 320 using the power supply voltage VDD2. It is desirable to set the power supply voltage VDD2 to a value satisfying the following formula.

[0295] VDD2 = VDD1 - Vgs...Equation 5

[0296] In the above formula, Vgs represents the gate-source voltage of the pre-stage amplifier transistor 315 .

[0297] By setting a value satisfying Formula 5, the potential fluctuation between the previous stage node 320 and the next stage node 340 in darkness can be reduced. Therefore, the photo response non-uniformity (PRNU) can be improved.

[0298] The previous stage selection transistor 324 opens and closes a path between the previous stage circuit 310 and the previous stage node 320 according to a previous stage selection signal sel from the vertical scanning circuit 211 .

[0299] Fig.39 1 is a timing diagram showing an example of global shutter operation in the first variant of the fifth embodiment of the present technology. The timing diagram of the first variant of the fifth embodiment is different from the timing diagram of the first embodiment in that the vertical scanning circuit 211 further provides a front-stage reset signal rsta and a front-stage selection signal sel. In the figure, rsta_[n] and sel_[n] indicate signals to pixels in the nth row.

[0300] The vertical scanning circuit 211 supplies the high-level previous-stage selection signal sel to all pixels in the period from the time T2 immediately before the end of exposure to the time T5. The previous-stage reset signal rsta is controlled to be low level.

[0301] Moreover, when reading each row, the previous stage selection signal sel is controlled to a low level. This control causes the previous stage selection transistor 324 to enter an off state to disconnect the previous stage node 320 from the previous stage circuit 310. Therefore, the noise from the previous stage node 320 can be blocked when reading.

[0302] As described above, according to the first modification of the fifth embodiment of the present technology, the previous stage selection transistor 324 turns to the off state at the time of reading, and the previous stage circuit 310 is disconnected from the previous stage node 320 , so that noise from the previous stage circuit 310 can be cut off.

[0303] [Second Modification]

[0304] In the fifth embodiment described above, the circuit in the solid-state imaging element 200 is arranged on a single semiconductor chip. However, in this configuration, when the pixel 300 is miniaturized, the element may not fit into the semiconductor chip. The solid-state imaging element 200 according to the second modification of the fifth embodiment is different from the solid-state imaging element 200 of the fifth embodiment in that the circuit in the solid-state imaging element 200 is dispersedly arranged on two semiconductor chips.

[0305] Fig.40 1 is a diagram showing an example of a stacked structure of a solid-state imaging element 200 in a second modification of the fifth embodiment of the present technology. The solid-state imaging element 200 according to the second modification of the fifth embodiment includes a circuit chip 202 and a pixel chip 201 stacked on the circuit chip 202. These chips are electrically connected by, for example, Cu-Cu bonding. Note that in addition to Cu-Cu bonding, vias or bumps can also be used for connection.

[0306] The upper pixel array section 221 is provided in the pixel chip 201. The lower pixel array section 222 and the column signal processing circuit 260 are provided in the circuit chip 202. For each pixel in the pixel array section 220, a part of the pixels are provided in the upper pixel array section 221, and the remaining pixels are provided in the lower pixel array section 222.

[0307] Furthermore, in the circuit chip 202, a vertical scanning circuit 211, a timing control circuit 212, a DAC 213, and a load MOS circuit block 250 are also arranged. These circuits are not shown in the figure.

[0308] Furthermore, the pixel chip 201 is manufactured, for example, by a pixel-dedicated process, and the circuit chip 202 is manufactured, for example, by a complementary MOS (CMOS) process.

[0309] Fig.41 is a circuit diagram showing a configuration example of a pixel 300 in a second variant of the fifth embodiment of the present technology. In the pixel 300, the pre-stage circuit 310 is arranged on the pixel chip 201, and other circuits and elements (such as capacitor elements 321 and 322) are arranged on the circuit chip 202. Note that the current source transistor 316 may be further provided in the circuit chip 202. As shown in the figure, by dispersively arranging the elements in the pixel 300 on the stacked pixel chip 201 and the circuit chip 202, the area of ​​the pixel can be reduced, and the miniaturization of the pixel is facilitated.

[0310] As described above, according to the second modification example of the fifth embodiment of the present technology, since the circuits and elements in the pixel 300 are dispersedly arranged on two semiconductor chips, miniaturization of the pixel is facilitated.

[0311] [Third Modification]

[0312] In the second modification of the fifth embodiment described above, a part of the pixel 300 and the peripheral circuits (such as the column signal processing circuit 260) are provided in the lower circuit chip 202. However, with this configuration, the arrangement area of ​​the circuits and elements on the circuit chip 202 side is larger than the arrangement area of ​​the pixel chip 201 by the peripheral circuits, and there is a possibility that an unnecessary space without the circuits and elements is generated in the pixel chip 201. The solid-state imaging element 200 according to the third modification of the fifth embodiment is different from the second modification of the fifth embodiment in that the circuits in the solid-state imaging element 200 are dispersedly arranged on three semiconductor chips.

[0313] Fig.42 1 is a diagram showing an example of a stacked structure of a solid-state imaging element 200 in a third modification of the fifth embodiment of the present technology. The solid-state imaging element 200 according to the third modification of the fifth embodiment includes an upper pixel chip 203, a lower pixel chip 204, and a circuit chip 202. These chips are stacked and electrically connected by, for example, Cu-Cu bonding. Note that in addition to Cu-Cu bonding, vias or bumps can also be used for connection.

[0314] The upper pixel array section 221 is provided in the upper pixel chip 203. The lower pixel array section 222 is provided in the lower pixel chip 204. For each pixel in the pixel array section 220, a part of the pixels are provided in the upper pixel array section 221, and the remaining pixels are provided in the lower pixel array section 222.

[0315] Furthermore, in the circuit chip 202, a column signal processing circuit 260, a vertical scanning circuit 211, a timing control circuit 212, a DAC 213, and a load MOS circuit block 250 are arranged. Circuits other than the column signal processing circuit 260 are not shown in the drawing.

[0316] Compared to a two-layer configuration, adopting the three-layer configuration shown in the drawings allows for reduction of unnecessary space and allows for further pixel miniaturization. In addition, the lower pixel chip 204 as the second layer can be manufactured by a dedicated process for capacitors and switches.

[0317] As described above, in the third modification of the fifth embodiment of the present technology, since the circuits in the solid-state imaging element 200 are dispersedly arranged on three semiconductor chips, the pixels can be further miniaturized compared to the case where the circuits are dispersedly arranged on two semiconductor chips.

[0318] <6. Sixth Implementation>

[0319] In the fifth embodiment described above, the reset level is sampled and held during the exposure period, but in this configuration, the exposure period cannot be made shorter than the sampling and holding period of the reset level. The solid-state imaging element 200 of the sixth embodiment is different from the solid-state imaging element of the fifth embodiment in that the exposure period is further shortened by adding a transistor that releases charge from the photoelectric conversion element.

[0320] Fig.43 2 is a circuit diagram showing a configuration example of a pixel 300 according to a sixth embodiment of the present technology. The pixel 300 of the second embodiment is different from the pixel 300 of the fifth embodiment in that a discharge transistor 317 is further provided in the previous stage circuit 310 .

[0321] The discharge transistor 317 functions as an overflow drain that discharges electric charge from the photoelectric conversion element 311 in accordance with a discharge signal g from the vertical scanning circuit 211. As the discharge transistor 317, for example, an nMOS transistor is used.

[0322] In a configuration in which the discharge transistor 317 is not provided as in the fifth embodiment, blooming may occur when, for all pixels, charge is transferred from the photoelectric conversion element 311 to the FD 314. Then, at the time of FD reset, the potential of the FD 314 and the potential of the previous stage node 320 drop. In response to the potential drop, charging current and discharging current of the capacitance elements 321 and 322 continue to occur, and the IR drop in the power supply or ground changes from a stable state without blooming.

[0323] On the other hand, when the signal levels of all pixels are sampled and held, after the transfer of the signal charge, the photoelectric conversion element 311 has no charge, so that blooming does not occur, and the IR drop in the power supply or ground enters a stable state without blooming. Due to the difference in IR drop when sampling and holding the reset level and the signal level, tail noise occurs.

[0324] On the other hand, in the sixth embodiment in which the discharge transistor 317 is provided, the charge in the photoelectric conversion element 311 is discharged toward the overflow drain side. Therefore, the IR drop and the signal level when sampling and holding the reset level become almost the same, so that the tail noise can be suppressed.

[0325] Fig.44 1 is a timing diagram showing an example of global shutter operation according to the sixth embodiment of the present technology. At time T0 before starting exposure, the vertical scanning circuit 211 provides a high-level FD reset signal rst to all pixels during the pulse, and sets the discharge signal g to a high level for all pixels. Therefore, PD reset and FD reset are performed on all pixels. In addition, the reset level is sampled and held. Here, g_[n] in the figure represents a signal to a pixel in the nth row of N rows.

[0326] Then, at the time T1 as the start of exposure, the vertical scanning circuit 211 returns the discharge signal g to the low level for all pixels. Then, the vertical scanning circuit 211 supplies the high-level transfer signal trg to all pixels in the time period from the time T2 immediately before the end of exposure to the time T3 as the end of exposure. As a result, the signal level is sampled and held.

[0327] In a configuration in which the discharge transistor 317 is not provided as in the fifth embodiment, both the transfer transistor 312 and the FD reset transistor 313 need to be turned on at the start of exposure (i.e., at the time of PD reset). Under this control, the FD 314 also needs to be reset at the time of PD reset. Therefore, it is necessary to perform FD reset again within the exposure period to sample and hold the reset level so that the exposure period cannot be made shorter than the sampling and holding period of the reset level. When the reset levels of all pixels are sampled and held, a certain waiting time is required until the voltage and current are stabilized, and, for example, a sampling and holding period of several microseconds (μs) to several tens of microseconds (μs) is required.

[0328] On the other hand, in the sixth embodiment in which the discharge transistor 317 is provided, the PD reset and the FD reset can be performed separately. Therefore, as shown in the figure, the reset level can be sampled and held by performing the FD reset before the cancellation of the PD reset (the start of exposure). Therefore, the exposure period can be made shorter than the sampling and holding period of the reset level.

[0329] Furthermore, the first to third modified examples of the fifth embodiment can also be applied to the sixth embodiment.

[0330] As described above, according to the sixth embodiment of the present technology, since the discharge transistor 317 for releasing charge from the photoelectric conversion element 311 is provided, the reset level can be sampled and held by performing FD reset before starting exposure. Therefore, the exposure period can be made shorter than the sampling and holding period of the reset level.

[0331] <7. Seventh Implementation Method>

[0332] In the fifth embodiment described above, the FD 314 is initialized by the power supply voltage VDD. However, in this configuration, there is a possibility that the photoresponse non-uniformity (PRNU) deteriorates due to variations in the capacitance elements 321 and 322 and the parasitic capacitance. The solid-state imaging element 200 of the seventh embodiment is different from the solid-state imaging element of the fifth embodiment in that the PRNU is improved by reducing the power supply of the FD reset transistor 313 at the time of reading.

[0333] Fig.452 is a circuit diagram showing a configuration example of a pixel 300 according to a seventh embodiment of the present technology. The pixel 300 of this seventh embodiment is different from the pixel 300 of the first embodiment in that the power supply of the FD reset transistor 313 is separated from the power supply voltage VDD of the pixel 300 .

[0334] The FD reset transistor 313 of the seventh embodiment has a drain connected to a reset power supply voltage VRST. The reset power supply voltage VRST is controlled by, for example, the timing control circuit 212.

[0335] Here, degradation of PRNU in the pixel 300 according to the fifth embodiment will be considered. In the first embodiment, at time T0 immediately before the exposure start time, the potential of the FD 314 decreases due to reset feedthrough of the FD reset transistor 313. This change amount is set to Vft.

[0336] In the first embodiment, since the power supply voltage of the FD reset transistor 313 is VDD, the potential of the FD 314 changes from VDD to VDD-Vft at time T0. In addition, the potential of the previous stage node 320 when exposed is VDD-Vft-Vgs.

[0337] In addition, in the first embodiment, the FD reset transistor 313 is turned to the on state at the time of reading, and the FD 314 is fixed to the power supply voltage VDD. Due to the change Vft of the FD 314, the potential of the previous node 320 and the potential of the next node 340 are higher than approximately Vft at the time of reading. However, due to the change in the capacitance value or parasitic capacitance of the capacitance elements 321 and 322, the offset voltage amount changes for each pixel, which causes PRNU degradation.

[0338] The shift amount of the subsequent node 340 when the previous node 320 is shifted by Vft is expressed by, for example, the following formula.

[0339] {(Cs+δCs) / (Cs+δCs+Cp)}*Vft...Formula 6

[0340] In the above formula, Cs is the capacitance value of the capacitance element 322 on the signal level side, and δCs is the change in Cs. Cp is the capacitance value of the parasitic capacitance of the subsequent-stage node 340.

[0341] Formula 6 can be approximated by the following formula.

[0342] {1-(δCs / Cs)*(Cp / Cs)}*Vft...Formula 7

[0343] According to Formula 7, the change in the subsequent node 340 can be expressed by the following formula.

[0344] {(δCs / Cs)*(Cp / Cs)}*Vft...Formula 8

[0345] When (δCs / Cs) is set to 10 -2 , (Cp / Cs) is set to 10 -1 And when Vft is set to 400 millivolts (mV), according to Formula 8, PRNU is 400 μVrms, which is a relatively large value.

[0346] Specifically, in order to reduce the kTC noise when sampling and holding the input conversion capacitance, it is necessary to increase the charge-to-voltage conversion efficiency of FD 314. In order to increase the charge-to-voltage conversion efficiency, it is necessary to reduce the capacitance of FD 314, but the smaller the capacitance of FD 314, the greater the change Vft (which can be several hundred millivolts (mV)). In this case, the impact of PRNU can be at a non-negligible level of Formula 8.

[0347] Fig.46 : is a timing chart showing an example of voltage control according to the seventh embodiment of the present technology.

[0348] The timing control circuit 212 performs control so that the reset power supply voltage VRST during the row-by-row reading after the time T9 is different from the reset power supply voltage VRST during the exposure period.

[0349] For example, for the exposure period, the timing control circuit 212 sets the reset power supply voltage VRST to be the same as the power supply voltage VDD. On the other hand, during reading, the timing control circuit 212 reduces the reset power supply voltage VRST to VDD-Vft. That is, during reading, the timing control circuit 212 reduces the reset power supply voltage VRST by an amount substantially equal to the variation Vft caused by reset feedthrough. This control allows the reset level of the FD 314 at the time of exposure and the reset level at the time of reading to be the same as each other.

[0350] As shown in the figure, controlling the reset power supply voltage VRST allows reduction of voltage variation of the FD 314 and the previous-stage node 320. Therefore, variation of the capacitance elements 321 and 322 and PRNU degradation due to parasitic capacitance can be suppressed.

[0351] In addition, Modifications 1-3 of Embodiment 5 and Embodiment 6 can also be applied to Embodiment 7.

[0352] As described above, according to the seventh embodiment of the present technology, since the timing control circuit 212 reduces the reset power supply voltage VRST by the change Vft caused by the reset feedthrough at the time of reading, the reset level at the time of exposure and the reset level at the time of reading can be made the same as each other. Therefore, it is possible to suppress the degradation of the photoresponse non-uniformity (PRNU).

[0353] <8. Eighth Implementation Method>

[0354] In the fifth embodiment described above, the vertical scanning circuit 211 performs control (i.e., global shutter operation) to expose all rows (all pixels) at the same time. However, in the case where simultaneity of exposure is not required but low noise is required, such as when testing or analyzing, it is desirable to perform a rolling shutter operation. The solid-state imaging element 200 of the eighth embodiment is different from the solid-state imaging element 200 of the fifth embodiment in that a rolling shutter operation is performed when testing, etc.

[0355] Fig.47 2 is a timing chart showing an example of the rolling shutter operation according to the eighth embodiment of the present technology. The vertical scanning circuit 211 performs control to sequentially select a plurality of rows and start exposure. This figure shows exposure control of the n-th row.

[0356] In the time period from time T0 to time T2, the vertical scanning circuit 211 supplies the high-level post-stage selection signal selb, the high-level selection signal Φr, and the high-level selection signal Φs to the n-th row. In addition, at time T0 as the start of exposure, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the high-level post-stage reset signal rstb to the n-th row within the pulse period. At time T1 as the end of exposure, the vertical scanning circuit 211 supplies the transfer signal trg to the n-th row. The rolling shutter operation in the figure allows the solid-state imaging element 200 to produce low-noise image data.

[0357] Note that the solid-state imaging element 200 according to the eighth embodiment performs the global shutter operation similarly to the fifth embodiment except during testing.

[0358] Furthermore, the first to third modified examples of the fifth embodiment, and the sixth and seventh embodiments can also be applied to the eighth embodiment.

[0359] As described above, according to the eighth embodiment of the present technology, the vertical scanning circuit 211 performs control (ie, rolling shutter operation) to sequentially select a plurality of rows and start exposure, and therefore, low-noise image data can be generated.

[0360] <9. Ninth Implementation Method>

[0361] In the fifth embodiment described above, the source of the front-stage source follower (the front-stage amplifier transistor 315 and the current source transistor 316) is connected to the power supply voltage VDD, and reading is performed in units of rows when the source follower is turned on. However, there is a possibility that the driving method causes the circuit noise of the front-stage source follower to propagate to the subsequent stage when reading row by row, and the random noise increases accordingly. The solid-state imaging element 200 of the ninth embodiment differs from the solid-state imaging element of the fifth embodiment in that noise is reduced by disconnecting the front-stage source follower during reading.

[0362] Fig.48 1 is a block diagram showing a configuration example of a solid-state imaging element 200 according to a ninth embodiment of the present technology. The solid-state imaging element 200 of the ninth embodiment differs from the solid-state imaging element of the fifth embodiment in that it further includes a regulator 420 and a switch section 440. In addition, in the pixel array section 220 of the seventh embodiment, a plurality of effective pixels 301 and a predetermined number of dummy pixels 430 are arranged. The dummy pixels 430 are arranged around the area where the effective pixels 301 are arranged.

[0363] In addition, the power supply voltage VDD is supplied to each dummy pixel 430, and the power supply voltage VDD and the source voltage Vs are supplied to each effective pixel 301. The signal line for supplying the power supply voltage VDD to the effective pixel 301 is not shown in the figure. In addition, the power supply voltage VDD is supplied from the pad 410 located outside the solid-state imaging element 200.

[0364] The regulator 420 generates a constant generated voltage Vgen based on the input voltage Vi from the dummy pixel 430, and supplies the generated voltage Vgen to the switch section 440. The switch section 440 selects the power supply voltage VDD from the pad 410 or the generated voltage Vgen from the regulator 420, and supplies the selected voltage as the source voltage Vs to each of the columns of the effective pixels 301.

[0365] Fig.49 1 is a circuit diagram showing a configuration example of a dummy pixel 430, a regulator 420, and a switch section 440 according to a ninth embodiment of the present technology. In the drawings, a represents a circuit diagram of a dummy pixel 430 and a regulator 420, and b in the drawings represents a circuit diagram of a switch section 440.

[0366] As shown in a of the accompanying drawings, the dummy pixel 430 includes a reset transistor 431, an FD 432, an amplifier transistor 433, and a current source transistor 434. The reset transistor 431 initializes the FD 432 according to the reset signal RST from the vertical scanning circuit 211. The FD 432 accumulates charge and generates a voltage according to the amount of charge. The amplifier transistor 433 amplifies the level of the voltage of the FD 432 and provides the amplified voltage as the input voltage Vi to the regulator 420.

[0367] In addition, the FD reset transistor 431 and the amplifier transistor 433 have respective sources connected to the power supply voltage VDD. The current source transistor 434 is connected to the drain of the amplifier transistor 433. The current source transistor 434 supplies the current id1 under the control of the vertical scanning circuit 211.

[0368] The regulator 420 includes a low-pass filter 421, a buffer amplifier 422, and a capacitor 423. The low-pass filter 421 passes a component in a low frequency band below a predetermined frequency in a signal of an input voltage Vi as an output voltage Vj.

[0369] The output voltage Vj is input to the non-inverting input terminal (+) of the buffer amplifier 422. The inverting input terminal (-) of the buffer amplifier 422 is connected to its output terminal. The capacitor 423 maintains the voltage of the output terminal of the buffer amplifier 422 at Vgen. This Vgen is provided to the switch section 440.

[0370] As shown in b in the figure, the switch section 440 includes an inverter 441 and a plurality of switching circuits 442. Each of the switch circuits 442 is provided for a corresponding one of the columns of the effective pixels 301.

[0371] The inverter 441 inverts the switching signal SW sent from the timing control circuit 212. The inverter 441 supplies an inverted signal to each switch circuit 442.

[0372] The switch circuits 442 each select the power supply voltage VDD or the generated voltage Vgen and provide the selected voltage as the source voltage Vs to the corresponding column in the pixel array section 220. The switch circuits 442 each include switches 443 and 444. The switch 443 opens and closes the path between the node of the power supply voltage VDD and the corresponding column according to the switching signal SW. The switch 444 opens and closes the path between the node of the generated voltage Vgen and the corresponding column according to the inverted signal of the switch signal SW.

[0373] Fig.50is a timing diagram showing an example of the operation of the dummy pixel 430 and the regulator 420 according to the ninth embodiment of the present technology. At time T10 immediately before reading a specific row, the vertical scanning circuit 211 supplies a high-level reset signal RST (here, the power supply voltage VDD) to each dummy pixel 430. The potential Vfd of the FD 432 in the dummy pixel 430 is initialized to the power supply voltage VDD. Then, when the reset signal RST becomes low level, the reset feedthrough causes a change to VDD-Vft.

[0374] In addition, the input voltage Vi is reduced to VDD-Vgs-Vsig after reset. By passing through the low-pass filter 421, Vj and Vsen become almost constant voltages.

[0375] After the timing T20 immediately before the next row is read, similar control is performed for each row, and the constant generated voltage Vgen is supplied.

[0376] Fig.51 is a circuit diagram showing a configuration example of each effective pixel 301 according to a ninth embodiment of the present technology. The circuit configuration of the effective pixel 301 is similar to that of the pixel 300 of the fifth embodiment, except that the source voltage Vs from the switch section 440 is supplied to the source of the pre-amplification transistor 315.

[0377] Fig.52 1 is a timing diagram showing an example of global shutter operation according to the ninth embodiment of the present technology. In the ninth embodiment, when exposure is performed simultaneously in all pixels, the switch section 440 selects the power supply voltage VDD and provides the power supply voltage VDD as the source voltage Vs. In addition, the voltage of the previous node decreases from VDD-Vgs-Vth to VDD-Vgs-Vsig at time T4. Here, Vth represents the threshold voltage of the transfer transistor 312.

[0378] In the ninth embodiment, at the time of reading, the switch unit 440 selects the generated voltage Vgen and supplies the generated voltage Vgen as the source voltage Vs. The generated voltage Vgen is adjusted to VDD-Vgs-Vft.

[0379] Fig.53 3 is a diagram for explaining the effect in the ninth embodiment of the present technology. In the first embodiment, the source follower (pre-stage amplifier transistor 315 and current source transistor 316) of the pixel 300 to be read is turned on during the row-by-row reading. However, there is a possibility that the driving method causes the circuit noise of the pre-stage source follower to propagate to the subsequent stage (capacitive element, and the post-stage source follower and ADC), and the readout noise increases accordingly.

[0380] For example, in the fifth embodiment, as shown in the figure, the kTC noise generated in the pixel during the global shutter operation is 450 (μVrms). In addition, the noise generated in the front-stage source follower (the front-stage amplifier transistor 315 and the current source transistor 316) during the row-by-row reading is 380 (μVrms). The noise generated after the source follower in the subsequent stage is 160 (μVrms). Therefore, the total noise is 610 (μVrms). As described above, in the fifth embodiment, the contribution of the noise of the front-stage source follower to the total value of the noise becomes relatively large.

[0381] In order to reduce the noise of the front-stage source follower, in the ninth embodiment, as described above, an adjustable voltage (Vs) is supplied to the source of the front-stage source follower. During the global shutter (exposure) operation, the switch section 440 selects the power supply voltage VDD and provides the selected power supply voltage as the source voltage Vs. Then, after the exposure ends, the switch section 440 switches the source voltage Vs to VDD-Vgs-Vft. In addition, the timing control circuit 212 turns on the front-stage current source transistor 316 during the global shutter (exposure) operation, and turns off the front-stage current source transistor 316 after the exposure ends.

[0382] By the above control, as described above, the potential of the front-stage node is the same when the global shutter is operated and when each row is read, and PRNU can be improved. In addition, since the front-stage source follower is turned off when each row is read, the circuit noise of the source follower does not occur and becomes 0 (μVrms). Note that in the front-stage source follower, the front-stage amplifier transistor 315 is in the on state.

[0383] As described above, according to the ninth embodiment of the present technology, since the preceding-stage source follower is turned off at the time of reading, noise generated in the source follower can be reduced.

[0384] <10. Application examples of mobile objects>

[0385] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology of the present disclosure can be implemented in the form of a device installed on any kind of mobile body (such as, a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot).

[0386] Fig.54 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving body control system to which the technology according to the present disclosure can be applied.

[0387] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. Fig.54In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. In addition, as a functional structure of the integrated control unit 12050, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are exemplified.

[0388] The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 serves as a control device for a drive force generating device (such as an internal combustion engine, a drive motor, etc.) for generating a drive force of the vehicle, a drive force transmitting mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, and the like.

[0389] The body system control unit 12020 controls the operation of various devices provided on the vehicle body according to various programs. For example, the body system control unit 12020 is used as a control device for a keyless entry system, a smart key system, a power window device, or various lights such as a headlight, a backup light, a brake light, a turn signal, a fog light, etc. In this case, a radio wave transmitted from a mobile device as a substitute for a key or a signal of various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals, and controls the door lock device, the power window device, the lights, etc. of the vehicle.

[0390] The vehicle exterior information detection unit 12030 detects information outside the vehicle including the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to an imaging unit 12031. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receive the captured image. In addition, the vehicle exterior information detection unit 12030 may also detect objects such as people, vehicles, obstacles, signs, and text on the road surface, or detect their distances, based on the received image.

[0391] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 may output the electrical signal as an image, or may output the electrical signal as information about the measured distance. In addition, the light received by the imaging unit 12031 may be visible light, or may be invisible light such as infrared rays.

[0392] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that photographs the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's fatigue or the driver's concentration, or can determine whether the driver is dozing off.

[0393] The microcomputer 12051 can calculate the control target value of the driving force generation device, the steering mechanism, or the braking device based on the information about the inside or outside of the vehicle obtained by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of the advanced driver assistance system (ADAS), which includes collision prevention or shock absorption for the vehicle, follow-up driving based on the following distance, maintaining the vehicle speed for driving, warning of vehicle collision, warning of vehicle deviation from the lane, etc.

[0394] In addition, the microcomputer 12051 can perform collaborative control for automatic driving by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about outside or inside the vehicle obtained by the outside information detection unit 12030 or the inside information detection unit 12040, thereby enabling the vehicle to travel automatically without relying on the driver's operation, etc.

[0395] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle obtained by the vehicle outside information detection unit 12030. For example, the microcomputer 12051 can perform cooperative control aimed at preventing glare by controlling the headlights to change from high beam to low beam according to the position of the front vehicle or the oncoming vehicle detected by the vehicle outside information detection unit 12030.

[0396] The sound / image output unit 12052 sends an output signal of at least one of sound and image to an output device, which can visually or auditorily notify the occupants of the vehicle or the outside of the vehicle of information. Fig.54 In the example of FIG. 1 , an audio speaker 12061, a display portion 12062, and a dashboard 12063 are shown as output devices. For example, the display portion 12062 may include at least one of an on-board display and a head-up display.

[0397] Fig.55 12031 is a diagram illustrating an example of the installation position of the imaging unit 12031.

[0398] exist Fig.55In the imaging unit 12031, imaging units 12101, 12102, 12103, 12104 and 12105 are included.

[0399] The imaging units 12101, 12102, 12103, 12104, 12105 are provided, for example, at locations such as the front nose, side mirrors, rear bumper, rear door, and upper portion of the windshield in the interior of the vehicle 12100. The imaging unit 12101 provided at the front nose in the interior of the vehicle and the imaging unit 12105 provided at the upper portion of the windshield mainly obtain images in front of the vehicle 12100. The imaging units 12102 and 12103 provided to the side mirrors mainly obtain images of the sides of the vehicle 12100. The imaging unit 12104 provided to the rear bumper or rear door mainly obtains images of the rear portion of the vehicle 12100. The imaging unit 12105 provided at the upper portion of the windshield in the interior of the vehicle is mainly used to detect vehicles ahead, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0400] Please note that Fig.55 An example of the imaging range of the imaging units 12101 to 12104 is shown. The imaging range 12111 represents the imaging range of the imaging unit 12101 set to the front nose. The imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging units 12102 and 12103 set to the side mirrors. The imaging range 12114 represents the imaging range of the imaging unit 12104 set to the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 12100 viewed from above is obtained by superimposing the image data imaged by the imaging units 12101 to 12104.

[0401] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0402] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging range 12111 to 12114 and the time change of the distance (relative speed relative to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting objects that exist on the driving path of the vehicle 12100 and are traveling in the same direction as the vehicle 12100 at a prescribed speed (e.g., equal to or greater than 0 km / hour). In addition, the microcomputer 12051 can pre-set the following distance to keep ahead of the leading vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. As a result, coordinated control for automatic driving that allows the vehicle to travel automatically without relying on the driver's operation, etc., can be performed.

[0403] For example, the microcomputer 12051 can classify the three-dimensional object data related to the three-dimensional object into three-dimensional object data of a two-wheeled vehicle, a standard vehicle, a large vehicle, a pedestrian, a utility pole, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that can be visually recognized by the driver of the vehicle 12100 and obstacles that are difficult for the driver of the vehicle 12100 to visually recognize. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than the set value and therefore there is a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or evasive steering via the drive system control unit 12010. The microcomputer 12051 can thereby assist driving to avoid collisions.

[0404] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 may, for example, identify a pedestrian by determining whether a pedestrian exists in the captured images of the imaging units 12101 to 12104. This identification of a pedestrian is performed, for example, by a process of extracting feature points in the imaged images of the imaging units 12101 to 12104 as infrared cameras and by performing pattern matching processing on a series of feature points representing the outline of an object to determine whether it is a pedestrian. When the microcomputer 12051 determines that a pedestrian exists in the imaged images of the imaging units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 so that a square outline for emphasis is displayed as superimposed on the identified pedestrian. The sound / image output unit 12052 may also control the display unit 12062 so that an icon representing a pedestrian or the like is displayed at a desired position.

[0405] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 in the above configuration. Specifically, for example, Figure 1 The imaging device 100 in the embodiment can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, the accuracy of the image recognition process can be improved, so that the security of the system can be improved.

[0406] It should be noted that the above-mentioned embodiments show examples for embodying the present technology, and the matters in the embodiments and the matters specifying the present invention in the claims have a corresponding relationship. Similarly, the matters specifying the present invention in the claims have a corresponding relationship with the matters with the same names in the embodiments of the present technology. However, the present technology is not limited to the embodiments, and can be embodied by applying various modifications to the embodiments without departing from the scope of the present technology.

[0407] It should be noted that the advantageous effects described in this specification are merely examples and are not limiting, and other advantageous effects may be provided.

[0408] It should be noted that the present technology may also have the following configurations.

[0409] (1) A solid-state imaging element comprising:

[0410] a pixel array section in which a plurality of pixels each generating and holding an analog pixel signal are arranged in a two-dimensional grid pattern;

[0411] a plurality of integrating circuits that temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among a plurality of pixels and output integrated signals; and

[0412] The analog-to-digital converter performs analog addition on the integrated signals of each of the plurality of integrating circuits to convert the analog added integrated signals into digital signals.

[0413] (2) The solid-state imaging element according to (1), wherein

[0414] The integration time for each pixel of the integration circuit includes a time according to the absolute value of the filter coefficient corresponding to the pixel.

[0415] (3) The solid-state imaging element according to (2), wherein

[0416] The pixel signal includes a predetermined reset level and a signal level according to the exposure amount, and when the sign of the filter coefficient is positive, the pixel outputs the reset level and the signal level in a predetermined order, and when the sign of the filter coefficient is negative, the pixel outputs the reset level and the signal level in an order opposite to the predetermined order.

[0417] (4) The solid-state imaging element according to any one of (1) to (3) above, wherein

[0418] The plurality of integrating circuits include first and second integrating circuits, and

[0419] Analog-to-digital converters include:

[0420] a first capacitive element including a terminal connected to the first integrator circuit;

[0421] a second capacitive element including a terminal connected to a second integrating circuit;

[0422] a comparator including two input terminals, one of the two input terminals being connected to the other end of each of the first capacitive element and the second capacitive element; and

[0423] The counter counts the count value over a period of time until the output signal of the comparator is inverted.

[0424] (5) The solid-state imaging element according to (4), wherein

[0425] The analog-to-digital converter further includes a connection switch that connects the other end of the first capacitance element and the other end of the second capacitance element according to a predetermined switching signal.

[0426] (6) The solid-state imaging element according to any one of (1) to (5) above, wherein

[0427] Each pixel includes a photodiode in which a charge storage region is embedded in a predetermined semiconductor substrate.

[0428] (7) The solid-state imaging element according to any one of (1) to (6) above, wherein

[0429] The pixel signal includes a predetermined reset level and a signal level according to the exposure amount, and

[0430] Each pixel consists of:

[0431] first and second capacitive elements;

[0432] A front stage circuit, which sequentially generates a reset level and a signal level, and causes the first capacitance element and the second capacitance element to respectively maintain the reset level and the signal level;

[0433] a first subsequent stage circuit that reads and outputs a reset level held in a first capacitance element; and

[0434] The second subsequent stage circuit reads and outputs the signal level held in the second capacitance element.

[0435] (8) The solid-state imaging element according to any one of (1) to (6) above, wherein

[0436] The pixel signal includes a predetermined reset level and a signal level according to the exposure amount, and

[0437] Each pixel consists of:

[0438] first and second capacitive elements;

[0439] A front stage circuit, which sequentially generates a reset level and a signal level, and causes the first capacitance element and the second capacitance element to respectively maintain the reset level and the signal level;

[0440] a selection circuit that sequentially performs control to connect one of the first capacitance element and the second capacitance element to a predetermined subsequent-stage node, controls to disconnect both the first capacitance element and the second capacitance element from the subsequent-stage node, and controls to connect the other of the first capacitance element and the second capacitance element to the subsequent-stage node;

[0441] a subsequent-stage reset transistor that initializes a level of the subsequent-stage node when both the first capacitance element and the second capacitance element are disconnected from the subsequent-stage node; and

[0442] The subsequent stage circuit sequentially reads the reset level and the signal level from the first capacitance element and the second capacitance element via the subsequent stage node and outputs the reset level and the signal level.

[0443] (9) An imaging device comprising:

[0444] a pixel array section in which a plurality of pixels each generating and holding an analog pixel signal are arranged in a two-dimensional grid pattern;

[0445] a plurality of integrating circuits that temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among a plurality of pixels and output integrated signals;

[0446] an analog-to-digital converter that adds the integrated signal of each of the plurality of integrating circuits to convert the added integrated signal into a digital signal; and

[0447] The image recognition unit performs predetermined image recognition processing using the digital signal.

[0448] (10) A method for controlling a solid-state imaging element, the method comprising:

[0449] a sample and hold process in which each of a plurality of pixels arranged in a two-dimensional grid pattern generates and holds an analog pixel signal;

[0450] an integration process in which a plurality of integration circuits temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among a plurality of pixels and output integrated signals; and

[0451] An analog-to-digital conversion process of analog-adding the integrated signal of each of a plurality of integrating circuits to convert the analog-added integrated signal into a digital signal.

[0452] Reference Symbols List

[0453] 100 Imaging Device

[0454] 110 Imaging lens

[0455] 120 Records Department

[0456] 130 Imaging Control Unit

[0457] 200 Solid-state imaging element

[0458] 201 pixel chip

[0459] 202 Circuit Chip

[0460] 203 pixel chip

[0461] 204 lower pixel chip

[0462] 211 vertical scanning circuit

[0463] 212 Timing Control Circuit

[0464] 213DAC

[0465] 220 Pixel Array Section

[0466] 221 upper pixel array unit

[0467] 222 Lower pixel array part

[0468] 250 load MOS circuit block

[0469] 251 Load MOS Transistor

[0470] 260 signal processing circuits

[0471] 261, 261-1, 261-2 switch circuit

[0472] 262-1 to 262-K, 263-1 to 263-K Bypass Switch

[0473] 270-1 to 270-K Integrator Circuits

[0474] 271 Input switch

[0475] 272 Resistor

[0476] 273 Operational Amplifier

[0477] 274, 281, 282-1 to 282-K, 423 capacitor elements

[0478] 275, 286 automatic zero switch

[0479] 283-1 to 283-(K-1) connection switch

[0480] 280-1, 280-2ADC

[0481] 285 Comparator

[0482] 287 Counter

[0483] 289 CDS Circuit

[0484] 290 Image Recognition Department

[0485] 291 CNN Processing Unit

[0486] 292 Data Comparison Department

[0487] 300 pixels

[0488] 301 effective pixels

[0489] 310 Preamplifier Circuit

[0490] 311 Photoelectric conversion element

[0491] 312 Transfer transistor

[0492] 313 FD reset transistor

[0493] 314, 432FD

[0494] 315 Pre-amplifier transistor

[0495] 316, 434 Current source transistor

[0496] 317 Discharge transistor

[0497] 318 Pre-charge transistor

[0498] 321, 322 Capacitor elements

[0499] 323 Pre-stage reset transistor

[0500] 324 Pre-select transistor

[0501] 330 Selection Circuit

[0502] 331, 332 Select transistor

[0503] 341 Rear stage reset transistor

[0504] 350, 350-r, 350-s rear stage circuit

[0505] 351, 351-r, 351-s post-amplifier transistor

[0506] 352, 352-r, 352-s rear stage selection transistor

[0507] 420 Regulator

[0508] 421 Low-pass filter

[0509] 422 Buffer Amplifier

[0510] 430 Fake Pixels

[0511] 431 Reset transistor

[0512] 433 Amplifier Transistor

[0513] 440 Switch Department

[0514] 441 Inverter

[0515] 442 Switching Circuit

[0516] 443, 444 switch

[0517] 501, 511 substrate

[0518] 502 Charge storage unit

[0519] 503 p layer

[0520] 504 Silicon oxide film

[0521] 505n+ Zone

[0522] 514 Floating Electrode

[0523] 12031 Imaging Department.

Claims

1. A solid-state imaging element, comprising: a pixel array section in which a plurality of pixels are arranged in a two-dimensional grid pattern, each pixel generating and holding an analog pixel signal; a plurality of integration circuits that temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among the plurality of pixels and output integrated signals; as well as The analog-to-digital converter performs analog addition on the integrated signal of each of the plurality of integrating circuits to convert the analog added integrated signal into a digital signal.

2. The solid-state imaging element according to claim 1, wherein The integration time of each pixel of the integration circuit includes a time according to an absolute value of a filter coefficient corresponding to the pixel.

3. The solid-state imaging element according to claim 2, wherein The pixel signal includes a predetermined reset level and a signal level according to an exposure amount, and The pixel outputs the reset level and the signal level in a predetermined order when the sign of the filter coefficient is positive, and outputs the reset level and the signal level in an order opposite to the predetermined order when the sign of the filter coefficient is negative.

4. The solid-state imaging element according to claim 1, wherein The plurality of integration circuits include a first integration circuit and a second integration circuit, and the analog-to-digital converter includes: a first capacitive element including a terminal connected to the first integrator circuit; a second capacitive element including a terminal connected to the second integrating circuit; a comparator including two input terminals, one of the two input terminals being connected to the other end of each of the first capacitive element and the second capacitive element; and The counter counts the count value within a period of time until the output signal of the comparator is reversed.

5. The solid-state imaging element according to claim 4, wherein The analog-to-digital converter further includes a connection switch that connects the other end of the first capacitance element and the other end of the second capacitance element according to a predetermined switching signal.

6. The solid-state imaging element according to claim 1, wherein Each of the pixels includes a photodiode in which a charge storage region is embedded in a predetermined semiconductor substrate.

7. The solid-state imaging element according to claim 1, wherein The pixel signal includes a predetermined reset level and a signal level according to an exposure amount, and Each of the pixels comprises: a first capacitive element and a second capacitive element; a preceding stage circuit that sequentially generates the reset level and the signal level, and causes the first capacitor and the second capacitor to respectively maintain the reset level and the signal level; a first subsequent stage circuit that reads and outputs the reset level held in the first capacitive element; and The second subsequent stage circuit reads and outputs the signal level held in the second capacitive element.

8. The solid-state imaging element according to claim 1, wherein The pixel signal includes a predetermined reset level and a signal level according to an exposure amount, and Each pixel consists of: a first capacitive element and a second capacitive element; a preceding stage circuit that sequentially generates the reset level and the signal level, and causes the first capacitor and the second capacitor to respectively maintain the reset level and the signal level; a selection circuit that sequentially performs control of connecting one of the first capacitance element and the second capacitance element to a predetermined subsequent node; control of disconnecting both the first capacitance element and the second capacitance element from the subsequent node; and control of connecting the other of the first capacitance element and the second capacitance element to the subsequent node; a subsequent-stage reset transistor that initializes a level of the subsequent-stage node when both the first capacitance element and the second capacitance element are disconnected from the subsequent-stage node; and The subsequent stage circuit sequentially reads the reset level and the signal level from the first capacitance element and the second capacitance element via the subsequent stage node and outputs the reset level and the signal level.

9. An imaging device comprising: a pixel array section in which a plurality of pixels are arranged in a two-dimensional grid pattern, each pixel generating and holding an analog pixel signal; a plurality of integration circuits that temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among the plurality of pixels and output integrated signals; an analog-to-digital converter, adding the integrated signals in each of the plurality of integration circuits to convert the added integrated signals into a digital signal; as well as The image recognition unit performs predetermined image recognition processing using the digital signal.

10. A method for controlling a solid-state imaging element, the method comprising: a sample and hold process in which each of a plurality of pixels arranged in a two-dimensional grid pattern generates and holds an analog pixel signal; an integration process in which a plurality of integration circuits temporally integrate pixel signals held in each of a predetermined number of pixels arranged in a vertical direction among the plurality of pixels and output integrated signals; as well as The analog-to-digital conversion process performs analog addition on the integrated signal of each of the multiple integration circuits to convert the analog added integrated signal into a digital signal.

Images