Image pickup element and electronic apparatus

Abstract

In the present invention, the correct number of incident photons can be derived (estimated) even if the incident illuminance varies over time during the exposure period. An imaging element according to the present technology includes: a light receiving element that is provided for each pixel and detects whether or not photons are present; and a detection circuit that is provided corresponding to the light-receiving element and detects photons incident on the light-receiving element. In the image pickup element, the detection circuit includes: a photon detection unit that detects photons incident on the light receiving element by performing a plurality of cycles of recharge control on the light receiving element; a plurality of memory cells; and a selection control unit that selects one of the plurality of storage units as a storage unit that stores the number of detected photons detected by the photon detection unit.

Description

Technical Field

[0001] The present technology relates to an imaging element and, more particularly, to an imaging element including a light receiving element that detects the presence or absence of a photon for each pixel, and an electronic device including the imaging element. Background Art

[0002] As an imaging element installed in an electronic device such as an imaging device, an imaging element is known that includes a light receiving element that detects the presence or absence of photons for each pixel, and performs photon detection by performing a plurality of cycles of recharging control on the light receiving element. For example, a technique is known that combines a plurality of cycles of recharging pulses (reset pulses) to control the recharging of the light receiving element so that the monotonic increase of the count value of the number of photon detections is not lost even when photons are incident at a high frequency, while maintaining the detection capability when photons are incident at a low frequency (for example, see Patent Document 1). Citation List Patent Literature

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 07-067043 Summary of the invention Problems to be solved by the present invention

[0004] In the above-mentioned related art, by combining multiple types of recharge pulse cycles (i.e., recharge control cycles) to perform recharge control, the nonlinearity of the input / output characteristics can be enhanced and the dynamic range can be expanded. However, since it is necessary to read the period of photon detection after the end of the exposure period, there is a problem of reducing the proportion of the light receiving period when shooting at a high frame rate.

[0005] The present technology has been proposed in view of such circumstances, and an object of the present technology is to improve the frame rate without compromising the light receiving period in a plurality of cycles of recharging control. Solution to the problem

[0006] The present technology is proposed to solve the above-mentioned problems, and the first aspect of the present technology is an imaging element, comprising: a light receiving element, which is set for each pixel and detects whether there is a photon; and a detection circuit, which is set corresponding to the light receiving element and detects the photon incident on the light receiving element, wherein the detection circuit comprises: a photon detection unit, which detects the photon incident on the light receiving element by performing a plurality of cycles of recharging control on the light receiving element; a plurality of storage units; and a selection control unit, which selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit. Therefore, this brings about the following effect: in the recharging control of a plurality of cycles, the frame rate is significantly improved by the number of storage units.

[0007] In addition, in the first aspect, the detection circuit may further include a counter that counts pulses generated when the photon detection unit detects photons, the plurality of storage units may be configured using memories, each of the memories storing count values ​​of the counter, and the selection control unit may be arranged between the counter and the plurality of storage units, and may perform control to selectively input the count values ​​of the counter to the plurality of storage units. Therefore, this brings about the following effect: in the recharging control of a plurality of cycles, the frame rate is significantly improved by the number of storage units.

[0008] In addition, in the first aspect, the detection circuit may further include a read control unit that reads a hold value of another storage unit during a hold period of one storage unit among the plurality of storage units. Therefore, this brings about the following effects: pipeline operation can be achieved, and interruption of exposure during the read period can be minimized in a global shutter operation that simultaneously scans all areas of an image.

[0009] In addition, in the first aspect, the plurality of storage units may be configured using counters, each of which counts pulses generated when the photon detection unit detects a photon, and the selection control unit may be arranged between the photon detection unit and the plurality of storage units, and performs control to selectively input the number of photon detections to the plurality of storage units. Therefore, this brings about an effect that the number of photon detections can be directly counted by the plurality of storage units.

[0010] In addition, in the first aspect, each counter of the plurality of storage units can repeat a period of performing a counting operation and a period of not performing a counting operation multiple times. Therefore, this brings about the following effect: since the light receiving period in each counter constituting the plurality of storage units can be dispersed, it is possible to simultaneously obtain an image pickup result to which an electronic neutral density (ND: neutralensity) filter is applied and an image pickup result obtained by normal exposure.

[0011] Furthermore, in the first aspect, the selection control unit may perform an operation of selecting one of the plurality of storage cells in synchronization with the switching of the cycle of the recharging control. Therefore, this brings about the following effect: since count values ​​of the number of photon detections from a plurality of different input / output characteristics can be obtained by one exposure, even in the case where the incident illuminance changes with time during the exposure period, the correct number of incident photons can be derived (estimated).

[0012] Furthermore, in the first aspect, the selection control unit may generate a selection pulse for selecting one of the plurality of storage cells, and the photon detection unit may perform the plurality of cycles of recharge control in synchronization with the selection pulse. Therefore, this brings about the effect that the gamma curve (input / output characteristics) can be adjusted after reading the count value without losing the original SN ratio.

[0013] Furthermore, in the first aspect, the detection circuit may further include a recharge cycle switching control unit that switches the cycle of the recharge control, and the selection control unit may generate a selection pulse for selecting one of the plurality of storage cells in synchronization with a switching signal output from the recharge cycle switching control unit. Therefore, this brings about the following effect: since the illuminance change in exposure can be estimated from the count value of the number of photon detections in the plurality of input / output characteristics, it is possible to cope with the illuminance change of a fast-moving subject.

[0014] Furthermore, in the first aspect, the detection circuit may further include a recharge cycle switching control unit that switches the cycle of the recharge control, and the photon detection unit may generate a selection pulse for selecting one of the plurality of storage cells based on a switching signal output from the recharge cycle switching control unit. Therefore, this brings about the following effect: since the number of photon detections can be counted by a combination of long and short recharge cycles, the dynamic range of the input / output characteristics in each counter can be expanded.

[0015] In addition, in the first aspect, the imaging element may have a stacked structure in which at least two semiconductor substrates are stacked, the at least two semiconductor substrates including: a first semiconductor substrate on which the light receiving element is formed; and a second semiconductor substrate on which the detection circuit is formed. Therefore, this brings about the following effect: when manufacturing an imaging element having a stacked structure, process optimization can be achieved for each semiconductor substrate.

[0016] In addition, in the first aspect, the image pickup element may further include a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage cells, and the selection pulse generating unit may be arranged outside the pixel array unit in which the detection circuit is arranged in an array on the second semiconductor substrate. Therefore, this brings about the effect that the circuit area of ​​the region of the detection circuit arranged in units of pixels on the second semiconductor substrate can be reduced to a smaller size compared to the case where the selection pulse generating unit is arranged within the pixel.

[0017] In addition, in the first aspect, the image pickup element may further include a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage units, and the selection pulse generating unit may be arranged in a pixel array unit in which the detection circuits are arranged in an array on the second semiconductor substrate, and the selection pulse generating unit may be arranged in pairs with each of the detection circuits. Therefore, this brings about the effect that the circuit area of ​​a region on the second semiconductor substrate where the circuit outside the pixel array unit is formed can be reduced to a smaller size compared to a case where the selection pulse generating unit is arranged outside the pixel array unit.

[0018] In addition, in the first aspect, the imaging element may further include an arithmetic processing unit that obtains an image output based on each output value of the plurality of storage units. Therefore, this brings about the following effect: one image output or a plurality of image outputs can be calculated based on each output value of the plurality of storage units.

[0019] In addition, in the first aspect, the arithmetic processing unit can obtain the incident illuminance in a plurality of exposure periods based on each output value of the plurality of storage units, and obtain the average illuminance of the incident illuminance in the plurality of exposure periods. Therefore, this brings about the following effect: by obtaining the average illuminance in consideration of the illuminance change in the exposure period, it is possible to cope with the illuminance change of a fast-moving subject.

[0020] Furthermore, in the first aspect, the light receiving element may be an avalanche diode, for example, a single photon avalanche diode. Therefore, this brings about the effect that a signal can be generated in response to the reception of a photon.

[0021] In addition, the second aspect of the present technology is an electronic device including an imaging element, wherein the imaging element includes: a light receiving element, which is set for each pixel and detects whether there is a photon; and a detection circuit, which is set corresponding to the light receiving element and detects the photon incident on the light receiving element, the detection circuit includes: a photon detection unit, which detects the photon incident on the light receiving element by performing a plurality of cycles of recharging control on the light receiving element; a plurality of storage units; and a selection control unit, which selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit. Therefore, this brings about the following effect: in the imaging element of the electronic device, the frame rate is significantly improved by the number of storage units in the recharging control of a plurality of cycles. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a perspective view schematically showing the structure of a semiconductor chip of the image pickup element according to the first embodiment of the present technology. Figure 2 1 is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a first embodiment of the present technology. Figure 3 It is a timing chart for explaining a circuit operation example of a pixel of the image pickup element according to the first embodiment of the present technology. Figure 4 : is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a second embodiment of the present technology. Figure 5 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a second embodiment of the present technology. Figure 6 : is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a third embodiment of the present technology. Figure 7 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a second embodiment of the present technology. Figure 8 : is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a fourth embodiment of the present technology. Fig. 9 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a fourth embodiment of the present technology. Fig.10 It is a diagram for explaining a method of estimating incident illuminance from count values ​​of different input / output characteristics in the fourth embodiment of the present technology. Fig.11 is a diagram for explaining a method of acquiring count values ​​from a plurality of input / output characteristics in the fourth embodiment of the present technology. Fig.12 : is a graph showing input / output characteristics A and B obtained for each memory cell in the fourth embodiment of the present technology. Fig.13 It is a diagram for explaining the technology according to the reference example. Fig.14 It is a diagram for explaining the case where the input / output characteristic is nonlinear and the case where the input / output characteristic is linear. Fig.15 : is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a fifth embodiment of the present technology. Fig.16 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a fifth embodiment of the present technology. Fig.17 : is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a sixth embodiment of the present technology. Fig.18 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a sixth embodiment of the present technology. Fig.19 This is a block diagram showing a circuit configuration example of a pixel of an image pickup element according to a seventh embodiment of the present technology. Fig. 20 It is a timing chart for explaining a circuit operation example of a pixel of an image pickup element according to a seventh embodiment of the present technology. Fig.21 It is a block diagram showing a configuration example of an image pickup element according to an eighth embodiment of the present technology. Fig. 22 It is a block diagram showing a configuration example of an image pickup element according to a ninth embodiment of the present technology. Fig.23 is a block diagram showing a circuit configuration example of an arithmetic processing unit according to a tenth embodiment of the present technology. Fig.24 It is a diagram for explaining an arithmetic processing unit according to an eleventh embodiment of the present technology. Fig.25 is a flowchart showing a specific processing flow of the arithmetic processing unit according to the eleventh embodiment of the present technology. Fig.26 It is a diagram for explaining an arithmetic processing unit according to a twelfth embodiment of the present technology. Fig. 27 This is a block diagram showing a configuration example of an imaging device as an example of an electronic device to which the present technology is applied. Fig.28 : is a diagram showing an example of a field to which an embodiment of the present technology is applied. Fig.29 It is a block diagram showing a schematic configuration example of a vehicle control system. Fig.30 It is an explanatory diagram showing an example of the installation position of the imaging unit. DETAILED DESCRIPTION

[0023] A mode for carrying out the present technology (hereinafter, referred to as an embodiment) will be described below. The description will be given in the following order. 1. First Embodiment (an example in which the number of photon detections is counted by using a counter, a plurality of storage units storing count values ​​are provided, and a memory is used as the storage unit) 2. Second Embodiment (Variation of the First Embodiment: Example of Implementing Pipeline Operation) 3. Third Embodiment (Example of using a counter to form a plurality of storage units and counting the number of photon detections by the plurality of storage units) 4. Fourth Embodiment (Example of synchronizing the switching of the recharge control period with the operation of selecting one of the plurality of storage cells in the configuration of the first embodiment) 5. Fifth Embodiment (Example of synchronizing the switching of the recharge control period with the operation of selecting one of the plurality of storage cells in the configuration of the third embodiment) 6. Sixth Embodiment (Example of Switching Between Multiple Storage Units Based on a Signal for Switching a Recharge Control Period) 7. Seventh Embodiment (Example of Generating a Selection Pulse for Switching Between Multiple Memory Cells in a Recharge Control Unit Generating a Reset Pulse) 8. Eighth Embodiment (Example of Generating a Selection Pulse for Switching Between Multiple Memory Cells Outside a Pixel Array Unit) 9. Ninth Embodiment (Example of Generating a Selection Pulse for Switching Between Multiple Memory Cells in Each Pixel in a Pixel Array Unit) 10. Tenth Embodiment (Example of Obtaining Image Output Based on Each Output Value of Multiple Storage Units) 11. Eleventh Embodiment (Example of Obtaining Average Illuminance and Estimating Incident Illuminance in Consideration of Illuminance Changes During Exposure Period) 12. Twelfth Embodiment (Example of Obtaining an Estimated Illuminance Most Matching a Combination of Output Values ​​of Multiple Storage Cells Using a Preliminary Inverse Function Table) 13. Modifications 14. Application examples of electronic devices (electronic devices of this technology) 15. Application examples of implementation schemes of the present technology 16. Application examples for mobile objects 17. The technology can be used in the following structures

[0024] <First embodiment> The first embodiment of the present technology is an example in which the number of photon detections is counted by using a counter, a plurality of storage units storing count values ​​are provided, and a memory is used as the storage unit.

[0025] For example, for the imaging element according to the first embodiment of the present technology, a single photon avalanche diode (SPAD) element operating in Geiger mode is used as a light receiving element that is set for each pixel and detects the presence or absence of photons. That is, the light receiving element of the pixel is not limited to the SPAD element, and in addition to the SPAD element, various elements operating in Geiger mode such as an avalanche photodiode (APD) can also be used. The same applies to the embodiments described later.

[0026] [Semiconductor chip structure of imaging element] Figure 1 1 is a perspective view schematically showing a semiconductor chip structure of an image pickup element 1 according to a first embodiment of the present technology. Here, as a semiconductor chip structure of the image pickup element 1 according to the first embodiment, a stacked semiconductor chip structure in which at least two semiconductor substrates (chips) of a first semiconductor substrate 81 and a second semiconductor substrate 82 are stacked is exemplified.

[0027] In this stacked semiconductor chip structure (so-called stacked structure), the first semiconductor substrate 81 is a sensor chip provided for each pixel 2, and the first semiconductor substrate 81 is provided with SPAD elements 10 arranged in a two-dimensional array as an example of a light receiving element that detects the presence or absence of photons. The second semiconductor substrate 82 is a logic chip, in which detection circuits 20 that detect photons incident on the SPAD elements 10 are arranged in a two-dimensional array, and the detection circuits 20 correspond to the SPAD elements 10 in the first semiconductor substrate 81, respectively.

[0028] The SPAD element 10 on the first semiconductor substrate 81 and the detection circuit (logic circuit) 20 on the second semiconductor substrate 82 are electrically connected via a connection portion 84 formed in a wiring layer 83 interposed between the first semiconductor substrate 81 and the second semiconductor substrate 82. As an example of the connection portion 84 of the wiring layer 83, a Cu-Cu connection for directly connecting a Cu electrode can be exemplified.

[0029] According to the above-mentioned stacked semiconductor chip structure, a process suitable for manufacturing the SPAD element 10 can be applied to the first semiconductor substrate 81, and a process suitable for manufacturing the detection circuit (logic circuit) 20 can be applied to the second semiconductor substrate 82. That is, when manufacturing the imaging element 1 having a stacked structure according to the first embodiment, process optimization can be achieved for each semiconductor substrate.

[0030] Note that here, although a stacked semiconductor chip structure is exemplified as the semiconductor chip structure of the image pickup element 1 according to the first embodiment, a flat semiconductor chip structure in which the SPAD element 10 and the detection circuit 20 are formed on the same semiconductor substrate may be adopted.

[0031] [Pixel Circuit Configuration Example] Figure 2 1 is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the first embodiment of the present technology. Figure 2 The circuit configuration example of one pixel is shown, and the case where two storage units for storing the number of photon detections are provided is shown. The same applies to the embodiments described later.

[0032] The pixel 2 of the image pickup element 1 according to the first embodiment of the present technology includes a SPAD element 10 that detects the presence or absence of photons and a detection circuit 20 that is provided corresponding to the SPAD element 10 and detects photons incident on the SPAD element 10 .

[0033] For example, the SPAD element 10 has a grounded anode and operates at a reverse bias voltage equal to or higher than a breakdown voltage. When light is incident and even a single electron-hole pair is generated, the electron-hole pair acts as a seed to generate an avalanche current. As a result, even if one photon is incident, the SPAD element 10 can detect with a certain photon detection efficiency (PDE: photon detection efficiency).

[0034] The detection circuit 20 includes a photon detection unit 30 , a counter 40 , a holding pulse generation unit 50 , a storage unit 60 , and a selection control unit 70 .

[0035] In the detection circuit 20 having the above-described configuration, the photon detection unit 30 includes a recharging transistor 31, a recharging control unit 32, and a waveform shaping unit 33, and detects photons incident on the SPAD element 10 by performing multiple cycles of recharging control on the SPAD element 10 to generate a detection pulse.

[0036] For example, the recharge transistor 31 includes a P-type metal oxide semiconductor (MOS) transistor connected to the power supply voltage VDD between the node of and the cathode of the SPAD element 10 , and recharges the SPAD element 10 .

[0037] The output terminal of the recharge control unit 32 is connected to the gate of the recharge transistor 31, and the recharge control unit 32 outputs a plurality of reset pulses RST of different cycles and supplies the reset pulse RST to the gate of the recharge transistor 31. That is, the reset pulse RST is a recharge pulse for controlling the recharge timing of the SPAD element 10.

[0038] For example, the waveform shaping unit 33 includes a CMOS inverter, performs waveform shaping on the cathode voltage Vk as the output of the SPAD element 10 , and outputs the result as the detection pulse DET.

[0039] The counter 40 counts the detection pulses DET generated by the photon detection unit 30. The count value CNT of the counter 40 corresponds to the number of photon detections of the SPAD element 10.

[0040] The sustain pulse generating unit 50 generates a sustain pulse PLS to be supplied to the counter 40 , and controls the counter 40 by applying the sustain pulse PLS.

[0041] The storage unit 60 includes a plurality of storage units, for example, formed of two storage units, a first storage unit 61 and a second storage unit 62. In the first embodiment, the first storage unit 61 and the second storage unit 62 are configured using, for example, a memory. The first storage unit 61 and the second storage unit 62 have a 2-input configuration, and the count value CNT of the counter 40 is used as one input.

[0042] The selection control unit 70 includes a 2-input first AND circuit 71 , a 2-input second AND circuit 72 , and a selection pulse generating unit 73 .

[0043] The 2-input first AND circuit 71 uses the holding pulse PLS generated by the holding pulse generating unit 50 as one input and uses the selection pulse SEL1 generated by the selection pulse generating unit 73 as the other input, and generates a holding signal HLD1 based on the two inputs. The holding signal HLD1 generated by the first AND circuit 71 is provided to the first storage unit 61 as the other input.

[0044] The 2-input second AND circuit 72 uses the holding pulse PLS generated by the holding pulse generating unit 50 as one input and uses the selection pulse SEL2 generated by the selection pulse generating unit 73 as the other input, and generates a holding signal HLD2 based on the two inputs. The holding signal HLD2 generated by the second AND circuit 72 is provided to the second storage unit 62 as the other input.

[0045] The selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 to select one of the multiple storage cells of the storage unit 60 (in this example, the first storage cell 61 or the second storage cell 62) as a storage cell that stores the count value CNT of the counter 40 corresponding to the number of photon detections, and the selection pulse generating unit 73 provides the selection pulses SEL1 and SEL2 as another input to the first AND circuit 71 and the second AND circuit 72.

[0046] The first storage unit 61 stores the count value CNT output from the counter 40 under the control of the hold signal HLD1 applied from the first AND circuit 71. The hold value REG1 of the first storage unit 61 corresponds to the number of photon detections detected by the photon detection unit 30, and is output as a pixel signal corresponding to the photons incident on the SPAD element 10 in a predetermined exposure period.

[0047] The second storage unit 62 stores the count value CNT output from the counter 40 under the control of the hold signal HLD2 applied from the second AND circuit 72. The hold value REG2 of the second storage unit 62 corresponds to the number of photon detections detected by the photon detection unit 30, and is output as a pixel signal corresponding to the photons incident on the SPAD element 10 in a predetermined exposure period.

[0048] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the first embodiment configured as described above will be described. Figure 3 1 is a timing chart for explaining an example of circuit operation of the pixel 2 of the image pickup element 1 according to the first embodiment. Here, the timing relationship in one exposure period is shown.

[0049] Figure 3 The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the selection pulses SEL1 and SEL2, the hold pulse PLS, the hold signals HLD1 and HLD2, the count value CNT, and the hold values ​​REG1 and REG2 relative to the incident photons is shown.

[0050] like Figure 3As shown in the timing diagram of , in the first half of one exposure period, the selection pulse SEL1 is high level, and the selection pulse SEL2 is low level; in the second half of one exposure period, the selection pulse SEL1 is low level, and the selection pulse SEL2 is high level. As a result, in the image pickup element 1 according to the first embodiment, a circuit operation is performed in which the first storage unit 61 stores the count value N1 of the counter 40 in the first half of one exposure period, and the second storage unit 62 stores the count value N2 of the counter 40 in the second half of one exposure period. Through this circuit operation, the first storage unit 61 and the second storage unit 62 respectively store the count values ​​N1 and N2 of different time periods in one exposure period.

[0051] As described above, in the image pickup element 1 according to the first embodiment, an operation is performed to store the count values ​​N1 and N2 of the counter 40 at different time periods in one exposure period in different storage cells under the recharge control of multiple cycles for the SPAD element 10, and output the number of storage cells as an image pickup result. As a result, in the recharge control of multiple cycles, the frame rate can be significantly improved by the number of storage cells.

[0052] <Second Implementation Method> The second embodiment of the present technology is a modification of the first embodiment by counting the number of photon detections using a counter, providing a plurality of storage units storing count values, and using a memory as the storage unit, and is an example of realizing pipeline operation.

[0053] [Pixel Circuit Configuration Example] Figure 4 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the second embodiment of the present technology.

[0054] Similar to the case of the first embodiment, the pixel 2 of the imaging element 1 according to the second embodiment has the following structure: the detection circuit 20 for detecting photons incident on the SPAD element 10 includes a photon detection unit 30, a counter 40, a holding pulse generation unit 50, a storage unit 60 and a selection control unit 70.

[0055] In the detection circuit 20 having the above-mentioned configuration, the configurations of the photon detection unit 30, the counter 40, the holding pulse generation unit 50, and the selection control unit 70 are the same as those of the first embodiment, but the configuration of the storage unit 60 is different from that of the first embodiment. Specifically, in addition to the first storage unit 61 and the second storage unit 62 configured using, for example, a memory, the storage unit 60 also includes a read control unit 63.

[0056] The read control unit 63 performs control to generate read control signals RD1 and RD2, reads the held value REG1 by applying the read control signal RD1 to the first storage unit 61, and reads the held value REG2 by applying the read control signal RD2 to the second storage unit 62 at a timing different from that of the read control signal RD1.

[0057] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the second embodiment configured as described above will be described. Figure 5 1 is a timing chart for explaining an example of circuit operation of the pixel 2 of the image pickup element 1 according to the second embodiment. Here, the timing relationship in one exposure period is shown.

[0058] Figure 5 The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the selection pulses SEL1 and SEL2, the hold pulse PLS, the hold signals HLD1 and HLD2, the count value CNT, the hold values ​​REG1 and REG2, and the read control signals RD1 and RD2 relative to the incident photons is shown.

[0059] like Figure 5 As shown in the timing diagram of FIG. 1 , under the control of the hold signals HLD1 and HLD2, the count value CNT of the counter 40 is held twice alternately in the first storage unit 61 and the second storage unit 62 in one exposure period. In this case, the holding is performed at the falling edge of the selection pulse SEL (SEL1 and SEL2), and the holding is released at the rising edge. Thereafter, under the control of the read control signals RD1 and RD2 outputted from the read control unit 63 at different timings, during the period in which one of the first storage unit 61 and the second storage unit 62 is holding, the held value REG1 or REG2 of the other is read.

[0060] As described above, in the image pickup element 1 according to the second embodiment, a circuit operation is performed in one exposure period to alternately switch the holding period for the first storage unit 61 and the second storage unit 62 so that the increase in the count value CNT of the counter 40 is always reflected in one storage unit. Then, a so-called pipeline operation is performed, that is, during a period in which one of the first storage unit 61 and the second storage unit 62 performs holding, the held value REG1 or REG2 of the other is read. By such a pipeline operation, it is possible to minimize the interruption of exposure in the reading period in the global shutter operation of scanning all areas of the image at the same time.

[0061] <Third Implementation Method> The third embodiment of the present technology is an example in which a plurality of storage cells are constituted using counters and the number of photon detections is counted by the plurality of storage cells.

[0062] [Pixel Circuit Configuration Example] Figure 6 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the third embodiment of the present technology.

[0063] The pixel 2 of the image pickup element 1 according to the third embodiment has a configuration in which the detection circuit 20 for detecting photons incident on the SPAD element 10 includes a photon detection unit 30, a storage unit 60, and a selection control unit 70. In the detection circuit 20 having such a configuration, the configuration of the photon detection unit 30 is the same as that of the first embodiment.

[0064] For example, the storage unit 60 includes two storage units, a first storage unit 61 and a second storage unit 62. In the third embodiment, the first storage unit 61 and the second storage unit 62 are configured using, for example, a counter. That is, in the third embodiment, the first storage unit 61 and the second storage unit 62 configured using a counter count the detection pulses DET generated by the photon detection unit 30, that is, count the number of photon detections.

[0065] The count value CNT1 of the first storage unit 61 and the count value CNT2 of the second storage unit 62 correspond to the number of photon detections detected by the photon detection unit 30 , and are output as pixel signals corresponding to photons incident on the SPAD element 10 in a predetermined exposure period.

[0066] In the selection control unit 70, the 2-input first AND circuit 71 receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL1 generated by the selection pulse generation unit 73 as the other input, and outputs a first detection pulse DET1 based on the two inputs. This first detection pulse DET1 is counted by the counter constituting the first storage unit 61. The count value CNT1 of the first storage unit 61 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0067] The second AND circuit 72 having two inputs receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL2 generated by the selection pulse generation unit 73 as the other input, and outputs a second detection pulse DET2 based on the two inputs. The second detection pulse DET2 is counted by the counter constituting the second storage unit 62. The count value CNT2 of the second storage unit 62 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0068] The selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 to select one of the multiple storage cells of the storage unit 60 (in this example, the first storage cell 61 or the second storage cell 62) as a storage cell that stores the detection pulse DET corresponding to the number of photon detections, and the selection pulse generating unit 73 provides the selection pulses SEL1 and SEL2 as another input to the first AND circuit 71 and the second AND circuit 72.

[0069] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the third embodiment configured as described above will be described. Figure 7 1 is a timing chart for explaining a circuit operation example of the pixel 2 of the image pickup element 1 according to the third embodiment.

[0070] Figure 7 The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the selection pulses SEL1 and SEL2, the first and second detection pulses DET1 and DET2, and the first and second count values ​​CNT1 and CNT2 with respect to the incident photons is shown.

[0071] like Figure 7 As shown in the timing chart of , in the image pickup element 1 according to the third embodiment, under the control of the selection pulses SEL1 and SEL2, the switching between the first storage unit 61 and the second storage unit 62 is repeated multiple times. Then, regarding the counting of the number of photon detections by the respective counters constituting the first storage unit 61 and the second storage unit 62, specifically, the counting of the first count value CNT1 and the second count value CNT2, the period during which the counting operation is performed and the period during which the counting operation is not performed are repeated multiple times. The count values ​​CNT1 and CNT2 of the first storage unit 61 and the second storage unit 62 are output as pixel signals corresponding to the number of photons detected in a predetermined exposure period.

[0072] As described above, in the image pickup element 1 according to the third embodiment, while the switching between the first storage unit 61 and the second storage unit 62 is repeated multiple times, the period during which the counting operation for the number of photon detections is performed and the period during which the counting operation is not performed are repeated multiple times in the respective counters constituting the first storage unit 61 and the second storage unit 62. As a result, the light receiving period in the respective counters constituting the first storage unit 61 and the second storage unit 62 can be dispersed, and the image pickup result to which the electronic neutral density (ND) filter is applied and the image pickup result obtained by normal exposure can be obtained at the same time.

[0073] <Fourth Implementation Plan> The fourth embodiment of the present technology is an example in which switching of a recharge control cycle (hereinafter, may be simply referred to as a "recharge cycle") is synchronized with an operation of selecting one of a plurality of storage cells in the configuration of the first embodiment.

[0074] [Pixel Circuit Configuration Example] Figure 8 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the fourth embodiment of the present technology.

[0075] Similar to the case of the first embodiment, the pixel 2 of the image pickup element 1 according to the fourth embodiment has the following configuration: wherein the detection circuit 20 for detecting photons incident on the SPAD element 10 includes a photon detection unit 30, a counter 40, a holding pulse generation unit 50, a storage unit 60, and a selection control unit 70. The configurations of the photon detection unit 30, the counter 40, the holding pulse generation unit 50, the storage unit 60, and the selection control unit 70 are the same as those of the first embodiment. In addition, in the storage unit 60, the first storage unit 61 and the second storage unit 62 are configured using a memory.

[0076] The image pickup element 1 of the fourth embodiment according to the above-described configuration is different from the case of the first embodiment in that the switching of the recharge control period in the recharge control unit 32 is synchronized with the operation of selecting one of the plurality of storage units (the first storage unit 61 or the second storage unit 62). Here, the recharge control period is a period of the reset pulse RST output from the recharge control unit 32 that recharges the SPAD element 10. The reset pulse RST may also be referred to as a recharge pulse.

[0077] The selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 to select the first storage unit 61 or the second storage unit 62 as a storage unit storing the count value CNT of the counter 40 corresponding to the number of photon detections, and supplies the selection pulses SEL1 and SEL2 to the first AND circuit 71 and the second AND circuit 72. The selection pulse generating unit 73 also supplies the selection pulses SEL1 and SEL2 to the recharging control unit 32.

[0078] The recharge control unit 32 generates a reset pulse RST having a plurality of different cycles based on the selection pulses SEL1 and SEL2 supplied from the selection pulse generating unit 73. Specifically, the recharge control unit 32 switches the cycle of the reset pulse RST, that is, switches the cycle of the recharge control, based on the selection pulses SEL1 and SEL2. As a result, the switching of the cycle of the recharge control is synchronized with the operation of selecting the first storage unit 61 or the second storage unit 62.

[0079] The first storage unit 61 stores the count value CNT output from the counter 40 under the control of the hold signal HLD1 applied from the first AND circuit 71. The hold value REG1 of the first storage unit 61 corresponds to the number of photon detections detected by the photon detection unit 30, and is output as a pixel signal corresponding to the photons incident on the SPAD element 10 in a predetermined exposure period.

[0080] The second storage unit 62 stores the count value CNT output from the counter 40 under the control of the hold signal HLD2 applied from the second AND circuit 72. The hold value REG2 of the second storage unit 62 corresponds to the number of photon detections detected by the photon detection unit 30, and is output as a pixel signal corresponding to the photons incident on the SPAD element 10 in a predetermined exposure period.

[0081] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the fourth embodiment configured as described above will be described. Fig. 9 1 is a timing chart for explaining a circuit operation example of the pixel 2 of the image pickup element 1 according to the fourth embodiment.

[0082] Fig. 9 The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the selection pulses SEL1 and SEL2, the hold pulse PLS, the hold signals HLD1 and HLD2, the count value CNT, and the hold values ​​REG1 and REG2 relative to the incident photons is shown.

[0083] like Fig. 9 As shown in the timing diagram of , at the timing of the logic switching of the selection pulses SEL1 and SEL2, the switching of the recharging control period is performed, that is, in this example, switching from the period B of the long recharging period to the period A of the short recharging period. Then, a circuit operation is performed, in which the first storage unit 61 stores the count value N1 of the counter 40 in the period B in which the selection pulse SEL1 is at a high level and the selection pulse SEL2 is at a low level, and the second storage unit 62 stores the count value N2 of the counter 40 in the period A in which the selection pulse SEL1 is at a low level and the selection pulse SEL2 is at a high level.

[0084] As described above, in the image pickup element 1 according to the fourth embodiment, the input of the count value CNT of the counter 40 to the first storage unit 61 and the second storage unit 62 configured using memories is synchronized with the switching of the cycle of the reset pulse RST. As a result, in addition to different exposure periods, the first storage unit 61 and the second storage unit 62 can store the count value N1 and the count value N2 of the counter 40 under different incident illuminances. As a result, the gamma curve (input / output characteristics) can be adjusted after reading the count value without losing the SN ratio obtained in the case of performing exposure with a single storage unit.

[0085] As in the fourth embodiment described above, photon detection is performed by a plurality of cycles of recharge control, a plurality of storage units for storing the number of photon detections are provided, and the switching of the cycles of the recharge control is synchronized with the switching of the storage units for storing the number of photon detections, so that the following effects and results can be obtained. That is, since count values ​​of the number of photon detections from a plurality of different input / output characteristics can be obtained by a single exposure, even in the case where the incident illuminance changes with time during the exposure period, the correct number of incident photons can be derived (estimated).

[0086] Here, the following two points will be explained: (1) A method for estimating incident illuminance based on count values ​​of different input / output characteristics; and (2) A method for obtaining count values ​​through multiple input / output characteristics.

[0087] [Method for estimating incident illuminance based on count values ​​of different input / output characteristics] Fig.10 4 is a diagram for explaining a method of estimating incident illuminance according to count values ​​of different input / output characteristics in the fourth embodiment of the present technology. Fig.10 In the figure, a is used to describe the incident illuminance f H and f L to achieve a count value N of the number of photon detections from the input / output characteristic B B and in the same figure, b is a diagram for explaining the illumination change to obtain the count value N of the number of photon detections from the input / output characteristic A. A and the count value N of the number of photon detections from the input / output characteristic B B .

[0088] Assume that the count value N of the number of photon detections is obtained by two input / output characteristics A and B A and N B Assuming that the illuminance change occurs in the middle of the exposure period, the incident illuminance f in the first half of the exposure period can be estimated Hand the incident illuminance f in the second half of the exposure period L .

[0089] First, obtain the realization count value N B The incident illuminance f H and f L Based on the known input / output characteristic A, for the incident illuminance f H and f L For each combination of , the count value of the number of photon detections from the input / output characteristic A is calculated. In this case, the count value N of the number of photon detections actually obtained is obtained. A f H and f L The combination of is the incident illuminance.

[0090] [Method for obtaining the count value of the number of photon detections through multiple input / output characteristics] Fig.11 It is a diagram for explaining a method of acquiring a count value of the number of photon detections through a plurality of input / output characteristics in the fourth embodiment of the present technology.

[0091] In the recharging control of multiple cycles, the reset pulse RST is supplied from the recharging control unit 32, while switching multiple different cycles over time. In this case, a plurality of storage units storing the detection pulse DET generated by the photon detection unit 30 are prepared, and the switching of the plurality of storage units is synchronized with the cycle of the reset pulse RST supplied from the recharging control unit 32.

[0092] Specifically, for example, in the case where the reset pulse RST is supplied while switching the periods of period A and period B, two storage units, the first storage unit 61 and the second storage unit 62, are prepared as the storage unit 60. Then, switching between the storage units is performed so that the first storage unit 61 stores the count value of the number of photon detections only in period B, and the second storage unit 62 stores the count value of the number of photon detections only in period A. By performing such an operation, Fig.12 As shown, different input / output characteristics A and B of each storage unit (first storage unit 61 / second storage unit 62) can be obtained. Then, the first storage unit 61 obtains the count value n of the number of photon detections from the input / output characteristic B. B , and the second storage unit 62 obtains the count value n of the number of photon detections from the input / output characteristic A A .

[0093] [Technology based on reference example] Here, a case where the count value of the number of photon detections is nonlinear with respect to the number of incident photons per unit time is described as a technique according to a reference example. As a problem of the technique according to this reference example, due to the nonlinear input / output characteristics, it is difficult to derive the correct number of incident photons when the incident illuminance changes over time. This will be described below.

[0094] The count value of the number of photon detections obtained by exposing for a certain time is obtained by integrating the input / output characteristics per unit time with respect to time. If the incident illuminance does not change with time during the exposure period, if the count value is divided by the exposure time, the input / output characteristics per unit time can be maintained, so that the number of incident photons can be derived from the count value of the number of photon detections.

[0095] On the other hand, when the incident illuminance changes with time during the exposure period, due to the nonlinearity of the input / output characteristics, the same count value of the number of photon detections can be obtained even if the number of incident photons changes. Therefore, the number of incident photons cannot be correctly derived from the count value of the number of photon detections.

[0096] For example, in Fig.13 When the incident illuminance shown in a does not change during a specific exposure period, as shown in FIG. Fig.13 As shown in b, the count value of the number of photon detections relative to the incident illuminance is a constant multiple of the input / output characteristics per unit time. Since the incident illuminance and the count value of the number of photon detections correspond one to one, the number of incident photons can be derived from the count value of the number of photon detections.

[0097] On the other hand, as an example of a case where the illuminance changes, such as Fig.13 As shown in c, it can be considered that the incident illuminance switches to high illuminance in the first half of the exposure period and switches to low illuminance in the second half. In this case, since the incident illuminance f in the first half of the exposure period is H , the count value of the number of photon detections increases by the count value N of the number of photon detections H , and due to the incident illumination f in the second half of the exposure period L , the count value of the number of photon detections increases by the count value N of the number of photon detections L , thus obtaining the count value N of the number of photon detections out =N H +N L .

[0098] The count value N of the number of photon detections that cannot be obtained in this case out Derive the number of incident photons. For example, suppose the illuminance variation is known and only the incident illuminance f H and fL is unknown. Since the number of incident photons is related to the average illumination f M So we only need to know f M =(f H +f L ) / 2. However, the count value N used to obtain the number of photon detections out The incident illuminance f H and f L The combination of and the corresponding average illumination f M There are countless of them, so it is impossible to determine them uniquely. Fig.14 This is a problem unique to nonlinear input / output characteristics. This is because, as Fig.14 As shown in b, when the input / output characteristic is linear, the incident illuminance f H and f L There are countless combinations of, but the average illumination f M are all equal.

[0099] Because the correct number of incident photons is not known even if the illumination variation is known, it is also impossible to derive the correct number of incident photons in more complex real-world situations where the illumination variation is unknown.

[0100] <Fifth Implementation Plan> The fifth embodiment of the present technology is an example in which switching of the cycle of the recharge control is synchronized with an operation of selecting one of a plurality of storage cells in the configuration of the third embodiment.

[0101] [Pixel Circuit Configuration Example] Fig.15 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the fifth embodiment of the present technology.

[0102] Similar to the case of the third embodiment, the pixel 2 of the imaging element 1 according to the fifth embodiment has the following configuration: wherein the detection circuit 20 for detecting photons incident on the SPAD element 10 includes a photon detection unit 30, a storage unit 60, and a selection control unit 70. The configuration of the photon detection unit 30 is the same as that of the first embodiment. In addition, in the storage unit 60, the first storage unit 61 and the second storage unit 62 are configured using a counter.

[0103] The imaging element 1 of the fifth embodiment constructed as described above is different from that of the third embodiment in that the switching of the recharging control cycle in the recharging control unit 32 is synchronized with the operation of selecting one of the plurality of storage units (the first storage unit 61 or the second storage unit 62).

[0104] The selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 to select the first storage unit 61 or the second storage unit 62 as a storage unit storing the number of photon detections generated by the photon detecting unit 30, and supplies the selection pulses SEL1 and SEL2 to the first AND circuit 71 and the second AND circuit 72. The selection pulse generating unit 73 also supplies the selection pulses SEL1 and SEL2 to the recharging control unit 32.

[0105] The recharge control unit 32 generates a reset pulse RST having a plurality of different cycles based on the selection pulses SEL1 and SEL2 supplied from the selection pulse generating unit 73. Specifically, the recharge control unit 32 switches the cycle of the reset pulse RST, that is, switches the cycle of the recharge control, based on the selection pulses SEL1 and SEL2. As a result, the switching of the cycle of the recharge control is synchronized with the operation of selecting the first storage unit 61 or the second storage unit 62.

[0106] In the selection control unit 70, the 2-input first AND circuit 71 receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL1 generated by the selection pulse generation unit 73 as the other input, and outputs a first detection pulse DET1 based on the two inputs. This first detection pulse DET1 is counted by the counter constituting the first storage unit 61. The count value CNT1 of the first storage unit 61 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0107] The second AND circuit 72 having two inputs receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL2 generated by the selection pulse generation unit 73 as the other input, and outputs a second detection pulse DET2 based on the two inputs. The second detection pulse DET2 is counted by the counter constituting the second storage unit 62. The count value CNT2 of the second storage unit 62 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0108] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the fifth embodiment configured as described above will be described. Fig.16 1 is a timing chart for explaining a circuit operation example of the pixel 2 of the image pickup element 1 according to the fifth embodiment.

[0109] Fig.16The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the selection pulses SEL1 and SEL2, the first and second detection pulses DET1 and DET2, and the first and second count values ​​CNT1 and CNT2 with respect to the incident photons is shown.

[0110] like Fig.16 As shown in the timing chart of , in the image pickup element 1 according to the fifth embodiment, at the timing of the logic switching of the selection pulses SEL1 and SEL2, the switching of the recharging control period is performed, that is, in this example, switching from the period B of the long recharging period to the period A of the short recharging period. Then, the respective counters constituting the first storage unit 61 and the second storage unit 62 count the number of photon detections by repeating the switching between the first storage unit 61 and the second storage unit 62 under the control of the selection pulses SEL1 and SEL2. The count values ​​CNT1 and CNT2 of the first storage unit 61 and the second storage unit 62 are output as pixel signals corresponding to the number of photons detected in a predetermined exposure period.

[0111] As described above, in the image pickup element 1 according to the fifth embodiment, the input of the detection pulse DET generated by the photon detection unit 30 to the first storage unit 61 and the second storage unit 62 constituted using a counter is synchronized with the switching of the cycle of the reset pulse RST. Therefore, it is possible to obtain exposure results from a plurality of different input / output characteristics in the same exposure period without bias during the exposure period. As a result, since the illuminance change in the exposure can be estimated from the count value of the number of photon detections in the plurality of input / output characteristics, it is possible to cope with the illuminance change of a fast-moving subject.

[0112] <Sixth Implementation Plan> The sixth embodiment of the present technology is an example of switching a plurality of storage cells based on a signal for switching a cycle (recharging cycle) of recharging control in the configuration of the third embodiment. Similar to the fourth embodiment, in this example, the switching of the cycle of recharging control can be synchronized with the operation of selecting one of the plurality of storage cells.

[0113] [Pixel Circuit Configuration Example] Fig.17 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the sixth embodiment of the present technology.

[0114] Similar to the case of the third embodiment, the pixel 2 of the imaging element 1 according to the sixth embodiment has the following configuration: wherein the detection circuit 20 for detecting photons incident on the SPAD element 10 includes a photon detection unit 30, a storage unit 60, and a selection control unit 70. The configuration of the photon detection unit 30 is the same as that of the first embodiment. In addition, in the storage unit 60, the first storage unit 61 and the second storage unit 62 are configured using a counter.

[0115] The image pickup element 1 according to the sixth embodiment includes a recharging cycle switching control unit 90 in addition to the photon detection unit 30, the storage unit 60, and the selection control unit 70. The recharging cycle switching control unit 90 generates four types of switching signals A, B, C, and D to switch the cycle of the reset signal RST output from the recharging control unit 32, that is, the recharging cycle, and the recharging cycle switching control unit 90 supplies the switching signals A, B, C, and D to the recharging control unit 32.

[0116] The recharging control unit 32 generates the reset signal RST having four types of cycle A, cycle B, cycle C, and cycle D based on the recharging cycle switching signals A, B, C, and D supplied from the recharging cycle switching control unit 90. In other words, in the recharging control unit 32, switching of the cycle of the reset signal RST to be generated is performed based on the recharging cycle switching signals A, B, C, and D.

[0117] The recharge cycle switching control unit 90 also supplies the generated recharge cycle switching signals A, B, C, and D to the selection pulse generating unit 73 of the selection control unit 70. The selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 to select the first storage unit 61 or the second storage unit 62 as a storage unit for storing (counting) the number of photon detections based on the recharge cycle switching signals A, B, C, and D supplied from the recharge cycle switching control unit 90. The selection pulses SEL1 and SEL2 generated by the selection pulse generating unit 73 are supplied to the first AND circuit 71 and the second AND circuit 72.

[0118] The selection pulse generating unit 73 may include a logic circuit. Specifically, Fig.17 As shown in the lower right of , the selection pulse generating unit 73 may include two 2-input AND circuits 731 and 732. One AND circuit 731 receives two inputs of a switching signal A and a switching signal C to generate a selection pulse SEL1. The other AND circuit 732 receives two inputs of a switching signal B and a switching signal D to generate a selection pulse SEL2.

[0119] In this way, by generating selection pulses SEL1 and SEL2 for selecting the first storage unit 61 or the second storage unit 62 based on the recharging cycle switching signals A, B, C and D, the switching of the cycle of the recharging control can be synchronized with the operation of selecting the first storage unit 61 or the second storage unit 62.

[0120] In the selection control unit 70, the 2-input first AND circuit 71 receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL1 generated by the selection pulse generation unit 73 as the other input, and outputs the first detection pulse DET1 based on the two inputs. The first detection pulse DET1 is counted by the counter constituting the first storage unit 61. The count value CNT1 of the first storage unit 61 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0121] The second AND circuit 72 with 2 inputs receives the detection pulse DET generated by the photon detection unit 30 as one input and receives the selection pulse SEL2 generated by the selection pulse generation unit 73 as the other input, and outputs the second detection pulse DET2 based on the two inputs. The second detection pulse DET2 is counted by the counter constituting the second storage unit 62. The count value CNT2 of the second storage unit 62 is output as a pixel signal corresponding to the number of photons detected in a predetermined exposure period.

[0122] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the sixth embodiment configured as described above will be described. Fig.18 1 is a timing chart for explaining a circuit operation example of the pixel 2 of the image pickup element 1 according to the sixth embodiment.

[0123] Fig.18 The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the recharging cycle switching signals A, B, C, D, the selection pulses SEL1 and SEL2, the first detection pulse DET1 and the second detection pulse DET2, and the first count value CNT1 and the second count value CNT2 relative to the incident photon is shown.

[0124] like Fig.18As shown in the timing chart of , in the image pickup element 1 according to the sixth embodiment, in the recharging control unit 32, four types of recharging cycles of cycle A, cycle B, cycle C, and cycle D are set as the cycles of the reset pulse RST based on four types of recharging cycle switching signals A, B, C, and D. Specifically, cycle A is set during the high level period of the switching signal A, cycle B is set during the high level period of the switching signal B, cycle C is set during the high level period of the switching signal C, and cycle D is set during the high level period of the switching signal D. Here, for example, the length relationship between the cycles of the reset pulse RST is cycle A>cycle B>cycle C>cycle D.

[0125] Furthermore, in the selection pulse generating unit 73 of the selection control unit 70, the selection pulse SEL1 of the high level is generated in the period in which the recharging cycle switching signals A and C are at the high level, and the selection pulse SEL2 of the high level is generated in the period in which the recharging cycle switching signals B and D are at the high level. As a result, the logic switching of the selection pulses SEL1 and SEL2 is performed when the recharging cycle is switched.

[0126] Then, the respective counters constituting the first storage unit 61 and the second storage unit 62 count the number of photon detections by repeating switching between the first storage unit 61 and the second storage unit 62 under the control of the selection pulses SEL1 and SEL2. The count values ​​CNT1 and CNT2 of the first storage unit 61 and the second storage unit 62 are output as pixel signals corresponding to the number of photons detected in a predetermined exposure period.

[0127] As described above, in the image pickup element 1 according to the sixth embodiment, switching of the cycle of the reset pulse RST is synchronized with switching between the first storage unit 61 and the second storage unit 62 based on the recharging cycle switching signals A, B, C, and D. As a result, count values ​​CNT1 and CNT2 of the number of photons detected in a plurality of recharging cycles whose cycles are different from each other can be stored in the respective counters of the first storage unit 61 and the second storage unit 62. As a result, since the number of photon detections can be counted by a combination of long and short recharging cycles, the dynamic range of the input / output characteristics in each counter can be expanded.

[0128] <Seventh Implementation Plan> The seventh embodiment of the present technology is an example of generating selection pulses SEL1 and SEL2 for switching between a plurality of memory cells in the recharge control unit 32 that generates the reset pulse RST. Similar to the fourth embodiment, also in this example, the switching of the cycle of the recharge control can be synchronized with the operation of selecting one of the plurality of memory cells.

[0129] [Pixel Circuit Configuration Example] Fig.19 : is a block diagram showing a circuit configuration example of the pixel 2 of the image pickup element 1 according to the seventh embodiment of the present technology.

[0130] The pixel 2 of the imaging element 1 according to the seventh embodiment has a structure in which a selection pulse generating unit 73 generates selection pulses SEL1 and SEL2 for selecting the first storage unit 61 or the second storage unit 62, and the other structures are basically the same as those of the seventh embodiment, that is, the first storage unit 61 and the second storage unit 62 are constructed using counters.

[0131] Specifically, in the pixel 2 of the image pickup element 1 according to the seventh embodiment, the recharging control unit 32 generates a plurality of cycles of the reset signal RST based on four types of switching signals A, B, C, and D for switching the recharging cycle supplied from the recharging cycle switching control unit 90. In other words, in the recharging control unit 32, switching of the cycle of the reset signal RST to be generated is performed based on the recharging cycle switching signals A, B, C, and D.

[0132] The recharge control unit 32 further generates selection pulses SEL1 and SEL2 to select the first storage unit 61 or the second storage unit 62 as a storage unit for storing (counting) the number of photon detections based on the recharge cycle switching signals A, B, C, and D supplied from the recharge cycle switching control unit 90. The selection pulses SEL1 and SEL2 generated by the recharge control unit 32 are supplied to the first AND circuit 71 and the second AND circuit 72.

[0133] In this way, in the recharging control unit 32 that generates the reset pulse RST, by generating selection pulses SEL1 and SEL2 for switching between multiple storage cells, that is, switching between the first storage cell 61 and the second storage cell 62, the switching of the recharging control cycle can be synchronized with the operation of selecting the first storage cell 61 or the second storage cell 62.

[0134] [Circuit Operation Example of Pixel] An example of circuit operation of the pixel 2 according to the seventh embodiment configured as described above will be described. Fig. 20 1 is a timing chart for explaining a circuit operation example of the pixel 2 of the image pickup element 1 according to the seventh embodiment.

[0135] Fig. 20The timing relationship of the reset pulse RST, the cathode voltage Vk of the SPAD element 10, the detection pulse DET, the recharging cycle switching signals A, B, C, D, the selection pulses SEL1 and SEL2, the first detection pulse DET1 and the second detection pulse DET2, and the first count value CNT1 and the second count value CNT2 relative to the incident photon is shown.

[0136] like Fig. 20 As shown in the timing chart of , in the image pickup element 1 according to the seventh embodiment, in the recharging control unit 32, four types of recharging cycles of cycle A, cycle B, cycle C, and cycle D are set as the cycles of the reset pulse RST based on four types of recharging cycle switching signals A, B, C, and D. Specifically, cycle A is set during the high level period of the switching signal A, cycle B is set during the high level period of the switching signal B, cycle C is set during the high level period of the switching signal C, and cycle D is set during the high level period of the switching signal D. Here, for example, the length relationship between the cycles of the reset pulse RST is cycle A>cycle B>cycle C>cycle D.

[0137] Furthermore, in the recharging control unit 32, a high-level selection pulse SEL1 is generated during a period in which the recharging cycle switching signals A and C are at a high level, and a high-level selection pulse SEL2 is generated during a period in which the recharging cycle switching signals B and D are at a high level. As a result, logic switching of the selection pulses SEL1 and SEL2 is performed when the recharging cycle is switched.

[0138] Then, the respective counters constituting the first storage unit 61 and the second storage unit 62 count the number of photon detections by repeating switching between the first storage unit 61 and the second storage unit 62 under the control of the selection pulses SEL1 and SEL2. The count values ​​CNT1 and CNT2 of the first storage unit 61 and the second storage unit 62 are output as pixel signals corresponding to the number of photons detected in a predetermined exposure period.

[0139] As described above, in the image pickup element 1 according to the seventh embodiment, the recharge control unit 32 that generates the reset pulse RST performs control by generating the selection pulses SEL1 and SEL2 for switching between the plurality of storage units based on the recharge cycle switching signals A, B, C, and D so that the switching of the cycle of the reset pulse RST is synchronized with the operation of selecting the first storage unit 61 or the second storage unit 62. As a result, at the pulse generation timing in the recharge control unit 32, the switching timing of the recharge cycle and the switching timing of the first storage unit 61 and the second storage unit 62 can be made consistent.

[0140] <Eighth Implementation Plan> The eighth embodiment of the present technology is an example in which selection pulses SEL1 and SEL2 for switching between a plurality of memory cells are generated outside the pixel array unit.

[0141] Fig.21 : is a block diagram showing a configuration example of an image pickup element 1 according to an eighth embodiment of the present technology. Figure 1 In the stacked semiconductor chip structure shown, on the second semiconductor substrate 82, the detection circuit 20 for detecting photons incident on the SPAD elements 10 arranged in an array on the first semiconductor substrate 81 is arranged in an array with the SPAD elements 10 as units of pixels 2 to form a pixel array unit 3.

[0142] On the second semiconductor substrate 82, the recharging control unit 32 that generates the reset pulse RST, the selection pulse generating unit 73 that generates the selection pulses SEL1 and SEL2, and the recharging period switching control unit 90 that generates the recharging period switching signals A, B, C, and D are arranged outside the area of ​​the pixel array unit 3. The reset pulse RST generated by the recharging control unit 32 and the selection pulses SEL1 and SEL2 generated by the selection pulse generating unit 73 are provided to each detection circuit 20 in the pixel array unit 3.

[0143] The configuration illustrated here for generating the reset pulse RST and the selection pulses SEL1 and SEL2 based on the recharge cycle switching signals A, B, C and D corresponds to Fig.17 The same applies to the ninth embodiment described later.

[0144] As described above, in the image pickup element 1 according to the eighth embodiment, the selection pulse generating unit 73 is arranged outside the region of the pixel array unit 3, and the selection pulses SEL1 and SEL2 for switching between the plurality of memory cells, that is, selecting the first memory cell 61 or the second memory cell 62 are generated outside the pixel array unit 3, and the selection pulses SEL1 and SEL2 are supplied to the detection circuit 20 of each pixel 2. Therefore, compared with the case where the selection pulse generating unit 73 is arranged inside the pixel 2, the circuit area of ​​the region of the detection circuit 20 arranged in units of the pixel 2 on the second semiconductor substrate 82 can be reduced to be smaller.

[0145] <Ninth Implementation Plan> The ninth embodiment of the present technology is an example in which selection pulses SEL1 and SEL2 for switching between a plurality of memory cells are generated within a pixel array unit.

[0146] Fig. 22 : is a block diagram showing a configuration example of an image pickup element 1 according to a ninth embodiment of the present technology. Figure 1In the stacked semiconductor chip structure shown, on the second semiconductor substrate 82, the detection circuit 20 for detecting photons incident on the SPAD elements 10 arranged in an array on the first semiconductor substrate 81 is arranged in an array in units of pixels 2 corresponding to the SPAD elements 10, so as to constitute a pixel array unit 3 together with the selection pulse generating unit 73.

[0147] On the second semiconductor substrate 82, the recharge control unit 32 that generates the reset pulse RST and the recharge cycle switching control unit 90 that generates the recharge cycle switching signals A, B, C, and D are arranged outside the area of ​​the pixel array unit 3. The reset pulse RST generated by the recharge control unit 32 is provided to each detection circuit 20 in the pixel array unit 3. The recharge cycle switching signals A, B, C, and D generated by the recharge cycle switching control unit 90 are provided to each selection pulse generating unit 73 in the pixel array unit 3.

[0148] As described above, in the image pickup element 1 according to the ninth embodiment, the selection pulse generating unit 73 is arranged in the pixel array unit 3 in units of pixels 2, that is, in pairs with the detection circuit 20, and generates the selection pulses SEL1 and SEL2 for selecting the first storage unit 61 or the second storage unit 62 in each pixel 2 within the pixel array unit 3, and supplies the selection pulses SEL1 and SEL2 to the detection circuit 20. Therefore, compared with the case where the selection pulse generating unit 73 is arranged outside the pixel array unit 3, the circuit area of ​​the region on the second semiconductor substrate 82 where the circuit outside the pixel array unit 3 is formed can be reduced to be smaller.

[0149] <Tenth Implementation Plan> The tenth embodiment of the present technology is an example of obtaining an image output based on each output value of a plurality of storage units. The arithmetic processing to obtain the image output may be performed in the imaging element 1, or may be performed by, for example, an application processor outside the imaging element 1. The same applies to the embodiments described later.

[0150] Fig.23 1 is a block diagram showing a circuit configuration example of an arithmetic processing unit according to a tenth embodiment of the present technology. In the tenth embodiment, arithmetic operation of image output is performed based on each output value of a plurality of storage units, for example, an output value out1 of a first storage unit 61 and an output value out2 of a second storage unit 62. The output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62 are hold values ​​REG1 and REG2 when the first storage unit 61 and the second storage unit 62 are configured using memories, and are count values ​​CNT1 and CNT2 when the first storage unit 61 and the second storage unit 62 are configured using counters.

[0151] [Arithmetic Circuit Example 1] Fig.23 1 shows a circuit configuration example of an arithmetic processing unit according to arithmetic circuit example 1. For example, arithmetic circuit example 1 includes an arithmetic unit 91 configured using a central processing unit (CPU). The arithmetic unit 91 calculates an image output based on each output value of a plurality of storage units (in this example, the output value out1 of the first storage unit 61 and the output value out2 of the second storage unit 62).

[0152] According to the circuit configuration example of Arithmetic Circuit Example 1, the image output calculated based on each output value of the plurality of storage cells can be used as an image pickup result as it is.

[0153] [Arithmetic Circuit Example 2] Fig.23 'b' shows a circuit configuration example of an arithmetic processing unit according to arithmetic circuit example 2. Similar to arithmetic circuit example 1, arithmetic circuit example 2 includes an arithmetic unit 91 configured using, for example, a CPU. The arithmetic unit 91 calculates a plurality of image outputs (in this example, two image outputs 1 and 2) based on each output value of a plurality of storage units (in this example, an output value out1 of the first storage unit 61 and an output value out2 of the second storage unit 62).

[0154] According to the circuit configuration example of arithmetic circuit example 2, a plurality of images can be acquired from a plurality of image outputs (in this example, two image outputs 1 and 2) calculated based on each output value of a plurality of storage units, or an image can be acquired by arbitrarily combining images.

[0155] <Eleventh Implementation Plan> The eleventh embodiment of the present technology is an example of obtaining an average illuminance and estimating the incident illuminance by taking into account the illuminance change during the exposure period. Specifically, in the eleventh embodiment, the objects for estimating the illuminance change are limited to the two illuminances of the first half and the second half within one exposure period (see Figure 3 ), and then estimates the incident illumination that best matches the combination of the outputs of multiple storage units.

[0156] Fig.24 It is a diagram for explaining an arithmetic processing unit according to an eleventh embodiment of the present technology. Fig.24 a is a block diagram showing a circuit configuration example of an arithmetic processing unit, and Fig.24 b is a characteristic diagram showing the relationship between the incident photon rate f and the count values ​​CNT1 and CNT2. In the case where the count values ​​CNT1 and CNT2 are constructed using a counter, the count values ​​CNT1 and CNT2 are output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62.

[0157] exist Fig.24 In the characteristic diagram shown in b, the broken line curves g1(f) and g2(f) are known input / output characteristics obtained from a table created according to the setting of the cycle of the recharging control (recharging cycle).

[0158] like Fig.24 As shown in a, for example, the arithmetic processing unit according to the eleventh embodiment includes an arithmetic unit 91 formed using a CPU, a calculation unit 92-1 for the incident illuminance f1 of the first half of the exposure period, a calculation unit 92-2 for the incident illuminance f2 of the second half of the exposure period, and an average value calculation unit 93.

[0159] Similar to arithmetic circuit example 2 in the tenth embodiment, the arithmetic unit 91 calculates multiple image outputs (in this example, two image outputs 1 and 2) based on each output value of multiple storage units (in this example, the output value out1 of the first storage unit 61 and the output value out2 of the second storage unit 62).

[0160] The calculation unit 92-1 for the incident illuminance in the first half of the exposure period calculates the incident illuminance f1 in the first half of the exposure period based on the image output 1 output from the arithmetic unit 91, and the calculation unit 92-2 for the incident illuminance in the second half of the exposure period calculates the incident illuminance f2 in the second half of the exposure period based on the image output 2 output from the arithmetic unit 91.

[0161] The average value calculation unit 93 obtains the average value of the incident illuminance f1 of the first half of the exposure period calculated by the calculation unit 92-1 for the incident illuminance of the first half of the exposure period and the incident illuminance f2 of the second half of the exposure period calculated by the calculation unit 92-2 for the incident illuminance of the second half of the exposure period. The average value obtained by the average value calculation unit 93 is output as the estimated illuminance that best matches the combination of the output values ​​of the plurality of storage units (in this example, the output value out1 of the first storage unit 61 and the output value out2 of the second storage unit 62).

[0162] Fig.25 is a flowchart showing a specific processing flow of the arithmetic processing unit according to the eleventh embodiment of the present technology.

[0163] First, the output value out1 of the first storage unit 61 and the output value out2 of the second storage unit 62 are set as input values ​​(step S11), and then initial values ​​of the incident illuminance f1 in the first half of the exposure period and the incident illuminance f2 in the second half of the exposure period are provided (step S12).

[0164] Next, the estimated values ​​c1 and c2 of the output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62 with respect to the incident illuminance f1 of the first half of the exposure period and the incident illuminance f2 of the second half of the exposure period are calculated (step S13). The arithmetic processing of calculating the estimated values ​​c1 and c2 can be performed based on the calculation equation of Formula 1 shown in step S13. Here, in the case where the recharging cycle is represented by τ and the number of repetitions of repeated recharging within the exposure period is represented by n, the known input / output characteristic g(f) with respect to the incident photon rate f is as follows: g(f)=(1-e f·τ )·n

[0165] Subsequently, the evaluation value L is calculated based on the estimated values ​​c1 and c2 calculated in step S13 and the actually obtained output values ​​out1 and out2 of the first and second storage units 61 and 62 (step S14). The arithmetic processing for calculating the evaluation value L can be performed based on the calculation equation of Formula 2 shown in step S14.

[0166] Next, it is determined whether the evaluation value L is below the minimum or predetermined reference value (step S15), and if the evaluation value L is not below the minimum or predetermined reference value (No in S15), the incident illuminances f1 and f2 are updated (step S16), after which the process returns to step S13. In addition, in the case where the evaluation value L is below the minimum or predetermined reference value (Yes in S15), that is, in the case where the evaluation value L is sufficiently small, it is determined that the estimated values ​​c1 and c2 substantially match the output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62, and assuming that the values ​​of the incident illuminances f1 and f2 are correct, the average value thereof is obtained (step S17), and a series of arithmetic processing is terminated.

[0167] As described above, in the arithmetic processing unit according to the eleventh embodiment of the present technology, based on each output value of a plurality of storage units (in this example, the output value out1 of the first storage unit 61 and the output value out2 of the second storage unit 62), the following processing is performed: the estimated illuminance (estimated incident illuminance) of a period obtained by dividing one exposure period by the number of storage units is obtained. In this way, by obtaining the average illuminance in consideration of the illuminance change in the exposure period, it is possible to cope with the illuminance change of a fast-moving subject.

[0168] <Twelfth Implementation Plan> The twelfth embodiment of the present technology is an example of obtaining an estimated illuminance that best matches a combination of output values ​​of a plurality of storage cells using an inverse function table prepared in advance.

[0169] Fig.26 It is a diagram for explaining an arithmetic processing unit according to a twelfth embodiment of the present technology. Fig.26 a is a block diagram showing a circuit configuration example of an arithmetic processing unit, and Fig.26 b is a characteristic diagram showing the relationship between the incident photon rate f and the count values ​​CNT1 and CNT2. In the case where the count values ​​CNT1 and CNT2 are constructed using a counter, the count values ​​CNT1 and CNT2 are output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62.

[0170] For example, in the twelfth embodiment, the arithmetic unit 91 constituted using the CPU uses the output values ​​out1 and out2 of the first storage unit 61 and the second storage unit 62 as inputs, and has g with respect to these inputs. -1 Inverse conversion table of (n1-n2). Fig.26 In b, g(f) is the input / output characteristic that can be calculated according to the setting of the recharging cycle.

[0171] Here, where the recharging period is represented by τ1 and τ2 and the number of repetitions is represented by n1 and n2, the input / output characteristic g(f) with respect to the incident photon rate f is as follows: g(f)=(1-e f·τ1 )·n1+(1-e f·τ2 )·n2 For g -1 , the inverse function of g is stored in advance as a table in the arithmetic unit 91.

[0172] As described above, in the arithmetic processing unit according to the twelfth embodiment of the present technology, a process of obtaining an estimated illuminance that best matches a combination of output values ​​of a plurality of storage units using a pre-prepared inverse function table is performed, and more specifically, a process of obtaining an average illuminance without considering illuminance changes during exposure is performed. By this simple process, an image output proportional to the incident illuminance can be obtained for a still image.

[0173] <Modification> Note that the above-mentioned embodiments show examples of embodying the present technology, and the matters in the embodiments and the matters specifying the present invention in the claims have a corresponding relationship. Similarly, the matters specifying the present invention in the claims and the matters with the same names in the embodiments of the present technology have a corresponding relationship. However, the present technology is not limited to these embodiments, and the present technology can be embodied by various modifications of the embodiments without departing from its spirit.

[0174] <Application examples of electronic equipment> The imaging element according to the above-mentioned embodiment of the present technology can be applied to various electronic devices with imaging functions, for example, imaging devices such as digital cameras or video cameras, mobile terminal devices with imaging functions such as mobile phones, and copiers using imaging devices in image reading units.

[0175] [Example of an imaging device] Fig. 27 This is a block diagram showing a configuration example of an imaging device as an example of an electronic device to which the present technology is applied.

[0176] The imaging device 100 according to the present application example is a device for capturing an image of a subject, and includes an imaging optical system 101 including a lens group and the like, an imaging unit 102, a digital signal processor (DSP) circuit 103, a display unit 104, an operation unit 105, a storage unit 106, and a power supply unit 107. These elements are connected to each other via a bus 108. As the imaging device 100, for example, in addition to digital cameras such as digital cameras, smartphones and personal computers having an imaging function, car-mounted cameras, and the like are also envisioned.

[0177] The imaging unit 102 generates pixel data by photoelectric conversion. As the imaging unit 102, the imaging element in the embodiment of the present technology can be used. Light from the subject is converged by the imaging optical system 101 arranged on the incident light side and guided to the light receiving surface of the imaging unit 102. The imaging unit 102 provides the pixel data generated by the photoelectric conversion to the DSP circuit 103 in the subsequent stage.

[0178] The DSP circuit 103 performs predetermined signal processing on the pixel data from the image pickup unit 102. The display unit 104 displays the pixel data. As the display unit 104, for example, a liquid crystal panel or an organic electroluminescent (EL) panel is envisioned. The operation unit 105 generates an operation signal according to the user's operation. The storage unit 106 stores various types of data such as pixel data. The power supply unit 107 supplies power to the image pickup unit 102, the DSP circuit 103, the display unit 104, and the like.

[0179] In the imaging device 100 having the above-described structure, since the frame rate can be significantly improved by the number of storage units in multiple cycles of recharging control by using the imaging element according to the embodiment of the present technology as the imaging unit 102, a imaging image with higher image quality can be obtained.

[0180] <Application examples of the present technical implementation scheme> The image pickup element according to the above-described embodiment of the present technology can be applied to various technologies as exemplified below.

[0181] Fig.28Examples of fields to which the image pickup element according to the embodiment of the present technology is applied are shown.

[0182] For example, the image pickup element according to the embodiment of the present technology can be used as a device that captures images for appreciation, such as a digital camera or a portable device having a camera function.

[0183] In addition, the imaging element can be used as equipment for traffic, such as an on-board sensor for capturing images of the surroundings and interior of a car, a surveillance camera for monitoring moving vehicles and roads, and a ranging sensor for measuring distances between vehicles, etc., to achieve safe driving such as automatic parking, driver condition recognition, etc.

[0184] Furthermore, the imaging element can be used as a device for home appliances such as televisions, refrigerators, and air conditioners to capture images of user gestures and perform device operations according to the gestures.

[0185] Furthermore, the imaging element can be used as equipment for medical care, such as an endoscope and equipment for performing angiography by receiving infrared light.

[0186] Furthermore, the imaging element can be used as equipment for security, such as a security surveillance camera and a personal authentication camera.

[0187] Furthermore, the imaging element can be used as an apparatus for beauty care, such as a skin measuring instrument for imaging the skin and a microscope for imaging the scalp.

[0188] Furthermore, the imaging element can be used as a device for sports, such as a sports camera or a wearable camera for sports applications.

[0189] Furthermore, the imaging element can be used as equipment for agriculture, such as a camera for monitoring the conditions of fields or crops.

[0190] <Application example of mobile body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure can be implemented as a device equipped on any type of mobile body such as a car, an electric car, a hybrid car, a two-wheeled automatic vehicle, a bicycle, a personal mobile device, an airplane, an unmanned aerial vehicle, a ship, a robot, etc.

[0191] Fig.29 : is a block diagram showing a schematic configuration example of a vehicle control system as an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0192] The vehicle control system 12000 includes a plurality of electronic control units interconnected via a communication network 12001. Fig.29In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown as functional components of the integrated control unit 12050.

[0193] The drive system control unit 12010 controls the operation of the devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 functions as a control device for the following devices: a drive force generating device such as an internal combustion engine, a drive motor, etc. for generating a drive force for the vehicle; a drive force transmitting mechanism for transmitting the drive force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; a braking device for generating a braking force for the vehicle, etc.

[0194] The body system control unit 12020 controls the operation of various devices equipped on the body according to various programs. For example, the body system control unit 12020 functions as a control device for the following devices: a keyless entry system; a smart key system; a power window device; or various lights such as headlights, reverse lights, brake lights, turn signal lights, fog lights, etc. In this case, radio waves or signals of various switches emitted from a portable device that replaces the key may be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals and controls the door lock device, power window device, lights, etc. of the vehicle.

[0195] The vehicle exterior information detection unit 12030 detects information outside the vehicle including the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to the camera unit 12031. The vehicle exterior information detection unit 12030 causes the camera unit 12031 to capture an image of the vehicle exterior and receives the captured image. Based on the received image, the vehicle exterior information detection unit 12030 may perform a process of detecting objects such as a person, a vehicle, an obstacle, a sign, text on the road surface, or a process of detecting the distance to the above-mentioned object.

[0196] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 may output the electrical signal as an image, or may output the electrical signal as distance measurement information. In addition, the light received by the imaging unit 12031 may be visible light, or may be non-visible light such as infrared rays.

[0197] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. For example, the in-vehicle information detection unit 12040 is connected to a driver state detection unit 12041 that detects the state of the driver. For example, the driver state detection unit 12041 includes a camera that takes a picture of the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's fatigue level or the driver's concentration level, or can determine whether the driver is dozing off.

[0198] Based on the information outside or inside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, the microcomputer 12051 can calculate the control target value of the driving force generating device, the steering mechanism or the braking device, and output the control instruction to the driving system control unit 12010. For example, the microcomputer 12051 can perform coordinated control for realizing the functions of the advanced driver assistance system (ADAS), which includes collision avoidance or impact mitigation of the vehicle, following driving based on the vehicle-to-vehicle distance, vehicle constant speed driving, vehicle collision warning, vehicle lane departure warning, etc.

[0199] In addition, based on the information outside or inside the vehicle obtained by the external information detection unit 12030 or the internal information detection unit 12040, the microcomputer 12051 can perform coordinated control such as automatic driving aimed at enabling the vehicle to drive autonomously without the need for driver operation by controlling the driving force generating device, steering mechanism, braking device, etc.

[0200] Furthermore, based on the information outside the vehicle obtained by the vehicle exterior information detection unit 12030, the microcomputer 12051 may output a control instruction to the body system control unit 12020. For example, based on the position of the preceding vehicle or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, the microcomputer 12051 may perform coordinated control such as realizing anti-glare by controlling the headlights to switch from high beam to low beam.

[0201] The sound / image output unit 12052 sends an output signal of at least one of sound and image to an output device capable of visually or auditorily notifying the vehicle occupants or the outside of the vehicle of information. Fig.29 In the example of , an audio speaker 12061, a display portion 12062, and an instrument panel 12063 are shown as output devices. For example, the display portion 12062 may include at least one of an onboard display and a head-up display.

[0202] Fig.30 This is a diagram showing an example of the installation position of the camera unit 12031.

[0203] exist Fig.30 In the figure, the camera unit 12031 includes camera units 12101, 12102, 12103, 12104 and 12105.

[0204] For example, the camera units 12101, 12102, 12103, 12104, and 12105 are disposed at locations such as the front nose, side mirrors, rear bumper, rear door, and upper portion of the windshield in the vehicle 12100. The camera unit 12101 disposed at the front nose and the camera unit 12105 disposed at the upper portion of the windshield in the vehicle mainly acquire images in front of the vehicle 12100. The camera units 12102 and 12103 disposed at the side mirrors mainly acquire images on the sides of the vehicle 12100. The camera unit 12104 disposed at the rear bumper or rear door mainly acquires images at the rear of the vehicle 12100. The camera unit 12105 disposed at the upper portion of the windshield in the vehicle is mainly used to detect vehicles in front, pedestrians, obstacles, signal lights, traffic signs, lanes, and the like.

[0205] Notice, Fig.30 An example of the imaging range of the imaging units 12101 to 12104 is shown. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided at the front nose. The imaging ranges 12112 and 12113 respectively indicate the imaging ranges of the imaging units 12102 and 12103 provided at the side mirrors. The imaging range 12114 indicates the imaging range of the imaging unit 12104 provided at the rear bumper or the rear door. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above is obtained.

[0206] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0207] For example, based on the distance information obtained from the camera units 12101 to 12104, the microcomputer 12051 can determine the distance to each solid object within the camera ranges 12111 to 12114 and the change in the distance over time (relative speed relative to the vehicle 12100), so that the following solid object can be extracted as the front vehicle: it is particularly the closest solid object on the driving path of the vehicle 12100, and is a solid object that travels at a predetermined speed (for example, 0 km / h or more) in the same direction as the vehicle 12100. In addition, the microcomputer 12051 can pre-set the inter-vehicle distance to be ensured with the front vehicle, and can perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. Therefore, it is possible to perform coordinated control for realizing automatic driving, etc., which enables the vehicle to travel autonomously without the driver's operation.

[0208] For example, based on the distance information obtained from the camera units 12101 to 12104, the microcomputer 12051 can classify the three-dimensional object data of the three-dimensional object into the three-dimensional object data of two-wheeled vehicles, ordinary cars, large vehicles, pedestrians, electric poles and other three-dimensional objects, extract the classified three-dimensional object data, and use the extracted three-dimensional object data to automatically avoid obstacles. For example, the microcomputer 12051 distinguishes the obstacles around the vehicle 12100 into obstacles that can be visually identified by the driver of the vehicle 12100 and obstacles that are difficult for the driver of the vehicle 12100 to visually identify. Then, the microcomputer 12051 determines the collision risk indicating the danger of collision with each obstacle. In the case where the collision risk is equal to or greater than the set value and there is a possibility of collision, the microcomputer 12051 issues a warning to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or evasive steering via the drive system control unit 12010. Therefore, the microcomputer 12051 can assist driving to avoid collision.

[0209] At least one of the camera units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 may identify a pedestrian by determining whether there is a pedestrian in the camera images of the camera units 12101 to 12104. For example, such identification of pedestrians is performed by the following process: a process of extracting feature points from the camera images of the camera units 12101 to 12104 as infrared cameras; and a process of performing pattern matching processing on a series of feature points representing the contour of the object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that there is a pedestrian in the camera images of the camera units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 to display a square contour line for emphasis in a superimposed manner on the identified pedestrian. The sound / image output unit 12052 may also control the display unit 12062 to display an icon or the like for indicating a pedestrian at a desired position.

[0210] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. For example, the technology according to the present disclosure can be applied to the camera unit 12031 in the above-mentioned configuration. Specifically, the noise correction circuit included in the embodiment of the technology according to the present disclosure Figure 1 The imaging element 10 in the embodiment can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, the frame rate can be significantly improved by the number of storage units in the recharging control of multiple cycles to obtain a camera image with higher image quality, thereby reducing the fatigue of the driver.

[0211] <Configurations that can be adopted by this technology> Note that the present technology may also have the following configurations. (1) An imaging element comprising: a light receiving element which is provided for each pixel and detects the presence or absence of a photon; and a detection circuit which is provided corresponding to the light receiving element and detects photons incident on the light receiving element, wherein: The detection circuit comprises: a photon detection unit that detects photons incident on the light receiving element by performing a recharging control on the light receiving element for a plurality of cycles, multiple storage units, and A selection control unit selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit. (2) The imaging element according to (1), wherein: The detection circuit further includes a counter that counts pulses generated when the photon detection unit detects photons. The plurality of storage units are formed using memories, each of which stores a count value of the counter, and The selection control unit is disposed between the counter and the plurality of storage units, and performs control to selectively input the count value of the counter to the plurality of storage units. (3) The imaging element according to (2), wherein: The detection circuit further includes a read control unit configured to read a held value of one of the plurality of storage cells during a holding period of another storage cell. (4) The imaging element according to (1), wherein: The plurality of storage units are constituted using counters, each of which counts pulses generated when the photon detection unit detects a photon, and The selection control unit is arranged between the photon detection unit and the plurality of storage units, and performs control to selectively input the photon detection number to the plurality of storage units. (5) The imaging element according to (4), wherein: Each counter of the plurality of storage units repeats a period in which a counting operation is performed and a period in which a counting operation is not performed a plurality of times. (6) The imaging element according to any one of (1) to (5), wherein The selection control unit performs an operation of selecting one of the plurality of storage cells in synchronization with switching of a cycle of the recharging control. (7) The imaging element according to (6), wherein: The selection control unit generates a selection pulse for selecting one of the plurality of memory cells, and The photon detection unit performs the plurality of cycles of recharge control in synchronization with the selection pulse. (8) The imaging element according to (6), wherein: The detection circuit further includes a recharging cycle switching control unit, the recharging cycle switching control unit switching the cycle of the recharging control, and The selection control unit generates a selection pulse for selecting one of the plurality of memory cells in synchronization with a switching signal output from the recharge cycle switching control unit. (9) The imaging element according to (6), wherein: The detection circuit further includes a recharging cycle switching control unit, the recharging cycle switching control unit switching the cycle of the recharging control, and The photon detection unit generates a selection pulse for selecting one of the plurality of memory cells based on a switching signal output from the recharge period switching control unit. (10) The imaging device according to any one of (1) to (9), wherein The imaging element has a stacked structure in which at least two semiconductor substrates are stacked, and the at least two semiconductor substrates include: a first semiconductor substrate on which the light receiving element is formed, and The second semiconductor substrate has the detection circuit formed thereon. (11) The imaging element according to (10), wherein: The imaging element further includes a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage cells, and The selection pulse generating unit is arranged outside a pixel array unit in which the detection circuits are arranged in an array on the second semiconductor substrate. (12) The imaging element according to (10), wherein: The imaging element further includes a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage cells, and The selection pulse generating unit is arranged in a pixel array unit in which the detection circuits are arranged in an array shape on the second semiconductor substrate, and the selection pulse generating unit is arranged in a pair with each of the detection circuits. (13) The imaging element according to any one of (1) to (12), wherein: The image pickup element further includes an arithmetic processing unit that obtains an image output based on each output value of the plurality of storage units. (14) The imaging element according to (13), wherein: The arithmetic processing unit obtains incident illuminance in a plurality of exposure periods from each output value of the plurality of storage units, and obtains an average illuminance of the incident illuminance in the plurality of exposure periods. (15) The imaging element according to any one of (1) to (14), wherein The light receiving element is an avalanche photodiode. (16) The imaging element according to (15), wherein: The light receiving element is a single photon avalanche diode. (17) An electronic device including an imaging element, wherein: The imaging element comprises: a light receiving element which is provided for each pixel and detects the presence or absence of a photon; and a detection circuit which is provided corresponding to the light receiving element and detects photons incident on the light receiving element, The detection circuit comprises: a photon detection unit that detects photons incident on the light receiving element by performing a recharging control on the light receiving element for a plurality of cycles, multiple storage units, and A selection control unit selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit. Reference numerals list

[0212] 1 Camera element 2 pixels 3 pixel array unit 10SPAD element (light receiving element) 20 Detection circuit 30-photon detection unit 31 Recharge transistor 32 Recharging control unit 33 Waveform shaping unit 40 counters 50 Holding pulse generation unit 60 storage units 61 First storage unit 62 Second storage unit 63 Read control unit 70 Select control unit 71 First AND circuit 72 Second AND circuit 73 Select pulse generation unit 81 first semiconductor substrate 82 second semiconductor substrate 83 wiring layer 84 connection 90 recharge cycles switch control unit 91 Arithmetic Unit 92-1 Calculation unit for incident illumination during the first half of the exposure period 92-2 Calculation unit for incident illumination during the second half of the exposure period 93Average calculation unit

Claims

1. An imaging element, comprising: a light receiving element which is provided for each pixel and detects the presence or absence of a photon; and a detection circuit which is provided corresponding to the light receiving element and detects photons incident on the light receiving element, wherein: The detection circuit comprises: a photon detection unit that detects photons incident on the light receiving element by performing a recharging control on the light receiving element for a plurality of cycles, multiple storage units, and A selection control unit selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit.

2. The imaging element according to claim 1, wherein: The detection circuit further includes a counter that counts pulses generated when the photon detection unit detects photons. The plurality of storage units are formed using memories, each of which stores a count value of the counter, and The selection control unit is disposed between the counter and the plurality of storage units, and performs control to selectively input the count value of the counter to the plurality of storage units.

3. The imaging element according to claim 2, wherein: The detection circuit further includes a read control unit configured to read a held value of one of the plurality of memory cells during a holding period of another memory cell.

4. The imaging element according to claim 1, wherein: The plurality of storage units are constituted using counters, each of which counts pulses generated when the photon detection unit detects a photon, and The selection control unit is arranged between the photon detection unit and the plurality of storage units, and performs control to selectively input the photon detection number to the plurality of storage units.

5. The imaging element according to claim 4, wherein: Each counter of the plurality of storage units repeats a period in which a counting operation is performed and a period in which a counting operation is not performed a plurality of times.

6. The imaging element according to claim 1, wherein: The selection control unit performs an operation of selecting one of the plurality of storage cells in synchronization with switching of a cycle of the recharging control.

7. The imaging element according to claim 6, wherein: The selection control unit generates a selection pulse for selecting one of the plurality of memory cells, and The photon detection unit performs the plurality of cycles of recharge control in synchronization with the selection pulse.

8. The imaging element according to claim 6, wherein: The detection circuit further includes a recharging cycle switching control unit, the recharging cycle switching control unit switching the cycle of the recharging control, and The selection control unit generates a selection pulse for selecting one of the plurality of memory cells in synchronization with a switching signal output from the recharge cycle switching control unit.

9. The imaging element according to claim 6, wherein: The detection circuit further includes a recharging cycle switching control unit, the recharging cycle switching control unit switching the cycle of the recharging control, and The photon detection unit generates a selection pulse for selecting one of the plurality of memory cells based on a switching signal output from the recharge period switching control unit.

10. The imaging element according to claim 1, wherein The imaging element has a stacked structure in which at least two semiconductor substrates are stacked, and the at least two semiconductor substrates include: a first semiconductor substrate on which the light receiving element is formed, and The second semiconductor substrate has the detection circuit formed thereon.

11. The imaging element according to claim 10, wherein: The imaging element further includes a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage cells, and The selection pulse generating unit is arranged outside a pixel array unit in which the detection circuits are arranged in an array on the second semiconductor substrate.

12. The imaging element according to claim 10, wherein: The imaging element further includes a selection pulse generating unit that generates a selection pulse for selecting one of the plurality of storage cells, and The selection pulse generating unit is arranged in a pixel array unit in which the detection circuits are arranged in an array shape on the second semiconductor substrate, and the selection pulse generating unit is arranged in a pair with each of the detection circuits.

13. The imaging element according to claim 1, wherein: The image pickup element further includes an arithmetic processing unit that obtains an image output based on each output value of the plurality of storage units.

14. The imaging element according to claim 13, wherein: The arithmetic processing unit obtains incident illuminance in a plurality of exposure periods from each output value of the plurality of storage units, and obtains an average illuminance of the incident illuminance in the plurality of exposure periods.

15. The imaging element according to claim 1, wherein: The light receiving element is an avalanche photodiode.

16. The imaging element according to claim 15, wherein: The light receiving element is a single photon avalanche diode.

17. An electronic device comprising an imaging element, wherein: The imaging element comprises: a light receiving element which is provided for each pixel and detects the presence or absence of a photon; and a detection circuit which is provided corresponding to the light receiving element and detects photons incident on the light receiving element, The detection circuit comprises: a photon detection unit that detects photons incident on the light receiving element by performing a recharging control on the light receiving element for a plurality of cycles, multiple storage units, and A selection control unit selects one of the plurality of storage units as a storage unit for storing the number of photon detections detected by the photon detection unit.

Images