Photoelectric conversion element, photoelectric conversion element control method, and information processing device

Abstract

To provide a photoelectric conversion element that detects an address event signal by using a photon counting sensor.SOLUTION: The photoelectric conversion element according to the present invention for solving the above-mentioned problem includes: first measuring means that measures a signal output from a pixel including conversion means for outputting the signal in response to incidence of photons; second measuring means that measures the time required for the signal measured by the first measuring means to satisfy a first threshold; first storage means that stores the past result of measurement performed by the second measuring means as a first time; and comparison means that compares the first time stored by the first storage means with a second time measured by the second measuring means.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a photoelectric conversion element. [Background technology]

[0002] There is known a photoelectric conversion element (hereinafter referred to as a photon counting sensor) that digitally counts the number of photons arriving at an avalanche photodiode (hereinafter referred to as an APD) and outputs the counted value from a pixel as a photoelectrically converted digital signal (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent No. 9210350 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to detect an address event signal using a photon counting sensor. [Means for solving the problem]

[0005] The photoelectric conversion element of the present invention, which solves the above-mentioned problems, is a pixel equipped with a conversion means for outputting a signal in response to the incidence of a photon, and includes: a first measurement means for measuring the signal output from the pixel; a second measurement means for measuring the time required for the signal measured by the first measurement means to satisfy a first threshold value; a first storage means for storing a past measurement result of the second measurement means as a first time; and a comparison means for comparing the first time stored by the first storage means with a second time measured by the second measurement means. an output means for outputting a signal in response to a request to output address event data when it is determined based on a comparison result by the comparison means that the difference or ratio between the first time and the second time is greater than a second threshold value; The present invention is characterized by having the following. [Effects of the Invention]

[0006] A photon counting sensor can be used to detect the address event signal. [Brief explanation of the drawings]

[0007] [Figure 1] A block diagram showing an example of the functional configuration of an information processing device including a photoelectric conversion element. [Figure 2] FIG. 1 is a diagram showing an example of a stack structure of a photoelectric conversion element according to Embodiment 1. [Figure 3] FIG. 1 is a diagram showing an example of a sensor chip in the element of embodiment 1. [Figure 4] FIG. 1 is a diagram showing an example of a circuit chip in the element of embodiment 1. [Figure 5] FIG. 1 is a diagram showing an example of a pixel according to the first embodiment; [Figure 6] 1 is a timing chart illustrating a method for driving the element according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a pixel according to a second embodiment; [Figure 8] 10 is a flowchart illustrating a process executed by an information processing device. [Figure 9] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a photoelectric conversion element according to an embodiment of the present invention will be described with reference to the drawings. In this regard, elements having the same functions in all the drawings will be assigned the same numbers, and repeated description thereof will be omitted.

[0009] <Embodiment 1> <Information processing device> FIG. 1 shows an example of the functional configuration of an information processing device 1000 including a photoelectric conversion element according to this embodiment. The information processing device 1000 includes a light receiving lens 1001, a photoelectric conversion element 100, a control unit 1002, a storage unit 1003, an image processing unit 1004, a display unit 1005, an analysis processing unit 1006, and an acquisition unit 1007. Specifically, the information processing device 1000 is an imaging device or a measurement device. The light receiving lens 1001 receives incident light and forms an image on the photoelectric conversion element 100. The photoelectric conversion element 100 outputs a signal corresponding to the received incident light. Details will be described later. The control unit 1002 controls the focus driving and aperture driving of the light receiving lens 1001, the driving of the photoelectric conversion element 100, etc. The storage unit 1003 stores, for example, signals output by the photoelectric conversion element 100 and image data processed by the image processing unit 1004. The information processing device 1000 also stores various settings related to the information processing device. The image data is processed based on the signal output by the photoelectric conversion element 100. The image processing unit 1004 determines pixel values ​​for each pixel of the image according to the signal and the measured time, and generates image data. While the information processing device 1000 has been described as an example in which the address event signal from the photoelectric conversion element 100 is converted into image data by the image processing unit 1004, the information processing device 1000 does not necessarily need to generate image data. The display unit 1005 displays the image and various information generated by the image processing unit 1004. The functions of the storage unit 1003 and the display unit 1005 may be external to the information processing device. Examples of other functional components not shown include an operation unit through which the user inputs various instructions such as image capture, and an audio output unit that outputs audio, such as video, as an output other than image display. The analysis processing unit 1006 performs predetermined analysis processing based on the address event signal output by the photoelectric conversion element. Specifically, this includes object detection, moving object detection, and the like. Detailed processing will be described later. The acquiring unit 1007 acquires the address event signal from the photoelectric conversion element 100 .

[0010] 2 is a diagram showing an example of the configuration of a photoelectric conversion element according to this embodiment. The photoelectric conversion element 100 is configured by stacking and electrically connecting two chips: a sensor chip 11 and a circuit chip 21. The sensor chip 11 includes a pixel region 12. The circuit chip 21 includes a pixel circuit region 22 that processes signals detected in the pixel region 12, and a readout circuit region 23 that reads out signals from the pixel circuit region 22.

[0011] <Pixel substrate> FIG. 3 is a diagram showing an example of the configuration of the sensor chip 11. The pixel region 12 of the sensor chip 11 includes a plurality of pixels 101 arranged two-dimensionally. Each pixel 101 has a photoelectric conversion unit 102 including an avalanche photodiode (APD). FIG. 3 shows 36 pixels 101 arranged in six rows from row 0 to row 5 and six columns from column 0 to column 5, along with reference symbols indicating the row and column numbers. For example, the unit pixel 11 arranged in the first row and fourth column is assigned the reference symbol "P14." The number of rows and columns of the pixel array constituting the pixel region 12 is not particularly limited.

[0012] <Circuit board> FIG. 4 is a diagram showing an example of the configuration of the circuit chip 21. The circuit chip 21 includes a pixel circuit region 22 and a readout circuit region 23. The pixel circuit region 22 includes a plurality of signal processing units 103 arranged two-dimensionally to correspond to each pixel 103 of the sensor chip 11. FIG. 4 shows 36 signal processing units 103 arranged in six rows (rows 0 to 5) and six columns (columns 0 to 5), along with reference numerals indicating the row and column numbers. For example, the signal processing unit 103 arranged in the first row and fourth column is designated by the reference numeral "S14." Note that the number of rows and columns of the signal processing unit array constituting the pixel circuit region 22 is not particularly limited. The readout circuit region 23 includes a vertical arbitration circuit 110, a column circuit 112, a horizontal readout circuit 111, and a signal output circuit 117.

[0013] <Vertical arbitration circuit> In each row of the signal processing unit array of the pixel circuit region 22, a request signal output line 114VREQ and a response input line 115VACT are arranged, extending in a first direction (the horizontal direction in FIG. 4). The request signal output line 114VREQ and the response input line 115VACT are respectively connected to the signal processing units 103 arranged in the first direction and form signal lines. The first direction in which the request signal output line 114VREQ and the response input line 115VACT extend may be referred to as the row direction or the horizontal direction. Note that in FIG. 4, the control lines VREQ and VACT are shown together with symbols indicating the row number. For example, the request signal line in the first row is assigned the symbol "VREQ[1]."

[0014] The control lines VREQ and VACT of each row are connected to a vertical arbitration circuit 110. The vertical arbitration circuit 110 supplies a control signal for driving the signal processing unit 103 to the signal processing unit 103 via a request signal output line 114VREQ and a response input line 115VACT.

[0015] The signal processing unit 103 outputs a request signal requesting the output of address event data to the vertical arbitration circuit 110 via a request signal output line 114VREQ. The vertical arbitration circuit 110 arbitrates the requests from the signal processing units 103 of each pixel, and returns a response indicating whether or not the output of address event data is permitted to the signal processing unit 103 via a response input line 115VACT.

[0016] <Signal output> In each column of the signal processing unit array of the pixel circuit region 22, a signal line 116 is arranged, extending in a second direction (vertical direction in FIG. 4) intersecting the first direction. The signal line 116 is connected to each of the signal processing units 103 arranged in the second direction, forming a common signal line. The second direction in which the signal line 116 extends may be referred to as the column direction or vertical direction. Note that in FIG. 4, the signal line 116 is shown together with a symbol indicating the column number. For example, the signal line 116 in the fourth column is assigned the symbol "POUT4."

[0017] Upon receiving an output permission response from the vertical arbitration circuit 110, each signal processing unit 103 outputs address event data from the signal processing unit 103 to the column circuit 112 via a signal line 116. The column circuits 112 are provided corresponding to each column of the signal processing unit array in the pixel circuit unit region 22, and are connected to the signal line 116 of the corresponding column. The column circuits 112 have a latch function that holds the signal read out from the signal processing unit 103 via the signal line 116 of the corresponding column.

[0018] <Horizontal readout> The horizontal readout circuit 111 supplies the column circuits 112 with control signals for reading out signals from the column circuits 112, and receives address event data from the column circuits 112 for each column. The signal output circuit 117 outputs the address event data measured at each pixel as an output signal SOUT. The address event data includes coordinate information for the unit pixel where a change in the number of incident photons per unit time occurred as an event, and information about the time when the change in the number of incident photons per unit time occurred. In addition, the address event data can also include the polarity (positive or negative) of the change in the number of incident photons per unit time. The method for counting address event data will be described later. By using address event data, it becomes possible to support high-speed processing that was not possible with conventional synchronous photoelectric conversion elements in use cases that require high-speed processing, such as robot or car control. (The advantage of minute brightness changes in SPADs will be described later.) <Pixel section> FIG. 5 is an example of an equivalent circuit and a block diagram of the pixel 101 in FIG. 3 and the signal processing unit 103 in FIG.

[0019] Each pixel 101 in the sensor chip 11 includes an APD 201, which is a photoelectric conversion unit. When light is incident on the APD 201, photoelectric conversion generates charge pairs corresponding to the incident light. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage is supplied to the anode and cathode so that the APD 201 performs avalanche multiplication. With such a voltage supplied, charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0020] The signal processing unit 103 in the sensor chip 21 includes a quench element 202, a waveform shaping unit 210, a first counter circuit 211, a second counter circuit 212, a first judgment circuit 213, a memory 214, a comparator 215, a second judgment circuit 216, a response circuit 217, and a selection circuit 218.

[0021] The quench element 202 is connected to a power supply that supplies a voltage VH and the APD 201. The quench element 202 has a function of converting a change in avalanche current generated in the APD 201 into a voltage signal. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and suppresses the voltage supplied to the APD 201 to suppress avalanche multiplication (quench operation).

[0022] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. The waveform shaping unit 210 may be, for example, an inverter circuit or a buffer circuit.

[0023] <Pixel counter section> The first counter circuit 211 counts the pulse signals (output signals from the pixels) output from the waveform shaping unit 210. That is, the first counter circuit 211 is a counter circuit that counts the number of photons incident on each APD. Furthermore, the first counter circuit 211 receives a reset signal from the first determination circuit 213 and resets the counter value.

[0024] On the other hand, the second counter circuit 212 counts the time taken for the first counter circuit 211 to count photons, using a clock supplied from outside the sensor (or a clock generated within the sensor using that clock). That is, the second counter circuit 212 is a time to digital converter (hereinafter referred to as TDC). The output of the second counter circuit 212 may be output as time (seconds) or the number of clocks.

[0025] When the number of photons counted by the first counter circuit 211 reaches a first threshold, the first decision circuit 213 resets the count value of the first counter circuit 211. After the comparison by the comparator 215 (described later) is completed, the first decision circuit 213 overwrites the counter value (second time count value) of the second counter circuit 212 in the memory 214 and also resets the count value of the second counter circuit 212. That is, the first decision circuit has the role of resetting the first counter circuit that counts the number of photons and the second counter circuit that counts the time during which the photons are incident, every time the number of photons incident on the APD reaches the first threshold.

[0026] The memory 214 stores past counter values ​​(first time count values) of the second counter circuit 212, and the counter value in the memory 214 is overwritten each time the count value of the second counter circuit 212 is reset. The comparator 215 calculates a count value 219 that is the difference between the current counter value (second time count value) of the second counter circuit 212 and the past counter value (first time count value) of the second counter circuit 212. That is, the comparator 215 outputs a predetermined signal (e.g., a differential count value) in accordance with the comparison result between a first time required for the number of photons measured by the first counter circuit to exceed a threshold value the Nth time and a second time required for the number of photons to similarly exceed the threshold value the (N+1)th time. The comparator 215 may detect a change in luminance using the ratio between the counter value of the first counter circuit and the counter value of the second counter circuit.

[0027] <Linkage with peripheral circuits> If the difference count value 219 is equal to or greater than the second threshold, the second determination circuit 216 sends a request signal to the vertical arbitration circuit 110 via a request signal output line 118VREQ. Then, the response circuit 217 receives a response from the vertical arbitration circuit 110 via a response input line 119VACT, indicating whether or not the output of address event data is permitted. On the other hand, if the difference count value 219 is less than the second threshold, the response circuit 217 does not send a request signal.

[0028] When the response circuit 217 receives a response indicating permission to output, the selection circuit 218 switches the connection between the memory 214 and the signal line 113 using the control signal VSEL. As a result, the counter value (second time count value) of the second counter circuit 212 held in the first memory 314 is output to the column circuit 112.

[0029] <Summary> The differential counter value 219 calculated by the comparator 215 is the time interval at which the second counter value was reset. As described above, the second counter value is reset when the number of photons incident on the APD reaches the first threshold, so the differential counter value is the time interval at which the number of incident photons reaches the first threshold. In other words, the differential counter value 218 corresponds to the reciprocal of the incidence frequency of photons. Therefore, the photoelectric conversion element 100 has the function of measuring "changes in the incidence frequency of photons," that is, changes in luminance.

[0030] Then, using the second determination circuit 216, address event data is output only when the difference in the intervals at which the number of incident photons reached the first threshold is equal to or greater than the second threshold. In other words, this photoelectric conversion element outputs the incidence frequency when the difference in incidence frequency is large, and does not output the incidence frequency when the difference is small. This configuration realizes an asynchronous photoelectric conversion element that detects luminance changes as address events in real time for each pixel address. By implementing the DVS as a photon counting type in this way, it becomes possible to detect address events with changes at the single photon level and to set the detection conditions for address events more precisely.

[0031] <Timing chart> Fig. 6 is a timing chart illustrating a driving method for detecting an address event using the photoelectric conversion element 100. Fig. 6 shows a case where both the first threshold and the second threshold are 4 counts. 250 indicates a clock, 251 indicates the timing at which a photon is detected by the first counter circuit, 211 indicates the count value of the first counter circuit, and 214 indicates the counter value of the second counter circuit stored in memory.

[0032] At the first time T0, the first counter circuit 211 starts counting photons. At this time, the first time count value counted by the second counter circuit at the previous timing is stored in the memory 214. (In FIG. 6, it is assumed that the first time T0=10 is stored as the initial time count value.) After the first counter circuit 211 starts counting photons, at a second time T1, the count value of the first counter circuit 211 reaches a first threshold value of 4. The second time T1 is counted by the second counter circuit 212, and FIG. 6 shows a case where the time count value counted by the second counter circuit is T1=8. The first determination circuit 213 determines that the count value of the first counter circuit 211 has reached the first threshold value of 4, and resets the first counter circuit 211.

[0033] At the same time, the comparator 215 obtains a counter value ΔT1=2, which is the difference between T1 and T0. The second judgment circuit 216 compares ΔT1=2 counts with a second threshold value=4 counts. At time T1, ΔT1 is less than the second threshold value, so T1 is not output. After the judgment by the comparator 215 is completed, the time count value T1 counted by the second counter circuit 212 is written to the memory 214, and the count value of the second counter circuit 212 is reset.

[0034] Next, photon counting is resumed from time T1. After starting photon counting, the count value of the first counter circuit 211 again reaches the first threshold value (=4) at time T2. FIG. 5 shows the case where the time count value T2=16. As in the case of time T1, the first determination circuit 213 resets the first counter circuit 211 and uses the comparator 215 to calculate the counter value ΔT2=6 counts, which is the difference between T2 and T1. At time T2, ΔT2 is greater than or equal to the second threshold value, so a request signal to output T2 is sent to the vertical arbitration circuit 110. Then, upon receiving a response signal from the vertical arbitration circuit 110, the second counter circuit 212 outputs the time count value T2 to the column circuit 112.

[0035] <Advantages of photon counting type> In conventional photoelectric conversion elements, changes in luminance are detected as analog signals based on changes in the amount of light incident on a photodiode, making it difficult to measure changes in luminance at the level of a single photon. On the other hand, the photoelectric conversion element of this embodiment measures changes in luminance using a first counting circuit 211 that counts the number of photons and a second counter circuit 212 that counts time. This makes it possible to detect changes in luminance at the level of a single photon. Specifically, by setting the first threshold to 1, changes in luminance at the level of a single photon can be detected. By detecting changes in luminance at the level of a single photon, an address event signal can be acquired even in night vision conditions, such as at night.

[0036] Furthermore, the photoelectric conversion element of this embodiment uses a comparator 215 and a second determination circuit 216 to determine whether or not there is a change in luminance. Because the detection conditions for address events can be set digitally using the second threshold value, the photoelectric conversion element of this embodiment allows for detailed setting of the detection conditions for address events. By detailed setting of the detection conditions for address events, it is possible to remove luminance changes due to photon shot noise depending on the scene and obtain only the necessary address event information. Specifically, if you want to capture only large luminance changes, you can set the threshold value to be large, and if you want to capture small luminance changes as well, you can set the threshold value to be small.

[0037] <Variation 1: Photon Count> While Figure 5 shows a case where an APD is used as the photoelectric conversion unit to achieve photon counting, a different type of photoelectric conversion unit may also be used. For example, photon counting may be achieved using a photoelectric conversion unit that suppresses readout noise to approximately 0.5 electrons or less, such as the QIS (Quanta Image Sensor) disclosed in Reference 1. However, using an APD as shown in Figure 5 results in less readout noise. (Reference 1: Jiaju Ma, et.al, "Photon-number-resolving megapixel image sensor at room temperature without avalanche gain," Optica, 2017) <Variation 2: First Threshold> The first threshold value is preferably set to a value equal to the number of saturation bits of the first counter circuit. Specifically, the first count value and the second count value may be reset when the most significant bit of the first counter circuit becomes 1.

[0038] The smaller the first threshold, the more frequently the presence or absence of a change in the differential counter value 219 is determined, and thus changes in luminance can be captured instantaneously, which is preferable. On the other hand, the larger the first threshold, the more photon incidence frequency is estimated using a larger number of photons, which improves estimation accuracy and allows changes in luminance to be measured accurately, which is preferable. Therefore, it is even more preferable to configure the first threshold so that it can be appropriately changed depending on the use case, subject, etc.

[0039] DVS has two main advantages. One is that it can capture high-speed phenomena such as those in vehicles, and the other is that it can reduce the amount of data by not outputting when there is no change in brightness, such as in fixed-point monitoring of parking lots. When you want to capture high-speed phenomena, as in the former case, it is preferable to set the first threshold low so that changes in brightness can be captured instantaneously. On the other hand, when you want to suppress output when there is no change in brightness, as in the latter case, it is preferable to set the first threshold high and accurately measure changes in brightness to reliably suppress output when there is no change in brightness.

[0040] <Variation: Second Threshold> The second threshold for determining whether to output the differential counter value 219 may be a fixed value or may be variable over time. Because the differential counter value 218 corresponds to the inverse of the frequency of incident photons, it is preferable to change the second threshold according to the differential counter value 219. Specifically, it is preferable to change the second threshold so that it is inversely proportional to the differential counter value 219 each time the first count value and the second count value are reset. With this configuration, the second threshold can be determined so that the ratio (ΔL / L) of the luminance change ΔL to the original luminance L is constant. It is even more preferable to configure the second threshold to be variable for each pixel. Because the frequency of incident photons varies depending on the pixel, the second threshold needs to be variable for each pixel in order to keep ΔL / L constant.

[0041] The second threshold may be configured to have different thresholds when the difference counter value increases and when the difference counter value decreases. By having different thresholds, it is possible to selectively detect only an increase in luminance or only a decrease in luminance.

[0042] <Variation: If there is no photon count, output pixel value> When the frequency of incident photons is extremely low, it takes a long time for the count value of the first counter circuit to reach the first threshold, which may reduce the frequency of address event detection. Therefore, it is preferable to configure the device so that, when the count value of the second counter circuit reaches the third threshold but the count value of the first counter circuit does not reach the first threshold, the count value of the first counter circuit at that time is output. Specifically, a third determination circuit is provided that determines whether the count value of the first counter circuit is greater than or equal to the first threshold or less than the first threshold when the count value of the second counter circuit reaches the third threshold. Furthermore, the third determination circuit is connected to the request signal line, the response input line, the selection circuit, and the column circuit. This configuration allows the presence or absence of an address event to be detected even when the frequency of incident photons is low.

[0043] <Variations: Peripheral circuits> 4 shows a case where the address events of each pixel are read out using a vertical arbitration circuit and a horizontal readout circuit, but other configurations are also possible. Specifically, a three-layer configuration may be used in which a memory chip 31 is added to the sensor chip 11 and circuit chip 21, and the time count values ​​measured by each pixel are transferred to the memory chip for each pixel, temporarily stored in the memory within the memory chip, and then output sequentially from the memory chip.

[0044] <Variation: Analysis processing using address event signals> An information processing device that executes various analytical processes based on the address event signal output by the photoelectric conversion element described above will now be described. FIG. 9 is a block diagram showing an example of the hardware configuration of the information processing device. The CPU 1100 reads and executes an OS and other programs stored in the memory 1200, controls each connected component, and performs calculations and logical judgments for various processes. The processes executed by the CPU 1100 include the information processing of the embodiment. The memory 1200 is a hard disk drive, an external storage device, or the like, and stores programs and various data related to the information processing of the embodiment. FIG. 8 is a flowchart illustrating the process executed by the information processing device that acquires the address event signal output by the photoelectric conversion element described in this embodiment and executes various analytical processes. The process shown in the flowchart of FIG. 8 is executed by the CPU 1100 shown in FIG. 9, which is a computer, in accordance with a computer program stored in the memory 1200. In the following description, each process (step) will be denoted by prefixing it with "S" to omit the process (step). In S801, the acquisition unit 1007 acquires the address event signal detected by the photoelectric conversion element 100. In S802, the analysis processing unit 1006 executes a predetermined analysis process based on the acquired address event signal. The analysis process may be, for example, various analysis processes such as moving object detection and vibration detection. In S830, the analysis processing unit 1006 outputs the analysis result of the address event signal. For example, in the case of moving object detection, image data that can display the time and location where the moving object was detected is generated and output to the display unit.

[0045] The system may also be configured to include a trained neural network that receives address event signals as input and outputs control signals, and to learn data to be used for post-processing from the address event signals. For example, the address event signals may be used directly as recognition signals for machine learning to perform analytical processing such as subject detection and subject classification. Furthermore, the results of calculating the motion vector of a subject from the address event signals may be output to the control unit 1002 and used for subject vibration isolation, tracking, motion prediction, and motion analysis. Furthermore, the frequency of a blinking light source may be analyzed from temporal changes in the address events, and used for position estimation or failure analysis of a vibrating object.

[0046] <Variation: Luminance output> In the above, the second counter value is output when the difference between the intervals at which the number of incident photons reaches the first threshold is equal to or greater than the second threshold, using the second determination circuit 216. However, it is more preferable to also output information on the brightness of the subject in addition to the second count value.

[0047] Specifically, if the difference in the intervals of time when the number of incident photons reaches the first threshold is equal to or greater than the second threshold, the second counter is reset once. After that, it is sufficient to have a first memory unit 220 that stores the second count value until the difference in the intervals of time when the number of incident photons reaches the first threshold again becomes equal to or greater than the second threshold, and a second memory unit 221 that stores the number of times the first counter is reset.

[0048] For example, if the number of times the first counter has been reset is C, the second count value is T, and the first threshold value is N, the luminance L of the subject can be expressed as follows: L=K(C×N / T) Here, K is an arbitrary coefficient.

[0049] This allows pixel values ​​to be acquired using changes in light intensity as a trigger, making it possible to acquire images that reduce the effects of camera shake, or images of the moment a subject begins to move.

[0050] <Embodiment 2> The photoelectric conversion element 300 of the second embodiment differs from the photoelectric conversion element 100 of the first embodiment only in the configuration of the signal processing unit. Fig. 7 shows an example of an equivalent circuit and block diagram of the pixels and signal processing unit 301 of the photoelectric conversion element 300. The configuration of the pixel unit is the same as that of the photoelectric conversion element 100 shown in Fig. 5, and therefore a description thereof will be omitted.

[0051] Similar to the signal processing unit 103 shown in FIG. 5, the signal processing unit 301 includes a quench element 302, a waveform shaping unit 310, a first counter circuit 311, a second counter circuit 312, a first judgment circuit 313, a second judgment circuit 316, a response circuit 317, and a selection circuit 318. In addition to a first memory 314 and a first comparator 315, the signal processing unit 301 also includes a second memory 324 and a second comparator 325. This configuration allows the "degree of change" in brightness to be detected as an address event. Since the "change in brightness" is the first derivative of luminance, the "degree of change in brightness" corresponds to the second derivative of luminance. Using the second derivative of luminance as an address event allows the detection of only abnormal phenomena, such as detecting an abnormality in an object whose brightness changes uniformly.

[0052] The roles and configurations of the quench element 302, waveform shaping section 310, first counter circuit 311, and second counter circuit 312 are the same as those in FIG. 5, and therefore will not be described.

[0053] When the number of photons counted by the first counter circuit 311 reaches a first threshold, the first decision circuit 313 resets the count value of the first counter circuit 311. After the comparison by the first comparator 315 (described later) is completed, the first decision circuit 313 overwrites the counter value (second time count value) of the second counter circuit 312 in the first memory 314, and also resets the count value of the second counter circuit 312.

[0054] The first memory 314 stores past counter values ​​(first time count values) of the second counter circuit 312, and the counter value in the first memory 314 is overwritten every time the count value of the second counter circuit 312 is reset. The first comparator 315 calculates a count value 319 that is the difference between the current counter value (second time count value) of the second counter circuit 312 and the past counter value (first time count value) of the second counter circuit 312.

[0055] The second memory stores a count value 329 of the past difference (first difference count value), and the counter value in the second memory 324 is overwritten every time the count value of the second counter circuit 312 is reset. The second comparator 325 calculates a count value 339 of the difference between the current counter value of the difference (second difference count value) and the counter value of the past difference (first difference count value).

[0056] If the difference count value 339 is equal to or greater than the fourth threshold, the second determination circuit 316 sends a request signal to the vertical arbitration circuit via a request signal output line. The response circuit 317 then receives a response from the vertical arbitration circuit via a response input line, indicating whether or not the output of address event data is permitted.

[0057] When the response circuit 317 receives a response indicating permission to output, the selection circuit 318 switches the connection between the second memory 324 and the signal line, thereby outputting the differential count value 339 stored in the second memory 324 to the column circuit.

[0058] <Summary> The differential counter value 339 is the difference in the time interval at which the second counter value was reset. Therefore, the differential counter value 339 corresponds to the degree of change in the frequency of incident photons. Therefore, the photoelectric conversion element 100 of the present invention measures the second derivative of luminance. With the above configuration, an asynchronous photoelectric conversion element can be realized that detects the second derivative of luminance as an address event in real time for each pixel address.

[0059] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0060] 11 Sensor chip 21 Circuit Chip 101 pixels 103 Signal Processing Unit 201 Avalanche photodiode 202 Quench element 210 Waveform shaping section 211 first counter circuit 212 second counter circuit 213 First decision circuit 214 memory 215 Comparator 216 Second decision circuit 217 Response Circuit 218 Selection Circuit

Claims

1. A photoelectric conversion element having pixels that output signals in response to incident photons, a first measuring means for measuring the signal output from the pixel; a second measuring means for measuring a time required for the signal measured by the first measuring means to reach a first threshold value; a first storage means for storing a measurement result of the second measurement means at a first time point as a first time; a comparison means for comparing the first time stored by the first storage means with a second time measured by the second measurement means at a second time point that is later than the first time point; and an output means for outputting a signal requesting output of address event data when it is determined based on the comparison result by the comparison means that the difference or ratio between the first time and the second time is greater than a second threshold value.

2. The photoelectric conversion element described in Claim 1, characterized in that the output means outputs the signal requesting the output of address event data to the vertical arbitration circuit when it is determined based on the comparison result by the comparison means that the difference or ratio between the first time and the second time is greater than a second threshold value.

3. 2. The photoelectric conversion element according to claim 1, wherein the second threshold value is variable.

4. 2. The photoelectric conversion element according to claim 1, wherein the second threshold value is variable for each pixel.

5. 2. The photoelectric conversion element according to claim 1, wherein the second threshold value is different when the second time is greater than the first time and when the second time is less than the first time.

6. 6. The photoelectric conversion element according to claim 1, wherein the first measuring means measures the number of photons output from the pixel.

7. 7. The photoelectric conversion element according to claim 1, further comprising a conversion means formed of an avalanche photodiode.

8. 8. The photoelectric conversion element according to claim 1, wherein the first threshold value is equal to a saturation bit number of the first measurement means.

9. 9. The photoelectric conversion element according to claim 1, wherein the first threshold value is variable.

10. 10. The photoelectric conversion element according to claim 1, wherein the second measurement means resets the measurement by the second measurement means when the first time reaches a preset third threshold value.

11. 11. The photoelectric conversion element according to claim 10, wherein the comparison means outputs the measurement result of the first measurement means when the measurement result of the second measurement means reaches the third threshold value.

12. 12. The photoelectric conversion element according to claim 1, further comprising: second storage means for storing a difference between the second time and the first time based on a comparison result by the comparison means.

13. 13. The photoelectric conversion element according to claim 12, wherein the comparison means further compares the difference in time stored by the second storage means with the difference between the measurement result by the second measurement means and the time stored by the first storage means.

14. 14. The photoelectric conversion element according to claim 1, further comprising a determination means for resetting the measurement by the first measurement means when the measurement result of the signal by the first measurement means exceeds the first threshold value.

15. The photoelectric conversion element according to claim 14, characterized in that, when the measurement result of the first measurement means satisfies the first threshold value, the determination means outputs the measurement result of the first time by the second measurement means to the first storage means, and further resets the first time to the measurement result of the second measurement means.

16. 16. The photoelectric conversion element according to claim 14, wherein the determining means outputs the luminance of the subject based on the number of times the measurement by the first measuring means is reset.

17. 2. The information processing apparatus according to claim 1, wherein the second measuring means measures the time required for the signal measured by the first measuring means to reach the first threshold value by measuring the number of clocks.

18. An information processing device comprising the photoelectric conversion element according to any one of claims 1 to 16, an acquisition means for acquiring the signal output by the photoelectric conversion element; and analyzing means for performing a predetermined analysis process based on the acquired signal.

19. A method for controlling a photoelectric conversion element having a pixel that outputs a signal in response to incidence of a photon, comprising: a first measurement step of measuring the signal output from the pixel; a second measuring step of measuring a time required for the signal measured in the first measuring step to reach a first threshold value; a first storage step of storing the measurement result of the second measurement step at a first time point as a first time; a comparing step of comparing the first time stored in the first storing step with a second time measured in the second measuring step at a second time point later than the first time point; and an output step of outputting a signal requesting output of address event data when it is determined based on the comparison result in the comparison step that the difference or ratio between the first time and the second time is greater than a second threshold value.

20. A control method as described in Claim 19, characterized in that in the output process, if it is determined based on the comparison result in the comparison process that the difference or ratio between the first time and the second time is greater than a second threshold value, the signal requesting the output of address event data is output to a vertical arbitration circuit.

21. A program for causing a computer to execute the control method according to claim 20.

22. A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 20.

Images