Imaging device

Abstract

To provide a technique for grasping information on the brightness of an image in the termination of the exposure time of one frame and realizing adjustment of the brightness in the next frame.SOLUTION: An imaging apparatus comprises: a plurality of SPAD pixels which are two-dimensionally arrayed; and a plurality of pieces of signal processing means which are provided respectively corresponding to the plurality of SPAD pixels. Each of the plurality of pieces of signal processing means comprises: requesting means which issues a readout request according to the detection of a prescribed number of photons by the corresponding SPAD pixels; and mediation means which selects and outputs one of another readout request received from other signal processing means and the readout request issued by the requesting means. The imaging apparatus further includes request detection means which detects the readout request output by the mediation means. The request detection means detects and counts the readout requests issued in a prescribed period.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] There is known a photoelectric conversion device that digitally counts the number of photons arriving at an avalanche photodiode and outputs the counted value from a pixel as a digitally photoelectrically converted signal. In terms of noise and signal arithmetic processing, the advantage of digitizing pixel signals is great, and imaging devices in which a plurality of pixels that output digitally photoelectrically converted signals are arranged have begun to spread.

[0003] Patent Document 1 discloses a method of measuring the time when the number of counted photons reaches a threshold value in a time shorter than one frame, and obtaining the number of photons per frame from the time information and the number of photons.

[0004] In the method of obtaining the number of photons per frame in Patent Document 1, a photon counter and a time measurement unit are provided for each pixel, and the counting of photons and the time measurement are started at the start of exposure. When the photon counter reaches the threshold value in a time shorter than the exposure time of one frame, a calculation of photon threshold value × (exposure time of one frame / time until reaching the threshold value) is performed to calculate the number of photons per frame. Since the photon threshold value and the exposure time of one frame are set to predetermined values, when the time until reaching the threshold value is known, that is, when the photon counter reaches the threshold value, the number of photons per frame of that pixel is determined. Also, in order to perform the above calculation of the number of photons per frame, it can be seen that the pixels in which the photon counter has reached the threshold value have a certain level of brightness or more compared to the pixels that have not reached the threshold value.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Generally, the brightness adjustment of an image is performed based on the read pixel values. However, in the conventional technology, since the brightness calculation is performed after the reading of all pixels is completed, it is reflected in the exposure conditions with a delay of one frame or more.

[0007] The present invention has been made in view of the above problems, and an object thereof is to provide a technique for grasping information on the brightness of an image at the end of the exposure time of one frame and realizing brightness adjustment in the next frame.

Means for Solving the Problems

[0008] An imaging device according to an aspect of the present invention that achieves the above object includes a plurality of SPAD pixels two-dimensionally arranged, a plurality of signal processing means provided corresponding to each of the plurality of SPAD pixels, and includes each of the plurality of signal processing means request means for issuing a read request in response to a corresponding SPAD pixel detecting a predetermined number of photons, arbitration means for selecting and outputting one of another read request received from another signal processing means and the read request issued by the request means, and includes the imaging device further includes request detection means for detecting the read request output by the arbitration means, the request detection means is characterized by detecting and counting the read requests issued within a predetermined period.

Effects of the Invention

[0009] According to the present invention, it is possible to grasp information on the brightness of an image at the end of the exposure time of one frame and realize brightness adjustment in the next frame.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0012] (First Embodiment) In this embodiment, an example in which a pixel in a two-dimensional array issues a read request and performs a read operation will be described. A pixel for which a pixel counter has reached a threshold value within an imaging time issues a read request, and the count value of each pixel is read during a batch read, thereby acquiring brightness information earlier than in the prior art. As a result, it becomes possible to adjust the brightness in the next frame.

[0013] <Configuration example of imaging device> FIG. 1 is a diagram showing a configuration example of an imaging device according to this embodiment. The imaging device 100 is configured by stacking and electrically connecting two chips, a sensor chip 11 and a circuit chip 21. The sensor chip 11 includes a pixel region 12. The circuit chip 21 includes a pixel circuit region 22 that processes signals detected in the pixel region 12 and a control circuit region 23 that reads signals from the pixel circuit region 22.

[0014] <Configuration example of sensor chip> FIG. 2 is a diagram showing a configuration example of the sensor chip 11 included in the imaging device according to this embodiment. The pixel region 12 of the sensor chip 11 includes a plurality of pixels 101 two-dimensionally arranged in a plurality of rows and columns. The pixel 101 includes a photoelectric conversion unit 102 including, for example, an avalanche photodiode (hereinafter, APD). The pixel 101 is, for example, a SPAD (Single Photon Avalanche Diode) pixel. FIG. 2 shows m×n pixels 101 arranged in m rows from the first row to the m-th row and n columns from the first column to the n-th column, together with symbols indicating row numbers and column numbers. For example, the unit pixel 101 arranged in the first row and the third column is labeled with the symbol "P13". Note that the number of rows and columns of the pixel array constituting the pixel region 12 is not particularly limited.

[0015] <Configuration example of circuit chip> FIG. 3 is a diagram showing a configuration example of a circuit chip 21 included in the imaging device 100 according to the present embodiment. The circuit chip 21 includes a pixel circuit region 22 and a control circuit region 23. The pixel circuit region 22 includes a plurality of signal processing units 103 arranged two-dimensionally over a plurality of row and column directions. In FIG. 3, a part of m×n signal processing units 103 arranged from the first row to the m-th row and from the first column to the n-th column is shown together with symbols indicating row numbers and column numbers. For example, the signal processing unit 103 arranged in the first row and the third column is assigned the symbol "S13". Note that the number of rows and columns of the signal processing unit array constituting the pixel circuit region 22 is not particularly limited.

[0016] The control circuit region 23 includes a control pulse generation circuit 110 and a signal readout circuit 111. Each row of the signal processing unit array in the pixel circuit region 22 extends in a first direction (the horizontal direction in FIG. 3), and a control line 112 and a data signal line 113 are arranged in each row. The control line 112 and the data signal line 113 are respectively connected to the signal processing units 103 arranged in the first direction. The first direction in which the control line 112 extends may be referred to as the row direction or the horizontal direction. Further, a readout request signal line 114 is connected to each row of the signal processing unit array in the pixel circuit region 22 in such a manner that the signal processing units 103 in the first direction are connected.

[0017] The control line 112 of each row is connected to the control pulse generation circuit 110. The control pulse generation circuit 110 supplies a control signal for driving the signal processing unit 103 to the signal processing unit 103 via the control line 112. Further, the control pulse generation circuit 110 and the signal readout circuit 111 are connected via a control line 115. The control pulse generation circuit 110 supplies a control signal for driving the signal readout circuit 111 in synchronization with the signal processing unit 103 via the control line 115. The control pulse generation circuit 110 may generate a control signal based on an external trigger (not shown) or may generate a control signal based on an internal signal.

[0018] The data signal lines 113 and the read request signal lines 114 of each row are connected to the signal readout circuit 111. The data signal lines 113 are signal lines for transmitting the data held by the signal processing unit 103. The data referred to here is a pixel counter value or a time counter value, the details of which will be described later. The read request signal lines 114 are signal lines for transmitting the read requests transmitted by each signal processing unit 103. The signal readout circuit 111 acquires a plurality of data from the data signal lines 113 in response to the read requests on the read request signal lines 114.

[0019] <Equivalent Circuit and Block Diagram> Subsequently, FIG. 4 is an example of an equivalent circuit and a block diagram of the pixel 101 in FIG. 2 and the signal processing unit 103 in FIG. 3.

[0020] The pixel 101 in the sensor chip 11 includes an APD 201 which is a photoelectric conversion unit. When light is incident on the APD 201, electron-hole pairs corresponding to the incident light are generated by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. Also, a voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage is supplied between the anode and the cathode so that the APD 201 operates in an avalanche multiplication mode. By supplying such a voltage, the charges generated by the incident light cause avalanche multiplication, and an avalanche current is generated.

[0021] Note that when a reverse bias voltage is supplied, there are a Geiger mode in which the potential difference between the anode and the cathode is operated at a potential difference larger than the breakdown voltage, and a linear mode in which the potential difference between the anode and the cathode is operated at a voltage difference near or below the breakdown voltage. An APD operated in the Geiger mode is called an SPAD. For example, the voltage VL (first voltage) is -30V, and the voltage VH (second voltage) is 1V.

[0022] The signal processing unit 103 in the circuit chip 21 includes a pulse generation unit 210 and a pulse processing unit 220.

[0023] The pulse generation unit 210 includes a quench element 211 and a waveform shaping unit 212. The quench element 211 is connected to a power supply that supplies the voltage VH and the APD 201. The quench element 211 has a function of replacing the change in the avalanche current generated in the APD 201 with a voltage signal. The quench element 211 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing the voltage supplied to the APD 201 to suppress avalanche multiplication (quench operation).

[0024] The waveform shaping unit 212 shapes the potential change of the cathode of the APD 201 obtained at the time of photon detection and outputs a pulse signal. For the waveform shaping unit 212, for example, an inverter circuit or a buffer circuit is used.

[0025] The pulse processing unit 220 receives the photon detection pulse generated by the pulse generation unit 210 and transfers a plurality of data via the data signal line 113. This will be described in detail below with reference to FIG. 5.

[0026] <Configuration example of the pulse processing unit> FIG. 5 is a diagram showing a configuration example of the pulse processing unit 220 according to the present embodiment. The pulse processing unit 220 includes a pixel counter 221, a read request unit 222, a time counter 223, and a request arbitration unit 224.

[0027] The drive signal received via the control line 112 is transmitted to the pulse processing unit 220 of each signal processing unit 103 and is used for controlling the pixel counter 221, the read request unit 222, the time counter 223, and the request arbitration unit 224. When this drive signal is valid (High), it indicates that imaging is in progress, and when it is invalid (Low), it indicates that reading is in progress.

[0028] The pixel counter 221 counts the photons detected by the corresponding SPAD pixel. The pixel counter 221 starts counting the number of pulse signals output from the pulse generation unit 210 at the rising edge of the drive signal received via the control line 112. Also, the pixel counter 221 stops counting at the falling edge of the drive signal or the rising edge of the control signal output from the time counter 223, and outputs the count value to the request arbitration unit 224. The pixel counter 221 is, for example, an 8-bit counter, and outputs a saturation pulse signal when it reaches saturation (all bits are 1). Note that the saturation pulse signal output by the pixel counter 221 may be issued in response to the count value exceeding (or reaching) a predetermined threshold value.

[0029] The read request unit 222 issues a read request in response to the corresponding SPAD pixel 101 detecting a predetermined number of photons. The read request unit 222 transmits the read request to the request arbitration unit 224, for example, either by the rising edge of the saturation pulse signal output from the pixel counter 221 or by the falling edge of the drive signal received via the control line 112. Also, when the read request unit 222 detects the rising edge of the saturation pulse signal, it transmits a request to read the time counter value. When the rising edge of the saturation pulse signal is not detected, it transmits a request to read the pixel counter. Alternatively, the read request unit 222 may be configured to transmit the read request to the request arbitration unit 224 either by the rising edge of the saturation pulse signal output from the pixel counter 221 or by the rising edge of the control signal output from the time counter 223. Even in that case, when the read request unit 222 detects the rising edge of the saturation pulse signal, it transmits a request to read the time counter value. When the rising edge of the saturation pulse signal is not detected, it transmits a request to read the pixel counter value.

[0030] The read request unit 222 may be configured to issue a read request when the value of the pixel counter 221 is equal to or greater than the threshold value at the time when the value of the time counter 223 reaches the threshold value.

[0031] The time counter 223 counts the time since the corresponding SPAD pixel started counting photons. The time counter 223 starts counting time at the rising edge of the drive signal received via the control line 112. Also, the time counter 223 stops counting at the falling edge of the drive signal or the rising edge of the saturation pulse signal output from the pixel counter 221, and outputs the time counter value to the request arbitration unit 224. The time counter 223 may issue a control signal when the time counter value exceeds (or reaches) a predetermined threshold value.

[0032] The request arbitration unit 224 selects and outputs one of the other read requests received from other signal processing units and the read request issued by the read request unit 222. The request arbitration unit 224 controls the read request according to the presence or absence of a high-priority read request from the signal processing unit 103 with a high priority. Each signal processing unit 103 has a determined priority. In FIG. 3, for example, the signal processing unit 103 (S11) at the left end has the highest priority, and the priority is determined to decrease as the column numbers progress horizontally for S12 and S13. By doing so, the wiring of the read request line 114 can be simplified. Of course, the method of assigning priorities is not limited to this. Also, the signal processing unit 103 with the highest priority operates in the same manner as the configuration example shown in FIG. 5 on the premise that the read request of the signal processing unit 103 with a high priority is always stopped (Low).

[0033] Next, a method for controlling the read request of the request arbitration unit 224 will be described. When no read request occurs in the signal processing unit 103, the request arbitration unit 224 directly transmits the high-priority read request from the signal processing unit 103 with high priority. When a read request occurs in the signal processing unit 103 and at the same time a read request (High) from the signal processing unit 103 with high priority occurs, the read request is transmitted to the signal readout circuit 111, and the read request of the signal processing unit 103 is extended (continued). When the read request from the signal processing unit 103 with high priority is stopped (Low), it is determined that the read request of the signal processing unit 103 is permitted, and the read request signal is stopped (Low). Simultaneously with the stop of the read request signal, when the drive signal received via the control line 112 is valid (High), the request arbitration unit 224 does not output the count value, and when the drive signal is invalid (Low), the request arbitration unit 224 outputs the count value to the data signal line 113. The request arbitration unit 224 selects and outputs the count value of the pixel counter 221 or the count value of the time counter 223 according to the read request signal output from the read request unit 222.

[0034] <Connection between signal readout circuit and signal processing unit> FIG. 6 is a diagram showing a configuration example of the signal readout circuit 111 according to the present embodiment and its connection with the signal processing unit 103. The signal readout circuit 111 includes a row readout circuit 230 connected for each row and an output interface 231 for selecting the output of each row. The row readout circuit 230 includes a data reception unit 232 and a request detection unit 233.

[0035] The output interface 231 selects the row readout circuit of each row and outputs the count value and the count value (number of requests) of the request detection unit 233 to the outside.

[0036] The data reception unit 232 receives the read request received from the signal processing unit 103 via the read request signal line 114 and acquires the pixel counter value or the time counter value of the data signal line 113.

[0037] The request detection unit 233 detects and counts read requests issued within a predetermined time after the start of imaging. The value counted by the request detection unit 233 is the number of pixels for which the pixel counter has saturated within a predetermined time after the start of imaging, and is the number of pixels having a brightness of a certain level or more. By using this count value, it is possible to adjust the brightness in the next frame.

[0038] <Timing Chart> FIG. 7 is a timing chart showing the control of read requests in each signal processing unit 103 according to the present embodiment and the operation of the line read circuit 230. The signal processing units S11 to S14 in the first row are illustrated as examples, and it is assumed that no reading is performed from the other signal processing units 103. Note that the illustrated timing chart is an example of the present embodiment, and the present embodiment is not limited to the transition timings of these signals.

[0039] Times T0 to T10 are the times at the rising edges of CLK, respectively.

[0040] Here, a flow will be described in which only pixels for which the pixel counter has saturated from the start of imaging until the end of imaging issue read requests and count the number of read requests, and then all pixels are read out in a batch after the end of imaging. Time T4 is the end time of imaging. From time T0 to T4, read requests are issued by the signal processing units among S11 to S14 for which the pixel counter has exceeded the threshold value according to the instruction of the drive signal, and the requests are counted by the request detection unit 233. From time T4 to T10, an instruction to perform batch reading is issued to each pixel by the drive signal, and each pixel issues a read request and outputs a count value. Note that among the signal processing units S11 to S14, the smaller the horizontal number, the higher the priority of reading.

[0041] First, each of the signals shown in FIG. 7 will be described. The time counter value is the value of the time counter 223. In FIG. 7, the time counter of S11 is extracted and shown. The pixel counter saturation pulse is a signal issued by the pixel counter 221 upon receiving a pulse from the pulse generation unit 210. The read request is a signal issued by the read request unit 222 in synchronization with CLK upon receiving the pixel counter saturation pulse. The output counter value is the count value output by the request arbitration unit 224 of each pixel to the data signal line 113. The request detection unit count value is the count value of the requests issued and detected by time T4. The drive signal is a signal transmitted to each pixel via the control line 112. In this example, a signal for ending imaging and performing burst readout is issued at time T4.

[0042] Hereinafter, the behavior of the signals at each time will be described. At time T0, the time counter value changes from 239 to 240. The signal processing units S11 and S14 issue a pixel counter saturation pulse at an arbitrary timing between times T0 and T1. Due to this pixel counter saturation pulse, after T1, the time counter 223 stops and the time counter value does not change.

[0043] Upon receiving the pixel counter saturation pulse, the signal processing unit S11 issues a read request at time T1. Since this signal processing unit has the highest priority, the read request is stopped at time T2.

[0044] Upon receiving the pixel counter saturation pulse, the signal processing unit S14 issues a read request at time T1. When the signal processing unit S11 stops the read request at time T2, the signal processing unit S14 becomes the signal processing unit with the highest priority at time T3. Therefore, the signal processing unit S14 stops the read request at time T3.

[0045] Since the pixel counter saturation pulse is not output until time T4 for the signal processing units S12 and S13, they do not issue a read request until the burst readout performed from time T5. The request detection unit 233 in the line readout circuit 230 detects the requests of S11 at time T2 and S14 at time T3 and counts them respectively.

[0046] In the drive signal, a signal for performing burst readout is issued at time T4, and in response to this, S11, S12, S13, and S14 issue readout requests simultaneously at time T5. Also, since the pixel counter has reached the threshold by time T4 for S11 and S14, they output the time counter value, while S12 and S13 output the pixel counter value because they have not reached the threshold. Since S11 is the signal processing unit with the highest priority, it outputs the count value at time T6 and ends the issuance of the readout request.

[0047] After that, at time T7, S12, at time T8, S13, and at time T9, S14 become the ones with the highest priority among the processing units that have issued readout requests, so they sequentially output the count value and end the issuance of the readout request.

[0048] As described above, in this embodiment, when the pixel counter saturation pulse is output, each pixel issues a readout request, and by counting the number of such requests, it becomes possible to grasp the number of pixels for which the pixel counter has saturated before the burst readout of the pixels. That is, since each pixel issues a readout request when the number of photons in each pixel reaches the threshold, by measuring the number of such requests, it becomes possible to grasp the number of pixels for which the number of photons has reached the threshold within the exposure time of one frame at the end of the exposure time of one frame. Reaching the threshold indicates that the brightness is above a certain level, and this number can be used for adjusting the brightness of the image.

[0049] Conventionally, since the brightness information was acquired after the burst readout of the pixels, it was not possible to adjust the brightness in time for the next frame (the brightness adjustment would be delayed by one frame), but in this embodiment, since the brightness information is acquired before the burst readout of the pixels, it becomes possible to adjust the brightness in the next frame based on this information.

[0050] (Second Embodiment) In the first embodiment, a pixel that reaches the threshold within the imaging time issues a read request, and the count value of each pixel is read during batch readout, showing a configuration capable of acquiring brightness information faster than before. In contrast, in this embodiment, an example of adding a storage means (memory) for storing the count value of each pixel and reading the pixel counter value when the threshold is reached will be described. Thereby, it becomes possible to improve the frame rate according to the brightness.

[0051] <Configuration example of pulse processing unit> FIG. 8 is a diagram showing a configuration example of the pulse processing unit 220 according to this embodiment. Note that the configurations according to FIGS. 1 to 4 in the first embodiment are the same as those in this embodiment, and thus the description thereof is omitted.

[0052] The pulse processing unit 220 includes a pixel counter 221, a read request unit 225, a time counter 223, a request arbitration unit 226, and a coordinate holding unit 227. Since the pixel counter 221 and the time counter 223 operate in the same manner as in the first embodiment, they are given the same numbers and the description thereof is omitted.

[0053] The read request unit 225 transmits a single read request to the request arbitration unit 226 either by the rising edge of the saturation pulse signal output from the pixel counter 221 or the falling edge of the drive signal received via the control line 112 for a single rising edge of the drive signal. In the first embodiment, two read requests are issued at saturation and during batch readout, whereas in this embodiment, only one read request is issued. Further, when the read request unit 225 detects the rising edge of the saturation pulse signal, it transmits a request to read the time counter value. When the rising edge of the saturation pulse signal is not detected, it transmits a request to read the pixel counter value.

[0054] Alternatively, the read request unit 225 may be configured to transmit a single read request to the request arbitration unit 226 based on either the rising edge of the saturation pulse signal output from the pixel counter 221 or the rising edge of the control signal output from the time counter 223. Even in this case, when the read request unit 225 detects the rising edge of the saturation pulse signal, it transmits a request to read the time counter value. When the rising edge of the saturation pulse signal is not detected, it transmits a request to read the pixel counter value.

[0055] Since the request arbitration unit 226 is the same as the request arbitration unit 224 regarding the assignment of priorities, the description thereof is omitted.

[0056] Subsequently, the control method of the read request of the request arbitration unit 226 will be described. When no read request occurs in the signal processing unit 103, the request arbitration unit 226 directly transmits a high-priority read request from a signal processing unit with a high priority. When a read request occurs in the signal processing unit and at the same time a read request from a signal processing unit with a high priority occurs (High), the request arbitration unit 226 transmits the read request to the signal readout circuit 111 and extends the read request of the signal processing unit. When the read request from the signal processing unit 103 with a high priority is stopped (Low), it is determined that the read request of the signal processing unit is permitted, the read request signal is stopped (Low), and the request arbitration unit 226 outputs the count value and the coordinate information to the data signal line 113. The request arbitration unit 226 selects and outputs either the count value of the pixel counter 221 or the count value of the time counter 223 according to the read request signal output from the read request unit 225.

[0057] The coordinate holding unit 227 pre-holds the coordinate information within each row or column of each pixel and transmits it to the request arbitration unit 226.

[0058] <Connection between signal readout circuit and signal processing unit> FIG. 9 is a diagram showing a configuration example of the signal readout circuit 111 according to the present embodiment and its connection to the signal processing unit 103.

[0059] The signal readout circuit 111 includes a row readout circuit 230 connected for each row, a memory 235 that stores the read signal values, an exposure control unit 236, and an output interface 237. The row readout circuit 230 includes a data reception unit 234 and a request detection unit 233. Since the request detection unit 233 operates in the same manner as in the first embodiment, it is given the same number and the description thereof is omitted.

[0060] The data reception unit 234 receives a read request received from the signal processing unit 103 via the read request signal line 114, and acquires the count value of the data signal line 113 and the coordinate information of each pixel. Then, the count value of each pixel is stored in the memory 235 corresponding to the acquired coordinate information. Alternatively, the coordinate information and the count value may be stored together in the memory 235.

[0061] One memory may be implemented for each row in the memory 235. Thereby, arbitration of reading between rows can be avoided.

[0062] The exposure control unit 236 refers to the control signal transmitted via the control line 115 and the read request signal line 114 for each row, and determines whether the reading of all pixels is completed. When it is determined that the reading is completed, a rising edge of a drive signal is requested to the control pulse generation circuit 110 via the control line 115 to start the next imaging. Further, the exposure control unit 236 refers to the count value of the request detection unit 233, and when the threshold value determined by a predetermined time is not reached, a rising edge of a drive signal may be requested to the control pulse generation circuit 110 via the control line 115 to start the next imaging. That is, when the number of pixels of the SPAD pixels read in a predetermined period is less than the threshold value, the current exposure may be aborted and the exposure of the next frame may be started.

[0063] Further, the exposure control unit 236 may refer to the count value of the request detection unit 233 and, when the threshold value set until a predetermined time is reached, request the control pulse generation circuit 110 via the control line 115 to lower the drive signal in order to read out the remaining pixels. After requesting the lowering of the drive signal, the exposure control unit 236 may request the control pulse generation circuit 110 via the control line 115 to raise the drive signal in order to start the next imaging as soon as the reading of all the pixels is confirmed.

[0064] The output interface 237 refers to the memory 235 and outputs the count value and the count value of the request detection unit 233 to the outside.

[0065] <Timing Chart> FIG. 10 is a timing chart showing the control of the read request in each signal processing unit 103 according to the present embodiment and the operation of the row read circuit 230. The signal processing units S11 to S14 in the first row are illustrated as an example, and it is assumed that no reading is performed from the other signal processing units 103. Note that the illustrated timing chart is an example of the present embodiment, and the present embodiment is not limited to the transition timings of these signals.

[0066] Times T0 to T10 are the times at the rising edges of CLK, respectively. Here, after the start of imaging, only the pixels for which the pixel counter has saturated until the end of imaging issue a read request, store the count value in the corresponding location of the memory based on the coordinate information, and after the end of imaging, the process of collectively reading all the unread pixels will be described.

[0067] Time T4 is the end time of imaging. During the times from T0 to T4, a read request is issued by the signal processing unit among S11 to S14 in which the pixel counter has exceeded the threshold value, and the count value and the coordinate information are output according to the result of arbitration between the pixels. Between times T4 and T10, an instruction to collectively read the unread pixels is issued to each pixel by the drive signal, and each pixel issues a read request and outputs the count value and the coordinate information. Note that among the signal processing units S11 to S14, the smaller the horizontal number, the higher the priority of reading.

[0068] First, the signals shown in FIG. 10 will be described respectively. Since the time counter value, pixel counter saturation pulse, readout request, output count value, and request detector count value are the same as those in FIG. 7 of the first embodiment, the description thereof will be omitted. The drive signal outputs a signal that issues a readout request and reads up to the count value for the pixels whose pixel counters have saturated by time T4, and outputs a signal that reads the count values of the remaining pixels that have not been read since time T5. The coordinate information is the coordinate information of the pixels received by the data receiving unit 234.

[0069] Hereinafter, the behavior of the signals at each time will be described. The signal processing units S11 and S14 issue a pixel counter saturation pulse at an arbitrary timing between times T0 and T1. Upon receiving the pixel counter saturation pulse, the signal processing unit S11 issues a readout request at time T1. Since this signal processing unit has the highest priority, at time T2, it outputs the time counter value and the coordinate information and ends the issuance of the readout request.

[0070] Upon receiving the pixel counter saturation pulse, the signal processing unit S14 issues a readout request at time T1. When the signal processing unit S11 ends the issuance of the readout request at time T2, the signal processing unit S14 becomes the signal processing unit with the highest priority at time T3. Therefore, the signal processing unit S14 outputs the time counter value and the coordinate information at time T3 and ends the issuance of the readout request.

[0071] Since the pixel counter saturation pulse is not output by time T4, the signal processing units S12 and S13 do not issue a readout request until the batch readout performed from time T5. The drive signal issues a signal for batch readout at time T4, and in response to this, the signal processing units S12 and S13, which have not output the count value, issue a readout request simultaneously at time T5.

[0072] At time T6, since S12 is the signal processing unit with the highest priority, it outputs the pixel counter value and the coordinate information, and ends the issuance of the read request. After that, at time T7, since the signal processing unit S13 has the highest priority among the signal processing units that have issued read requests, it outputs the pixel counter value and the coordinate information, and ends the issuance of the read request.

[0073] As described above, in this embodiment, when the pixel counter saturation pulse is output, a read request is issued to read the count value and the coordinate information. When imaging is completed and batch reading of pixels is performed, the remaining number of pixels to be read decreases. Conventionally, all pixels were read during batch reading of pixels, but in this embodiment, by reading the count value of the pixels whose pixel counter value has reached the threshold before batch reading of pixels, the time required for batch reading after imaging can be reduced. In addition, after completing the reading of the count value from the pixels earlier than before, by advancing the exposure of the next frame by the exposure control unit 236, it is possible to improve the frame rate according to the brightness during imaging.

[0074] Therefore, in a situation where the situation changes sequentially, such as in an in-vehicle camera, it is possible to immediately reflect the exposure conditions to improve the recognition system, or to improve the frame rate in a specific environment to improve the recognition response speed.

[0075] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0076] The invention is not limited to the above-described embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0077] 100: Imaging device, 101: Pixel, 103: Signal processing unit, 210: Pulse generation unit, 221: Pixel counter, 222: Readout request unit, 223: Time counter, 224: Request arbitration unit

Claims

1. An imaging device, comprising: a plurality of SPAD pixels arranged two-dimensionally; a plurality of signal processing means provided corresponding to each of the plurality of SPAD pixels, wherein each of the plurality of signal processing means has a request means for issuing a readout request in response to a corresponding SPAD pixel detecting a predetermined number of photons, and an arbitration means for selecting and outputting one of another readout request received from another signal processing means and the readout request issued by the request means; The imaging device further comprises a request detection means for detecting the readout request output by the arbitration means, wherein the request detection means is characterized by detecting and counting the readout requests issued within a predetermined period.

2. Each of the plurality of signal processing means further comprises a pixel counter for counting the photons detected by the corresponding SPAD pixel, wherein the request means issues the readout request when the value of the pixel counter reaches a threshold value. The imaging device according to claim 1.

3. Each of the plurality of signal processing means further comprises a time counter for counting the time since the corresponding SPAD pixel started counting photons, wherein the request means issues the readout request when the value of the time counter reaches a threshold value. The imaging device according to claim 1 or 2.

4. Each of the plurality of signal processing means comprises a pixel counter for counting the photons detected by the corresponding SPAD pixel, and a time counter for counting the time since the corresponding SPAD pixel started counting photons, wherein the request means issues the readout request when the value of the pixel counter is equal to or greater than the threshold value when the value of the time counter reaches the threshold value. The imaging device according to claim 1.

5. The request means controls the issuance of the readout request based on an external instruction. The imaging device according to any one of claims 1 to 4.

6. The request detection means detects and counts the readout requests issued from the start of exposure of the imaging device to the batch readout of the SPAD pixels, and outputs the count value of the number of requests in accordance with the batch readout. The imaging device according to any one of claims 1 to 4.

7. The imaging device further comprises an exposure control means for controlling the start of exposure of the next frame. ​ The imaging device according to any one of claims 1 to 6, wherein the exposure control means starts the exposure of the next frame when the number of read SPAD pixels satisfies a predetermined condition.

8. The imaging device according to claim 7, wherein the exposure control means starts the exposure of the next frame when the reading of all SPAD pixels is completed.

9. The imaging device according to claim 7, wherein the exposure control means aborts the current exposure and starts the exposure of the next frame when the number of SPAD pixels read in a predetermined period is less than a threshold value.

10. Further comprising exposure control means for controlling the start of exposure of the next frame, The imaging device according to any one of claims 1 to 6, wherein the exposure control means, when the number of SPAD pixels read by a predetermined time reaches a threshold value, starts reading the remaining pixels that have not been read yet based on an instruction from the outside, and starts the exposure of the next frame as soon as the reading of all SPAD pixels is completed.

11. Each of the plurality of signal processing means, A pixel counter that counts photons detected by the corresponding SPAD pixel, A time counter that counts the time since the corresponding SPAD pixel started counting photons, And coordinate holding means for holding coordinate information of the corresponding SPAD pixel, The imaging device according to claim 1, wherein the arbitration means outputs the value of the pixel counter or the value of the time counter and the coordinate information.

12. The imaging device, Data receiving means for receiving the value of the pixel counter or the value of the time counter output by the arbitration means and the coordinate information, Storage means for associating and storing the value of the pixel counter or the value of the time counter and the coordinate information, The imaging device according to claim 11, further comprising the above.

Images