Imaging device, imaging method, and computer program

Abstract

To provide an imaging device capable of improving image quality in response to changes in the brightness of the subject. [Solution] The imaging device includes a counter that counts the output of an avalanche photodiode provided for each pixel, a determination means that determines whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure and outputs a determination result, and a control means that changes at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter.

Description

Technical Field

[0001] The present invention relates to an imaging device, an imaging method, a computer program, and the like.

Background Art

[0002] In recent years, an imaging device has been proposed that digitally counts the number of photons arriving at an avalanche photodiode (APD) and performs photoelectric conversion by outputting the count value from each pixel.

[0003] Patent Document 1 describes an imaging device including an APD, a detection unit that detects an avalanche current, and a switch disposed between the APD and the detection unit. Further, Patent Document 1 describes a reset unit that applies a predetermined potential to the input unit of the switch to reset the node between the switch and the detection unit.

[0004] In Patent Document 1, by resetting the reset unit with a clock pulse at a constant period, it is possible to suppress the power consumption of the imaging device even when photons are incident on the APD at a high frequency. Further, accurate signal information in which the linearity between the number of photons incident on the APD and the count value of the photons detected by the imaging device is maintained can be obtained.

[0005] Also, when the count values corresponding to each exposure time reach a predetermined threshold at a plurality of exposure times shorter than the maximum exposure time, the operating voltage of the APD is changed. Thereby, the counting of the number of photons is paused, and the estimated value of the count is calculated and substituted, so that the power consumption can be reduced.

[0006] Here, the estimated value of the count is a count value that is expected to be obtained when the APD is exposed for the length of the maximum exposure time, based on the count value when the count is paused at an exposure time shorter than the maximum exposure time.

Prior Art Documents

Patent Documents

[0007] [Patent Document 1] Japanese Patent Publication No. 2021-019281 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] However, in the prior art described in Patent Document 1, the exposure time is fixed at a short duration in order to prioritize the dynamic range. Consequently, errors are likely to occur in the incidence frequency of photons during the exposure and non-exposure periods, and the accuracy of the count estimate deteriorates.

[0009] In other words, because image information is generated from count values ​​that are out of sync with the frequency of photon incidence to each pixel, it becomes difficult to accurately estimate the number of photons containing information about the subject, which presents a challenge in representing the brightness of the subject in the image.

[0010] One of the objectives of the present invention is to provide an imaging device capable of improving image quality in response to changes in the brightness of the subject, in order to solve the above-mentioned problems. [Means for solving the problem]

[0011] The imaging device according to an embodiment of the present invention is A counter that counts the output of an avalanche photodiode provided for each pixel, A determination means that determines whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure, and outputs a determination result, Control means for changing at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter, It is characterized by having the following features. [Effects of the Invention]

[0012] According to the present invention, an imaging device capable of improving image quality in response to changes in the brightness of the subject can be realized. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows an example configuration of a photoelectric conversion element according to Embodiment 1 of the present invention. [Figure 2] This figure shows an example configuration of the sensor substrate 11 according to Embodiment 1 of the present invention. [Figure 3] This figure shows an example configuration of a circuit board 21 according to Embodiment 1 of the present invention. [Figure 4] Figures 2 and 3 show examples of equivalent circuits for the photoelectric conversion unit 102 and the signal processing circuit 103 corresponding to the photoelectric conversion unit 102. [Figure 5] This figure schematically illustrates an example of the relationship between the operation of the APD201 and the output signal according to Embodiment 1 of the present invention. [Figure 6] This is a timing chart illustrating an example of the operation of the signal processing circuit 103 according to Embodiment 1 of the present invention. [Figure 7] This figure shows an example of the relationship between the exposure time for each pixel 101 included in the photoelectric conversion element 100 according to Embodiment 1 of the present invention and the count value of the counter circuit 211. [Figure 8] This figure shows an example of the relationship between the exposure time for each pixel 101 and the count value of the counter circuit 211 according to Embodiment 1 of the present invention. [Figure 9] This is a functional block diagram showing an example configuration of the imaging device 900 according to Embodiment 1. [Figure 10] This flowchart shows an example of the process for determining the exposure time of the next frame in Embodiment 1. [Figure 11] (A) and (B) are diagrams illustrating examples of the relationship between exposure time and count distribution within one frame, respectively, to explain the processing example in Figure 10. [Figure 12] This flowchart shows an example of the process for determining the exposure time of the next frame in Embodiment 2. [Figure 13] (A) and (B) are diagrams illustrating examples of the relationship between exposure time and count distribution within one frame, respectively, to explain the processing example in Figure 12. [Figure 14]It is a functional block diagram showing a configuration example of an imaging device 900 according to Embodiment 3. [Figure 15] It is a flowchart showing an example of a process for determining the exposure time of the next frame in Embodiment 3. [Figure 16] (A) and (B) are diagrams each showing an example of the distribution of Tcode output from each pixel of the photoelectric conversion element 100 within one frame. [Figure 17] It is a diagram showing an example of the relationship between the exposure time and the count distribution within one frame in Embodiment 4.

Modes for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same members or elements are given the same reference numerals, and overlapping descriptions are omitted or simplified.

[0015] <Embodiment 1> FIG. 1 is a diagram showing a configuration example of a photoelectric conversion element according to Embodiment 1 of the present invention. In this embodiment, the photoelectric conversion element 100 is configured by laminating two substrates, a sensor substrate 11 and a circuit substrate 21, and electrically connecting the substrates. The circuit substrate 21 includes a circuit region 22 that processes signals detected in the pixel region 12.

[0016] FIG. 2 is a diagram showing a configuration example of the sensor substrate 11 according to Embodiment 1 of the present invention. The pixel region 12 of the sensor substrate 11 includes a plurality of pixels 101 two-dimensionally arranged in a plurality of rows and columns.

[0017] The pixel 101 includes a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter, APD). The number of rows and columns of the plurality of pixels forming the pixel region 12 is not particularly limited.

[0018] Figure 3 shows an example of the configuration of a circuit board 21 according to Embodiment 1 of the present invention. The circuit board 21 has a signal processing circuit 103 for processing the charge photoelectrically converted by the photoelectric conversion unit 102 in Figure 2, a vertical scanning circuit unit 110, a horizontal scanning circuit unit 111, a readout circuit 112, a signal line 113, an output circuit 114, and a pulse signal generation unit 115.

[0019] The signals output from the photoelectric conversion unit 102 of each pixel are processed by the signal processing circuit 103 corresponding to each pixel. The signal processing circuit 103 includes a counter and memory, and the memory stores a digital value that counts the number of photons.

[0020] The horizontal scanning circuit unit 111 inputs control pulses to the readout circuit 112 that sequentially select the signal lines 113 of each column in order to read the signal from the memory of each pixel in which the digital value is held.

[0021] Signals are output to the signal line 113 from the signal processing circuits 103 of multiple pixels in the row selected by the vertical scanning circuit unit 110. The signals output to the signal line 113 of each column are sequentially selected by the readout circuit 112 and output to the outside of the photoelectric conversion element 100 via the output circuit 114.

[0022] As shown in Figures 2 and 3, multiple signal processing circuits 103 are arranged in the area that overlaps with the pixel area 12 in a plan view. Then, in a plan view, the vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the pulse signal generation section 115 are arranged so as to overlap with the peripheral area (non-pixel area) between the edge of the sensor substrate 11 and the edge of the pixel area 12.

[0023] In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. The vertical scanning circuit section 110, the horizontal scanning circuit section 111, the readout circuit 112, the output circuit 114, and the pulse signal generation section 115 are arranged in the region that overlaps with the non-pixel region in a plan view.

[0024] The vertical scanning circuit 110 receives control pulses supplied from the pulse signal generation unit 115 and supplies multiple types of control pulses to multiple pixels on a row-by-row basis. The vertical scanning circuit 110 may also be composed of a shift register or address decoder that connects multiple rows as a single unit, and may be configured to enable high-speed reading by reading multiple rows at once.

[0025] In particular, in imaging devices that digitally count the number of photons arriving at the APD and output the count value as a photoelectrically converted digital signal from the pixel, the operation of the counter circuit that digitally counts the number of photons takes time. Therefore, it is preferable to read out multiple rows simultaneously for high-speed readout.

[0026] In other words, the vertical scanning circuit unit 110, which functions as a readout circuit for reading pixel signals from pixels, may simultaneously read out pixel signals from pixels in the first row and pixel signals from pixels in the second row. The pulse signal generation unit 115 sets threshold information that serves as the judgment criterion and exposure time information that indicates the timing for performing the judgment to the count judgment circuit, which will be described later.

[0027] Note that the arrangement of the signal line 113, the read circuit 112, and the output circuit 114 is not limited to the example shown in Figure 3. For example, the signal line 113 may be arranged so as not to extend in the row direction, and the read circuit 112 may be placed at the end of the signal line 113.

[0028] Furthermore, the signal processing circuit 103 does not necessarily need to be provided in one unit for each photoelectric conversion unit; a single signal processing unit may be shared by multiple photoelectric conversion units, and sequential signal processing may be performed.

[0029] Figure 4 shows an example of the equivalent circuits of the photoelectric conversion unit 102 and the signal processing circuit 103 corresponding to the photoelectric conversion unit 102, as seen in Figures 2 and 3.

[0030] The APD201 generates charge pairs corresponding to incident light through photoelectric conversion. One of the two nodes of the APD201 is connected to a power line that supplies the drive voltage VL (first voltage).

[0031] Furthermore, the other of the two nodes of the APD201 is connected to a power line supplied with a drive voltage VH (second voltage) higher than voltage VL via a quench element 202. In Figure 4, one node of the APD201 is the anode, and the other node of the APD is the cathode.

[0032] A reverse bias voltage is supplied to the anode and cathode of the APD201, causing it to perform avalanche multiplication. This voltage supply causes the charge generated by the incident light to undergo avalanche multiplication, resulting in the generation of an avalanche current.

[0033] Furthermore, the APD201's operating mode is classified into two types depending on the value of the reverse bias voltage used to operate it. The two modes are Geiger mode, in which the Anode and cathode voltage difference is greater than the breakdown voltage, and Linear mode, in which the Anode and cathode voltage difference is near or below the breakdown voltage.

[0034] An APD that operates in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In the case of a SPAD, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V.

[0035] The quench element 202 is connected to the power line to which the drive voltage VH is supplied and to one of the nodes, either the anode or cathode, of the APD201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, suppressing the voltage supplied to the APD201 and thereby suppressing avalanche multiplication (quench operation).

[0036] Furthermore, the quench element 202 has the function of returning the voltage supplied to the APD201 to the drive voltage VH by supplying current to compensate for the voltage drop caused by the quench operation (recharge operation).

[0037] In this embodiment, the quench element 202 is composed of a MOS transistor, and the on / off state of the quench element 202 is controlled by a control signal CLK connected to the gate of the quench element. The control signal CLK is generated by the pulse signal generation unit 115. In the following description, the control signal CLK may be referred to as control CLK.

[0038] The waveform shaping unit 210 shapes the voltage change at the cathode of the APD201 obtained when the APD201 detects a photon, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210.

[0039] Figure 4 shows an example where one inverter is used as the waveform shaping unit 210, but a circuit with multiple inverters connected in series may also be used, or other circuits that have a waveform shaping effect may be used.

[0040] The counter circuit 211 functions as a counter for counting the output of the avalanche photodiode provided for each pixel, counting the pulse signal output from the waveform shaping unit 210 and storing the count value.

[0041] Furthermore, when the control pulse RES is supplied via the drive line 214, the signal held in the counter circuit 211 is reset. In addition, when the control pulse STOP is supplied via the drive line 217, the counter circuit 211 continues to hold the count value until the control pulse RES is supplied.

[0042] The count determination circuit 212 is supplied with the count value held by the counter circuit 211 via the drive line 216, and a control pulse φt is supplied from the pulse signal generation unit 115 via the drive line 218 at predetermined intervals, for example, four times during one frame period.

[0043] The count determination circuit 212 compares the above count value with a predetermined threshold (default value) at each timing when it receives a control pulse φt. If it determines that the count value exceeds the above threshold (default value), it supplies a control pulse STOP to the counter circuit 211 via the drive line 217.

[0044] Furthermore, the pixel-by-pixel count determination circuit 212 functions as a determination means that determines whether the count value of the counter for each pixel has reached a specified value within a predetermined determination period from the start of exposure, and outputs a determination result. The predetermined determination period is, for example, the shortest determination period among multiple configurable determination periods, i.e., the minimum determination period.

[0045] Furthermore, the count determination circuit 212 outputs exposure time information (the exposure time code when the count threshold is exceeded) and the count value at the timing of the control pulse φt to the selection circuit 213 via the drive line 219.

[0046] For example, if each pixel output is 14 bits, 3 bits are allocated to exposure time information (Tcode), and the remaining 11 bits are allocated to the count value. In this way, the count determination circuit 212 outputs this 14-bit signal as the output of each pixel. The timing chart of the control pulse φt will be described later using Figure 6.

[0047] The selection circuit 213 receives control pulses SEL from the vertical scanning circuit section 110 in Figure 3 via the drive line 215 (not shown in Figure 3) in Figure 4, which switches the electrical connection between the count determination circuit 212 and the signal line 113. The selection circuit 213 includes, for example, a buffer circuit for outputting signals, and outputs the output signal from the pixel count determination circuit 212 to the signal line 113.

[0048] Furthermore, switches such as transistors may be placed between the quench element 202 and the APD201, or between the photoelectric conversion unit 102 and the signal processing circuit 103, to switch the electrical connections. Similarly, the supply of voltage VH or voltage VL to the photoelectric conversion unit 102 may be electrically switched using switches such as transistors.

[0049] Figure 5 is a schematic diagram illustrating an example of the relationship between the operation of the APD201 and the output signal according to Embodiment 1 of the present invention. Figure 5 schematically shows the relationship between the control signal CLK to the quench element 202, the voltage at node A, the voltage at node B, and the count value of the counter circuit 211 in the photoelectric conversion element.

[0050] When the control signal CLK is at a high level, the drive voltage VH is less likely to be supplied to the APD201, and when the control signal CLK is at a low level, the drive voltage VH is supplied to the APD201.

[0051] A high level control CLK is, for example, 1V, and a low level control CLK is, for example, 0V. When the control CLK is high level, the quench element 202 is turned off, and when the control CLK is low level, the quench element 202 is turned on.

[0052] The resistance of the quench element 202 when the control CLK is at a high level is higher than the resistance of the quench element 202 when the control CLK is at a low level.

[0053] When the control signal CLK is at a high level, even if avalanche multiplication occurs in the APD201, recharge operation is less likely to occur, and the voltage supplied to the APD201 becomes below the breakdown voltage of the APD201. Therefore, the avalanche multiplication operation in the APD201 stops.

[0054] At time t1, the control CLK changes from a high level to a low level, the quench element 202 turns on, and the recharge operation of the APD201 begins. As a result, the cathode voltage of the APD201 transitions to a high level.

[0055] Then, the difference in voltage applied to the anode and cathode of the APD201 puts the APD201 into an avalanche multiplication state. The cathode voltage is the same as the voltage of node A. Subsequently, when the cathode voltage transitions from a low level to a high level, the voltage of node A becomes greater than or equal to the threshold at time t2.

[0056] When the voltage at node A exceeds the threshold, the pulse signal output from node B of the waveform shaping unit 210 is inverted, changing from a high level to a low level.

[0057] Once recharging is complete, a voltage of (drive voltage VH - drive voltage VL) is applied to APD201. Then, between time t2 and time t3, the control CLK becomes high, and the quench element 202 turns off.

[0058] Next, at time t3, when a photon is incident on APD201, avalanche multiplication occurs in APD201, an avalanche multiplication current flows through the quench element 202, and the cathode voltage, i.e., the voltage at node A, drops.

[0059] If the voltage at nodeA falls below the threshold during its voltage drop, the voltage at nodeB changes from a low level to a high level. In other words, the portion of the output waveform at nodeA that exceeds the threshold is reshaped by the waveform shaping unit 210 and output as a low-level signal at nodeB.

[0060] Then, the rising edge of nodeB from low level to high level is counted by the counter circuit 211, and the count value of the counter signal output from the counter circuit 211 increases by 1 LSB.

[0061] Although photons are incident on APD201 between time t3 and time t4, the quench element 202 is in the off state, and the voltage applied to APD201 is not a voltage difference that allows for avalanche multiplication, so the voltage level of node A does not exceed the judgment threshold.

[0062] At time t4, the control CLK changes from a high level to a low level, and the quench element 202 turns on. Consequently, a current flows through node A to compensate for the voltage drop from the drive voltage VL, and the voltage at node A returns to its original voltage level. At this time, at time t5, the voltage at node A exceeds the threshold, so the pulse signal at node B inverts, changing from a high level to a low level.

[0063] Subsequently, at time t6, the control CLK changes from a low level to a high level, and node A settles back to its original voltage level. In principle, the period during which the control CLK is at a low level only needs to be set to be longer than the period during which node A transitions from a low level to a high level.

[0064] In Figure 5, the period during which the control CLK is at a low level is set to be the same as the period during which nodeA transitions from a low level to a high level. This allows the frequency of the control CLK to be set higher, thereby reducing the effect of the "nonlinear relationship between the number of output signals and the number of input signals," which will be discussed later.

[0065] Subsequently, as explained in the sections from time t1 to t6, the voltages of each node and signal line change in response to the control CLK and the incidence of photons. However, when the APD recharge frequency is controlled by the control CLK, the relationship between the number of output signals and the number of input signals is not linear. The number of input signals refers to the number of photons incident on the APD, and the number of output signals refers to the photon count value detected by the imaging device.

[0066] In SPADs, when avalanche breakdown occurs, secondary photons are emitted, causing emission crosstalk with adjacent pixels. However, if we ignore the effects of emission crosstalk, the relationship between the number of output signals and the number of input signals can be theoretically derived as follows.

[0067] Specifically, when the number of input signals is Nph, the number of output signals is Nct, the frequency of the control CLK (number of CLK cycles per unit time) is f, and the exposure time is T, the following equation 1 holds true.

number

[0068] Figure 6 is a timing chart illustrating an example of the operation of the signal processing circuit 103 according to Embodiment 1 of the present invention. In this embodiment, the exposure time of each pixel is set by the pulse signal generation unit 115 to a predetermined exposure time represented by T / (n to the power of (m-1)).

[0069] T is the maximum exposure time within one frame, and m is any integer satisfying m≧1. Figure 6 shows the timing chart when 1≦m≦4 is set. Therefore, the drive line 218 in Figure 4 is supplied with a control pulse φt that becomes high only at the moment the exposure time determined by t=T / (n^(m-1)) is reached.

[0070] Furthermore, if the count value of the counter circuit 211 reaches a predetermined count threshold at the moment of exposure time T / (n^(m-1)) for four exposure times when m=1, 2, 3, and 4, the photoelectric conversion element 100 is switched from Geiger mode to linear mode and the APD201 is put into standby mode.

[0071] Once paused, the pulse signal from the waveform shaping unit 210 is not output, and the count of the counter circuit 211 remains unchanged. The count determination circuit 212 then outputs a 14-bit signal to the selection circuit 213, consisting of, for example, a 3-bit Tcode corresponding to T / (n to the power of (m-1)), which represents the exposure time according to the control pulse φt, and, for example, an 11-bit count value.

[0072] Figure 7 shows an example of the relationship between the exposure time for each pixel 101 included in the photoelectric conversion element 100 according to Embodiment 1 of the present invention and the count value of the counter circuit 211. In Figure 7, similar to Figure 6, the pulse signal generation unit 115 sets a predetermined exposure time T / (n to the power of (m-1)), (where 1 ≤ m ≤ 4).

[0073] When the upper limit of the count of the counter circuit 211 is set to Cmax, the count threshold is set to Cmax / n for reasons described later. In Figure 7, it is assumed that the count value of a certain pixel increases in proportion to time.

[0074] As shown in Figure 6, the count determination circuit 212 determines whether the count value exceeds the count threshold at the moment T / (n^(m-1)) in order of shortest exposure time. As shown in Figure 7, when the count value increases, it is determined that the count threshold Cmax / n was not exceeded at T / n^3 and T / n^2, but was exceeded at T / n. Note that n^3 means n to the power of 3, and n^2 means n to the power of 2.

[0075] Furthermore, if it is determined that the count threshold Cmax / n has been exceeded at the time T / n, a control pulse STOP is supplied from the count determination circuit 212 to the counter circuit 211 via the drive line 217, stopping the count of the counter circuit 211.

[0076] Then, a total of 14 bits, consisting of, for example, an 11-bit count value Cout for the exposure time T / n and, for example, a 3-bit exposure time information Tcode corresponding to the exposure time T / n, is output from the count determination circuit 212 to the selection circuit 213.

[0077] In other words, the count determination circuit 212, which serves as the determination means, outputs the Tcode as the determination result and the counter's count value for each pixel.

[0078] Table 1 below shows an example of the relationship between the exposure time when the count threshold is exceeded in Embodiment 1 and the exposure time information (Tcode) corresponding to each exposure time. [Table 1]

[0079] Furthermore, while the count determination circuit 212 may output the exposure time T / n directly as time information to the selection circuit 213, as shown in Table 1, it is also possible to determine and output exposure time information (Tcode) corresponding to the exposure time when the count threshold is exceeded. Note that the Tcode shown in Table 1 is just one example of the data format output from the count determination circuit 212, and is not limited to this.

[0080] Furthermore, the count estimate (Cest) shown in Figure 7 is calculated using the formula Cest = Cout × n, assuming that the count increases at the same rate during the exposure time from 0 to T / n and the non-exposure time from T / n to T.

[0081] As mentioned earlier, the above count estimation method assumes that the count increases at the same rate during the exposure time from 0 to T / n and the non-exposure time from T / n to T. Therefore, if the upper limit of the count, Cmax, is reached before the exposure time T / n is reached, the rate of increase during the exposure time from 0 to T / n cannot be estimated correctly, and the accuracy of the count estimation decreases.

[0082] Therefore, in this embodiment, a threshold determination is performed before the upper count limit Cmax is reached in order to maintain the accuracy of the count estimation. Also, when the recharge frequency of the APD is controlled by the control CLK, the relationship between the number of output signals and the number of input signals is not linear, so in this embodiment, linearity correction is performed before the count estimation.

[0083] Figure 8 shows an example of the relationship between the exposure time for each pixel 101 and the count value of the counter circuit 211 according to Embodiment 1 of the present invention. Figure 8 shows how the count increases when the count value becomes equal to the count threshold at the moment of each exposure time for which threshold determination is performed.

[0084] As described above, in this embodiment, in order to maintain the accuracy of the count estimation, a threshold determination is made before the upper limit of the count, Cmax, is reached. Therefore, the timing of the threshold determination when the count increases at a rate that reaches Cmax at the moment of the maximum exposure time T is defined as t1. Also, the timing of the threshold determination when the count increases at a rate that reaches Cmax at the moment of t1 is defined as t2.

[0085] In other words, when the count threshold is Cmax / n, if the count increases at a rate that reaches Cmax at the moment of maximum exposure time T, the time until Cmax / n is reached is calculated as T × (1 / n), so t1 = T / n.

[0086] Similarly, t2 is calculated as t1 × (1 / n) = T / n^2. In this way, each exposure time used for threshold determination is calculated as T / (n^(m-1)).

[0087] Furthermore, as shown in Figure 4, when the APD recharge frequency is controlled by the control CLK, the relationship between the number of output signals and the number of input signals is not linear, as shown in Equation 1. Therefore, in this embodiment, linearity correction is performed based on the estimated count value calculated by the signal processing circuit 103.

[0088] Here, linearity correction refers to determining the number of input signals Nph from the number of output signals Nct per exposure time using the following equation 2, where f is the frequency of the control CLK (number of CLK cycles per unit time) and T is the length of the exposure time.

number

[0089] In this embodiment, the input signal number Nph derived by Equation 2 is the number of photons per exposure time for threshold determination. Therefore, the number of photons in one frame exposure time is calculated as Nph × (n to the power of (m-1)).

[0090] Figure 9 is a functional block diagram showing an example configuration of the imaging device 900 according to Embodiment 1. Note that some of the functional blocks shown in Figure 9 are realized by having the CPU 907, which acts as a computer included in the imaging device 900, execute a computer program stored in the memory 908, which acts as a storage medium.

[0091] However, some or all of these can be implemented in hardware. Hardware options include dedicated circuits (ASICs) and processors (reconfigurable processors, DSPs).

[0092] Furthermore, each functional block shown in Figure 9 does not necessarily have to be housed in the same enclosure; they may be composed of separate devices connected to each other via signal paths. The above explanation regarding Figure 9 also applies to Figure 14.

[0093] In Figure 9, the imaging device 900 includes the photoelectric conversion element 100 described in Figures 1 to 8, an optical imaging system 901, a signal processing unit 902, a timing generation unit 903, a control unit 904, a memory unit 905, and an I / F unit 906.

[0094] The 901 optical imaging system consists of a zoom lens, a focusing lens, an aperture, a shutter, an optical low-pass filter, an IR cut filter, and other components.

[0095] The signal processing unit 902 performs calculation processing to calculate an estimated count value based on the count value output from the photoelectric conversion element 100, the exposure time information, and the timing signal output from the timing generation unit 903. In other words, the signal processing unit 902 functions as a signal processing means that estimates the count value within a predetermined exposure time based on the determination result and the count value.

[0096] Furthermore, the signal processing unit 902 performs linearity correction processing and development processing based on the estimated count value. That is, the signal processing unit 902 functions as a correction means that outputs a signal Nph that corrects the nonlinearity of the count value using Equation 2, when the recharge frequency of the avalanche photodiode is f, the exposure time is T, and the count value per exposure time is Nct.

[0097] The timing generation unit 903 outputs various timing signals to the photoelectric conversion element 100 and the signal processing unit 902.

[0098] The control unit 904 outputs a timing signal control signal to the timing generation unit 903 based on the image signal output from the signal processing unit 902. The memory unit 905 stores exposure time information and a correction LUT (Look Up Table) for linearity correction.

[0099] Furthermore, the control unit 904 has a built-in CPU 907, which functions as a control means to control the operation of each part of the imaging device 900 based on a computer program stored in the memory 908, which serves as a storage medium.

[0100] The I / F unit 906 outputs video in formats such as SDI (Serial Digital Interface) and HDMI (High Definition Multimedia Interface) to a display device connected to the outside of the imaging device 900.

[0101] Figure 10 is a flowchart showing an example of the process for determining the exposure time of the next frame in Embodiment 1, and is executed in frame cycles. The CPU 907, which acts as a computer within the control unit 904, executes the computer program stored in memory 908, thereby sequentially performing each step of the flowchart in Figure 10.

[0102] The next frame refers to the frame in which the imaging device 900 will perform exposure time processing, relative to the frame in which it is currently processing exposure time.

[0103] In step S1001, the signal processing unit 902 acquires exposure time information included in each pixel output from the photoelectric conversion element 100. The exposure time information in the embodiment of the present invention is output as the exposure time information Tcode shown in Table 1.

[0104] In step S1002, the signal processing unit 902 determines whether or not the determination condition 1 is met based on the exposure time information Tcode acquired by the signal processing unit 902. In Embodiment 1, determination condition 1 is the condition that a pixel with Tcode "111", indicating that the count stopped at the minimum exposure time, exists within the 1-frame period.

[0105] Since the count value and exposure time information (Tcode) are output on a pixel-by-pixel basis, a check is performed each time exposure time information for a pixel is output. If the Tcode "111" is present even once within a frame, the condition 1 is met, so the result is "Yes," and the process ends without changing the exposure time.

[0106] If the Tcode "111" does not appear even once within a frame, the determination condition 1 is not met, so the result is determined to be No, and the exposure time is changed in step S1003 to increase the minimum exposure time. Specifically in this embodiment, the minimum exposure time is increased by decreasing n in each exposure time T / (n to the power of (m-1)).

[0107] Figures 11(A) and 11(B) illustrate examples of the relationship between exposure time and count distribution within a single frame, respectively, to explain the processing example in Figure 10. Count distribution 1101 and count distribution 1102 show examples of the distribution of count values ​​for each pixel within a single frame, respectively. Note that the exposure time shown in Figure 11(A) is for an example where n=4 and m≦4, and the exposure time shown in Figure 11(B) is for an example where n=2 and m≦4.

[0108] In FIG. 11(A), since the count distribution 1101 does not overlap with the intersection of T / 64 and the count threshold Cmax / 4, it indicates that there is no pixel where the count stops at the timing of the minimum exposure time T / 64.

[0109] At this time, since Tcode "111" is not output from any pixel in one frame, it is determined as No in step S1002. Incidentally, the fact that Tcode "111" is not output from any pixel in one frame indicates that the minimum exposure time is set too short.

[0110] Incidentally, if the exposure time is short and the count value when exposed at the maximum exposure time is estimated, the count estimation accuracy will decrease. Therefore, as long as Tcode "111" is not output, it is desirable to increase the count estimation accuracy by decreasing n and increasing the minimum exposure time. Therefore, the process proceeds to step S1003, and a change process for decreasing n in the next frame is performed.

[0111] As described above, the change in the exposure time in step S1003 is a process of decreasing n in T / (n to the power of (m - 1)). The exposure time shown in FIG. 11(A) shows an example of n = 4 and m ≤ 4. In Embodiment 1, n is decreased to increase the minimum exposure time. That is, in the next frame, n is set within the range of 1 < n < 4.

[0112] Within this range, the value of n can be arbitrarily determined. However, in reality, since the arithmetic resources for linearity correction described later are limited, the value of n that can be set is restricted in advance in the imaging device.

[0113] Therefore, the exposure time is changed using n = 2, the next smallest settable n after n = 4. That is, in FIG. 11(B), for simplicity of calculation, n is set as a multiple of two, and an example of the relationship between the exposure time and the count distribution when the next smallest n after n = 4 is n = 2 is shown.

[0114] At this point, the intersection of the minimum exposure time T / 8 and the count threshold Cmax / 2 coincides with the count distribution 1102. If the Tcode "111" is not output from any pixel in a frame even when the minimum exposure time is increased by setting n=2, the minimum exposure time is further increased in the next frame.

[0115] n is not limited to an integer, but it must be greater than 1. In the above embodiment, a method for setting the next smallest n after the currently set n is described, but it is also possible to further reduce n based on whether or not other Tcodes "011" and "001" were output from the photoelectric conversion element 100 within one frame.

[0116] Alternatively, the exposure time may be changed by using Tcodes other than those in Table 1 above, corresponding to the number of n and m, to determine whether or not a Tcode corresponding to the minimum exposure time has been output from any pixel of the photoelectric conversion element 100.

[0117] In Figure 11(A) of Embodiment 1, the count threshold is set to Cmax / 4, but in Figure 11(B), the count threshold is set to Cmax / 2. That is, since the default count threshold is set to Cmax / n, when n=4 is changed to n=2 in order to increase the minimum exposure time, the count threshold is increased accordingly. This improves the accuracy of the estimation.

[0118] In step S1003, both the default value and the judgment period are changed, but it is sufficient to change at least one of the default value and the judgment period. That is, in this embodiment, if the pixel count value has not reached the default value during the shortest judgment period among the multiple configurable judgment periods, it is sufficient to change at least one of the default value and the judgment period.

[0119] In this embodiment, steps S1002 and S1003 function as control steps (control means). Furthermore, the control steps (control means) of this embodiment change at least one of the specified value and determination period for the next frame based on the determination results of multiple pixels for one frame and the count value of the counter.

[0120] Furthermore, a holding means for storing the above-mentioned determination period and specified value may be provided, and at least one of the specified value and determination period may be changed based on the determination period and specified value that are pre-stored in the holding means.

[0121] In this embodiment, as described above, after estimating the count from the count value and Tcode, the linearity correction shown in Equation 2 is performed. This makes it possible to maintain more accurate count estimation when photons are incident on the photoelectric conversion element 100 at a high frequency.

[0122] Furthermore, while the calculation in Equation 2 may be performed by the signal processing unit 902, a LUT for linearity correction may be stored in the memory unit 905, and linearity correction may be performed based on the LUT. That is, the signal processing unit 902, as a correction means, may perform linearity correction using a lookup table that stores data corresponding to Equation 2.

[0123] Furthermore, the minimum exposure time for the photoelectric conversion element 100 is Tmin, and the estimated count value is calculated as Cest = Cout × (1 / Tmin). If a LUT is created so that all possible exposure times from the minimum exposure time to the maximum exposure time can be set, the amount of data input to the LUT will increase by Tmax / Tmin times when the maximum exposure time is Tmax.

[0124] In this case, the amount of data to be stored is calculated as (counter upper limit) × (1 / Tmin), but this consumes the capacity of the memory unit 905. Therefore, when using a LUT, it is desirable to pre-determine the exposure time and create a LUT that uses the count value calculated from that exposure time as input data.

[0125] In this case, when changing the exposure time, the LUT referenced during linearity correction is also changed at the same time as the exposure time. This minimizes the amount of data in the LUT while maintaining more accurate estimation of the count value.

[0126] As described above, in the method according to Embodiment 1, when the minimum exposure time setting of the imaging device is too short relative to the frequency of photons incident on the photoelectric conversion element 100, the minimum exposure time is readjusted to be longer. This ensures that the dynamic range is maintained while maximizing the accuracy of the count estimation, thereby avoiding black crushing and white blowout in the image and correctly representing the brightness of the subject in the image.

[0127] <Embodiment 2> The imaging device used in the description of Embodiment 2 has the same configuration as the system block diagram of Embodiment 1, but the determination process in step S1002 and the method of changing the exposure time in step S1003 are different.

[0128] Figure 12 is a flowchart showing an example of the process for determining the exposure time of the next frame in Embodiment 2. The CPU 907, which acts as a computer within the control unit 904, executes a computer program stored in the memory 908, thereby sequentially performing each step of the flowchart in Figure 12.

[0129] Furthermore, Figures 13(A) and 13(B) are diagrams illustrating examples of the relationship between exposure time and count distribution within one frame, respectively, to explain the processing example in Figure 12. In Figures 13(A) and 13(B), count distribution 1301 and count distribution 1302 show examples of the distribution of count values ​​for each pixel within one frame. Step S1001 is the same process as in Embodiment 1, so its explanation is omitted.

[0130] In step S1202 of Figure 12, the signal processing unit 902 determines whether or not the determination condition 2 is met based on the exposure time information Tcode acquired by the signal processing unit 902. Determination condition 2 is the condition that there is a pixel that has reached the upper count limit with the minimum exposure time.

[0131] In other words, in Embodiment 2, it is determined whether the count value of any pixel has reached the upper count limit at the minimum exposure time. If it is determined to be Yes, the process proceeds to step S1203. In step S1203, the minimum exposure time is shortened based on the exposure time information Tcode acquired by the signal processing unit 902.

[0132] As mentioned above using Figure 8, the exposure time is set so that, in principle, a threshold check is performed before the counter reaches its upper limit. However, when photographing an extremely bright subject, depending on the setting of the minimum exposure time, it is possible that the upper limit may be reached at the minimum exposure time.

[0133] Figure 13(A) shows one example, indicating that there are pixels that have already reached the upper count limit at the minimum exposure time T / 8. In this case, even though the upper count limit is reached at an exposure time shorter than T / 8, the Tcode output from the photoelectric conversion element 100 is T / 8, resulting in an error in the count estimation.

[0134] Therefore, in step S1202, it is determined whether the condition that there is a pixel that has reached the upper count limit is met, and if it is determined to be Yes, in step S1203 the exposure time is changed to shorten the minimum exposure time.

[0135] In step S1003, the exposure time is changed by modifying n in T / (n to the power of (m-1)). The exposure time shown in Figure 13(A) is calculated with n=2 and m≦4. In Embodiment 1, we want to increase n in order to shorten the minimum exposure time, so in the next frame, we set n within the range of n>2.

[0136] Within this range, the value of n can be arbitrarily determined, but in practice, the values ​​of n that can be set are predetermined within the imaging device, due to the computational resources required for linearity correction mentioned above. Therefore, it is desirable to change the exposure time using n=4, which is the next largest possible value after n=2. The timing for changing the Tcode is the same as in Embodiment 1, at the next frame.

[0137] Figure 13(B) shows the exposure time settings when the exposure time is changed, with n=4 being the next largest n after n=2, demonstrating that there are no pixels that reach the upper limit of the count at the minimum exposure time.

[0138] In other words, setting a shorter minimum exposure time will not improve the dynamic range, meaning that the longest possible minimum exposure time is set while still ensuring the necessary dynamic range.

[0139] In Figure 13(A) of Embodiment 2, the count threshold is set to Cmax / 2, while in Figure 13(B), the count threshold is set to Cmax / 4. By reducing the count threshold in accordance with shortening the minimum exposure time, the dynamic range can be ensured.

[0140] In step S1203, both the default value and the determination period are changed, but it is sufficient to change at least one of the default value and the determination period. That is, in this embodiment, if the pixel count value reaches the upper limit of the counter during the shortest determination period among the multiple configurable determination periods, it is sufficient to change at least one of the determination period and the default value.

[0141] Thus, steps S1202 and S1203 function as control steps (control means) that change at least one of a predetermined value and a determination period based on the determination results of multiple pixels and the count value of a counter.

[0142] As described above, in the method described in Embodiment 2, when the minimum exposure time setting of the imaging device is too long relative to the frequency of photons incident on the photoelectric conversion element 100, the minimum exposure time is readjusted to be shorter. This ensures that the dynamic range is maintained while maximizing the accuracy of the count estimation, thereby avoiding underexposure and overexposure in the image and correctly representing the brightness of the subject in the image.

[0143] <Embodiment 3> Figure 14 is a functional block diagram showing an example configuration of the imaging device 900 according to Embodiment 3. Note that some of the functional blocks shown in Figure 14 are realized by having the CPU 907, which acts as a computer included in the imaging device 900, execute a computer program stored in the memory 908, which acts as a storage medium.

[0144] Embodiment 3 includes, in addition to the imaging device 900 system described in Embodiments 1 and 2, an input unit 1401 and an output unit 1402.

[0145] The input unit 1401 has a function for the user to input data for determining the level of predetermined thresholds (for example, threshold 1 and threshold 2 described later) to be compared with the count value, and for determining exposure time information. In other words, the input unit 1401 functions as an input means that receives signals from outside the imaging device for setting the conditions for changes in the control means.

[0146] Furthermore, the control unit 904 generates a timing signal based on the input from the input unit 1401 and outputs it to the timing generation unit 903.

[0147] The output unit 1402 outputs a display signal for displaying Tcode information (judgment results) output from multiple pixels of the photoelectric conversion element 100 on a display device (not shown). The output unit 1402 functions as an output means for outputting a display signal to display the judgment results of multiple pixels.

[0148] Figure 15 is a flowchart showing an example of the process for determining the exposure time of the next frame in Embodiment 3. The CPU 907, which acts as a computer within the control unit 904, executes a computer program stored in the memory 908, thereby sequentially performing each step of the flowchart in Figure 15.

[0149] Figures 16(A) and 16(B) show examples of the distribution of Tcode output from each pixel of the photoelectric conversion element 100 within one frame, respectively. In Figure 15, step S1001 is the same as in Embodiment 1, so its explanation is omitted.

[0150] In step S1502 of Figure 15, it is determined whether or not the judgment condition 3 is met. Judgment condition 3 is the condition that the distribution of Tcode "111" shown in Figure 16(A) is within a predetermined proportion range, that is, for example, between threshold 1 and threshold 2 (where threshold 1 < threshold 2). The acquisition method and calculation method are irrelevant as long as the predetermined proportion range is uniquely determined within the imaging device 900.

[0151] One possible method for specifying and calculating the predetermined percentage range is to pre-specify an acceptable range for the percentage of Tcode "111" for each Tcode within the imaging device 900. Alternatively, one could display the Tcode "111" information output from the output unit 1402 on a display unit connected to the outside of the imaging device 900, and then calculate the percentage based on that information and the data input from the input unit 1401.

[0152] The information displayed on the external display unit can be in any format as long as the user can identify the proportion of Tcode "111". For example, the Tcode distribution diagram shown in Figure 16(A) would be acceptable, or a diagram mapping the Tcode information for each pixel would also be acceptable.

[0153] When the Tcode distribution shown in Figure 16(A) is obtained, suppose in step S1502 the signal processing unit 902 determines that the proportion of Tcode "111" corresponding to the minimum exposure time is greater than a predetermined proportion (threshold 2). In that case, the determination condition 3 is not met, and therefore the result in step S1502 is determined to be No.

[0154] Then, in step S1503, the minimum exposure time is shortened so that the proportion of Tcode "111" falls within a predetermined proportion (threshold 1 or greater and threshold 2 or less). The timing of the judgment process can be the same as in Embodiment 1 and Embodiment 2, and only needs to be performed once per frame.

[0155] The change in the minimum exposure time in step S1503 can be controlled as in Embodiment 1 if the minimum exposure time is to be shortened relative to the frequency of photons incident on the photoelectric conversion element 100, and as in Embodiment 2 if the minimum exposure time is to be lengthened.

[0156] In other words, if the proportion of Tcode "111" is T2 or higher, the minimum exposure time should be shortened as in Embodiment 1, and if the proportion of Tcode "111" is less than T1, the minimum exposure time should be lengthened as in Embodiment 1. Furthermore, the count threshold should be decreased in proportion to shortening the minimum exposure time, and increased in proportion to lengthening the minimum exposure time.

[0157] Figure 16(B) is an example showing the distribution of Tcode after changing the minimum exposure time from Figure 16(A) to a shorter minimum exposure time. In this example, by shortening the minimum exposure time using the method of Embodiment 1, the distribution of Tcode shown in Figure 16(B) can be obtained from Figure 16(A).

[0158] By acquiring the distribution of Tcodes output from the photoelectric conversion element 100 using the method described in Embodiment 3, and controlling the exposure time with the control unit 904, the Tcodes can be controlled to a predetermined ratio.

[0159] Furthermore, by allowing the user to arbitrarily specify levels such as threshold 1 or threshold 2, it becomes possible to set the exposure time in accordance with the frequency of photons incident on the photoelectric conversion element 100. Therefore, by user specification, the accuracy of the count estimation can be maintained to the maximum extent while ensuring the dynamic range, thus avoiding underexposure and overexposure in the image and accurately representing the brightness of the subject in the image.

[0160] In step S1503, both the specified value and the judgment period are changed, but it is sufficient to change at least one of the specified value and the judgment period. That is, in this embodiment, it is sufficient to change at least one of the specified value and the judgment period based on the distribution state of the judgment results of multiple pixels.

[0161] Thus, steps S1502 and S1503 function as control steps (control means) that change at least one of a predetermined value and a determination period based on the determination results of multiple pixels and the count value of a counter.

[0162] <Embodiment 4> The imaging device described in Embodiment 4 is the same as the functional block diagram in Figure 14 of Embodiment 3, and the flowchart is also the same as in Embodiment 3, but the method for setting the exposure time is different.

[0163] Figure 17 shows an example of the relationship between exposure time and count distribution within one frame in Embodiment 4. Embodiments 1 to 3 assumed that the exposure time was set to T / (n to the power of (m-1)), but Embodiment 4 describes an example in which the exposure time is set to a time other than T / (n to the power of (m-1)).

[0164] In Figure 17, the exposure times are set to T / 2, T / 4, and T / 32, and do not conform to the formula T / (n to the power of (m-1)). Count distributions 1701, 1702, and 1703 show examples of the distribution of count values ​​for each pixel within a single frame.

[0165] Furthermore, count distribution 1702 shows a distribution that is spread to the left in Figure 17 compared to count distribution 1701, and count distribution 1703 shows a distribution that is spread even further to the left in Figure 17 compared to count distribution 1702.

[0166] Pixels belonging to count distribution 1701 and count distribution 1702 have not reached the upper limit of the count at the minimum exposure time T / 32, so their count values ​​can be estimated with accuracy.

[0167] However, pixels belonging to count distribution 1703 reach the upper count limit before the count threshold can be determined at the minimum exposure time T / 32, thus reducing the accuracy of the brightness count estimation for pixels distributed in the middle range.

[0168] On the other hand, in shooting situations where you want to accurately photograph dark subjects while also having bright subjects, exposure time settings such as T / 2, T / 4, and T / 32 shown in Figure 17 are effective. With the exposure time settings shown in Figure 17, the accuracy of the pixel count estimation for pixels belonging to the count distribution 1701 corresponding to the dark subject area can be maintained.

[0169] Furthermore, by using an exposure time setting such as T / 32, it is possible to improve the estimation accuracy of pixels belonging to count distribution 1703, which cannot be accurately counted with T / 8, which is set according to the formula T / (n to the power of (m-1)).

[0170] In other words, in Embodiment 4, the visibility of the image can be improved by setting the exposure time according to the brightness of the subject that the user wants to prioritize, rather than following the formula T / (n^(m-1)).

[0171] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible in accordance with the spirit of the present invention, and these are not excluded from the scope of the present invention. Furthermore, some of the above embodiments may be combined as appropriate.

[0172] Furthermore, the present invention includes, for example, a system that realizes the functions of the above embodiment using at least one processor such as a CPU, memory, and circuitry (e.g., an ASIC). Alternatively, multiple processors may be used for distributed processing.

[0173] Furthermore, in order to implement some or all of the control in the above embodiment, a computer program that implements the functions of the above embodiment may be supplied to the imaging device, etc., via a network or various storage media.

[0174] The computer (or CPU or MPU, etc.) in the imaging device may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention. The present invention includes the following combinations.

[0175] (Configuration 1) An imaging apparatus characterized by comprising: a counter for counting the output of an avalanche photodiode provided for each pixel; determination means for determining whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure and outputting a determination result; and control means for changing at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter.

[0176] (Configuration 2) The imaging apparatus according to Configuration 1, characterized in that the determination means outputs the determination result and the count value of the counter for each pixel.

[0177] (Composition 3) The imaging apparatus according to configuration 1 or 2, characterized in that the control means changes at least one of the specified value and the determination period if the count value of the pixel has not reached the specified value during the minimum determination period among a plurality of configurable determination periods.

[0178] (Configuration 4) The imaging apparatus according to any one of Configurations 1 to 3, characterized in that the control means changes at least one of the determination period and the specified value when the count value of the pixel reaches the upper limit of the counter during the minimum determination period among a plurality of settable determination periods.

[0179] (Configuration 5) The imaging apparatus according to any one of Configurations 1 to 4, characterized in that the control means changes at least one of the specified value and the determination period based on the determination result of a plurality of pixels for one frame and the count value of the counter.

[0180] (Configuration 6) The imaging apparatus according to any one of Configurations 1 to 5, characterized in that the control means changes at least one of the predetermined value and the determination period for the next frame based on the determination result of a plurality of pixels for one frame and the count value of the counter.

[0181] (Configuration 7) Having a holding means for holding the determination period and the specified value, The imaging apparatus according to any one of configurations 1 to 6, characterized in that the control means changes at least one of the determination period and the determination period based on the determination period and the determination period which are held in advance by the holding means.

[0182] (Configuration 8) An imaging device according to any one of Configurations 1 to 7, characterized in that it has a signal processing means for estimating a count value within a predetermined exposure time based on the determination result and the count value.

[0183] (Configuration 9) When the recharge frequency of the avalanche photodiode is f, the exposure time is T, and the count value per exposure time is Nct, the following equation

number

[0184] (Configuration 10) The imaging apparatus according to Configuration 9, characterized in that the correction means performs the correction using a lookup table that stores data corresponding to the formula.

[0185] (Configuration 11) The imaging apparatus according to any one of Configurations 1 to 10, characterized in that the control means changes at least one of the specified value and the determination period based on the distribution state of the determination results of the plurality of pixels.

[0186] (Configuration 12) An imaging device according to any one of Configurations 1 to 11, characterized in that it has an input means for receiving a signal from outside the imaging device for setting the conditions for the change in the control means.

[0187] (Configuration 13) An imaging device according to any one of Configurations 1 to 12, characterized in that it has an output means for outputting a display signal for displaying the determination results of a plurality of pixels.

[0188] (Method) An imaging method for controlling an imaging apparatus having a counter for counting the output of an avalanche photodiode provided for each pixel, and determination means for determining whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure and outputting a determination result, the method comprising a control step of controlling at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter.

[0189] (Program) A computer program for controlling each means of the imaging device described in any one of configurations 1 to 13 by computer. [Explanation of symbols]

[0190] 100: Photoelectric conversion element 11: Sensor board 12: Pixel area 21: Circuit board 22: Circuit area 101: Pixels 102: Photoelectric conversion unit 103: Signal Processing Circuit 110: Vertical scanning circuit 111: Horizontal scanning circuit 112: Readout circuit 113: Signal line 114: Output Circuit 115: Pulse signal generation unit 201: Avalanche photodiode 202: Quench element 210: Waveform shaping section 211: Counter circuit 213: Selection Circuit 900: Imaging device 901: Optical imaging system 902: Signal Processing Unit 903: Timing generation unit 904: Control Unit 905: Memory section 906:I / F section 1401: Input section 1402: Output section

Claims

1. A counter that counts the output of an avalanche photodiode provided for each pixel, A determination means that determines whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure, and outputs a determination result, Control means for changing at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter, An imaging device characterized by having the following features.

2. The imaging apparatus according to claim 1, characterized in that the determination means outputs the determination result and the count value of the counter for each pixel.

3. The imaging apparatus according to claim 1, wherein the control means changes at least one of the specified value and the determination period if the count value of the pixel has not reached the specified value during the minimum determination period among a plurality of configurable determination periods.

4. The imaging apparatus according to claim 1, wherein the control means changes at least one of the determination period and the specified value when the count value of the pixel reaches the upper limit of the counter during the minimum determination period among a plurality of configurable determination periods.

5. The imaging apparatus according to claim 1, characterized in that the control means changes at least one of the specified value and the determination period based on the determination results of a plurality of pixels for one frame and the count value of the counter.

6. The imaging apparatus according to claim 1, characterized in that the control means changes at least one of the predetermined value and the determination period for the next frame based on the determination results of a plurality of pixels for one frame and the count value of the counter.

7. The system has a holding means for holding the aforementioned determination period and the aforementioned specified value, The imaging apparatus according to claim 1, characterized in that the control means changes at least one of the determination period and the determination period based on the determination period and the determination period which are held in advance by the holding means.

8. The imaging apparatus according to claim 1, characterized in that it has a signal processing means for estimating a count value within a predetermined exposure time based on the determination result and the count value.

9. When the recharge frequency of the avalanche photodiode is f, the exposure time is T, and the count value per exposure time is Nct, the following equation [Math 4] The imaging apparatus according to claim 1, further comprising a correction means for outputting a signal Nph obtained by correcting the nonlinearity of the count value using the method described above.

10. The imaging apparatus according to claim 9, characterized in that the correction means performs the correction using a lookup table that stores data corresponding to the formula.

11. The imaging apparatus according to claim 1, characterized in that the control means changes at least one of the specified value and the determination period based on the distribution state of the determination results of the plurality of pixels.

12. The imaging device according to claim 1, characterized in that it has an input means for receiving a signal from outside the imaging device for setting the conditions for the change in the control means.

13. The imaging apparatus according to claim 1, further comprising output means for outputting a display signal for displaying the determination results of a plurality of pixels.

14. An imaging method for controlling an imaging apparatus having a counter for counting the output of an avalanche photodiode provided for each pixel, and determination means for determining whether the count value of the counter for each pixel has reached a predetermined value within a predetermined determination period from the start of exposure and outputting a determination result, wherein An imaging method characterized by having a control step that controls at least one of the predetermined value and the determination period based on the determination results of a plurality of pixels and the count value of the counter.

15. A computer program for controlling each means of an imaging apparatus according to any one of claims 1 to 13 by computer.

Images