Information processing device and information processing method

Abstract

To provide an information processing device that can provide accurate correction values. [Solution] The information processing device includes an input unit that receives a signal based on the number of avalanches in a photodetector, an accumulation unit that accumulates a value corresponding to the number of avalanches based on the signal, and a generation unit that generates a correction value for the signal based on the accumulated value in the accumulation unit. This allows the information processing device to provide an accurate correction value. The information processing device further includes a correction unit that corrects the signal based on the correction value. The correction value corresponds to the DCR of the signal, and the correction unit subtracts the correction value from the signal.

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus and an information processing method. [Background technology]

[0002] In recent years, photodetectors capable of detecting weak light at the single-photon level have been used in a wide range of fields. One such photodetector is the APD (Avalanche PhotoDiod). APDs can amplify the signal charge excited by photons by several times to about a million times through avalanche multiplication generated by a strong electric field induced at the pn junction of a semiconductor. By greatly amplifying the signal of weak light using the high gain of avalanche multiplication, the signal-to-noise ratio (SNR) can be increased. In photon counting using APDs, the brightness of the input light, which has traditionally been treated as a continuous value, is counted as a discrete value called the number of photons. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2008-139586 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, the characteristics of the photodetector can change depending on the cumulative number of avalanche cycles. Patent Document 1 describes a method for correcting the voltage applied to a liquid crystal panel according to the operating time, but it is not possible to correct it according to the cumulative value.

[0005] Therefore, the object of the present invention is to provide an information processing device and an information processing method that can provide accurate correction values. [Means for solving the problem]

[0006] According to one disclosure of this specification, an information processing device is provided, comprising: an input unit that inputs a signal based on the number of avalanches in a photodetector; an accumulation unit that accumulates a value corresponding to the number of avalanches based on the signal; and a generation unit that generates a correction value for the signal based on the accumulated value in the accumulation unit.

[0007] According to one disclosure of this specification, an information processing method is provided which includes the steps of: inputting a signal based on the number of avalanches in a photodetector; accumulating a value corresponding to the number of avalanches based on the signal; and generating a correction value for the signal based on the accumulated value. [Effects of the Invention]

[0008] According to the present invention, it is possible to realize an information processing device and an information processing method that can provide accurate correction values. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of the information processing system according to the first embodiment. [Figure 2] This is a block diagram of the imaging device according to the first embodiment. [Figure 3] This is a block diagram of pixels according to the first embodiment. [Figure 4] This figure shows the relationship between avalanche multiplication and pulse signals according to the first embodiment. [Figure 5] This is a flowchart showing the operation of the information processing device according to the first embodiment. [Figure 6] This is a block diagram of the information processing device according to the second embodiment. [Figure 7] This figure shows the relationship between random numbers and pixel values ​​according to the second embodiment. [Figure 8] This is a flowchart showing the compression method according to the second embodiment. [Figure 9] This is a block diagram of the information processing device according to the third embodiment. [Figure 10] This diagram illustrates the weighting process according to the third embodiment. [Figure 11] It is a block diagram of an information processing apparatus according to the fourth embodiment. [Figure 12] It is a block diagram of an information processing apparatus according to the fifth embodiment. [Figure 13] It is a diagram showing a method of recording cumulative values according to the fifth embodiment. [Figure 14] It is a diagram showing a moving body according to the sixth embodiment. [Figure 15] It is a block diagram of a device according to the seventh embodiment.

Embodiments for Carrying Out the Invention

[0010] [First Embodiment] FIG. 1 is a block diagram of an information processing system according to the present embodiment. The information processing system includes an imaging device 100 and an information processing device 200. The imaging device 100 includes a plurality of pixels, and each of the plurality of pixels includes a light receiving element. The light receiving element can be an APD (Avalanche PhotoDiod) that amplifies the amount of signal charge excited by photons by avalanche multiplication. The imaging device 100 detects the incident light and outputs the count value (pixel value) of the pulse signal to the information processing device 200. The pixel value corresponds to the number of occurrences of avalanche multiplication (hereinafter referred to as the avalanche count) by the light receiving element.

[0011] Here, avalanche multiplication may occur even though no photons are incident on the light receiving element. Therefore, the pixel value may include the count value when avalanche multiplication occurs regardless of the incidence of photons. This count value is called DCR (Dark Count Rate). The information processing device 200 estimates the DCR and corrects the pixel value from the imaging device 100.

[0012] The information processing device 200 includes an input unit 201, an accumulation unit 202, a storage unit 203, a generation unit 204, and a correction unit 205.

[0013] The input unit 201 receives the pixel values ​​of multiple pixels from the imaging device 100. The input unit 201 outputs the input pixel values ​​to the accumulation unit 202 and the correction unit 205.

[0014] When a pixel value is output from the input unit 201, the accumulation unit 202 reads the accumulated value of the avalanche count for each pixel from the storage unit 203. The accumulation unit 202 adds the pixel value from the input unit 201 to the accumulated value and updates the accumulated value in the storage unit 203.

[0015] The memory unit 203 may consist of registers or RAM (Random Access Memory). The memory unit 203 stores the cumulative value of the avalanche count for each pixel.

[0016] The generation unit 204 reads the cumulative value from the storage unit 203 and estimates the DCR based on the cumulative value. As the cumulative value increases, the DCR tends to increase as well. This allows the generation unit 204 to estimate the DCR based on the cumulative value. The generation unit 204 generates a value (hereinafter referred to as the correction value) to correct the pixel value based on the estimated DCR. The correction value corresponds to the DCR in the pixel value. The generation unit 204 outputs the correction value to the correction unit 205.

[0017] The correction unit 205 corrects the pixel values ​​from the imaging device 100 based on the correction values ​​from the generation unit 204. Specifically, the correction unit 205 subtracts the correction value from the pixel value. This allows the correction unit 205 to perform DCR correction on the pixel value. The correction unit 205 outputs the corrected pixel value to the outside of the information processing device 200.

[0018] Figure 2 is a block diagram of the imaging device 100 according to this embodiment. The imaging device 100 includes a pixel area 10, a vertical scanning circuit section 40, a readout circuit section 50, a horizontal scanning circuit section 60, an output circuit section 70, and a control pulse generation section 80.

[0019] The pixel region 10 is provided with multiple pixels 12 arranged in an array such that they form multiple rows and multiple columns. Each pixel 12 includes a photoelectric conversion unit including a light-receiving element and a pixel signal processing unit that processes the signal output from the photoelectric conversion unit, as will be described later. The number of pixels 12 is not particularly limited. For example, the pixel region 10 can be composed of multiple pixels 12 arranged in an array of several thousand rows x several thousand columns. Alternatively, the pixel region 10 may be composed of multiple pixels 12 arranged in one row or one column. Alternatively, the pixel region 10 may be composed of a single pixel 12.

[0020] Each row of the pixel array in the pixel region 10 has a control line 14 extending in a first direction (horizontal direction in Figure 2). The control line 14 is connected to each pixel 12 arranged in the first direction and forms a common signal line for these pixels 12. Each of the control lines 14 may include multiple signal lines for supplying multiple types of control signals to the pixels 12. The control lines 14 in each row are connected to the vertical scanning circuit section 40.

[0021] Each row of the pixel array in the pixel region 10 has data lines 16 extending in a second direction (vertical direction in Figure 2) that intersects the first direction. The data lines 16 are connected to each of the pixels 12 arranged in the second direction and form a common signal line for these pixels 12. Each of the data lines 16 may include multiple signal lines for transferring multi-bit digital signals output from the pixels 12 bit by bit. The data lines 16 of each row are connected to the readout circuit 50.

[0022] The vertical scanning circuit 40 receives a control signal output from the control pulse generation unit 80, generates a control signal to drive the pixels 12, and supplies it to the pixels 12 via the control line 14. Logic circuits such as a shift register and an address decoder may be used in the vertical scanning circuit 40. The vertical scanning circuit 40 sequentially scans the pixels 12 within the pixel area 10 row by row, and outputs the pixel signal of each pixel 12 to the sequential readout circuit 50 via the data line 16.

[0023] The readout circuit 50 includes a plurality of holding units (not shown) corresponding to each column of the pixel array in the pixel region 10. The readout circuit 50 holds the pixel signals of the pixels 12 of each column, which are output row by row from the pixel region 10 via the data line 16, in the holding unit of the corresponding column.

[0024] The horizontal scanning circuit unit 60 receives a control signal output from the control pulse generation unit 80, generates a control signal for reading pixel signals from the holding units of each column of the reading circuit unit 50, and supplies it to the reading circuit unit 50. Logic circuits such as shift registers and address decoders may be used in the horizontal scanning circuit unit 60. The horizontal scanning circuit unit 60 sequentially scans the holding units of each column of the reading circuit unit 50 and sequentially outputs the pixel signals held in each to the output circuit unit 70.

[0025] The output circuit section 70 has an external interface circuit and outputs the pixel signal (pixel value) output from the readout circuit section 50 to the information processing device 200.

[0026] The control pulse generation unit 80 generates control signals to control the operation and timing of the vertical scanning circuit unit 40, the readout circuit unit 50, and the horizontal scanning circuit unit 60, and supplies them to each functional block.

[0027] Figure 3 is a block diagram of the pixel 12 according to this embodiment. The pixel 12 includes a photoelectric conversion unit 20 and a pixel signal processing unit 30.

[0028] The photoelectric conversion unit 20 includes a light-receiving element 22 and a quench element 24. The pixel signal processing unit 30 includes a waveform shaping unit 32, a counter circuit 34, and a selection circuit 36.

[0029] The photodetector 22 may be an APD as described above. The anode of the photodetector 22 is connected to a node to which voltage VL is supplied. The cathode of the photodetector 22 is connected to one terminal of the quench element 24. The connection node between the photodetector 22 and the quench element 24 is the output node of the photoelectric conversion unit 20. The other terminal of the quench element 24 is connected to a node to which a voltage VH higher than voltage VL is supplied. Voltages VL and VH are set to supply applied voltages (reverse bias voltages) that induce avalanche multiplication in the photodetector 22. Here, a negative high voltage is supplied as voltage VL, and a positive voltage of approximately the power supply voltage is supplied as voltage VH.

[0030] By supplying the above-mentioned applied voltage to the photodetector 22, the charge generated by the incidence of light on the photodetector 22 undergoes avalanche multiplication, generating an avalanche current. There are two operating modes when the applied voltage is supplied to the photodetector 22: Geiger mode and linear mode. Geiger mode is an operating mode in which the voltage applied between the anode and cathode is greater than the breakdown voltage of the photodetector 22. Linear mode is an operating mode in which the voltage applied between the anode and cathode is near or below the breakdown voltage of the photodetector 22. A photodetector 22 operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). The photodetector 22 can be configured to operate in linear mode or in Geiger mode.

[0031] The quench element 24 converts the change in avalanche current generated in the photodetector 22 into a voltage signal. Furthermore, the quench element 24 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, reducing the voltage applied to the photodetector 22 and suppressing avalanche multiplication. This operation by the quench element 24 to suppress avalanche multiplication is called the quench operation. The quench element 24 also restores the voltage supplied to the photodetector 22 to voltage VH by supplying current to compensate for the voltage drop caused by the quench operation. This operation by the quench element 24 to restore the voltage supplied to the photodetector 22 to voltage VH is called the recharge operation. The quench element 24 can be composed of a resistor, a MOS transistor, or the like.

[0032] The waveform shaping unit 32 has an input node (node ​​a) and an output node (node ​​b) to which the output signal from the photoelectric conversion unit 20 is input. The waveform shaping unit 32 converts the analog signal output from the photoelectric conversion unit 20 into a pulse signal. The waveform shaping unit 32 is configured to include a NOT circuit (inverter circuit). The output node of the waveform shaping unit 32 is connected to the counter circuit 34.

[0033] The counter circuit 34 includes an input node to which the output signal from the waveform shaping unit 32 is input, an input node connected to the control line 14, and an output node. The counter circuit 34 counts the pulse signal superimposed on the output signal from the waveform shaping unit 32 and holds the count value. The control signals supplied from the vertical scanning circuit unit 40 to the counter circuit 34 via the control line 14 may include an enable signal to control the counting period (exposure period) of the pulse signal, a reset signal to reset the count value held by the counter circuit 34, and so on. The output node of the counter circuit 34 is connected to the data line 16 via a selection circuit 36.

[0034] The selection circuit 36 ​​switches the electrical connection between the counter circuit 34 and the data line 16. The selection circuit 36 ​​switches the connection between the counter circuit 34 and the data line 16 in response to a control signal supplied from the vertical scanning circuit section 40 via the control line 14. The selection circuit 36 ​​may include a buffer circuit (not shown) for outputting a signal.

[0035] Figure 4 shows the relationship between avalanche multiplication and a pulse signal according to this embodiment. At time t0, the photodetector 22 is supplied with an applied voltage corresponding to (voltage VH - voltage VL). An applied voltage that induces avalanche multiplication is supplied between the anode and cathode of the photodetector 22, but when no photons are incident on the photodetector 22, there are no carriers that serve as seeds for avalanche multiplication. Therefore, avalanche multiplication does not occur in the photodetector 22, and no current flows through the photodetector 22.

[0036] Assume that a photon is incident on the light-receiving element 22 at time t1. When a photon is incident on the light-receiving element 22, an electron-hole pair is generated by photoelectric conversion. Avalanche multiplication occurs using these carriers as a seed, and an avalanche current flows through the light-receiving element 22. This avalanche current flows through the quench element 24, causing a voltage drop across the quench element 24, and the voltage at node a of the waveform shaping unit 32 begins to drop. When the voltage drop at node a becomes large and avalanche multiplication stops at time t3, the voltage at node a stops dropping further. The difference between the voltage at node a and the applied voltage at time t3 corresponds to the breakdown voltage Va.

[0037] When the avalanche multiplication in the photodetector 22 stops, a current flows from the node to which voltage VL is supplied through the photodetector 22 to node a to compensate for the voltage drop, and the voltage at node a gradually increases. Subsequently, at time t5, node a settles to its original voltage.

[0038] The waveform shaping unit 32 binarizes the signal input from node a according to a predetermined threshold voltage and outputs it from node b. Specifically, the waveform shaping unit 32 outputs a low-level signal from node b when the voltage at node a exceeds the threshold voltage, and outputs a high-level signal from node b when the voltage at node a is below the threshold voltage. For example, as shown in Figure 4, suppose the voltage at node a is below the threshold voltage during the period from time t2 to time t4. In this case, the signal level at node b will be low during the period from time t0 to time t2 and from time t4 to time t5, and high during the period from time t2 to time t4.

[0039] Thus, the analog signal input from node a is waveform-shaped into a digital signal by the waveform shaping unit 32. A pulse signal is output from the waveform shaping unit 32 in response to the incidence of photons on the light-receiving element 22.

[0040] The exposure period shown in Figure 4 includes the first to fourth determination periods. In each determination period, one pulse signal is generated in response to the incident light. That is, even if multiple photons are incident in each determination period, only one pulse signal is generated.

[0041] During the first and third determination periods, photons are incident and pulse signals are generated, while during the second and fourth determination periods, no photons are incident and no pulse signals are generated. Therefore, the count value of the pulse signal during the exposure period is 2. The count value of the pulse signal can be output to the information processing device 200 for each frame period that includes multiple exposure periods.

[0042] Figure 5 is a flowchart showing the operation of the information processing device 200 according to this embodiment. In step S101, the input unit 201 receives pixel values ​​from the imaging device 100. At this time, the input unit 201 sequentially receives pixel values ​​for each pixel 12. The input unit 201 outputs the pixel values ​​from the imaging device 100 to the accumulation unit 202 and the correction unit 205.

[0043] In step S102, when a pixel value is output from the input unit 201, the accumulation unit 202 reads the accumulated value of the avalanche count from the storage unit 203. Here, the accumulation unit 202 specifies the address of the storage unit 203 based on the identification information of the pixel 12 that output the pixel value, and reads the accumulated value. The accumulation unit 202 adds the pixel value from the imaging device 100 to the accumulated value and updates the accumulated value in the storage unit 203.

[0044] In step S103, the generation unit 204 generates a correction value based on the cumulative value. Experiments have confirmed that DCR does not increase linearly with respect to the cumulative value, but rather increases logarithmically. Therefore, the generation unit 204 can generate the correction value D using the following equation (1).

[0045] D = α * log(1 + β * B) + D0 ... (1) α and β are constants, D0 is the initial correction value, and B is the cumulative value. α, β, and D0 are determined based on experiments. The generation unit 204 outputs the calculated correction value D to the correction unit 205.

[0046] In step S104, the correction unit 205 corrects the pixel values ​​from the imaging device 100 based on the correction value D from the generation unit 204. Specifically, as shown in equation (2) below, the correction unit 205 subtracts the correction value D from the pixel value B to obtain the corrected pixel value B'. B'=BD ···(2)

[0047] In step S105, the correction unit 205 outputs the corrected pixel value B' to the outside of the information processing device 200.

[0048] As described above, the information processing device 200 can provide an accurate correction value for DCR based on the cumulative value of the avalanche count, and can correct the pixel values ​​from the imaging device 100 based on this correction value. As a result, the information processing device 200 can output pixel values ​​with suppressed DCR.

[0049] The information processing method, the program for causing a computer to execute the information processing method, and the recording medium on which the program is stored according to this embodiment also provide the same effects as the information processing device 200.

[0050] Note that the formula for generating the correction value D is not limited to formula (1), and any other formula based on the cumulative number of avalanche attempts may be used.

[0051] Furthermore, considering that DCR is affected by temperature, the generation unit 204 may generate a correction value D based on the temperature information of the photodetector 22 in addition to the cumulative value. For example, the correction value D can be generated by adding a term or coefficient related to the temperature information of the photodetector 22 to equation (1).

[0052] Furthermore, the input unit 201 may adjust the timing of outputting the pixel values. Specifically, the input unit 201 may delay the timing of outputting the pixel values ​​to the correction unit 205 compared to the timing of outputting the pixel values ​​to the accumulation unit 202. This allows the correction unit 205 to synchronize the timing of inputting the pixel values ​​with the timing of inputting the correction values, enabling smoother correction processing.

[0053] [Second Embodiment] Figure 6 is a block diagram of the information processing device 300 according to this embodiment. The information processing device 300 differs from the information processing device 200 according to the first embodiment in that it compresses the cumulative value using random numbers. In this embodiment, the same reference numerals are used for the same components as in the information processing device 200 according to the first embodiment, and detailed descriptions are omitted.

[0054] The information processing device 300 includes an input unit 201, a storage unit 203, a generation unit 204, a correction unit 205, a random number unit 301, a compression unit 302, and an accumulation unit 303.

[0055] The random number unit 301 generates random numbers using a predetermined random number generation algorithm. Here, the lower limit of the random numbers is set to 0, and the upper limit of the random numbers is set based on the maximum pixel value. For example, the upper limit of the random numbers is set to 1000 times the maximum pixel value. Here, when the maximum pixel value is 2047, the upper limit of the random numbers is 2047000, and the cumulative value is compressed to 1 / 2047000. Note that the upper limit of the random numbers is not limited to this value and can be set to an upper limit according to the desired compression ratio.

[0056] When the compression unit 302 receives a pixel value output from the input unit 201, it compresses the pixel value based on a random number from the random number unit 301 and outputs the compressed pixel value (hereinafter referred to as the compressed value) to the accumulation unit 303.

[0057] When the compression unit 302 outputs a compressed value, the accumulation unit 303 reads the accumulated value from the storage unit 203. The accumulation unit 303 adds the compressed value from the compression unit 302 to the accumulated value and updates the accumulated value in the storage unit 203.

[0058] Figure 7 shows the relationship between random numbers and pixel values ​​according to this embodiment. In Figure 7, the relationship between random numbers and pixel values ​​is shown for each frame output from the imaging device 100. The pixel value in Figure 7 represents one of several pixel values ​​included in the frame.

[0059] The compression unit 302 compares the pixel value "1045" with the random number "520" in the first frame. Since the pixel value is greater than the random number, the compression unit 302 converts the pixel value to "1" (the first value). Note that the compressed value does not necessarily have to be "1", but it is desirable that it be a value smaller than the pixel value.

[0060] In the second frame, the pixel value "733" is less than or equal to the random number "4518", so the compression unit 302 converts the pixel value to "0" (the second value). Similarly, the pixel values ​​of the third to fifth frames are converted to "0", "0", and "1".

[0061] Here, if the pixel values ​​of the first to fifth frames are not compressed, the cumulative value will be "5371" (=1045+733+1128+450+2015). On the other hand, according to this embodiment, the cumulative value is compressed from "5371" to "2" (=1+0+0+0+1), so the storage capacity of the storage unit 203 can be reduced.

[0062] Figure 8 is a flowchart showing the compression method according to this embodiment. In step S201, the compression unit 302 receives pixel values ​​from the imaging device 100.

[0063] In step S202, the random number unit 301 generates a random number. In step S203, the compression unit 302 compares the pixel value with the random number. If the pixel value is greater than the random number (step S203; YES), in step S204, the compression unit 302 compresses the pixel value by converting the pixel value to 1.

[0064] If the pixel value is less than or equal to a random number (step S203; NO), in step S205, the compression unit 302 compresses the pixel value by converting the pixel value to 0.

[0065] In step S206, the accumulation unit 303 adds the compressed value from the compression unit 302 to the accumulated value and updates the accumulated value in the storage unit 203.

[0066] As described above, the information processing device 300 can compress the cumulative value, thereby reducing the storage capacity of the storage unit 203.

[0067] When the generation unit 204 calculates the correction value D using the above formula (1), α and β should be adjusted according to the compression ratio.

[0068] Furthermore, as another compression method, the accumulation unit 303 may accumulate pixel values ​​from the imaging device 100 for multiple frames. For example, if pixel values ​​are accumulated every 1000 frames, the accumulated value is compressed to 1 / 1000. Alternatively, a compression method using random numbers and a compression method accumulating pixel values ​​for multiple frames may be combined. That is, compression of the bit length of the pixel value and compression of the time axis (accumulation frequency) may be combined. This makes it possible to compress the accumulated value at a higher compression ratio and further reduce the storage capacity of the storage unit 203.

[0069] [Third Embodiment] Figure 9 is a block diagram of the information processing device 400 according to this embodiment. The information processing device 400 differs from the information processing device 300 according to the second embodiment in that it estimates unweighted pixel values ​​from weighted pixel values. In this embodiment, the same reference numerals are used for the same components as in the information processing device 300 according to the second embodiment, and detailed explanations are omitted.

[0070] Figure 10 is a diagram illustrating the weighting process according to this embodiment. The exposure period includes multiple determination periods. Each of the multiple determination periods is divided into a first period, a second period, and a third period. The first to third periods are examples of divided periods. In each of the first to third periods, the value used to count up the pulse signal is different; that is, weighting is set. Higher weighting is set for earlier timings among the first to third periods. Here, the timings are in the order of first period, second period, and third period. Therefore, the weighting of the first period is the highest, the weighting of the second period is the next highest, and the weighting of the third period is the lowest. By weighting, the effect of so-called afterpulses is suppressed, and thus the decrease in the signal-to-noise ratio can be suppressed.

[0071] When the counter circuit 34 of pixel 12 detects a pulse signal in the first period, it counts up the number of avalanches by three. When the counter circuit 34 detects a pulse signal in the second period, it counts up the number of avalanches by two. When the counter circuit 34 detects a pulse signal in the third period, it counts up the number of avalanches by one. Thus, the weighting value for the first period is "3", the weighting value for the second period is "2", and the weighting value for the third period is "1". When the counter circuit 34 does not detect a pulse signal in the first to third periods, it does not count up the number of avalanches. The counter circuit 34 sums the count values in the plurality of determination periods to obtain the count value for the exposure period. The imaging device 100 outputs the count value (weighted pixel value) to the information processing device 400 for each frame period including a plurality of exposure periods. Note that the number of divisions and the weighting value in the determination period are merely examples and are not limited thereto.

[0072] Let the number of avalanches (pixel value) generated by the light receiving element 22 be B a and the weighted pixel value be O p . Here, the pixel value is not weighted and is the value obtained by counting up the number of avalanches by one each time a pulse signal is detected. The relationship between the pixel value and the weighted pixel value is expressed by the following formula (3). O p = f(B a ) ···(3)

[0073] The inverse transformation of formula (3) is expressed by the following formula (4). The pixel value B a can be obtained from formula (4). B a = f -1 (O p )···(4)

[0074] Note that when the inverse function f -1 cannot be obtained, the pixel value can be obtained from the weighted pixel value based on an approximate calculation or an experimental correspondence relationship.

[0075] Based on the relationship in equation (4), we will explain a specific method for estimating pixel values ​​from weighted pixel values ​​using equations (5) to (7).

[0076] p = P / N / M ... (5) In equation (5), N is the number of judgment periods in the exposure period, P is the number of photons in the exposure period, and M is the maximum weighting value. Here, N is 7 and M is 3. The probability p of avalanche multiplication occurring in the first to third periods can be calculated using equation (5) above.

number

number

[0077] As shown in Figure 10, the information processing device 400 includes an input unit 201, a storage unit 203, a generation unit 204, a correction unit 205, a random number unit 301, a compression unit 302, an accumulation unit 303, and an estimation unit 401.

[0078] The input unit 201 receives weighted pixel values ​​from the imaging device 100. The input unit 201 outputs the input weighted pixel values ​​to the estimation unit 401 and the correction unit 205.

[0079] The estimation unit 401 uses equation (7) above to obtain an estimated value A from the weighted pixel values ​​and outputs it to the compression unit 302.

[0080] The compression unit 302 compresses the estimated value A based on a random number from the random number unit 301 and outputs the compressed estimated value A (hereinafter referred to as the compressed value) to the accumulation unit 303. The accumulation unit 303 adds the compressed value to the accumulation value and updates the accumulation value in the storage unit 203.

[0081] The generation unit 204 estimates the DCR based on the cumulative value read from the storage unit 203. Based on the estimated DCR, the generation unit 204 generates correction values ​​for weighted pixel values ​​and outputs them to the correction unit 205.

[0082] The correction unit 205 corrects the weighted pixel values ​​from the imaging device 100 based on the correction values ​​from the generation unit 204. Specifically, the correction unit 205 subtracts the correction value from the weighted pixel values.

[0083] As described above, the information processing device 400 includes an estimation unit 401 that estimates pixel values ​​from weighted pixel values ​​weighted according to the timing of detecting pulse signals based on avalanche multiplication by the light-receiving element 22 within the exposure period. Then, by determining a correction value based on the cumulative value of the estimated pixel values, the weighted pixel values ​​from the imaging device 100 can be corrected based on the correction value. As a result, the information processing device 400 can output weighted pixel values ​​with suppressed DCR.

[0084] It is not necessary to compress the estimated value. In this case, the estimation unit 401 outputs the estimated value to the accumulation unit 303, and the accumulation unit 303 adds the estimated value to the accumulated value to update the accumulated value in the storage unit 203.

[0085] [Fourth Embodiment] Figure 11 is a block diagram of the information processing device 500 according to this embodiment. The information processing device 500 differs from the information processing device 200 according to the first embodiment in that it corrects the applied voltage at the light receiving element 22. In this embodiment, the same reference numerals are used for the same components as in the information processing device 200 according to the first embodiment, and detailed descriptions are omitted.

[0086] The information processing device 500 comprises an input unit 201, an accumulation unit 202, a storage unit 203, a generation unit 501, and a control unit (correction unit) 502.

[0087] The generation unit 501 estimates the breakdown voltage of the photodetector 22 based on the cumulative value read from the storage unit 203. As the cumulative value of avalanche counts increases, the breakdown voltage of the photodetector 22 tends to increase as well. The generation unit 501 estimates the breakdown voltage based on the cumulative value and generates a correction value for the applied voltage of the photodetector 22 based on the estimated breakdown voltage. Experiments have confirmed that the breakdown voltage increases logarithmically. That is, as the cumulative value increases, the rate of increase in the breakdown voltage tends to decrease. The generation unit 204 generates a correction value ΔV based on this trend using the following equation (8). ΔV=γ*log(1+δ*B)+D0···(8)

[0088] γ and δ are constants, D0 is the initial correction value, and B is the cumulative value. γ, δ, and D0 are determined based on experiments. The generation unit 501 outputs the correction value ΔV to the control unit 502.

[0089] The control unit 502 corrects the applied voltage at the photodetector 22 based on the correction value ΔV from the generation unit 501. Specifically, as shown in equation (9) below, the control unit 502 adds the correction value ΔV to the applied voltage V before correction to obtain the corrected applied voltage V'. V' = V + ΔV ... (9)

[0090] The control unit 502 outputs voltage information indicating the applied voltage V' to the imaging device 100. Based on the voltage information from the control unit 502, the imaging device 100 controls the applied voltage supplied to the photodetector 22.

[0091] As described above, the information processing device 500 can provide a correction value for the applied voltage based on the cumulative value of the avalanche count, and can correct the applied voltage of the photodetector 22 based on this correction value. By correcting the applied voltage according to the cumulative value of the avalanche count, the information processing device 500 can reduce the change in pixel value over time.

[0092] [Fifth Embodiment] Figure 12 is a block diagram of the information processing device 600 according to this embodiment. The information processing device 600 differs from the information processing device 200 according to the first embodiment in that it inputs image data containing multiple pixel values, rather than sequentially inputting pixel values ​​for each pixel 12. In this embodiment, the same reference numerals are used for the same components as in the information processing device 200 according to the first embodiment, and detailed descriptions are omitted.

[0093] The imaging device 100 outputs image data containing multiple pixel values ​​to the information processing device 600. The multiple pixel values ​​in the image data are stored in association with XY coordinates, and the pixel value can be identified by specifying the XY coordinates.

[0094] The information processing device 600 includes an input unit 201, a generation unit 204, a correction unit 205, an accumulation unit 601, and a storage unit 602.

[0095] Figure 13 shows a method for storing cumulative values ​​according to this embodiment. The storage unit 602 stores a plurality of cumulative values ​​in relation to the pixel array of the pixel region 10. When the first direction and second direction of the pixel region 10 are assigned to coordinates (x, y), the storage unit 602 stores, for example, the cumulative value "3254" in relation to coordinate (0, 0) and the cumulative value "1825" in relation to coordinate (1, 0).

[0096] The input unit 201 receives image data from the imaging device 100. The input unit 201 outputs the input image data to the accumulation unit 601 and the correction unit 205.

[0097] When image data is output from the input unit 201, the accumulation unit 601 reads the accumulated values ​​from the storage unit 602 in a predetermined order. Specifically, as shown by the arrows in Figure 13, the accumulation unit 601 reads the accumulated values ​​in the order of coordinates (0,0), (1,0), ... (6,0), (0,1), (1,1), ... (6,1). This allows the accumulation unit 601 to read the accumulated value corresponding to each of the multiple pixel values ​​in the image data. The accumulation unit 601 updates the accumulated value in the storage unit 602 by adding the pixel value to the read accumulated value. The accumulation unit 202 performs the same process for all pixel values ​​included in the image data.

[0098] The generation unit 204 reads cumulative values ​​from the storage unit 602 in a predetermined order. Specifically, as shown by the arrows in Figure 13, the generation unit 204 reads cumulative values ​​in the order of coordinates (0,0), (1,0), ... (6,0), (0,1), (1,1), ... This allows the generation unit 204 to read cumulative values ​​corresponding to each of the multiple pixel values ​​in the image data. The generation unit 204 estimates the DCR based on the read cumulative values. The generation unit 204 generates correction values ​​based on the estimated DCR. In this way, the generation unit 204 generates correction values ​​corresponding to all pixel values ​​in the image data. The generation unit 204 outputs the generated multiple correction values ​​to the correction unit 205.

[0099] When image data is output from the imaging device 100, the correction unit 205 reads out correction values ​​in a predetermined order. This allows the correction unit 205 to read out correction values ​​corresponding to each of the multiple pixel values ​​in the image data. The correction unit 205 corrects the pixel values ​​based on the read-out correction values. The correction unit 205 corrects all pixel values ​​included in the image data.

[0100] As described above, the information processing device 600 can provide image data with suppressed DCR.

[0101] Furthermore, the components of the information processing devices 200 to 600 may be combined as appropriate. For example, the random number unit 301 and the compression unit 302 may be combined with the information processing devices 500 and 600 to compress the cumulative value. Alternatively, the estimation unit 401 may be combined with the information processing devices 500 and 600 to process weighted pixel values.

[0102] [Sixth Embodiment] Figure 14 shows a mobile body according to this embodiment. Figure 14(a) shows an example of the configuration of a device 700 mounted on a vehicle as an in-vehicle camera. The device 700 includes a distance measuring unit 703 that measures the distance to an object, and a collision determination unit 704 that determines whether or not there is a possibility of collision based on the distance measured by the distance measuring unit 703. The distance measuring unit 703 includes a light source device having a light-emitting element that emits light, and a light-receiving device having a light-receiving element that receives light emitted from the light source device and reflected by the object to be measured. The distance measuring unit 703 further includes an information processing device of any of the first to fifth embodiments that processes signals from the light-receiving device, and a distance information acquisition unit. The distance information acquisition unit acquires information regarding the distance to an object based on the time difference between the timing at which light is emitted from the light-emitting element and the timing at which the light-receiving element receives the light emitted from the light-emitting element and reflected by the object.

[0103] Device 700 is connected to a vehicle information acquisition device 710 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Device 700 is also connected to a control ECU 720, which is a control device that outputs a control signal to generate braking force on the vehicle based on the collision determination result of the collision determination unit 704. Furthermore, device 700 is also connected to a warning device 730 that issues a warning to the driver based on the collision determination result of the collision determination unit 704. For example, if the collision determination result of the collision determination unit 704 indicates a high probability of collision, the control ECU 720 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 730 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seatbelt or steering wheel. These devices of device 700 function as a mobile control unit that controls the vehicle's operations as described above.

[0104] In this embodiment, the device 700 measures the distance around the vehicle, for example, in front of or behind it. Figure 14(b) shows the device when measuring the distance in front of the vehicle (distance measurement range 750). The vehicle information acquisition device 710, acting as a distance measurement control means, sends an instruction to the device 700 or the distance measurement unit 703 to perform the distance measurement operation. This configuration allows for further improvement of the accuracy of distance measurement.

[0105] The above example described controlling a vehicle to avoid collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles, or control systems that automatically stay within their lane. Furthermore, the equipment is not limited to vehicles such as automobiles, but can be applied to mobile objects (mobile devices) such as ships, aircraft, satellites, industrial robots, and consumer robots. In addition, it can be applied not only to mobile objects, but also to a wide range of devices that utilize object recognition or biometric recognition, such as intelligent transportation systems (ITS) and surveillance systems.

[0106] [Seventh Embodiment] Figure 15 is a block diagram of the EQP device according to this embodiment. The EQP device includes a light source device having a light-emitting element that emits light, and a photoelectric converter APR that has a light-receiving element that receives light emitted from the light-emitting element of the light source device and reflected by an object, and converts the optical signal into an electrical signal. All or part of the photoelectric converter APR is a semiconductor device IC. The photoelectric converter APR can be used, for example, as an image sensor, an AF (Auto Focus) sensor, a photometering sensor, or a distance measuring sensor. The semiconductor device IC has a pixel area PX in which pixel circuits PXC including a photoelectric conversion unit are arranged in a matrix. The semiconductor device IC may have a peripheral area PR around the pixel area PX. Circuits other than pixel circuits can be arranged in the peripheral area PR.

[0107] The photoelectric converter APR may have a stacked structure (chip stacking structure) comprising a first semiconductor chip equipped with multiple photoelectric conversion units and a second semiconductor chip equipped with peripheral circuits. The peripheral circuits on the second semiconductor chip can each be a column circuit corresponding to a pixel row of the first semiconductor chip. Alternatively, the peripheral circuits on the second semiconductor chip can each be a matrix circuit corresponding to a pixel or pixel block of the first semiconductor chip. Connections between the first and second semiconductor chips can be made using through-silicon vias (TSVs), direct bonding of conductors such as copper for inter-chip wiring, microbump connections between chips, or wire bonding.

[0108] The photoelectric converter APR may include a semiconductor device IC as well as a package PKG that houses the semiconductor device IC. The package PKG may include a substrate on which the semiconductor device IC is fixed, a lid made of glass or the like that faces the semiconductor device IC, and connecting members such as bonding wires or bumps that connect terminals provided on the substrate to terminals provided on the semiconductor device IC.

[0109] The device EQP may further comprise at least one of the following: an optical device OPT, a control device CTRL, a processing device PRCS, a display device DSPL, a memory device MMRY, and a mechanical device MCHN. The optical device OPT corresponds to the photoelectric converter APR and is, for example, a lens, shutter, or mirror. The control device CTRL controls the photoelectric converter APR and is, for example, a semiconductor device such as an ASIC. The processing device PRCS processes the signals output from the photoelectric converter APR and is a semiconductor device such as a CPU (central processing unit) or an ASIC (application-specific integrated circuit). The processing device PRCS includes any of the information processing devices of the first to fifth embodiments described above. The display device DSPL is an EL display device or a liquid crystal display device that displays information (images) obtained from the photoelectric converter APR. The memory device MMRY is a magnetic device or a semiconductor device that stores information (images) obtained from the photoelectric converter APR. The memory device MMRY is a volatile memory such as SRAM or DRAM, or a non-volatile memory such as flash memory or a hard disk drive. The mechanical device MCHN has movable parts or propulsion parts such as motors and engines. The device EQP displays signals output from the photoelectric converter APR on the display device DSPL, or transmits them to the outside using a communication device (not shown) provided by the device EQP. For this purpose, it is preferable that the device EQP further includes a memory device MMRY and a processing device PRCS, separate from the memory circuit and arithmetic circuit of the photoelectric converter APR.

[0110] The EQP (Equipment Equipped Device) shown in Figure 15 can be an electronic device such as an information terminal with imaging capabilities (e.g., a smartphone or wearable device) or a camera (e.g., an interchangeable lens camera, a compact camera, a video camera, or a surveillance camera). In a camera, the mechanical device MCHN can drive components of the optical device OPT for zooming, focusing, and shutter operation. The EQP can also be a transport device (mobile object) such as a vehicle or a ship. Furthermore, the EQP can be a medical device such as an endoscope or a CT scanner.

[0111] The mechanical device MCHN in transport equipment can be used as a mobile device. The device EQP as transport equipment is suitable for transporting the photoelectric converter APR, or for assisting and / or automating driving (operation) through its imaging function. The processing device PRCS for assisting and / or automating driving (operation) can perform processing to operate the mechanical device MCHN as a mobile device based on information obtained from the photoelectric converter APR.

[0112] The photoelectric converter APR according to this embodiment can provide high value to its designers, manufacturers, distributors, buyers, and / or users. Therefore, by installing the photoelectric converter APR in the EQP (Equipment Equipment), the value of the EQP can also be increased. Thus, when manufacturing and selling the EQP, deciding to install the photoelectric converter APR of this embodiment in the EQP is advantageous in increasing the value of the EQP.

[0113] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways. For example, an example in which a part of the configuration of one embodiment is added to another embodiment, or in which a part of the configuration of another embodiment is replaced, is also an embodiment of the present invention.

[0114] The above explanation describes an example of performing DCR correction and applied voltage correction based on the cumulative value of avalanche counts, but it is not limited to this. For example, pixel scratch correction and determination of the usage period of the photodetector may also be performed.

[0115] The above-disclosed embodiments include the following configurations and methods. (Composition 1) An input section that receives a signal based on the number of avalanche cycles in the photodetector, A cumulative unit that accumulates a value corresponding to the number of avalanches based on the signal, An information processing device characterized by comprising a generation unit that generates a correction value for the signal based on the cumulative value in the accumulation unit. (Configuration 2) The information processing apparatus according to configuration 1, characterized by comprising a correction unit that corrects the signal based on the correction value. (Composition 3) The correction value corresponds to the DCR (Dark Count Rate) of the signal. The information processing apparatus according to configuration 2, characterized in that the correction unit subtracts the correction value from the signal. (Composition 4) When B is the number of avalanche steps, α and β are constants, and D0 is the initial correction value, The aforementioned correction value (D) is, D = α * log(1 + β * B) + D0 An information processing device according to any one of configurations 1 to 3, characterized by being determined by the method described above. (Composition 5) The system further includes a compression unit that compresses the number of avalanche cycles, The information processing apparatus according to any one of configurations 1 to 4, characterized in that the accumulation unit accumulates the compressed value of the number of avalanche operations. (Composition 6) The information processing apparatus according to configuration 5, characterized in that the compression unit compresses the number of avalanche operations based on a random number. (Composition 7) The information processing apparatus according to configuration 6, characterized in that the compression unit converts the number of avalanche steps to a first value smaller than the number of avalanche steps if the number of avalanche steps is greater than the number of random numbers, and converts the number of avalanche steps to a second value smaller than the first value if the number of avalanche steps is less than or equal to the number of random numbers. (Composition 8) The information processing device according to configuration 7, characterized in that the first value is "1" and the second value is "0". (Composition 9) The information processing device according to any one of configurations 1 to 8, characterized in that the accumulation unit accumulates the number of avalanche operations for each of the multiple frames. (Composition 10) The system further comprises an estimation unit that estimates the number of avalanche cycles from a signal weighted according to the timing of detecting a pulse signal based on avalanche multiplication by the light-receiving element within the exposure period, The information processing apparatus according to any one of configurations 1 to 9, characterized in that the accumulation unit accumulates an estimated value corresponding to the estimated number of avalanches. (Composition 11) The exposure period includes a plurality of determination periods, and each of the plurality of determination periods includes a plurality of division periods. When m is the number of the multiple division periods, th[i] is the weighting value, i is an integer between 1 and m, p is the probability of avalanche multiplication occurring in the multiple division periods, d[i] is the probability of avalanche multiplication occurring considering the weighting value in each of the multiple division periods, and N is the number of the multiple determination periods, The aforementioned estimate (A) is,

number

number

[0116] 200, 300, 400, 500, 600... Information Processing Devices 201...Input section 202, 303, 601... Cumulative portion 204, 501...Generation section 205...Correction section 502...Control Unit (Correction Unit) 302... Compression section 401...Estimation section

Claims

1. An input section that receives a signal based on the number of avalanche cycles in the photodetector, A cumulative unit that accumulates a value corresponding to the number of avalanches based on the signal, An information processing device characterized by comprising a generation unit that generates a correction value for the signal based on the cumulative value in the accumulation unit.

2. The information processing apparatus according to claim 1, further comprising a correction unit that corrects the signal based on the correction value.

3. The correction value corresponds to the DCR (Dark Count Rate) of the signal. The information processing apparatus according to claim 2, characterized in that the correction unit subtracts the correction value from the signal.

4. B is the number of avalanche steps, α and β are constants, D ０ When this is the initial correction value, The aforementioned correction value (D) is, D = α * log (1 + β * B) + D ０ The information processing apparatus according to claim 3, characterized by being determined by

5. The system further includes a compression unit that compresses the number of avalanche cycles, The information processing apparatus according to claim 1, characterized in that the accumulation unit accumulates the compressed value of the number of avalanche operations.

6. The information processing apparatus according to claim 5, characterized in that the compression unit compresses the number of avalanche operations based on a random number.

7. The information processing apparatus according to claim 6, characterized in that the compression unit converts the number of avalanche steps to a first value smaller than the number of avalanche steps if the number of avalanche steps is greater than the number of random numbers, and converts the number of avalanche steps to a second value smaller than the first value if the number of avalanche steps is less than or equal to the number of random numbers.

8. The information processing apparatus according to claim 7, characterized in that the first value is "1" and the second value is "0".

9. The information processing apparatus according to claim 1, characterized in that the accumulation unit accumulates the number of avalanche operations for each of the multiple frames.

10. The system further comprises an estimation unit that estimates the number of avalanche cycles from a signal weighted according to the timing of detecting a pulse signal based on avalanche multiplication by the light-receiving element within the exposure period, The information processing apparatus according to claim 1, characterized in that the accumulation unit accumulates an estimated value corresponding to the estimated number of avalanches.

11. The exposure period includes a plurality of determination periods, and each of the plurality of determination periods includes a plurality of division periods. When m is the number of the multiple division periods, th[i] is the weighting value, i is an integer between 1 and m, p is the probability of avalanche multiplication occurring in the multiple division periods, d[i] is the probability of avalanche multiplication occurring considering the weighting value in each of the multiple division periods, and N is the number of the multiple determination periods, The aforementioned estimate (A) is, [Math 1] [Math 2] The information processing apparatus according to claim 10, characterized by being determined by

12. The information processing apparatus according to claim 2, characterized in that the correction unit corrects the applied voltage to the light receiving element based on the correction value.

13. B is the number of avalanche steps, γ and δ are constants, D ０ When this is the initial correction value, The aforementioned correction value (ΔV) is, ΔV=γ*λog(1+δ*B)+D ０ The information processing apparatus according to claim 12, characterized by being determined by

14. Multiple light-receiving elements are arranged in an array. The information processing apparatus according to claim 1, characterized in that the accumulation unit holds the accumulated value for each of the plurality of light-receiving elements.

15. The information processing apparatus according to claim 1, characterized in that the generation unit generates the correction value based on the temperature information of the light receiving element and the cumulative value.

16. A light source device that emits light, A light receiving device having a light receiving element according to claim 1 that receives light emitted from the light source device and reflected by the object to be measured, An information processing device according to claim 1, which processes a signal from the light receiving device, A distance measuring device comprising: a distance information acquisition unit that acquires information regarding the distance to the object to be measured based on the time difference between the timing at which light is emitted from the light source device and the timing at which the light receiving device receives the light.

17. A photoelectric converter having a light-receiving element as described in claim 1 and converting an optical signal into an electrical signal, An information processing device according to claim 1, which processes a signal output from the photoelectric converter, Optical device corresponding to the aforementioned photoelectric converter, A control device for controlling the aforementioned photoelectric converter, A mechanical device controlled based on information obtained from the aforementioned photoelectric converter, A display device for displaying information obtained by the aforementioned photoelectric converter, and A device characterized by comprising at least one of the following: a storage device for storing information obtained by the photoelectric converter.

18. A step of inputting a signal based on the number of avalanches in the photodetector, A step of accumulating a value corresponding to the number of avalanches based on the signal, An information processing method characterized by comprising the step of generating a correction value for the signal based on a cumulative value.

19. A program for causing a computer to execute the information processing method described in claim 18.

20. A recording medium storing a program for causing a computer to execute the information processing method described in claim 18.

Images