Photovoltaic conversion device, control method, and computer program

Abstract

To provide a photoelectric conversion device capable of suppressing image quality deterioration due to miscounts caused by avalanche radiation.SOLUTION: A photoelectric conversion device of the present invention: comprises a pixel region in which pixels formed from avalanche photo-diodes, which photo-electric converts an optical image, are arranged two-dimensionally; a photoelectric conversion device including a read circuit simultaneously reading signals from pixels included in a first pixel group and signals from pixels included in a second pixel group; generation means of generating a first image based on signals from the photoelectric conversion device; and correction means of correcting the first image by using first characteristic information relating to cross-talk between the pixels of the photoelectric conversion device. In the correction means, correction information based on the first characteristic information differs between the first pixel group and the second pixel group.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device. [Background technology]

[0002] 2. Description of the Related Art In recent years, a photoelectric conversion device has been proposed that digitally counts the number of photons arriving at an avalanche photodiode (APD) and outputs the counted value from a pixel as a photoelectrically converted digital signal.

[0003] It is known that a phenomenon called avalanche emission occurs in photoelectric conversion devices having APDs (Non-Patent Document 1). When avalanche emission occurs, the generated secondary electrons enter adjacent pixels, increasing the count number of the adjacent pixel values ​​and causing erroneous counting. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] I. Rech et al., “Optical crosstalk in single photon avalanche diode arrays: a new complete model”, OpEx 16(12), 2008 Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide a photoelectric conversion device that suppresses deterioration of image quality due to erroneous counting caused by avalanche light emission. [Means for solving the problem]

[0006] The photoelectric conversion device of the present invention, which solves the above-mentioned problems, comprises a pixel region in which pixels composed of avalanche photodiodes for photoelectrically converting an optical image are arranged two-dimensionally, a photoelectric conversion element having a readout circuit that simultaneously reads out signals from pixels included in a first pixel group and signals from pixels included in a second pixel group, a generation means for generating a first image based on the signals obtained from the photoelectric conversion element, and a correction means for performing a correction process on the first image using first characteristic information related to crosstalk between pixels of the photoelectric conversion element, wherein the correction information based on the first characteristic information differs between the first pixel group and the second pixel group. Effect of the Invention

[0007] According to the present invention, it is possible to provide a photoelectric conversion device that suppresses deterioration of image quality caused by erroneous counting due to avalanche light emission. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a configuration example of a photoelectric conversion element; [Diagram 2] FIG. 1 shows an example of the configuration of a sensor substrate 11. [Diagram 3] FIG. 2 shows an example of the configuration of a circuit board 21. [Figure 4] Timing chart of the vertical scanning circuit 110 [Diagram 5] FIG. 1 is a diagram showing an example of an equivalent circuit of a pixel 101 and a signal processing circuit 104 corresponding to the pixel 101. [Figure 6] A schematic diagram showing the relationship between the APD201 operation and the output signal [Figure 7] FIG. 1 is a block diagram showing an example of the functional configuration of a photoelectric conversion device 300 according to a first embodiment. [Figure 8] FIG. 1 is a diagram showing a probability distribution of occurrence of miscounting between adjacent pixels of a photoelectric conversion element 100 as first array data. [Figure 9] A diagram explaining light emission crosstalk when multiple lines are read simultaneously. [Figure 10]A diagram explaining image quality degradation caused by miscounting due to light emission crosstalk [Figure 11] 1 is a flowchart showing signal processing performed by the photoelectric conversion device according to the first embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of correction processing. [Figure 13] Timing chart when the exposure time is changed [Figure 14] FIG. 1 is a diagram showing an equivalent circuit corresponding to a pixel of a photoelectric conversion element according to a second embodiment. [Figure 15] Photoelectric conversion element timing chart [Figure 16] 1 is a flowchart showing signal processing performed by a photoelectric conversion device according to a second embodiment of the present invention; [Figure 17] FIG. 1 is a diagram illustrating an example of correction processing. [Figure 18] A diagram showing an example of the functional configuration of a photoelectric conversion system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following describes in detail the embodiments for carrying out the present invention. Note that the embodiments described below are examples for realizing the present invention, and should be appropriately modified or adjusted depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments. Also, in all drawings, parts having the same functions are given the same numbers, and repeated explanations are omitted.

[0010] <Embodiment 1> 1 is a diagram showing an example of the configuration of a photoelectric conversion element. In the following, a photoelectric conversion device having a so-called laminated structure in which a photoelectric conversion element 100 is configured by laminating and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21, will be described as an example. However, a so-called non-laminated structure in which the configuration included in the sensor substrate and the configuration included in the circuit substrate are arranged on a common semiconductor layer may also be used. The sensor substrate 11 includes a pixel region 12. The circuit substrate 21 includes a circuit region 22 that processes a signal detected in the pixel region 12.

[0011] <Sensor board> 2 is a diagram showing a configuration example of the sensor substrate 11. The pixel region 12 of the sensor substrate 11 includes a plurality of pixels 101 arranged two-dimensionally across a plurality of rows and columns. Each pixel 101 includes a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter, APD). Note that the number of rows and columns of the pixel array constituting the pixel region 12 is not particularly limited.

[0012] <Circuit board> Fig. 3 is a diagram showing an example of the configuration of the circuit board 21. The circuit board 21 has a signal processing circuit 103 that processes charges photoelectrically converted by the photoelectric conversion unit 102 in Fig. 2, a readout circuit 112, a control pulse generation unit 115, a horizontal scanning circuit 111, a signal line 113, and a vertical scanning circuit 110.

[0013] The signal output from the photoelectric conversion unit 102 of the pixel is processed by the signal processing circuit 103. The signal processing circuit 103 is provided with a counter, a memory, etc., and the memory holds digital values.

[0014] The horizontal scanning circuit 111 inputs a control pulse for sequentially selecting each column to the signal processing circuit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0015] A signal is output to the signal line 113 from the signal processing circuit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column. The signal output to the signal line 113 is output to the outside of the photoelectric conversion device 100 via the output circuit 114.

[0016] <Connection between sensor board and circuit board> 2 and 3, a plurality of signal processing circuits 103 are arranged in a region overlapping the pixel region 12 in a planar view. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. A plurality of signal processing circuits 103 are arranged in a region overlapping the non-pixel region in a planar view. A vertical scanning circuit section 110, a horizontal scanning circuit section 111, a readout circuit 112, an output circuit 114, and a control pulse generating section 115 are arranged.

[0017] <Vertical scanning circuit> The vertical scanning circuit 110 receives a control pulse supplied from the control pulse generating unit 115 and supplies a control pulse to each pixel. The vertical scanning circuit 110 is composed of a shift register and an address decoder in which multiple rows are connected as one unit, and realizes high-speed readout by reading multiple rows at once. In particular, in the case of a photoelectric conversion device that digitally counts the number of photons arriving at the APD and outputs the counted value from the pixel as a photoelectrically converted digital signal, it takes time for the counter circuit that digitally counts the number of photons to operate. Therefore, it is preferable to read multiple rows simultaneously for high-speed readout. That is, the vertical scanning circuit 110, which functions as a readout circuit that reads pixel signals from pixels, simultaneously reads pixel signals from the pixels included in the first row and pixel signals from the pixels included in the second row.

[0018] FIG. 4 is a diagram showing a timing chart of the vertical scanning circuit 110. In the following, as shown in FIG. 4, when rows to be read out simultaneously are grouped into one group, the rows are distinguished according to the position of the row within the group and expressed as rows A to F. Note that, although FIG. 4 shows a case where the number of rows to be read out simultaneously is six, this is not limited to six. Also, FIG. 4 shows a case where only six rows are simultaneously subjected to an exposure operation, but if the exposure time is long, the number of rows to be simultaneously subjected to an exposure operation will be an integer multiple of six. Differences due to exposure time will be described later.

[0019] The arrangement of the signal lines 113, the readout circuits 112, and the output circuits 114 is not limited to that shown in FIG. 3. For example, the signal lines 113 may be arranged extending in the row direction, and the readout circuits 112 may be arranged at the ends of the signal lines 113. The function of a signal processing unit does not necessarily need to be provided for each photoelectric conversion unit, and one signal processing unit may be shared by a plurality of photoelectric conversion units to perform signal processing sequentially.

[0020] <Pixel equivalent circuit> FIG. 5 is a diagram showing an equivalent circuit of the pixel 101 and the signal processing circuit 103 corresponding to the pixel 101 in FIGS.

[0021] The APD201 generates a pair of electric charges according to the incident light by photoelectric conversion. One of the two nodes of the APD201 is connected to a power supply line to which a driving voltage VL (first voltage) is supplied. The other of the two nodes of the APD201 is connected to a power supply line to which a driving voltage VH (second voltage) higher than the voltage VL is supplied. In FIG. 4, one node of the APD201 is an anode, and the other node of the APD is a cathode. A reverse bias voltage is supplied to the anode and cathode of the APD201 such that the APD201 performs an avalanche multiplication operation. By supplying such a voltage, the electric charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0022] When a reverse bias voltage is supplied, there are two modes: Geiger mode, in which the voltage difference between the anode and cathode is greater than the breakdown voltage, and linear mode, in which the voltage difference between the anode and cathode is close to or less than the breakdown voltage. An APD that operates in Geiger mode is called a SPAD. In the case of a SPAD, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V.

[0023] The quench element 202 is connected to a power supply line to which a drive voltage VH is supplied and to one of the anode and cathode nodes of the APD 201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quench operation). The quench element 202 also has a function of returning the voltage supplied to the APD 201 to the drive voltage VH by flowing a current equivalent to the voltage drop caused by the quench operation (recharge operation).

[0024] 5 shows an example in which the signal processing circuit 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212 in addition to the quench element 202. However, it is sufficient for the signal processing circuit 103 to have at least one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212 in addition to the quench element 202.

[0025] The waveform shaping unit 210 shapes the voltage change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 5, a circuit in which a plurality of inverters are connected in series may be used, or another circuit having a waveform shaping effect may be used.

[0026] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse RES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0027] 3 through a drive line 214 (not shown in FIG. 3) in FIG. 5, and switches between electrical connection and non-connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal, and outputs an output signal from the counter circuit 211 of the pixel to the vertical signal line 113.

[0028] The electrical connection may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing circuit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0029] <Circuit drive> FIG. 6 is a diagram showing a schematic diagram of the relationship between the operation of the APD and the output signal. The input side of the waveform shaping unit 210 is nodeA, and the output side is nodeB. Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201. When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage of nodeA drops. When the voltage drop amount becomes larger and the potential difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops as shown at time t2, and the voltage level of nodeA does not drop below a certain value. After that, between time t2 and time t3, a current that compensates for the voltage drop from the voltage VL flows through nodeA, and at time t3, nodeA is stabilized to the original potential level. At this time, the portion of the output waveform at nodeA that exceeds a certain threshold value is shaped by the waveform shaping section 210 and is output as a pulse signal at nodeB.

[0030] Photoelectric conversion devices according to respective embodiments of the present invention will be described below.

[0031] <System block diagram> Fig. 7 shows a system block diagram of the photoelectric conversion device 300 shown in the embodiment 1. Note that some of the functional blocks shown in Fig. 7 are realized by causing a computer (not shown) included in the photoelectric conversion device 300 to execute a computer program stored in a memory serving as a storage medium (not shown).

[0032] However, some or all of them may be realized by hardware. A dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) may be used as the hardware. Furthermore, the functional blocks shown in Fig. 6 do not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths. The above explanation regarding Fig. 6 also applies to Fig. 18.

[0033] The photoelectric conversion device 300 includes the photoelectric conversion element 100 of FIGS. 1 to 6, an imaging optical system 301, and a signal processing unit 302 that processes signals acquired by the photoelectric conversion element. The photoelectric conversion element 100 is configured with an avalanche photodiode for photoelectrically converting an optical image. The avalanche photodiode configures a pixel region that is two-dimensionally arranged. The photoelectric conversion element 100 includes a readout circuit that simultaneously reads out signals from pixels included in a first pixel group and signals from pixels included in a second pixel group.

[0034] The signal processing unit 302 has an image generation unit 303 that generates a first image from a signal acquired by the photoelectric conversion unit, a correction processing unit 304, and a memory unit 305 as a memory means that stores first array data based on the first characteristic information of the photoelectric conversion element 101.

[0035] The above-mentioned first characteristic information is information on the crosstalk characteristics between pixels caused by the avalanche light emission phenomenon of the photoelectric conversion element 100. The storage unit 306 may download the first characteristic information (first array data, etc.) from an external server or the like and temporarily store it. The first characteristic information is two-dimensional array data, and includes a numerical value indicating the occurrence probability for each element.

[0036] The correction processing unit 304 performs a first correction process using first array data based on the first characteristic information of the photoelectric conversion element 101. In the photoelectric conversion element of the present invention, the first array data is changed for each row among the multiple pixel groups (rows A to F in FIG. 4) that are read simultaneously. That is, in the correction processing unit that functions as a correction processing means, the first characteristic information used for correction is different between the first row and the second row that are read simultaneously. In the following description, an example in which read scanning is performed for each row of the pixel area will be described, but the scanning direction may also be the column direction. That is, the readout circuit simultaneously reads out multiple signals from the one-dimensionally arranged pixels included in each pixel group (pixel column) from the pixel group (pixel column) consisting of multiple pixels in the pixel area. Data in which the pixel group is one-dimensionally arranged in a predetermined scanning direction is called one row. In summary, the correction information used for the correction process is different between the first pixel group and the second pixel group. Here, it is not necessary that the correction information for each of the first pixel group and the second pixel group is different, and it is sufficient that at least one set of pixel groups uses different correction information. That is, in the correction process, correction information extracted or converted based on different parts of the first characteristic information is used for different pixel groups.

[0037] <Sequence data> FIG. 8 is a diagram showing a probability distribution of occurrence of erroneous counting between adjacent pixels of the photoelectric conversion element 100 as first array data, and shows an example of the first array data corresponding to the first characteristic information of the photoelectric conversion element 100. FIGS. 8(a), (b), (c), (d), (e), and (f) respectively show array data used for pixels located in rows A, B, C, D, E, and F that are simultaneously read out in FIG. 4. Each array data is two-dimensional array data used by the correction processing unit 304 and stored in the storage unit 305. The storage unit 306 stores the basic two-dimensional array data (first characteristic information), and the correction processing unit 304 as a correction processing means performs correction processing on the image using the whole or part of this two-dimensional array data (first array data). The first characteristic information and the first array data (correction information) corresponding to the pixel to be processed may be stored as a table in the storage unit 306, etc., or may be saved as a function.

[0038] Here, as described in Non-Patent Document 1, when the pixel is an avalanche photodiode, avalanche light emission phenomenon causes erroneous counting in adjacent pixels, that is, crosstalk between adjacent pixels (hereinafter referred to as light emission crosstalk).

[0039] The influence of a photon incident on one pixel on an adjacent pixel is determined by the probability of occurrence of light emission crosstalk. Since the probability of occurrence of light emission crosstalk is determined by the pixel structure of the photoelectric conversion element, the probability of occurrence of light emission crosstalk can be predicted by the pixel structure of the photoelectric conversion element. In the photoelectric conversion device 300 shown in the first embodiment, the first characteristic information related to the information on the probability of occurrence of light emission crosstalk is used to perform signal processing that suppresses the influence of erroneous counting, thereby improving image quality.

[0040] In particular, in the photoelectric conversion element in this embodiment, the first array data is changed for each row among multiple rows (rows A to F in FIG. 4) that are read simultaneously. This makes it possible to suppress deterioration of image quality due to miscounting. The reason for this will be explained below.

[0041] <Explanation of the principle and effect> In the photoelectric conversion element of this embodiment, multiple rows are read out simultaneously as shown in Fig. 4. Therefore, as shown in Fig. 9, the amount of light emission crosstalk received from surrounding pixels differs depending on the position in the multiple rows that are read out simultaneously. Specifically, row A receives only light emission crosstalk from the pixels in the lower half, while row D receives light emission crosstalk from the pixels above, below, left and right. Therefore, if the brightness of the subject is uniform, as shown in Fig. 10, rows A and F are dark and rows C and D are the brightest, causing horizontal stripes in the image and deteriorating image quality due to the effects of light emission crosstalk.

[0042] Therefore, in the photoelectric conversion element in this embodiment, the first array data (correction information) representing the probability of occurrence of miscounting between adjacent pixels is changed between multiple rows that are read simultaneously. This reflects in the correction that the influence of miscounting due to the amount of light emission crosstalk is different between multiple rows that are read simultaneously. Specifically, the first array data for pixels located in row A has the values ​​of the array elements in the upper half set to zero as shown in FIG. 8(a). On the other hand, the first array data for pixels located in row D is as shown in FIG. 8(d). Similarly, for rows B, C, E, and F, the values ​​of each element of the array data are made different as shown in FIG. 8 depending on the position of the pixel that is read simultaneously. This makes it possible to suppress deterioration of image quality due to miscounting.

[0043] <Flowchart> FIG. 11 is a diagram for explaining a flow chart of signal processing, and FIG. 12 is a diagram for explaining an example of correction processing. In the following explanation, each process (step) is represented by adding an S to the beginning, and the notation of the process (step) is omitted. First, in S401, the image generating unit 303 generates an image based on a signal acquired from a photoelectric conversion element. That is, the image generating unit 303 generates a first image in which the signal acquired by the photoelectric conversion unit 101 is arranged in a two-dimensional frame shape.

[0044] Next, in S402, the correction processing unit 304 performs a correction process using the first array data on the first image to generate a second image. At this time, as shown in Fig. 8, the first array data used for the correction process is changed among multiple rows that are read simultaneously. In other words, the first array data used for the correction process differs between the first pixel group and the second pixel group.

[0045] As described above, since the probability of occurrence of light emission crosstalk is predictable, by performing a convolution operation, the second image becomes a signal representing a miscount caused by light emission crosstalk. The first characteristic information (first array data, etc.) may be acquired from the storage unit 306 or an external server, etc. The second array data may also be stored as a table or function in the storage unit 306, etc. Here, S402 functions as an acquisition step (acquisition means) for acquiring the first characteristic information of the photoelectric conversion element.

[0046] Then, in S403, a correction process is performed on the first image using first characteristic information related to crosstalk between pixels of the photoelectric conversion element. That is, the correction processing unit 304 subtracts the second image from the first image to generate a third image. As described above, since the second image is a signal representing a counting error caused by light emission crosstalk, the third image is an image obtained by restoring a signal that would be obtained if no counting error due to light emission crosstalk occurred. That is, the effects of counting errors caused by crosstalk can be reduced by the processes of S402 to S403.

[0047] Fig. 12(a) shows the first image, Fig. 12(b) and (c) show the first array data of rows A and D, respectively, Fig. 12(d) shows the second image, and Fig. 12(e) shows the third image. For simplicity, the figures only show the first array data used for the pixels of rows A and D, but the array data shown in Fig. 8 is also used to perform convolution operations for the pixels of rows B, C, E, and F. Instead of performing the convolution operations, the first image and the second array data may be Fourier transformed and then the product may be found.

[0048] <Crosstalk matrix:> The first array data obtained from the first characteristic information shown in FIG. 8 may be one-dimensional or two-dimensional as long as it includes two or more pieces of data, but is preferably two-dimensional array data. Furthermore, considering the symmetry of the crosstalk probability, the first array data is preferably a matrix with an odd number of rows and columns, and is symmetrical on the left and right sides with the center as the center. Since the crosstalk probability value increases closer to the pixel that is the defect, the first array data has a peak value in the center and has a distribution that changes monotonically toward the ends of the data, except for the pixel. Specifically, when one row of the first array data is extracted as one-dimensional data, it has a distribution that monotonically decreases from the center, which is the peak value, toward both the left and right ends.

[0049] In multiple lines to be read simultaneously, it is preferable that the first array data is vertically symmetrical in the central line, and that the vertical symmetry is broken toward the top and bottom. Specifically, it is preferable that the values ​​of the upper elements of the first array data are smaller in the upper lines, and that the values ​​of the lower elements are smaller in the lower lines. In addition, it is preferable that the first array data is vertically symmetrical between two lines (lines A and F, lines B and E, and lines C and D in the example of FIG. 4) that are the same distance apart from the central line (between lines C and D in the example of FIG. 4) among multiple lines to be read simultaneously.

[0050] Further, consider the case where the first array data has N rows. In the multiple rows read simultaneously, it is preferable that the values ​​of the first array data are equal between the rows below the (N-1)÷2 row from the top (hereinafter referred to as the upper boundary row) and above the (N-1)÷2 row from the bottom (hereinafter referred to as the lower boundary row). (In the example of FIG. 4, rows C and D, hereinafter referred to as the center row). The first array data used in the upper boundary row (row B in FIG. 4) has a smaller value of the element in the top row than the first array data used in the center row. And, it is preferable that the first array data used in the row M rows above the upper boundary row (row A when M=1 in FIG. 4) has a smaller value of the element in the M+1 rows from the top than the first array data used in the center row. Similarly, the first array data used in the lower boundary row (row E in FIG. 4) has a smaller value of the element in the bottom row than the first array data used in the center row. It is preferable that the first array data used in the row L rows below the lower boundary row (row F when L=1 in Fig. 4) has smaller element values ​​in the L+1 rows from the bottom than the first array data used in the center row. In fact, it can be seen that the above conditions are met in the array data shown in Fig. 8.

[0051] The above describes the array data when multiple rows of pixel groups are simultaneously read in the row direction in the pixel region, but the array data when multiple columns are simultaneously scanned in the column direction in the pixel region also has a certain degree of symmetry. The first array data when the column direction is the scanning direction has a characteristic that the above-mentioned first array data is rotated 90 degrees. To summarize the above, first, the first characteristic information is array data with an odd number of rows and columns, and has symmetry around the center. The readout circuit can simultaneously read out signals of multiple one-dimensionally arranged pixel groups including a first pixel group (for example, the first column in the pixel region) and a second pixel group (the second column in the pixel region) in a predetermined scanning direction. The predetermined scanning direction is the row direction or the column direction. The first array data (correction information) is data extracted from the first characteristic information so as to have symmetry around the center pixel group in multiple pixel rows (pixel groups) that are simultaneously read out. Specifically, if the pixel groups are in the row direction, they are vertically symmetrical with respect to the center row, and if the pixel groups are in the column direction, they are horizontally symmetrical with respect to the center column. Furthermore, in the first array data (correction information), the element value of the first array data corresponding to the pixel group farther from the center in a plurality of one-dimensionally arranged pixel groups read out simultaneously is smaller than the element value of the first array data corresponding to the pixel group closer to the center. Specifically, for example, when the first pixel group is the S+1th column from the center and the second pixel group is the Sth column from the center, the element value of the first array data corresponding to the first pixel group is smaller than the element value of the first array data corresponding to the second pixel group. Also, consider a case where the predetermined scanning direction is the column direction of the pixel area and the number of columns of the first array data (correction information) is T. In a plurality of pixel groups (columns) read out simultaneously, the element values ​​of the corresponding first array data are equal between columns to the left of (T-1)÷2 columns from the left and to the right of (N-1)÷2 columns from the right.

[0052] <Change the arrangement according to the exposure time> It is preferable to change the first array data according to the exposure time of the photoelectric conversion element 101. The reason is explained below. FIG. 13(a) is a timing chart of the photoelectric conversion element when the exposure time is short, and FIG. 13(b) is a timing chart of the photoelectric conversion element when the exposure time is short. FIG. 13(a) and (b) also show the number of rows that are simultaneously performing the exposure operation at each timing. In FIG. 13(a), the number of rows that are simultaneously performing the exposure operation is six rows at all timings. That is, pixels other than the multiple rows that are simultaneously read are not performing the exposure operation, so they are only affected by the light emission crosstalk from the pixels in the multiple rows that are simultaneously read. On the other hand, in FIG. 13(b), there is a period during the exposure operation when pixels other than the multiple rows that are simultaneously read are also performing the exposure operation. Therefore, they are also affected by the light emission crosstalk from the pixels other than the multiple rows that are simultaneously read.

[0053] Therefore, in the case where only the effects of light emission crosstalk from pixels in multiple rows that are read simultaneously are present, as in Fig. 13(a), it is preferable to set the values ​​(hatched parts) of the elements of the array that indicate the probability of light emission crosstalk from pixels that are not operating simultaneously to zero, as in Fig. 8(a), (b), (e), and (f).

[0054] As the exposure time becomes longer, the influence of light emission crosstalk from pixels other than the multiple rows being read simultaneously becomes stronger, so it is preferable to increase the values ​​of the elements in the hatched parts in Figures 8(a), (b), (e), and (f). In other words, the shorter the exposure time, the greater the difference between the first array data used in the first row and the first array data used in the second row. In other words, the shorter the exposure time, the greater the difference between the correction information used in the first pixel group and the correction information used in the second pixel group.

[0055] In the case of so-called full accumulation in which exposure is performed in all periods, it is preferable to use the same first array data for all rows because all pixels are performing exposure in all periods. In other words, when exposure is performed in all periods, it is preferable that the correction information in the correction processing means is equal between the first row (pixel group) and the second row (pixel group).

[0056] <Color sensor> The photoelectric conversion element 101 may be a monochrome sensor that does not have an on-chip color filter, or may be a so-called color sensor that has at least two or more types of pixels with different spectral characteristics. In the case of a color sensor, it is preferable to change the correction process for each color.

[0057] In the case of a color sensor, there is no difference in the probability of crosstalk between colors, but since the signal level changes for each different color pixel depending on the color of the subject, there are colors that are more susceptible to miscounting due to crosstalk. For example, in the case of a color sensor with a Bayer array of RGGB photoelectric conversion elements, for a typical subject, the brightness of the B pixel is the lowest and the brightness of the G pixel is the highest. Therefore, the B pixel is most susceptible to miscounting due to crosstalk, and the G pixel is the least susceptible.

[0058] <Change the size of the array for each color> The larger the size of the array data used in the convolution calculation, the more the impact of miscounting due to crosstalk can be reduced. On the other hand, it is more susceptible to the impact of differences in the probability of light emission crosstalk caused by manufacturing variations due to cluster scratches. Therefore, it is preferable that the first array data has the minimum size that suppresses the impact of miscounting due to crosstalk. Therefore, in the case of a color sensor with a Bayer array type RGGB photoelectric conversion element, it is preferable to use a larger size of the array data used in S502 for B pixels than for G pixels.

[0059] <Embodiment 2: Clock-driven type> The photoelectric conversion device shown in the embodiment 2 has a different method of driving the photoelectric conversion element from the photoelectric conversion device shown in the embodiment 1. Specifically, as shown in Fig. 14, the quench element 202 is composed of a MOS transistor, and the on / off of the quench element 202 is controlled by a control signal CLK connected to the gate of the quench element. The control signal CLK is controlled by a signal generating unit in the control pulse generating unit 115.

[0060] <Clock Drive Explanation> FIG. 15 is a diagram showing a schematic diagram of the relationship between the control signal CLK of the switch, the voltage of the node nodeA, the voltage of the node nodeB, and the output signal in the photoelectric conversion element shown in FIG. 14. In the photoelectric conversion element of the second embodiment, when the control signal CLK is at a high level, the drive voltage VH is not easily supplied to the APD, and when the control signal CLK is at a low level, the drive voltage VH is supplied to the APD. The high level of the control signal CLK is, for example, 1V, and the low level of the control signal CLK is, for example, 0V. When the control signal CLK is at a high level, the switch is turned off, and when the control signal CLK is at a low level, the switch is turned on. The resistance value of the switch when the control signal CLK is at a high level is higher than the resistance value of the switch when the control signal CLK is at a low level. When the control signal CLK is at a high level, even if avalanche multiplication occurs in the APD, the recharge operation is not easily performed, so the voltage supplied to the APD is a voltage equal to or lower than the breakdown voltage of the APD. Therefore, the avalanche multiplication operation in the APD stops.

[0061] At time t1, the control signal CLK changes from high level to low level, the switch is turned on, and the recharge operation of the APD is started. This causes the voltage of the cathode of the APD to transition to high level. Then, the voltage difference between the voltages applied to the anode and cathode of the APD becomes a state in which avalanche multiplication is possible. The voltage of the cathode is the same as that of the node nodeA. Therefore, when the voltage of the cathode transitions from low level to high level, the voltage of the node nodeA becomes equal to or higher than the judgment threshold at time t2. At this time, the pulse signal output from the node nodeB is inverted and goes from high level to low level. When the recharge is completed, the voltage difference of the drive voltage VH-drive voltage VL is applied to the APD 201. After that, the control signal CLK becomes high level, and the switch is turned off.

[0062] Next, at time t3, when a photon is incident on the APD 201, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the switch 202, and the voltage at the cathode drops. That is, the voltage at the node A drops. When the voltage at the node A drops below the decision threshold, the voltage at the node B goes from low to high. That is, the part of the output waveform at the node A that exceeds the decision threshold is shaped by the waveform shaping unit 210 and output as a signal at the node B. Then, the counter circuit counts, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0063] Between time t3 and time t4, photons are incident on the APD. However, since the switch is in the off state and the voltage applied to the APD 201 does not have a voltage difference that allows avalanche multiplication, the voltage level of the node nodeA does not exceed the decision threshold.

[0064] At time t4, the control signal CLK changes from high to low, turning the switch on. As a result, a current flows through node A to compensate for the voltage drop from the drive voltage VL, and the voltage of node A transitions to its original voltage level. At this time, the voltage of node A becomes equal to or higher than the decision threshold at time t5, so the pulse signal of node B is inverted and goes from high to low.

[0065] At time t6, the node A is stabilized to the original voltage level, and the control signal CLK goes from low to high. After this, the voltages of the nodes and signal lines change in response to the control signal CLK and the incidence of photons, as described from time t1 to time t6.

[0066] In this way, the recharge frequency of the APD can be controlled by applying the control signal CLK to the switch 202 to switch the switch 202 on and off. If the control signal CLK is not used, there is a problem that the actual count value becomes smaller than the count value corresponding to the luminance of the incident light at high luminance. In that case, this problem can be solved by applying the control signal CLK to the switch 202 to switch the switch 202 on and off.

[0067] However, when the APD recharge frequency is controlled by the control signal CLK, the relationship between the number of input signals and the number of output signals is not linear. If the effects of light emission crosstalk are ignored, the relationship between the number of input signals and the number of output signals can be theoretically derived. Specifically, when the number of input signals is Nph, the number of output signals is Nct, the frequency of the control signal CLK (the inverse of the number of CLKs per unit time) is f, and the length of the exposure period is T, this is expressed by the following equation 1.

[0068]

number

[0069] In the photoelectric conversion device shown in the second embodiment, a correction process is performed to simultaneously reduce the influence of the nonlinear response caused by the control signal CLK and the influence of the miscounting caused by crosstalk. That is, the nonlinearity of the image is corrected based on the number of pulse signals and the length of the exposure time. The following will explain this.

[0070] <Flowchart> Fig. 16 is a flowchart of signal processing in the photoelectric conversion device shown in the embodiment 2. The following mainly describes the parts that are different from the flowchart in Fig. 11. First, in the same way as in Fig. 11, in S501, the image processing unit 303 generates a first image.

[0071] Next, in S502, similarly to S402, the correction processing unit 304 performs a correction process using the first array data on the first image to generate a second image. At this time, the first array data used for the correction process is changed among multiple rows that are read simultaneously.

[0072] In the photoelectric conversion device of the second embodiment, the correction processing unit 304 performs signal processing of S503, branching from S502. In S503, the correction processing unit 304 performs nonlinearity correction processing on the first image to return the nonlinear response generated by the control signal CLK to linearity, and generates a third image. That is, the correction processing unit 304 performs nonlinearity correction on the image based on the number of pulse signals and the length of exposure time. Specifically, the number of input signals Nph can be calculated from the number of output signals Nct using the following formula 2.

[0073]

number

[0074] Then, in S504, the correction processing unit 304 subtracts the second image from the third image to generate a fourth image. As described above, since the second image is a signal representing a counting error caused by light emission crosstalk, the fourth image is an image obtained by restoring a signal that would be obtained if no counting error caused by light emission crosstalk occurred. That is, the processes of S503 to S505 can simultaneously reduce the influence of the nonlinear response caused by the control signal CLK and the influence of the counting error caused by crosstalk.

[0075] Figure 17(a) shows the first image, Figures 17(b) and (c) show the first array data of rows A and D, respectively, Figure 17(d) shows the second image, Figure 17(e) shows the third image, and Figure 17(f) shows the fourth image.

[0076] <Embodiment 3: Photoelectric Conversion System> 18 shows a system block diagram of a photoelectric conversion system using the photoelectric conversion device according to Embodiments 1 to 3. The photoelectric conversion system 800 includes a photoelectric conversion device 300 having a photoelectric conversion element 100, a control unit 801, a storage unit 802, and a communication unit 803.

[0077] The photoelectric conversion element 100 captures an optical image formed by an imaging optical system 301. The signal read from the photoelectric conversion element 100 is subjected to image generation and correction processing in a signal processing unit 302, as well as black level correction, gamma curve adjustment, noise reduction, data compression, and other processing to generate a final image. When the photoelectric conversion element 100 has an on-chip RGB color filter, it is further preferable to perform processing such as white balance correction and color conversion.

[0078] The control unit 801 has a built-in CPU as a computer, and functions as a control means for controlling the operation of each part of the entire photoelectric conversion device 800 based on a computer program stored in a memory as a storage medium. In addition, the control unit 802 controls the length of the exposure period of each frame of the photoelectric conversion element 100 and the timing of the control signal CLK via a control pulse generation unit of the photoelectric conversion element 100.

[0079] The recording unit 803 includes a recording medium such as a memory card, a hard disk, etc. The communication unit 804 includes a wireless or wired interface, and outputs a generated image to the outside of the photoelectric conversion device 800 and receives a signal from the outside.

[0080] A computer program for implementing all or part of the control in this embodiment may be provided to the photoelectric conversion device via a network or various storage media. A computer (or a CPU, MPU, etc.) in the photoelectric conversion device may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0081] 300 Photoelectric conversion device 301 Imaging Optical System 302 Signal Processing Section 303 Image Generation Unit 304 Correction processing section 305 Storage section

Claims

1. a pixel region in which pixels, each of which is an avalanche photodiode for photoelectrically converting an optical image, are arranged two-dimensionally; a photoelectric conversion element having a readout circuit that simultaneously reads out signals from pixels included in the first pixel group and signals from pixels included in the second pixel group; A generating means for generating a first image based on a signal obtained from the photoelectric conversion element; a correction unit that performs a correction process on the first image by using first characteristic information related to crosstalk between pixels of the photoelectric conversion element, The photoelectric conversion device, wherein the correction means provides different correction information based on the first characteristic information for the first pixel group and the second pixel group.

2. 2. The photoelectric conversion device according to claim 1, wherein the first pixel group and the second pixel group are two or more pixels arranged one-dimensionally in a row direction or a column direction.

3. The first characteristic information is two-dimensional array data, The correction information is a portion of the two-dimensional array data corresponding to the pixel region, 3. The photoelectric conversion device according to claim 2, wherein at least one of the one-dimensional data corresponding to the scanning direction of the two-dimensional array data has a peak value in the center.

4. 4. The photoelectric conversion device according to claim 3, wherein the one-dimensional data has a distribution that monotonically decreases from the peak value toward a data end.

5. 3. The photoelectric conversion device according to claim 2, wherein the first characteristic information is array data having an odd number of rows and columns and having symmetry around the center.

6. the readout circuit simultaneously reads out signals of a plurality of one-dimensionally arranged pixel groups including the first pixel group and the second pixel group in a predetermined scanning direction; 3. The photoelectric conversion device according to claim 2, wherein the correction information is data extracted from the first characteristic information so as to have symmetry about a central pixel group among a plurality of pixel groups read out simultaneously.

7. The photoelectric conversion device according to claim 6, characterized in that, in the correction information, in a plurality of one-dimensionally arranged pixel groups that are read out simultaneously, the element values ​​of the correction information corresponding to pixel groups farther from the center are smaller than the element values ​​of the correction information corresponding to pixel groups closer to the center.

8. 7. The photoelectric conversion device according to claim 6, characterized in that, when the first pixel group and the second pixel group are two pixel groups spaced the same distance from a center, the correction information is characterized in that the value of an element in the correction information corresponding to the first pixel group and the value of an element in the correction information corresponding to the second pixel group are symmetrical to each other.

9. The photoelectric conversion device of claim 6, characterized in that when the specified scanning direction is the row direction of the pixel area, and the number of rows of the correction information is N, in a group of pixels that are read out simultaneously, the values ​​of each element of the corresponding correction information are equal between rows below the (N-1)÷2 row from the top and rows above the (N-1)÷2 row from the bottom.

10. The photoelectric conversion device according to claim 1 , wherein the first characteristic information varies depending on an exposure time of the photoelectric conversion element.

11. 11. The photoelectric conversion device according to claim 10, wherein the correction means uses the same first characteristic information for the first pixel group and the second pixel group when performing an exposure operation for the entire period.

12. The photoelectric conversion device according to claim 10 , wherein the correction means is configured to adjust the difference between the correction information used for the first pixel group and the correction information used for the second pixel group to be greater as the exposure time is shorter.

13. 2. The photoelectric conversion device according to claim 1, wherein the correction means includes a process for performing a convolution operation of the correction information on the first image.

14. The photoelectric conversion device according to claim 1 , wherein the first pixel group and the second pixel group are different pixel groups adjacent to each other.

15. The photoelectric conversion device according to claim 1 , wherein the photoelectric conversion element has at least two types of pixels having different spectral characteristics.

16. The photoelectric conversion device according to claim 15 , wherein the first characteristic information has a size that is changed for each pixel having a different spectral characteristic.

17. 2. The photoelectric conversion device according to claim 1, characterized in that the photoelectric conversion element is connected to one of the anode and cathode nodes of the avalanche photodiode and a power supply line to which a drive voltage is applied, and has a switch that switches a resistance value between the one node and the power supply line, and a signal generating unit that generates a pulse signal for controlling the switching of the switch.

18. 18. The photoelectric conversion device according to claim 17, wherein the correction means performs nonlinear correction on the first image based on the number of the pulse signals and the length of the exposure time.

19. a pixel region in which pixels, each of which is an avalanche photodiode for photoelectrically converting an optical image, are arranged two-dimensionally; A method for controlling a photoelectric conversion device including a photoelectric conversion element having a readout circuit that simultaneously reads out signals from pixels included in a first pixel group and signals from pixels included in a second pixel group, the method comprising: A generation step of generating a first image based on a signal acquired from the photoelectric conversion element; a correction step of performing a correction process on the first image using first characteristic information related to crosstalk between pixels of the photoelectric conversion element, The correction step is characterized in that the first characteristic information used in the correction process differs between the first pixel group and the second pixel group.

20. A computer program for controlling each unit of the photoelectric conversion device according to any one of claims 1 to 18 by a computer.

Images