Imaging apparatus, imaging system, movable body, and apparatus

Abstract

To prevent a reduction in image quality by improving the accuracy of correcting output of pixels.SOLUTION: An imaging apparatus of the present disclosure has a photoelectric conversion device in which a plurality of pixels each including a photoelectric conversion element having an avalanche photodiode are two-dimensionally arranged in matrix, and a processing device that processes output signals from the plurality of pixels. The plurality of pixels each has a first pixel part having a pixel for imaging and a second pixel part having at least one pair of pixels having an aperture pixel and a light shielding pixel surrounding at least part of the aperture pixel. The processing device uses output signals from the aperture pixel and the light shielding pixel in the pair to create a correction pattern for correcting an output signal from the pixel for imaging corresponding to output patterns of the aperture pixel and the light shielding pixel in the pair.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an imaging device, an imaging system, a moving object, and an apparatus. [Background technology]

[0002] As a means for correcting the output signal of so-called defective pixels in an imaging device, a method is known in which a pixel having a higher output level than the surrounding pixels is extracted as the defective pixel, and the output signal of the defective pixel is replaced with an output signal obtained by averaging the outputs of the surrounding pixels (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2007 / 0030365 Summary of the Invention [Problem to be solved by the invention]

[0004] However, simply replacing the output of a defective pixel with the output signal of another pixel, as in the correction method described in Patent Document 1, reduces the accuracy of correction of the output signal of the defective pixel, resulting in a problem of reduced image quality in the generated image.

[0005] The present disclosure has been made in view of the above, and aims to suppress deterioration in image quality by improving the accuracy of correction of pixel output signals. [Means for solving the problem]

[0006] In order to achieve the above object, an imaging device according to the present disclosure includes: a photoelectric conversion device in which a plurality of pixels, each including a photoelectric conversion element having an avalanche photodiode, are two-dimensionally arranged in a matrix; A processing device for processing output signals of the plurality of pixels; having the plurality of pixels include a first pixel section having an imaging pixel, and a second pixel section having at least one pixel set including an aperture pixel and a light-shielding pixel surrounding at least a part of the aperture pixel; The processing device generates a correction pattern for correcting an output signal of the imaging pixel, the correction pattern corresponding to an output pattern of the aperture pixel and the light-shielding pixel of the pixel set, using output signals of the aperture pixel and the light-shielding pixel of the pixel set. The present invention includes an imaging device characterized by the above-mentioned.

[0007] In order to achieve the above object, the imaging system according to the present disclosure includes: The imaging device, a signal processing device for processing a signal output from the imaging device; The imaging system includes:

[0008] In order to achieve the above object, the moving body according to the present disclosure is The imaging device, A mobile device; a signal processing device for acquiring information from a signal output from the imaging device; a control device that controls the moving device based on the information; The present invention relates to a moving object having a

[0009] In order to achieve the above object, the device according to the present disclosure comprises: An apparatus including the imaging device, an optical device corresponding to the imaging device; A control device for controlling the imaging device; a processing device that processes a signal output from the imaging device; a display device for displaying information obtained by the imaging device; a storage device that stores information obtained by the imaging device; and and a mechanical device that operates based on information obtained by the imaging device. Effect of the Invention

[0010] According to the present disclosure, it is possible to improve the accuracy of correction of output signals from defective pixels in an imaging device and the quality of images generated. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram of a photoelectric conversion device according to an embodiment. [Diagram 2] 2 is a schematic diagram of a substrate of a photoelectric conversion device according to one embodiment. FIG. [Diagram 3] 2 is a schematic diagram of a circuit board of a photoelectric conversion device according to an embodiment. FIG. [Figure 4] 2 is a configuration example of a pixel circuit of a photoelectric conversion device according to an embodiment. [Diagram 5] 3 is a schematic diagram showing driving of a pixel circuit of a photoelectric conversion device according to an embodiment; FIG. [Figure 6] 1 shows a block diagram of an imaging device according to a first embodiment. [Figure 7] 1 shows a pixel arrangement diagram of an imaging device according to a first embodiment. [Figure 8] FIG. 2 shows a block diagram of a correction pattern generation process of the imaging device according to the first embodiment. [Figure 9] 3A to 3C are schematic diagrams illustrating correction processing of the imaging device according to the first embodiment. [Figure 10] 11 shows a pixel arrangement diagram of an imaging device according to a second embodiment. [Figure 11] FIG. 11 is a block diagram of a correction pattern generation process of an imaging device according to a second embodiment. [Figure 12] 13 shows a pixel arrangement diagram of an imaging device according to a third embodiment. [Figure 13] FIG. 13 is a block diagram of a correction pattern generation process of an imaging device according to a third embodiment. [Figure 14] FIG. 13 shows a block diagram of an imaging device according to a fourth embodiment. [Figure 15] FIG. 13 is a schematic diagram of an apparatus including a semiconductor device according to a fifth embodiment. [Figure 16] FIG. 13 is a diagram illustrating an example of the configuration of an imaging system according to a sixth embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of the configuration of an imaging system and a moving object according to a seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Note that the present disclosure is not limited to the following embodiment, and can be modified as appropriate without departing from the gist of the present disclosure. In addition, in the drawings described below, parts having the same functions are given the same reference numerals, and their description may be omitted or simplified.

[0013] In addition, in each embodiment described below, a solid-state image sensor (image sensor) will be mainly described as an example of a photoelectric conversion device. However, each embodiment is not limited to a solid-state image sensor (image sensor), and can be applied to other examples of photoelectric conversion devices. For example, there are distance measuring devices (devices for measuring distance using focus detection or TOF (Time Of Flight)), photometry devices (devices for measuring the amount of incident light, etc.), etc.

[0014] First Embodiment Hereinafter, the embodiments will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) will be used as necessary. The use of these terms is for the purpose of facilitating understanding of the embodiments with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms. It is not something that can be done.

[0015] In this specification, the plan view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer. The cross-sectional view refers to a surface in a direction perpendicular to the light incident surface of the semiconductor layer. When the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0016] In the imaging device of this embodiment, the anode of the APD (Avalanche Photodiode) is set to a fixed potential, and a signal is taken out from the cathode side. Therefore, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is an N-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is a P-type semiconductor region. This embodiment is also valid when the cathode of the APD is set to a fixed potential, and the signal is taken out from the anode side. In this case, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is a P-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is an N-type semiconductor region. In the following, a case where one node of the APD is set to a fixed potential will be described, but the potentials of both nodes may fluctuate.

[0017] In addition, when the term "impurity concentration" is used simply in this specification, it means the net impurity concentration minus the amount compensated for by the impurity of the opposite conductivity type. In other words, "impurity concentration" refers to the net doping concentration. A region where the P-type added impurity concentration is higher than the N-type added impurity concentration is a P-type semiconductor region. Conversely, a region where the N-type added impurity concentration is higher than the P-type added impurity concentration is an N-type semiconductor region.

[0018] Hereinafter, a photoelectric conversion device according to this embodiment and a method for driving the same will be described with reference to FIGS.

[0019] FIG. 1 is a diagram showing the configuration of a stacked photoelectric conversion device 100 according to this embodiment. The photoelectric conversion device 100 is configured by stacking two substrates, a sensor substrate 11 and a circuit substrate 21, and electrically connecting the sensor substrate 11 and the circuit substrate 21. The sensor substrate 11 has a first semiconductor layer having a photoelectric conversion element 102 described later, and a first wiring structure. The circuit substrate 21 has a second semiconductor layer having circuits such as a signal processing portion 103 described later, and a second wiring structure. The photoelectric conversion device 100 is configured by stacking the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order from the bottom. Note that the photoelectric conversion device described in each of the following embodiments is a back-illuminated photoelectric conversion device in which light is incident from the first surface and a circuit substrate is disposed on the second surface.

[0020] In the following, the sensor substrate 11 and the circuit substrate 21 are described as diced chips, but the form of each substrate is not limited to a chip. For example, each substrate may be a wafer. In addition, each substrate may be stacked in a wafer state and then diced, or may be chipped and then stacked and bonded.

[0021] A pixel region 12 is disposed on the sensor substrate 11, and a circuit region 22 that processes signals detected in the pixel region 12 is disposed on the circuit substrate 21.

[0022] 2 is a diagram showing an example of the arrangement of the sensor substrate 11. In a plan view of the sensor substrate 11, pixels 101 each having a photoelectric conversion element 102 including an APD are arranged in a two-dimensional array to form a pixel region 12.

[0023] The pixel 101 is typically a pixel for forming an image, but is also a pixel for TOF (Time of Flight). In the case of a 3D camera for use in a 3D camera of flight, it is not necessary to form an image. That is, the pixel 101 may be a pixel for measuring the time and amount of light that arrives.

[0024] Fig. 3 is a configuration diagram of the circuit board 21. The circuit board 21 has a signal processing unit 103 that processes charges photoelectrically converted by the photoelectric conversion element 102 in Fig. 2, a column circuit 112, a control pulse generating unit 115, a horizontal scanning circuit unit 111, a signal line 113, and a vertical scanning circuit unit 110. The photoelectric conversion element 102 in Fig. 2 and the signal processing unit 103 in Fig. 3 are electrically connected via connection wiring provided for each pixel.

[0025] The vertical scanning circuit section 110 receives a control pulse supplied from a control pulse generating section 115 and supplies the control pulse to each pixel. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0026] The signal output from the photoelectric conversion element 102 of the pixel is processed by a signal processing unit 103. The signal processing unit 103 is provided with a counter, a memory, etc., and digital values ​​are held in the memory.

[0027] The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing unit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0028] For a selected column, a signal is output from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 to the signal line 113. The signal output to the signal line 113 is output via the output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100.

[0029] 2, the photoelectric conversion elements in the pixel region may be arranged one-dimensionally. The function of the signal processing unit does not necessarily need to be provided for each photoelectric conversion element, and for example, one signal processing unit may be shared by multiple photoelectric conversion elements to perform signal processing sequentially.

[0030] 2 and 3, a plurality of signal processing units 103 are arranged in a region overlapping the pixel region 12 in a planar view. A vertical scanning circuit unit 110, a horizontal scanning circuit unit 111, a column circuit 112, an output circuit 114, and a control pulse generating unit 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a planar view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12, and the vertical scanning circuit unit 110, the horizontal scanning circuit unit 111, the column circuit 112, the output circuit 114, and the control pulse generating unit 115 are arranged in a region overlapping the non-pixel region in a planar view.

[0031] Fig. 4 is an example of a block diagram including the equivalent circuits of Fig. 2 and Fig. 3. In Fig. 2, a photoelectric conversion element 102 having an APD 201 is provided on a sensor substrate 11, and other members are provided on a circuit substrate 21.

[0032] The APD201 is a photoelectric conversion unit that generates charge pairs according to incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD201. A reverse bias voltage that causes the APD201 to perform avalanche multiplication is supplied to the anode and cathode. By supplying such a voltage, charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0033] When a reverse bias voltage is applied, the potential difference between the anode and the cathode is There are two modes: Geiger mode, in which the device is operated with a potential difference greater than the breakdown voltage, and linear mode, in which the potential difference between the anode and cathode is close to or less than the breakdown voltage.

[0034] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). For example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD 201 may be operated in either linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger than that of an APD in linear mode, and the effect of withstanding voltage is more pronounced, so it is preferable that the APD 201 is a SPAD.

[0035] The quench element 202 is connected to a power supply that supplies a voltage VH and to the APD 201. The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quench operation). The quench element 202 also has a function of returning the voltage supplied to the APD 201 to the voltage VH by flowing a current equivalent to the voltage drop caused by the quench operation (recharge operation).

[0036] The signal processing section 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212. In this specification, it is sufficient that the signal processing section 103 has any one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212.

[0037] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 4, a circuit in which a plurality of inverters are connected in series may be used, or another circuit having a waveform shaping effect may be used.

[0038] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0039] A control pulse pSEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via a drive line 214 (not shown in Fig. 3) in Fig. 4, and switches between electrical connection and non-connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

[0040] The electrical connection may be switched by disposing a switch such as a transistor between the quench element 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0041] In this embodiment, a configuration using the counter circuit 211 has been shown. However, instead of the counter circuit 211, a photoelectric conversion device 100 may be configured to acquire the pulse detection timing using a time-to-digital converter (hereinafter, TDC) and a memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. To measure the timing of the pulse signal, a control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 in FIG. 1 via a drive line. The TDC acquires, as a digital signal, a signal obtained by converting the input timing of the signal output from each pixel via the waveform shaping unit 210 into a relative time based on the control pulse pREF.

[0042] 5A and 5B are diagrams showing the relationship between the operation of the APD of this embodiment and the output signal. Fig. 5A is a diagram showing the APD 201, quench element 202, and waveform shaping unit 210 of Fig. 4. Here, the input side of the waveform shaping unit 210 is nodeA, and the output side is nodeB. In Fig. 5B, the upper graph shows the waveform change of nodeA in Fig. 5A, and the lower graph shows the waveform change of nodeB in Fig. 5A.

[0043] Between time t0 and time t1, a potential difference of VH-VL is applied to the APD 201 in FIG. 5A. When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201, an avalanche multiplication current flows through the quench element 202, and the voltage at nodeA drops. When the voltage drop amount becomes larger and the potential difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops as at time t2, and the voltage level at nodeA does not drop by more than a certain value. After that, between time t2 and time t3, a current that compensates for the voltage drop from the voltage VL flows through nodeA, and at time t3, nodeA is stabilized to the original potential level. At this time, the part of the output waveform at nodeA that exceeds a certain threshold is shaped by the waveform shaping unit 210 and output as a signal at nodeB.

[0044] The arrangement of the signal lines 113, the column circuits 112, and the output circuits 114 is not limited to that shown in Fig. 3. For example, the signal lines 113 may be arranged to extend in the row direction, and the column circuits 112 may be arranged at the ends of the signal lines 113.

[0045] The imaging device 300 according to the present embodiment will be described below. The imaging device according to the first embodiment will be described with reference to FIGS.

[0046] 6 shows an overall block diagram of the imaging device 300. The imaging device 300 includes the above-mentioned photoelectric conversion device 100, a pre-processing unit 400, a crosstalk correction pattern generating unit 500, and a crosstalk correcting unit 600.

[0047] The photoelectric conversion device 100 has a plurality of pixels arranged two-dimensionally in a matrix. Specifically, the photoelectric conversion device 100 includes an aperture pixel section 120 arranged in a matrix for outputting an image signal, a light-shielding pixel section whose light incident surface side is shielded by a light-shielding film, and a crosstalk correction pixel section 130 in a form in which the aperture pixel section is surrounded by the light-shielding pixel section. The aperture pixel section 120 is a first pixel section having pixels for imaging. The crosstalk correction pixel section 130 is a second pixel section having at least one pixel set having an aperture pixel and a light-shielding pixel surrounding at least a part of the aperture pixel.

[0048] The crosstalk correction pixel section 130, which is a feature of the imaging device 300 of this embodiment, is configured to surround the aperture pixel section with light-shielding pixels. Therefore, the crosstalk correction pixel section 130 has at least one pixel set having an aperture pixel and a light-shielding pixel surrounding at least a part of the aperture pixel. The reason for this will be explained. For example, when the pixel is an avalanche photodiode (APD), crosstalk may occur due to the avalanche light emission phenomenon that occurs when avalanche-multiplied charges are recombined. By providing light-shielding pixels around the aperture pixel, the influence of crosstalk due to the avalanche light emission phenomenon occurring in the aperture pixel appears in the output of the surrounding light-shielding pixels, making it possible to calculate the influence of crosstalk on the output signal.

[0049] The pre-processing unit 400 performs pre-processing of the signal. The output signal from the photoelectric conversion device 100 is input to the pre-processing unit 400. In the pre-processing, corrections such as offset correction and gain correction of each signal are appropriately performed. The pre-processing unit 400 processes pixel signals for crosstalk correction and image signals for imaging. The aperture pixel signal is separated from the crosstalk correction pattern generating unit 500 and the crosstalk correction unit 600, and output to the downstream signal processing unit (crosstalk correction pattern generating unit 500, crosstalk correction unit 600).

[0050] The crosstalk correction pattern generation unit 500 generates a crosstalk correction pattern that indicates, as a matrix of crosstalk probability, the influence that an aperture pixel has on surrounding pixels when avalanche multiplication occurs, using pixel signals for crosstalk correction. This crosstalk correction pattern is a correction pattern that corresponds to the output pattern of an aperture pixel and a light-shielded pixel that constitute one pixel pair of the crosstalk correction pixel unit 130.

[0051] The crosstalk correction unit 600 corrects the aperture pixel signal for imaging input from the pre-processing unit 400 using the crosstalk correction pattern that is the output of the crosstalk correction pattern generation unit 500, and outputs the corrected aperture pixel signal for imaging.

[0052] In the following, in order to further explain the embodiment of the present invention in detail, a description will be given with reference to FIG. 7A and FIG. 7B. FIG. 7A and FIG. 7B are diagrams showing pixel arrangements of the photoelectric conversion device 100. FIG. 7B is a partially enlarged diagram showing a part of the crosstalk correction pixel section 130 of FIG. 7A. In the photoelectric conversion device 100 of this embodiment, the pixel arrangement of the crosstalk correction pixel section 130 is a 5×5 matrix arrangement, and the central pixel is an aperture pixel. In addition, in the aperture pixel, avalanche multiplication occurs due to light incidence, and the avalanche light emission phenomenon occurs accordingly. On the other hand, in the light-shielded pixel, avalanche multiplication does not occur due to light incidence, so the output of the light-shielded pixel is an output according to the crosstalk associated with the avalanche light emission phenomenon in the aperture pixel. In addition, although the pixel arrangement of the crosstalk correction pixel section 130 is a 5×5 matrix arrangement here, the pixel arrangement is not limited to this, and may be, for example, a 7×7 or 3×3 matrix arrangement depending on the degree to which the influence of crosstalk is to be corrected.

[0053] In the photoelectric conversion device 100, in order to suppress the influence of crosstalk from the aperture pixel section 120 for imaging, it is desirable to separate the crosstalk correction pixel section 130 from the aperture pixel section 120 for imaging. In this embodiment, as shown in FIG. 7B, two light-shielding pixels are arranged between the aperture pixel section 120 for imaging and the crosstalk correction pixel section 130. It is desirable to separate the crosstalk correction pixel section 130 from the aperture pixel section 120 for imaging by the same number or more as the number of light-shielding pixels surrounding the aperture pixel in the crosstalk correction pixel section. In addition, it is not necessarily necessary to provide a pixel structure between the aperture pixel section for imaging and the crosstalk correction pixel section, and it is sufficient that there is a distance equivalent to the area where crosstalk from the aperture pixel section 120 for imaging occurs.

[0054] 8A and 8B are block diagrams showing an example of a correction pattern generation process executed by the crosstalk correction pattern generation unit 500. In this embodiment, as shown in FIG. 8A, the output signals of each of the 5×5 pixels constituting the crosstalk correction pixel unit 130 are input to the standardization processing unit 501. The standardization processing unit 501 then generates a crosstalk correction pattern by standardization based on the ratio between the output signal of the aperture pixel unit (output signal of "500" in the figure), which is the pixel input for crosstalk correction, and the output signal of the light-shielding pixel unit (output signal other than "500" in the figure). The crosstalk correction pattern thus generated constitutes two-dimensional data of N rows and M columns (one of N and M is an integer of 2 or more, and the other is an integer of 1 or more).

[0055] In addition, when a black level offset occurs, the normalization processing unit 501 may obtain a difference between an output signal of a light-shielded pixel unit not affected by crosstalk and a pixel input signal for crosstalk correction. Then, the normalization processing unit 501 may obtain a ratio of an output signal of the light-shielded pixel unit to an output signal of the aperture pixel unit in the pixel input signal for crosstalk correction.

[0056] FIG. 8B is a diagram showing a modified example of the generation of the above-mentioned crosstalk correction pattern. In this example, the pixel input signal for crosstalk correction is set as the output signal of each pixel of the aperture pixel and the 3×3 light-shielding pixels on the lower right side of the light-shielding pixel section. In a plan view of the pixel section, crosstalk between pixels occurs isotropically. Therefore, instead of setting the pixel input signal for crosstalk correction as the output signal of all pixels of the crosstalk correction pixel section 130, a folding process may be performed on the output signals of the aperture pixel and some light-shielding pixels to generate a crosstalk correction pattern. As shown in FIG. 8B, the crosstalk correction pattern generating section 500 has a folding process section 502. In this modification, it is assumed that the influence of crosstalk on each pixel on the lower right side of the light-shielding pixel section is the same as the influence of crosstalk on each pixel on the upper right side, upper left side, and lower left side of the light-shielding pixel section. Therefore, the folding process section 502 performs folding process on the output signals of each pixel of the aperture pixel and the 3×3 light-shielding pixels on the lower right side of the light-shielding pixel section to generate an output signal of pixels corresponding to a 5×5 matrix. The normalization processing unit 501 generates a crosstalk correction pattern using the output signals of 5×5 pixels output from the folding processing unit 502. According to this modification, the crosstalk correction pattern can be generated by folding processing without processing the output signals of the pixels on the upper right, upper left, and lower left sides, and therefore it is expected that the crosstalk correction pattern can be generated more efficiently.

[0057] As shown in FIG. 8B, the crosstalk correction pattern generated by the folding processing unit 502 is two-dimensional array data. When one row or one column of this two-dimensional array data is extracted as one-dimensional data, this one-dimensional data has a peak value in the center. This one-dimensional data has a distribution that changes monotonically from the center of the data, which is the peak value, toward the data ends, which are both ends of the one-dimensional data. Furthermore, in the two-dimensional array data on which the one-dimensional data is based, one-dimensional data that shares the data center, which is the peak value of the extracted one-dimensional data, and is arranged in a direction intersecting the one-dimensional data also has a distribution that changes monotonically from the center of the data toward the data ends. The above peak value may be a bottom value.

[0058] 9 is a diagram showing a schematic diagram of the correction process executed by the crosstalk correction unit 600. The crosstalk correction unit 600 divides the output signal of the aperture pixel for imaging before correction by the crosstalk correction pattern generated by the crosstalk correction pattern generation unit 500. Here, the division is, for example, a deconvolution operation. By the correction process using the division of the crosstalk correction unit 600, it is possible to obtain a pixel signal in which the influence of crosstalk has been corrected ("output after crosstalk correction" in the figure), as shown in FIG. 9.

[0059] As a result, in the imaging device 300, it is possible to extract only the output signal component of the defective pixel, which is free from the influence of the light emission crosstalk, from an image in which the pixels around the so-called defective pixel are affected by the light emission crosstalk of the defective pixel. This makes it possible to contribute to high accuracy of the defect correction in the defect correction circuit provided in the subsequent stage of the crosstalk correction unit 600. Furthermore, in the imaging device 300, even if the occurrence of light emission crosstalk causes a deterioration in image quality, such as blurring the edges of the subject and reducing the sense of resolution, in a normal image, the effect of improving image quality, such as improving the sense of resolution, can be obtained by correcting the influence of the light emission crosstalk.

[0060] As described above, the occurrence of pixel emission crosstalk depends on the temperature conditions, voltage conditions, individual variations, and changes over time of the imaging device, so crosstalk correction using fixed correction conditions according to conventional technology may result in poor correction accuracy and residual correction. In contrast, according to this embodiment, a crosstalk correction pixel section is provided on the same substrate as an aperture pixel section for imaging, and a crosstalk correction pattern for the output signal of the aperture pixel section is generated in real time. This makes it possible to generate a crosstalk correction pattern in a form that follows variable factors such as the temperature conditions, voltage conditions, individual variations, and changes over time of the imaging device, thereby improving the correction accuracy.

[0061] <Second embodiment> Next, an imaging device according to a second embodiment will be described with reference to Figures 10A to 10C and 11. In the following description, differences from the first embodiment will be mainly described, and the same components as those in the first embodiment will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0062] 10A is a diagram showing a pixel arrangement of a photoelectric conversion device 100 according to the second embodiment. Unlike the pixel arrangement in the first embodiment, in the photoelectric conversion device 100 of this embodiment, a plurality of crosstalk correction pixel sections 130 are provided around an aperture pixel section 120 for imaging. Therefore, in the photoelectric conversion device 100 of this embodiment, a plurality of pairs of aperture pixels constituting the crosstalk correction pixel section 130 and light-shielding pixels surrounding the aperture pixels are arranged. Therefore, in this embodiment, the crosstalk correction pixel section 130 has at least two pairs of aperture pixels and light-shielding pixels.

[0063] In the imaging device 300, the output of the aperture pixel of the crosstalk correction pixel unit may be saturated or may be close to zero depending on the imaging conditions. In such a case, in a configuration in which a single crosstalk correction pixel unit 130 is arranged for the aperture pixel unit 120 for imaging as in the first embodiment, there is a possibility that the crosstalk correction pattern cannot be generated normally. However, in this embodiment, a plurality of crosstalk correction pixel units are arranged for the aperture pixel unit 120 for imaging. Therefore, the crosstalk correction pattern generating unit can generate the crosstalk correction pattern using only the output signals within an appropriate output range among the output signals of each aperture pixel of the crosstalk correction pixel unit. Since the crosstalk correction pattern can be generated based on the result of averaging the output signals of a plurality of aperture pixels, it is possible to reduce the influence of noise in the correction.

[0064] In addition, considering that the amount of incident light to the aperture pixel unit may change within the screen due to imaging conditions or depending on the incident angle to the edge of the screen, it is desirable to arrange multiple crosstalk correction pixel units 230 around the aperture pixel unit 120 for imaging, as illustrated in Figure 10A.

[0065] FIG. 10B also shows a schematic configuration of the crosstalk correction pixel section 230 arranged above the aperture pixel section 120 for imaging. As shown in FIG. 10B, in a pair of aperture pixels and light-shielding pixels constituting one crosstalk correction pixel section 230, light-shielding pixels are arranged around one aperture pixel, forming a 5×5 matrix arrangement. As shown in FIG. 10B, each aperture pixel of the crosstalk correction pixel section 230 is arranged in the same row, and the interval between adjacent aperture pixels is 15 pixels. This interval between aperture pixels is set for the purpose of eliminating the influence of crosstalk between the aperture pixels of the crosstalk correction pixel section 230. Therefore, in the crosstalk correction pixel section 230 of this embodiment, for the pair of aperture pixels and light-shielding pixels, at least two pairs of pixel pairs have a first pixel pair having a first aperture pixel and a second pixel pair having a second aperture pixel. The first aperture pixel and the second aperture pixel are configured to be separated by at least two pixels.

[0066] As a modification of this embodiment, as shown in FIG. 10C, adjacent aperture pixels of the crosstalk correction pixel section 230 may be arranged in different rows and diagonally to each other. The intensity of the luminescence crosstalk generated in the aperture pixel is strong in the up, down, left, and right directions of the aperture pixel, and weak in the diagonal direction of the aperture pixel. Therefore, by arranging adjacent aperture pixels of the crosstalk correction pixel section 230 diagonally to each other, there is an advantage that the aperture pixels are less susceptible to the influence of crosstalk even if they are arranged closer to each other than in the example of FIG. 10B. Note that, in FIG. 10C, the output signals of the light-shielding pixels in the overlapping areas of the crosstalk correction pixel section 230 may be obtained by the above-mentioned folding process during the crosstalk correction pattern processing in the subsequent stage.

[0067] FIG. 11 shows the correction pattern generation process of the crosstalk correction pattern generating unit 500 of this embodiment. 11 shows a block diagram of a processing unit that executes the above-mentioned process. As shown in Fig. 11, in the crosstalk correction pattern generation unit 500, a crosstalk correction pattern is generated by processing of an opening pixel determination processing unit 503, a standardization processing unit 501, a crosstalk pattern storage unit 505, and a region averaging processing unit 504. In the following description, it is assumed that N crosstalk correction pixel units 230 are arranged in the imaging device 300, and that the crosstalk correction pixel units 230 occupy regions 1 to N.

[0068] First, the aperture pixel determination processing unit 503 determines whether or not the output signals of the aperture pixels of the crosstalk correction pixel unit 230 of each of the regions 1 to N are within a predetermined output range. This makes it possible to select the cases where the outputs of the aperture pixels of the crosstalk correction pixel unit 230 of each of the regions 1 to N are saturated or low output, that is, the output signals of the aperture pixels that are inappropriate for generating a crosstalk correction pattern.

[0069] Next, in the normalization processing unit 501, the ratio between the output signal of the aperture pixel and the output signal of the light-shielding pixel is calculated as in the first embodiment, and the output signal of the aperture pixel deemed appropriate by the aperture pixel determination processing unit 503 is stored in the crosstalk pattern storage unit 505. Then, in the region averaging processing unit 504, the output signals of the aperture pixels within an appropriate output range are averaged among the output signals of the aperture pixels of the crosstalk correction pixel units 230 of each of the regions 1 to N. This averaging processing is expected to improve the generation accuracy of the crosstalk correction pattern when the crosstalk correction pattern is generated in real time.

[0070] <Third embodiment> An imaging device according to the third embodiment will be described with reference to Figures 12A to 12C and 13. In the following description, differences from the first and second embodiments will be mainly described, and the same configurations as those in the first and second embodiments will be denoted by the same reference numerals, and detailed description will be omitted.

[0071] 12A is a diagram showing a pixel arrangement of a photoelectric conversion device 100 according to the third embodiment. The difference from the first embodiment is that in this embodiment, the pixel arrangement of the aperture pixel section 120 for imaging is an RGB Bayer array for color imaging, and the aperture pixels of the crosstalk correction pixel section 230 are configured using monochromatic color filters. Therefore, in this embodiment, the aperture pixel section 120 for imaging is configured of two or more types of pixels (here, R pixels, G pixels, and B pixels) having different spectral sensitivities, and the aperture pixels of the crosstalk correction pixel section 230 are configured of pixels having one type of spectral sensitivity.

[0072] Fig. 12A shows an example of a pixel arrangement of the photoelectric conversion device 100 of this embodiment. As shown in Fig. 12A, in the imaging device 300 of this embodiment, light-shielding pixel regions of the crosstalk correction pixel section 230 are provided on the upper and left sides of the aperture pixel section 120 for imaging. A plurality of crosstalk correction pixel sections 130 are provided around the aperture pixel section 120 for imaging. Therefore, similar to the second embodiment, the output signals of the aperture pixels can be averaged and selected by the plurality of crosstalk correction pixel sections 230 when generating a crosstalk correction pattern.

[0073] Fig. 12B shows the arrangement of color filters of each pixel in the aperture pixel section 120 for imaging, and as shown in Fig. 12B, the pixel arrangement of the aperture pixel section 120 for imaging is a Bayer pattern using general RGB color filters. Note that although the Bayer pattern is used here, another pattern may be used, or an RGBW pixel arrangement using a White pixel in addition to the Bayer pattern may be used.

[0074] 12C shows an example of a pixel arrangement in the crosstalk correction pixel section 230. As shown in FIG. 1, the aperture pixels of each crosstalk correction pixel section 230 are configured with a single-color green color filter. This is because the output signal of the pixel for crosstalk correction does not require color information, but only luminance information is required. In this embodiment, therefore, in consideration of the fact that sensitivity is required when photographing in a dark place, green, which provides more luminance information than other colors, is used for the color filter of the aperture pixel. Note that the light-shielding pixels of the crosstalk correction pixel section 230 may be arranged in a Bayer array similar to the aperture pixels for imaging for ease of manufacturing, or only green color filters may be used.

[0075] 12D shows a modified example of the imaging device 300 of this embodiment in which the aperture pixels of the crosstalk correction pixel section 230 are white pixels. By using white pixels instead of green pixels as aperture pixels, it is possible to further improve the sensitivity of the aperture pixels to incident light. Note that since the pixels of the crosstalk correction pixel section 230 are not particularly used for imaging, the use of white pixels does not affect the generation of the crosstalk correction pattern.

[0076] Furthermore, in order to further improve the generation accuracy of the output signal of the crosstalk correction pixel unit 230 when the imaging device 300 captures an image in a dark place, the charge accumulation time of the crosstalk correction pixel unit 230 may be made longer than the charge accumulation time of the imaging aperture pixel unit 120. By making the charge accumulation time longer, it is possible to increase the output signals of the aperture pixels and the light-shielded pixels affected by the emission crosstalk in the crosstalk correction pixel unit 230.

[0077] 13 is a block diagram of a processing unit that executes correction pattern generation processing of the crosstalk correction pattern generation unit 500 of this embodiment. In this embodiment, the crosstalk correction pattern generation unit 500 executes processing by an inter-frame processing unit 506 on the output of the aperture pixels held in the crosstalk pattern holding unit 505. When the output signals of the aperture pixels within an appropriate output range among the output signals of the aperture pixels of the crosstalk correction pixel units 230 of each of the regions 1 to N are small, the influence of noise can be reduced by performing frame addition processing of the output signals of the aperture pixels for each region.

[0078] In the crosstalk correction pattern generating unit 500 of this embodiment, first, the aperture pixel determination processing unit 503 determines whether the output signal of the aperture pixel of the crosstalk correction pixel unit 230 of each of the regions 1 to N is within a predetermined output range. Then, the inter-frame processing unit 506 adds the signal data of the previous frame held in the crosstalk correction pixel output holding unit 507 to the signal data of the current frame. Here, the signal data of the output signal of the aperture pixel determined by the aperture pixel determination processing unit 503 to be outside the threshold is not added. The data after the addition processing by the inter-frame processing unit 506 is held in the crosstalk correction pixel output holding unit 507 and is output to the subsequent normalization processing unit 501. The processing of the normalization processing unit 501, the crosstalk pattern holding unit 505, and the region averaging processing unit 504 is the same as that of the second embodiment, and therefore will not be described here.

[0079] According to this embodiment, in capturing a color image by the imaging device 300, the crosstalk correction pixels are set to be green pixels or white pixels with high sensitivity, and frame averaging processing is performed in the crosstalk correction pattern generating unit 500. This is expected to improve the generation accuracy of the crosstalk correction pattern even under low illuminance, and thus improve the crosstalk correction accuracy.

[0080] <Fourth embodiment> An imaging device according to the fourth embodiment will be described with reference to Fig. 14. In the following description, differences from the first to third embodiments will be mainly described, and the same configurations as those in the first to third embodiments will be given the same reference numerals, and detailed description will be omitted.

[0081] 14 shows an overall block diagram of an image pickup device 300 of this embodiment. The image pickup device 300 differs from the image pickup devices 300 of the first to third embodiments in that the image pickup device 300 includes a voltage control unit 700 that controls the voltage of the photoelectric conversion device 100 by using a crosstalk correction pattern as an input.

[0082] The degree of crosstalk caused by the avalanche emission of the aperture pixel varies depending on the voltage state of the photoelectric conversion unit. In the imaging device 300 of the first to third embodiments, the state of crosstalk that varies depending on temperature conditions and aging is corrected by the crosstalk correction pattern, but in this case, there is a concern about the influence of correction residues when the degree of crosstalk increases. Therefore, in the imaging device 300 of this embodiment, a crosstalk correction pattern is generated in real time, and the voltage control unit 700 compares it with a predetermined crosstalk correction pattern and changes the voltage based on the comparison result. Then, the imaging device 300 performs feedback control to keep the degree of correction of the pixel output signal by the crosstalk correction pattern within a certain range of levels. The imaging device 300 of this embodiment controls the voltage VL, but may also control the voltage VH. In addition, the imaging device 300 may perform feedback control so that the value of the light-shielded pixel part of the crosstalk correction pattern becomes minimum.

[0083] According to this embodiment, the degree of correction of pixel output signals by the crosstalk correction pattern falls within a certain range, so that the accuracy of crosstalk correction can be further improved.

[0084] <Fifth embodiment> Any of the first to fourth embodiments can be applied to the fifth embodiment. FIG. 15 is a schematic diagram for explaining a device 1591 including a semiconductor device 1530 of this embodiment. The semiconductor device 1530 can be any of the imaging devices explained in the first to fourth embodiments, or an imaging device combining a plurality of the embodiments. The device 1591 including the semiconductor device 1530 will be explained in detail. As described above, the semiconductor device 1530 can include a package 1520 that houses the semiconductor device 1510 in addition to the semiconductor device 1510 having a semiconductor layer. The package 1520 can include a base to which the semiconductor device 1510 is fixed, and a cover such as glass that faces the semiconductor device 1510. The package 1520 can further include a bonding member such as a bonding wire or a bump that connects a terminal provided on the base and a terminal provided on the semiconductor device 1510.

[0085] The device 1591 can include at least one of an optical device 1540, a control device 1550, a processing device 1560, a display device 1570, a storage device 1580, and a mechanical device 1590. The optical device 1540 corresponds to the semiconductor device 1530. The optical device 1540 is, for example, a lens, a shutter, or a mirror. The control device 1550 controls the semiconductor device 1530. The control device 1550 is, for example, a semiconductor device such as an ASIC.

[0086] The processing device 1560 processes the signal output from the semiconductor device 1530. The processing device 1560 is a semiconductor device such as a CPU or ASIC for configuring an AFE (analog front end) or a DFE (digital front end). The display device 1570 is an EL display device or a liquid crystal display device that displays information (images) obtained by the semiconductor device 1530. The storage device 1580 is a magnetic device or a semiconductor device that stores information (images) obtained by the semiconductor device 1530. The storage device 1580 is a volatile memory such as an SRAM or a DRAM, or a non-volatile memory such as a flash memory or a hard disk drive.

[0087] Mechanical device 1590 has a moving or propulsive part, such as a motor or engine. In 1591, the signal output from the semiconductor device 1530 is displayed on a display device 1570, or transmitted to the outside by a communication device (not shown) included in the device 1591. For this purpose, the device 1591 preferably further includes a storage device 1580 and a processing device 1560 in addition to the memory circuit and arithmetic circuit included in the semiconductor device 1530. The mechanical device 1590 may be controlled based on the signal output from the semiconductor device 1530.

[0088] The device 1591 is also suitable for electronic devices such as information terminals (e.g., smartphones and wearable devices) with a photographing function and cameras (e.g., interchangeable lens cameras, compact cameras, video cameras, and surveillance cameras). The mechanical device 1590 in the camera can drive components of the optical device 1540 for zooming, focusing, and shutter operation. Alternatively, the mechanical device 1590 in the camera can move the semiconductor device 1530 for vibration isolation operation.

[0089] The device 1591 may be a transport device such as a vehicle, a ship, or an aircraft. The mechanical device 1590 in the transport device may be used as a moving device. The device 1591 as a transport device is suitable for transporting the semiconductor device 1530 or for assisting and / or automating driving (operation) by using a photographing function. The processing device 1560 for assisting and / or automating driving (operation) can perform processing for operating the mechanical device 1590 as a moving device based on information obtained by the semiconductor device 1530. Alternatively, the device 1591 may be a medical device such as an endoscope, a measuring device such as a distance measuring sensor, an analytical device such as an electron microscope, an office machine such as a copier, or an industrial device such as a robot.

[0090] According to the fifth embodiment, it is possible to obtain good pixel characteristics. Therefore, the value of the semiconductor device 1530 can be increased. Increasing the value here means at least one of adding a function, improving performance, improving characteristics, improving reliability, improving manufacturing yield, reducing environmental load, reducing costs, making the device smaller, and reducing weight.

[0091] Therefore, if the semiconductor device 1530 according to the fifth embodiment is used in the equipment 1591, the value of the equipment can be improved. For example, by mounting the semiconductor device 1530 on a transport equipment, excellent performance can be obtained when photographing the outside of the transport equipment or measuring the external environment. Therefore, in manufacturing and selling the transport equipment, it is advantageous to decide to mount the semiconductor device 1530 according to the fifth embodiment on the transport equipment in order to improve the performance of the transport equipment itself. In particular, the semiconductor device 1530 is suitable for transport equipment that performs driving assistance and / or automatic driving of the transport equipment using information obtained by the semiconductor device 1530.

[0092] Sixth Embodiment Next, an imaging system according to the sixth embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing a schematic configuration of the imaging system according to this embodiment.

[0093] The imaging devices described in the first to fourth embodiments are applicable to various imaging systems. Applicable imaging systems are not particularly limited, but include various devices such as digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and medical cameras. Camera modules equipped with an optical system such as a lens and an imaging device (photoelectric conversion device) are also included in the imaging system. FIG. 16 illustrates a block diagram of a digital still camera as an example of these.

[0094] 16, the imaging system 2000 includes an imaging device 300, an imaging optical system 2002, a CPU 2010, a lens control unit 2012, an imaging device control unit 2014, and an image processing unit 2016. The imaging system 2000 further includes an aperture shutter control unit 2018, a display unit 2020, an operation switch 2022, and a recording medium 2024.

[0095] The imaging optical system 2002 is an optical system for forming an optical image of a subject, and includes a lens group, an aperture 2004, etc. The aperture 2004 has a function of adjusting the amount of light during shooting by adjusting its opening diameter, and also functions as a shutter for adjusting the exposure time when shooting a still image. The lens group and aperture 2004 are held so that they can move forward and backward along the optical axis, and their linked operations realize a variable magnification function (zoom function) and a focus adjustment function. The imaging optical system 2002 may be integrated into the imaging system, or may be an imaging lens that can be attached to the imaging system.

[0096] The imaging device 300 is disposed in the image space of the imaging optical system 2002 so that its imaging surface is located therein. The imaging device 300 is the imaging device described in the first to fourth embodiments, and is configured to include a CMOS sensor (pixel section) and its peripheral circuit (peripheral circuit region). The imaging device 300 has pixels, each having a plurality of photoelectric conversion sections, arranged two-dimensionally, and a color filter is arranged for these pixels, thereby forming a two-dimensional single-plate color sensor. The imaging device 300 photoelectrically converts the subject image formed by the imaging optical system 2002, and outputs the image signal and focus detection signal.

[0097] The lens control unit 2012 controls the forward and backward movement of the lens group of the imaging optical system 2002 to perform variable magnification and focus adjustment, and is composed of circuits and a processing device configured to realize this function. The aperture shutter control unit 2018 changes the aperture diameter of the aperture 2004 (variable aperture value) to adjust the amount of light for photographing, and is composed of circuits and a processing device configured to realize this function.

[0098] The CPU 2010 is a control device within the camera that handles various controls of the camera body, and includes an arithmetic unit, ROM, RAM, an A / D converter, a D / A converter, a communication interface circuit, etc. The CPU 2010 controls the operation of each unit within the camera in accordance with a computer program stored in the ROM or the like, and executes a series of photographing operations such as AF, including detection of the focus state of the imaging optical system 2002 (focus detection), imaging, image processing, and recording. The CPU 2010 is also a signal processing device.

[0099] The imaging device control unit 2014 controls the operation of the imaging device 300, and also performs A / D conversion on signals output from the imaging device 300 and transmits the signals to the CPU 2010. The imaging device control unit 2014 is configured to realize these functions by circuits and control devices. The imaging device 300 may have the A / D conversion function. The image processing unit 2016 is a processing device that performs image processing such as gamma conversion and color interpolation on A / D converted signals to generate image signals, and is configured to realize these functions by circuits and control devices. The display unit 2020 is a display device such as a liquid crystal display (LCD), and displays information on the shooting mode of the camera, a preview image before shooting, a confirmation image after shooting, a focus state at the time of focus detection, and the like. The operation switch 2022 is configured to include a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The recording medium 2024 is for recording shot images, and may be built into the imaging system, or may be a removable device such as a memory card.

[0100] In this manner, by configuring the imaging system 2000 to which the imaging device 300 according to the first to fourth embodiments is applied, a high-performance imaging system can be realized.

[0101] Seventh embodiment Next, an imaging system and a moving object according to the seventh embodiment will be described with reference to Figs. 17A and 17B. Figs. 17A and 17B show an imaging system and a moving object according to the present embodiment. FIG. 1 is a diagram showing a configuration of a moving body.

[0102] FIG. 17A shows an example of an imaging system 2100 related to a vehicle-mounted camera. The imaging system 2100 has an imaging device 2110. The imaging device 2110 is any one of the imaging devices described in the first to third embodiments. The imaging system 2100 has an image processing unit 2112, which is a processing device that performs image processing on a plurality of image data acquired by the imaging device 2110. The imaging system 2100 also has a parallax acquisition unit 2114, which is a processing device that calculates a parallax (phase difference of parallax images) from a plurality of image data acquired by the imaging device 2110. The imaging system 2100 also has a distance acquisition unit 2116, which is a processing device that calculates a distance to an object based on the calculated parallax, and a collision determination unit 2118, which is a processing device that determines whether or not there is a possibility of a collision based on the calculated distance. Here, the parallax acquisition unit 2114 and the distance acquisition unit 2116 are examples of information acquisition means that acquire information such as distance information to an object. That is, the distance information is information on the parallax, the defocus amount, the distance to the object, and the like. The collision determination unit 2118 may determine the possibility of a collision using any of these distance information. The above-mentioned processing device may be realized by dedicated hardware, or may be realized by general-purpose hardware that performs calculations based on software modules. The processing device may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like. Alternatively, the processing device may be realized by a combination of FPGA, ASIC, and the like.

[0103] The imaging system 2100 is connected to a vehicle information acquisition device 2120, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The imaging system 2100 is also connected to a control ECU 2130, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 2118. That is, the control ECU 2130 is an example of a moving body control means that controls a moving body based on distance information. The imaging system 2100 is also connected to an alarm device 2140 that issues an alarm to the driver based on the judgment result of the collision judgment unit 2118. For example, when the judgment result of the collision judgment unit 2118 indicates that there is a high possibility of a collision, the control ECU 2130 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 2140 warns the user by sounding an alarm, displaying alarm information on a screen of a car navigation system, etc., or vibrating a seat belt or steering wheel.

[0104] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the imaging system 2100. Fig. 17B shows the imaging system 2100 when imaging the area in front of the vehicle (imaging range 2150). The vehicle information acquisition device 2120 sends an instruction to operate the imaging system 2100 to perform imaging. By using the imaging device of the above-mentioned first to fourth embodiments as the imaging device 2110, the imaging system 2100 of this embodiment can further improve the accuracy of distance measurement.

[0105] In the above description, an example of control to avoid collision with other vehicles has been described, but the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to avoid going out of a lane, and the like. Furthermore, the imaging system is not limited to vehicles such as automobiles, but can be applied to moving bodies (transportation equipment) such as ships, aircraft, and industrial robots. The moving devices in moving bodies (transportation equipment) are various driving sources such as engines, motors, wheels, and propellers. In addition, the present invention can be applied not only to moving bodies, but also to a wide range of equipment that uses object recognition, such as intelligent transport systems (ITS).

[0106] The disclosure of this embodiment includes the following configuration. (Configuration 1) a photoelectric conversion device in which a plurality of pixels, each including a photoelectric conversion element having an avalanche photodiode, are two-dimensionally arranged in a matrix; A processing device for processing output signals of the plurality of pixels; having the plurality of pixels include a first pixel section having an imaging pixel, and a second pixel section having at least one pixel set including an aperture pixel and a light-shielding pixel surrounding at least a part of the aperture pixel; The processing device generates a correction pattern for correcting an output signal of the imaging pixel, the correction pattern corresponding to an output pattern of the aperture pixel and the light-shielding pixel of the pixel set, using output signals of the aperture pixel and the light-shielding pixel of the pixel set. 1. An imaging device comprising: (Configuration 2) The imaging device according to configuration 1, wherein the one-dimensional data corresponding to at least one row or one column of the plurality of pixels included in the correction pattern has a peak value or a bottom value in the center of the one-dimensional data. (Configuration 3) 3. The imaging device according to configuration 2, wherein the one-dimensional data has a distribution that changes monotonically from the peak value or the bottom value toward a data end. (Configuration 4) The imaging device according to configuration 2 or 3, wherein another one-dimensional data including the peak value or the bottom value of the one-dimensional data and arranged in a direction intersecting the one-dimensional data has a distribution that changes monotonically from the peak value or the bottom value toward the end of the data. (Configuration 5) 5. The imaging device according to configuration 4, wherein the correction pattern is configured as two-dimensional data with N rows and M columns (one of N and M is an integer of 2 or more, and the other is an integer of 1 or more). (Configuration 6) 6. The imaging device according to any one of configurations 1 to 5, wherein the processing device generates the correction pattern using a ratio between an output signal of the aperture pixel and an output signal of the light-shielding pixel. (Configuration 7) 7. The imaging device according to claim 6, wherein the processing device generates the correction pattern by normalizing the output signals of the light-shielding pixels using the output signals of the aperture pixels as a reference. (Configuration 8) 8. The imaging device according to any one of configurations 1 to 7, wherein the processing device corrects output signals of the imaging pixels using the correction pattern. (Configuration 9) The imaging device described in any one of configurations 1 to 8, characterized in that the second pixel section has at least two sets of pixels, the at least two sets of pixels having a first set of pixels having a first aperture pixel and a second set of pixels having a second aperture pixel, and a distance between the first aperture pixel and the second aperture pixel is at least two pixels or more. (Configuration 10) 10. The imaging device according to configuration 9, wherein the imaging pixel is disposed between the first aperture pixel and the second aperture pixel. (Configuration 11) The imaging device described in configuration 9, characterized in that the processing device generates a correction pattern for correcting the output signal of the imaging pixel using a correction pattern generated using the output signal of the first aperture pixel and an output signal of a light-shielding pixel of a pixel set including the first aperture pixel, and a correction pattern generated using the output signal of the second aperture pixel and an output signal of a light-shielding pixel of a pixel set including the second aperture pixel. (Configuration 12) the first pixel unit is composed of two or more types of pixels having different spectral sensitivities; The aperture pixels of the second pixel section are each composed of pixels having one type of spectral sensitivity. 12. The imaging device according to any one of configurations 1 to 11, (Configuration 13) the first pixel section is composed of an R pixel, a G pixel, and a B pixel; The aperture pixel of the second pixel portion is a G pixel. 13. The imaging device according to configuration 12, (Configuration 14) the first pixel section is composed of an R pixel, a G pixel, and a B pixel; The aperture pixel of the second pixel portion is a W pixel. 13. The imaging device according to configuration 12, (Configuration 15) 15. The imaging device according to any one of configurations 1 to 14, characterized in that a charge accumulation time in the imaging pixels of the first pixel section and a charge accumulation time in the aperture pixels and the light-shielding pixels of the second pixel section are different from each other. (Configuration 16) 16. The imaging device according to any one of configurations 1 to 15, characterized in that the processing device performs frame addition processing on output signals of the aperture pixel and the light-shielding pixel in the pixel set of the second pixel section. (Configuration 17) 17. The imaging device according to any one of configurations 1 to 16, wherein the processing device performs voltage control in driving the photoelectric conversion element by the photoelectric conversion device in accordance with the correction pattern. (Configuration 18) An imaging device according to any one of configurations 1 to 17; a signal processing device for processing a signal output from the imaging device; An imaging system comprising: (Configuration 19) A mobile object, An imaging device according to any one of configurations 1 to 17; A mobile device; a signal processing device for acquiring information from a signal output from the imaging device; a control device that controls the moving device based on the information; A moving object comprising: (Configuration 20) An apparatus including the imaging device according to any one of configurations 1 to 17, an optical device corresponding to the imaging device; A control device for controlling the imaging device; a signal processing device that processes a signal output from the imaging device; a display device for displaying information obtained by the imaging device; a storage device that stores information obtained by the imaging device; and and a mechanical device that operates based on information obtained by the imaging device. [Explanation of symbols]

[0107] 100 Photoelectric conversion device, 101 Pixel, 201 Avalanche photodiode, 500 Crosstalk correction pattern generator, 300 Imaging device

Claims

1. a photoelectric conversion device in which a plurality of pixels, each including a photoelectric conversion element having an avalanche photodiode, are two-dimensionally arranged in a matrix; a processing device for processing output signals of the plurality of pixels; and the plurality of pixels include a first pixel section having an imaging pixel and a second pixel section having at least one pixel set including an aperture pixel and a light-shielding pixel surrounding at least a part of the aperture pixel; The processing device generates a correction pattern for correcting the output signals of the imaging pixels, the correction pattern corresponding to the output patterns of the aperture pixels and the light-shielding pixels of the pixel set, using output signals of the aperture pixels and the light-shielding pixels of the pixel set. An imaging device characterized by:

2. 2. The imaging device according to claim 1, wherein one-dimensional data corresponding to at least one row or one column of the plurality of pixels included in the correction pattern has a peak value or a bottom value at a position corresponding to the aperture pixel.

3. 3. The imaging device according to claim 2, wherein the one-dimensional data has a distribution that changes monotonically from the peak value or the bottom value toward a data end.

4. 4. The imaging device according to claim 2, wherein another one-dimensional data set including the peak value or the bottom value of the one-dimensional data and arranged in a direction intersecting the one-dimensional data set has a distribution that changes monotonically from the peak value or the bottom value toward the end of the data set.

5. 5. The imaging device according to claim 4, wherein the correction pattern is configured as two-dimensional data with N rows and M columns (one of N and M is an integer of 2 or more, and the other is an integer of 1 or more).

6. 4. The imaging device according to claim 1, wherein the processing device generates the correction pattern using a ratio between the output signals of the aperture pixels and the output signals of the light-shielding pixels. Place.

7. 7. The imaging device according to claim 6, wherein the processing device generates the correction pattern by normalizing the output signals of the light-shielding pixels using the output signals of the aperture pixels as a reference.

8. 4. The imaging device according to claim 1, wherein the processing device corrects the output signals of the imaging pixels using the correction pattern.

9. 4. The imaging device according to claim 1, wherein the second pixel section has at least two sets of pixels, the at least two sets of pixels having a first set of pixels having a first aperture pixel and a second set of pixels having a second aperture pixel, and the first aperture pixel and the second aperture pixel are separated by a distance of at least two pixels.

10. 10. The imaging device according to claim 9, wherein the imaging pixel is disposed between the first aperture pixel and the second aperture pixel.

11. 10. The imaging device according to claim 9, wherein the processing device generates a correction pattern for correcting the output signals of the imaging pixels using a correction pattern generated using the output signal of the first aperture pixel and the output signals of light-shielding pixels of a pixel set including the first aperture pixel, and a correction pattern generated using the output signal of the second aperture pixel and the output signals of light-shielding pixels of a pixel set including the second aperture pixel.

12. the first pixel unit is composed of two or more types of pixels having different spectral sensitivities, The aperture pixel of the second pixel unit is configured from pixels having one type of spectral sensitivity.

4. The imaging device according to claim 1, wherein the imaging device is a lens.

13. the first pixel unit is composed of an R pixel, a G pixel, and a B pixel, The aperture pixel of the second pixel unit is a G pixel.

13. The imaging device according to claim 12.

14. the first pixel unit is composed of an R pixel, a G pixel, and a B pixel, The aperture pixel of the second pixel unit is a W pixel.

13. The imaging device according to claim 12.

15. 4. The imaging device according to claim 1, wherein the charge accumulation time in the imaging pixels of the first pixel section and the charge accumulation time in the aperture pixels and the light-shielded pixels of the second pixel section are different from each other.

16. 4. The imaging device according to claim 1, wherein the processing device performs frame addition processing on output signals of the aperture pixel and the light-shielding pixel in the pixel pair of the second pixel unit.

17. 4. The imaging device according to claim 1, wherein the processing device controls a voltage when the photoelectric conversion device drives the photoelectric conversion elements in accordance with the correction pattern.

18. The imaging device according to any one of claims 1 to 3; a signal processing device that processes a signal output from the imaging device; An imaging system comprising:

19. A mobile object, The imaging device according to any one of claims 1 to 3; A mobile device; a signal processing device that acquires information from a signal output from the imaging device; a control device that controls the moving device based on the information; A moving object characterized by having:

20. An apparatus comprising the imaging device according to any one of claims 1 to 3, an optical device corresponding to the imaging device; a control device that controls the imaging device; a signal processing device that processes a signal output from the imaging device; a display device that displays information obtained by the imaging device; a storage device that stores information obtained by the imaging device; and and a mechanical device that operates based on information obtained by the imaging device.

Images