Light receiving device, control method and ranging system

Abstract

Provided are a light receiving device, a control method, and a distance measurement system capable of suppressing the influence of incident light other than measurement light. According to the present disclosure, there is provided a light receiving device including: a pixel array section having measurement pixels for measuring a distance to a target object and a plurality of phase difference pixels in pairs for pupil-dividing incident light from the target object to detect a phase difference; a first distance measurement unit that generates a first distance value to the target on the basis of information relating to the difference between the timing at which the measurement pixel receives the photons and a predetermined time point; and a second distance measurement unit that generates a second distance value to the target on the basis of information corresponding to the number of photons incident on each of the plurality of phase difference pixels.

Description

Technical Field

[0001] The present disclosure relates to an optical receiving device, a control method and a distance measuring system. Background Art

[0002] There is a light receiving device that uses an element that generates a signal according to the reception of a photon as a light receiving element. In a distance measuring device including such a light receiving device, ToF (Time of Flight) is used as a measurement method for measuring the distance to a distance measuring target (subject). In ToF, the time until measurement light, which is irradiated from a light source toward the distance measuring target, is reflected by the distance measuring target and returns is measured.

[0003] However, the distance measuring system may perform erroneous measurement in response to incident light other than the measurement light.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-156688 Summary of the invention

[0007] Technical problem to be solved by the invention

[0008] Therefore, the present disclosure provides a light receiving device, a control method, and a distance measuring system capable of suppressing the influence of incident light other than measurement light.

[0009] Solutions for solving technical problems

[0010] In order to solve the above technical problems, according to the present disclosure, a light receiving device is provided, comprising:

[0011] The pixel array unit comprises: a measuring pixel for measuring the distance to a target object; and a plurality of paired phase difference pixels for performing pupil division on incident light from the target object to detect a phase difference;

[0012] a first distance measuring unit that generates a first distance value to the target object based on information about a difference between a timing at which the measuring pixel receives a photon and a predetermined time point; and

[0013] The second distance measuring unit generates a second distance value to the target object based on information corresponding to the number of photons respectively incident on the plurality of phase difference pixels.

[0014] The measuring pixel may include a photoelectric conversion unit that performs photoelectric conversion based on incident photons.

[0015] The phase difference pixel includes a photoelectric conversion unit that performs photoelectric conversion according to incident photons.

[0016] Alternatively, the photoelectric conversion unit is a single photon avalanche photodiode (SPAD).

[0017] The light receiving device may further include a control unit configured to set a driving period of the measurement pixel according to a second distance based on output signals of the plurality of phase difference pixels.

[0018] Alternatively, the first distance measuring unit generates the first distance value based on a value of a histogram, the histogram having occurrence frequency information of a difference value between a timing at which the measurement pixel receives a photon and a predetermined time point,

[0019] The second distance measuring unit generates the second distance value based on a phase difference using a signal value corresponding to the number of photons incident on each of the plurality of phase difference pixels.

[0020] Alternatively, the measuring pixel may include a plurality of the photoelectric conversion units.

[0021] The measuring pixel and the plurality of phase difference pixels forming pairs are arranged as a unit,

[0022] The light receiving device further includes a first conversion section corresponding to the unit,

[0023] The first conversion unit generates a reception signal having information on a difference between a timing at which the measurement pixel receives a photon and a predetermined time point.

[0024] Alternatively, the light receiving device further includes a first infrared light pulse laser,

[0025] The control unit controls the light amount of laser light irradiated by the first infrared light pulse laser according to the second distance.

[0026] Alternatively, the light receiving device further includes a second infrared light pulse laser.

[0027] The plurality of phase difference pixels generate signals according to laser light irradiated by the second infrared light pulse laser.

[0028] The pixel array section and the first conversion section may be stacked, and the first conversion section corresponding to the cell may be disposed directly below the cell.

[0029] Alternatively, the optical receiving device further includes a second conversion unit corresponding to the unit,

[0030] The second conversion unit generates a second reception signal having information corresponding to the number of photons incident on each of the plurality of phase difference pixels.

[0031] The second conversion unit may generate a third reception signal and generate the second reception signal according to the number of the third reception signals, wherein the third reception signal has information on a difference between a timing at which the phase difference pixel receives a photon and a predetermined time point.

[0032] The measuring pixel and the plurality of phase difference pixels forming a pair may generate signals based on photons received in the same time period.

[0033] The measurement pixel and the plurality of phase difference pixels forming pairs may be arranged on different semiconductor device chips.

[0034] Alternatively, the light receiving device further includes a first infrared light pulse laser,

[0035] The first chip on which the measuring pixels are arranged, the second chip on which the plurality of phase difference pixels in pairs are arranged, and the first infrared light pulse laser are arranged in an L shape.

[0036] Alternatively, the optical receiving device includes:

[0037] a pixel array portion, which is configured by a plurality of pixels; and

[0038] a control unit, controlling the pixel array unit,

[0039] The pixel array section comprises:

[0040] A plurality of phase difference pixels in pairs, performing pupil division on incident light from a target object to detect a phase difference; and

[0041] Determine pixels for determining the distance to the target object,

[0042] The control unit sets a driving period of the measurement pixel according to a second distance based on output signals of the plurality of phase difference pixels.

[0043] Alternatively, the phase difference pixel may include a photoelectric conversion unit that receives visible light and performs photoelectric conversion.

[0044] The measurement pixel includes a photoelectric conversion unit that performs photoelectric conversion based on incident photons.

[0045] The photoelectric conversion unit may further include a reflection prevention unit on the incident side.

[0046] The photoelectric conversion unit may include an on-chip lens formed of a high refractive index material on the incident side.

[0047] Alternatively, the photoelectric conversion unit may include:

[0048] On-chip lens; and

[0049] The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion.

[0050] The diffusion layer is disposed according to the position of the principal optical axis of the on-chip lens.

[0051] Alternatively, in the phase difference pixel,

[0052] The photoelectric conversion unit has:

[0053] The aperture of the incident area; and

[0054] The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion.

[0055] The diffusion layer is positioned according to the position of the aperture.

[0056] Alternatively, two phase-difference pixels in the plurality of phase-difference pixels in a pair,

[0057] An elliptical on-chip lens is provided in the photoelectric conversion parts of the two phase difference pixels.

[0058] Alternatively, the optical receiving device may include:

[0059] The circular on-chip lens is disposed in the photoelectric conversion sections of two phase difference pixels among the plurality of phase difference pixels forming a pair and in the photoelectric conversion sections of the two measurement pixels.

[0060] Alternatively, the photoelectric conversion unit of the phase difference pixel receives light via a filter that transmits visible light.

[0061] The photoelectric conversion section of the measurement pixel receives light via a filter that transmits infrared light.

[0062] In order to solve the above technical problems, according to the present disclosure, a control method is provided, which is a control method of a pixel array unit, wherein the pixel array unit has:

[0063] A plurality of phase difference pixels in pairs, performing pupil division on incident light from a target object to detect a phase difference; and

[0064] Determine pixels for determining the distance to the target object,

[0065] In the control method,

[0066] A driving period of the measurement pixel is set according to a second distance based on output signals of the plurality of phase difference pixels.

[0067] In order to solve the above technical problems, according to the present disclosure, a ranging system is provided, which includes:

[0068] a light receiving device; and

[0069] A lens that focuses incident light from the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a block diagram schematically showing a schematic configuration example of a ranging system.

[0071] Figure 2 is a diagram showing a configuration example of a plurality of units arranged in a pixel array section.

[0072] Figure 3 is a circuit diagram showing a configuration example of a measuring pixel.

[0073] Figure 4 It is shown that it is placed in Figure 2 The left side is a circuit diagram of a configuration example of a phase difference pixel.

[0074] Figure 5 It is shown that it is placed in Figure 2 The right side is a circuit diagram of a configuration example of a phase difference pixel.

[0075] Figure 6 is a diagram showing a circuit configuration example of a reference pixel unit.

[0076] Figure 7 FIG. 1 is a schematic cross-sectional view of a SPAD formed in a pixel array portion.

[0077] Figure 8 : is a diagram showing an arrangement example of the TDC 1 and the counter arranged on the substrate.

[0078] Fig. 9 is a block diagram showing a configuration example of a control section.

[0079] Fig.10 1 is a diagram schematically showing a state after pulse light of a dot pattern arrives as return light.

[0080] Fig.11 is a diagram schematically showing a processing example of the TDC of the unit.

[0081] Fig.12 is a diagram schematically showing a visible image and a ranging image.

[0082] Fig.13 It is a diagram for explaining phase difference detection.

[0083] Fig.14 1 is a diagram showing the positions of phase difference pixels and count values ​​on a single lateral axis of a pixel array unit.

[0084] Fig.15 is a diagram schematically showing an example of disturbance light.

[0085] Fig.16 It is shown Fig.15 FIG. 4 is a diagram showing an example of a histogram of reference pixels and a histogram of measurement pixels when not attacked.

[0086] Fig.17 It is shown Fig.15 A diagram showing a histogram of reference pixels and a histogram of measured pixels during an attack.

[0087] Fig.18 1 is a diagram showing the positions of phase difference pixels and count values ​​on a single horizontal axis of the pixel array unit when irradiated with disturbance light.

[0088] Fig.19 It is a timing chart showing a control example of the integrated control unit.

[0089] Fig. 20 It is a flowchart which shows the control example of the integrated control part.

[0090] Fig.21A This is a schematic diagram of a simplified cross section of a SPAD.

[0091] Fig. 21B is a schematic diagram showing a cross section of a SPAD in which a reflection preventing portion is arranged.

[0092] Fig. 22 This is a schematic diagram showing a case where the position of the diffusion layer is changed according to the position of either the on-chip lens or the aperture.

[0093] Fig.23 is a schematic diagram showing a configuration example of a pixel array section.

[0094] Fig.24 1 is a schematic diagram showing a configuration example of a pixel array section according to Modification 4 of the first embodiment.

[0095] Fig.25 This is a diagram schematically showing an example of changing the light amount according to the distance value obtained by the phase difference method.

[0096] Fig.26 This is a flowchart showing a control example of the integrated control unit according to the second embodiment.

[0097] Fig. 27 : is a block diagram showing a configuration example of a control section according to the third embodiment.

[0098] Fig.28 It is a diagram schematically showing a processing example of a unit according to the third embodiment.

[0099] Fig.29 3 is a diagram showing a histogram corresponding to each output of the phase difference pixel generated by the histogram generating unit.

[0100] Fig.30 It is a diagram for explaining phase difference detection according to the fourth embodiment.

[0101] Fig.31 : is a diagram showing the relationship between the focal position and the output value of a pair of phase difference pixels.

[0102] Fig.32 : is a diagram showing the relationship between the focal position and the output value of the phase difference pixel when it is irradiated with disturbance light.

[0103] Fig.33 It is a timing chart showing a control example of the integrated control unit according to the fourth embodiment.

[0104] Fig.34 This is a flowchart showing a control example of the integrated control unit according to the fourth embodiment.

[0105] Fig.35 This is a flowchart showing a control example of the integrated control unit according to the fifth embodiment.

[0106] Fig.36 : is a block diagram schematically showing a schematic configuration example of a distance measurement system according to the sixth embodiment.

[0107] Fig.37A : is a block diagram schematically showing a schematic configuration example of a distance measurement system according to the seventh embodiment.

[0108] Fig.37B is a block diagram schematically showing a schematic configuration example of a distance measurement system related to a comparative example.

[0109] Fig.38 1 is a diagram showing an example of arrangement of a plurality of phase difference pixels and a plurality of measurement pixels arranged in a pixel array section.

[0110] Fig.39 This is a diagram showing an example of an inverted L-shaped structure in which an infrared light pulse laser is brought closer to one chip.

[0111] Fig.40 It is a diagram showing an example of an oblique L-shaped structure.

[0112] Fig.41 It is a diagram showing an example of an L-shaped structure.

[0113] Fig.42 FIG. 1 is a schematic cross-sectional view of a SPAD formed in a pixel array portion.

[0114] Fig.43 It is a diagram showing an example of arrangement of phase difference pixels and measurement pixels in a plan view.

[0115] Fig.44 It is a simplified cross-sectional schematic diagram of a phase difference pixel.

[0116] Fig.45 FIG. 1 is a diagram showing an example of arrangement of phase difference pixels and measurement pixels formed in a pixel array unit in a plan view.

[0117] Fig.46 is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0118] Fig.47 It is an explanatory diagram showing an example of the installation positions of the vehicle exterior information detection unit and the imaging unit. DETAILED DESCRIPTION

[0119] The following description will focus on the main configuration parts of the optical receiving device, the control method and the ranging system, but there may be configuration parts and functions not shown or described in the drawings in the optical receiving device, the control method and the ranging system. The following description does not exclude configuration parts and functions not shown or described in the drawings.

[0120] (First Embodiment)

[0121] <Configuration example of distance measurement system>

[0122] Figure 1 1 is a block diagram schematically showing a schematic configuration example of a distance measurement system 1 to which the present technology is applied. Figure 1 As shown, the distance measuring system 1 according to the first embodiment includes a light receiving device 10, a control device 22, a display device 24, an operation device 26, a lens 30 on the emission side, and a lens 40 on the incident side. Figure 1 Target object 50 to be measured is further shown in FIG.

[0123] The light receiving device 10 has a substrate 11, a pixel array section 12, a circuit section 13, and an infrared light pulse laser 14. The control section 20 is configured to include, for example, a CPU, and controls the light receiving device 10.

[0124] The control device 22 is a device that controls the light receiving device 10 according to the operation of the operation device 26. The display device 24 is, for example, a monitor, and displays the image impression generated by the light receiving device 10. In addition, the operation device 26 is configured to include a keyboard and a mouse, etc., and inputs the operator's operation signal to the control device 22.

[0125] The distance measuring system 1 irradiates the target object 50 with light emitted by the infrared pulse laser 14 via the lens 30. The system measures the distance to the target object 50 using light obtained by causing the light reflected by the target object 50 to enter the pixel array unit 12 via the lens 40. Incidentally, the details of the control unit 20 will be described later.

[0126] The pixel array unit 12 is mounted on the substrate 11 and includes a plurality of measurement pixels 110i, a plurality of phase difference pixels 110z, and a plurality of reference pixels 110r. For example, a single photon avalanche photodiode (SPAD) is used, and each pixel of the pixel array unit 12 is formed as a semiconductor element. The circuit unit 13 is a circuit that performs control and signal processing of the pixel array unit 12.

[0127] The measuring pixel 110i is a pixel used for ToF (Time of Flight), in which the time from when the measuring light irradiated toward the ranging target is reflected by the ranging target and returns is measured. On the other hand, the phase difference pixel 110z is a phase difference pixel that performs pupil division on the incident light from the target 50 to detect the image plane phase difference. The phase difference pixel 110z is a pixel used for PDAF (Phase Detection Auto Focus). In this embodiment, the phase difference pixel 110z formed using SPAD is sometimes referred to as a PDAF pixel. In addition, the reference pixel 110r is used to measure the emission timing of the infrared light pulse laser 14. In addition, in this embodiment, the measurement using ToF (Time of Flight) is sometimes referred to as the ToF method, and the measurement using the phase difference pixel 110z is sometimes referred to as the phase difference method.

[0128] The infrared light pulse laser 14 is mounted on the substrate 11. For example, a VCSEL (Vertical Cavity Surface Emitting LASER) light source is used to form the infrared light pulse laser 14. In addition, by placing a diffraction optical element 14a (DOE) on the irradiation side of the infrared light pulse laser 14, a dot pattern of 100 points of focused light arranged in a matrix can be irradiated onto the target 50.

[0129] The wavelength of the measuring light emitted by the infrared pulse laser 14 is, for example, 850 nm. The measuring light is emitted in synchronization with the emission control signal input from the control unit 20 .

[0130] like Figure 1 As shown, in the ToF method, the reference pixel 110r receives the pulse light emitted by the infrared light pulse laser 14 in the pixel array unit 12. For example, the pulse light emitted by the infrared light pulse laser 14 is guided to the reference pixel 110r via the light guide component. On the other hand, the pulse light emitted via the lens 30 is incident on the plurality of measurement pixels 110i at different timings via the lens 40.

[0131] In the plurality of phase difference pixels 110z, the pulse light emitted through the lens 30 is received respectively through the lens 40. As described later, in the plurality of phase difference pixels 110z, the pulse light is received through the paired apertures in the left and right or in the top and bottom. Thus, in the paired phase difference pixels 110z, the image plane phase difference corresponding to the distance to the target object 50 can be detected. In the paired phase difference pixels 110z, without using the plurality of reference pixels 110r, the distance to the target object 50 can be measured at a timing different from the measurement in the ToF method or at a time overlapping with the measurement in the ToF method.

[0132] <Unit Configuration Example>

[0133] Figure 2 1 is a diagram showing an example of the configuration of a plurality of units 11u disposed in the pixel array section 12. The pixel array section 12 is provided with a plurality of units 11u in a matrix. In addition, a reference pixel unit 166u in which a reference pixel 110r is disposed is disposed at the end of the pixel array section 12. The unit 11u has a measurement pixel 110i configured by SPADs 331 to 334 and a phase difference pixel 110z configured by SPADs 335 and 336, respectively. In addition, as described later, a pixel having an aperture WL in the right half of the incident area of ​​the SPAD 335 is set as a phase difference pixel 110zL, and a pixel having an aperture WR in the left half of the incident area of ​​the SPAD 336 is set as a phase difference pixel 110zR.

[0134] Figure 3 1 is a circuit diagram showing a configuration example of a measurement pixel 110i. The measurement pixel 110i includes a quenching / detection circuit 310, selection transistors 321 to 324, and SPADs 331 to 334. The quenching / detection circuit 310 includes a resistor 311 and an inverter 312. Figure 3In the wiring example of , for example, pMOS (p-channel Metal Oxide Semiconductor) transistors are used as the selection transistors 321 to 324. In addition, logic signals 309-1 to 309-4 are input from the control unit 20 to the gates of the corresponding selection transistors 321 to 324.

[0135] SPADs (Single Photon Avalanche photodiode) 331 to 334 perform photoelectric conversion on photons to perform avalanche multiplication and generate current. A negative bias voltage VRLD is applied to the anodes of SPADs 331 to 334. The cathode of SPAD 331 is connected to the drain of the selection transistor 321, and the cathode of SPAD 332 is connected to the drain of the selection transistor 322. In addition, the cathode of SPAD 333 is connected to the drain of the selection transistor 323, and the cathode of SPAD 334 is connected to the drain of the selection transistor 324.

[0136] The sources of the selection transistors 321 to 324 are connected to the common node 319. In addition, the selection signal XSEL1 from the control unit 20 is input to the gates of the selection transistors 321 to 324 via the selection lines 309-1 to 309-4. By making the timing of the selection signal XSEL1 different, the SPADs 331 to 334 can also be driven as separate pixels. When the selection signal XSEL1 is at a high level and the selection transistors 321 to 324 are in the on state, the potentials VK_1 to VK_4 of the SPADs 331 to 334 are conducted to the common node 319.

[0137] The resistor 311 is inserted between the node of the predetermined power supply voltage VDD and the common node 319. The inverter 312 generates a pulse signal TOUT based on the potential of the common node 319 and supplies it to the TDC (Time-to-Digital Converter) 170. In addition, the resistor 311 reduces the potential of the common node 319 that exceeds the predetermined potential due to avalanche multiplication, and suppresses avalanche multiplication. Incidentally, the TDC 170 involved in this embodiment corresponds to the first conversion unit.

[0138] TDC170 converts the light reception timing into a digital value based on the pulse signal TOUT. For example, the circuit section 13 has a clock circuit. Thus, TDC170 uses the information of the clock circuit to convert the time difference of the input time point of the pulse signal TOUT into a digital value according to the reference time point, taking the measurement start time as the reference time point. The TDC170 supplies the digital value to the control section 20. In addition, TDC170 can be configured for each unit 11u. By the way, in the following description, the circuit configuration other than SPAD331 to SPAD334 is configured in the circuit section 13.

[0139] Figure 4 It is shown that the Figure 2 1 is a circuit diagram of a configuration example of a phase difference pixel 110zL on the left side of FIG. The phase difference pixel 110zL includes a quenching / detection circuit 310, a selection transistor 325, and a SPAD 335. In addition, a logic signal 309-5 is input to the gate of the corresponding selection transistor 325.

[0140] A negative bias voltage VRLD is applied to the anode of SPAD335. The cathode of SPAD335 is connected to the drain of the selection transistor 325. The source of the selection transistor 325 is connected to the node 319. In addition, the selection signal XSEL2 from the control unit 20 is input to the gate of the selection transistor 325 via the selection line 309-5. The inverter 312 generates a pulse signal COUT based on the potential of the node 319 and supplies it to the counter 172. Incidentally, the counter 172 involved in this embodiment corresponds to the second conversion unit.

[0141] The counter 172 counts the number of photons incident on the SPAD 335 based on the pulse signal COUT. The counter 172 counts the number of pulses of the pulse signal COUT as a value corresponding to the number of photons, and supplies the count value to the control unit 20. In this way, when the selection signal XSEL2 is at a high level and the selection transistor 325 is in an on state, the potential of the SPAD 335 is conducted to the node 319. In addition, the counter 172 can be configured for each unit 11u.

[0142] Figure 5 To show the placement Figure 2 1 is a circuit diagram of a configuration example of a phase difference pixel 110zR on the right side of . The phase difference pixel 110zR has the same configuration as the phase difference pixel 110zL. That is, the phase difference pixel 110zR includes a quenching / detection circuit 310, a selection transistor 326, and a SPAD 336. In addition, a logic signal 309-6 is input to the gate of the corresponding selection transistor 326.

[0143] A negative bias voltage VRLD is applied to the anode of SPAD336. The cathode of SPAD336 is connected to the drain of the selection transistor 326. The source of the selection transistor 326 is connected to the node 319. In addition, the selection signal XSEL3 from the control unit 20 is input to the gate of the selection transistor 326 via the selection line 309-6. The inverter 312 generates a pulse signal COUT based on the potential of the node 319 and supplies it to the counter 172. The counter 172 counts the number of photons incident on the SPAD336 based on the pulse signal COUT. The counter 172 counts the number of pulses of the pulse signal COUT as a value corresponding to the number of photons, and supplies the count value to the control unit 20. In this way, when the selection signal XSEL3 is at a high level and the selection transistor 326 is in the on state, the potential of the SPAD336 is conducted to the node 319.

[0144] Figure 6 1 is a diagram showing an example of a circuit configuration of a reference pixel unit 166u. The reference pixel 110r is also configured in the same manner as the measurement pixel 110i. That is, the reference pixel 110r includes a quenching / detection circuit 310, selection transistors 327 to 330, and SPADs 337 to 340. The quenching / detection circuit 310 includes a resistor 311 and an inverter 312. In addition, logic signals 309-7 to 309-10 are input to the gates of the corresponding selection transistors 327 to 330.

[0145] The sources of the selection transistors 327 to 330 are connected to the common node 319. The selection signal XSEL0 from the control unit 20 is input to the gates of the selection transistors 327 to 330 via the selection lines 309-7 to 309-10. When the selection signal XSEL0 is at a high level and the selection transistors 321 to 324 are in an on state, the potentials VK_1 to VK_4 of the SPADs 337 to 340 are conducted to the common node 319.

[0146] Figure 7 331, 332, 335, 336 are formed in the pixel array unit 12. The SPADs 331, 332, 335, 336 correspond to the phase difference pixel 110zL, the phase difference pixel 110zR, and two of the four pixels for configuring the measurement pixel 110i (see Figure 2 ).

[0147] SPAD331 and SPAD332 are of the same configuration. In addition, SPAD333 and 334 are also of the same configuration as SPAD331 and 332. That is, the four pixels used to configure the measurement pixel 110i are of the same configuration. The phase difference pixel 110zL and the phase difference pixel 110zR are different from the measurement pixel 110ia in that they have apertures WL and WR, respectively. The apertures WL and WR are configured by light-shielding components, and pupil division is performed on the incident light L1 and L2 from the target object 50, respectively. By the way, SPAD337 to 340 are also of the same configuration as SPAD331 and 332.

[0148] like Figure 7 As shown in FIG. 1 , the SPAD 331 includes an on-chip lens 1110, an N-well 1112, a diffusion layer 1114, a metal wiring 1116, a metal pad 1118, and an inter-pixel separation portion 1120. Incidentally, the detailed description of the wiring layer is omitted. Incidentally, the configuration of the SPADs 331 to 340 can adopt a general configuration and is not limited to Figure 7 Configuration.

[0149] The on-chip lens 1110 focuses the light incident through the lens 40 into the N-well 1112. The N-well 1112 is formed by controlling the impurity concentration of the sensor substrate to be n-type, and the N-well 1112 forms an electric field that transports the electrons generated by the photoelectric conversion in the SPAD to the avalanche multiplication region. The on-chip lens 1110 can be configured, for example, from a high-refractive material. For example, amorphous silicon, SiN, etc. can be used as a high-refractive material. When configured with a high-refractive material, the sensitivity of infrared light is improved, the counting rate of the phase difference pixel 110zL, the phase difference pixel 110zR, and the measurement pixel 110i is improved, and the focusing efficiency of the phase difference pixel 110zL and the phase difference pixel 110zR is improved.

[0150] The diffusion layer 1114 is configured by a P-type diffusion layer, an N-type diffusion layer, a hole accumulation layer, a pinning layer, and a high-concentration P-type diffusion layer. For example, in this SPAD, an avalanche multiplication region is formed by a depletion layer formed in a region where the P-type diffusion layer and the N-type diffusion layer are connected.

[0151] The metal line 1116 is formed wider than the diffusion layer 1114, for example, so as to cover the avalanche multiplication region. The metal pad 1118 is used to electrically and mechanically connect to the metal pad formed in the logic side wiring layer by forming the respective metals (Cu). The pixel separation unit 1120 insulates and separates each SPAD by a double-layer structure formed by a metal film and an insulating film formed between adjacent SPADs.

[0152] <Example of placement on a substrate>

[0153] Figure 8 1 is a diagram showing an example of placement of the TDC 170 and the counter 172 placed on the substrate 11. The TDC 170 has a TDC 170a placed at the left end of the pixel array section 12 and a TDC 170b placed at the right end of the pixel array section 12. Similarly, the counter 172 has a counter 172a placed at the left end of the pixel array section 12 and a counter 172b placed at the right end of the pixel array section 12. In this way, the circuit section 13 has two TDCs 170 and two counters 172, for example.

[0154] The TDC 170a is wired to the plurality of cells 11u in the left half of the pixel array section 12, and the TDC 170b is wired to the plurality of cells 11u in the right half of the pixel array section 12. Similarly, the counter 172a is wired to the plurality of cells 11u in the left half of the pixel array section 12, and the counter 172b is wired to the plurality of cells 11u in the right half of the pixel array section 12. Thus, the arrangement can be performed in a manner that reduces the amount of wiring.

[0155] <Configuration example of control unit>

[0156] Fig. 9 2 is a block diagram showing a configuration example of the control unit 20. Fig. 9 As shown, the control unit 20 includes a light emission control unit 200, a drive control unit 202, a first distance measuring unit 204, a second distance measuring unit 206, and a comprehensive control unit 208. In addition, the first distance measuring unit 204 includes a histogram generating unit 204a and a processing unit 204b.

[0157] The light control unit 200 controls the irradiation timing of the pulse light of the dot pattern formed by the diffractive optical element 14a by supplying a light control signal to the infrared light pulse laser 14. In the ToF method, the frequency of the light control signal is, for example, 20 [MHz]. By the way, the frequency of the light control signal is not limited to 20 [MHz], and may also be 5 [MHz] or the like. In addition, if the light control signal is a periodic signal, it is not limited to a rectangular wave. For example, the light control signal may also be a sine wave. On the other hand, in the measurement using a plurality of phase difference pixels 110z, for example, before performing the measurement of the ToF method, pulse light is irradiated several times at, for example, 20 [MHz]. By the way, the irradiation cycle and period are examples and are not limited thereto.

[0158] Fig.10 Schematically shows the state after the pulse light of the dot pattern reaches the light receiving surface of the pixel array unit 12 as the return light from the target 50. Fig.10 As shown, the pulse light of the dot pattern is received as focused light 11s in a manner dispersed in each unit 11u, for example.

[0159] The drive control unit 202 generates the selection transistors 321 to 330 (see Figures 3 to 5 ) selection signals XSEL0 to 3, logic signals 309-1 to 309-10, etc., and controls the driving of each unit 11u and unit 116u.

[0160] The first distance measuring unit 204 includes a histogram generating unit 204a and a processing unit 204b. The histogram generating unit 204a generates a histogram based on the digital value obtained by the TDC 170. The processing unit 204b performs various processes based on the histogram generated by the histogram generating unit 204a. For example, the processing unit 204b can perform FIR (Finite Impulse Response) filtering, echo determination, depth value (distance value) calculation, peak detection, etc. The distance image (depth image) generated by the processing unit 204b is output via an interface.

[0161] Fig.11 11 is a diagram schematically showing a processing example of the TDC 170 of the unit 11u. As described above, the SPADs 331 to 334 are connected to the TDC 170, and the SPADs 335 and 336 are respectively connected to the counter 172. The horizontal axis of the TDC histogram represents time, and the vertical axis represents frequency.

[0162] The example in which photons ph1 to ph7 are incident on SPADs 331 to 334 in time series is schematically shown. Unit 11u generates pulse signals TOUT corresponding to these photons ph1 to ph7. Then, in TDC 170, a digital value proportional to the difference between the input timing of the pulse signals TOUT generated in time series due to photons ph1 to ph7 and the reference time t0 is generated and supplied to the histogram generation unit 204a.

[0163] Every time a digital value is input, the histogram generator 204a adds, for example, 1 to the frequency of the time interval corresponding to the digital value. By repeatedly irradiating the target 50 with pulse light, the frequency of the time interval corresponding to the digital value corresponding to the distance to the target 50 increases.

[0164] Similarly, photons p are introduced from the infrared light pulse laser 14 to the SPADs 337 to 340 of the unit 116u in a time series. The unit 116u generates a pulse signal TOUT corresponding to the photon. Then, in the TDC 170, a digital value proportional to the difference between the input timing of the pulse signal TOUT generated in a time series due to the photon and the reference time t0 is generated, and supplied to the histogram generation unit 204a. By repeatedly irradiating the pulse light, the frequency of the time interval corresponding to the digital value corresponding to the irradiation timing of the pulse light increases.

[0165] The processing unit 204b generates, for example, the time corresponding to the maximum frequency of the histogram generated by the histogram generating unit 204a for the reference pixel 110r of the cell 166u as time t1. The time t1 is a value substantially equal to the irradiation start time of the pulse light.

[0166] On the other hand, the processing unit 204b generates the time corresponding to the maximum frequency of the histogram generated by the histogram generating unit 204a for the plurality of measurement pixels 110i as time t2n, for example. n represents each unit 11u. Thus, the processing unit 204b generates the distance Dn to the target object 50 detected by each unit 11u as (t2n-t1)×speed of light / 2.

[0167] Fig.12 110a and the distance measurement image 110b are schematically shown. Each concentration point Cp of the distance measurement image 110b has a concentration value corresponding to the distance Dn calculated by the processing unit 204b. In this way, the processing unit 204b can generate a two-dimensional distribution of distance values ​​based on the pulse signal TOUT measured by each unit 11u as a distance measurement image (depth image) 110b. For example, the distance measurement image 110b is displayed on the display device 24 via the control device 22.

[0168] Fig.13 : is a diagram for explaining the phase difference detection involved in the embodiment of the present disclosure. A in the figure is a diagram showing the relationship between the surface position of the target object 50, the lens 40 and the pixel array unit 12 and the optical path of the incident light. In the figure, 301 and 302 represent the light passing through the left and right sides of the lens 40, respectively. For convenience, for the light 301 and 302, only such light passing through the end of the lens 40 is recorded. The left, center and right figures of A in the figure respectively represent the situation of being in the pixel array unit 12 (imaging surface) (focus state), the situation of being on the side opposite to the pixel array unit 12 (imaging surface) (so-called rear focus state) and the situation of being on one side of the pixel array unit 12 (so-called front focus state).

[0169] In contrast, in the case of the center and right images of B in the figure, the optical images 303 and 304 are displaced shapes. The dislocation of the image indicates a phase difference. In the case of the rear focus state of the center image of B in the figure, the optical images 303 and 304 are images displaced to the left and right, respectively. In the case of the front focus state of the right image of B in the figure, the optical images 303 and 304 are images displaced in opposite directions.

[0170] Fig.141 is a diagram showing the positions and count values ​​of the phase difference pixels 110zL and zR on a single horizontal axis of the pixel array unit 12. The horizontal axis corresponds to the positions of the phase difference pixel 110zL and the phase difference pixel 110zR. The vertical axis is the count value of the counter 172. That is, the signal value L20 corresponds to the number of photons of the pulse signal TOUT based on the phase difference pixel 110zL. On the other hand, the signal value R20 corresponds to the number of photons of the pulse signal TOUT based on the phase difference pixel 110zR.

[0171] When the signal value L20 or the signal value R20 is shifted in the horizontal direction, the phase difference corresponds to the shift amount at which the sum of the difference values ​​between the signal value L20 and the signal value R20 is minimized.

[0172] Again, if Fig. 9 As shown, the second distance measuring unit 206 calculates the displacement based on the pulse signal TOUT of each unit 11u. Then, the second distance measuring unit 206 calculates the distance to the target object 50 based on the displacement. Since the displacement corresponds to the distance to the target object 50, the accuracy is reduced compared to the ToF method, but the distance can be measured. In this way, in this embodiment, by using SPAD335, 336, the distance to the target object 50 can be measured using the light amount at the photon level.

[0173] Since the second distance measuring unit 206 calculates the displacement amount that minimizes the sum of the difference values ​​between the signal value L20 and the signal value R20, the influence of ambient light including noise and intentional disturbance light is suppressed in the second distance measuring unit 206 compared with the ToF method.

[0174] Fig.15 1 is a diagram schematically showing an example of interference light. FIG. A schematically shows a state without attack, and FIG. B schematically shows an attack by intentional interference light. The attack circuit 1000 emits pulse light of the same wavelength band as the pulse light to the target 50.

[0175] Fig.16 It is shown Fig.15 10 is a diagram showing examples of a histogram of a reference pixel 110r and a histogram of a measurement pixel 110i when not attacked. FIG. A shows a histogram generated by the histogram generating unit 204a based on the output signal of the reference pixel 110r, and FIG. B shows a histogram generated by the histogram generating unit 204a based on the output signal of the measurement pixel 110i. The time difference between the peak time point of the histogram of the reference pixel and the peak time point of the histogram of the measurement pixel corresponds to the actual distance R to the target object 50.

[0176] Fig.17 It is shown Fig.15110r and the histogram of the measuring pixel 110i during the attack. FIG. A shows the histogram generated by the histogram generating unit 204a based on the output signal of the reference pixel 110r, and FIG. B shows the histogram generated by the histogram generating unit 204a based on the output signal of the measuring pixel 110i. The histogram of the measuring pixel includes the pulse signal DOUT caused by the interference light, and the peak of the histogram of the measuring pixel is misaligned. The time difference between the peak time point of the histogram of the reference pixel and the peak time point of the histogram of the measuring pixel corresponds to the distance Rf to the pseudo target 50f.

[0177] Thus, in the ToF method, since the incident timing of photons is converted into a digital value, an error signal is generated according to the incident timing of the pulse light of the attack circuit 1000, thereby causing, for example, the position of the target object 50 to be erroneously measured as the position of the target object 50f. As can be seen from this, the measurement accuracy in the ToF method is higher than the measurement accuracy based on the phase difference method, but it is easily affected by the attack light or ambient light.

[0178] Fig.18 1 is a diagram showing the positions and count values ​​of the phase difference pixels 110zL and zR on the horizontal single axis of the pixel array unit 12 when irradiated with disturbing light. The horizontal axis corresponds to the positions of the phase difference pixel 110zL and the phase difference pixel 110zR. The vertical axis is the count value of the counter 172. That is, the signal value L20a corresponds to the number of photons of the pulse signal TOUT based on the phase difference pixel 110zL. On the other hand, the signal value R20a corresponds to the number of photons of the pulse signal TOUT based on the phase difference pixel 110zR. When the signal value L20a or the signal value R20a is displaced in the horizontal direction, the phase difference corresponds to the displacement amount at which the sum of the differential values ​​of the signal value L20a and the signal value R20a is minimized.

[0179] In this way, even if the target object 50 is irradiated with interfering light, only the count value based on the photon caused by the interfering light is increased as an offset in the count values ​​of the phase difference pixel 110zL and the phase difference pixel 110zR. Therefore, even if the target object 50 is irradiated with interfering light, the influence on the displacement amount is suppressed. In this way, in the phase difference method using the phase difference pixel 110zL and the phase difference pixel 110zR, even if the pulse light of the attack circuit 1000 is irradiated to the target object 50, only the signal value L20a and the signal value R20a increase to the same extent, so the influence on the displacement amount is suppressed, and the influence of the pulse light of the attack circuit 1000 and the ambient light is suppressed.

[0180] <Control example of control equipment>

[0181] Fig.192 is a timing chart showing an example of control by the integrated control unit 208. The horizontal axis represents time. The vertical axis of FIG. A shows the light control signal generated by the light control unit 200 in accordance with the control of the integrated control unit 208, and the selection signal XSEL2, the selection signal XSEL3, the selection signal XSEL0, and the selection signal XSEL1 generated by the drive control unit 202. The high level of these signals indicates the drive state.

[0182] Fig.19 FIG. B schematically shows photons incident on the pixel array unit 12. That is, it schematically shows photons generated by the attacking light or the ambient light and photons generated by the secondary measurement light.

[0183] Fig. 20 2 is a flowchart showing a control example of the integrated control unit 208. Fig.19 A control example of the integrated control unit 208 will be described.

[0184] like Fig.19 As shown, the integrated control unit 208 makes the light emission control signal a high-level signal for the first time to measure the phase difference signal, and makes the infrared light pulse laser 14 emit a measurement pulse light (step S100).

[0185] Next, the integrated control unit 208 sets the selection signal XSEL2 and the selection signal XSEL3 for driving the phase difference pixel 110zL and the phase difference pixel 110zR to high level according to the timing of issuing the first measurement pulse, so as to drive the phase difference pixel 110zL and the phase difference pixel 110zR to measure the phase difference signal (step S102). The range of the first measurement period in which the selection signal XSEL2 and the selection signal XSEL3 are set to high level can be set according to the predetermined measurement distance range to the target object 50.

[0186] Next, the second distance measuring unit 206 calculates the displacement based on the pulse signal COUT of the phase difference pixel 110zL, zR of each unit 11u, and generates each first distance to the target object 50 (step S104). As described above, the distance value based on the phase difference method is generated in a state where the influence of attack light, ambient light, etc. is suppressed.

[0187] Next, the integrated control unit 208 controls the period during which the second light emission control signal is at a high level and the period during which the selection signal XSEL0 is at a high level based on the first distance generated by the second distance measuring unit 206. As a result, the second measurement pulse emits light (step S106).

[0188] Then, the integrated control unit 208 sets the selection signal XSEL0 for driving the reference pixel 110r to a high level according to the timing of the second measurement pulse emission, so as to measure the signal of the reference light (step S108). Similarly, the integrated control unit 208 sets the selection signal XSEL1 for driving the measurement pixel 110i to a high level according to the timing of the second measurement pulse emission, so as to measure the signal of the measurement light (step S110). In this way, the integrated control unit 208 sets the range of the measurement time for the photon to be reflected from the target object 50 and return to the second measurement period corresponding to the first distance value based on the phase difference method.

[0189] Next, while the selection signal XSEL0 is at a high level, whenever a digital value is input, the histogram generation unit 204a of the first distance measurement unit 204 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to the output of the reference pixel 110r. The processing unit 204b generates, for example, a time corresponding to the maximum frequency of the histogram generated by the histogram generation unit 204a for the reference pixel 110r as time t1.

[0190] Similarly, when the selection signal XSEL1 is at a high level, each time a digital value is input, the histogram generation unit 204a of the second distance measurement unit 206 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to each of the plurality of measurement pixels 110i. Then, the processing unit 204b generates, for example, time t2n, the time corresponding to the maximum frequency of the histogram generated by the histogram generation unit 204a for each of the plurality of measurement pixels 110i. Thus, the processing unit 204b generates the distance Dn to the target object 50 detected by each unit 11u as (t2n-t1)×speed of light / 2 (step S112).

[0191] As described above, the integrated control unit 208 sets the second measurement period of the photons reflected and returned from the target object 50 to correspond to the first distance value based on the phase difference method. Fig.19 As shown in FIG. B, photons caused by attack light or ambient light also enter the pixel array unit 12 during the second measurement period in which the measurement pixel 110i is driven. However, the attack light entering during the measurement period is incident at a high density, for example, outside the second measurement period, and the photons are randomly incident due to ambient light.

[0192] On the other hand, regarding the photons of the measurement light incident during the second measurement period, the density of the photons increases with the time corresponding to the position of the target object 50 as the center. Therefore, for the peak of the histogram generated by the histogram generation unit 204a, the influence of the photons caused by the measurement light becomes dominant. In this way, by setting the second measurement period corresponding to the first distance value based on the phase difference method, the reduction in measurement accuracy in the ToF method can be suppressed. It can be seen from this that because the measurement accuracy of the ToF method is higher than that of the phase method in the absence of attacking light, even in the presence of attacking light, a measurement distance value with a higher measurement accuracy than the phase method can be obtained.

[0193] As described above, the distance measurement system 1 involved in this embodiment is configured such that the integrated control unit 208 sets the second measurement period of the photons reflected and returned from the target object 50 to correspond to the first distance value based on the phase difference method. Thus, since the second measurement period is set according to the first distance value at which the influence of the attacking light or the ambient light is suppressed, the influence of the attacking light or the ambient light outside the second measurement period in the ToF method is suppressed, and the reduction in the distance measurement of the ToF method in the presence of the attacking light, etc. is suppressed.

[0194] (Modification 1 of the first embodiment)

[0195] The distance measuring system 1 according to the first modification of the first embodiment is different from the distance measuring system 1 according to the first embodiment in that a reflection preventing portion (moth eye) 1122 is arranged in the SPADs 331, 332, 335, and 336 formed in the pixel array portion 12. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0196] Fig.21A This is a simplified schematic diagram of the SPAD cross section. Fig.21A As shown, visible light is almost absorbed in the N-well 1112. On the other hand, about half of infrared light is reflected. Therefore, in the SPAD according to this embodiment, a reflection prevention portion (moth eye) 1122 is provided.

[0197] Fig. 21B 1 is a schematic diagram showing a cross section of SPADs 331, 332, 335, and 336 in which a reflection prevention portion (moth eye) 1122 is arranged. Fig. 21BAs shown, in SPAD331, 332, 335, 336 involved in the modified example 1 of the first embodiment, a reflection prevention structure formed by fine protrusions, a so-called moth-eye structure, is provided in the surface (plate surface) on the side for light incidence. In addition to preventing reflection, the reflection prevention portion 1122 also extends the effective optical path length by diffraction. In this way, a concave-convex structure portion arranged at a predetermined pitch, i.e., a reflection prevention portion (moth-eye) 1122, is arranged on the surface of the photoelectric conversion element side. By the way, SPAD333, 334 also have the same configuration as SPAD331, 332.

[0198] As described above, according to this embodiment, the antireflection unit (moth eye) 1122 allows the light entering the SPADs 331 , 332 , 335 , and 336 to travel back and forth in the N-well 1112 , thereby improving the sensitivity of the SPADs 331 , 332 , 335 , and 336 .

[0199] (Variation Example 2 of the First Embodiment)

[0200] The distance measuring system 1 according to the second modification of the first embodiment is different from the distance measuring system 1 according to the first embodiment in that, in the SPADs 331, 332, 335, 336 formed in the pixel array section 12, the position of the diffusion layer 1114 is changed according to the position of the on-chip lens 1110 and the apertures WL and WR. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0201] Fig. 22 1 is a schematic diagram showing the cross-section of SPADs 331, 332, 335, and 336 after the position of the diffusion layer 1114 is changed according to the position of the on-chip lens 1110 and the aperture WL and WR. Fig. 22 As shown, in the SPAD 335, the sensitivity can be further improved by arranging the diffusion layer 1114 on the side without the aperture WL. Similarly, in the SPAD 336, the sensitivity can be further improved by arranging the diffusion layer 1114 on the side without the aperture WR.

[0202] The focal position of each SPAD 331, 332 changes according to the placement position due to the relationship between the placement position on the pixel array unit 12 and the optical axis of the lens 40. Therefore, for the SPAD 331, 332 involved in this modified example, the diffusion layer 1114 is placed to match the focal position of the on-chip lens 1110. That is, the diffusion layer 1114 is placed to match the position of the main optical axis of the on-chip lens 1110. This will have a so-called pupil correction effect and can improve the sensitivity of the SPAD 331, 332. By the way, the SPAD 333, 334 is also configured in the same manner as the SPAD 331, 332.

[0203] (Variation Example 3 of the First Embodiment)

[0204] The distance measuring system 1 according to the modification 3 of the first embodiment is different from the distance measuring system 1 according to the first embodiment in that the circuit unit 13 formed in the pixel array unit 12 is stacked in two layers. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0205] Fig.23 is a schematic diagram showing a configuration example of the pixel array section 12. Fig.23 As shown, the pixel array unit 12 is arranged on the first substrate 11a, and the inverter 312, TDC 170a, b and counter 172a, b of each pixel 110r, 110i, 110zR, 110zL are arranged on the second substrate 11b. Thus, the upper surface area of ​​the light receiving device 10 can be reduced, and the light receiving device 10 can be made more compact.

[0206] (Variation Example 4 of the First Embodiment)

[0207] The distance measuring system 1 according to the modification 4 of the first embodiment is different from the distance measuring system 1 according to the first embodiment in that the circuit unit 13 formed in the pixel array unit 12 is stacked in three layers. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0208] Fig.24 1 is a schematic diagram showing a configuration example of the pixel array section 12 according to the modification example 4 of the first embodiment. Fig.24 As shown, the pixel array unit 12 is arranged on the first substrate 11c, the inverter 312 of each pixel 110r, 110i, 110zR, 110zL is arranged on the second substrate 11d, and the TDC 170 and the counter 172 are placed on the third substrate 11e. In addition, the TDC 170 corresponding to each unit 11u of the pixel array unit 12 is placed directly below each unit 11u. Similarly, the counter 172 corresponding to each unit 11u of the pixel array unit 12 is placed directly below each unit 11u. As a result, the wiring distance of the light receiving device 10 can be further shortened, and the upper surface area can be further reduced, which can make the light receiving device 10 more compact.

[0209] (Second Embodiment)

[0210] The distance measuring system 1 according to the second embodiment is different from the distance measuring system 1 according to the first embodiment in that the amount of the measuring light used in the measurement of the ToF method can be further changed based on the first distance value based on the phase difference method generated by the second distance measuring unit 206. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0211] Fig.25 This is a diagram schematically showing an example of changing the light amount according to the distance value based on the phase difference method. Fig.25 Figure A is a close-up example. Fig.25 The B graph is an example of a long distance that is farther than the A graph.

[0212] Fig.26 2 is a flowchart showing a control example of the integrated control unit 208 according to the second embodiment. Fig. 20 The flowchart of the control example of the integrated control unit 208 according to the first embodiment is different in that the light intensity of the infrared light pulse laser 14 is set in step S200 .

[0213] That is, in step S200, the integrated control unit 208 sets the light intensity of the infrared pulse laser 14 based on the first distance value generated by the second distance measuring unit 206 using the phase difference method. The integrated control unit 208 stores, for example, a table that associates distance values ​​with light intensity, and sets the light intensity by referring to the table.

[0214] As described above, according to the present embodiment, the integrated control unit 208 sets the second measurement period of the photons reflected and returned from the target object 50 and the light amount of the infrared light pulse laser 14 to correspond to the first distance value based on the phase difference method. Thus, since the second measurement period is set according to the first distance value where the influence of the attack light or the ambient light is suppressed, and the distance variation of the amount of photons reflected and returned from the target object 50 can be suppressed, the reduction of the measurement accuracy in the ToF method is further suppressed.

[0215] (Third Embodiment)

[0216] The phase difference pixels 110zL and 110zR involved in the distance measurement system 1 of the first embodiment generate count values ​​through the counter 172, whereas the phase difference pixels 110zL and 110zR involved in the distance measurement system 1 of the third embodiment output the output values ​​to the TDC 170 and generate count values ​​using the TDC 170, which is different from the distance measurement system 1 of the first embodiment. The differences from the distance measurement system 1 of the first embodiment are described below.

[0217] Fig. 27 2 is a block diagram showing a configuration example of the control unit 20 according to the third embodiment. Fig. 27 As shown, the control unit 20 according to the third embodiment further includes a third distance measuring unit 210. The third distance measuring unit 210 includes a histogram generating unit 210a and a processing unit 210b. Details of the processing of the third distance measuring unit 210 will be described later.

[0218] Fig.28 3 is a diagram schematically showing an example of processing by the unit 11u according to the third embodiment. Figure 3 ) is connected to TDC 170, and SPADs 335 and 336 involved in the third embodiment are respectively connected to TDC 170, which is different from the SPADs 335 and 336 involved in the first embodiment.

[0219] An example is schematically shown in which photons ph8 to ph11 are incident on SPAD335 and 336 respectively in a time series. The phase difference pixel 110zL with SPAD335 and the phase difference pixel 110zR with SPAD336 of the unit 11u generate pulse signals TOUT corresponding to these photons ph8 to ph10 respectively. Then, in the TDC170 corresponding to the SPAD335, a digital value proportional to the difference between the input timing of the pulse signal TOUT caused by photons ph8 to ph9 and generated in a time series and the reference time t0 is generated, and supplied to the histogram generation unit 210a. Similarly, in the TDC170 corresponding to the PAD336, a digital value proportional to the difference between the input timing of the pulse signal TOUT caused by photons ph10 to ph11 and generated in a time series and the reference time t0 is generated, and supplied to the histogram generation unit 210a.

[0220] Every time a digital value is input, the histogram generator 210a adds, for example, 1 to the frequency of the time interval corresponding to the digital value. By repeatedly irradiating the target 50 with pulse light, the frequency of the time interval corresponding to the digital value corresponding to the distance to the target 50 increases.

[0221] Fig.29 Graphs A and B show histograms corresponding to the outputs of the phase difference pixels 110zL and 110zR, respectively, generated by the histogram generation unit 210a. Graph A is a histogram corresponding to the phase difference pixel 110zL, and Graph B is a histogram corresponding to the phase difference pixel 110zR.

[0222] The processing unit 210b generates a count value by, for example, integrating the histogram generated by the histogram generating unit 210a. That is, the integrated value is proportional to the number of photons reaching the phase difference pixels 110zL and 110zR. In other words, the processing unit 210b integrates the number of pulse signals TOUT generated by the TDC 170 to generate a count value.

[0223] like Fig.14As shown, the processing unit 210b calculates the displacement based on the count values ​​corresponding to the phase difference pixels 110zL and 110zR arranged in a row. Then, the processing unit 210b calculates the distance to the target object 50 based on the displacement. Since the displacement corresponds to the distance to the target object 50, the accuracy is reduced compared to the ToF method, but the distance can be measured while suppressing the influence of attack light and ambient light.

[0224] As described above, according to the present embodiment, the SPADs 335 and 336 are connected to the TDC 170, respectively, and the third distance measuring unit 210 generates a count value by accumulating the number of pulse signals TOUT generated by the TDC 170, and calculates the distance value. Thus, in addition to the same effect as the distance measuring system 1 of the first embodiment to the second embodiment, the same processing effect can be obtained by the TDC 170 without using the counter 172. Therefore, the distance measuring system 1 can be configured only by the TDC 170.

[0225] (Fourth Embodiment)

[0226] The distance measuring system 1 of the first embodiment measures the distance by the measuring pixel 110i after the distance measurement by the phase difference pixels 110zL and 110zR, whereas the distance measuring system 1 according to the fourth embodiment performs the distance measurement by the phase difference pixels 110zL and 110zR and the measuring pixel 110i of each unit 11u in parallel, which is different from the distance measuring system 1 according to the first embodiment. The differences from the distance measuring system 1 of the first embodiment will be described below.

[0227] Fig.30 1 is a diagram for explaining phase difference detection according to the fourth embodiment. The diagram shows the relationship between the surface positions of the target object 50, the lens 40, and the pixel array unit 12 and the optical path of the incident light. In the diagram, 301 and 302 represent the light passing through the left and right sides of the lens 40, respectively. For convenience, only the light 301 and 302 passing through the ends of the lens 40 are recorded. Figure 7 ), light 301 actually passes through the right side of the on-chip lens 1110, and light 302 actually passes through the left side of the on-chip lens 1110. Figure 7 The position of the shielding portion WR is schematically shown on the opposite side.

[0228] The - (negative) position of the figure indicates a so-called rear focus state in which the focus position is on the opposite side of the pixel array unit 12 (the imaging surface). In addition, the 0 position of the figure indicates a focused state. In addition, the + (positive) position of the figure indicates a so-called front focus state in which the focus position is on the side of the pixel array unit 12 (the imaging surface).

[0229] Fig.31 1 is a diagram showing the relationship between the focus position and the output value of the paired phase difference pixels 110zL and 110zR. On the horizontal axis, the focus position is 0, the - position is the rear focus state, and the + position is the front focus state.

[0230] In addition, the output value of the phase difference pixel 110zL is L20a, and the output value of the phase difference pixel 110zR is R20a. That is, the output value L20a is the count value of the counter 172 corresponding to the phase difference pixel 110zL. Similarly, the output value R20a is the count value of the counter 172 corresponding to the phase difference pixel 110zR. Fig.31 As shown, the values ​​of L20a and R20a vary according to the degree of the rear focus state and the front focus state.

[0231] Again, if Fig. 9 As shown, the output value of the phase difference pixel 110zL of each unit 11u is set to L20a, and the output value of the phase difference pixel 110zR is set to R20a, and the second distance measuring unit 206 calculates the difference value D20 based on the output values ​​L20a and R20a. In addition, the second distance measuring unit 206 calculates the distance to the target object 50 according to the unit 11u based on the difference value D20. Since the difference value D20 corresponds to the distance to the target object 50, the accuracy is reduced compared to the ToF method, but the distance can be measured. In this way, in this embodiment, by using SPAD335 and 336, the distance to the target object 50 can be measured according to each unit 11u using the light amount at the photon level.

[0232] Fig.32 1 is a diagram showing the relationship between the focus position and the output value of the phase difference pixels 110zL and 110zR when irradiated with disturbance light. On the horizontal axis, the focus position is 0, the - position is the rear focus state, and the + position is the front focus state.

[0233] In addition, the output value of the phase difference pixel 110zL is L20a, and the output value of the phase difference pixel 110zR is R20a. That is, the output value L20a is the count value of the counter 172 corresponding to the phase difference pixel 110zL. Similarly, the output value R20a is the count value of the counter 172 corresponding to the phase difference pixel 110zR.

[0234] On the other hand, even if the target object 50 is irradiated with disturbance light, the count values ​​of the phase difference pixel 110zL and the phase difference pixel 110zR are only increased by the offset of the number of photons caused by the disturbance light. Therefore, even if the target object 50 is irradiated with disturbance light, the influence on the differential value D20 is suppressed. In this way, in the phase difference method using the phase difference pixel 110zL and the phase difference pixel 110zR, even if the pulse light of the attack circuit 1000 is irradiated to the target object 50, only the signal value L20a and the signal value R20a increase to the same extent, so the influence of the pulse light of the attack circuit 1000, the ambient light, etc. is suppressed.

[0235] Fig.33 1 is a timing chart showing a control example of the integrated control unit 208 according to the fourth embodiment. The horizontal axis represents time. The vertical axis of FIG. A represents the light control signal generated by the light control unit 200 in accordance with the control of the integrated control unit 208, and the selection signal XSEL2, the selection signal XSEL3, the selection signal XSEL0, and the selection signal XSEL1 generated by the drive control unit 202. The high level of these signals represents the drive state.

[0236] Fig.33 FIG. B schematically shows photons incident on the pixel array unit 12. That is, it schematically shows photons generated by the attacking light or the ambient light and photons generated by the secondary measurement light.

[0237] Fig.34 4 is a flowchart showing a control example of the integrated control unit 208 according to the fourth embodiment. Fig.33 A control example of the integrated control unit 208 according to the fourth embodiment will be described.

[0238] like Fig.33 As shown, the integrated control unit 208 makes the light emission control signal a high-level signal, causing the infrared light pulse laser 14 to emit measurement pulse light (step S300).

[0239] (Phase pixel side driving)

[0240] Next, the integrated control unit 208 sets the selection signal XSEL2 and the selection signal XSEL3 for driving the phase difference pixel 110zL and the phase difference pixel 110zR to high level according to the timing of issuing the measurement pulse, so as to drive the phase difference pixel 110zL and the phase difference pixel 110zR, and measure the phase difference signal according to the unit u (step S302). The range in which the selection signal XSEL2 and the selection signal XSEL3 are set to high level can be set according to the predetermined measurement distance range to the target object 50.

[0241] Next, the second distance measuring unit 206 calculates the difference value D20 for each unit 11u based on the pulse signal COUT of each unit 11u, and generates a first distance value to the target object 50 for each unit 11u (step S304). As described above, the distance value is generated while suppressing the influence of the attack light and the like.

[0242] (Measurement of pixel side driving)

[0243] The integrated control unit 208 sets the selection signal XSEL0 driving the reference pixel 110r to a high level according to the timing of the measurement pulse emission to measure the signal of the reference light (step S306). Next, during the period when the selection signal XSEL0 is at a high level, whenever a digital value is input, the histogram generation unit 204a of the first distance measurement unit 204 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to the output of the reference pixel 110r. The processing unit 204b generates, for example, the time corresponding to the maximum frequency of the histogram generated by the histogram generation unit 204a for the reference pixel 110r as time t1.

[0244] Similarly, the integrated control unit 208 sets the selection signal XSEL1 for driving each measurement pixel 110i to a high level according to the timing of the measurement pulse emission, and measures the signal of the measurement light (step S308). Next, while the selection signal XSEL1 is at a high level, each time a digital value is input, the histogram generation unit 204a of the first distance measurement unit 204 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to each of the plurality of measurement pixels 110i. Then, the processing unit 204b generates, for example, time t2n, the time corresponding to the maximum frequency of the histogram generated by the histogram generation unit 204a for each of the plurality of measurement pixels 110i. Thus, the processing unit 204b generates the distance Dn to the target object 50 detected by each unit 11u as (t2n-t1)×speed of light / 2 (step S310).

[0245] As described above, the distance measurement values ​​of the phase difference pixels 110zL and 110zR have lower measurement accuracy than the measurement pixel 110i, but the influence of the attack light and the ambient light is suppressed. On the other hand, the distance measurement values ​​of the measurement pixel 110i have higher measurement accuracy than the distance measurement values ​​of the phase difference pixels 110zL and 110zR, and tend to be easily affected by the attack light and the ambient light.

[0246] Therefore, if the distance measurement value of the measurement pixel 110i of each unit 11u is within the predetermined range of the distance measurement values ​​of the phase difference pixels 110zL and 110zR, the integrated control unit 208 selects the distance measurement value of the measurement pixel 110i. On the other hand, if the distance measurement value of the measurement pixel 110i is not within the predetermined range of the distance measurement values ​​of the phase difference pixels 110zL and 110zR, the distance measurement value of the phase difference pixels 110zL and 110zR is selected (step S312). For example, the range of the predetermined value is set to a range of 90% or more and 110% or less of the distance measurement values ​​of the phase difference pixels 110zL and 110zR.

[0247] As described above, according to the present embodiment, the distance measurement based on the phase difference pixels 110zL and 110zR and the related measuring pixel 110i is performed in parallel for each unit u11, and the first distance value of the phase difference pixels 110zL and 110zR is selected when the unit u11 is irradiated with disturbance light, etc., according to the value of the second distance value based on the first distance value, and the second distance value of the measuring pixel 110i is selected when the unit u11 is not irradiated with disturbance light, etc. Thus, the measurement time can be shortened, and the influence of the disturbance light is suppressed by selecting the distance measurement value based on the phase difference pixels 110zL and 110zR when the unit u11 is irradiated with disturbance light, etc., and a more accurate distance measurement value can be generated by selecting the distance measurement value of the measuring pixel 110i when the unit u11 is not irradiated with disturbance light, etc.

[0248] (Fifth Embodiment)

[0249] In the distance measuring system 1 according to the fourth embodiment, the distance measurement based on the phase difference pixels 110zL and 110zR and the measuring pixel 110i is performed in parallel for each unit 11u, whereas the distance measuring system 1 according to the fifth embodiment performs the distance measurement based on the measuring pixel 110i for each unit 11u after performing the distance measurement based on the phase difference pixels 110zL and 110zR for each unit 11u, and is different from the distance measuring system 1 according to the first embodiment. The following describes the differences between the distance measuring system 1 according to the first embodiment.

[0250] Fig.35 1 is a flowchart showing a control example of the integrated control unit 208 according to the fifth embodiment. Fig.35 As shown, the integrated control unit 208 and Fig.34 (Step S300) to (Step S304) are performed in the same manner.

[0251] Next, the integrated control unit 208 sets the selection signal XSEL0 while the second light emission control signal is at a high level, and sets the second measurement period during which the selection signal XSEL1 of each cell 11u is at a high level according to the first distance value of each cell 11u (step S400).

[0252] Next, the integrated control unit 208 sets the selection signal XSEL0 driving the reference pixel 110r to a high level according to the timing of the second measurement pulse emission to measure the signal of the reference light (step S404). Next, during the period when the selection signal XSEL0 is at a high level, each time a digital value is input, the histogram generation unit 204a of the first distance measurement unit 204 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to the output of the reference pixel 110r. The processing unit 204b generates, for example, the time corresponding to the maximum frequency of the histogram generated by the histogram generation unit 204a for the reference pixel 110r as time t1.

[0253] Next, the integrated control unit 208 sets the selection signal XSEL1 for driving each measurement pixel 110i to a high level according to the timing of the second measurement pulse emission, and measures the signal of the measurement light (step S406). In this way, the integrated control unit 208 sets the measurement time range of the photons reflected and returned from the target object 50 to the second measurement period corresponding to the first distance value based on the phase difference method for each unit 11u, according to the unit 11u.

[0254] Next, while the selection signal XSEL0 is at a high level, whenever a digital value is input, the histogram generation unit 204a of the first distance measurement unit 204 adds, for example, 1 to the frequency of the time interval corresponding to the digital value, and generates a histogram corresponding to the output of each cell 11u and the reference pixel 110r. The processing unit 204b generates, for example, the time corresponding to the maximum frequency of each histogram generated by the histogram generation unit 204a for the reference pixel 110r as time t1.

[0255] Then, the processing unit 204b generates the time corresponding to the maximum frequency of the histogram generated by the histogram generating unit 204a for each of the plurality of measurement pixels 110i as, for example, time t2n. Thus, the processing unit 204b generates the distance Dn to the target object 50 detected by each unit 11u as (t2n-t1)×speed of light / 2 (step S408).

[0256] As described above, the distance measurement system 1 involved in this embodiment is configured such that the integrated control unit 208 makes the measurement period of the photons reflected and returned from the target object 50 correspond to the first distance value measured by the phase difference method of each unit 11u, and sets it according to the unit 11u. Thus, since the second measurement period of each unit 11u is set according to the first distance value at which the influence of the attack light or ambient light of each unit 11u is suppressed, the influence caused by the attack light or ambient light outside the second measurement period in the ToF method is suppressed, and the reduction in the measurement accuracy of the ToF method of each unit 11u is suppressed.

[0257] (Sixth Embodiment)

[0258] The distance measuring system 1 according to the sixth embodiment is different from the distance measuring system 1 according to the first embodiment in that it further includes an infrared pulse laser 16 for the phase difference pixels 110zL and 110zR. The differences from the distance measuring system 1 according to the first embodiment will be described below.

[0259] Fig.36 1 is a block diagram schematically showing a schematic configuration example of a distance measuring system 1 according to the sixth embodiment. Fig.36 As shown, the distance measuring system 1 according to the sixth embodiment differs from the distance measuring system 1 according to the first embodiment in that it further includes an infrared light pulse laser 16 and a diffractive optical element 16a (DOE) for phase difference pixels 110zL and 110zR.

[0260] The infrared light pulse laser 16 is mounted on the substrate 11. For example, a VCSEL (Vertical Cavity Surface Emitting LASER) light source is used to form the infrared light pulse laser 16. In addition, by placing a diffraction optical element 16a (DOE) on the irradiation side of the infrared light pulse laser 16, a dot pattern of 100 points of focused light arranged in a matrix can be irradiated onto the target 50.

[0261] As described above, according to this embodiment, the infrared light pulse laser 16 for the phase difference pixels 110zL and 110zR is further provided. Thus, the measurement for the phase difference pixels 110zL and 110zR and the measurement for the measurement pixel 110i can be performed by independent light sources.

[0262] (Seventh Embodiment)

[0263] The distance measuring system 1 according to the seventh embodiment is different from the distance measuring system 1 according to the first embodiment in that the phase difference pixels 110zL and 110zR, the reference pixel 110r, and the measurement pixel 110i are arranged on different chips.

[0264] Fig.37A Schematically shows a block diagram of a schematic configuration example of a distance measurement system 1 according to the seventh embodiment. Fig.37A As shown, in the distance measurement system 1 according to the seventh embodiment, the phase difference pixels 110zL and 110zR, the reference pixel 110r, and the measurement pixel 110i are arranged in different chips.

[0265] Fig.37B 1 is a block diagram schematically showing a schematic configuration example of a distance measuring system 1 according to a comparative example. Fig.37B As shown, in the distance measurement system 1 involved in the comparative example, the chips of the phase difference pixels 110zL and 110zR are placed between the chip of the reference pixel 110r and the infrared light pulse laser 14. Therefore, in the comparative example, stray light from the infrared light pulse laser 14 is easily incident on the chips of the phase difference pixels 110zL and 110zR. In contrast, in the pixel array unit 12 involved in this embodiment, the infrared light pulse laser 14 is placed between the chip of the reference pixel 110r and the chip of the phase difference pixels 110zL and 110zR, so the stray light is suppressed.

[0266] As described above, according to this embodiment, the phase difference pixels 110zL, 110zR, the reference pixel 110r, and the measurement pixel 110i are configured as different chips. This makes it easy to make the circuit configurations of the phase difference pixels 110zL, 110zR, the reference pixel 110r, and the measurement pixel 110i different, and the degree of freedom in design is further improved.

[0267] (Eighth Embodiment)

[0268] The distance measuring system 1 according to the eighth embodiment is different from the distance measuring system 1 according to the first embodiment in that each of the phase difference pixels 110zL and 110zR and the reference pixel 110r is arranged in one SPAD.

[0269] Fig.38 1 is a diagram showing an example of arrangement of a plurality of phase difference pixels 110zL and 110zR and a plurality of measurement pixels 110i arranged in the pixel array section 12 according to the eighth embodiment. Fig.38As shown in FIG. 1 , each of the plurality of phase difference pixels 110zL and 110zR and the plurality of reference measurement pixels 110i is configured by one SPAD. Apertures WL and WR are arranged in the middle of the phase difference pixels 110zL and 110zR.

[0270] This can further improve the resolution of the reference measurement pixel 110 i.

[0271] (Ninth Embodiment)

[0272] The distance measuring system 1 according to the ninth embodiment is different from the distance measuring system 1 of the first embodiment in that the phase difference pixels 110zL and 110zR are arranged on one chip and the measuring pixel 110i is arranged on another chip.

[0273] Figure 39 to Figure 41 1 is a diagram schematically showing a chip 110zch disposed in the pixel array unit 12 and the positional relationship between the chip 110ich and the infrared light pulse laser 14. Phase difference pixels 110zL and 110zR are arranged in the semiconductor element chip 110zch, and a measurement pixel 110i is arranged in the semiconductor element chip 110ich. By the way, although each chip is schematically shown in a circular shape, the pixels in each chip are arranged in a square matrix.

[0274] Fig.39 110zL and 110zR are arranged in an inverted L-shaped structure in which the chips 110zch and 110ich are arranged to be long in the vertical direction and the infrared light pulse laser 14 is brought closer to the chip 110ich. In such a configuration, stray light from the infrared light pulse laser 14 is suppressed from entering the phase difference pixels 110zL and 110zR.

[0275] Fig.40 1 is a diagram showing an example of an oblique L-shaped structure in which the chip 110zch and the chip 110ich are arranged obliquely and the chip 110ich is brought closer to the infrared light pulse laser 14. In such a configuration, stray light from the infrared light pulse laser 14 is suppressed from entering the phase difference pixels 110zL and 110zR.

[0276] Fig.41 1 is a diagram showing an example of an L-shaped structure in which the chip 110zch and the chip 110ich are arranged horizontally and the infrared light pulse laser 14 is brought closer to the chip 110ich. In such a configuration, stray light from the infrared light pulse laser 14 is suppressed from entering the phase difference pixels 110zL and 110zR. Incidentally, the inverted L-shaped structure and the oblique L-shaped structure involved in this embodiment also correspond to the L-shaped structure, respectively.

[0277] As described above, according to this embodiment, the chip 110zch on which the phase difference pixels 110zL and 110zR are mounted and the chip 110ich on which the measurement pixels 110i are mounted are independently configured, and an L-shaped structure is adopted in which the infrared light pulse laser 14 is brought closer to the chip 110ich. This suppresses stray light from the infrared light pulse laser 14 from entering the phase difference pixels 110zL and 110zR.

[0278] (Tenth Embodiment)

[0279] The distance measuring system 1 according to the tenth embodiment is different from the distance measuring systems 1 according to the first to ninth embodiments in that the phase difference pixels 110zL and 110zR are arranged using a CIS (CMOS Image Sensor) so that distance measurement can be performed using visible light. The differences from the distance measuring system 1 according to the first embodiment are described below.

[0280] In the distance measuring system 1 of the first to ninth embodiments, the measurement using the phase difference pixels 110zL and 110zR is performed by irradiating with the infrared light pulse laser 14 or the infrared light pulse laser 16. However, by configuring the phase difference pixels 110zL and 110zR with a CIS (CMOS Image Sensor), the measurement using the phase difference pixels 110zL and 110zR can be performed without irradiating with the pulse laser. Fig.36 ), a visible light source can be installed instead of the infrared light pulse laser 16. Thus, even in an environment with little light, such as at night, measurement using the phase difference pixels 110zL and 110zR can be performed.

[0281] Fig.42 3310, 3320, CIS3350, 3360 formed in the pixel array unit 12. SPAD3310, 3320, CIS3350, 3360 respectively correspond to the phase difference pixel 110zL, the phase difference pixel 110zR and two of the four pixels for configuring the measurement pixel 110i (see Figure 2 ).

[0282] SPAD3310 and SPAD3320 are respectively provided with a red filter 1122R and a blue filter 1122B. Figure 7 The SPADs 331 and 332 shown in FIG. 1 are different. Thus, the incidence of visible light on the N-well 1112 can be suppressed.

[0283] On the other hand, a green filter 1122G is configured in CIS3350 and 3360, and the photoelectric conversion unit 1112a is configured by CIS (CMOS Image Sensor). In this way, the phase difference pixels 110zL and 110zR can measure the distance using visible light. In addition, the sensitivity of the phase difference pixels 110zL and 110zR in distance measurement can be further improved.

[0284] (Eleventh Embodiment)

[0285] The distance measuring system 1 according to the eleventh embodiment is different from the distance measuring systems 1 according to the first to tenth embodiments in that the phase difference pixels 110zL and 110zR are arranged using elliptical or circular on-chip lenses. The differences from the first to tenth distance measuring systems 1 are described below.

[0286] In the distance measuring system 1 of the first to tenth embodiments, the phase difference pixels 110zL and 110zR are arranged by apertures WL and WR, while in the distance measuring system 1 according to the eleventh embodiment, the phase difference pixels 110zL and 110zR are arranged by an elliptical or circular on-chip lens.

[0287] Fig.43 110 is a diagram showing a planar arrangement example of phase difference pixels 110zL, 110zR and a measurement pixel 110i formed in the pixel array unit 12. Each pixel is configured by one SPAD. An elliptical on-chip lens Lz10 is arranged in the phase difference pixels 110zL, 110zR.

[0288] Fig.44 1 is a simplified cross-sectional schematic diagram of the phase difference pixels 110zL and 110zR. Fig.43 as well as Fig.44 As shown, pupil division of the phase difference pixels 110zL and 110zR can be performed by the elliptical on-chip lens Lz10. Since an aperture is not used, the sensitivity can be further improved.

[0289] Fig.45 1 is a diagram showing an example of a top view of the phase difference pixels 110zL, 110zR and the measurement pixel 110i formed in the pixel array unit 12. Each pixel is configured by one SPAD. A circular on-chip lens Lz20 is arranged in the phase difference pixels 110zL, 110zR and the two measurement pixels 110i. Thus, by accumulating the outputs of the SPADs in which the on-chip lenses Lz20 are arranged, one pixel can be configured.

[0290] On the other hand, by using the outputs of the phase difference pixels 110zL and 110zR, the phase difference pixels 110zL and 110zR can be arranged so as to perform pupil division. The phase difference pixels 110zL and 110zR can perform pupil division. Since an aperture is not used, the sensitivity can be further improved.

[0291] <<Application Examples>>

[0292] The technology disclosed herein can be applied to various products. For example, the technology disclosed herein can also be implemented as a device mounted on any type of mobile object, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, an unmanned aerial vehicle, a ship, a robot, a construction machine, an agricultural machine (tractor), etc.

[0293] Fig.46 7000 is a block diagram showing an example of a schematic configuration of a vehicle control system 7000, which is an example of a mobile body control system to which the technology according to the embodiment of the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected to each other via a communication network 7010. Fig.46 In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 that connects the plurality of control units to each other may be an in-vehicle communication network conforming to any standard, such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), FlexRay (registered trademark), and the like.

[0294] Each control unit includes: a microcomputer that performs calculation processing according to various programs; a storage unit that stores programs executed by the microcomputer, parameters for various operations, etc.; and a drive circuit that drives various control target devices. Each control unit further includes: a network interface (I / F) for performing communication with other control units via the communication network 7010; and a communication I / F for performing communication with devices, sensors, etc. inside and outside the vehicle through wired communication or radio communication. Fig.46 The functional configuration of the integrated control unit 7600 shown includes a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, a sound / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690. Other control units also similarly include a microcomputer, a communication I / F, a storage unit, and the like.

[0295] The drive system control unit 7100 controls the operation of the devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 is used as a control device to control: a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine, a drive motor, etc.; a drive force transmission mechanism for transmitting the drive force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; a brake device for generating the braking force of the vehicle, etc. The drive system control unit 7100 may have the function of a control device for an anti-lock braking system (ABS), an electronic stability control (ESC), etc.

[0296] The drive system control unit 7100 is connected to a vehicle state detection unit 7110. The vehicle state detection unit 7110 includes, for example, at least one of the following items: a gyro sensor for detecting the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor for detecting the acceleration of the vehicle, and a sensor for detecting the operation amount of the accelerator pedal, the operation amount of the brake pedal, the steering angle of the steering wheel, the engine speed or the rotation speed of the wheel, etc. The drive system control unit 7100 performs calculation processing using the signal input from the vehicle state detection unit 7110 to control the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0297] The body system control unit 7200 controls the operation of various devices installed on the body according to various programs. For example, the body system control unit 7200 is used as a control device to control: a keyless entry system, a smart key system, a power window device, or various lights such as headlights, reverse lights, brake lights, turn signals, fog lights, etc. In this case, the body system control unit 7200 can receive radio waves transmitted from a mobile device that replaces the key or signals from various switches as input. The body system control unit 7200 receives these input radio waves or signals to control the vehicle's door lock devices, power window devices, lights, etc.

[0298] The battery control unit 7300 controls the secondary battery 7310 used as a power source for driving the motor according to various programs. For example, the battery control unit 7300 receives information about battery temperature, battery output voltage, remaining battery power, etc. from a battery device including the secondary battery 7310. The battery control unit 7300 uses these signals to perform calculation processing, perform temperature adjustment control of the secondary battery 7310, or control a cooling device of the battery device, etc.

[0299] The vehicle exterior information detection unit 7400 detects information outside the vehicle including the vehicle control system 7000. For example, the vehicle exterior information detection unit 7400 is connected to at least one of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The vehicle exterior information detection unit 7420 may include at least one of the following items: an environmental sensor for detecting current atmospheric conditions or weather conditions, and a peripheral information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle including the vehicle control system 7000.

[0300] The environmental sensor may be, for example, at least one of the following: a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunshine sensor for detecting the degree of sunshine, and a snow sensor for detecting snowfall. The peripheral information detection sensor may be at least one of the following: an ultrasonic sensor, a radar device, and a LIDAR device (a light detection and ranging device, or a laser imaging detection and ranging device). Each of the imaging unit 7410 and the vehicle exterior information detection unit 7420 may be configured as an independent sensor or device, or may be configured as a device in which a plurality of sensors or devices are integrated.

[0301] Fig.47 An example of the installation position of the imaging unit 7410 and the vehicle exterior information detection unit 7420 is shown. The imaging units 7910, 7912, 7914, 7916 and 7918 may be arranged at the positions of the front nose, side mirrors, rear bumper, rear door and the upper part of the windshield inside the vehicle 7900. The imaging unit 7910 arranged at the front nose and the imaging unit 7918 arranged at the upper part of the windshield inside the vehicle mainly obtain the image in front of the vehicle 7900. The imaging units 7912 and 7914 arranged at the side mirrors mainly obtain the image of the side of the vehicle 7900. The imaging unit 7916 arranged at the rear bumper or rear door mainly obtains the image of the rear of the vehicle 7900. The imaging unit 7918 arranged at the upper part of the windshield inside the vehicle is mainly used to detect the front vehicle, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0302] By the way, Fig.47 An example of the shooting range of each imaging unit 7910, 7912, 7914 and 7916 is shown. Imaging range a represents the imaging range of the imaging unit 7910 arranged at the front nose. Imaging ranges b and c respectively represent the imaging ranges of the imaging units 7912 and 7914 arranged at the side mirrors. Imaging range d represents the imaging range of the imaging unit 7916 arranged at the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 7900 observed from above can be obtained by superimposing image data imaged by the imaging units 7910, 7912, 7914 and 7916.

[0303] The vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930 arranged at the front, rear, side, and corner of the vehicle 7900 and the upper portion of the windshield inside the vehicle may be ultrasonic sensors or radar devices. The vehicle exterior information detection units 7920, 7926, and 7930 arranged at the front nose of the vehicle 7900, the rear bumper of the vehicle 7900, the rear door, and the upper portion of the windshield inside the vehicle may be LIDAR devices. These vehicle exterior information detection units 7920-7930 are mainly used to detect vehicles ahead, pedestrians, obstacles, and the like.

[0304] Back to Fig.46 , continue to describe. The vehicle exterior information detection unit 7400 causes the imaging unit 7410 to image the image outside the vehicle and receive the imaged image data. In addition, the vehicle exterior information detection unit 7400 receives detection information from the vehicle exterior information detection unit 7420 connected to the vehicle exterior information detection unit 7400. When the vehicle exterior information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the vehicle exterior information detection unit 7400 causes ultrasonic waves, electromagnetic waves, etc. to be transmitted, and receives information about the received reflected waves. Based on the received information, the vehicle exterior information detection unit 7400 may perform processing of detecting objects (such as people, vehicles, obstacles, signs, symbols, etc. on the road), or perform processing of detecting the distance of the object. The vehicle exterior information detection unit 7400 may perform environmental recognition processing based on the received information to identify rainfall, fog, road conditions, etc. The vehicle exterior information detection unit 7400 may calculate the distance to the object outside the vehicle based on the received information.

[0305] In addition, based on the received image data, the vehicle exterior information detection unit 7400 may perform image recognition processing for identifying an object (such as a person, vehicle, obstacle, sign, symbol, etc. on the road), or perform processing for detecting the distance of the object. The vehicle exterior information detection unit 7400 may perform processing such as distortion correction, alignment, etc. on the received image data, and generate a bird's-eye view image or a panoramic image by combining image data imaged by a plurality of different imaging units 7410. The vehicle exterior information detection unit 7400 may perform viewpoint conversion processing using image data imaged by different imaging units 7410.

[0306] The in-vehicle information detection unit 7500 detects information inside the vehicle. The in-vehicle information detection unit 7500 may be connected to a driver state detection unit 7510 that detects the driver's state. The driver state detection unit 7510 may include a camera for photographing the driver, a biosensor for detecting the driver's biometric information, a microphone for collecting sounds inside the vehicle, and the like. The biosensor may be arranged on a seat surface, a steering wheel, and the like, and detects biometric information of a passenger sitting in a seat or a driver holding the steering wheel. Based on the detection information input from the driver state detection unit 7510, the in-vehicle information detection unit 7500 may calculate the driver's fatigue level or the driver's concentration level, or may distinguish whether the driver is dozing off. The in-vehicle information detection unit 7500 may perform processing such as noise elimination processing on the audio signal obtained by sound collection.

[0307] The integrated control unit 7600 controls the overall operation within the vehicle control system 7000 according to various programs. The integrated control unit 7600 is connected to the input unit 7800. The input unit 7800 is a device that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, a joystick, etc. The integrated control unit 7600 can receive data obtained by voice recognition of the voice input via the microphone. The input unit 7800 can be a remote control device using infrared or other radio waves, or can be an external connection device such as a mobile phone, a personal digital assistant (PDA), etc. that supports the operation of the vehicle control system 7000. The input unit 7800 can be a camera. In this case, the passenger can input information through gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger can be input. In addition, the input unit 7800 may include an input control circuit, etc., which generates an input signal based on the information input by the passenger, etc. using the above-mentioned input unit 7800, and outputs the generated input signal to the integrated control unit 7600. Passengers and the like can input various data into the vehicle control system 7000 through the operation input unit 7800 to process operation instructions.

[0308] The storage unit 7690 may include a read-only memory (ROM) storing various programs executed by the microcomputer and a random access memory (RAM) storing various parameters, operation results, sensor values, etc. In addition, the storage unit 7690 may be a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, etc.

[0309] The general communication I / F 7620 is a widely used communication I / F that mediates communication with various devices present in the external environment 7750. The general communication I / F 7620 can implement: cellular communication protocols such as Global System for Mobile Communications (GSM (registered trademark)), Worldwide Interoperability for Microwave Access (WiMAX (registered trademark)), Long Term Evolution (LTE (registered trademark)), LTE Advanced (LTE-A), etc., or other wireless communication protocols such as wireless LAN (also known as Wireless Fidelity (Wi-Fi (registered trademark)), Bluetooth (registered trademark), etc. The general communication I / F 7620 can be connected to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, the general communication I / F 7620 can use peer-to-peer (P2P) technology to connect to a terminal present near the vehicle (the terminal is, for example, a terminal of a driver, a pedestrian, or a store, or a machine type communication (MTC) terminal).

[0310] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol developed for vehicle use. The dedicated communication I / F 7630 can implement: a standard protocol, for example, Wireless Access in Vehicular Environments (WAVE) (which is a combination of the Institute of Electrical and Electronics Engineers (IEEE) 802.11p as a lower layer and IEEE 1609 as an upper layer), a dedicated short-range communication (DSRC), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication including one or more of the following concepts: communication between vehicles (vehicle-to-vehicle), communication between roads and vehicles (vehicle-to-infrastructure), communication between vehicles and homes (vehicle-to-home), and communication between pedestrians and vehicles (vehicle-to-pedestrian).

[0311] The positioning unit 7640 can perform positioning by receiving a global navigation satellite system (GNSS) signal from a GNSS satellite (e.g., a GPS signal from a global positioning system (GPS) satellite) to generate position information including the latitude, longitude, and altitude of the vehicle. Incidentally, the positioning unit 7640 can identify the current position by exchanging signals with a wireless access point, and can also obtain position information from a terminal such as a mobile phone, a personal handyphone system (PHS), or a smart phone with a positioning function.

[0312] The beacon receiving unit 7650 can receive radio waves or electromagnetic waves transmitted from a radio station installed on a road or the like, thereby obtaining information on the current position, congestion, road closure, required time, etc. Incidentally, the function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

[0313] The in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 can establish a wireless connection using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless universal serial bus (WUSB). In addition, the in-vehicle device I / F 7660 can establish a wired connection through a universal serial bus (USB), a high-definition multimedia interface (HDMI (registered trademark)), a mobile high-definition link (MHL), etc. via a connection terminal (and a cable, if necessary) not shown in the figure. The in-vehicle device 7760 may include at least one of the following items: a mobile device and a wearable device owned by a passenger and an information device loaded into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a path to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0314] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like in accordance with a predetermined protocol supported by the communication network 7010.

[0315] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the vehicle network I / F 7680. For example, the microcomputer 7610 can calculate a control target value for a driving force generation device, a steering mechanism, or a braking device based on the obtained vehicle interior or vehicle exterior related information, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control for functions intended to implement an advanced driver assistance system (ADAS), including collision avoidance or impact buffering for a vehicle, following driving based on a vehicle-to-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 can control the driving force generation device, the steering mechanism, the braking device based on the information obtained about the vehicle surrounding environment, thereby performing cooperative control intended for automatic driving, etc. that does not depend on the operation of the driver.

[0316] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the vehicle network I / F 7680, and generate local map information including information about the surrounding environment in which the vehicle is currently located. In addition, the microcomputer 7610 may predict dangers such as collision of the vehicle, approach of pedestrians, etc., entering a closed road, etc. based on the obtained information, and generate an alarm signal. The alarm signal may be a signal for generating a warning sound or lighting an alarm lamp.

[0317] The sound / image output unit 7670 transmits an output signal of at least one of sound and image to an output device, which can visually or auditorily notify information to the passengers of the vehicle or the outside of the vehicle. Fig.46 In the example, the audio speaker 7710, the display unit 7720 and the instrument panel 7730 are shown as output devices. The display unit 7720 may include at least one of an on-board display and a head-up display. The display unit 7720 may have an augmented reality (AR) display function. The output device may be other devices other than these devices, such as headphones, wearable devices such as eyeglass displays worn by passengers, projectors, lights, etc. In the case where the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610, or displays information received from other control units in various forms (such as text, images, tables, graphs, etc.). In addition, in the case where the output device is an audio output device, the audio output device converts an audio signal composed of played audio data or sound data into an analog signal and outputs the analog signal in an auditory manner.

[0318] By the way, in Fig.46 In the example shown, at least two control units connected to each other via the communication network 7010 may be integrated into one control unit. Alternatively, each individual control unit may include multiple control units. In addition, the vehicle control system 7000 may include other control units not shown in the figure. In addition, part or all of the functions performed by one of the control units in the above description may be assigned to another control unit. That is, a predetermined operation process may be performed by any control unit as long as the information is transmitted and received via the communication network 7010. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0319] By the way, it is possible to use Figure 1The computer program of each function of the distance measuring system 1 involved in the described embodiment is loaded on any control unit, etc. In addition, a recording medium storing such a computer program and readable by a computer can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. In addition, the above-mentioned computer program can also be distributed, for example, via a network without using a recording medium.

[0320] In the vehicle control system 7000 described above, the Figure 1 The optical receiving device 10 of the distance measuring system 1 according to the embodiment described above can be applied to Fig.46 The imaging unit 7410 of the application example shown in FIG. For example, the control unit 20 can be applied to Fig.46 The vehicle exterior information detection unit 7400 is shown in FIG.

[0321] In addition, use Figure 1 At least a part of the configuration elements of the described distance measurement system 1 may also be Fig.46 , for example, an integrated circuit module configured from one die. Alternatively, using Figure 1 The described distance measuring system 1 can also be realized by Fig.46 The vehicle control system 7000 shown in FIG. 7 is implemented by multiple control units.

[0322] Incidentally, the present technology can adopt a configuration like the following. (1)

[0324] A light receiving device, comprising:

[0325] The pixel array unit comprises: a measuring pixel for measuring the distance to a target object; and a plurality of paired phase difference pixels for performing pupil division on incident light from the target object to detect a phase difference;

[0326] a first distance measuring unit that generates a first distance value to the target object based on information about a difference between a timing at which the measuring pixel receives a photon and a predetermined time point; and

[0327] The second distance measuring unit generates a second distance value to the target object based on information corresponding to the number of photons respectively incident on the plurality of phase difference pixels. (2)

[0329] The light receiving device according to (1), wherein

[0330] The measuring pixel has a photoelectric conversion unit that performs photoelectric conversion based on incident photons.

[0331] The phase difference pixel includes a photoelectric conversion unit that performs photoelectric conversion according to incident photons. (3)

[0333] The light receiving device according to (2), wherein

[0334] The photoelectric conversion unit is a single-photon avalanche photodiode (SPAD). (4)

[0336] The light receiving device according to (3), wherein

[0337] The light receiving device further includes a control unit configured to set a driving period of the measurement pixel according to a first distance based on output signals of the plurality of phase difference pixels. (5)

[0339] The light receiving device according to (1), wherein

[0340] The first distance measuring unit generates the first distance value based on a value of a histogram having occurrence frequency information of a difference value between a timing at which the measurement pixel receives a photon and a predetermined time point,

[0341] The second distance measuring unit generates the second distance value based on a phase difference using a signal value corresponding to the number of photons incident on each of the plurality of phase difference pixels. (6)

[0343] The light receiving device according to (5), wherein

[0344] The measuring pixel has a plurality of the photoelectric conversion units.

[0345] The measuring pixel and the plurality of phase difference pixels forming pairs are arranged as a unit,

[0346] The light receiving device further includes a first conversion section corresponding to the unit,

[0347] The first conversion unit generates a reception signal having information on a difference between a timing at which the measurement pixel receives a photon and a predetermined time point. (7)

[0349] The light receiving device according to (3), wherein

[0350] The light receiving device also includes a first infrared light pulse laser,

[0351] The control unit controls the light amount of laser light irradiated by the first infrared light pulse laser according to the second distance. (8)

[0353] The light receiving device according to (7), wherein

[0354] The light receiving device also includes a second infrared light pulse laser,

[0355] The plurality of phase difference pixels generate signals according to laser light irradiated by the second infrared light pulse laser. (9)

[0357] The light receiving device according to (6), wherein

[0358] The pixel array section and the first conversion section are stacked, and the first conversion section corresponding to the cell is disposed directly below the cell. (10)

[0360] The light receiving device further includes a second conversion section corresponding to the unit,

[0361] The second conversion unit generates a second reception signal having information corresponding to the number of photons incident on each of the plurality of phase difference pixels. (11)

[0363] The light receiving device according to (10), wherein

[0364] The second conversion section generates a third reception signal having information on a difference between a timing at which the phase difference pixel receives a photon and a predetermined time point, and generates the second reception signal according to the number of the third reception signals. (12)

[0366] The light receiving device according to (11), wherein

[0367] The measuring pixel and the plurality of phase difference pixels forming a pair generate signals based on photons received in the same time period. (13)

[0369] The light receiving device according to (2), wherein

[0370] The measurement pixel and the plurality of phase difference pixels forming pairs are arranged on different semiconductor device chips. (14)

[0372] The light receiving device according to (13), wherein

[0373] The light receiving device also includes a first infrared light pulse laser,

[0374] The first chip on which the measuring pixels are arranged, the second chip on which the plurality of phase difference pixels in pairs are arranged, and the first infrared light pulse laser are arranged in an L shape. (15)

[0376] A light receiving device, comprising:

[0377] a pixel array portion, which is configured by a plurality of pixels; and

[0378] a control unit, controlling the pixel array unit,

[0379] The pixel array section comprises:

[0380] A plurality of phase difference pixels in pairs, performing pupil division on incident light from a target object to detect a phase difference; and

[0381] Determine pixels for determining the distance to the target object,

[0382] The control unit sets a driving period of the measurement pixel according to a second distance based on output signals of the plurality of phase difference pixels. (16)

[0384] The light receiving device according to (15), wherein

[0385] The phase difference pixel has a photoelectric conversion unit that receives visible light and performs photoelectric conversion.

[0386] The measurement pixel includes a photoelectric conversion unit that performs photoelectric conversion based on incident photons. (17)

[0388] The light receiving device according to (2), wherein

[0389] The photoelectric conversion unit further includes a reflection prevention unit on the incident side. (18)

[0391] The light receiving device according to (2), wherein

[0392] The photoelectric conversion unit has an on-chip lens formed of a high refractive index material on the incident side. (19)

[0394] The light receiving device according to (2), wherein

[0395] The photoelectric conversion unit has:

[0396] On-chip lens; and

[0397] The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion.

[0398] The diffusion layer is disposed according to the position of the principal optical axis of the on-chip lens. (20)

[0400] The light receiving device according to (2), wherein

[0401] In the phase difference pixel,

[0402] The photoelectric conversion unit has:

[0403] The aperture of the incident area; and

[0404] The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion.

[0405] The diffusion layer is positioned according to the position of the aperture. (twenty one)

[0407] The light receiving device according to (2), wherein

[0408] Two phase difference pixels in the plurality of phase difference pixels in pairs,

[0409] An elliptical on-chip lens is provided in the photoelectric conversion parts of the two phase difference pixels. (twenty two)

[0411] The light receiving device according to (2), wherein

[0412] The light receiving device has a circular on-chip lens, photoelectric converters of two phase difference pixels arranged in the plurality of phase difference pixels forming a pair, and photoelectric converters of the two measurement pixels. (twenty three)

[0414] The light receiving device according to (15), wherein

[0415] The photoelectric conversion unit of the phase difference pixel receives light via a filter that transmits visible light.

[0416] The photoelectric conversion section of the measurement pixel receives light via a filter that transmits infrared light. (twenty four)

[0418] A control method is a control method of a pixel array unit, wherein the pixel array unit has:

[0419] A plurality of phase difference pixels in pairs, performing pupil division on incident light from a target object to detect a phase difference; and

[0420] Determine pixels for determining the distance to the target object,

[0421] In the control method,

[0422] A driving period of the measurement pixel is set according to a second distance based on output signals of the plurality of phase difference pixels. (25)

[0424] A distance measuring system, comprising:

[0425] The light receiving device described in (1); and

[0426] A lens that focuses incident light from the target object.

[0427] The scheme of the present disclosure is not limited to the above-mentioned embodiments, but also includes various modifications that can be thought of by those skilled in the art, and the effects of the present disclosure are not limited to the above-mentioned contents. That is, various additions, changes and partial deletions can be made within the scope of the conceptual ideas and purposes of the present disclosure derived from the contents specified in the claims and their equivalents.

[0428] Description of Reference Numerals

[0429] 1: ranging system; 10: light receiving device; 12: pixel array unit; 14: infrared light pulse laser; 16: infrared light pulse laser; 20: control unit; 24: display device; 40: lens; 110i: measurement pixel; 110r: reference pixel; 110z, 110zL, 110zR: phase difference pixel; 331~340: SPAD; 1110: on-chip lens; 3350, 3360: CIS; Lz10, Lz20: on-chip lens; WL, WR: aperture.

Claims

1. A light receiving device, comprising: The pixel array unit has: a measuring pixel for measuring the distance to the target object; and a plurality of phase difference pixels in pairs, which perform pupil division on incident light from the target object to detect phase difference; a first distance measuring unit that generates a first distance value to the target object based on information about a difference between a timing at which the measuring pixel receives a photon and a predetermined time point; as well as The second distance measuring unit generates a second distance value to the target object based on information corresponding to the number of photons respectively incident on the plurality of phase difference pixels.

2. The light receiving device according to claim 1, wherein The measurement pixel includes a photoelectric conversion section that performs photoelectric conversion based on incident photons, and the phase difference pixel includes a photoelectric conversion section that performs photoelectric conversion based on incident photons.

3. The light receiving device according to claim 2, wherein: The photoelectric conversion unit is a single-photon avalanche photodiode (SPAD).

4. The light receiving device according to claim 3, wherein The light receiving device further includes a control unit configured to set a driving period of the measurement pixel according to a second distance based on output signals of the plurality of phase difference pixels.

5. The light receiving device according to claim 1, wherein The first distance measuring unit generates the first distance value based on a value of a histogram having occurrence frequency information of a difference value between a timing at which the measurement pixel receives a photon and a predetermined time point, The second distance measuring unit generates the second distance value based on a phase difference using a signal value corresponding to the number of photons incident on each of the plurality of phase difference pixels.

6. The light receiving device according to claim 2, wherein: The measuring pixel has a plurality of the photoelectric conversion units. The measuring pixel and the plurality of phase difference pixels forming pairs are arranged as a unit, The light receiving device further includes a first conversion section corresponding to the unit, The first conversion unit generates a reception signal having information on a difference between a timing at which the measurement pixel receives a photon and a predetermined time point.

7. The light receiving device according to claim 4, wherein: The light receiving device also includes a first infrared light pulse laser, The control unit controls the light amount of laser light irradiated by the first infrared light pulse laser according to the second distance.

8. The light receiving device according to claim 7, wherein The light receiving device also includes a second infrared light pulse laser, The plurality of phase difference pixels generate signals according to laser light irradiated by the second infrared light pulse laser.

9. The light receiving device according to claim 6, wherein: The pixel array section and the first conversion section are stacked, and the first conversion section corresponding to the cell is disposed directly below the cell.

10. The light receiving device according to claim 6, wherein The light receiving device further includes a second conversion section corresponding to the unit, The second conversion unit generates a second reception signal having information corresponding to the number of photons incident on each of the plurality of phase difference pixels.

11. The light receiving device according to claim 10, wherein The second conversion section generates a third reception signal having information on a difference between a timing at which the phase difference pixel receives a photon and a predetermined time point, and generates the second reception signal according to the number of the third reception signals.

12. The light receiving device according to claim 11, wherein The measuring pixel and the plurality of phase difference pixels forming a pair generate signals based on photons received in the same time period.

13. The light receiving device according to claim 2, wherein: The measurement pixel and the plurality of phase difference pixels forming pairs are arranged on different semiconductor device chips.

14. The light receiving device according to claim 13, wherein The light receiving device also includes a first infrared light pulse laser, The first chip on which the measuring pixels are arranged, the second chip on which the plurality of phase difference pixels in pairs are arranged, and the first infrared light pulse laser are arranged in an L shape.

15. A light receiving device, comprising: A pixel array portion, configured by a plurality of pixels; as well as a control unit, controlling the pixel array unit, The pixel array section comprises: A plurality of phase difference pixels are paired to perform pupil division on incident light from a target object to detect phase difference; as well as Determine pixels for determining the distance to the target object, The control unit sets a driving period of the measurement pixel according to a second distance based on output signals of the plurality of phase difference pixels.

16. The light receiving device according to claim 15, wherein The phase difference pixel includes a photoelectric conversion section that receives visible light and performs photoelectric conversion, and the measurement pixel includes a photoelectric conversion section that performs photoelectric conversion based on incident photons.

17. The light receiving device according to claim 2, wherein: The photoelectric conversion unit further includes a reflection prevention unit on the incident side.

18. The light receiving device according to claim 2, wherein: The photoelectric conversion unit has an on-chip lens formed of a high refractive index material on the incident side.

19. The light receiving device according to claim 2, wherein: The photoelectric conversion unit has: On-chip lens; as well as The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion. The diffusion layer is disposed according to the position of the principal optical axis of the on-chip lens.

20. The light receiving device according to claim 2, wherein In the phase difference pixel, The photoelectric conversion unit has: The aperture of the incident area; as well as The diffusion layer has at least an avalanche multiplication region for multiplying carriers generated by the photoelectric conversion. The diffusion layer is positioned according to the position of the aperture.

21. The optical receiving device according to claim 2, wherein: Two of the plurality of phase difference pixels forming a pair have elliptical on-chip lenses provided in photoelectric conversion portions of the two phase difference pixels.

22. The light receiving device according to claim 2, wherein: The light receiving device has a circular on-chip lens, photoelectric converters of two phase difference pixels arranged in the plurality of phase difference pixels forming a pair, and photoelectric converters of the two measurement pixels.

23. The light receiving device according to claim 15, wherein The photoelectric conversion unit of the phase difference pixel receives light via a filter that transmits visible light. The photoelectric conversion section of the measurement pixel receives light via a filter that transmits infrared light.

24. A control method, a control method of a pixel array unit, the pixel array unit comprising: a plurality of phase difference pixels in pairs, which perform pupil division on incident light from a target object to detect a phase difference; and a measuring pixel, which is used to measure a distance to the target object, wherein: A driving period of the measurement pixel is set according to a second distance based on output signals of the plurality of phase difference pixels.

25. A distance measurement system, comprising: The light receiving device according to claim 1; as well as A lens that focuses incident light from the target object.

Images