Ranging device and ranging method

Abstract

[Problem] The problem according to the present disclosure is to provide a ranging device and a ranging method which are capable of suppressing a decrease in the accuracy of measuring a ranging value generated on the basis of a phase difference. [Solution] To this end, provided is a ranging device which, according to the present disclosure, comprises: a ranging sensor having a first charge storage part for storing signal charges using a first drive frequency and a second charge storage part for storing signal charges using a second drive frequency; a control unit for setting a switching period for switching from first irradiation light to second irradiation light shorter than a reading period for reading a first signal corresponding to an accumulated charge of the first charge storage part and a second signal corresponding to an accumulated charge of the second charge storage part; a reading unit for reading the first signal and the second signal; and a signal processing unit for generating a distance measurement value by generating, on the basis of the first signal and as a first phase difference, the time until light is received as reflected light, and generating, on the basis of the second signal and as a second phase difference, the time until light is received as reflected light.

Description

Distance measuring device and distance measuring method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method.

[0002] A commonly known distance measurement method for distance measurement modules is the indirect time of flight (ToF) method, which generates a phase difference representing the time from when a pattern of light is irradiated onto an object until the light is received as reflected light, and generates a distance measurement value based on this phase difference.

[0003] Japanese Patent Application Laid-Open No. 2022-111663

[0004] However, in indirect ToF distance measurement, AC modulated light is repeatedly irradiated. Furthermore, multiple modulation frequencies are set and distributed to the two charge storage units. This modulation frequency has four phases (0, 90, 180, and 270 degrees), requiring four data readouts. Furthermore, measurements may be performed using multiple AC modulated light beams to further improve measurement accuracy. In this case, for the multiple AC modulated light beams, the modulation frequency has four phases (0, 90, 180, and 270 degrees), requiring eight data readouts. Therefore, when the object being measured is a moving object, the measurement period may be extended, potentially reducing measurement accuracy.

[0005] Therefore, the present disclosure provides a distance measuring device and a distance measuring method that can suppress a decrease in measurement accuracy of distance values ​​generated based on phase differences.

[0006] In order to solve the above problem, according to the present disclosure, there is provided a distance measuring device that sequentially irradiates first irradiation light of a first modulation frequency and second irradiation light of a second modulation frequency different from the first modulation frequency, and generates a distance value using light reflected by an object, the distance measuring sensor having: a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency; a control unit that controls a switching period for switching from the first irradiation light to the second irradiation light to be shorter than a readout period for reading out a first signal corresponding to the accumulated charges in the first charge accumulation unit and a second signal corresponding to the accumulated charges in the second charge accumulation unit after irradiation with the second irradiation light; a signal processing unit that generates a first phase difference based on the first signal, which is the time until the light is received as the reflected light, and that generates a second phase difference based on the second signal, which is the time until the light is received as the reflected light, and that generates a distance measurement value using the first phase difference and the second phase difference.

[0007] The distance measuring sensor may include a plurality of pixel groups each having the first charge accumulation portion and a plurality of pixel groups each having the second charge accumulation portion.

[0008] The distance measurement sensor may further include a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit may be configured to make the first charge discharging transistor conductive and the second charge discharging transistor non-conductive during a period in which the second irradiation light is irradiated, and to make the first charge discharging transistor non-conductive and the second charge discharging transistor conductive during a period in which the first irradiation light is irradiated.

[0009] The distance measurement sensor may further include a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit may cause the first charge discharging transistor to alternate between a non-conductive state and a conductive state during the period in which the second irradiation light is irradiated, and may cause the second charge discharging transistor to alternate between a non-conductive state and a conductive state during the period in which the first irradiation light is irradiated.

[0010] a first pixel having the first photoelectric conversion unit accumulates charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the first modulation frequency, and accumulates charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency; a second pixel adjacent to the first pixel and having the first photoelectric conversion unit accumulates charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, and accumulates charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency; and the signal processing unit may generate the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

[0011] a third pixel having the second photoelectric conversion unit accumulates charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the second modulation frequency, and accumulates charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency; a fourth pixel adjacent to the third pixel and having the second photoelectric conversion unit accumulates charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the second modulation frequency, and accumulates charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency; and the signal processing unit may generate the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

[0012] The fifth pixel having the first photoelectric conversion unit may accumulate charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the first modulation frequency, accumulate charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, accumulate charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and accumulate charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency, and the signal processing unit may generate the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

[0013] The sixth pixel having the second photoelectric conversion unit may accumulate charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the second modulation frequency, accumulate charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the second modulation frequency, accumulate charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency, and accumulate charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency, and the signal processing unit may generate the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

[0014] The fifth pixel and the sixth pixel may be adjacent to each other.

[0015] The first photoelectric conversion unit and the second photoelectric conversion unit may be the same photoelectric conversion unit.

[0016] The seventh pixel may be synchronized with the first modulation frequency, accumulate charge in one charge storage section of the first charge storage section in accordance with one of two pulse drive signals having different phases, and accumulate charge in the other charge storage section of the first charge storage section in accordance with the other of the two pulse drive signals having different phases; may be synchronized with the second modulation frequency, accumulate charge in one charge storage section of the second charge storage section in accordance with one of two pulse drive signals having different phases, and accumulate charge in the other charge storage section of the second charge storage section in accordance with the other of the two pulse drive signals having different phases.

[0017] In the seventh pixel, the two pulse drive signals synchronized with the first modulation frequency and having different phases may be a 0-degree pulse drive signal that is in phase with the first modulation frequency and a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases may be a 0-degree pulse drive signal that is in phase with the second modulation frequency and a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency.

[0018] An eighth pixel adjacent to the seventh pixel may have the same configuration as the seventh pixel, and the two pulse drive signals synchronized with the first modulation frequency and having different phases may be a 90-degree pulse drive signal that is in phase with the first modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases may be a 90-degree pulse drive signal that is in phase with the second modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency.

[0019] The first modulation frequency may be lower than the second modulation frequency.

[0020] The signal processing unit may generate the distance measurement value based on a repetition period of the second phase difference corresponding to the first phase difference and the second phase difference.

[0021] The first modulation frequency and the second modulation frequency may have the same period.

[0022] the signal processing unit generates a first reliability using a first component using a signal based on the accumulated charges of a first charge accumulation unit of the first charge accumulation unit and a signal based on the accumulated charges of a third charge accumulation unit of the second charge accumulation unit, and a second component using a signal based on the accumulated charges of a second charge accumulation unit of the first charge accumulation unit and a signal based on the accumulated charges of the second charge accumulation unit of the second charge accumulation unit; and a second reliability using a third component using a signal based on the accumulated charges of the first charge accumulation unit of the second charge accumulation unit and a signal based on the accumulated charges of the third charge accumulation unit of the second charge accumulation unit, and a fourth component using a signal based on the accumulated charges of the second charge accumulation unit of the first charge accumulation unit and a signal based on the accumulated charges of the second charge accumulation unit of the second charge accumulation unit; and when a difference between the first reliability and the second reliability is within a predetermined value, the signal processing unit may generate the distance measurement value based on the second phase difference and a repetition period of the second phase difference corresponding to the first phase difference.

[0023] In order to solve the above problem, according to the present disclosure, there is provided a distance measuring device that irradiates first irradiation light of a first modulation frequency and generates a distance value using reflected light reflected by an object, the distance measuring sensor having ten pixels that accumulate signal charges generated in a photoelectric conversion unit in a first charge accumulation unit in accordance with a 0-degree pulse drive signal that is synchronized with the first modulation frequency and has the same phase as the first modulation frequency, accumulate the signal charges in a second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, accumulate the signal charges in a third charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and accumulate the signal charges in a fourth charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency; a first sum signal of a signal based on the accumulated charges in the first charge accumulation unit and a signal based on the accumulated charges in the third charge accumulation unit, A distance measuring device is provided, comprising: a second sum signal of a signal based on the accumulated charge in the second charge storage unit and a signal based on the accumulated charge in the third charge storage unit; and a signal processing unit that generates a distance measurement value by generating a first phase difference corresponding to a frequency lower than a first modulation frequency based on the first sum signal and the second sum signal, which represents the time until the reflected light is received.

[0024] In order to solve the above problem, according to the present disclosure, there is provided a distance measuring device that sequentially irradiates an object with first irradiation light of a first modulation frequency, second irradiation light of a second modulation frequency different from the first modulation frequency, and third irradiation light of the first modulation frequency, and generates a distance value using light reflected by the object and returning, the distance measuring sensor having: a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency; and a readout unit that, after irradiation with the first irradiation light, reads out a first signal corresponding to the accumulated charges in the first charge accumulation unit, reads out a second signal corresponding to the accumulated charges in the second charge accumulation unit, and, after irradiation with the third irradiation light, reads out a third signal corresponding to the accumulated charges in the first charge accumulation unit. a signal processing unit that generates a first phase difference based on the first signal, which is the time until the light is received as reflected light; that generates a second phase difference based on the second signal, which is the time until the light is received as reflected light; that generates a third phase difference based on the third signal, which is the time until the light is received as reflected light; and that generates a distance measurement value using the average value of the first phase difference and the third phase difference and the second phase difference.

[0025] In order to solve the above problem, the present disclosure provides a distance measurement method using a distance measurement sensor that sequentially irradiates an object with first irradiation light of a first modulation frequency and second irradiation light of a second modulation frequency different from the first modulation frequency, and that has: a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency; and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency, wherein a switching period for switching from the first irradiation light to the second irradiation light is shorter than a readout period for reading out a first signal corresponding to the accumulated charges in the first charge accumulation unit and a second signal corresponding to the accumulated charges in the second charge accumulation unit after irradiation with the second irradiation light; and based on the first signal, generates a first phase difference that indicates the time until the light is received as reflected light from the object, and based on the second signal, generates a second phase difference that indicates the time until the light is received as reflected light, and generates a distance measurement value using the first phase difference and the second phase difference.

[0026] 1 is a diagram showing an example of a schematic configuration of a distance measuring device to which the present technology is applied. FIG. 1 is a block diagram showing an example of a configuration of a light receiving element. FIG. 2 is a diagram showing an example of an arrangement of pixels of a distance measuring sensor according to function. FIG. 3 is a diagram showing an example of an arrangement of four adjacent pixels with a modulation frequency Fmod1 and four adjacent pixels with a modulation frequency Fmod2. FIG. 4 is a diagram showing an example of a detailed circuit configuration of a pixel. FIG. 5 is a diagram explaining the emission of irradiation light and the driving of pixels. FIG. 6 is a block diagram showing an example of a configuration of a signal processing unit. FIG. 7 is a diagram showing the exposure periods of the first taps of pixels at each phase, arranged so that the phase difference is easy to understand. FIG. 8 is a schematic explanatory diagram showing that the reliability is constant regardless of the phase. FIG. 9 is a time chart showing an example of a driving method according to the present embodiment. FIG. 10 is a diagram showing pixels used by an interpolation processing unit and example combinations of pixels. FIG. 11 is a diagram showing a method of calculating a distance value in an interpolation processing unit using a first distance value and a second distance value. FIG. 12 is a diagram showing the relationship between the reflectance of an object and reliability. FIG. 13 is a diagram showing reliability when two types of measurement light are used. FIG. 14 is a diagram showing a driving state of Comparative Example 1. 20. A diagram schematically showing the relationship between an object and depth values ​​(measured distance values) in Comparative Example 1. A diagram schematically showing a driving state in Comparative Example 2. A diagram schematically showing the relationship between an object and depth values ​​(measured distance values) in Comparative Example 2. A diagram schematically showing the relationship between an object and depth values ​​(measured distance values) according to the present embodiment. A diagram showing three driving modes. A diagram showing three driving modes different from that of FIG. 20. A diagram showing an example of interpolation processing. A diagram showing an example of interpolation processing for a pixel group. A flowchart showing an example of control according to the present embodiment. A time chart showing a driving example according to Modification 1 of the first embodiment. A diagram showing an example of the configuration of a distance measuring sensor according to the second embodiment. A time chart showing a driving example according to the second embodiment. A time chart showing a driving example according to the third embodiment. A time chart showing an example of driving of a distance measuring device according to the fourth embodiment. A diagram showing an example in which pixels are configured with 4TAP. A diagram showing an example of interpolation processing in the arrangement example of FIG. 31. A diagram showing three driving modes. A diagram showing three driving modes different from that of FIG. 33. A diagram showing three driving modes further different from that of FIG. 33. 34A and 34B are diagrams showing three driving modes different from those in FIG. 33;

[0027] Hereinafter, embodiments of a distance measuring device and a distance measuring method will be described with reference to the drawings. The following description will focus on the main components of the distance measuring device and the distance measuring method, but the distance measuring device and the distance measuring method may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0028] First Embodiment <Configuration Example of Range Finder> Fig. 1 is a diagram showing a schematic configuration example of a range finder to which the present technology is applied. The range finder 1 in Fig. 1 is a system capable of reading all phase data for a plurality of AC modulated lights in a single readout.

[0029] The distance measuring device 1 includes a control device 11, a light receiving device 12, and an illumination device 13. The light receiving device 12 includes a light receiving element 31 and a lens 32. The illumination device 13 includes an LD 21 and a light emitting unit 22.

[0030] The control device 11 is a device that controls the overall operation of the distance measuring device 1. The control device 11 specifies an operation mode and supplies a measurement request to the light receiving device 12.

[0031] Furthermore, the control device 11 acquires measurement data generated based on the results of light received by the light receiving device 12 in response to a measurement request from the light receiving device 12. The control device 11 is not a part of the configuration of the distance measuring device 1, but may be provided outside the distance measuring device 1, for example, as a control unit of a host device in which the distance measuring device 1 is incorporated.

[0032] Here, the operating modes that the control device 11 can specify for the light receiving device 12 include a ranging mode that measures the distance to an object using an indirect ToF method, and a 2D image capture mode that generates a two-dimensional luminance image. The indirect ToF ranging method detects the time of flight from the time when irradiated light is emitted to the time when reflected light is received as a phase difference to calculate the distance to the object. The 2D image capture mode is a mode that outputs a two-dimensional luminance image according to the amount of received light, like a normal image sensor. The ranging device 1 according to this embodiment also has multiple ranging modes. Details of the ranging modes will be described later.

[0033] When a measurement request is supplied from the control device 11, the light receiving element 31 of the light receiving device 12 supplies a light emission pulse to the LD 21 under the control of the control unit 51 (see FIG. 2 described below), causing the light emitter 22 to emit irradiation light. The light receiving element 31 then receives reflected light from subjects such as objects 41 and 42 via the lens 32. The lens 32 forms an image of the reflected light from the subjects as incident light on the light receiving surface of the light receiving element 31. The lens 32 may have any configuration, and for example, the lens 32 may be formed of a group of multiple lenses.

[0034] The light emission pulse is a timing signal that serves as a reference for the light emission operation of the light emitting unit 22 and the light reception operation of the light receiving element 31, and is, for example, a pulse signal that is repeatedly on (High) and off (Low). For example, according to this embodiment, a pulse signal with a modulation frequency Fmod1 is generated in a first period, and then a pulse signal with a modulation frequency Fmod2 is generated in a second period. The modulation frequency Fmod1 is a low-frequency frequency, for example, 20 MHz. On the other hand, the modulation frequency Fmod2 is a high-frequency frequency, for example, 100 MHz. The ranging accuracy on the Fmod1 side is, for example, 7.5 m per 360 degrees. On the other hand, the ranging accuracy on the Fmod2 side is, for example, 1.5 m per 360 degrees. That is, when generating one distance image (depth image), distance measurement (coarse depth) using a pulse signal of modulation frequency Fmod1 and distance measurement (fine depth) using a pulse signal of modulation frequency Fmod2 are performed. Note that, in this embodiment, modulation frequency Fmod1 corresponds to the first modulation frequency, and modulation frequency Fmod2 corresponds to the second modulation frequency.

[0035] When the operation mode is the distance measurement mode, the light receiving element 31 generates a depth image storing distance information to the subject based on the result of receiving reflected light, and outputs the image as measurement data to the control device 11. On the other hand, when the operation mode is the 2D image capturing mode, the light receiving device 12 generates a luminance image of the subject based on the result of receiving reflected light, and outputs the luminance image to the control device 11 as measurement data. The specific configuration of the light receiving element 31 will be described in detail from FIG. 2 onwards.

[0036] The LD 21 of the lighting device 13 is, for example, a laser driver that drives the light-emitting unit 22, and drives the light-emitting unit 22 based on the light-emitting pulse from the light-receiving element 31, causing the light-emitting unit 22 to output AC modulated light as irradiation light. The light-emitting unit 22 is formed of, for example, a VCSEL LED (Vertical Cavity Surface Emitting Laser LED), and emits irradiation light when the LD 21 is driven. For example, infrared light (IR light) having a wavelength in the range of approximately 850 nm to 940 nm is used as the irradiation light.

[0037] <Configuration Example of Light-Receiving Element> FIG. 2 is a block diagram showing a configuration example of the light-receiving element 31 in FIG.

[0038] The light receiving element 31 includes a control unit 51 , a distance measurement sensor 52 , a pulse generating circuit 53 , a tap driving unit 54 , a vertical driving unit 55 , a column processing unit 56 , a horizontal driving unit 57 , a signal processing unit 58 , and an output unit 59 .

[0039] The control unit 51 controls the overall operation of the light receiving element 31. For example, the control unit 51 acquires the operation mode and measurement request supplied from the control device 11 via an input unit (not shown). The control unit 51 also instructs the pulse generation circuit 53 to generate a light emission pulse with a predetermined modulation frequency Fmod according to the operation mode. The control unit 51 also supplies control signals for performing operations according to the operation mode to the tap drive unit 54, vertical drive unit 55, column processing unit 56, horizontal drive unit 57, etc. In other words, the control unit 51 sets and controls the charge accumulation time and readout period of the accumulated charge in the distance measurement sensor 52, the irradiation time of the irradiation light using the pulse generation circuit 53, the switching period for switching the modulation frequency of the irradiation light, etc. Furthermore, the control unit 51 instructs the signal processing unit 58 to perform predetermined signal processing according to the operation mode, such as generating distance values ​​based on multiple phase differences to generate a depth image (distance image) and generating a luminance image.

[0040] <Example of Pixel Arrangement by Function> A plurality of pixels 61 are arranged two-dimensionally in a matrix in the distance measurement sensor 52. Although the configuration of the pixels 61 according to this embodiment is the same, it is possible to vary the functions of the pixels by varying the drive signals sent via the tap drive unit 54.

[0041] 3 is a diagram showing an example of the functional arrangement of pixels in the distance measurement sensor 52. As shown in FIG. 3, the light receiving frequency processed by the pixels 61 varies every two rows. For example, the pixels 61 in the two rows from the top process measurement light with a modulation frequency Fmod1 = 20 MHz. That is, the pixels 61 in the two rows from the top receive and process 20 MHz measurement light having high and low illuminance levels.

[0042] Furthermore, the pixels 61 in the next two rows process the measurement light with a modulation frequency Fmod2=100 MHz. That is, the pixels 61 in the next two rows receive and process the measurement light with a frequency of 100 MHz having high and low illuminance levels.

[0043] Furthermore, the pixel group that processes the measurement light with a modulation frequency Fmod1 = 20 MHz is an example of a combination when, for example, four pixels adjacent to each other above, below, left, and right calculate a distance measurement value. In Figure 3, "0" and "180" correspond to measurement pixels corresponding to phases of 0 and 180 degrees, and "90" and "270" correspond to measurement pixels corresponding to phases of 90 and 270 degrees.

[0044] Similarly, the pixel group that processes measurement light with a modulation frequency Fmod1 = 100 MHz is, for example, a combination of four adjacent pixels vertically and horizontally when calculating a distance measurement value. In FIG. 3 , "0" and "180" correspond to measurement pixels corresponding to phases of 0 and 180 degrees, and "90" and "270" correspond to measurement pixels corresponding to phases of 90 and 270 degrees. Note that these arrangements are merely examples and are not limiting. That is, the pixels 61 that process modulation frequency Fmod1 and the pixels 61 that process modulation frequency Fmod2 can be interchanged. Similarly, the measurement pixels corresponding to phases of 0 and 180 degrees and the measurement pixels corresponding to phases of 90 and 270 degrees can be interchanged. Alternatively, the pixels 61 that process modulation frequency Fmod1 and the pixels 61 that process modulation frequency Fmod2 can be arranged in a checkerboard pattern.

[0045] 4 is a diagram schematically illustrating an example of the arrangement of four adjacent pixels 61A, B, C, and D with a modulation frequency Fmod1 and four adjacent pixels 61a, b, c, and d with a modulation frequency Fmod2. As shown in FIG. 4, the pixel 61 distributes accumulated charge to two taps (charge accumulation units) 73A and 73B via two transfer transistors 72A and 72B. Note that the taps (charge accumulation units) 73A and 73B of the pixels 61A, B, C, and D according to this embodiment correspond to first charge accumulation units. Furthermore, the taps (charge accumulation units) 73A and 73B of the pixels 61a, b, c, and d according to this embodiment correspond to second charge accumulation units.

[0046] 4 , four adjacent pixels 61A, 61B, 61C, and 61D receive reflected light by tap 73A at light reception timings that are shifted in phase by 0°, 90°, 180°, and 270° relative to the irradiation timing of irradiation light having modulation frequency Fmod1, and accumulate electric charges. More specifically, pixel 61A receives light with a phase of 0° relative to the irradiation timing of the irradiation light, pixel 61B receives light with a phase of 90° relative to the irradiation timing of the irradiation light, pixel 61C receives light with a phase of 180° relative to the irradiation timing of the irradiation light, and pixel 61D receives light with a phase of 270° relative to the irradiation timing of the irradiation light, and so on.

[0047] Unless otherwise specified, a phase of 0°, 90°, 180°, or 270° refers to the phase at tap 73A of pixel 61. Tap 73B has an inverted phase from the first tap, so when the first tap has a phase of 0°, 90°, 180°, or 270°, the second tap has a phase of 180°, 270°, 0°, or 90°, respectively. Furthermore, tap 73A may also be referred to as the first tap, and tap 73B may also be referred to as the second tap.

[0048] Similarly, the four adjacent pixels 61 a, b, c, and d receive reflected light at tap 73A (first tap) at light reception timings that are shifted in phase by 0°, 90°, 180°, and 270° relative to the irradiation timing of the illumination light having the modulation frequency Fmod2. More specifically, pixel 61 a receives reflected light with a phase of 0° relative to the irradiation timing of the illumination light, pixel 61 b ​​receives light with a phase of 90°, pixel 61 c receives light with a phase of 180°, and pixel 61 d receives light with a phase of 270°. In this way, each of the four adjacent pixels 61 A, B, C, and D arranged two-dimensionally in the distance measurement sensor 52 and each of the four adjacent pixels 61 a, b, c, and d can simultaneously generate light reception signals with different phases by changing the phase relative to the irradiation timing of the illumination light for each pixel 61 position. In this way, signal charges can be accumulated in each of the four adjacent pixels 61A, B, C, and D in the distance measurement sensor 52 and in the taps 73A and B of each of the four adjacent pixels 61a, b, c, and d, so that phase information at the modulation frequency Fmod1 required for long-distance distance measurement and phase information at the modulation frequency Fmod2 required for high-precision distance measurement can be read out together in a series of readout sequences in a single readout. Note that in this embodiment, after measurements are made at each of the four adjacent pixels 61A, B, C, and D, measurements are made at each of the four adjacent pixels 61a, b, c, and d, and sequence processing is executed to read out the data all at once, but this is not limiting.

[0049] Referring again to FIG. 2 , the distance measurement sensor 52 has a pixel drive line 63 wired horizontally for each pixel row, and two vertical signal lines 64A and 64B wired vertically for each pixel column. The pixel drive line 63 transmits a drive signal for driving each pixel 61 when reading out a detection signal VSL. While FIG. 2 shows the pixel drive line 63 as a single line, the number of lines is not limited to one. One end of the pixel drive line 63 is connected to an output terminal of the vertical drive unit 55 corresponding to each pixel row. The vertical signal line 64A is a signal line that transmits the detection signal VSLA of the first tap to the column processing unit 56, and the vertical signal line 64B is a signal line that transmits the detection signal VSLB of the second tap to the column processing unit 56.

[0050] The pulse generating circuit 53 generates light emitting pulses of a predetermined modulation frequency Fmod under the control of the control unit 51 and outputs them to the LD 21 of the lighting device 13. As described above, the modulation frequency Fmod1 of the light emitting pulses is set to 20 MHz, and the modulation frequency Fmod2 is set to 100 MHz. Note that these modulation frequencies are merely examples and are not limited to these numerical values.

[0051] Furthermore, the pulse generating circuit 53 generates drive signals GDA and GDB corresponding to the light emission pulses of the modulation frequency Fmod and supplies them to the tap driving unit 54. The drive signal GDA is a drive signal for transferring the charge generated in the photoelectric conversion unit of each pixel 61 to the first tap, and the drive signal GDB is a drive signal for transferring the charge generated in the photoelectric conversion unit of each pixel 61 to the second tap. In this embodiment, for example, the drive signals GDA and GDB are signals in which one has an inverted phase with respect to the other.

[0052] The tap driver 54 distributes the two drive signals GDA and GDB supplied from the pulse generator circuit 53 to generate drive signals GDA' and GDB' for each pixel 61, and supplies them to each pixel 61 of the distance measurement sensor 52. The tap driver 54 supplies the drive signals GDA' and GDB' to each pixel column of the distance measurement sensor 52, thereby controlling the distribution of charge to the two taps of each pixel 61. That is, the drive signals GDA' and GDB' are a combination of pulse signals having a frequency of 20 MHz and phases of [0 deg] and [180 deg] when the modulation frequency is Fmod1, or a combination of pulse signals having a frequency of 20 MHz and phases of [90 deg] and [270 deg]. That is, the combined pulse signals having phases of [0°] and [180°] are supplied to pixels 61A and 61D (see FIG. 4), and the combined pulse signals having phases of [90°] and [270°] are supplied to pixels 61B and 61C (see FIG. 4).

[0053] Similarly, drive signals GDA' and GDB' are a combination of pulse signals having a phase of [0 deg] and [180 deg] at a frequency of 100 MHz, or a combination of pulse signals having a phase of [90 deg] and [270 deg] at a modulation frequency of Fmod2. That is, the combined pulse signals having phases of [0 deg] and [180 deg] are supplied to pixels 61a and 61d (see FIG. 4), and the combined pulse signals having phases of [90 deg] and [270 deg] are supplied to pixels 61b and 61c (see FIG. 4).

[0054] The vertical drive unit 55 is a pixel drive unit that is configured with a shift register, an address decoder, etc., and drives each pixel 61 of the distance measurement sensor 52 simultaneously or row by row via pixel drive lines 63 that are wired in the horizontal direction for each pixel row. Detection signals VSLA and VSLB output from each pixel 61 in a pixel row selected and scanned by the vertical drive unit 55 are supplied as pixel signals to the column processing unit 56 via vertical signal line 64A or 64B.

[0055] The column processing unit 56 performs predetermined signal processing on pixel signals input from each pixel 61 in a selected row via the vertical signal line 64A or 64B for each pixel column of the distance measurement sensor 52, and temporarily stores the pixel signals after signal processing. For example, the column processing unit 56 performs AD (analog-digital) conversion processing of the pixel signals as signal processing. Note that the column processing unit 56 according to this embodiment corresponds to a readout unit.

[0056] The horizontal drive unit 57 is configured with a shift register, an address decoder, etc., and sequentially selects unit circuits corresponding to pixel columns in the column processing unit 56. By selective scanning by this horizontal drive unit 57, pixel signals that have been signal-processed by the column processing unit 56 are sequentially output to the signal processing unit 58.

[0057] The signal processing unit 58 has a predetermined calculation processing function, and performs predetermined calculation processing on the pixel signals output from the column processing unit 56 as necessary, and outputs the signals to the control device 11 (see Figure 1) via the output unit 59.

[0058] For example, when the operating mode is the ranging mode, the signal processing unit 58 calculates distance information to the subject based on the pixel data (detection signals VSLA and VSLB) for each tap of each pixel 61 supplied from the column processing unit 56, and performs processing to generate a depth image in which the distance information is stored as the pixel value of each pixel.

[0059] For example, when the operation mode is the 2D image capturing mode, the signal processing unit 58 performs processing to generate a luminance image of the subject based on pixel data for each tap of each pixel 61 supplied from the column processing unit 56. Details of the signal processing unit 58 will be described later.

[0060] The output unit 59 is composed of a predetermined communication interface such as MIPI (Mobile Industry Processor Interface), and outputs the depth image and brightness image, which are the results of the calculation processing by the signal processing unit 58, to the control device 11 as measurement data.

[0061] The light receiving element 31 configured as described above performs a light receiving operation in an operation mode specified by the control device 11. When a distance measurement mode is specified as the operation mode, the light receiving element 31 performs a light receiving operation by switching the modulation frequency Fmod of the light emission pulse between at least two modulation frequencies, a low-speed drive Fmod1 and a high-speed drive Fmod2, and generates and outputs a depth image. Furthermore, when a 2D image capture mode is specified as the operation mode, the light receiving element 31 generates and outputs a luminance image of the subject. In the 2D image capture mode, as in the distance measurement mode, the light receiving element 31 may or may not emit irradiation light at a predetermined modulation frequency Fmod. When irradiation light is not emitted, the output of the light emission pulse to the illumination device 13 is stopped. In this embodiment, the light receiving element 31 emits irradiation light at a second modulation frequency Fmod2 for low-speed drive and receives the light. The modulation frequency Fmod used when emitting irradiation light in the 2D image capture mode does not need to be the same as either of the modulation frequencies Fmod used when operating in the distance measurement mode.

[0062] <Pixel Circuit Configuration Example> FIG. 5 is a diagram showing a detailed circuit configuration example of a pixel 61. The pixel 61 in FIG. 5 includes a photoelectric conversion element (photodiode) 71 as a photoelectric conversion unit. The pixel 61 also includes a transfer transistor 72, an FD (floating diffusion region) 73, an FD gate transistor 74, an amplifier transistor 75, a reset transistor 76, and a selection transistor 77, two of which correspond to each of the two taps. That is, the pixel 61 includes transfer transistors 72A and 72B, FDs 73A and 73B, FD gate transistors 74A and 74B, amplifier transistors 75A and 75B, reset transistors 76A and 76B, and selection transistors 77A and 77B. Elements marked with an A are elements on the first tap side, and elements marked with a B are elements on the second tap side. The FDs 73A and 73B correspond to two taps that store the charge generated by the photoelectric conversion element 71. The pixel 61 further includes a charge discharging transistor 78. Each transistor included in the pixel 61 is configured as an N-type MOSFET. As described above, the FDs (floating diffusion regions) 73A and 73B according to this embodiment correspond to the charge accumulation portion.

[0063] The photoelectric conversion element 71 generates and accumulates electric charges according to the amount of reflected light received. The photoelectric conversion element 71 according to this embodiment corresponds to a photoelectric conversion unit.

[0064] When a drive signal GDA' supplied to its gate electrode becomes active (high), the transfer transistor 72A becomes conductive in response, thereby transferring the charge stored in the photoelectric conversion element 71 to the FD 73A. When a drive signal GDB' supplied to its gate electrode becomes active (high), the transfer transistor 72B becomes conductive in response, thereby transferring the charge stored in the photoelectric conversion element 71 to the FD 73B.

[0065] The FD 73A is a first tap charge storage unit that temporarily stores and holds the charge transferred from the photoelectric conversion element 71. The FD 73B is a second tap charge storage unit that temporarily stores and holds the charge transferred from the photoelectric conversion element 71.

[0066] When the FD drive signal FDG supplied to its gate electrode becomes active, the FD gate transistor 74A becomes conductive in response, thereby connecting the additional capacitance between the FD gate transistor 74A and the reset transistor 76A to the FD 73A. When the FD drive signal FDG supplied to its gate electrode becomes active, the FD gate transistor 74B becomes conductive in response, thereby connecting the additional capacitance between the FD gate transistor 74B and the reset transistor 76B to the FD 73B. The storage capacitance can be changed by dynamically controlling the on / off of the FD gate transistor 74 in accordance with the amount of incident light. For simplicity's sake, FIG. 3 shows a configuration in which a single FD drive signal line 81 is used, and the FD drive signal FDG is shared by the FD gate transistors 74A and 74B. However, in reality, a separate FD drive signal line 81 is provided for each of the FD gate transistors 74A and 74B, and the on / off of each is controlled so that the FD gate transistors are operated exclusively.

[0067] The amplifier transistor 75A has a source electrode connected to the vertical signal line 64A via the selection transistor 77A, and is therefore connected to a constant current source (not shown) to form a source follower circuit. The amplifier transistor 75B has a source electrode connected to the vertical signal line 64B via the selection transistor 77B, and is therefore connected to a constant current source (not shown) to form a source follower circuit.

[0068] When the reset drive signal RST supplied to its gate electrode becomes active, the reset transistor 76A becomes conductive in response, thereby resetting the potential of the FD 73A. When the reset drive signal RST supplied to its gate electrode becomes active, the reset transistor 76B becomes conductive in response, thereby resetting the potential of the FD 73B. When the reset transistors 76A and 76B are activated, the FD gate transistors 74A and 74B are also simultaneously activated. For simplicity's sake, FIG. 3 shows a configuration in which a single reset drive signal line 82 is used, and the reset drive signal RST is shared by the reset transistors 76A and 76B. However, in reality, a separate reset drive signal line 82 is provided for each of the reset transistors 76A and 76B, and the on / off state of each is controlled so that the reset transistors operate exclusively. A predetermined drain voltage RSTDRAIN is supplied to the drains of the reset transistors 76A and 76B.

[0069] The selection transistor 77A is connected between the amplifying transistor 75A and the vertical signal line 64A, and becomes conductive when a selection signal SEL supplied to its gate electrode is activated, thereby outputting a detection signal VSLA output from the amplifying transistor 75A to the vertical signal line 64A. The selection transistor 77B is connected between the amplifying transistor 75B and the vertical signal line 64B, and becomes conductive when a selection signal SEL supplied to its gate electrode is activated, thereby outputting a detection signal VSLB output from the amplifying transistor 75B to the vertical signal line 64B. For simplicity's sake, FIG. 5 shows a configuration in which a single selection signal line 83 is used, and the selection signal SEL is shared by the selection transistors 77A and 77B. However, in reality, a separate selection signal line 83 is provided for each of the selection transistors 77A and 77B, and the on / off control is performed so that each transistor operates exclusively.

[0070] When the discharge drive signal OFG supplied to the gate electrode via the discharge drive signal line 84 becomes active, the charge discharge transistor 78 becomes conductive in response to this, thereby discharging the charge accumulated in the photoelectric conversion element 71.

[0071] <Explanation of Pixel Operation and Pixel Signal> Fig. 6 is a diagram illustrating the emission of irradiation light and the driving of pixels. An example of the operation of pixel 61A will be described based on Fig. 6 while referring to Fig. 5. Here, drive signals GDA' and GDB' are an example of a combination of pulse signals having phases of [0 deg] and [180 deg].

[0072] The horizontal axis represents time, and from top to bottom, it shows the irradiated light, reflected light, drive signal GDA' with a phase of 0 degrees, and drive signal GDB' with a phase of 180 degrees. The irradiated light indicates the modulation frequency Fmod(1 / 2T) of the light emission pulse, where high level indicates high luminous intensity and low level indicates low luminous intensity. The reflected light indicates the returning light that has returned from the object 41 with a time ΔT delay from the irradiated light.

[0073] The drive signal GDA' is a gate signal supplied to the transfer transistor 72A, and the drive signal GDB' is a gate signal supplied to the transfer transistor 72B.

[0074] First, before light reception, a reset operation is performed on pixel 61A to reset the charge of pixel 61. That is, FD gate transistors 74A and 74B and reset transistors 76A and 76B are turned on to discharge the accumulated charge of FDs 73A and 73B, and charge discharge transistor 78 is turned on to discharge the accumulated charge of photoelectric conversion element 71.

[0075] After the accumulated charge is discharged, light reception begins at a modulation frequency Fmod1. Specifically, as shown in FIG. 6 , the light emitter 22 outputs irradiated light modulated to repeatedly turn on and off over an irradiation time T, and the reflected light is received by the photoelectric conversion element 71 with a delay time ΔT corresponding to the distance to the subject. One cycle (=2T) of the irradiated light is 1 / Fmod1. Furthermore, the drive signal GDA' controls the on / off of the transfer transistor 72A, and the drive signal GDB' controls the on / off of the transfer transistor 72B. For example, the drive signal GDA' is a signal in phase with the irradiated light, and the drive signal GDB' has an inverted phase from the drive signal GDA'.

[0076] 6, charges generated by the photoelectric conversion element 71 receiving reflected light are transferred to FD 73A while the transfer transistor 72A is on in accordance with the drive signal GDA', and transferred to FD 73B while the transfer transistor 72B is on in accordance with the drive signal GDB'. As a result, during a predetermined period in which irradiation light is periodically applied for the irradiation time T, the charges transferred via the transfer transistor 72A are sequentially stored in FD 73A, and the charges transferred via the transfer transistor 72B are sequentially stored in FD 73B. When the FD gate transistors 74A and 74B are on, the charges are also stored in the additional capacitances.

[0077] As described above, the pixel 61 distributes the charge generated by the reflected light received by the photoelectric conversion element 71 to the first tap (FD73A) and the second tap (FD73B) according to the delay time ΔT, and outputs the detection signal VSLA of the first tap and the detection signal VSLB of the second tap. Note that when the FD gate transistors 74A and 74B are on, the charge is also stored in the additional capacitance.

[0078] Pixel 61B is a combination of pulse signals having phases of [90°] and [270°], where the [90°] drive signal GDA' is delayed in phase by T / 2 relative to the [0°] drive signal GDA', and the [270°] drive signal GDA' is delayed in phase by T+T / 2 relative to the [0°] drive signal GDA'. Similarly, pixel 61C is a combination of pulse signals having phases of [270°] and [90°], where the [270°] drive signal GDA' is delayed in phase by T+T / 2 relative to the [0°] drive signal GDA', and the [90°] drive signal GDA' is delayed in phase by T / 2 relative to the [0°] drive signal GDA'. Similarly, pixel 61D is a combination of pulse signals having phases of [180°] and [0°], where the [180°] drive signal GDA' is delayed in phase by T relative to the [0°] drive signal GDA', and the [0°] drive signal GDA' is in phase with the [0°] drive signal GDA'. The subsequent operation is similar to that of pixel 61A, so a detailed explanation of the operation example will be omitted. Similarly, pixels 61a, 61b, 61c, and 61d differ from pixels 61A, 61B, 61C, and 61D in that one cycle of the irradiated light (=2T') is 1 / F mod 2. The explanation of the respective operations is similar to that of pixels 61A, 61B, 61C, and 61D, so will be omitted.

[0079] However, in the distance measurement sensor 52, differences in the characteristics (sensitivity differences) of each element of pixel transistors such as the photoelectric conversion element 71 and transfer transistor 72 of each pixel 61 may have different effects on the detection signals VSLA and VSLB for each pixel 61. For this reason, the distance measuring device 1 using the indirect ToF method employs a method of obtaining detection signals VSLA and VSLB obtained by receiving reflected light with different phases at the same pixel 61, thereby eliminating the sensitivity differences between the taps of each pixel 61 and improving the S / N ratio.

[0080] <Configuration of signal processing unit and arithmetic processing> Here, an example configuration and an example arithmetic processing of the signal processing unit 58 will be described with reference to Fig. 7 to Fig. 12. Fig. 7 is a block diagram showing an example configuration of the signal processing unit 58. As shown in Fig. 7, the signal processing unit 58 has a distance generation unit 580, a reliability generation unit 582, an interpolation processing unit 584, and a determination unit 586.

[0081] <Depth Value Calculation and Reliability> The distance generation unit 580 can use, for example, a four-phase detection method (four-phase method) or a two-phase detection method (two-phase method) as a method for calculating depth values.

[0082] 8 is a diagram showing the exposure periods of the first taps of pixels 61A, 61B, 61C, and 61D at phases of 0°, 90°, 180°, and 270°, arranged so that the phase differences can be easily understood. From the top, the diagram shows the irradiated light, the reflected light, and the exposure periods GDA_0, GDA_90, GDA_180, and GDA_270 of the first taps of pixels 61A, 61B, 61C, and 61D.

[0083] As shown in Figure 8, at the first tap, the detection signal VSLA obtained by receiving light at the same phase as the irradiated light (phase 0°) is called detection signal A0, the detection signal VSLA obtained by receiving light at a phase shifted by 90 degrees from the irradiated light (phase 90°) is called detection signal A90, the detection signal VSLA obtained by receiving light at a phase shifted by 180 degrees from the irradiated light (phase 180°) is called detection signal A180, and the detection signal VSLA obtained by receiving light at a phase shifted by 270 degrees from the irradiated light (phase 270°) is called detection signal A270.

[0084] Also, although not shown in the figure, at the second tap, the detection signal VSLB obtained by receiving light at the same phase as the irradiated light (phase 0°) is referred to as detection signal B0, the detection signal VSLB obtained by receiving light at a phase shifted by 90 degrees from the irradiated light (phase 90°) is referred to as detection signal B90, the detection signal VSLB obtained by receiving light at a phase shifted by 180 degrees from the irradiated light (phase 180°) is referred to as detection signal B180, and the detection signal VSLB obtained by receiving light at a phase shifted by 270 degrees from the irradiated light (phase 270°) is referred to as detection signal B270.

[0085] In a 4-phase detection method (4-phase method), the distance generation unit 580 performs calculation processing of distance information according to equations (1) to (3). That is, in the indirect ToF method, the distance generation unit 580 performs calculation of the depth value d according to the following equation (1): d = (c × ΔT) / 2 = (c × η) / 4πf (1) In equation (1), c is the speed of light, ΔT is the delay time, and f is the modulation frequency Fmod of light. Furthermore, η in equation (1) represents the phase difference [rad] of the reflected light, and is expressed by the following equation (2): η = Arctan(Q / I) (2) where η is in the range of 0≦φ<2π.

[0086] In the 4-phase method, I and Q in equation (2) are calculated by the following equation (3) using detection signals A0 to A270 and detection signals B0 to B270 obtained by setting the phase to 0°, 90°, 180°, and 270°. I and Q are signals obtained by converting the phase of the sine wave from polar coordinates to a Cartesian coordinate system (IQ plane) assuming that the luminance change of the irradiated light is a sine wave.

[0087] I=c0-c180=(A0-B0)-(A180-B180) Q=c90-c270=(A90-B90)-(A270-B270) (3)

[0088] The reliability generation unit 582 calculates the reliability Cf using the following equation (4) in both the 2-phase method and the 4-phase method.

[0089] 9 is a schematic explanatory diagram showing that the confidence Cf is constant regardless of the phase. FIG. 9(a) is a diagram showing, from above, the irradiated light, c0, c180, c90, and c270. FIG. 9(b) is a diagram showing, from above, the irradiated light, c0, c180, c90, and c270, and also showing the external light (ambient light) La100 and LA102. FIG. 9(c) is a diagram showing, from above, the irradiated light, c0, c180, c90, and c270, and the confidence cnf, and also showing the absolute values ​​La104 and LA106 of the external light (ambient light) La100 and LA102.

[0090] As shown in Fig. 9, in the 4-phase method, by taking the difference between detection signals of opposite phases at the same pixel, such as "A0-A180" or "A90-A270" in equation (3), it is possible to eliminate the characteristic variations between taps present in each pixel, i.e., the sensitivity differences between taps and the difference in external light. As a result, as shown in Fig. 9(c), the reliability Cf becomes a constant value regardless of the phase. Cf = Sqrt(I x I + Q x Q) (4)

[0091] In this way, the level of reliability Cf is calculated by multiplying the integral time of pixel 61 by the duty of the light source by the reflectance of the subject. As can be seen from equation (4), reliability Cf corresponds to the magnitude of the reflected light received by pixels 61A, 61B, 61C, and 61D, i.e., brightness information (brightness value).

[0092] On the other hand, in the 2-phase system, the distance generation unit 580 executes calculation processing of distance information according to equations (1), (2), and (5). In the 2-phase system, I and Q in equation (2) can be calculated using detection signals of two phases, phase 0° and phase 90°. That is, I and Q in equation (2) in the 2-phase system are expressed by the following equation (5). These values ​​are substituted into equations (1) and (2) to calculate distance information. I=c0-c180=(A0-B0) Q=c90-c270=(A90-B90) (5)

[0093] Although the 2-phase method cannot eliminate the characteristic variations between taps present in each pixel, it can obtain the depth value d to the object using only the detection signals of two phases, and therefore can measure distances with twice the resolution of the 4-phase method. The characteristic variations between taps can be adjusted using correction parameters such as gain and offset, for example.

[0094] FIG. 10 is a time chart showing a driving example according to this embodiment. From the top, the diagram shows the light emission amount of the light source, the driving state, the GDA1 signal, the GDB1 signal, the OFG1 signal, the GDA2 signal, the GDB2 signal, and the OFG2 signal. The horizontal axis represents time. The light emission amount of the light source, the GDA1 signal, the GDB1 signal, the OFG1 signal, the GDA2 signal, the GDB2 signal, and the OFG2 signal represent high-level and low-level signals, respectively. Note that, as described above, the phases of the GDA1 signal and the GDB1 signal differ for each of the pixels 61A, B, C, and D, and the phases of the GDA2 signal and the GDB2 signal differ for each of the pixels 61a, b, c, and d, but for ease of explanation, FIG. 9 shows only one set of signals as a representative.

[0095] 4 and 5, as shown in FIG. 10, at time T0, light emission of Fmod1=20 MHz is started, and the pixels 61A, B, C, and D enter a charge accumulation state (Integration 1). That is, a low-level OFG1 signal is applied to the gates of the charge drain transistors 78 in the pixels 61A, B, C, and D. As a result, the photoelectric conversion elements 71 in the pixels 61A, B, C, and D start photoelectric conversion.

[0096] At this time, phase-shifted signals, the GDA1 signal and the GDB1 signal, are supplied at 20 MHz to the gates of the transfer transistors 72A and 72B of the pixels 61A, B, C, and D. As a result, in the pixels 61A, B, C, and D, the charge corresponding to the potential of the photoelectric conversion element 71 is distributed at 20 MHz to the first tap FD73A and the second tap FD73B.

[0097] Meanwhile, a high-level OFG2 signal is applied to the gates of the charge discharging transistors 78 in the pixels 61 a, b, c, and d. As a result, the potential VDD is applied to the cathodes of the photoelectric conversion elements 71 in the pixels 61 a, b, c, and d, maintaining the initial state. Similarly, a low-level signal is supplied to the gates of the transfer transistors 72A and 72B in the pixels 61 a, b, c, and d, maintaining a disconnected state.

[0098] At time T1, a high-level OFG1 signal is applied to the gate of the charge discharging transistor 78 in each of the pixels 61A, B, C, and D, and potential VDD is applied to the cathode of the photoelectric conversion element 71, thereby returning to the initial state and ending the charge accumulation state (Integration 1). Also, a time T / 2 before time T1, a low-level signal is supplied to the gates of the transfer transistors 72A and 72B in each of the pixels 61A, B, C, and D, causing them to enter a disconnected state. These states are maintained until time T4.

[0099] At time T2, the blank period of the light-emitting unit 22 ends, light emission at Fmod2=100 MHz begins, and the pixels 61a, b, c, and d enter a charge accumulation state (Integration 2). That is, a low-level OFG2 signal is applied to the gates of the charge drain transistors 78 in the pixels 61a, b, c, and d. As a result, the photoelectric conversion elements 71 in the pixels 61a, b, c, and d start photoelectric conversion.

[0100] At this time, phase-inverted signals, the GDA2 signal and the GDB3 signal, are supplied at 100 MHz to the gates of the transfer transistors 72A and 72B of the pixels 61a, 61b, 61c, and 61d, respectively. As a result, in the pixels 61a, 61b, 61c, and 61d, charges corresponding to the potential of the photoelectric conversion elements 71 are distributed at 100 MHz to the first tap FD73A and the second tap FD73B.

[0101] At time T3, a high-level OFG2 signal is applied to the gate of the charge discharging transistor 78 in each of the pixels 61a, 61b, 61c, and 61d, and potential VDD is applied to the cathode of the photoelectric conversion element 71, thereby returning to the initial state and terminating the charge accumulation state (Integration 2). T / 2 hours before time T3, a low-level signal is supplied to the gates of the transfer transistors 72A and 72B in each of the pixels 61a, 61b, 61c, and 61d, causing them to become disconnected. These states are maintained until time T4.

[0102] At time T3, high-level signals are sequentially or simultaneously supplied to transistors 77A and 77B of pixels 61A, B, C, and D and pixels 61a, b, c, and d, respectively, to read out the accumulated charges. In this manner, the accumulated charges of pixels 61A, B, C, and D and pixels 61a, b, c, and d are read out during a data readout period from time T3 to t5. The periods from time T0 to T1 and from time T2 to T3 are set to, for example, 0.1 to 1 ms. The period from time T1 to t2 is set to, for example, 0.1 ms, and the data readout period from time T3 to T4 is set to, for example, 5 ms. As can be seen from these, the data readout period from time T3 to T4 accounts for a large proportion of the processing time required to generate distance information, for example, approximately 80%. Furthermore, because Fmod1=20 MHz data is relatively more robust against distance phase noise than Fmod2=100 MHz data, stable Dual Freq operation is possible by first acquiring low-frequency data and then acquiring high-frequency data. Furthermore, by having each pixel 61 distribute signal charge only when it is desired to receive reflected light from the object 41, and by keeping the charge drain transistor 78 conductive during other periods, signal charge generated by sunlight or ambient light from other distance measuring sensors is drained, minimizing degradation of distance measurement accuracy due to ambient light.

[0103] <Combinations and Depth Value Calculation> Fig. 11 is a diagram showing an example of combinations of pixels 61A, B, C, and D used by the interpolation processing unit 584 with pixels 61a, b, c, and d. Similar to Fig. 3, Fig. 11(a) is a diagram showing an example of the functional arrangement of pixels in the distance measurement sensor 52, and Fig. 11(b) is a diagram showing an example of combinations of pixels 61A, B, C, and D with pixels 61a, b, c, and d in the interpolation calculation of the interpolation processing unit 584. In this embodiment, the interpolation calculation of the interpolation processing unit 584 uses, for example, combinations of adjacent pixels 61A, B, C, and D with pixels 61a, b, c, and d.

[0104] Here, the combination of pixels 61A, B, C, and D and pixels 61a, b, c, and d is represented by Grm, n. That is, n indicates the row and m indicates the column. In Fig. 11B, when n indicates an even number, it corresponds to pixels 61A, B, C, and D, and when n indicates an odd number, it corresponds to pixels 61a, b, c, and d.

[0105] The signals read out from the pixels 61A, B, C, and D are converted into digital signals (A0, A90, A180, A270) and (B0, B90, B180, B270). The distance generation unit 580 calculates a first distance value (Coarse Depth) ψm,n (n is an even number) for each of the pixels 61A, B, C, and D according to equation (1). Then, the reliability generation unit 582 calculates a reliability Cfm,n (n is an even number) according to equation (4).

[0106] Meanwhile, signals read from pixels 61a, b, c, and d are converted into digital signals (A0, A90, A180, A270) and (B0, B90, B180, B270). The distance generation unit 580 calculates a second distance value distance (Fine Depth) Φm,n (n is an odd number) for each pixel 61a, b, c, and d according to equation (1). The reliability generation unit 582 then calculates a reliability Cfm,n (n is an odd number) according to equation (4). The interpolation processing unit 584 performs, for example, an interpolation calculation using the upper and lower distance values, as shown in FIG. 11(b).

[0107] 12 is a diagram schematically illustrating a method of calculating distance values ​​in the interpolation processing unit 584 using a first distance value (Coarse Depth) and a second distance value (Fine Depth). The horizontal axis represents the measured distance value (Distance), and the vertical axis represents the first distance value (Coarse Depth) and the second distance value (Fine Depth).

[0108] For example, when calculating the distance value of Grm,n (n is an even number), the values ​​of ψm,n (n is an even number), Φm,n-1, and Φm,n+1 are used. The number of cycles of the second distance value is determined by ψm,n (n is an even number). In Figure 11, the repetition cycle is 2 cycles, and the second distance value is calculated as (Φm,n-1 + Φm,n+1) / 2.

[0109] As described above, the first distance value (Coarse Depth) corresponds to 7.5 m for 360 degrees (one cycle), and the ranging accuracy of the second distance value (Fine Depth) corresponds to 1.5 m for 360 degrees (one cycle). In other words, in Figure 11, the repeating cycle is the second cycle, and the distance value (Distance) of Grm,n (n is an even number) is 1.5 m + (Φm,n-1 + Φm,n+1) / 2.

[0110] Furthermore, for example, when calculating the distance value of Grm,n (n is an odd number), the values ​​of Φm,n (n is an odd number), ψm,n-1, and ψm,n+1 are used. The cycle of the second distance value is determined by (ψm,n-1 + ψm,n+1) / 2. In FIG. 11, this is the second cycle, and the second distance value is determined by Φm,n (n is an odd number). That is, in FIG. 11, the repetition cycle is the second cycle, and 1.5m + Φm,n (n is an odd number) is the distance value (Distance) of Grm,n (n is an odd number). In this way, the signal processing unit 58 generates a distance measurement value (Distance) based on the second phase difference η2 (see Equation 1) and the repetition period of the second phase difference η2 (see Equation 1) calculated based on the signals of the pixels 61a, b, c, and d corresponding to the first phase difference η1 (see Equation 1) calculated based on the signals of the pixels 61A, B, C, and D. This type of processing is sometimes referred to as de-aliasing processing.

[0111] <Conditions for Interpolation Calculation> Here, an example of the determination process of the determination unit 586 will be described with reference to FIGS. 13 and 14 . The determination unit 586 determines whether or not to perform the interpolation calculation of the interpolation processing unit 584. FIG. 13 is a diagram showing the relationship between the reflectance of the object 41 and the reliability Cf. The horizontal axis represents the distance value (Distance), and the vertical axis represents the reliability Cf. As described above, the level of the reliability Cf is calculated by multiplying the integration time (Integ) of the pixel 61 by the light source duty (Duty) by the subject reflectance divided by the square of the distance value. In other words, the reliability Cf100 from an object 41 with high reflectance is inversely proportional to the square of the distance value and exhibits a higher value than the reliability Cf102 from an object 41 with low reflectance.

[0112] 14 is a diagram showing the reliability Cf when two types of measurement light, 100 MHz and 20 MHz, are used. The horizontal axis represents the distance value (Distance), and the vertical axis represents the signal (Signal) indicating the value of the reliability Cf. As shown in FIG. 14, the reliability Cf does not depend on the frequency of the measurement light.

[0113] Therefore, in the determination unit 586 according to this embodiment, when performing the interpolation calculation shown in FIG. 11 , the determination unit 586 refers to the value of the reliability Cf to determine whether or not to perform the interpolation calculation. For example, when calculating the distance value of Grm,n (n is an odd number), if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n+1 is less than a threshold th1, the determination unit 586 determines to use the value of ψm,n+1 for interpolation, and if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n-1 is less than the threshold th1, the determination unit 586 determines to use the value of ψm,n-1 for interpolation. On the other hand, if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n-1 is equal to or greater than the threshold th1, the determination unit 586 determines not to use the value of ψm,n+1 for interpolation. In other words, if the determination unit 586 determines not to use the values ​​of ψm,n-1 and ψm,n-1 for interpolation, the interpolation processing unit 584 outputs only the second distance value (Fine Depth). In this way, if the value of the reliability Cf calculated based on the four adjacent pixels is less than a predetermined value, the interpolation processing unit 584 determines that the light is reflected from the same object. Then, the interpolation processing unit 584 performs an interpolation calculation.

[0114] Similarly, when calculating the distance value of Grm,n (n is an even number), for example, if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n+1 is less than threshold th1, the determination unit 586 determines to use the value of Φm,n+1 for interpolation, and if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n-1 is less than threshold th1, the determination unit 586 determines to use the value of Φm,n-1 for interpolation. On the other hand, if the absolute value of the difference between the values ​​of Cfm,n and Cfm,n-1 is equal to or greater than threshold th1, the determination unit 586 determines not to use the value of Φm,n+1 for interpolation. In other words, if the determination unit 586 determines that the values ​​of Φm,n-1 and Φm,n-1 are not to be used for interpolation, the interpolation processing unit 584 outputs only the first distance value (Coarse Depth).

[0115] <Comparative Example 1> Figure 15 is a diagram schematically illustrating the driving state of Comparative Example 1. The diagram shows the light emission state of the light source from the receiver and the driving state of the pixel 61. The horizontal axis represents time. In a general indirect ToF method such as Comparative Example 1, data is accumulated and read out in a time-division manner in one pixel 61. That is, using a modulation frequency Fmod1, data is accumulated and read out sequentially at four frames t0, t1, t2, and t3 corresponding to four phases (0 degrees, 90 degrees, 180 degrees, 270 degrees) and (180 degrees, 270 degrees, 0 degrees, 90 degrees).

[0116] Similarly, using modulation frequency Fmod2, data accumulation and readout are performed sequentially for four frames t4, t5, t6, and t7 corresponding to four phases (0 degrees, 90 degrees, 180 degrees, 270 degrees) and (180 degrees, 270 degrees, 0 degrees, 90 degrees). Thus, when measuring distances at multiple modulation frequencies Fmod1 and Fmod2, light emission and readout for at least eight frames are required to output one depth image. The readout time is, for example, approximately five times the data accumulation time.

[0117] 16 is a diagram schematically illustrating the relationship between an object 41 and depth values ​​(distance measurement values) in Comparative Example 1. The object 41 is a moving object, and the readout time requires, for example, about five times the data accumulation time, so the depth value changes over eight frames t0, t1, t2, t3, t4, t5, t6, and t7.

[0118] <Comparative Example 2> Fig. 17 is a diagram schematically illustrating the driving state of Comparative Example 2. The diagram shows the light emission state of the light source from the receiver and the driving state of the pixel 61. The horizontal axis represents time. Comparative Example 2 is an example that has pixels 61A, B, C, and D, as in this embodiment, but does not have pixels 61a, b, c, and d. That is, after pixels 61A, B, C, and D accumulate and read one frame t0 corresponding to four phases using a modulation frequency Fmod1, pixels 61A, B, C, and D further accumulate and read one frame t1 corresponding to four phases using a modulation frequency Fmod2.

[0119] 18 is a diagram showing a schematic diagram of the relationship between an object 41 and depth values ​​(distance measurement values) in Comparative Example 2. The object 41 is a moving object, and the readout time is, for example, about five times the data accumulation time, so the depth value changes between two frames t0 and t1.

[0120] <Present Embodiment> FIG. 19 is a diagram schematically illustrating the relationship between an object 41 and depth values ​​(distance measurements) according to this embodiment. The distance measurement sensor 52 according to this embodiment configures pixels 61A, B, C, and D and pixels 61a, b, c, and d as independently functioning pixels 61. This allows for simultaneous accumulation of charge. Therefore, as shown in FIG. 10, distance information can be generated without data readout during the blank period from time T1 to time T2. This shortens the time between time T0, when charge accumulation for pixels 61A, B, C, and D begins, and time T3, when charge accumulation for pixels 61a, b, c, and d ends. This further reduces the difference in distance values ​​between pixels 61A, B, C, and D and pixels 61a, b, c, and d, thereby improving the accuracy of distance information. In this way, there is no blurring of the object 41, and data can be read out all at once, contributing to low power consumption.

[0121] <Other Drive Modes> The drive mode can be changed by the drive signal supplied from the tap drive unit 54 to each pixel 61 of the distance measurement sensor 52 .

[0122] FIG. 20 is a diagram showing three driving modes. In FIG. 20, four adjacent pixels 61A, B, C, and D are shown as a pixel group 610, and four adjacent pixels 61a, b, c, and d are shown as a pixel group 612. FIG. 20(a) is an example of a horizontal stripe configuration similar to FIG. 4. In FIG. 20(a), pixel groups 610 and 612 are configured for each row. FIG. 20(b) is an example of a checkerboard configuration, in which pixel groups 610 and 612 are configured in a checkerboard pattern. FIG. 20(c) is an example of a horizontal stripe configuration, in which pixel groups 610 and 612 are configured for each column.

[0123] FIG. 21 illustrates three driving modes different from those illustrated in FIG. 20 . Four adjacent pixels 61D, C, B, and A are represented by pixel group 610a, and four adjacent pixels 61d, c, b, and a are represented by pixel group 612a. In other words, the four phases are changed from the arrangement order of (0 degrees, 90 degrees, 180 degrees, and 270 degrees) to the arrangement order of (180 degrees, 270 degrees, 0 degrees, and 90 degrees). In FIG. 21( a), pixel groups 610a and 612a are arranged in rows. In FIG. 21( b), a checkerboard arrangement is shown, in which pixel groups 610a and 612a are arranged in a checkerboard pattern. In FIG. 21( c), a horizontal stripe arrangement is shown, in which pixel groups 610a and 612a are arranged in columns. In this way, the control unit 51 can change the driving mode depending on the imaging purpose.

[0124] Fig. 22 is a diagram showing an example of interpolation processing in the arrangement example of Fig. 20(b). Fig. 22(a) is a diagram showing an example of a checkerboard configuration similar to Fig. 20(b). Fig. 22(b) is a diagram schematically showing a group of adjacent pixels to be interpolated.

[0125] FIG. 23 is a diagram showing an example of interpolation processing for a pixel group 610. FIG. 23( a) shows an example of a pixel group 610 of interest, and FIG. 23( b) shows four adjacent pixel groups 612 surrounding the pixel group 610 of interest. In the checkered configuration examples shown in FIGS. 20( b) and 21( b), it is possible to use the four surrounding pixel groups for interpolation. That is, for the pixel group 610, as shown in FIG. 23( b), if the absolute value of the difference in reliability Cf between the four adjacent pixel groups 612 is less than threshold th1, the determination unit 586 determines that the pixel group 610 should be used for interpolation, and if the absolute value is equal to or greater than threshold th1, it determines that the pixel group 610 should not be used for interpolation. As described above, since reliability Cf does not depend on modulation frequency, it returns approximately the same value for objects 41 that are equidistant and have the same reflectance, even if the modulation frequencies are different. Therefore, by using the correlation of the reliability C values ​​to make a direction judgment (a judgment to select the group of adjacent pixels to be used for interpolation), it becomes possible to perform de-aliasing processing with less degradation in the quality of the distance image than when simply interpolating using the group of adjacent pixels in the surrounding area.

[0126] 24A and 24B are diagrams showing an example of interpolation processing for a pixel group 612. FIG. 24A shows an example of a pixel group 612 of interest, and FIG. 24B shows four adjacent pixel groups 612 surrounding the pixel group 612 of interest, denoted by 6100, 6102, 6104, and 6106. In FIG. 23B, the difference in absolute value between the reliability Cf of the pixel group 612 and the reliability of the pixel groups 6104 and 6106 is equal to or greater than the threshold Th1, and the difference in absolute value between the reliability of the pixel groups 6100 and 6102 is less than the threshold Th1. In this case, the determination unit 586 determines that the pixel groups 6100 and 6102 should be used to interpolate the pixel group 610 of interest. In this way, for a pixel group 612, if the absolute value of the difference in reliability Cf between the four adjacent pixel groups 610 is less than the threshold value th1, the determination unit 586 determines that the pixel group 612 should be used for interpolation, and if the absolute value is greater than or equal to the threshold value th1, the determination unit 586 determines that the pixel group 612 should not be used for interpolation.

[0127] 25 is a flowchart showing an example of control according to this embodiment. Here, an example of a checkerboard configuration shown in FIG. 20(b) will be described.

[0128] First, under the control of the control unit 51, the pulse generation circuit 53 supplies a light emission pulse of Fmod1=20 MHz to the LD 21, and light emission of Fmod1=20 MHz begins (step S100). Next, the pixels 61A, 61B, 61C, and 61D enter a charge accumulation state (Integration 1) (step S102).

[0129] Next, under the control of the control unit 51, the pulse generation circuit 53 supplies a light emission pulse of Fmod2=100 MHz to the LD 21, and light emission of Fmod2=100 MHz begins (step S104). Next, the pixels 61a, 61b, 61c, and 61d enter a charge accumulation state (Integration 1) (step S106).

[0130] Next, the reliability generation unit 582 of the signal processing unit 58 calculates the reliability Cf as the AC light level for each pixel group 610, 612 according to equation (4) (step S108). The determination unit 586 of the signal processing unit 58 calculates the absolute value of the difference in reliability Cf between the four adjacent pixel groups 610 and 612 for all pixel groups 610, 612, and determines whether or not to perform de-aliasing processing (step S110).

[0131] Next, the distance generation unit 580 calculates distance information for all pixel groups 610 and all pixel groups 612 (step S108). Subsequently, the reliability generation unit 582 of the signal processing unit 58 calculates reliability Cf as an AC light level for each of the pixel groups 610 and 612 according to equation (4) (step S110).

[0132] The determination unit 586 calculates the absolute value of the difference between the reliability Cf of the four adjacent pixel groups 610 and the reliability Cf of the four adjacent pixel groups 612, and determines whether the absolute value is less than a threshold value th1 (step S112). If the absolute values ​​of the difference values ​​between the four adjacent pixel groups are all equal to or greater than the threshold value th1 (N in step S112), the interpolation processing unit 584 outputs the distance information for each of the pixel groups 610 and 612 without performing de-aliasing (step S114), and ends the process.

[0133] On the other hand, if the determination unit 586 determines that any of the four-adjacent pixel groups is less than the threshold value th1 (Y in step S112), it generates a distance image of the pixel group (see (b) in FIG. 22) (step S116). Next, the determination unit 586 determines the interpolation direction based on the values ​​of the four-adjacent pixel groups. For example, if the values ​​of all four-adjacent pixel groups are less than the threshold value th1, it selects the horizontal or vertical direction (step S116). In this case, if the values ​​of three pixel groups out of the four-adjacent pixel groups are to be used, any two pixel groups may be selected. Alternatively, all four-adjacent pixel groups may be selected.

[0134] Next, the interpolation processing unit 584 calculates the average value of the pixel group selected in step S116 as an interpolated intensity value (step S118), and then performs de-aliasing processing on all pixel groups 610 and 612 using the interpolated intensity value (step S120), thereby terminating the processing.

[0135] As described above, according to this embodiment, the pixels 61A, B, C, and D corresponding to the modulation frequency Fmod1 and the pixels 61a, b, c, and d corresponding to the modulation frequency Fmod2 are mixed in the distance measurement sensor 52. This makes it possible to drive the pixels 61A, B, C, and D and the pixels 61a, b, c, and d independently, and it is possible to control the switching period during which the illumination light having the modulation frequency Fmod1 is switched to the second illumination light having the modulation frequency Fmod2 to be shorter than the readout period during which signals are read from the pixels 61A, B, C, and D and the pixels 61a, b, c, and d after illumination with the illumination light having the modulation frequency Fmod2. This makes it possible to reduce the difference in distance (blur) between the position of the object 41 during the measurement period at the modulation frequency Fmod1 and the position of the object 41 during the measurement period at the modulation frequency Fmod2, based on the position of the object 41 during the measurement period at the modulation frequency Fmod1, and to reduce the decrease in the measurement accuracy of the distance value generated based on the phase difference.

[0136] (Variation 1 of First Embodiment) The distance measuring device 1 according to Variation 1 of the first embodiment differs from the distance measuring device 1 according to the first embodiment in that the overflow gate is turned on / off at the same high frequency during the period when ambient light is being discharged as during the period when signal charges are being distributed. The differences from the distance measuring device 1 according to the first embodiment will be described below.

[0137] 26 is a time chart showing a driving example according to Modification 1 of the first embodiment. From the top, the diagram shows the light emission amount of the light source, the driving state, the GDA1 signal, the GDB1 signal, the OFG1 signal, the GDA2 signal, the GDB2 signal, and the OFG2 signal. The horizontal axis shows time. The light emission amount of the light source, the GDA1 signal, the GDB1 signal, the OFG1 signal, the GDA2 signal, the GDB2 signal, and the OFG2 signal respectively indicate high-level and low-level signals.

[0138] 26, during the period from time T1 to time T3, the OFG1 signal is toggled to repeatedly turn on and off the charge discharging transistor 78. Similarly, during the period from time T0 to time T2, the OFG2 signal is toggled to repeatedly turn on and off the charge discharging transistor 78.

[0139] In this way, the charge drain transistor 78 is turned on / off at the same high frequency during the period when disturbance light is being discharged as during the period when signal charge is being distributed, thereby reducing the temporal fluctuation of power supply fluctuations and further improving the stability of distance measurement performance.

[0140] Second Embodiment The distance measuring device 1 according to the second embodiment differs from the distance measuring device 1 according to the first embodiment in that the Fmod2=100 MHz side is calculated using a 4-phase method, and the Fmod1=20 MHz side is calculated using a 2-phase method. The differences from the distance measuring device 1 according to the first embodiment will be described below.

[0141] Fig. 27 is a diagram showing an example of the configuration of the distance measurement sensor 52 according to the second embodiment. As shown in Fig. 27, a pixel group 612 of pixels 61a, 61b, 61c, and 61d constitutes a whole pixel group 612.

[0142] 28 is a time chart showing a driving example according to the second embodiment. From the top, the diagram shows the light emission intensity of the light source, the driving state, the TAPA signal, the TAPB signal, the TAPC signal, and the TAPD signal. The horizontal axis shows time. The light emission intensity of the light source, the TAPA signal, the TAPB signal, the TAPC signal, and the TAPD signal respectively indicate high-level and low-level signals. The TAPA signal, the TAPB signal, the TAPC signal, and the TAPD signal indicate gate signals of the transfer transistors 72A, 72B, 72C, and 72D. The transfer transistors 72A and 72B correspond to a phase of [0 degrees, 180 degrees], and the transfer transistors 72A and 72B correspond to a phase of [90 degrees, 270 degrees].

[0143] The distance generation unit 580 calculates the depth value d on the Fmod2=100 MHz side according to equations (1) to (3). On the other hand, the distance generation unit 580 calculates the depth value d on the Fmod1=20 MHz side according to equations (1), (2), and (5). In this case, the signal values ​​of TAPA+TAPB and TAPC+TAPD are used. That is, the signal value of TAPA+TAPB is the signal value of FD73A+FD73B, and the signal value of TAPC+TAPD is the signal value of FD73C+FD73D.

[0144] In this way, it is possible to obtain phase information of the low frequency component on the Fmod1=20 MHz side using the sum signal of [0+180] and [90+270] degrees. This enables uniform driving of the pixel group 612 and also makes it possible to eliminate the need for interpolation for spatial distribution.

[0145] (Third Embodiment) The distance measuring device 1 according to the third embodiment differs from the distance measuring device 1 according to the first embodiment in that distance measuring information is acquired twice at Fmod1=20 MHz and averaged over time. The following describes the differences from the distance measuring device 1 according to the first embodiment.

[0146] FIG. 29 is a time chart showing an example of driving according to the third embodiment. From the top, the light emission amount and driving state of the light source are shown. FIG. 29( a) is a diagram showing the relationship between the distance value L100 of the pixel group 610 in the first measurement, the distance value L102 of the pixel group 610 in the second measurement, and the average value L104. FIG. 29( b) is a diagram showing the results of de-aliasing processing using the distance value L106 and average value L104 of the pixel group 612. In this way, the signal processing unit 58 can eliminate the center of gravity shift (blur) of the object 41 by performing de-aliasing processing using the distance value L100 of the pixel group 610 in the first measurement, the average value L104 of the distance values ​​L102 of the pixel group 610 in the second measurement, and the distance value L106 of the pixel group 612.

[0147] (Fourth embodiment) The distance measuring device 1 according to the fourth embodiment differs from the distance measuring device 1 according to the first embodiment in that it has three or more types of modulation frequencies Fmod. The differences from the distance measuring device 1 according to the first embodiment will be described below.

[0148] The distance measurement sensor 52 of the distance measuring device 1 according to the fourth embodiment has a first pixel group corresponding to a modulation frequency Fmod1=20 MHz, a second pixel group corresponding to a modulation frequency Fmod2=100 MHz, and a third pixel group corresponding to a modulation frequency FmodN=200 MHz. The modulation frequency FmodN is, for example, 200 MHz, but is not limited to this.

[0149] FIG. 30 is a time chart showing an example of driving the distance measuring device 1 according to the fourth embodiment. From the top, the light emission amount and driving state of the light source are shown. The horizontal axis represents time. As shown in FIG. 30, distance information is generated at a modulation frequency Fmod1=20 MHz, then at a modulation frequency Fmod2=100 MHz, and then at a modulation frequency FmodN=200 MHz. In this way, it is possible to obtain information with three or more types of distance accuracy.

[0150] Fifth Embodiment A distance measuring device 1 according to a fifth embodiment differs from the distance measuring device 1 according to the first embodiment in that the pixel 61 is configured with 4TAPs. The differences from the distance measuring device 1 according to the first embodiment will be described below.

[0151] 31 is a diagram showing an example of a pixel configured with 4TAP. Pixels 614 and 616 configured with 4TAP have the same configuration. In the following description, driving with a modulation frequency Fmod1 = 20 MHz will be indicated by capital letters D, E, F, and G, and driving with a modulation frequency Fmod2 = 100 MHz will be indicated by lowercase letters d, e, f, and g. Furthermore, with respect to the reference phase of the light emission pulse, a phase of 0 degrees will be indicated by D and d, a phase of 90 degrees will be indicated by E and e, a phase of 180 degrees will be indicated by G and g, and a phase of 270 degrees will be indicated by E and e.

[0152] As shown in FIG. 31 , pixel 614 is driven at a modulation frequency Fmod1=20 MHz. Electrons generated by a single photoelectric conversion unit are distributed to four taps 73D, 73E, 73F, and 73G via transfer transistors 72D, 72E, 72F, and 72G, which store electric charges. That is, drive signals with a modulation frequency Fmod1=20 MHz, which have phase differences of 0 degrees, 90 degrees, 180 degrees, and 270 degrees with respect to the reference phase of the light-emitting pulse, are supplied to the gates of transfer transistors 72D, 72E, 72F, and 72G. That is, when a pixel is configured with 2 taps, one photoelectric conversion unit is required for each of the two taps, whereas when a pixel is configured with 3 taps, one photoelectric conversion unit is required for each of the four taps. Thus, using a pixel configured with 4 taps allows for a smaller distance measurement sensor 52.

[0153] On the other hand, pixel 616 is a pixel driven at a modulation frequency Fmod2=100 MHz, and distributes electrons generated in the photoelectric conversion unit to four taps 73d, 73e, 73f, and 73g that accumulate electric charges via transfer transistors 72d, 72e, 72f, and 72g. That is, drive signals with a modulation frequency Fmod2=100 MHz and phase differences of 0, 90, 180, and 270 degrees with respect to the reference phase of the light emission pulse are supplied to the gates of the transfer transistors 72d, 72e, 72f, and 72g.

[0154] FIG. 32 shows an example of interpolation processing for the example arrangement of FIG. 31. FIG. 32(a) shows an example of a checkerboard configuration similar to that of FIG. 31. FIG. 32(b) is a schematic diagram showing adjacent pixel groups to be interpolated. In this way, using pixels 614 and 616 configured with 4TAP, it is possible to generate a distance image similar to that of pixel groups 610 and 612 configured with 2TAP using a single photoelectric conversion unit. <4TAP Drive Mode> The drive mode can be changed by the drive signal supplied from the tap drive unit 54 to each pixel of the distance measurement sensor 52.

[0155] Fig. 33 shows three driving modes. Fig. 33(a) is a checkerboard configuration example, which is equivalent to Fig. 31. Fig. 32(b) is an example of horizontal stripes, in which pixels 614 and 616 are configured for each row. Fig. 33(c) is an example of vertical stripes, in which pixels 614 and 616 are configured for each column.

[0156] Figure 34 shows three driving modes different from those shown in Figure 33. Pixel 618 has transfer transistors 72D and 72E driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72d and 72e driven at a modulation frequency Fmod2 = 100 MHz. On the other hand, pixel 620 has transfer transistors 72F and 72G driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72f and 72g driven at a modulation frequency Fmod2 = 100 MHz.

[0157] Fig. 34(a) shows an example of horizontal stripes, where pixels 618 and 620 are arranged in each row. Fig. 34(b) shows an example of vertical stripes, where pixels 618 and 620 are arranged in each column. Fig. 34(c) shows an example of a checkerboard configuration. In this way, gate transistors driven at different modulation frequencies may be configured within the same pixels 618, 620.

[0158] Figure 35 shows three driving modes that are further different from those shown in Figure 33. Pixel 622 has transfer transistors 72D and 72G driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72d and 72g driven at a modulation frequency Fmod2 = 100 MHz. On the other hand, pixel 624 has transfer transistors 72F and 72E driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72f and 72e driven at a modulation frequency Fmod2 = 100 MHz.

[0159] Fig. 35(a) is an example of horizontal stripes, where pixels 622 and 624 are arranged in each row. Fig. 35(b) is an example of vertical stripes, where pixels 622 and 624 are arranged in each column. Fig. 35(c) is an example of a checkerboard configuration. In this way, gate transistors driven at different modulation frequencies may be configured within the same pixel 622 and pixel 624.

[0160] Figure 36 shows three driving modes that are further different from those shown in Figure 33. Pixel 626 has transfer transistors 72D and 72G driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72d and 72g driven at a modulation frequency Fmod2 = 100 MHz. On the other hand, pixel 624 has transfer transistors 72F and 72E driven at a modulation frequency Fmod1 = 20 MHz and transfer transistors 72f and 72e driven at a modulation frequency Fmod2 = 100 MHz.

[0161] Fig. 36(a) is an example of horizontal stripes, where pixels 622 and 624 are arranged in each row. Fig. 36(b) is an example of vertical stripes, where pixels 622 and 624 are arranged in each column. Fig. 36(c) is an example of a checkerboard configuration. In this way, gate transistors driven at different modulation frequencies may be configured within the same pixel 624 and pixel 626.

[0162] The present technology can be configured as follows:

[0163] (1) A distance measuring device that sequentially irradiates first irradiation light of a first modulation frequency and second irradiation light of a second modulation frequency different from the first modulation frequency, and generates a distance value using reflected light reflected by an object, comprising: a distance measuring sensor having a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency; a control unit that controls a switching period for switching from the first irradiation light to the second irradiation light to be shorter than a readout period in which a first signal corresponding to the accumulated charge in the first charge accumulation unit and a second signal corresponding to the accumulated charge in the second charge accumulation unit are read out after irradiation with the second irradiation light; and a signal processing unit that generates a first phase difference based on the first signal, which indicates the time until the light is received as reflected light, and generates a second phase difference based on the second signal, which indicates the time until the light is received as reflected light, and generates a distance measurement value using the first phase difference and the second phase difference.

[0164] (2) The distance measuring device according to (1), wherein the distance measuring sensor includes a plurality of pixel groups each having the first charge accumulation portion and a plurality of pixel groups each having the second charge accumulation portion.

[0165] (3) The distance measuring sensor further has a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit, and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit makes the first charge discharging transistor conductive and the second charge discharging transistor non-conductive during the period when the second irradiation light is irradiated, and makes the first charge discharging transistor non-conductive and the second charge discharging transistor conductive during the period when the first irradiation light is irradiated, in the distance measuring device described in (1).

[0166] (4) The distance measuring sensor further has a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit, and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit causes the first charge discharging transistor to alternate between a non-conductive state and a conductive state during the period in which the second irradiation light is irradiated, and causes the second charge discharging transistor to alternate between a non-conductive state and a conductive state during the period in which the first irradiation light is irradiated, in the distance measuring device described in (2).

[0167] (5) The ranging device described in (1), wherein a first pixel having the first photoelectric conversion unit accumulates charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the first modulation frequency, and accumulates charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency; a second pixel adjacent to the first pixel and having the first photoelectric conversion unit accumulates charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, and accumulates charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency; and the signal processing unit generates the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

[0168] (6) A distance measuring device as described in (5), wherein a third pixel having the second photoelectric conversion unit accumulates charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the second modulation frequency, and accumulates charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency; a fourth pixel adjacent to the third pixel and having the second photoelectric conversion unit accumulates charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the second modulation frequency, and accumulates charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency; and the signal processing unit generates the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

[0169] (7) The fifth pixel having the first photoelectric conversion unit accumulates charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the first modulation frequency, accumulates charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, accumulates charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and accumulates charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency, and the signal processing unit generates the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

[0170] (8) The sixth pixel having the second photoelectric conversion unit accumulates charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0-degree pulse drive signal that is in phase with the second modulation frequency, accumulates charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the second modulation frequency, accumulates charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency, and accumulates charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency, and the signal processing unit generates the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

[0171] (9) The distance measuring device according to (8), wherein the fifth pixel and the sixth pixel are adjacent to each other.

[0172] (10) The distance measuring device according to (1), wherein the first photoelectric conversion unit and the second photoelectric conversion unit are the same photoelectric conversion unit.

[0173] (11) A distance measuring device as described in (10), wherein the seventh pixel synchronizes with the first modulation frequency, accumulates charge in one charge accumulation section of the first charge accumulation section in accordance with one of two pulse drive signals having different phases, and accumulates charge in the other charge accumulation section of the first charge accumulation section in accordance with the other of the two pulse drive signals having different phases, synchronizes with the second modulation frequency, accumulates charge in one charge accumulation section of the second charge accumulation section in accordance with one of two pulse drive signals having different phases, and accumulates charge in the other charge accumulation section of the second charge accumulation section in accordance with the other of the two pulse drive signals having different phases.

[0174] (12) In the seventh pixel, the two pulse drive signals synchronized with the first modulation frequency and having different phases are a 0-degree pulse drive signal that is in phase with the first modulation frequency and a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases are a 0-degree pulse drive signal that is in phase with the second modulation frequency and a 180-degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency, in the distance measuring device described in (11).

[0175] (13) The eighth pixel adjacent to the seventh pixel has a configuration equivalent to that of the seventh pixel, and the two pulse drive signals synchronized with the first modulation frequency and having different phases are a 90-degree pulse drive signal that is in phase with the first modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases are a 90-degree pulse drive signal that is in phase with the second modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency, in the distance measuring device described in (12).

[0176] (14) The distance measuring device according to (1), wherein the first modulation frequency is lower than the second modulation frequency.

[0177] (15) The distance measuring device according to (14), wherein the signal processing unit generates the distance measurement value based on a repetition period of the second phase difference corresponding to the first phase difference and the second phase difference.

[0178] (16) The distance measuring device according to (1), wherein the first modulation frequency and the second modulation frequency have the same period.

[0179] (17) The distance measuring device described in (6) generates a first reliability using a first component using a signal based on the accumulated charges of a first charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges of a third charge accumulation section of the second charge accumulation section, and a second component using a signal based on the accumulated charges of a second charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges of the second charge accumulation section of the second charge accumulation section; and the signal processing unit generates a second reliability using a third component using a signal based on the accumulated charges of the first charge accumulation section of the second charge accumulation section and a signal based on the accumulated charges of the third charge accumulation section of the second charge accumulation section, and a fourth component using a signal based on the accumulated charges of the second charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges of the second charge accumulation section of the second charge accumulation section; and when a difference between the first reliability and the second reliability is within a predetermined value, generates the distance measurement value based on the second phase difference and the repetition period of the second phase difference corresponding to the first phase difference.

[0180] (18) A distance measuring device that irradiates a first irradiation light of a first modulation frequency and generates a distance value using reflected light reflected by an object, comprising: a distance measuring sensor having ten pixels that accumulates signal charges generated in a photoelectric conversion unit in a first charge accumulation unit in accordance with a 0-degree pulse drive signal that is synchronized with the first modulation frequency and has the same phase as the first modulation frequency, accumulates the signal charges in a second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, accumulates the signal charges in a third charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and accumulates the signal charges in a fourth charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency; a first sum signal of a signal based on the accumulated charges in the first charge accumulation unit and a signal based on the accumulated charges in the third charge accumulation unit; and a second sum signal of a signal based on the accumulated charges in the second charge accumulation unit and a signal based on the accumulated charges in the third charge accumulation unit. a signal processing unit that generates a distance measurement value by generating a first phase difference corresponding to a frequency lower than a first modulation frequency based on the first sum signal and the second sum signal, the first phase difference representing the time until the reflected light is received.

[0181] (19) A distance measuring device that sequentially irradiates an object with first irradiation light of a first modulation frequency, second irradiation light of a second modulation frequency different from the first modulation frequency, and third irradiation light of the first modulation frequency, and generates a distance value using light reflected by the object, the distance measuring sensor having: a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency; and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency; and a readout unit that, after irradiation with the first irradiation light, reads out a first signal corresponding to the accumulated charges in the first charge accumulation unit, reads out a second signal corresponding to the accumulated charges in the second charge accumulation unit, and, after irradiation with the third irradiation light, reads out a third signal corresponding to the accumulated charges in the first charge accumulation unit. a signal processing unit that generates a first phase difference based on the first signal, which is the time until the light is received as reflected light; that generates a second phase difference based on the second signal, which is the time until the light is received as reflected light; and that generates a third phase difference based on the third signal, which is the time until the light is received as reflected light; and that generates a distance measurement value using an average value of the first phase difference and the third phase difference, and the second phase difference.

[0182] (20) A distance measurement method using a distance measurement sensor that sequentially irradiates an object with first irradiation light of a first modulation frequency and second irradiation light of a second modulation frequency different from the first modulation frequency, and that has a first charge accumulation unit that accumulates signal charges generated in a first photoelectric conversion unit using a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit that accumulates signal charges generated in a second photoelectric conversion unit using a second drive frequency synchronized with the second modulation frequency, wherein a switching period for switching from the first irradiation light to the second irradiation light is shorter than a readout period for reading out a first signal corresponding to the accumulated charge in the first charge accumulation unit and a second signal corresponding to the accumulated charge in the second charge accumulation unit after irradiation with the second irradiation light, and based on the first signal, generates a first phase difference that represents the time until the light is received as reflected light from the object, and based on the second signal, generates a second phase difference that represents the time until the light is received as reflected light, and generates a distance measurement value using the first phase difference and the second phase difference.

[0183] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0184] 1: ranging device, 41, 42: object, 51: control unit, 52: ranging sensor, 56: column processing unit (readout unit), 58: signal processing unit, 71: photoelectric conversion element, 73, 73A, 73B, 73C, 73D, 73a, 73b, 73c, 73d: taps (charge storage units), 78: charge discharge transistor, 580: distance generation unit, 582: reliability generation unit, 586: judgment unit, Fmod1: first modulation frequency, Fmod2: second modulation frequency.

Claims

1. A distance measuring device which sequentially irradiates a first irradiation light having a first modulation frequency and a second irradiation light having a second modulation frequency different from the first modulation frequency, and generates a distance value using the reflected light reflected by an object, comprising: a distance measuring sensor having a first charge accumulation unit which accumulates a signal charge generated in a first photoelectric conversion unit by a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit which accumulates a signal charge generated in a second photoelectric conversion unit by a second drive frequency synchronized with the second modulation frequency; a control unit which controls a switching period for switching from the first irradiation light to the second irradiation light to be shorter than a readout period in which a first signal corresponding to the accumulated charge in the first charge accumulation unit and a second signal corresponding to the accumulated charge in the second charge accumulation unit are read out after irradiation of the second irradiation light; a signal processing unit which generates a first phase difference based on the first signal, which indicates the time until the light is received as the reflected light, and generates a second phase difference based on the second signal, which indicates the time until the light is received as the reflected light, and generates a distance measurement value using the first phase difference and the second phase difference; A distance measuring device comprising:

2. The distance measuring device according to claim 1, wherein the distance measuring sensor is equipped with a plurality of pixel groups having the first charge storage portion and a plurality of pixel groups having the second charge storage portion.

3. The distance measuring sensor further comprises a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit, and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit sets the first charge discharging transistor in a conductive state and the second charge discharging transistor in a non-conductive state during the period when the second irradiation light is irradiated, and sets the first charge discharging transistor in a non-conductive state and the second charge discharging transistor in a conductive state during the period when the first irradiation light is irradiated. The distance measuring device described in claim 1.

4. The distance measuring device of claim 2, wherein the distance measuring sensor further comprises a first charge discharging transistor capable of supplying a predetermined potential to the first photoelectric conversion unit, and a second charge discharging transistor capable of supplying a predetermined potential to the second photoelectric conversion unit, and the control unit causes the first charge discharging transistor to alternate between a non-conductive state and a conductive state during the period when the second irradiation light is irradiated, and causes the second charge discharging transistor to alternate between a non-conductive state and a conductive state during the period when the first irradiation light is irradiated.

5. The distance measuring device described in claim 1, wherein a first pixel having the first photoelectric conversion unit accumulates charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0 degree pulse drive signal having the same phase as the first modulation frequency, and accumulates charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 180 degree pulse drive signal having a phase difference of 180 degrees from the first modulation frequency; a second pixel adjacent to the first pixel and having the first photoelectric conversion unit accumulates charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 90 degree pulse drive signal having a phase difference of 90 degrees from the first modulation frequency, and accumulates charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270 degree pulse drive signal having a phase difference of 270 degrees from the first modulation frequency; and the signal processing unit generates the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

6. The distance measuring device described in claim 5, wherein a third pixel having the second photoelectric conversion unit accumulates charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0 degree pulse drive signal having the same phase as the second modulation frequency, and accumulates charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 180 degree pulse drive signal having a phase difference of 180 degrees from the second modulation frequency; a fourth pixel adjacent to the third pixel and having the second photoelectric conversion unit accumulates charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 90 degree pulse drive signal having a phase difference of 90 degrees from the second modulation frequency, and accumulates charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270 degree pulse drive signal having a phase difference of 270 degrees from the second modulation frequency; and the signal processing unit generates the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

7. The ranging device described in claim 1, wherein a fifth pixel having the first photoelectric conversion unit accumulates charge in a first charge accumulation unit of the first charge accumulation unit in accordance with a 0-degree pulse drive signal having the same phase as the first modulation frequency, accumulates charge in a second charge accumulation unit of the first charge accumulation unit in accordance with a 90-degree pulse drive signal having a phase difference of 90 degrees from the first modulation frequency, accumulates charge in a third charge accumulation unit of the first charge accumulation unit in accordance with a 180-degree pulse drive signal having a phase difference of 180 degrees from the first modulation frequency, and accumulates charge in a fourth charge accumulation unit of the first charge accumulation unit in accordance with a 270-degree pulse drive signal having a phase difference of 270 degrees from the first modulation frequency, and the signal processing unit generates the first phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the first charge accumulation unit.

8. The distance measuring device described in claim 7, wherein the sixth pixel having the second photoelectric conversion unit accumulates charge in a first charge accumulation unit of the second charge accumulation unit in accordance with a 0 degree pulse drive signal having the same phase as the second modulation frequency, accumulates charge in a second charge accumulation unit of the second charge accumulation unit in accordance with a 90 degree pulse drive signal having a phase difference of 90 degrees from the second modulation frequency, accumulates charge in a third charge accumulation unit of the second charge accumulation unit in accordance with a 180 degree pulse drive signal having a phase difference of 180 degrees from the second modulation frequency, and accumulates charge in a fourth charge accumulation unit of the second charge accumulation unit in accordance with a 270 degree pulse drive signal having a phase difference of 270 degrees from the second modulation frequency, and the signal processing unit generates the second phase difference based on the charges accumulated in each of the first to fourth charge accumulation units of the second charge accumulation unit.

9. The distance measuring device according to claim 8, wherein the fifth pixel and the sixth pixel are adjacent to each other.

10. A distance measuring device according to claim 1, wherein the first photoelectric conversion section and the second photoelectric conversion section are the same photoelectric conversion section.

11. The distance measuring device of claim 10, wherein a seventh pixel accumulates charge in one charge accumulation section of the first charge accumulation section in accordance with one of two pulse drive signals having different phases, synchronized with the first modulation frequency, and accumulates charge in the other charge accumulation section of the first charge accumulation section in accordance with the other of the two pulse drive signals having different phases, synchronizes with the second modulation frequency, and accumulates charge in one charge accumulation section of the second charge accumulation section in accordance with one of two pulse drive signals having different phases, and accumulates charge in the other charge accumulation section of the second charge accumulation section in accordance with the other of the two pulse drive signals having different phases.

12. The distance measuring device described in claim 11, wherein in the seventh pixel, the two pulse drive signals synchronized with the first modulation frequency and having different phases are a 0 degree pulse drive signal that is in phase with the first modulation frequency and a 180 degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases are a 0 degree pulse drive signal that is in phase with the second modulation frequency and a 180 degree pulse drive signal that is 180 degrees out of phase with the second modulation frequency.

13. A distance measuring device as described in claim 12, wherein an eighth pixel adjacent to the seventh pixel has a configuration equivalent to that of the seventh pixel, and the two pulse drive signals synchronized with the first modulation frequency and having different phases are a 90-degree pulse drive signal that is in phase with the first modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency, and the two pulse drive signals synchronized with the second modulation frequency and having different phases are a 90-degree pulse drive signal that is in phase with the second modulation frequency and a 270-degree pulse drive signal that is 270 degrees out of phase with the second modulation frequency.

14. The distance measuring device according to claim 1, wherein the first modulation frequency is lower than the second modulation frequency.

15. The distance measuring device according to claim 14, wherein the signal processing unit generates the distance measurement value based on a repetition period of the second phase difference corresponding to the first phase difference and the second phase difference.

16. The distance measuring device according to claim 1, wherein the first modulation frequency and the second modulation frequency have the same period.

17. A distance measuring device as described in claim 6, wherein the signal processing unit generates a first reliability using a first component using a signal based on the accumulated charges in a first charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges in a third charge accumulation section of the second charge accumulation section, and a second component using a signal based on the accumulated charges in the second charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges in the second charge accumulation section of the second charge accumulation section, and a third component using a signal based on the accumulated charges in the first charge accumulation section of the second charge accumulation section and a signal based on the accumulated charges in the third charge accumulation section of the second charge accumulation section, and a fourth component using a signal based on the accumulated charges in the second charge accumulation section of the first charge accumulation section and a signal based on the accumulated charges in the second charge accumulation section of the second charge accumulation section, and wherein when a difference between the first reliability and the second reliability is within a predetermined value, the distance measurement value is generated based on the repetition period of the second phase difference corresponding to the first phase difference and the second phase difference.

18. A distance measuring device that irradiates a first irradiation light of a first modulation frequency and generates a distance value using the reflected light reflected by an object, comprising: a distance measuring sensor having ten pixels that accumulates a signal charge generated in a photoelectric conversion unit in a first charge accumulation unit in accordance with a 0-degree pulse drive signal that is synchronized with the first modulation frequency and has the same phase as the first modulation frequency, accumulates the signal charge in a second charge accumulation unit in accordance with a 90-degree pulse drive signal that is 90 degrees out of phase with the first modulation frequency, accumulates the signal charge in a third charge accumulation unit in accordance with a 180-degree pulse drive signal that is 180 degrees out of phase with the first modulation frequency, and accumulates the signal charge in a fourth charge accumulation unit in accordance with a 270-degree pulse drive signal that is 270 degrees out of phase with the first modulation frequency; a first sum signal of a signal based on the accumulated charge in the first charge accumulation unit and a signal based on the accumulated charge in the third charge accumulation unit; a second sum signal of a signal based on the accumulated charge in the second charge accumulation unit and a signal based on the accumulated charge in the third charge accumulation unit; a signal processing unit that generates a distance measurement value by generating a first phase difference corresponding to a frequency lower than a first modulation frequency, based on the first sum signal and the second sum signal, which represents a time until the reflected light is received.

19. A distance measuring device that sequentially irradiates a first irradiation light of a first modulation frequency, a second irradiation light of a second modulation frequency different from the first modulation frequency, and a third irradiation light of the first modulation frequency, and generates a distance value using light reflected by an object, the distance measuring sensor having a first charge accumulation unit that accumulates a signal charge generated in a first photoelectric conversion unit by a first drive frequency synchronized with the first modulation frequency, and a second charge accumulation unit that accumulates a signal charge generated in a second photoelectric conversion unit by a second drive frequency synchronized with the second modulation frequency; and a readout unit that reads out a first signal corresponding to the accumulated charge in the first charge accumulation unit after irradiation with the first irradiation light, reads out a second signal corresponding to the accumulated charge in the second charge accumulation unit, and reads out a third signal corresponding to the accumulated charge in the first charge accumulation unit after irradiation with the third irradiation light; a signal processing unit that generates a first phase difference based on the first signal, a second phase difference based on the second signal, a third phase difference based on the third signal, and generates a distance measurement value using an average value of the first phase difference and the third phase difference and the second phase difference.

20. A distance measuring method using a distance measuring sensor having: a first irradiation light having a first modulation frequency and a second irradiation light having a second modulation frequency different from the first modulation frequency, which is irradiated to an object in sequence; a first charge accumulation unit which accumulates a signal charge generated in a first photoelectric conversion unit by a first drive frequency synchronized with the first modulation frequency; and a second charge accumulation unit which accumulates a signal charge generated in a second photoelectric conversion unit by a second drive frequency synchronized with the second modulation frequency; wherein a switching period for switching from the first irradiation light to the second irradiation light is made shorter than a readout period for reading out a first signal corresponding to the accumulated charge in the first charge accumulation unit and a second signal corresponding to the accumulated charge in the second charge accumulation unit after irradiation with the second irradiation light; the distance measuring method comprises: generating, based on the first signal, a first phase difference representing the time until the light is received as reflected light from the object; generating, based on the second signal, a second phase difference representing the time until the light is received as reflected light; and generating a distance measurement value using the first phase difference and the second phase difference.

Images