Ranging Device

Abstract

The present technology relates to a ranging device which achieves synchronization between the modulating frequency of a light source and the modulating frequency of a ToF sensor when the light source and the ToF sensor are configured as separate units, and to a light reception method for the ranging device. The ranging device is provided with: a reception unit which performs a synchronization process for a transmitted signal transmitted from another device via wireless communication, and thereby generates a signal synchronized with a synchronization clock signal of the other device; a reference clock generating unit which, on the basis of the synchronization clock signal, generates a reference clock signal serving as a reference when the light source emits light; and a ToF sensor which, on the basis of the reference clock signal, receives reflected light due to reflection by an object of irradiating light irradiated from the light source. The present technology is applicable to, for example, ranging devices for measuring distance to a subject.

Description

[Technical field]

[0001] This technology is Ranging Device In particular, when the light source and the ToF sensor are configured separately, the modulation frequency of the light source and the modulation frequency of the ToF sensor are synchronized. Ranging Device Regarding. [Background technology]

[0002] One of the distance measurement methods that uses light to measure the distance to an object is the ToF (Time of Flight) method. The ToF method uses a light source to irradiate the object with light, receives the reflected light with a ToF sensor, and analyzes the light reception results to measure the distance to the object.

[0003] Also, a technique has been proposed in which light is irradiated onto an object from a light source separate from the ToF sensor, and the reflected light is received by the ToF sensor to measure the distance to the object, etc. (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-31607 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when the light source and the ToF sensor are configured separately, the light source clock for the light source and the sensor clock for the ToF sensor are separate. Therefore, the modulation frequency of the light source and the modulation frequency of the ToF sensor are not necessarily synchronized, and there are cases where the frequencies are misaligned. And when the modulation frequency of the ToF sensor is misaligned from the modulation frequency of the light source, there is a risk of an error occurring in the distance measurement result.

[0006] The present technology has been developed in consideration of such circumstances, and makes it possible to synchronize the modulation frequency of a light source and the modulation frequency of a ToF sensor when the light source and the ToF sensor are configured separately. [Means for solving the problem]

[0007] A distance measuring device according to a first aspect of the present technology includes a receiving unit that performs synchronization processing of a transmission signal transmitted from another device via wireless communication to generate a signal synchronized with a synchronization clock signal of the other device, a reference clock generating unit that generates a reference clock signal that is a reference when a light source emits light, based on the synchronization clock signal, and a ToF sensor that receives reflected light that is returned after irradiation light irradiated from the light source is reflected by an object, based on the reference clock signal. The receiving unit receives light source information related to the irradiation light emitted from the light source via the wireless communication, the light source information including calibration data of the light source. .

[0008] A ranging device according to a second aspect of the present technology includes: The device includes a receiving unit that generates a signal synchronized with a synchronization clock signal of another device by performing synchronization processing of a transmission signal transmitted from another device via wireless communication, a reference clock generating unit that generates a reference clock signal that serves as a reference for when a light source emits light based on the synchronization clock signal, and a ToF sensor that receives reflected light that is reflected by an object and returned from the light source based on the reference clock signal, and the receiving unit receives light source information regarding the illumination light emitted from the light source via the wireless communication, and the light source information includes an operating mode.

[0009] In the first and second aspects of the present technology, a reference clock signal that is a reference when a light source emits light is generated based on a synchronous clock signal synchronized with a transmission signal of wireless communication, and reflected light that is returned after the irradiation light irradiated from the light source is reflected by an object is received based on the reference clock signal. Also, light source information regarding the irradiation light irradiated from the light source is received by the wireless communication.

[0013] The ranging device may be a stand-alone device or a module that is integrated into another device. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating the basic principle of distance measurement processing using a direct ToF method. [Diagram 2] FIG. 1 is a diagram illustrating the basic principle of distance measurement processing using a direct ToF method. [Diagram 3] 11A and 11B are diagrams illustrating a method for calculating a phase difference φ. [Figure 4] FIG. 2 is a diagram illustrating components and frames in the present disclosure. [Diagram 5] 1A and 1B are diagrams illustrating a concept of a correction process for a cyclic error. [Figure 6] 1A and 1B are diagrams illustrating a method for preventing interference between a plurality of light sources. [Figure 7] 1A and 1B are diagrams illustrating a method for preventing interference between a plurality of light sources. [Figure 8] FIG. 1 is a diagram illustrating a direct ToF method. [Figure 9] 1 is a diagram illustrating a configuration example of a distance measuring system according to a first embodiment of the present disclosure. [Figure 10] 1 is a block diagram showing an example of a functional configuration of a distance measuring device including a light source and a distance measuring apparatus; [Figure 11] FIG. 13 is a diagram illustrating an example of operation when the operation mode is a time division mode. [Figure 12] FIG. 13 is a diagram illustrating an example of operation when the operation mode is set to a modulation frequency mode. [Figure 13] 13A to 13C are diagrams illustrating an example of operation when the operation mode is set to a light emission pattern mode. [Figure 14] FIG. 2 is a functional block diagram of a distance measuring device having only a light emitting function. [Figure 15] FIG. 2 is a functional block diagram of a distance measuring device having only a light receiving function. [Figure 16] 11 is a flowchart for explaining a light emission control process of a light emitting distance measuring device. [Figure 17] 11 is a flowchart for explaining a light emission control process of a distance measuring device on a light receiving side. [Figure 18] FIG. 11 is a diagram illustrating a phase synchronization process. [Figure 19] FIG. 1 is a block diagram of a ranging device that performs time synchronization based on GNSS signals. [Figure 20] FIG. 13 is a diagram illustrating a configuration example of a distance measuring system according to a second embodiment of the present disclosure. [Figure 21] 1A and 1B are diagrams illustrating the principle of relative distance calculation; [Figure 22] 1A and 1B are diagrams illustrating the principle of relative distance calculation; [Diagram 23] 1A and 1B are diagrams illustrating the principle of relative distance calculation; [Figure 24] FIG. 1 is a block diagram of a ranging device built into a drone. [Diagram 25]FIG. 13 is a diagram showing an example of a confidence image observed by a drone. [Figure 26] 11A and 11B are diagrams illustrating a process of improving the accuracy of a position and orientation by using an accurate relative distance. [Figure 27] FIG. 4 is a diagram illustrating the relationship between the modulation frequency and the measured distance. [Figure 28] 13 is a flowchart illustrating a relative distance measurement process of a drone. [Figure 29] FIG. 13 is a diagram illustrating a configuration example of a distance measuring system according to a third embodiment of the present disclosure. [Diagram 30] 13 is a flowchart illustrating a distance calculation process of a distance measuring system in a third embodiment. [Diagram 31] FIG. 31 is a diagram for explaining detection of a phase difference in step S66 of FIG. 30. [Diagram 32] 11A and 11B are diagrams illustrating a method of calculating distance by receiving reflected light from a plurality of light sources. [Diagram 33] 11A and 11B are diagrams illustrating a method of calculating distance by receiving reflected light from a plurality of light sources. [Diagram 34] FIG. 2 is a diagram illustrating a configuration example of a light receiving unit of a ToF sensor. [Diagram 35] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Diagram 36] 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, a description will be given of an embodiment of the present technology with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted. The description will be given in the following order. 1. Overview of ToF distance measurement processing 2. First embodiment of distance measurement system 3. Block diagram of the ranging device 4. Processing flow of the ranging device 5. Modification of the First Embodiment 6. Second embodiment of distance measurement system 7. Third embodiment of distance measurement system 8. Distance calculation method by receiving reflected light from multiple light sources 9.ToF Sensor Configuration 10. Mobile application examples

[0016] <1. Overview of distance measurement processing using the ToF method> One of the distance measurement methods that uses light to measure the distance to an object is the ToF (Time of Flight) method. The ToF method uses a light source to irradiate the object with light, receives the reflected light with a ToF sensor, and analyzes the light reception results to measure the distance to the object.

[0017] The present disclosure relates to a distance measurement technique using a ToF method. Therefore, in order to facilitate understanding of each embodiment of the present disclosure, the basic principle of distance measurement processing using the ToF method will be described with reference to FIGS.

[0018] The ToF method is a method of irradiating an object with light and analyzing the reflected light to measure the distance (depth) to the object and the shape of the object. Note that the following explanation does not specifically mention the measurement process of 3D shapes, but it is possible to measure the 3D shape of an object by measuring the distance to the object's surface over the entire surface of the object.

[0019] (Configuration of distance measurement system) FIG. 1 shows the configuration of a distance measuring system.

[0020] 1 includes a light source 2 and a distance measuring device 3. The light source 2 irradiates light onto an object 4 as a measurement target. Irradiation light L1 emitted from the light source 2 is reflected by the object 4 and enters the distance measuring device 3 as reflected light L2.

[0021] The light source 2 and the distance measuring device 3 are disposed at approximately the same position. In this case, the distance (depth) d from the distance measuring device 3 to the object 4 can be calculated by the following formula (1).

number

[0022] In equation (1), Δt is the time it takes for the illuminating light L1 emitted from the light source 2 to be reflected by the object 4 and enter the distance measuring device 3, and c is the speed of light (2.9979×10 8 [m / sec]).

[0023] The light L1 emitted from the light source 2 is a pulsed light with an emission pattern that repeatedly turns on and off at high speed at a predetermined frequency f (modulation frequency) as shown in Fig. 2. One period of the emission pattern is 1 / f. The distance measuring device 3 detects the phase of the reflected light L2 shifted according to the time Δt it takes for the light to reach the distance measuring device 3 from the light source 2. If the amount of this phase shift (phase difference) is φ, the time Δt can be calculated by the following formula (2).

number

[0024] Therefore, the distance d from the distance measuring device 3 to the object 4 can be calculated from the formulas (1) and (2) using the following formula (3).

number

[0025] Next, a method for calculating the above-mentioned phase difference φ will be described with reference to FIG.

[0026] The distance measuring device 3 has a ToF sensor in which pixels that perform photoelectric conversion of incident light are arranged two-dimensionally in a matrix. Each pixel of the ToF sensor repeatedly turns ON / OFF at high speed, and accumulates electric charge only during the ON period.

[0027] When measuring the distance to an object using the ToF sensor, the distance measuring device 3 sequentially switches the execution timing of the ToF sensor between ON and OFF, accumulates charge at each execution timing, and outputs a detection signal according to the accumulated charge.

[0028] There are four types of ON / OFF execution timing, for example, phase 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees.

[0029] The execution timing of the phase 0 degrees is a timing at which the ON timing (light receiving timing) of the ToF sensor is set to the same phase as the phase of the pulsed light emitted by the light source 2, that is, the light emission pattern.

[0030] The execution timing of the 90-degree phase is a timing at which the ON timing (light receiving timing) of the ToF sensor is delayed by a phase of 90 degrees from the pulsed light (light emission pattern) emitted by the light source 2.

[0031] The execution timing of the 180 degree phase is a timing at which the ON timing (light receiving timing) of the ToF sensor is delayed by a phase of 180 degrees from the pulsed light (light emission pattern) emitted by the light source 2.

[0032] The execution timing of the phase 270 degrees is a timing at which the ON timing (light receiving timing) of the ToF sensor is delayed by a phase of 270 degrees from the pulsed light (light emission pattern) emitted by the light source 2.

[0033] The ToF sensor sequentially switches between these four types of execution timing, and acquires the amount of reflected light L2 received (accumulated charge) at each light receiving timing. In Fig. 3, the timing at which the reflected light L2 is incident is shaded at the ON timing (light receiving timing) of each phase.

[0034] In FIG. 3, the charges accumulated when the light receiving timing is set to phase 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees are respectively denoted as Q 0 , Q 90 , Q 180 , and Q270 Then, the phase difference φ is Q 0 , Q 90 , Q 180 , and Q 270 Using this, it can be calculated using the following formula (4).

number

[0035] By inputting the phase difference φ calculated by equation (4) into the above equation (3), the distance d from the distance measuring device 3 to the object 4 can be calculated.

[0036] In addition, in a ToF sensor, a value representing the intensity of light received by each pixel is called a confidence value, and is calculated using the following equation (5) or (6).

number

[0037] As shown in Fig. 4, the ToF sensor switches the light reception timing in the order of phase 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees, and accumulates the charge (charge Q 0 , charge Q 90 , charge Q 180 , and charge Q 270 ) and sequentially outputs detection signals according to the

[0038] In this disclosure, an image frame output by the ToF sensor with one phase, i.e., phase 0 degrees, phase 90 degrees, phase 180 degrees, or phase 270 degrees, is referred to as a "component," and a set of four components (four-phase image frames) with phases of 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees is referred to as a "frame."

[0039] (Cyclic error correction) The above-mentioned formula (4) is calculated assuming that the luminance change of the illumination light L1 emitted from the light source 2 is a sine wave. However, in reality, the light emitted from the light source 2 is a square wave as shown in Fig. 2, and therefore, by processing the square wave as a sine wave, a periodic error (hereinafter referred to as a cyclic error) occurs in the distance d. Therefore, a correction process is generally performed to correct the cyclic error for the detection signals of each phase output from the ToF sensor.

[0040] FIG. 5 is a diagram showing a concept of a cyclic error correction process.

[0041] The graph on the left side of Figure 5 shows the phase difference φ output from the ToF sensor. obs The relationship between the phase difference φ and the distance d is shown. obs and the distance d is ideally a linear relationship as shown by the dashed line, but in reality, it becomes a nonlinear relationship including an error due to a cyclic error as shown by the solid line.

[0042] Therefore, in the downstream signal processing unit that processes the detection signal from the ToF sensor, a correction process is performed to correct the cyclic error, and the corrected phase difference φ linear and the distance d is corrected to be linear.

[0043] Specifically, the correction process involves measuring an object with a known distance d using a ToF sensor, and calculating the phase difference φ obs and the true phase difference φ corresponding to the distance d to the object linear From the relationship, the phase difference φ obs The true phase difference φ linear A correction function f(φ obs ) is calculated. This correction function f(φ obs ) is stored in advance in the memory of the signal processing unit, and the signal processing unit receives a phase difference φ obs When is supplied, the phase difference φ obs A correction process is performed on

[0044] The correction process is performed using the correction function f(φ obs ) in memory and perform the calculation. obs and the true phase difference φ linear The combination of and is stored in a lookup table or the like, and the true phase difference φ corresponding to the measured value is calculated. linear Another method is to read it from a lookup table and output it.

[0045] Alternatively, the actually measured correction function f(x) may be approximated to another parameter function, and only a small number of correction coefficients may be stored in memory as parameters. For example, as shown in equation (7), the correction function f(x) may be expanded into a Fourier series, and the correction coefficients (a k , b k ) can be stored in memory.

number

[0046] (Method of preventing interference between multiple light sources) In the distance measuring system 1, when there are multiple sets of light sources 2 and distance measuring devices 3 within the measurement range of the distance measuring device 3, the phase difference φ may not be measured correctly due to the illumination light emitted from the other light sources 2.

[0047] For example, as shown in FIG. 6, assume that a set 5A of a light source 2A and a distance measuring device 3A and a set 5B of a light source 2B and a distance measuring device 3B are installed within a range (measurement range) in which light from each other's light source 2 can be received.

[0048] The distance measuring device 3A receives light reflected by an object 4 from the light source 2A and detects a phase difference φ A The distance measuring device 3B receives the light reflected by the object 4 from the light source 2B and calculates a phase difference φ B At this time, there is a risk that the light emitted from the light source 2A may adversely affect the light reception by the distance measuring device 3B, and the light emitted from the light source 2B may adversely affect the light reception by the distance measuring device 3A.

[0049] There are three ways to deal with the interference of the light irradiated from the other light sources 2: (1) a method of emitting pulsed light from each light source 2 in a time-division manner, (2) a method of varying the modulation frequency of the pulsed light from each light source 2, and (3) a method of varying the emission pattern of the pulsed light from each light source 2.

[0050] (1) In the method of emitting pulsed light from each light source 2 in a time-division manner, since only one type of pulsed light is emitted from each light source 2 at any given time, there is no influence from the light emitted by other light sources 2.

[0051] FIG. 7A shows an example of (2) a method for making the modulation frequencies of the pulsed light from each light source 2 different.

[0052] In the method of varying the modulation frequency of pulsed light, for example, the light source 2A of set A irradiates pulsed light with a modulation frequency f of 100 MHz, and the light source 2B of set B irradiates pulsed light with a modulation frequency f of 71 MHz.

[0053] The distance measuring device 3A of the set A drives the ToF sensor at the same drive frequency as the modulation frequency f=100 MHz, and the distance measuring device 3B of the set B drives the ToF sensor at the same drive frequency as the modulation frequency f=71 MHz. The drive frequency of the ToF sensor corresponds to the frequency of the ON period in FIG. 3 during which electric charge is accumulated.

[0054] The distance measuring device 3A is driven at the same driving frequency as the modulation frequency f of the pulsed light from the light source 2A, so that the light from the light source 2A can be received with high correlation, i.e., a signal having a high confidence value can be detected.

[0055] On the other hand, when the distance measuring device 3B receives pulsed light from the light source 2A, the drive frequency is different, so the phase difference φ cannot be calculated. 0 , Q 90 , Q 180 , and Q 270Since the values ​​of are almost the same and are not observed as modulated light sources, the calculation of the phase difference φ is not affected. Therefore, even if multiple light sources 2A and 2B are emitting irradiation light simultaneously, distance measuring devices 3A and 3B can each independently measure the accurate distance d.

[0056] FIG. 7B shows an example of (3) a method for making the emission patterns of the pulsed light from each light source 2 different.

[0057] The light source 2A of set A emits pulsed light in a predetermined light emission pattern A, and the light source 2B of set B emits pulsed light in a predetermined light emission pattern B. The distance measuring device 3A of set A drives the ToF sensor with the light emission pattern A, and the distance measuring device 3B of set B drives the ToF sensor with the light emission pattern B. The light emission pattern A and the light emission pattern B are different.

[0058] In each of the distance measuring devices 3A and 3B, the reflected light of the same light emission pattern shows a high correlation, and the confidence value is also high. On the other hand, the correlation is low for the reflected light of a different light emission pattern, so there is no effect on the calculation of the phase difference φ. Therefore, even if multiple light sources 2A and 2B simultaneously emit irradiation light, the distance measuring devices 3A and 3B can each independently measure the accurate distance d.

[0059] As a measure against interference of irradiated light, it is possible to set which of the following methods is adopted depending on the operation mode: (1) a method for identifying the pulsed light of each light source 2 in time division, (2) a method for identifying the pulsed light of each light source 2 by the modulation frequency, and (3) a method for identifying the pulsed light of each light source 2 by the emission pattern. The operation modes of (1) the method for identifying the pulsed light of each light source 2 in time division, (2) the method for identifying the pulsed light of each light source 2 by the modulation frequency, and (3) the method for identifying the pulsed light of each light source 2 by the emission pattern are referred to as (1) the time division mode, (2) the modulation frequency mode, and (3) the emission pattern mode, respectively.

[0060] (direct ToF method) In the ToF method, as described above, the charge Q detected at multiple phases with respect to the irradiation timing of the irradiation light L1 is 0 , Q 90 , Q 180 , and Q 270 The method of detecting the phase difference φ based on this and calculating the distance d to the object 4 is called an indirect ToF method.

[0061] In contrast to this, there is also a method called a direct ToF method, in which the time Δt from when light is emitted from the light source 2 until it reaches the distance measuring device 3 is directly counted.

[0062] The direct ToF method will be briefly described with reference to FIG.

[0063] A direct ToF ToF sensor has a SPAD (Single Photon Avalanche Diode) or APD (Avalanche photodiode) as a light receiving element in each pixel arranged two-dimensionally in a matrix. A SPAD or APD is a light receiving element that outputs a signal by avalanche amplifying the electrons generated when incident light is incident. The flight time of light from the time when the light source 2 starts emitting light to the time when the distance measuring device 3 receives the reflected light is converted into a digital count value (hereinafter referred to as TDC code) by a TDC (time to digital converter). The light is emitted and received multiple times to eliminate the effects of ambient light and multipath.

[0064] Then, a histogram of the TDC codes for multiple times is generated, as shown in Fig. 8. The TDC code with the highest frequency value is determined as the final flight time Δt, and the distance d can be calculated using the above equation (1).

[0065] Therefore, even in a direct ToF type ToF sensor, the time is counted based on the time when the light source 2 starts emitting light, so the light emission timing needs to be notified to the ToF sensor side.

[0066] The above is an overview of distance measurement processing using the ToF method.

[0067] As described above, in the ToF method, the light source 2 and the distance measuring device 3 are generally configured as an integrated unit, such as in the same module, and are disposed in approximately the same position.

[0068] However, there is also a method in which the light source 2 and the distance measuring device 3 are separated, light is irradiated onto the object 4 from the light source 2 installed at a distance, and the reflected light is received by the distance measuring device 3 to measure the distance d to the object 4.

[0069] When the light source 2 and the distance measuring device 3 are disposed separately at separate positions, the light source 2 can be disposed near the object 4, so the amount of light received by the ToF sensor can be improved compared to when the light source 2 and the distance measuring device 3 are configured as an integrated unit. This improves the distance measurement accuracy of the distance measuring device 3.

[0070] However, as described above, when measuring the distance d using the ToF method, the distance measuring device 3 needs to be synchronized with the light emission timing of the light source 2 with high precision. Therefore, if the light source 2 and the distance measuring device 3 are placed separately, a problem arises as to whether the ToF sensor of the distance measuring device 3 can achieve frequency synchronization and phase synchronization with the light emission timing of the light source 2 with high precision.

[0071] Therefore, a distance measuring system that realizes highly accurate synchronization between a light source and a distance measuring device that are separately arranged will be described below.

[0072] <2. First embodiment of distance measuring system> FIG. 9 illustrates an example configuration of a distance measuring system according to the first embodiment of the present disclosure.

[0073] The ranging system 11 according to the first embodiment in Fig. 9 includes a plurality of ranging devices 21. In the example of Fig. 9, the ranging system 11 includes five ranging devices 21A to 21E, but the number of ranging devices 21 is not limited to five and may be any number.

[0074] The distance measuring device 21 may have a configuration including both the light source 2 and the distance measuring device 3, a configuration including only the light source 2, or a configuration including only the distance measuring device 3. Specifically, the distance measuring device 21A includes only the light source 2, and the distance measuring device 21B includes the light source 2 and the distance measuring device 3. The distance measuring device 21C includes only the distance measuring device 3, and the distance measuring device 21D includes the light source 2 and the distance measuring device 3. The distance measuring device 21E includes only the distance measuring device 3.

[0075] Each distance measuring device 21 is identified by a unique ID assigned to it. In this embodiment, distance measuring device 21A is assigned a unique ID of "0" (ID0), distance measuring device 21B is assigned a unique ID of "1" (ID1), and distance measuring device 21C is assigned a unique ID of "2" (ID2). Similarly, distance measuring device 21D is assigned a unique ID of "3" (ID3), and distance measuring device 21E is assigned a unique ID of "4" (ID4).

[0076] The distance measuring device 3 includes a ToF sensor 52 (FIG. 10) that measures a distance d to an object using an indirect ToF method or a direct ToF method. For simplicity, the following description will be given assuming that the ToF sensor 52 included in each distance measuring device 3 is an indirect ToF ToF sensor, and supplementary explanations will be given as necessary for the case of the direct ToF method.

[0077] The distance measuring devices 21A to 21D are fixedly attached to, for example, a ceiling, etc. The distance measuring device 21E is attached to a predetermined position of the mobile transport vehicle 22, and the mobile transport vehicle 22 can change its traveling direction depending on the distance measuring result of the distance measuring device 21E, such as the distance to the object 24.

[0078] The five distance measuring devices 21A to 21E can communicate (wirelessly communicate) by a predetermined radio signal via the network 23. Alternatively, each of the five distance measuring devices 21A to 21E may directly communicate wirelessly with the other distance measuring device 21 without going through the network 23.

[0079] The network 23 is configured, for example, by a wide area communication network for wireless mobile devices such as WiFi (registered trademark), Bluetooth (registered trademark), or a so-called 4G line or 5G line.

[0080] The distance measuring device 21A, which is constituted only by the light source 2, emits irradiation light when another distance measuring device 21 (for example, the distance measuring device 21C) equipped with the distance measuring device 3 measures the distance.

[0081] Distance measuring device 21C and distance measuring device 21E, each of which is composed only of distance measuring device 3, receive reflected light emitted from light source 2 of other distance measuring device 21 and measure distance d to the object.

[0082] Distance measuring devices 21B and 21D, which are equipped with a light source 2 and a distance measuring device 3, can measure the distance d to an object by receiving reflected light of light emitted from their own light source 2, and can also measure the distance d to an object by receiving reflected light of light emitted from the light source 2 of another distance measuring device 21 (for example, distance measuring device 21C).

[0083] For example, a distance measuring device 21E constituted only by a distance measuring device 3 measures a distance d from the distance measuring device 21E to an object 24 by synchronizing with the emission timing of light emitted from a light source 2 of another distance measuring device 21D.

[0084] More specifically, the distance measuring device 21E synchronizes the light emission timing of the light source 2 with the light reception timing of its own distance measuring device 3 by synchronizing with the clock of a wireless signal emitted by the light source 2 of another distance measuring device 21D. For example, when the distance measuring device 21E and the distance measuring device 21D perform wireless communication by WiFi (registered trademark) or Bluetooth (registered trademark), the light emission timing is synchronized with the light reception timing of its own distance measuring device 3 by, for example, a 2.4 GHz clock signal. This makes it possible to achieve high-precision distance measurement even when the light source 2 and the distance measuring device 3 are arranged separately.

[0085] <3. Block diagram of distance measuring device> FIG. 10 is a block diagram showing an example of the functional configuration of a distance measuring device 21 including a light source 2 and a distance measuring apparatus 3. As shown in FIG.

[0086] The distance measuring device 21 includes a clock source 41 , a communication module 42 , an antenna 43 , a reference clock generating unit 44 , an operation mode setting unit 45 , a laser driver 46 , and a laser emitting unit 47 .

[0087] The distance measuring device 21 also includes a light source information acquisition unit 48, a memory (storage unit) 49, a time synchronization unit 50, a timing control unit 51, a ToF sensor 52, and a signal processing unit 53.

[0088] The clock source 41 is configured, for example, by a crystal oscillator, generates a master clock that serves as a reference for both wireless communication and light emission timing, and supplies it to the communication module 42 .

[0089] The communication module 42 controls the wireless communication performed by the distance measuring device 21 and processes data based on the wireless signals transmitted and received via the antenna 43 .

[0090] For example, when transmitting data, the communication module 42 performs encoding and modulation using a predetermined encoding method and modulation method corresponding to the data to be transmitted, and transmits the resultant transmission signal from the antenna 43. The data to be transmitted is supplied from a light source information acquisition unit 48, a time synchronization unit 50, a timing control unit 51, etc.

[0091] In addition, when receiving data, the communication module 42 performs processing (decoding and demodulation) opposite to that performed when transmitting data, in response to the data received via the antenna 43, and supplies the resulting data to one of the subsequent blocks, specifically, the operation mode setting unit 45, the light source information acquisition unit 48, the time synchronization unit 50, or the timing control unit 51.

[0092] The communication module 42 includes a synchronous clock generating unit 61 and a signal transmitting / receiving unit 62 .

[0093] When transmitting data, the synchronous clock generating unit 61 generates a transmission signal by modulating the carrier frequency to a predetermined modulation frequency, and transmits the signal via the antenna 43. The synchronous clock generating unit 61 also generates a synchronous clock signal corresponding to the carrier frequency or the modulation frequency, and supplies the synchronous clock signal to the reference clock generating unit 44.

[0094] When receiving data, the synchronous clock generating unit 61 performs synchronization processing on the signal received via the antenna 43 to generate a signal (synchronous clock signal) corresponding to the synchronous clock signal used when transmitting data, and supplies the signal to the reference clock generating unit 44.

[0095] When transmitting data, the signal transmitting / receiving unit 62 encodes the data to be transmitted using a predetermined encoding method and supplies the encoded data to the synchronization clock generating unit 61. Examples of the data to be transmitted include a unique ID for identifying the distance measuring device 21, which is supplied from the light source information acquiring unit 48, and light source information including emission start time information, emission duration, and operation mode of the irradiation light emitted by the laser emitting unit 47.

[0096] When receiving data, the signal transmitting / receiving unit 62 demodulates the signal supplied from the synchronization clock generating unit 61, acquires the transmitted data, and supplies it to any of the operation mode setting unit 45, the light source information acquiring unit 48, the time synchronization unit 50, or the timing control unit 51. For example, the signal transmitting / receiving unit 62 supplies the operation mode, which is one piece of light source information acquired from the received signal, to the operation mode setting unit 45, and supplies all of the light source information to the light source information acquiring unit 48.

[0097] When transmitting data, the antenna 43 amplifies a transmission signal supplied from the communication module 42 and transmits it as an electromagnetic wave. When receiving data, the antenna 43 receives a transmission signal transmitted from another device and supplies it to the communication module 42 as a received signal.

[0098] The reference clock generating unit 44 includes a PLL (Phase Locked Loop) circuit and the like, and generates a reference clock signal that serves as a reference for the laser emission unit 47 to emit light based on the synchronous clock signal supplied from the synchronous clock generating unit 61, and supplies the reference clock signal to the operation mode setting unit 45.

[0099] The operation mode setting unit 45 uses the reference clock signal supplied from the reference clock generating unit 44 to generate a light emission pulse signal corresponding to the operation mode supplied from the signal transmitting / receiving unit 62, and supplies the light emission pulse signal to the laser driver 46 and the ToF sensor 52.

[0100] For example, when the operating mode is the time division mode, the operating mode setting unit 45 supplies the reference clock signal from the reference clock generating unit 44 to the laser driver 46 and the ToF sensor 52 as an emission pulse signal in accordance with the time period during which the operating mode setting unit 45 itself (the ranging device 21) emits (or receives) light.

[0101] Also, for example, when the operating mode is the modulation frequency mode, the operating mode setting unit 45 adjusts the frequency of the reference clock signal from the reference clock generating unit 44 to the modulation frequency at which it (the ranging device 21) emits or receives light, and supplies it to the laser driver 46 and the ToF sensor 52 as an emission pulse signal.

[0102] Also, for example, when the operating mode is the light emission pattern mode, the operating mode setting unit 45 generates an emission pattern for the itself (the ranging device 21) to emit or receive light based on the reference clock signal from the reference clock generating unit 44, and supplies it to the laser driver 46 and the ToF sensor 52 as an emission pulse signal.

[0103] Based on the light emission pulse signal supplied from the operation mode setting unit 45, the laser driver 46 generates a drive signal for driving a VCSEL (Vertical Cavity Surface Emitting Laser), which is the light source of the laser emission unit 47, and supplies the drive signal to the laser emission unit 47.

[0104] Start timing control information that specifies the timing to start light emission (or light reception) is supplied to the laser driver 46 from the timing control unit 51. The start timing control information includes light emission start time information indicating the time to start light emission or light reception, the light emission time of the VCSEL (exposure time of the pixel), the component and frame cycles, etc. The laser driver 46 supplies a drive signal to the laser emission unit 47 at a predetermined timing based on the start timing control information.

[0105] The laser emission unit 47 is configured, for example, with a VCSEL array (light source array) in which a plurality of VCSELs serving as light sources are arranged in a plane, and repeats turning on and off light emission at a predetermined cycle in response to a drive signal from a laser driver 46.

[0106] When emitting light, the light source information acquisition unit 48 acquires light source information from the memory 49 and supplies it to the signal transmission / reception unit 62. When receiving light, the light source information acquisition unit 48 acquires the received light source information from the signal transmission / reception unit 62 and stores it in the memory 49. The light source information is also supplied to the timing control unit 51, the ToF sensor 52, and the signal processing unit 53 as necessary. For example, calibration data of the light source is supplied to the ToF sensor 52.

[0107] The memory 49 stores the light source information and supplies it to the light source information acquisition unit 48 as necessary.

[0108] Here, examples of the information stored in the memory 49 as the light source information include the following information: ·Unique ID Light emission time / exposure time -Light emission start time information / Exposure start time information Repetition frequency (direct ToF method) Modulation frequency (indirect ToF method) Component length (indirect ToF method) Position and orientation of the ranging device Frame length Light source wavelength Lighting pattern Light source calibration data (indirect ToF method) Operation mode

[0109] The unique ID is information for identifying the distance measuring device 21. The light emission time indicates the length of time that the laser emitter 47 (light source 2) emits light once (the ON period described above). The light emission time corresponds to the exposure time of the ToF sensor 52 on the light receiving side. The light emission start time information indicates the time when the laser emitter 47 starts emitting irradiation light. The light emission start time information corresponds to the time when the ToF sensor 52 starts exposure on the light receiving side.

[0110] In a direct ToF system in which light emission and reception are repeated multiple times, the repetition frequency represents the time interval between emission, that is, the time interval from the start of the previous emission to the start of the next emission.

[0111] The modulation frequency represents the modulation frequency in the indirect ToF system, and the component length represents the time length of one component in the indirect ToF system.

[0112] The position and attitude of the ranging device is information indicating its own position or attitude when the ranging device 21 is equipped with an IMU (Inertial Measurement Unit), a magnetic sensor, a GNSS (Global Navigation Satellite System) receiver, etc. Either the position or the attitude may be indicated, or both may be indicated.

[0113] The frame length indicates the time length of one frame in the indirect ToF system. The wavelength of the light source indicates the wavelength of the irradiated light emitted by the laser emitter 47. For example, when the irradiated light is infrared light, the wavelength is in the range of about 850 nm to 940 nm.

[0114] The light emission pattern represents information that allows the distance measuring device 21 to identify the light emission pattern of the pulsed light emitted by the light source 2.

[0115] The light source calibration data is data for correcting cyclic errors in the indirect ToF method, and is composed of, for example, a correction function or correction coefficients that correct the nonlinear relationship on the left side of FIG. 5 to the linear relationship on the right side of FIG.

[0116] The operation mode indicates information indicating one of (1) time division mode, (2) modulation frequency mode, and (3) light emission pattern mode for preventing interference of irradiated light, and operation mode detailed setting information required for operation in the operation mode. For example, the operation mode detailed setting information includes, in (1) time division mode, information indicating the time period to be used by the device itself among the time division, and in (3) light emission pattern mode, information indicating the light emission pattern to be used by the device itself among multiple light emission patterns.

[0117] FIG. 11 shows an example of the operation of a plurality of distance measuring devices 21 on the light-emitting side and a distance measuring device 21 on the light-receiving side when the operation mode is set to a time-division mode to prevent interference of irradiated light.

[0118] For example, the distance measuring device 21A and the distance measuring device 21B, which are the light emitting distance measuring devices 21, emit light in a time-division manner. The light receiving distance measuring device 21E performs a light receiving operation in synchronization with the light emission timing of the distance measuring device 21B, thereby preventing interference of the emitted light and receiving the reflected light of the emitted light emitted by the distance measuring device 21B. The light emission timing of the light receiving target distance measuring device 21B can be confirmed from light emission start time information indicating the light emission start time, which is part of the light source information.

[0119] FIG. 12 shows an example of the operation of a plurality of distance measuring devices 21 on the light-emitting side and a distance measuring device 21 on the light-receiving side when the operation mode is set to the modulation frequency mode to prevent interference of irradiated light.

[0120] As in FIG. 11, the distance measuring device 21 on the light-emitting side is the distance measuring device 21A and the distance measuring device 21B, and the distance measuring device 21 on the light-receiving side is the distance measuring device 21E.

[0121] The distance measuring device 21A emits irradiation light at a modulation frequency of 71 MHz. The distance measuring device 21B emits irradiation light at a modulation frequency of 100 MHz. The distance measuring device 21E on the light receiving side performs a light receiving operation in synchronization with the modulation frequency of the distance measuring device 21B, thereby preventing interference of the irradiation light and receiving the reflected light of the irradiation light emitted by the distance measuring device 21B. The modulation frequency of the distance measuring device 21B to be received can be confirmed from modulation frequency information representing the modulation frequency, which is part of the light source information.

[0122] FIG. 13 shows an example of the operation of a plurality of distance measuring devices 21 on the light-emitting side and a distance measuring device 21 on the light-receiving side when the operation mode is set to the light emission pattern mode to prevent interference of irradiated light.

[0123] As in FIG. 11, the distance measuring device 21 on the light-emitting side is the distance measuring device 21A and the distance measuring device 21B, and the distance measuring device 21 on the light-receiving side is the distance measuring device 21E.

[0124] The distance measuring device 21A emits irradiation light with an emission pattern A. The distance measuring device 21B emits irradiation light with an emission pattern B. The light-receiving distance measuring device 21E performs a light receiving operation in synchronization with the emission pattern B of the distance measuring device 21B, thereby preventing interference of the irradiation light and receiving reflected light of the irradiation light emitted by the distance measuring device 21B. The emission pattern of the distance measuring device 21B to be received can be confirmed from emission pattern information representing the emission pattern, which is part of the light source information.

[0125] Returning to the description of FIG. 10, the time synchronization unit 50 exchanges time stamps between the distance measuring device 21 that emits pulsed light and the distance measuring device 21 that receives pulsed light, and synchronizes the time with high accuracy. For example, the time synchronization unit 50 uses a protocol such as PTP (Precision Time Protocol) to synchronize the time with a precision that does not cause problems with light emission and reception between remote light sources and sensors. Alternatively, the time synchronization unit 50 may synchronize the time by adopting a method of synchronizing the shutter timing of multiple cameras using wireless communication, as disclosed in "Sameer Ansari; Neal Wadhwa; Rahul Garg; Jiawen Chen, 'Wireless Software Synchronization of Multiple Distributed Cameras', 2019 IEEE International Conference on Computational Photography, ICCP 2019".

[0126] The timing control unit 51 generates start timing control information based on the time information from the time synchronization unit 50 and the like, and supplies it to the laser driver 46 and the ToF sensor 52.

[0127] The ToF sensor 52 receives light reflected from an object that is emitted from the laser emission unit 47 of the ToF sensor 52 (the distance measuring device 21) or another distance measuring device 21, generates an image signal of the component, and supplies it to the signal processing unit 53. A part of the light source information is supplied to the ToF sensor 52 from the light source information acquisition unit 48 as necessary.

[0128] The ToF sensor 52 controls exposure of each pixel in the pixel array unit based on a light emission pulse signal according to the operation mode from the operation mode setting unit 45, light source information from the light source information acquisition unit 48, and start timing control information from the timing control unit 51, and supplies an image frame of a phase of 0 degrees, a phase of 90 degrees, a phase of 180 degrees, or a phase of 270 degrees to the signal processing unit 53. When light source calibration data is supplied from the light source information acquisition unit 48, the ToF sensor 52 performs a correction process to correct cyclic errors and calculates an image frame of each phase.

[0129] The signal processing unit 53 acquires image frames of each phase of 0 degrees, 90 degrees, 180 degrees, and 270 degrees from the ToF sensor 52. In addition, the signal processing unit 53 acquires information such as the modulation frequency of the received reflected light from the light source information acquisition unit 48.

[0130] Then, the signal processing unit 53 calculates the distance (depth value) d from the distance measuring device 3 to the object 24 in units of pixels using the above-mentioned formula (3) based on the acquired four-phase image frame and information such as the modulation frequency. The signal processing unit 53 generates a depth image in which the distance d from the distance measuring device 3 to the object 24 calculated in units of pixels is stored as the pixel value of each pixel, and outputs the depth image to a downstream device or block.

[0131] When the distance measuring device 21 includes both the light source 2 and the distance measuring device 3, in other words, when the distance measuring device 21 includes both a light emitting function and a light receiving function, the distance measuring device 21 is configured as described above.

[0132] When the distance measuring device 21 is configured with only either a light emitting function or a light receiving function, blocks necessary only for the light emitting function or the light receiving function are omitted as appropriate.

[0133] Specifically, when the distance measuring device 21 is configured only with a function of emitting irradiation light, such as the distance measuring device 21A, the functional block diagram of the distance measuring device 21 is configured as shown in FIG.

[0134] On the other hand, when the distance measuring device 21 is configured only with a reflected light receiving function like the distance measuring device 21C, for example, the functional block diagram of the distance measuring device 21 is configured as shown in FIG.

[0135] <4. Processing flow of the distance measuring device> Next, a description will be given of the control for achieving frequency synchronization and phase synchronization with the pulsed light output from each light source 2 of each distance measuring device 21 having a light emitting function, specifically, distance measuring device 21A, distance measuring device 21B, and distance measuring device 21D. By achieving frequency synchronization and phase synchronization with each distance measuring device 21 having a light emitting function, the distance measuring device 3 of any distance measuring device 21 can receive irradiation light emitted from the light source 2 of another distance measuring device 21 and measure the distance to any object 24.

[0136] First, a light emission control process of the distance measuring device 21 on the light emitting side that emits pulsed light will be described with reference to the flowchart of Fig. 16. This process is started, for example, when the light emitting distance measuring device 21 is turned on.

[0137] Note that, as a premise for the light emission control processing of Figure 16, which is the processing of the light-emitting distance measuring device 21, and the light reception control processing of Figure 17, which is the processing of the light-receiving distance measuring device 21, it is assumed that information regarding the position (relative position) of each distance measuring device 21 that has been measured in advance is stored as light source information in the memory 49 of each distance measuring device 21.

[0138] First, in step S1, the communication module 42 establishes a wireless communication connection with the distance measuring device 21 on the light receiving side that receives the pulsed light emitted by the communication module 42, either directly or via the network 23. For example, in a wireless LAN (Local Area Network) standardized by IEEE (The Institute of Electrical and Electronic Engineers) 802.11, a mechanism for avoiding mutual interference between terminals called CSMA / CA (Carrier Sense Multiple Access / Collision Avoidance) is used to acquire a transmission opportunity and connect to the communication partner.

[0139] In a wireless communication connection, the synchronous clock generating unit 61 of the communication module 42 generates a transmission signal based on a master clock supplied from the clock source 41, and transmits the signal via the antenna 43. The synchronous clock generating unit 61 also generates a synchronous clock signal corresponding to the frequency of the transmission signal (carrier frequency or modulation frequency), and supplies the synchronous clock signal to the reference clock generating unit 44.

[0140] In step S2, the time synchronization unit 50 exchanges time stamps with the distance measuring device 21 on the light receiving side that receives the pulsed light, and synchronizes the time with high accuracy. The synchronized time information is supplied to the timing control unit 51.

[0141] In step S3, the light source information acquisition unit 48 acquires the light source information from the memory 49 and supplies it to the signal transmission / reception unit 62.

[0142] In step S4, the signal transmitting / receiving unit 62 of the communication module 42 encodes the data to be transmitted using a predetermined encoding method and supplies the encoded data to the synchronization clock generating unit 61. The data to be transmitted includes, for example, at least one of the light source information supplied from the light source information acquiring unit 48, such as a unique ID, light emission duration, light emission start time information, and operation mode. The signal transmitting / receiving unit 62 also supplies the operation mode to the operation mode setting unit 45.

[0143] In step S 5 , the reference clock generating unit 44 generates a reference clock signal based on the synchronous clock signal supplied from the synchronous clock generating unit 61 , and supplies the reference clock signal to the operation mode setting unit 45 .

[0144] In step S6, the operation mode setting unit 45 uses the reference clock signal supplied from the reference clock generating unit 44 to generate a light emission pulse signal corresponding to the operation mode supplied from the signal transmitting / receiving unit 62, and supplies it to the laser driver 46.

[0145] In step S7, the timing control unit 51 generates start timing control information based on the time information from the time synchronization unit 50 and the like, and supplies it to the laser driver .

[0146] In step S8, the laser driver 46 supplies a drive signal to the laser emission unit 47 at a predetermined timing based on the start timing control information.

[0147] In step S9, the laser emission unit 47 repeats turning on and off the emission at a predetermined cycle in response to a drive signal from the laser driver .

[0148] This completes the light emission control process.

[0149] Next, a light reception control process of the distance measuring device 21 on the light receiving side that receives pulsed light will be described with reference to the flowchart of Fig. 17. This process is started, for example, when the power of the distance measuring device 21 on the light receiving side is turned on.

[0150] First, in step S21, the communication module 42 establishes a wireless communication connection with the distance measuring device 21 on the light emitting side that emits pulsed light, either directly or via the network 23.

[0151] In a wireless communication connection, the synchronous clock generating unit 61 of the communication module 42 performs synchronization processing on a transmission signal received via the antenna 43, thereby generating a signal equivalent to a synchronous clock signal for data transmission, and supplies the signal to the reference clock generating unit 44. As a result, the operation of the light receiving side is controlled based on a synchronous clock signal based on one clock source 41 on the light emitting side, and the frequencies of the light emitting side and the light receiving side are synchronized.

[0152] In step S22, the time synchronization unit 50 exchanges time stamps with the distance measuring device 21 on the light emitting side that emits pulsed light, and synchronizes the time with high accuracy. The synchronized time information is supplied to the timing control unit 51.

[0153] In step S23, the signal transmitting / receiving unit 62 demodulates the signal supplied from the synchronous clock generating unit 61 and acquires the transmitted data. The signal transmitting / receiving unit 62 supplies, for example, the operation mode acquired from the received signal to the operation mode setting unit 45 and supplies other light source information to the light source information acquiring unit 48.

[0154] In step S24, the light source information acquisition unit 48 stores the light source information acquired from the signal transmission / reception unit 62 in the memory 49, and supplies a part of the light source information to the ToF sensor 52 and the signal processing unit 53.

[0155] In step S 25 , the reference clock generating unit 44 generates a reference clock signal based on the synchronous clock signal supplied from the synchronous clock generating unit 61 , and supplies the generated reference clock signal to the operation mode setting unit 45 .

[0156] In step S26, the operation mode setting unit 45 uses the reference clock signal supplied from the reference clock generation unit 44 to generate a light emission pulse signal corresponding to the operation mode supplied from the signal transmission / reception unit 62, and supplies the light emission pulse signal to the ToF sensor 52.

[0157] In step S27, the ToF sensor 52 receives the reflected light reflected by a specific object 24 based on the light emission pulse signal, and the signal processing unit 53 detects the phase difference of the light emitted by the other ranging device 21 based on the light reception result, and aligns the phase.

[0158] Even if the frequencies are synchronized, the timing of the light emission pulses emitted by each distance measuring device 21 is slightly shifted, so that the phase difference must be aligned.

[0159] For example, as shown in FIG. 18, a distance measuring device 21B emits irradiation light from its own light source 2 and acquires the three-dimensional position of a point F of an object 24 in the light source coordinate system.

[0160] Next, the distance measuring device 21B receives the reflected light of the irradiation light emitted from the light source 2 of the distance measuring device 21A and reflected at the point F of the object 24 in a time-division manner so that the irradiation light does not interfere with each other, and calculates the distance D based on the light reception result. AFB Here, the distance D AFB can be expressed by the following equation:

number

[0161] That is, the distance D AFB is the distance |V from the distance measuring device 21A to a point F on the object 24. AF | and |V from point F of object 24 to distance measuring device 21B FB | and a distance C corresponding to the phase shift of the light source 2 of the distance measuring device 21A based on the phase of the distance measuring device 21B. AB where V is the sum of V and V is the sum of V and V is the AF represents a vector from the distance measuring device 21A to the point F of the object 24, and V FB represents a vector from point F of the object 24 to the distance measuring device 21B. Note that in the specification, a symbol representing a vector (V in FIG. 18) AF and V FB The arrow (→) symbol at the top of |V| represents the absolute value of vector V.

[0162] The signal processor 53 calculates the distance C using the speed of light. AB The phase shift T AB Then, the phase of the light source 2 of the distance measuring device 21A is converted into the phase shift T AB By issuing an instruction to shift the distance measuring devices 21A and 21B by this amount, the phase differences between the distance measuring devices 21A and 21B can be made uniform.

[0163] This completes the light reception control process.

[0164] According to the light emission control process by the light-emitting distance measuring device 21 and the light reception control process by the light-receiving distance measuring device 21, the light-receiving distance measuring device 21 performs synchronization processing on the wireless communication transmission signal sent by the light-emitting side, thereby generating a synchronous clock signal whose frequency is synchronized with the synchronous clock signal generated by the light-emitting distance measuring device 21. Then, time synchronization is performed, and a reference clock signal that serves as a reference clock for the light emission timing and the light reception timing is generated.

[0165] That is, the light emission timing and light reception timing of the distance measuring devices 21 on both the light emission side and the light reception side are controlled based on the single clock source 41 on the light emission side.

[0166] Therefore, even if the light-emitting distance measuring device 21 and both the light-receiving distance measuring devices 21 are configured to be located separately as separate devices, it is possible to achieve frequency synchronization between the laser light emitting unit 47 (light source) and the ToF sensor 52, thereby reducing errors in the distance measurement results.

[0167] As shown in Figures 11 to 13, when there are multiple ranging devices 21 that emit pulsed light, and three or more ranging devices 21 operate simultaneously in any one of the operating modes of (1) time division mode, (2) modulation frequency mode, or (3) emission pattern mode, all of the ranging devices 21 synchronize with the master clock of the clock source 41 of any one of the ranging devices 21 to generate a synchronization clock signal and a reference clock signal.

[0168] When the operation mode is the modulation frequency mode or the light emission pattern mode, frequency synchronization is necessary but time synchronization is not necessary.

[0169] Furthermore, according to the light reception control process described above, the phase difference between the distance measuring device 21 on the light emitting side and the distance measuring device 21 on the light receiving side can be detected and the phase differences can be aligned.

[0170] In the distance measuring system 11, by establishing a state in which the frequencies and phases of the light emission pulses of the multiple distance measuring devices 21 having a light emitting function are synchronized, for example, the distance measuring device 3 of the distance measuring device 21E attached to a movable mobile carrier 22 can measure the distance to an arbitrary object 24 based on the irradiation light emitted from the light source 2 of the multiple distance measuring devices 21. A method in which the distance measuring device 3 of the distance measuring device 21E attached to the mobile carrier 22 measures the distance to the arbitrary object 24 will be described later.

[0171] The distance measuring device 21 on the light emitting side and the distance measuring device 21 on the light receiving side can transmit and receive light source information by wireless communication. Various information such as a unique ID, light emission time, light emission start time information, repetition frequency, modulation frequency, component length, position and attitude of the distance measuring device, frame length, wavelength of the light source, light emission pattern, light source calibration data, and operation mode have been exemplified as information that can be transmitted and received as light source information, but it is not necessary to transmit and receive all of the information, and some of the information may be stored in advance in the memory 49 as fixed data.

[0172] When the light emitting side and the light receiving side are different distance measuring devices 21, it may be difficult for the light receiving side distance measuring device 21, which is another device, to hold the light source calibration data in advance. In such a case, by transmitting the light source calibration data from the light emitting side to the light receiving side as part of the light source information, it becomes possible to detect the phase difference accurately on the light receiving side.

[0173] 5. Modification of the First Embodiment In the first embodiment described above, the distance measuring system 11 is configured such that each distance measuring device 21 synchronizes the synchronous clock signal and the reference clock signal with the clock source 41 of any one of the multiple distance measuring devices 21.

[0174] However, the clock source to which each ranging device 21 synchronizes may be a clock source of another device other than the multiple ranging devices 21 constituting the ranging system 11 .

[0175] For example, signals of Global Navigation Satellite Systems (GNSS) such as Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Galileo, and Quasi-Zenith Satellite System (QZSS) (hereinafter referred to as GNSS signals) have a function of outputting highly accurate clock signals and time information. Each of the multiple ranging devices 21 constituting the ranging system 11 can adopt a configuration in which it receives a GNSS signal and unifies and synchronizes with the clock signal and time information acquired from the GNSS signal.

[0176] FIG. 19 is a block diagram of a ranging device 21' that performs time synchronization based on GNSS signals.

[0177] In FIG. 19, parts corresponding to the configuration shown in FIG. 10 are given the same reference numerals, and the description of those parts will be omitted as appropriate.

[0178] 19 shows two distance measuring devices 21' each having both a light emitting function and a light receiving function. One of the two distance measuring devices 21', distance measuring device 21'-1, is the light emitting side, and the other distance measuring device 21'-2 is the light receiving side. The distance measuring devices 21' having the same configuration operate as the light emitting side or the light receiving side depending on, for example, preset setting information (light source information).

[0179] The ranging device 21' includes a GNSS receiver 81, a time information setting unit 82, a reference clock generation unit 44, an operation mode setting unit 45, a laser driver 46, a laser emission unit 47, a light source information acquisition unit 48, a memory 49, a time synchronization unit 50, a timing control unit 51, a ToF sensor 52, and a signal processing unit 53.

[0180] In other words, compared to the ranging device 21 of FIG. 10, the ranging device 21′ has a configuration in which the clock source 41, the communication module 42, and the antenna 43 are replaced with a GNSS receiving unit 81 and a time information setting unit 82.

[0181] The GNSS receiver 81 receives a GNSS signal from a GNSS satellite 83 and supplies it to a time information setting unit 82 and a reference clock generator 44 .

[0182] The time information setting unit 82 obtains highly accurate time information from the GNSS signal supplied from the GNSS receiving unit 81, and synchronizes the time to standard time.

[0183] The reference clock generation unit 44 extracts a clock signal from the GNSS signal, and generates a reference clock signal that serves as a reference when the laser emission unit 47 emits light based on the clock signal, and supplies the reference clock signal to the operation mode setting unit 45.

[0184] The memory 49 stores light source information and supplies it to the light source information acquisition unit 48 as necessary. The light source information in the memory 49 stores information necessary for the distance measuring device 21' to operate as the light emitting side or the light receiving side. For example, the memory 49 of the light receiving side distance measuring device 21'-2 stores data such as the operation mode, light emission pattern, and light source calibration data of the light emitting side distance measuring device 21' in advance.

[0185] The distance measuring device 21' having the above configuration can synchronize the light emission timing and the light reception timing using a clock source of a device (GNSS satellite 83) different from both the light emitting side and the light receiving side distance measuring device 21'. That is, the frequency of the laser emission unit 47 on the light emitting side and the ToF sensor 52 on the light receiving side can be synchronized, and the error in the distance measurement result can be reduced.

[0186] In the above example, the GNSS signal is received, and the time and clock are synchronized based on the received GNSS signal, and the transmission and reception of data such as light source information is not performed as it is set in advance. However, it is also possible to use another wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark) to transmit and receive data such as light source information.

[0187] In addition to the above-mentioned GNSS signal, a radio signal for UWB (Ultra Wide Band) communication may be used as a signal for outputting a highly accurate clock signal and time information. In this case, the GNSS receiver 81 of the ranging device 21' is used as a UWB receiver 81, and the UWB receiver 81 receives a radio signal for UWB (Ultra Wide Band) communication.

[0188] 19, the distance measuring device 21' on the light-emitting side and the distance measuring device 21' on the light-receiving side are configured to be the same, and the distance measuring device 21' can be either the light-emitting side or the light-receiving side. However, when limited to either the light-emitting side or the light-receiving side, unnecessary components can be omitted, as in the case of FIGS. 14 and 15.

[0189] <6. Second embodiment of distance measuring system> FIG. 20 illustrates an example configuration of a distance measuring system according to the second embodiment of the present disclosure.

[0190] The distance measuring system 11 according to the second embodiment of Fig. 20 is composed of multiple drones 101. The drone 101 is an airborne vehicle having multiple rotors, and has a distance measuring device including both a light source 2 and a distance measuring device 3.

[0191] The ranging system 11 can be configured with three or more drones 101, but for ease of explanation, it is assumed below that the system is configured with two drones 101A and 101B as shown in Fig. 20. The drones 101A and 101B have the same configuration, but have different unique IDs for identifying the drones. For example, the unique ID of the drone 101A is "0" (ID0), and the unique ID of the drone 101B is "1" (ID1).

[0192] The drone 101A irradiates the drone 101B with irradiation light of a predetermined modulation frequency, and receives the irradiation light irradiated from the drone 101B. The drone 101B also irradiates the drone 101A with irradiation light of a predetermined modulation frequency, and receives the irradiation light irradiated from the drone 101A.

[0193] The synchronous clock signals and the reference clock signals of the drones 101A and 101B are controlled to be synchronized based on the GNSS signal from the GNSS satellite 102, as in the modified example of the first embodiment.

[0194] Drones 101A and 101B can accurately measure the distance to each other (hereinafter referred to as the relative distance) by exchanging the light propagation time or distance that they have measured.

[0195] The principle of relative distance calculation will be described with reference to FIGS.

[0196] First, as shown in FIG. 21, each of drones 101A and 101B includes a light source 2 and a distance measuring device 3.

[0197] The light source 2 of the drone 101A irradiates the drone 101B with light of a predetermined modulation frequency. The distance measuring device 3 of the drone 101A receives the light irradiated from the light source 2 of the drone 101B.

[0198] The light source 2 of the drone 101B irradiates the drone 101A with irradiation light of a predetermined modulation frequency. The distance measuring device 3 of the drone 101B receives the irradiation light irradiated from the light source 2 of the drone 101A.

[0199] Here, if the relative distance between drones 101A and 101B is d, the propagation time of light between drones 101A and 101B is Δt d can be expressed by equation (8).

number

[0200] FIG. 22 is a schematic diagram showing pulsed light emitted by drones 101A and 101B in the indirect ToF method.

[0201] Drones 101A and 101B can synchronize the emission timing of their illumination lights by sharing highly accurate time information based on GNSS signals from GNSS satellites 102. However, it is difficult to synchronize the timing of each individual light emission pulse.

[0202] Specifically, even if the light emission timings of the drones 101A and 101B are aligned, as shown in FIG. 22, the light emission pulse of the drone 101A is delayed by Δt offset Sometimes it just seems to be delayed.

[0203] That is, the light propagation time Δt between the drones 101 observed by the distance measuring device 3 of the drone 101B 1→2 is the offset Δt according to the difference in the light emission timing offset This is also influenced by and is expressed by the following equation (9). Δt 1→2 = Δt d +Δt offset (9)

[0204] On the other hand, from the perspective of drone 101A, the light emission pulse of drone 101B is Δtoffset Therefore, the light propagation time between drones 101 observed by the distance measuring device 3 of drone 101A is Δt 2→1 is expressed by the following equation (10). Δt 2→1 = Δt d -Δt offset (10)

[0205] Therefore, the drones 101A and 101B inform each other of the propagation time Δt measured by themselves, and the offset Δt is calculated by the following equation (11). offset The exact light propagation time Δt excluding the effect of d Then, the accurate relative distance d can be calculated by using equation (12), which is a modification of equation (8).

number

[0206] In addition, drones 101A and 101B may inform each other of the propagation time Δt, or may inform each other of the distance d obtained by multiplying the propagation time Δt by the speed of light c, and the average of the distances d of both drones may be calculated, thereby obtaining an accurate relative distance d.

[0207] FIG. 23 is a schematic diagram showing pulsed light emitted by drones 101A and 101B in the direct ToF method.

[0208] The direct ToF method can also be formulated in exactly the same way as the indirect ToF method described with reference to FIG.

[0209] That is, as shown in FIG. 23, when viewed from drone 101B, the light emission pulse of drone 101A is Δt offset If the distance between the drones 101B and 101C appears to be delayed by Δt, the light propagation time between the drones 101 observed by the distance measuring device 3 of the drone 101B is 1→2 is expressed by equation (9).

[0210] On the other hand, the light propagation time Δt between drones 101 observed by the distance measuring device 3 of drone 101A 2→1 is expressed by equation (10).

[0211] Therefore, according to equation (11), the offset Δt offset The exact light propagation time Δt excluding the effect of d Then, the accurate relative distance d can be calculated using equation (12).

[0212] <Drone block diagram> FIG. 24 is a block diagram of a ranging device built into the drone 101.

[0213] In FIG. 24, the same reference numerals are used to denote parts corresponding to the distance measuring device 21' in FIG. 19, and the description of those parts will be omitted as appropriate.

[0214] The drone 101 (the distance measuring device) has a configuration in which an IMU 121, a position and attitude calculation unit 122, a communication unit 123, and an inter-light source distance calculation unit 124 are further added to the configuration of the distance measuring device 21′ shown in FIG.

[0215] In the second embodiment, a light source information acquisition unit 48 acquires light source information such as an operation mode stored in a memory 49, and supplies the information to an operation mode setting unit 45, a timing control unit 51, a signal processing unit 53, and the like as appropriate.

[0216] The IMU 121 is a device that detects angles (or angular velocities) and accelerations in three axial directions of the drone 101, and supplies a signal indicating the detection results to a position and attitude calculation unit 122.

[0217] The position / attitude calculation unit 122 detects the position and attitude of the drone 101 based on the detection signal supplied from the IMU 121 and the position information supplied from the GNSS receiving unit 81, and supplies the position and attitude to the communication unit 123 and the inter-light source distance calculation unit 124. The position / attitude calculation unit 122 may detect either the position or the attitude of the drone 101.

[0218] Note that the IMU 121 is not an essential component and may be omitted. In this case, the position and attitude calculation unit 122 detects the position and attitude of the drone 101 based on the position information supplied from the GNSS receiving unit 81. Also, instead of the IMU 121, other sensors such as a magnetic sensor, a pressure sensor, etc. may be provided, and the position and attitude of the drone 101 may be detected based on a sensor detection signal. Also, both the IMU 121 and a magnetic sensor, etc. may be provided.

[0219] The signal processing unit 53 includes a depth image generating unit 141 and an external modulated light source detecting unit 142 .

[0220] The depth image generating unit 141 generates a depth image in which the distance d to the other drone 101 calculated in pixel units is stored as the pixel value of each pixel. The generated depth image is supplied to the communication unit 123 and the light source distance calculation unit 124.

[0221] The external modulated light source detection unit 142 detects the position (light source position) of the light source 2 (laser emission unit 47) emitted by the other drone 101.

[0222] In the indirect ToF method, the externally modulated light source detection unit 142 detects the light source position using a confidence image in which a confidence value indicating the intensity of light is stored as the pixel value of each pixel.

[0223] FIG. 25 shows an example of a confidence image when drone 101A observes drone 101B, and an example of a confidence image when drone 101B observes drone 101A.

[0224] The externally modulated light source detection unit 142 detects, from among the pixels of the confidence image, a pixel position where the pixel value (confidence value) is equal to or greater than a predetermined threshold and where the pixel value is maximum, as the light source position.

[0225] On the other hand, in the direct ToF method, the external modulated light source detection unit 142 detects, as the light source position, a pixel position where the peak value of the generated histogram is equal to or greater than a predetermined threshold value and where the peak value is the maximum.

[0226] The drones 101A and 101B can emit light in a time-division manner and operate by separating light emission and light reception. Alternatively, the drones 101A and 101B may emit light and receive light simultaneously in an environment where there are no other objects around the drones 101A and 101B, for example, when the background other than the drone 101 is the sky.

[0227] Then, the signal processing unit 53 detects the light source position detected by the external modulation light source detection unit 142 and the distance d detected at that light source position. 1 is supplied to the communication unit 123 and the light source distance calculation unit 124.

[0228] The communication unit 123 communicates with the other drone 101 by wireless communication such as WiFi (registered trademark), Bluetooth (registered trademark), or mobile communication such as so-called 4G line or 5G line. The communication unit 123 transmits the light source position and distance d of the other drone 101 obtained by the signal processing unit 53 through wireless communication. 1 The drone 101 transmits its own rough position detected by the position and attitude calculation unit 122 to the other drone 101. The communication unit 123 also transmits the light source position and distance d 2 The rough position of the other drone 101 is then received and supplied to the light source distance calculation unit 124.

[0229] The light source distance calculation unit 124 receives the light source position and the distance d from the other drone 101 via wireless communication. 2 The inter-light source distance calculation unit 124 obtains the distance d 1 and the light source position are obtained from the signal processing unit 53. The light source distance calculation unit 124 calculates the distance d 1 and the distance d measured by the other drone 1012 Using the above offset Δt offset Furthermore, the inter-light source distance calculation unit 124 uses its own rough position and orientation detected by the position and orientation calculation unit 122 to improve the accuracy of the relative distance d.

[0230] FIG. 26 is a diagram for explaining a process of improving the accuracy of the relative distance d by using the rough orientation of the object itself detected by the position / orientation calculation unit 122. In FIG.

[0231] The direction of the other party's drone 101 is grasped from the light source position detected in the confidence image by the principle of a pinhole camera. In addition, the distance d 1 and the distance d sent by the other party 2 Using this, the accurate relative distance d is calculated.

[0232] Under these conditions, as shown in A of FIG. 26, there is arbitrariness in the rotation around the line of sight, and the position and orientation cannot be uniquely determined.

[0233] 26B, the direction of gravity of the ToF sensor 52 can be identified by using the detection result of the IMU 121, and the arbitrariness is solved by using the detection result of the IMU 121. That is, for images observed by the ToF sensor 52 of either the drone 101A or the drone 101B, the relative position and orientation can be uniquely determined based on the detection result of the IMU 121. If a magnetic sensor is provided instead of the IMU 121, the magnetic sensor can also detect a direction common to both the drone 101A and the drone 101B, and therefore the relative position and orientation can be determined.

[0234] As described above, by using the highly accurate distance d calculated by mutually exchanging the distance d with the drone 101 and the attitude (tilt) detected by the IMU 121, the relative position and attitude can be highly accurately determined.

[0235] Figure 27 is a diagram explaining the process of improving the accuracy of the relative distance d by using the rough position of the drone itself detected by the position / attitude calculation unit 122 and the rough position of the other drone 101 transmitted from the other drone 101.

[0236] In the indirect ToF method, the measurable distance d is limited by the modulation frequency. For example, when the modulation frequency is 20 MHz, the phase completes one revolution in about 15 m, so as shown in Figure 27, the phase difference detected when the relative distance d is 15 m and 30 m is the same value, making it impossible to distinguish between distances of 15 m and 30 m.

[0237] As a countermeasure to this, it is possible to determine whether the relative distance d is 15 m or 30 m by using the rough position of the drone itself detected by the position / attitude calculation unit 122 and the rough position of the other drone 101 transmitted from the other drone 101. In other words, it is possible to improve the accuracy of the relative distance d by using the rough position of the drone itself detected by the position / attitude calculation unit 122.

[0238] In addition, to determine whether the relative distance d is 15 m or 30 m, a method may be used in which multiple modulation frequencies are set and the final distance is determined from the common distance obtained in the measurement results using each modulation frequency.

[0239] <Processing flow of relative distance measurement processing> Next, the relative distance measurement process of the drone 101 will be described with reference to the flowchart of Fig. 28. This process is started, for example, after the drone 101 starts flying.

[0240] First, in step S41, the GNSS receiver 81 receives a GNSS signal from a GNSS satellite 83 and supplies it to the time information setting unit 82 and the reference clock generation unit 44. The time information setting unit 82 acquires highly accurate time information from the GNSS signal supplied from the GNSS receiver 81 and synchronizes the time to standard time. The reference clock generation unit 44 extracts a clock signal from the GNSS signal, generates a reference clock signal based on the clock signal, and supplies it to the operation mode setting unit 45. The GNSS receiver 81 also supplies the position information acquired from the GNSS signal to the position and attitude calculation unit 122.

[0241] In step S42, the position / attitude calculation unit 122 detects the position and attitude of the drone 101 based on the detection signal supplied from the IMU 121 and the position information supplied from the GNSS reception unit 81, and supplies the position and attitude to the communication unit 123 and the inter-light-source distance calculation unit 124. When the IMU 121 is omitted, the position and attitude of the drone 101 are detected based only on the position information supplied from the GNSS reception unit 81.

[0242] In step S43, the timing control unit 51 generates start timing control information based on the highly accurate time information from the time information setting unit 82 and the like, and supplies the start timing control information to the laser driver 46 and the ToF sensor 52.

[0243] In step S44, the operation mode setting unit 45 uses the reference clock signal supplied from the reference clock generation unit 44 to generate an emission pulse signal corresponding to the operation mode supplied from the light source information acquisition unit 48, and supplies it to the laser driver 46 and the ToF sensor 52.

[0244] In step S45, the laser driver 46 generates a drive signal and supplies it to the laser emission unit 47 at a predetermined timing based on the start timing control information.

[0245] In step S46, the laser emission unit 47 repeats turning on and off the emission at a predetermined cycle in response to a drive signal from the laser driver 46.

[0246] In step S47, the ToF sensor 52 receives the light irradiated by the other drone 101 based on the start timing control information from the timing control unit 51 and the reference clock signal from the operation mode setting unit 45, and sequentially generates image frames of each phase, namely, phase 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees, and supplies them to the signal processing unit 53.

[0247] In step S48, the depth image generating unit 141 of the signal processing unit 53 calculates the distance d 1 The depth image storing unit 122 stores the above-mentioned pixel values ​​as pixel values ​​of each pixel, and supplies the image to the communication unit 123 and the light source distance calculation unit 124.

[0248] In step S49, the external modulated light source detection unit 142 detects the position (light source position) of the light source 2 (laser emission unit 47) emitted by the other drone 101, and supplies it to the communication unit 123 and the light source distance calculation unit 124.

[0249] In step S50, the communication unit 123 transmits, by wireless communication, the light source position and distance d of the other drone 101 obtained by the signal processing unit 53. 1 The communication unit 123 transmits the light source position and distance d detected by the drone 101 to the other drone 101. 2 The rough position of the other drone 101 is then received and supplied to the light source distance calculation unit 124.

[0250] In step S51, the light source distance calculation unit 124 calculates an accurate relative distance d to the other drone 101. Specifically, the light source distance calculation unit 124 specifies the direction of gravity using its own rough orientation detected by the position and orientation calculation unit 122, and obtains a relative position and orientation. In addition, the light source distance calculation unit 124 calculates the light source position and distance d received from the other drone 101 via wireless communication. 2The inter-light source distance calculation unit 124 obtains the distance d 1 and the distance d sent by the other party 2 and offset Δt offset In addition, the inter-light source distance calculation unit 124 uses the rough positions detected by the drone 101 itself and the other drone 101 to improve the accuracy of the relative distance d.

[0251] This is the end of the relative distance measurement process. The above-described relative distance measurement process is executed in parallel by each of the drones 101A and 101B.

[0252] According to the above relative distance measurement process, the distance d 1 (or d 2 ), the exact distance d can be calculated.

[0253] In the second embodiment, as in the first embodiment, frequency synchronization of drones 101A and 101B is performed by synchronizing based on received radio signals, but a method of performing frequency synchronization using reflected light (light emitted by the master) of the master drone 101, as disclosed in Patent Document 1, may also be adopted.

[0254] <7. Third embodiment of distance measurement system> FIG. 29 illustrates an example configuration of a distance measuring system according to the third embodiment of the present disclosure.

[0255] The ranging system 11 according to the third embodiment in Fig. 29 includes a plurality of ranging devices 201. In the example of Fig. 29, the ranging system 11 includes five ranging devices 201A to 201E, but the number of ranging devices 201 is not limited to five and may be any number.

[0256] In the third embodiment, the four distance measuring devices 201A to 201D each include both a light source 2 and a distance measuring device 3, and are fixed above, for example, on a ceiling. On the other hand, the distance measuring device 201E includes only the distance measuring device 3, and is placed at a position where it can receive the light emitted from the four distance measuring devices 201A to 201D or its reflected light, for example, on the ground.

[0257] The configurations of the distance measuring devices 201A to 201D, which have both the light source 2 and the distance measuring device 3, are similar to the configuration shown in FIG. 10 in the first embodiment, and the configuration of the distance measuring device 201E, which has only the distance measuring device 3, is similar to the configuration shown in FIG. 15 in the first embodiment, so the description thereof will be omitted.

[0258] In the third embodiment, a process will be described in which a distance measuring device 201E equipped only with a distance measuring apparatus 3 calculates the distance to an object 202 by receiving light emitted from multiple (four) distance measuring devices 201A to 201D fixed to a ceiling or the like.

[0259] FIG. 30 is a flowchart illustrating the distance calculation process of the distance measuring system 11 in the third embodiment.

[0260] First, in step S61, each of the distance measuring devices 201A to 201E establishes a wireless communication connection. Then, each of the distance measuring devices 201A to 201E exchanges time stamps and synchronizes the time with high accuracy.

[0261] In step S62, each of the distance measuring devices 201A to 201E reads out the light source information from its own memory 49 and transmits it by wireless communication to the other distance measuring devices 201. In this way, the light source information of each of the distance measuring devices 201A to 201E is shared.

[0262] In step S63, each of the ranging devices 201A to 201E generates a reference clock signal from a synchronous clock signal generated by detecting a transmission signal of the master ranging device 201 (for example, the ranging device 201A), and generates a light emission pulse signal according to the operation mode.

[0263] In step S64, each of the distance measuring devices 201A to 201D fixed to the ceiling emits and receives light based on the light emission pulse signal. Specifically, the light source 2 (laser emitter 47) of each of the distance measuring devices 201A to 201D emits light in response to the light emission pulse signal, and the distance measuring device 3 (ToF sensor 52) receives reflected light in response to the light emission pulse signal to sequentially generate image frames of phase 0 degrees, phase 90 degrees, phase 180 degrees, and phase 270 degrees.

[0264] In step S65, each of the distance measuring devices 201A to 201D fixed to the ceiling calculates its own position and orientation, in other words, its position and orientation relative to the other distance measuring devices 201.

[0265] The process of step S65 will now be described. Each of the distance measuring devices 201A to 201D is a depth camera that measures the distance d to the subject, and therefore can obtain the three-dimensional position of the subject in the camera coordinate system.

[0266] A method of determining the relative position and orientation of a 3D point and a camera by observing the known 3D point with a camera is generally known as PNP (Perspective-n-Point). Each of the ranging devices 201A to 201D calculates the position and orientation of its own ranging device 201 relative to the other ranging devices 201 using PNP.

[0267] For example, the distance measuring device 201A receives reflected light from a known object 202 to obtain coordinate values ​​of feature points of the object 202. Similarly, the distance measuring device 201B receives reflected light from the known object 202 to obtain coordinate values ​​of feature points of the object 202. The relative positions and orientations of the distance measuring devices 201A and 201B are calculated by solving the PNP problem using the coordinate values ​​of the feature points obtained by the distance measuring devices 201A and 201B and the coordinate values ​​of known three-dimensional points.

[0268] Alternatively, an optimization problem can be solved to best match the shapes of the surrounding environments created by the distance measuring devices 201A and 201B, and the relative orientation of the distance measuring devices 201A and 201B can be obtained. This method is known as ICP (Iterative Clisest Point). ICP is disclosed, for example, in "Digital Image Processing [Revised New Edition], Association for the Promotion of Education in Image Information" and in Szymon.R and Marc.L, "Efficient Variants of the ICP Algorithm", Proceedings Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 145-152.

[0269] In this manner, the distance measuring devices 201A to 201D each calculate the relative position and orientation, thereby grasping the relative relationship between the positions and orientations of all the distance measuring devices 201A to 201D.

[0270] In step S66, each of the distance measuring devices 201A to 201D fixed to the ceiling detects the phase difference of the irradiated light and aligns the phases.

[0271] As described in FIG. 22 of the second embodiment, even if frequency synchronization is performed, the timing of the light emission pulses emitted by each distance measuring device 201 is slightly shifted, and therefore the phase difference needs to be calibrated.

[0272] Specifically, for example, as shown in FIG. 31, distance measuring devices 201A and 201B irradiate an object 202 at the same physical position with irradiation light and receive the reflected light.

[0273] The phase difference between the light emission pulses of the distance measuring devices 201A and 201B is Δt offset_12 Then, the time it takes for the light emitted by the distance measuring device 201A to be received by the distance measuring device 201B is given by the following equation (13). Δt 1→2 = Δt d +Δt offset_12 (13)

[0274] On the other hand, the time it takes for the light emitted by the distance measuring device 201B to be received by the distance measuring device 201A is given by the following formula (14). Δt 2→1 = Δt d -Δt offset_12 (14)

[0275] Therefore, the difference (phase difference) Δt offset_12 can be calculated using the following equation (15). Δt offset_12 =(Δt 1→2 -Δt 2→1 ) / 2 (15)

[0276] In this manner, the phase difference between the light sources 2 of the distance measuring devices 201A to 201D is detected, and the phase of the illumination light of each of the distance measuring devices 201A to 201D is adjusted so that there is no phase difference. Note that instead of adjusting the phase of the illumination light, the phase difference may be taken into consideration when calculating in the signal processing at a later stage.

[0277] In step S67, the ToF sensor 52 of the distance measuring device 201E including only the distance measuring unit 3 receives the reflected light of the illumination light emitted by each of the distance measuring devices 201A to 201D fixed to the ceiling and reflected by the object 202. The frequency and phase of the illumination light emitted by each of the distance measuring devices 201A to 201D are synchronized by the process of step S66. In addition, since the light source information is shared, it is possible to identify which of the distance measuring devices 201A to 201D emitted the reflected light currently being received.

[0278] In step S68, the signal processing unit 53 of the distance measuring device 201E calculates the distance to the object 202 based on the phase difference between the reflected light received by each of the distance measuring devices 201A to 201D. The method of calculating the distance to the object 202 will be described later.

[0279] In the above distance calculation process, the light source information of the light irradiated from the ranging devices 201A to 201D fixed to the ceiling is known, and the phase and frequency are synchronized, so that the position and orientation of the ranging device 201E can be calculated by detecting the phase difference between the light irradiated from the ranging devices 201A to 201D, in the same way as when the ranging devices 201A to 201D are considered to be GNSS satellites and their own position is calculated from the difference in GNSS signals.

[0280] In the third embodiment, as in the first embodiment, frequency synchronization of drones 101A and 101B is performed by synchronizing based on received radio signals, but a method of performing frequency synchronization using reflected light (light emitted by the master) of a master ranging device 201, as disclosed in Patent Document 1, may also be adopted.

[0281] <8. Distance calculation method by receiving reflected light from multiple light sources> In the first embodiment described above, the distance measuring device 3 of the distance measuring device 21E can measure the distance to the object 24, for example, by receiving reflected light that is emitted from the light source 2 of the distance measuring device 21A and the light source 2 of the distance measuring device 21B and is reflected by the object 24.

[0282] In addition, in the above-mentioned third embodiment, the distance measuring device 3 of the distance measuring device 201E can measure the distance to the object 202, for example, by receiving reflected light that is emitted from the light source 2 of the distance measuring device 201B and the light source 2 of the distance measuring device 201D and reflected by the object 202.

[0283] In the following, a calculation method for measuring the distance to a predetermined object by receiving light emitted by a plurality of light sources 2 and reflected by the predetermined object will be described.

[0284] As a system arrangement for explanation, three light sources a to c, an object 301, and a distance measuring device 302 are assumed as shown in Fig. 32. Point D is the reflection position of the pulsed light from the light sources a to c on the object 301. Point O is the origin of the camera coordinate system, which is the coordinate system of the distance measuring device 302.

[0285] The light sources a to c correspond to, for example, the light source 2 of the distance measuring devices 21A, 21B, and 21D in the first embodiment, and correspond to, for example, the light source 2 of the distance measuring devices 201A, 201B, and 201D in the third embodiment. The object 301 corresponds to the object 24 in the first embodiment, and corresponds to the object 202 in the third embodiment. The distance measuring device 302 corresponds to the distance measuring device 3 of the distance measuring device 21E in the first embodiment, and corresponds to the distance measuring device 3 of the distance measuring device 201E in the third embodiment.

[0286] In FIG. 32, three light sources a to c are located at positions E 1 , E 2 , and E 3The relative positions of the light sources a to c in the light source coordinate system are known, but the locations of the three light sources a to c as a whole are unknown. The relative position of the distance measuring device 302 with respect to the positions of the light sources a to c is unknown. The distance measuring device 302 can perform frequency synchronization based on the wireless signals from the light sources a to c, but a phase shift remains.

[0287] The distance measuring device 302 calculates the distance (the distance from each of the light sources a to c to the distance measuring device 302) using frames acquired based on the light emitted by the light sources a to c and reflected by the object 301. The distance from the light sources a to c to the distance measuring device 302 is expressed by the following equation (16).

[0288]

number

[0289] In equation (16), u D =(u D ,v D ) is the pixel position where point D on the object 301 is projected, Obs(u D ,E 1 ) is at position E 1 When the light source a located at D The observed distance, Obs(u D ,E 2 ) is at position E 2 When the light source b located at D The observed distance, Obs(u D ,E 3 ) is at position E 3 When light source c located at D is the magnitude (distance) of the vector connecting the distance measuring device 302 and the object 301, and is |DE k | is position E krepresents the magnitude (distance) of the vector connecting each light source (k=1, 2, 3) and the reflection position on the object 301. C represents an offset term equivalent to the measurement distance error caused by the phase shift between the light sources a to c and the clock of the distance measuring device 302.

[0290] In addition, the vector notation (arrow (→) symbol) in the above formula (16) indicates a vector from the origin of a certain coordinate system, for example, a light source coordinate system. Note that vector notation is omitted in the explanation of the above formula in the specification. Note that in the following explanations in the specification, vector notation is omitted.

[0291] In this way, the distance measuring device 302 calculates the three distances from the three light sources a to c to the distance measuring device 302 as follows: (a) distance data |OD| between the distance measuring device 302 and the object 301; (b) Distance data |DE between the object 301 and each of the light sources a to c k |And, (c) an offset C corresponding to a distance measurement error occurring based on a phase difference between the clock (sensor clock) of the distance measuring device 302 and the clocks (light source clocks) of the light sources a to c; Three distance calculation formulas are calculated using the data.

[0292] Next, the distance measuring device 302 calculates the three distances Obs(u D ,E 1 ), distance Obs(u D ,E 2 ), and distance Obs(u D ,E 3 ) to generate the simultaneous equations of equation (17) consisting of the calculation formulas for the differences in distances.

[0293]

number

[0294] By solving the simultaneous equations shown in Equation (17) above using the light source coordinate system, multiple object positions L Dk and the corresponding pixel position u D It is possible to obtain the above-mentioned multiple object positions L Dk For example, the positions E of the light sources a to c are 1 ,E 2 ,E 3 and the surface position of the object 301.

[0295] Generally, if the correspondence between multiple points in a coordinate system and the pixel positions where those points are projected by a camera is known, the relative position and orientation between that coordinate system and the camera coordinate system can be obtained. In other words, by solving the so-called PnP problem, the relative relationship between the light source coordinate system and the camera coordinate system can be derived, and the subject position in the light source coordinate system can be converted to the camera coordinate system.

[0296] This process calculates the subject position L in the camera coordinate system. Dk and the corresponding pixel position u D From the pixel positions of the light sources a to c, the positions E 1 ,E 2 ,E 3 It is then possible to obtain the surface position of the object 301 in the camera coordinate system. By obtaining the surface position of the object 301 in the camera coordinate system, the distance to the object 301, i.e., the distance from the distance measuring device 302 to the object 301, can be calculated.

[0297] <9.ToF sensor configuration> The configuration of the ToF sensor 52 will be described.

[0298] FIG. 34 shows an example of the configuration of the light receiving section of the ToF sensor 52.

[0299] The ToF sensor 52 has a pixel array section 232 in which pixels 231 are arranged two-dimensionally in a matrix in the row and column directions, and the pixels 231 generate charges according to the amount of light received and output signals according to the charges, and a drive control circuit 233 arranged in the peripheral area of ​​the pixel array section 232.

[0300] The drive control circuit 233 outputs control signals (such as a distribution signal DIMIX, a selection signal ADDRESS DECODE, and a reset signal RST, which will be described later) for controlling the driving of the pixels 231 based on a light emission pulse signal or the like.

[0301] The pixel 231 has a photodiode 251, and a first tap 252A and a second tap 252B that detect charges photoelectrically converted by the photodiode 251. In the pixel 231, charges generated in one photodiode 251 are distributed to the first tap 252A or the second tap 252B. Of the charges generated in the photodiode 251, the charge distributed to the first tap 252A is output as a detection signal A from a signal line 253A, and the charge distributed to the second tap 252B is output as a detection signal B from a signal line 253B.

[0302] The first tap 252A is composed of a transfer transistor 241A, an FD (Floating Diffusion) section 242A, a selection transistor 243A, and a reset transistor 244A. Similarly, the second tap 252B is composed of a transfer transistor 241B, an FD section 242B, a selection transistor 243B, and a reset transistor 244B.

[0303] The operation of the pixel 231 will now be described.

[0304] As shown in FIG. 2, the light source 2 outputs the modulated irradiation light so as to repeat the on / off of irradiation in a 1 / f cycle, and the reflected light is received by the photodiode 251 after a delay time ΔT according to the distance to the object. The distribution signal DIMIX_A controls the on / off of the transfer transistor 241A, and the distribution signal DIMIX_B controls the on / off of the transfer transistor 241B. At the execution timing of phase 0 degrees, the distribution signal DIMIX_A is a signal with the same phase as the irradiation light (i.e., phase 0 degrees), and the distribution signal DIMIX_B has an inverted phase of the distribution signal DIMIX_A (i.e., phase 180 degrees). At the execution timing of phase 90 degrees, the distribution signal DIMIX_A is a signal with a phase shift of 90 degrees with respect to the irradiation light, and the distribution signal DIMIX_B has an inverted phase of the distribution signal DIMIX_A (i.e., phase 270 degrees).

[0305] 34, the charge generated by the photodiode 251 receiving reflected light is transferred to the FD section 242A while the transfer transistor 241A is on in accordance with the distribution signal DIMIX_A, and is transferred to the FD section 242B while the transfer transistor 241B is on in accordance with the distribution signal DIMIX_B. As a result, during a predetermined period in which irradiation with irradiation light is periodically performed at a 1 / f cycle, the charge transferred via the transfer transistor 241A is sequentially accumulated in the FD section 242A, and the charge transferred via the transfer transistor 241B is sequentially accumulated in the FD section 242B.

[0306] Then, after the accumulation period for accumulating the charges ends, when the selection transistor 243A is turned on in accordance with the selection signal ADDRESS DECODE_A, the charges accumulated in the FD section 242A are read out via the signal line 253A, and a detection signal A corresponding to the amount of the charges is output from the pixel array section 232. Similarly, when the selection transistor 243B is turned on in accordance with the selection signal ADDRESS DECODE_B, the charges accumulated in the FD section 242B are read out via the signal line 253B, and a detection signal B corresponding to the amount of the charges is output from the pixel array section 232. Also, the charges accumulated in the FD section 242A are discharged when the reset transistor 244A is turned on in accordance with the reset signal RST_A, and the charges accumulated in the FD section 242B are discharged when the reset transistor 244B is turned on in accordance with the reset signal RST_B.

[0307] In this way, the pixel 231 distributes the charge generated by the reflected light received by the photodiode 251 to the first tap 252A or the second tap 252B according to the delay time ΔT, and outputs the detection signal A and the detection signal B. The detection signal A and the detection signal B at the execution timing of phase 0 degrees are respectively the charge Q 0 , and a charge Q of 180 degrees in phase 180 The detection signals A and B at the execution timing of the phase 90 degrees correspond to the charge Q 90 , and a charge Q of phase 270 degrees 270 Corresponds to.

[0308] <10. Application examples for mobile devices> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.

[0309] FIG. 35 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.

[0310] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 35, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (interface) 12053 are illustrated.

[0311] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0312] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0313] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.

[0314] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0315] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.

[0316] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0317] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0318] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0319] The audio / video output unit 12052 transmits at least one output signal of audio and video to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 35, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

[0320] FIG. 36 is a diagram showing an example of the installation position of the imaging unit 12031.

[0321] In FIG. 36, a vehicle 12100 has imaging units 12101, 12102, 12103, 12104, and 12105 as an imaging unit 12031.

[0322] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The images of the front acquired by the imaging units 12101 and 12105 are mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0323] 36 shows an example of the imaging ranges of the imaging units 12101 to 12104. An imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and an imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.

[0324] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.

[0325] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.

[0326] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0327] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

[0328] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 in the configuration described above. Specifically, the above-mentioned distance measuring device 21 can be applied as the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, distance information can be acquired. In addition, the obtained distance information can be used to increase the safety of the driver and the vehicle.

[0329] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0330] The present technologies described in this specification can be implemented independently and individually, as long as no contradiction occurs. Of course, any of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in another embodiment. In addition, any of the present technologies described above can be implemented in combination with other technologies not described above.

[0331] Also, for example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Also, it is of course possible to add configurations other than those described above to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0332] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0333] It should be noted that the effects described in this specification are merely examples and are not limiting, and there may be effects other than those described in this specification.

[0334] The present technology can have the following configurations. (1) a receiving unit that performs synchronization processing of a transmission signal transmitted from another device via wireless communication to generate a signal synchronized with a synchronous clock signal of the other device; a reference clock generating unit that generates a reference clock signal that is a reference when the light source emits light, based on the synchronous clock signal; a ToF sensor that receives light reflected from an object that is irradiated with light from the light source based on the reference clock signal; A ranging device comprising: (2) The receiving unit receives light source information regarding the irradiation light emitted from the light source via the wireless communication. A ranging device as described in (1) above. (3) The light source information includes calibration data for the light source. A ranging device as described in (2) above. (4) The light source information includes an operation mode. A ranging device according to (2) or (3). (5) The operation mode includes information indicating one of a time division mode, a modulation frequency mode, and a light emission pattern mode. A ranging device as described in (4) above. (6) The light source information includes any one of a light emission time, light emission start time information, a modulation frequency, a wavelength of the light source, and a light emission pattern. A ranging device according to any one of (2) to (5). (7) The receiving unit receives a transmission signal transmitted by wireless communication from the other device via a network. A ranging device according to any one of (1) to (6). (8) The receiving unit receives a GNSS signal as the transmission signal. A ranging device according to any one of (1) to (6). (9) The device further includes a signal processing unit that calculates a distance to the other device based on a detection result of the ToF sensor. A ranging device according to any one of (1) to (8). (10) The device further includes a communication unit that transmits a distance to the other device as a first distance to the other device and receives a distance calculated by the other device as a second distance. The ranging device according to (9) above. (11) a distance calculation unit that calculates a relative distance to the other device from the first distance and the second distance. The ranging device according to (10) above. (12) The ToF sensor receives reflected light that is emitted from each of the plurality of light sources and is reflected by an object. A ranging device according to any one of (1) to (11). (13) The image sensor further includes a signal processing unit that calculates a distance to the object based on a phase difference of light irradiated from each of the plurality of light sources. The ranging device according to (12) above. (14) A communication unit that transmits a transmission signal for wireless communication; a reference clock generating unit that generates a reference clock signal that is a reference when the light source emits light, based on a synchronous clock signal that is synchronized with the transmission signal; a light source that emits irradiation light based on the reference clock signal; A ranging device comprising: (15) A communication unit that transmits and receives a transmission signal for wireless communication; a reference clock generating unit that generates a reference clock signal that is a reference when the light source emits light, based on a synchronous clock signal that is synchronized with the transmission signal; a light source that emits irradiation light based on the reference clock signal; a ToF sensor that receives light reflected from an object that is irradiated from another device based on the reference clock signal; A ranging device comprising: (16) The ranging device is A signal synchronized with a synchronous clock signal of another device is generated by performing a synchronous process of a transmission signal transmitted by wireless communication from the other device; generating a reference clock signal that is a reference when the light source emits light based on the synchronous clock signal; Based on the reference clock signal, the reflected light that is returned after being irradiated from the light source and reflected by an object is received. The light receiving method of the ranging device. [Explanation of symbols]

[0335] 1 ranging system, 2 light source, 3 ranging device, 11 ranging system, 21 (21', 21A to 21E) ranging device, 23 network, 41 clock source, 42 communication module, 44 reference clock generation unit, 45 operation mode setting unit, 47 laser emission unit, 48 light source information acquisition unit, 49 memory, 50 time synchronization unit, 51 timing control unit, 52 ToF sensor, 53 signal processing unit, 61 synchronization clock generation unit, 62 signal transmission / reception unit, 81 GNSS receiving unit, 82 time information setting unit, 101 (101A, 101B) drone, 122 position and attitude calculation unit, 123 communication unit, 124 light source distance calculation unit, 141 depth image generation unit, 142 external modulation light source detection unit

Claims

1. a receiving unit that performs synchronization processing of a transmission signal transmitted from another device via wireless communication to generate a signal synchronized with a synchronous clock signal of the other device; a reference clock generating unit that generates a reference clock signal that is a reference when the light source emits light, based on the synchronous clock signal; a ToF sensor that receives light reflected from an object that is irradiated with light from the light source based on the reference clock signal; Equipped with the receiving unit receives light source information regarding the irradiation light irradiated from the light source via the wireless communication; The light source information includes calibration data for the light source. Ranging device.

2. A receiving unit that performs synchronization processing of a transmission signal transmitted from another device by wireless communication to generate a signal synchronized with a synchronous clock signal of the other device; a reference clock generating unit that generates a reference clock signal that is a reference when the light source emits light, based on the synchronous clock signal; a ToF sensor that receives light reflected from an object that is irradiated with light from the light source based on the reference clock signal; Equipped with the receiving unit receives light source information regarding the irradiation light irradiated from the light source via the wireless communication; The light source information includes an operation mode. Ranging device.

3. The operation mode includes information indicating one of a time division mode, a modulation frequency mode, and a light emission pattern mode. A ranging device as claimed in claim 2.

Images