Ranging device and light receiving method thereof

Abstract

The present technology relates to a ranging device that achieves synchronization between a modulation frequency of a light source and a modulation frequency of a ToF sensor when the light source and the ToF sensor are configured as separate units, and a light reception method for a ranging device. The ranging device is provided with a reception unit that performs a synchronization process with respect to a transmitted signal transmitted by another device via wireless communication, thereby generating a signal synchronized with a synchronization clock signal of another device; a reference clock generation unit that generates a reference clock signal used as a reference when a light source emits light, based on the synchronization clock signal; and a ToF sensor that receives reflected light produced by irradiation from a light source, reflection by an object, based on the reference clock signal. The present technology is applicable to, for example, a ranging device used for measuring a distance to an object.

Description

TECHNICAL FIELD

[0001] The present technology relates to a distance measuring device and a light receiving method thereof, and more particularly, to a distance measuring device and a light receiving method thereof in which a modulation frequency of a light source is synchronized with a modulation frequency of a ToF sensor in a case where the light source and the ToF sensor are prepared separately. BACKGROUND

[0002] A distance measuring method called a ToF (Time of Flight) method is known as a distance measuring method that uses light to measure a distance to a target object. In the ToF method, a light source is used to perform light irradiation on an object, a ToF sensor receives reflected light from the object, and a distance to the object is measured by analyzing a light reception result.

[0003] Furthermore, a technique of measuring a distance to an object by irradiating the object with light emitted from a light source prepared separately from a ToF sensor and receiving the resulting reflected light using the ToF sensor and the like has also been proposed (see, for example, PTL 1).

[0004] [LIST OF CITATIONS]

[0005] [PATENT LITERATURE]

[0006] [PTL 1]

[0007] Japanese Patent Publication No. 2018-31607 SUMMARY

[0008] [TECHNICAL PROBLEM]

[0009] However, in a case where a light source and a ToF sensor are prepared separately from each other, a light source clock of the light source and a sensor clock of the ToF sensor are different. Therefore, a modulation frequency of the light source is not necessarily synchronized with a modulation frequency of the ToF sensor. In some cases, this causes a frequency deviation. Furthermore, in a case where the modulation frequency of the ToF sensor deviates from the modulation frequency of the light source, an error can occur in a result of distance measurement.

[0010] The present technology is proposed in view of the above-described circumstances, for achieving synchronization between a modulation frequency of a light source and a modulation frequency of a ToF sensor in a case where the light source and the ToF sensor are prepared separately.

[0011] [PROBLEM SOLUTION]

[0012] The ranging device according to the first aspect of the present technology includes a reception section that generates a signal synchronized with a synchronization clock signal of a separate device by performing synchronization processing on a transmission signal transmitted from the separate device through wireless communication, a reference clock generation section that generates a reference clock signal to be used as a reference for light emission of a light source based on the synchronization clock signal, and a ToF sensor that receives reflected light generated due to irradiation light applied from the light source and reflected by an object based on the reference clock signal.

[0013] The ranging device according to the second aspect of the present technology includes a communication section that transmits a transmission signal through wireless communication, a reference clock generation section that generates a reference clock signal to be used as a reference for light emission of a light source based on a synchronization clock signal synchronized with the transmission signal, and a light source that applies irradiation light based on the reference clock signal.

[0014] The ranging device according to the third aspect of the present technology includes a communication section that transmits and receives a transmission signal through wireless communication, a reference clock generation section that generates a reference clock signal to be used as a reference for light emission of a light source based on a synchronization clock signal synchronized with the transmission signal, a light source that applies irradiation light based on the reference clock signal, and a ToF sensor that receives reflected light generated due to irradiation light applied from a separate device and reflected by an object based on the reference clock signal.

[0015] The light reception method for a ranging device according to the fourth aspect of the present technology includes generating, by the ranging device, a signal synchronized with a synchronization clock signal of a separate device by performing synchronization processing on a transmission signal transmitted from the separate device through wireless communication, generating a reference clock signal to be used as a reference for light emission of a light source based on the synchronization clock signal, and receiving reflected light generated due to irradiation light applied from the light source and reflected by an object based on the reference clock signal.

[0016] According to the first, third, and fourth aspects of the present technology, a reference clock signal to be used as a reference for light emission of a light source is generated based on a synchronization clock signal synchronized with a transmission signal of wireless communication, and reflected light generated due to irradiation light applied from the light source and reflected by an object is received based on the reference clock signal.

[0017] According to the second aspect of the present technology, a reference clock signal to be used as a reference for light emission of a light source is generated based on a synchronization clock signal synchronized with a transmission signal of wireless communication, and irradiation light is applied from the light source based on the reference clock signal.

[0018] The ranging device can be a separate device or a module included in another device. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a diagram for explaining the basic principle of the distance measuring process using the direct ToF method.

[0020] Figure 2 is a diagram for explaining the basic principle of the distance measuring process using the direct ToF method.

[0021] Figure 3 is a diagram for explaining the calculation method of the phase difference .

[0022] Figure 4 is a diagram for explaining components and frames according to the present disclosure.

[0023] Figure 5 is a diagram for explaining the concept of the process of correcting the cycle error.

[0024] Figure 6 is a diagram for explaining the method of preventing interference between a plurality of light sources.

[0025] Figure 7 is a diagram for explaining the method of preventing interference between a plurality of light sources.

[0026] Figure 8 is a diagram for explaining the direct ToF method.

[0027] Figure 9 is a diagram showing a configuration example of a distance measuring system according to the first embodiment of the present disclosure.

[0028] Figure 10 is a block diagram showing a functional configuration example of a distance measuring apparatus including a light source and a distance measuring unit.

[0029] Figure 11 is a diagram showing an operation example in which the operation mode is set to the time division mode.

[0030] Figure 12 is a diagram showing an operation example in which the operation mode is set to the modulation frequency mode.

[0031] Figure 13 is a diagram showing an operation example in which the operation mode is set to the light emission pattern mode.

[0032] Figure 14 is a functional block diagram of a distance measuring apparatus having only a light emission function.

[0033] Figure 15 is a functional block diagram of a distance measuring apparatus having only a light reception function.

[0034] Figure 16 is a flowchart showing a light emission control process in a light emission side distance measuring apparatus.

[0035] Figure 17 is a flowchart showing light emission control processing in a light-receiving side distance measuring apparatus.

[0036] Figure 18 is a diagram for explaining processing for realizing phase synchronization.

[0037] Figure 19 is a block diagram of a distance measuring apparatus for realizing time synchronization based on GNSS signals.

[0038] Figure 20 is a diagram showing a configuration example of a distance measuring system according to a second embodiment of the present disclosure.

[0039] Figure 21 is a diagram for explaining a principle of calculating a relative distance.

[0040] Figure 22 is a diagram for explaining a principle of calculating a relative distance.

[0041] Figure 23 is a diagram for explaining a principle of calculating a relative distance.

[0042] Figure 24 is a block diagram of a distance measuring apparatus integrated in a drone.

[0043] Figure 25 is a diagram showing an example of a confidence image observed by a drone.

[0044] Figure 26 is a diagram for explaining processing for improving the accuracy of position and attitude using an accurate relative distance.

[0045] Figure 27 is a diagram for explaining a relationship between a modulation frequency and a measured distance.

[0046] Figure 28 is a flowchart depicting processing for measuring a relative distance of a drone.

[0047] Figure 29 is a diagram showing a configuration example of a distance measuring system according to a third embodiment of the present disclosure.

[0048] Figure 30 is a flowchart depicting distance calculation processing in a distance measuring system according to the third embodiment.

[0049] Figure 31 is a diagram for explaining phase difference detection in step S66 in Figure 30

[0050] Figure 32 ​is a diagram for explaining a distance calculation method involving receiving reflected light from multiple light sources.

[0051] Figure 33 is a diagram for explaining a distance calculation method involving receiving reflected light from multiple light sources.

[0052] Figure 34 is a diagram showing a configuration example of a light receiving section of a ToF sensor.

[0053] Figure 35 is a block diagram depicting an example of a schematic configuration of a vehicle control system.

[0054] Figure 36 is a diagram for assisting explanation of an example of installation positions of an outside-vehicle information detecting section and an imaging section.

[0055] Explanation of Embodiments

[0056] Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be explained with reference to the drawings. Note that in the present specification and the drawings, constituent elements having substantially the same function configuration are denoted by the same reference numerals. Therefore, repeated explanation thereof will be omitted. The explanation will be made in the following order.

[0057] 1. Overview of Ranging Processing Using ToF Method

[0058] 2. First Embodiment of Ranging System

[0059] 3. Block Diagram of Ranging Device

[0060] 4. Processing Flow of Ranging Device

[0061] 5. Modification of First Embodiment

[0062] 6. Second Embodiment of Ranging System

[0063] 7. Third Embodiment of Ranging System

[0064] 8. Distance Calculation Method Involving Receiving Reflected Light from Multiple Light Sources

[0065] 9. Configuration of ToF Sensor

[0066] 10. Example Applied to Mobile Body

[0067] <1. Overview of Ranging Processing Using ToF Method>

[0068] A distance measuring method called a ToF (Time of Flight) method is known as a distance measuring method for measuring the distance of a target object using light. In the ToF method, a light source is used to irradiate an object with light, the resulting reflected light is received by a ToF sensor, and the light reception result is analyzed, thereby measuring the distance of a distance object or the like.

[0069] The present disclosure relates to a distance measuring technology using the ToF method. For the convenience of understanding embodiments according to the present disclosure, the basic principle of a distance measuring process using the ToF method will be explained with reference to Figures 1 to 7

[0070] In the ToF method, light is applied to an object, and the resulting reflected light is analyzed, thereby measuring the distance (depth) of a distance object and the shape of this object. Note that the following explanation does not include any particular mention of measurement of a three-dimensional shape. However, the measurement of the distance of a distance object surface is performed for all surfaces of this object, thereby making it possible to measure the three-dimensional shape of this object.

[0071] (Configuration of distance measuring system)

[0072] Figure 1 The configuration of a distance measuring system is shown.

[0073] Figure 1 The distance measuring system 1 in FIG. 1 includes a light source 2 and a distance measuring unit 3. The light source 2 applies light to an object 4 that is a target object of measurement. The irradiation light L1 applied from the light source 2 is reflected by the object 4, and then becomes reflected light L2 and enters the distance measuring unit 3.

[0074] The arrangement positions of the light source 2 and the distance measuring unit 3 are substantially the same. In this case, the distance (depth) d from the distance measuring unit 3 to the object 4 can be calculated by the following expression (1).

[0075] [Mathematical calculation 1]

[0076]

[0077] In expression (1), Δt represents the time required for the irradiation light L1 emitted from the light source 2 to enter the distance measuring unit 3 after being reflected by the object 4, and c represents the speed of light (2.9979 x 10 8 [m / sec]).

[0078] As the irradiation light L1 applied from the light source 2, as Figure 2 ​The pulse light is shown to adopt a light emission pattern in which the ON state and the OFF state are repeated at a high speed with a predetermined frequency f (modulation frequency). One cycle of this light emission pattern is 1 / f. The ranging unit 3 detects the phase-shifted reflected light L2 based on the time Δt taken for the light to travel from the light source 2 to the ranging unit 3. When the amount of this phase shift (phase difference) is defined as The time Δt can be calculated by the following expression (2).

[0079] [Mathematical Calculation 2]

[0080]

[0081] Therefore, based on expression (1) and expression (2), the distance d from the ranging unit 3 to the object 4 can be calculated by the following expression (3).

[0082] [Mathematical Calculation 3]

[0083]

[0084] Next, the calculation method of the phase difference Figure 3 will be described with reference to .

[0085] The ranging unit 3 includes a ToF sensor in which pixels that perform photoelectric conversion on incident light are arranged two-dimensionally. Each pixel in the ToF sensor is repeatedly switched at a high speed between the ON state and the OFF state, and accumulates a charge only during the ON period.

[0086] In a case where the ToF sensor is used to measure the distance to an object, the ranging unit 3 sequentially switches between the ON execution timing and the OFF execution timing of the ToF sensor, accumulates the charge at the ON execution timing and the charge at the OFF execution timing, and outputs a detection signal in accordance with the accumulated charges.

[0087] There are four types of ON / OFF execution timing, such as a 0-degree phase, a 90-degree phase, a 180-degree phase, and a 270-degree phase.

[0088] At the execution timing of the 0-degree phase, the ON timing (light reception timing) of the ToF sensor matches the phase of the pulse light emitted by the light source 2 (i.e., the phase of the light emission pattern).

[0089] At the execution timing of the 90-degree phase, the ON timing (light reception timing) of the ToF sensor is delayed by 90 degrees from the pulse light (light emission pattern) emitted by the light source 2.

[0090] Under the 180-degree phase execution timing, the phase of the ToF sensor's ON timing (light reception timing) is delayed by 180 degrees from the pulse light (emission mode) emitted by the light source 2.

[0091] With a 270-degree phase execution timing, the phase of the ToF sensor's ON timing (light reception timing) is delayed by 270 degrees from the pulse light (emission mode) emitted by the light source 2.

[0092] The ToF sensor sequentially switches between these four types to execute timing, and acquires the amount of reflected light L2 received (accumulated charge) at each light reception timing. Figure 3 In the ON timing (optical reception timing) of each phase, the incident timing of the reflected light L2 is shaded.

[0093] exist Figure 3 In this context, the accumulated charge when the optical receiving timing is set to 0 degrees, 90 degrees, 180 degrees, and 270 degrees phase is defined as Q0, Q1, Q2, Q3, Q4, Q5, Q6, Q7, Q8, Q9, Q1 ...1, Q8, Q9, Q1, Q1, Q2, Q1, Q3, Q1, Q4, Q5, Q1, Q1, Q2, Q1, Q3, Q1, Q4 90 Q 180 and Q 270 You can use Q0, Q... 90 Q 180 and Q 270 The phase difference is calculated using the following expression (4).

[0094] [Mathematical Calculation 4]

[0095]

[0096] The phase difference calculated according to expression (4) Substituting into expression (3), the distance d from ranging unit 3 to object 4 can be calculated.

[0097] Furthermore, the value representing the light intensity received at each pixel in the ToF sensor is called the confidence value. The confidence value is calculated using either expression (5) or expression (6).

[0098] [Mathematical Calculation 5]

[0099]

[0100] Confidence value = |Q 180 -Q0|+|Q 90 -Q 270 |...(6)

[0101] like Figure 4As shown, the light reception timing of the ToF sensor is switched between 0-degree phase, 90-degree phase, 180-degree phase, and 270-degree phase in order, and detection signals according to the accumulated charges (charge Q0, charge Q 90 , charge Q 180 , and charge Q 270 ) in the respective phases are output in order.

[0102] In the present disclosure, an image frame of any one of the 0-degree phase, 90-degree phase, 180-degree phase, and 270-degree phase output from the ToF sensor is referred to as a "component", and a set including the four components (image frames of the four phases) of the 0-degree phase, 90-degree phase, 180-degree phase, and 270-degree phase is referred to as a "frame".

[0103] (Correction of cyclic error)

[0104] Assuming that the intensity variation of the irradiation light L1 emitted from the light source 2 is a sine wave, the calculation of the above expression (4) is performed. However, the light emitted from the light source 2 is actually a rectangular wave, as shown in Figure 2 . Therefore, because the rectangular wave is treated as a sine wave, an error (hereinafter referred to as a cyclic error) periodically occurs in the distance d. Therefore, the detection signals of the phases output from the ToF sensor are generally subjected to a correction process of the correction of the cyclic error.

[0105] Figure 5 is a diagram that illustrates the concept of the process of the correction of the cyclic error.

[0106] Figure 5 The diagram on the left side depicts the relationship between the phase difference and the distance d output from the ToF sensor. The linear relationship represented by the broken line is ideal for the relationship between the phase difference and the distance d. However, the relationship actually has a nonlinear shape, including the cyclic error represented by the solid line.

[0107] Therefore, the signal processing section of the rear stage that processes the detection signals output from the ToF sensor performs the correction process of the correction of the cyclic error. As a result, the corrected relationship between the phase difference and the distance d has a linear shape, as shown on the right side in Figure 5 .

[0108] Specifically, in the correction process, the ToF sensor measures an object of a known distance d, and then calculates a correction function f that converts the actually measured phase difference to a phase difference that is the true value corresponding to the distance d of the distance object, on the basis of the relationship between the phase difference . The correction function f is calculated on the basis of the relationship between the phase difference and the distance d. correction function is stored in advance in a memory of the signal processing section. Upon receiving a phase difference As a source of a measured value from the ToF sensor, the signal processing section calculates a phase difference The correction processing is executed.

[0109] Note that, in addition to the method of storing a correction function in the memory and performing a calculation using the correction function , examples of the correction processing also include a method of storing in pairs, each pair including a phase difference as a measured value and a phase difference as a true value in a lookup table or the like, and reading out and outputting from the lookup table a phase difference

[0110] Alternatively, it is also possible to approximate the actually measured correction function f(x) to another parameter function, and store only several correction coefficients in the memory. For example, as shown in expression (7), the correction function f(x) can be expanded into a Fourier series, and thus it is possible to store in the memory correction coefficients (a k , b k ) representing 0th to Nth order terms.

[0111] [mathematical calculation 6]

[0112]

[0113] (method of preventing interference of multiple light sources)

[0114] In the distance measuring system 1, in the case where there are multiple sets of light sources 2 and distance measuring units 3 within the measurement range of a certain distance measuring unit 3, accurate measurement of a phase difference may fail due to irradiation light emitted from individual light sources 2.

[0115] For example, assume that a set 5A including a light source 2A and a distance measuring unit 3A and a set 5B including a light source 2B and a distance measuring unit 3B are arranged within a range (measurement range) in which the light sources 2 are able to receive light from each other, as shown in Figure 6

[0116] The distance measuring unit 3A receives reflected light generated by irradiation light applied from the light source 2A and reflected by the object 4, and calculates a phase difference The distance measuring unit 3B receives reflected light generated by irradiation light applied from the light source 2B and reflected by the object 4, and calculates a phase difference ​In this case, there is a possibility that the irradiation light applied from the light source 2A affects the light reception at the distance measuring unit 3B and the irradiation light applied from the light source 2B affects the light reception at the distance measuring unit 3A.

[0117] As a countermeasure against the above-described interference with the irradiation light applied from the individual light sources 2, there are three methods, i.e., a method (1) of causing the light sources 2 to emit pulsed light in a time-division manner, a method (2) of causing the modulation frequencies of the pulsed light from the light sources 2 to be different from each other, and a method (3) of causing the light emission patterns of the pulsed light from the light sources 2 to be different from each other.

[0118] In the method (1) of causing the light sources 2 to emit pulsed light in a time-division manner, since one type of pulsed light is emitted from the light sources 2 at any point in time, there is no influence from the irradiation light applied from the individual light sources 2.

[0119] Figure 7 FIG. 2A of the drawings shows an example of the method (2) of causing the modulation frequencies of the pulsed light from the light sources 2 to be different from each other.

[0120] In the method of causing the modulation frequencies of the pulsed light from the light sources 2 to be different from each other, the light sources 2A of Group A apply pulsed light with a modulation frequency f of 100 MHz, and the light sources 2B of Group B apply pulsed light with a modulation frequency f of, for example, 71 MHz.

[0121] The distance measuring unit 3A of Group A drives the ToF sensor with a drive frequency equal to the modulation frequency f = 100 MHz. The distance measuring unit 3B of Group B drives the ToF sensor with a drive frequency equal to the modulation frequency f = 71 MHz. The drive frequency of the ToF sensor corresponds to Figure 3 the frequency of the ON period in which the charge is accumulated in FIG. 2B.

[0122] Since the distance measuring unit 3A is driven with a drive frequency equal to the modulation frequency f of the pulsed light emitted from the light source 2A, the light emitted from the light source 2A can be received with high correlation. That is, a signal with a high confidence value can be detected.

[0123] On the other hand, in the case where the distance measuring unit 3B receives the pulsed light emitted from the light source 2A, since there is a difference in the drive frequency, it is not possible to calculate the phase difference That is, in the distance measuring unit 3B, the values of the charges Q0, Q 90 , Q 180 , and Q 270 corresponding to the reflected light from the light source 2A are approximately equal. Accordingly, the light source 2A is not observed as a modulated light source, and thus there is no influence on the calculation of the phase difference Therefore, even when the multiple light sources 2A and 2B emit irradiation light at the same time, the distance measuring units 3A and 3B can measure the respective accurate distances d independently of each other.

[0124] Figure 7 B in FIG. 6 depicts an example of the method (3) of making the light emission patterns of the pulsed light from the light sources 2 different from each other.

[0125] The light sources 2A of the group A apply pulsed light with a predetermined light emission pattern A, while the light sources 2B of the group B apply pulsed light with a predetermined light emission pattern B. The ranging units 3A of the group A drive the ToF sensors with the light emission pattern A, while the ranging units 3B of the group B drive the ToF sensors with the light emission pattern B. The light emission pattern A and the light emission pattern B are different from each other.

[0126] Between the ranging units 3A and 3B, there is a high correlation for the reflected light of the same light emission pattern, and a high confidence value is obtained. On the other hand, the correlation of the reflected light of different light emission patterns is low, and therefore does not have an impact on the calculation of the phase difference . Therefore, even when the multiple light sources 2A and 2B emit irradiation light at the same time, the ranging units 3A and 3B can measure the respective accurate distances d independently of each other.

[0127] The method (1) of identifying the pulsed light from the light sources 2 by time division, the method (2) of identifying the pulsed light from the light sources 2 by the modulation frequency, or the method (3) of identifying the pulsed light from the light sources 2 by the light emission pattern thereof are selected as a measure to prevent interference of the irradiation light, which can be set in a manner depending on the operation mode. The method (1) of identifying the pulsed light from the light sources 2 by time division, the method (2) of identifying the pulsed light from the light sources 2 by the modulation frequency, and the method (3) of identifying the pulsed light from the light sources 2 by the light emission pattern thereof are respectively referred to as (1) a time division mode, (2) a modulation frequency mode, and (3) a light emission pattern mode, and the like.

[0128] (Direct ToF method)

[0129] As with the above-described manner, according to the charges Q0, Q 90 , Q 180 , and Q 270 detected at different phases at the irradiation timing of the irradiation light L1 and the distance d of the distance object 4 calculated therefrom, the phase difference of the ToF method is detected.

[0130] On the other hand, there is another method called a direct ToF method, in which the time Δt taken for the light emitted from the light sources 2 to reach the ranging unit 3 is directly counted.

[0131] Referring to Figure 8 , the direct ToF method will be briefly explained.

[0132] In a ToF sensor using a direct ToF method, pixels two-dimensionally arranged in a matrix form each include a SPAD (Single Photon Avalanche Diode) or an APD (Avalanche Photodiode) as a light-receiving element. The SPAD or the APD is a light-receiving element that amplifies electrons generated by incident light incident on the pixel in an avalanche manner and then outputs a signal. A TDC (Time-to-Digital Converter) converts a time of flight of light from a time when light is applied from a light source 2 to a time when reflected light is received by a ranging unit 3 into a digital count value (hereinafter, referred to as a TDC code). The application of light and the reception of light are performed a plurality of times to eliminate the influence of interference light and multipath.

[0133] Subsequently, a histogram of the plurality of TDC codes is generated as shown in Figure 8 The TDC code having the highest frequency value is determined as a final time of flight At, and then the distance d can be calculated according to the above equation (1).

[0134] Therefore, also in the ToF sensor using the direct ToF method, because the time is counted based on the time when the light is applied from the light source 2, it is necessary to notify the ToF sensor side of the light emission timing.

[0135] So far, the outline of the ranging processing using the ToF method has been described.

[0136] In the ToF method, as described above, the light source 2 and the ranging unit 3 are generally integrated in one module or the like in such a manner that they are located at substantially the same position.

[0137] However, the light source 2 can be separated from the ranging unit 3, the object 4 can be placed to be irradiated with light applied from the light source 2 at a separate position, the ranging unit 3 can receive the reflected light generated thereby, and the distance to the object 4 can be measured.

[0138] In the case where the light source 2 and the ranging unit 3 are placed separately at different positions, the light source 2 can be placed near the object 4. Therefore, compared with the case where the light source 2 and the ranging unit 3 are integrally formed, it is possible to increase the amount of light reception at the ToF sensor. As a result, it is possible to improve the ranging accuracy of the ranging unit 3.

[0139] However, when the distance d is measured by the ToF method, the ranging unit 3 needs to achieve high-precision synchronization with the light emission timing of the light source 2, as described above. Therefore, in the case where the light source 2 and the ranging unit 3 are placed separately, whether or not the ToF sensor of the ranging unit 3 can achieve high-precision frequency synchronization and phase synchronization with the light emission timing of the light source 2 is a key issue.

[0140] Therefore, the following will give an explanation of a ranging system that achieves synchronization between a separately placed light source and a ranging unit with high precision.

[0141] <2. First Embodiment of Ranging System>

[0142] Figure 9 is a diagram showing a configuration example of a distance measuring system according to a first embodiment of the present disclosure.

[0143] According to Figure 9 The distance measuring system 11 according to the first embodiment in Figure 9 A configuration of the distance measuring system 11 including five distance measuring devices 21 as distance measuring devices 21A to 21E is shown. However, it is not essential that the number of distance measuring devices 21 be set to five. Any number of distance measuring devices 21 can be prepared.

[0144] Some distance measuring devices 21 have a configuration including both the light source 2 and the distance measuring unit 3. Other distance measuring devices 21 have a configuration including only the light source 2 or a configuration including only the distance measuring unit 3. Specifically, the distance measuring device 21A includes only the light source 2, and the distance measuring device 21B includes the light source 2 and the distance measuring unit 3. The distance measuring device 21C includes only the distance measuring unit 3, and the distance measuring device 21D includes the light source 2 and the distance measuring unit 3. The distance measuring device 21E includes only the distance measuring unit 3.

[0145] The distance measuring devices 21 are identified by respective unique IDs given to the distance measuring devices 21, respectively. In the present embodiment, the unique ID of the distance measuring device 21A is "0" (ID0). The unique ID of the distance measuring device 21B is "1" (ID1). The unique ID of the distance measuring device 21C is "2" (ID2). The unique ID of the distance measuring device 21D is "3" (ID3). The unique ID of the distance measuring device 21E is "4" (ID4).

[0146] The distance measuring unit 3 includes a ToF sensor 52 Figure 10 ) that measures the distance d of a distance object using an indirect ToF method or a direct ToF method. Note that the following explanation will be given assuming that the ToF sensor 52 of each distance measuring unit 3 is a ToF sensor using the indirect ToF method. If necessary, the ToF sensor 52 using the direct ToF method will be additionally explained.

[0147] For example, the distance measuring devices 21A to 21D are fixedly installed on a ceiling. The distance measuring device 21E is installed at a predetermined position on a mobile carrier vehicle 22. The traveling direction of the mobile carrier vehicle 22 can be changed depending on a distance measurement result (e.g., the distance of the distance object 24 obtained by the distance measuring device 21E).

[0148] The five distance measuring devices 21A to 21E can communicate (wireless communication) using a predetermined wireless signal through the network 23. Alternatively, each of the five distance measuring devices 21A to 21E can directly wirelessly communicate with the other distance measuring device 21 without the network 23.

[0149] For example, the network 23 includes a WiFi (registered trademark), Bluetooth (registered trademark), or a wide area communication network such as a 4G or 5G line commonly called a wireless mobile body.

[0150] When the individual distance measuring device 21 including the distance measuring unit 3 (for example, the distance measuring device 21C) performs distance measurement, the distance measuring device 21A including only the light source 2 emits irradiation light.

[0151] The distance measuring devices 21C and 21E each including only the distance measuring unit 3 receive reflected light generated by the irradiation light applied from the light source 2 of the individual distance measuring device 21, and then measure the distance d of the distance object.

[0152] The distance measuring devices 21B and 21D each including the light source 2 and the distance measuring unit 3 can receive reflected light of the irradiation light applied from the light source 2 of the distance measuring device 21B and 21D, respectively, to measure the distance d of the distance object, or can receive reflected light of the irradiation light applied from the light source 2 of the individual distance measuring device 21 (for example, the distance measuring device 21C) to measure the distance d of the distance object.

[0153] For example, the distance measuring device 21E including only the distance measuring unit 3 measures the distance d from the distance measuring device 21E to the object 24 by implementing synchronization of the light emission timing with the irradiation light applied from the light source 2 of the individual distance measuring device 21D.

[0154] More specifically, by implementing synchronization with the clock of the wireless signal output from the light source 2 of the individual distance measuring device 21D, the distance measuring device 21E implements synchronization between the light emission timing of the light source 2 of the distance measuring device 21E and the light reception timing of the distance measuring unit 3. For example, in a case where the distance measuring device 21E and the distance measuring device 21D perform wireless communication with each other by WiFi (registered trademark) or Bluetooth (registered trademark), the distance measuring device 21E implements synchronization between the light emission timing and the light reception timing of the distance measuring unit 3 of the distance measuring device 21E according to, for example, a 2.4-GHz clock signal. Therefore, even in a case where the light source 2 and the distance measuring unit 3 are disposed separately, high-precision distance measurement can be performed.

[0155] <3. Block diagram of distance measuring device>

[0156] Figure 10 is a block diagram illustrating an example of a functional configuration of the distance measuring device 21 including the light source 2 and the distance measuring unit 3.

[0157] The distance measuring device 21 includes a clock source 41, a communication module 42, an antenna 43, a reference clock generation section 44, an operation mode setting section 45, a laser driver 46, and a laser emission section 47.

[0158] Further, the distance measuring device 21 includes a light source information acquisition section 48, a memory (storage section) 49, a time synchronization section 50, a timing control section 51, a ToF sensor 52, and a signal processing section 53.

[0159] For example, the clock source 41 includes a crystal oscillator. The clock source 41 generates a master clock serving as a reference for both wireless communication and light emission timing, and supplies the master clock to the communication module 42.

[0160] The communication module 42 controls wireless communication performed by the distance measuring device 21, and processes data based on wireless signals transmitted and received via the antenna 43.

[0161] For example, during data transmission, the communication module 42 performs encoding and modulation and the like using a predetermined encoding method and modulation method in accordance with the data to be transmitted, and causes the resultant transmission signal to be transmitted from the antenna 43. The data to be transmitted is supplied from the light source information acquisition section 48, the time synchronization section 50, the timing control section 51, and the like.

[0162] Further, during data reception, the communication module 42 performs processing (decoding and demodulation) opposite to the processing performed during data transmission in accordance with data received via the antenna 43, and supplies the resultant data to any one of the blocks in the subsequent stage. Specifically, the data is supplied to the operation mode setting section 45, the light source information acquisition section 48, the time synchronization section 50, the timing control section 51, and the like.

[0163] The communication module 42 includes a synchronization clock generation section 61 and a signal transmission / reception section 62.

[0164] During data transmission, the synchronization clock generation section 61 generates a transmission signal in which a carrier frequency is modulated to a predetermined modulation frequency, and transmits this transmission signal via the antenna 43. Further, the synchronization clock generation section 61 generates a synchronization clock signal corresponding to the carrier frequency or the modulation frequency, and supplies this synchronization clock signal to the reference clock generation section 44.

[0165] During data reception, the synchronization clock generation section 61 generates a signal (synchronization clock signal) corresponding to the synchronization clock signal at the time of data transmission by performing synchronization processing on a signal received via the antenna 43, and supplies the generated signal to the reference clock generation section 44.

[0166] During data transmission, the signal transmission / reception section 62 encodes data to be transmitted by a predetermined encoding method, and supplies the encoded data to the synchronous clock generation section 61. Examples of the data to be transmitted include a unique ID for identifying the distance measuring device 21, a unique ID supplied from the light source information acquisition section 48, light emission start time information on irradiation light to be emitted by the laser emission section 47, light emission time of the irradiation light, and light source information including an operation mode, and the like.

[0167] During data reception, the signal transmission / reception section 62 demodulates a signal supplied from the synchronous clock generation section 61, acquires transmitted data, and supplies the demodulated data to any one of the operation mode setting section 45, the light source information acquisition section 48, the time synchronization section 50, or the timing control section 51. For example, the signal transmission / reception section 62 supplies an operation mode as a part of light source information acquired from a received signal to the operation mode setting section 45, and supplies the entire light source information to the light source information acquisition section 48.

[0168] During data transmission, the antenna 43 amplifies a transmission signal supplied from the communication module 42, and transmits the amplified signal as an electromagnetic wave. Further, during data reception, the antenna 43 receives a transmission signal transmitted from a separate device, and supplies the transmitted signal as a reception signal to the communication module 42.

[0169] The reference clock generation section 44 includes a PLL (Phase Locked Loop) circuit or the like. The reference clock generation section 44 generates a reference clock signal to be used as a light emission reference from the laser emission section 47, based on a synchronous clock signal supplied from the synchronous clock generation section 61, and supplies the reference clock signal to the operation mode setting section 45.

[0170] Using the reference clock signal supplied from the reference clock generation section 44, the operation mode setting section 45 generates a light emission pulse signal corresponding to the operation mode supplied from the signal transmission / reception section 62, and supplies the light emission pulse signal to the laser driver 46 and the ToF sensor 52.

[0171] For example, in a case where the time division mode is set as the operation mode, the operation mode setting section 45 supplies the reference clock signal supplied from the reference clock generation section 44 as a light emission pulse signal to the laser driver 46 and the ToF sensor 52 during a period in which light emission (or light reception) is performed by the corresponding distance measuring device 21 itself.

[0172] Alternatively, in a case where the modulation frequency pattern is set as the operation mode, for example, the operation mode setting section 45 adjusts the frequency of the reference clock signal provided from the reference clock generation section 44 to the modulation frequency of the light emitted or received by the corresponding distance measuring device 21 itself, and provides the reference clock signal whose frequency has been adjusted as the light emission pulse signal to the laser driver 46 and the ToF sensor 52.

[0173] Alternatively, in a case where the light emission pattern mode is set as the operation mode, for example, the operation mode setting section 45 generates a light emission pattern of the light emitted or received by the corresponding distance measuring device 21 itself based on the reference clock signal provided from the reference clock generation section 44, and provides this light emission pattern as the light emission pulse signal to the laser driver 46 and the ToF sensor 52.

[0174] The laser driver 46 generates a drive signal for driving a VCSEL (Vertical Cavity Surface Emitting Laser) that is a light source of the laser emission section 47 based on the light emission pulse signal provided from the operation mode setting section 45, and provides the drive signal to the laser emission section 47.

[0175] Start timing control information that defines the timing of starting light emission (or light reception) is provided from the timing control section 51 to the laser driver 46. The start timing control information includes light emission start time information indicating the clock time of starting light emission or light reception, the light emission time of the VCSEL (exposure time of the pixel), the period of the component and the frame, and the like. At a predetermined timing based on the start timing control information, the laser driver 46 provides the drive signal to the laser emission section 47.

[0176] The laser emission section 47 includes, for example, a VCSEL array (light source array) in which a plurality of VCSELs that serve as light sources are arranged on a plane. The light emission from the laser emission section 47 is repeated on / off at a predetermined period according to the drive signal provided from the laser driver 46.

[0177] During light emission, the light source information acquisition section 48 acquires light source information from the memory 49, and provides the light source information to the signal transmission / reception section 62. Further, during light reception, the light source information acquisition section 48 acquires received light source information from the signal transmission / reception section 62, and causes the memory 49 to store the acquired light source information. The light source information is also provided to the timing control section 51, the ToF sensor 52, and the signal processing section 53 as necessary. For example, light source calibration data is provided to the ToF sensor 52.

[0178] The memory 49 stores the light source information, and provides the light source information to the light source information acquisition section 48 as necessary.

[0179] Here, examples of information stored in the memory 49 as the light source information are as follows.

[0180] - unique ID

[0181] - light emission time / exposure time

[0182] - light emission start time information / exposure start time information

[0183] - repetition frequency (direct ToF method)

[0184] - modulation frequency (indirect ToF method)

[0185] - component length (indirect ToF method)

[0186] - position and attitude of the ranging device

[0187] - frame length

[0188] - wavelength of the light source

[0189] - light emission pattern

[0190] - light source calibration data (indirect ToF method)

[0191] - operation mode

[0192] The unique ID is information for identifying the ranging device 21. The light emission time indicates the time length of one light emission of the laser emission section 47 (light source 2) (the above ON period). The light emission time on the light receiving side corresponds to the exposure time of the ToF sensor 52. The light emission start time information indicates the clock time for starting the emission of the irradiation light from the laser emission section 47. The light emission start time information on the light receiving side corresponds to the clock time at which the ToF sensor 52 starts exposure.

[0193] The repetition frequency indicates the irradiation time interval in the direct ToF method in which the light irradiation and reception are repeated multiple times, or the time interval from the last irradiation start time to the next irradiation start time.

[0194] The modulation frequency indicates the modulation frequency in the indirect ToF method. The component length indicates the time length of one component in the indirect ToF method.

[0195] In a case where the ranging device 21 is equipped with an IMU (inertial measurement unit), a magnetic sensor, a GNSS (global navigation satellite system) receiver, or the like, the position and attitude of the ranging device indicates the position and / or attitude of the ranging device 21. This information can indicate one of the position or the attitude, or can indicate both the position and the attitude.

[0196] The frame length indicates the duration of a frame in the indirect ToF method. The wavelength of the light source indicates the wavelength of the illumination light emitted by the laser emitting unit 47. For example, when infrared light is used as the illumination light, its wavelength range is approximately 850 nm to 940 nm.

[0197] The light emission pattern indicator is used to allow the ranging device 21 to identify information about the light emission pattern of the pulsed light emitted from the light source 2.

[0198] Light source calibration data is provided to correct for cyclic errors in indirect ToF methods. For example, the calibration data includes data for... Figure 5 The nonlinear relationship shown on the left is corrected to Figure 5 The right side depicts the correction function, correction coefficients, etc., for the linear relationship.

[0199] The operating mode indicates information for taking measures to prevent interference of the illumination light in any of the following: (1) time-division mode, (2) modulation frequency mode, and (3) light emission pattern mode, and indicates the required operating mode detail settings for each of the operating modes. For example, the operating mode detail settings for (1) time-division mode include information indicating the time frame used by the corresponding ranging device 21 in the divided time frame, and the operating mode detail settings for (3) light emission pattern mode include information indicating the light emission pattern used by the corresponding ranging device 21 in a plurality of light emission patterns.

[0200] Figure 11 An operational example is shown of multiple optical emitting side ranging devices 21 and optical receiving side ranging devices 21 operating in a time-division mode to prevent interference of the illumination light.

[0201] For example, ranging devices 21A and 21B, which are light-emitting side ranging devices 21, emit light in a time-division multiplexing manner. The light-receiving side ranging device 21E performs light-receiving operations synchronously with the light emission timing of ranging device 21B, thereby preventing interference of the illuminating light. Therefore, ranging device 21E can receive the reflected light generated by the illuminating light emitted by ranging device 21B. The light emission timing of ranging device 21B to receive target light can be confirmed from the light emission start time information, which indicates the start time of light emission and is part of the light source information.

[0202] Figure 12 An operational example is shown of multiple optical transmitting-side ranging devices 21 and optical receiving-side ranging devices 21 operating in an operating mode set to a modulation frequency mode to prevent interference of the illumination light.

[0203] like Figure 11 As shown, ranging devices 21A and 21B are light-emitting side ranging devices 21, and ranging device 21E is a light-receiving side ranging device 21.

[0204] The distance measuring device 21A emits irradiation light with a modulation frequency of 71 MHz. The distance measuring device 21B emits irradiation light with a modulation frequency of 100 MHz. The light-receiving-side distance measuring device 21E performs a light-receiving operation in synchronization with the modulation frequency of the distance measuring device 21B, thereby preventing interference of the irradiation light. Thus, the distance measuring device 21E can receive reflected light generated by the irradiation light emitted by the distance measuring device 21B. The modulation frequency of the reception target light emitted by the distance measuring device 21B can be confirmed from modulation frequency information indicating the modulation frequency and being part of the light source information.

[0205] Figure 13 An operation example is shown in which the plurality of light-emitting-side distance measuring devices 21 and the light-receiving-side distance measuring device 21 operate in an operation mode in which the light-emitting pattern mode is set to prevent interference of the irradiation light.

[0206] As shown in Figure 11 , the distance measuring device 21A and the distance measuring device 21B are light-emitting-side distance measuring devices 21, and the distance measuring device 21E is a light-receiving-side distance measuring device 21.

[0207] The distance measuring device 21A emits irradiation light in a light-emitting pattern A. The distance measuring device 21B emits irradiation light in a light-emitting pattern B. The light-receiving-side distance measuring device 21E performs a light-receiving operation in synchronization with the light-emitting pattern B of the distance measuring device 21B, so that the light-receiving-side distance measuring device 21E can receive reflected light generated by the irradiation light emitted by the distance measuring device 21B while preventing interference of the irradiation light. The light-emitting pattern of the light to be received from the distance measuring device 21B can be confirmed from light-emitting pattern information indicating the light-emitting pattern and being part of the light source information.

[0208] Referring back to Figure 10 , the time synchronization section 50 exchanges time stamps between the distance measuring device 21 that emits pulsed light and the distance measuring device 21 that receives pulsed light, and achieves time synchronization with high accuracy. For example, by using a protocol such as PTP (Precision Time Protocol), the time synchronization section 50 achieves time synchronization with an accuracy that does not affect the emission and reception of light at light sources and sensors located at remote locations. Alternatively, the time synchronization section 50 can employ a method that achieves synchronization between the shutter timings of a plurality of cameras using wireless communication, which is disclosed in “Sameer Ansari; Neal Wadhwa; Rahul Garg; Jiawen Chen, “Wireless Software Synchronization of Multiple Distributed Cameras”, 2019 IEEE International Conference on Computational Photography, ICCP2019”, to achieve time synchronization.

[0209] The timing control section 51 generates start timing control information based on the time information provided from the time synchronization section 50 or the like, and provides the start timing control information to the laser driver 46 and the ToF sensor 52.

[0210] The ToF sensor 52 receives reflected light generated from the irradiation light emitted from the laser emission section 47 of the same distance measuring device 21 or a separate distance measuring device 21 and reflected by an object, generates a component image signal, and provides the image signal to the signal processing section 53. If necessary, the ToF sensor 52 is provided with a part of the light source information from the light source information acquisition section 48.

[0211] The ToF sensor 52 controls exposure of the pixels in the pixel array section based on the light emission pulse signal corresponding to the operation mode provided from the operation mode setting section 45, the light source information provided from the light source information acquisition section 48, and the start timing control information provided from the timing control section 51, and provides an image frame of the 0-degree phase, the 90-degree phase, the 180-degree phase, or the 270-degree phase to the signal processing section 53. In a case where the light source calibration data is provided from the light source information acquisition section 48 to the ToF sensor 52, the ToF sensor 52 performs correction processing of correcting a cyclic error, and calculates an image frame of each phase.

[0212] The signal processing section 53 acquires the image frames of the 0-degree phase, the 90-degree phase, the 180-degree phase, and the 270-degree phase from the ToF sensor 52. Further, the signal processing section 53 acquires information on the modulation frequency or the like of the received reflected light from the light source information acquisition section 48.

[0213] Thereafter, for each pixel, the signal processing section 53 calculates the distance (depth value) d from the distance measuring unit 3 to the object 24 using the above-described expression (3) based on the acquired four-phase image frames and the information on the acquired modulation frequency or the like. The signal processing section 53 generates a depth image in which the distance d from the distance measuring unit 3 to the object 24 is stored as a pixel value of a pixel for which the distance d is calculated, and outputs the depth image to a unit or a block of a later stage.

[0214] In a case where the distance measuring device 21 includes both the light source 2 and the distance measuring unit 3, that is, in a case where the distance measuring device 21 is provided with both the light emission function and the light reception function, the distance measuring device 21 has the configuration explained so far.

[0215] Note that, in a case where the distance measuring device 21 has only one of the light emission function or the light reception function, the block necessary only for the light emission function or the light reception function is appropriately omitted.

[0216] Specifically, in the case where the distance measuring device 21 has only a function of emitting irradiation light, for example, similarly to the distance measuring device 21A, a functional block diagram of the distance measuring device 21 is as shown in FIG. 32. Figure 14

[0217] On the other hand, in the case where the distance measuring device 21 has only a function of receiving reflected light, for example, similarly to the distance measuring device 21C, a functional block diagram of the distance measuring device 21 is as shown in FIG. 35. Figure 15

[0218] <4. Processing flow of distance measuring device>

[0219] Next, control for achieving frequency synchronization and phase synchronization with the pulsed light output from each light source 2 of the distance measuring device 21 having a light emitting function, specifically, the distance measuring device 21A, the distance measuring device 21B, and the distance measuring device 21D will be described. As a result of achieving frequency synchronization and phase synchronization with each of the distance measuring devices 21 having a light emitting function, the distance measuring unit 3 of the freely selected distance measuring device 21 receives the irradiation light emitted from the light source 2 of the separate distance measuring device 21, and thus it is possible to measure the distance to the freely selected object 24.

[0220] First, the light emitting control processing in the light emitting side distance measuring device 21 which emits pulsed light will be described with reference to the flowchart in FIG. 32. For example, this processing starts when the light emitting side distance measuring device 21 is turned on. Figure 16

[0221] Note that the information on the previously measured position (relative position) of the distance measuring device 21 is stored as light source information in the memory 49 of the distance measuring device 21 under the condition that the light emitting control processing in Figure 16 performed by the light emitting side distance measuring device 21 and the light receiving control processing in Figure 17 performed by the light receiving side distance measuring device 21.

[0222] First, in step S1, the communication module 42 establishes a wireless communication connection with the light receiving side distance measuring device 21 which receives the pulsed light emitted from the light emitting side distance measuring device 21 through the network 23 or directly. For example, in the wireless LAN standardized by IEEE (Institute of Electrical and Electronics Engineers) 802.11, a connection with a communication partner is established by obtaining a transmission opportunity through a mechanism called CSMA / CA (Carrier Sense Multiple Access / Collision Avoidance), which is a mechanism for avoiding mutual interference between terminals.

[0223] ​​​During the wireless communication connection, the synchronous clock generation section 61 of the communication module 42 generates a transmission signal on the basis of the master clock supplied from the clock source 41, and transmits this transmission signal via the antenna 43. Further, the synchronous clock generation section 61 generates a synchronous clock signal corresponding to the frequency (carrier frequency or modulation frequency) of the transmission signal, and supplies this synchronous clock signal to the reference clock generation section 44.

[0224] In step S2, the time synchronization section 50 exchanges time stamps with the light-receiving-side distance measuring apparatus 21 that receives the pulsed light, and achieves high-precision time synchronization. Information on the synchronized time is supplied to the timing control section 51.

[0225] In step S3, the light source information acquisition section 48 acquires the light source information from the memory 49, and supplies the light source information to the signal transmission / reception section 62.

[0226] In step S4, the signal transmission / reception section 62 of the communication module 42 encodes the data to be transmitted by a predetermined encoding method, and supplies the encoded data to the synchronous clock generation section 61. For example, the data to be transmitted includes at least one of the unique ID included in the light source information supplied by the light source information acquisition section 48, the light emission time, the light emission start time information, the operation mode, and the like. Further, the signal transmission / reception section 62 supplies the operation mode to the operation mode setting section 45.

[0227] In step S5, the reference clock generation section 44 generates a reference clock signal on the basis of the synchronous clock signal supplied from the synchronous clock generation section 61, and supplies the reference clock signal to the operation mode setting section 45.

[0228] In step S6, the operation mode setting section 45 generates a light emission pulse signal corresponding to the operation mode supplied from the signal transmission / reception section 62, using the reference clock signal supplied from the reference clock generation section 44, and supplies the light emission pulse signal to the laser driver 46.

[0229] In step S7, the timing control section 51 generates start timing control information on the basis of the time information supplied from the time synchronization section 50 and the like, and supplies the start timing control information to the laser driver 46.

[0230] In step S8, the laser driver 46 supplies a drive signal to the laser emission section 47 at a predetermined timing on the basis of the start timing control information.

[0231] In step S9, the light emission of the laser emission section 47 is turned on and off repeatedly at a predetermined cycle in accordance with the drive signal supplied from the laser driver 46.

[0232] Thus, the light emission control processing is completed.

[0233] Next, the light reception control processing of the light reception side distance measuring apparatus 21 that receives pulsed light will be described with reference to the flowchart in Figure 17

[0234] First, in step S21, the communication module 42 establishes wireless communication with the light emission side distance measuring apparatus 21 that emits pulsed light through the network 23 or directly.

[0235] The synchronization clock generation section 61 of the communication module 42 generates a signal of a synchronization clock signal corresponding to the data transmission time by performing synchronization processing on a transmission signal received via the antenna 43 through the wireless communication connection, and supplies the generated signal to the reference clock generation section 44. As a result, the operation of the light reception side is controlled based on the synchronization clock signal that takes the one clock source 41 of the light emission side as a reference, and frequency synchronization is achieved between the light emission side and the light reception side.

[0236] In step S22, the time synchronization section 50 exchanges time stamps with the light emission side distance measuring apparatus 21 that emits pulsed light, and achieves high-precision time synchronization. Information on the synchronized time is supplied to the timing control section 51.

[0237] In step S23, the signal transmission / reception section 62 demodulates the signal supplied from the synchronization clock generation section 61, and acquires the transmitted data. The signal transmission / reception section 62 supplies the operation mode acquired from the received signal, for example, to the operation mode setting section 45, and supplies the remaining light source information to the light source information acquisition section 48.

[0238] In step S24, the light source information acquisition section 48 causes the memory 49 to store the light source information acquired from the signal transmission / reception section 62, and further supplies a part of the light source information to the ToF sensor 52 and the signal processing section 53.

[0239] In step S25, the reference clock generation section 44 generates a reference clock signal based on the synchronization clock signal supplied from the synchronization clock generation section 61, and supplies the reference clock signal to the operation mode setting section 45.

[0240] In step S26, the operation mode setting section 45 generates a light emission pulse signal corresponding to the operation mode supplied from the signal transmission / reception section 62, using the reference clock signal supplied from the reference clock generation section 44, and supplies the light emission pulse signal to the ToF sensor 52.

[0241] ​In step S27, the ToF sensor 52 receives reflected light reflected by the predetermined object 24 based on the light emission pulse signal, and the signal processing section 53 detects a phase difference of the irradiation light from the individual ranging apparatus 21 based on the light reception result, and performs phase matching.

[0242] Even when frequency synchronization is achieved, the timing of the light emission pulse applied by the respective ranging apparatuses 21 slightly deviates from each other. Therefore, it is necessary to compensate for the phase difference.

[0243] For example, the ranging apparatus 21B emits irradiation light from the light source 2 of the ranging apparatus 21B as shown in Figure 18 , and acquires a three-dimensional position of the point F on the object 24 in the light source coordinate system.

[0244] Next, in order to prevent interference of the irradiation light, for example, the ranging apparatus 21B receives reflected light generated by irradiation light emitted from the light source 2 of the ranging apparatus 21A in a time-division manner and reflected by the point F on the object 24, and acquires a distance D AFB from the ranging apparatus 21A to the point F on the object 24 based on the light reception result. AFB Here, the acquired distance D AFB is represented by the following equation.

[0245] [mathematical calculation 7]

[0246]

[0247] That is, the distance D AF is equal to the sum of the distance |V FB | from the ranging apparatus 21A to the point F on the object 24, the distance |V AB | from the point F on the object 24 to the ranging apparatus 21B, and the distance C AF corresponding to the phase difference of the phases of the light source 2 of the ranging apparatus 21A and the ranging apparatus 21B. Here, V FB represents a vector from the ranging apparatus 21A to the point F on the object 24, and V AF represents a vector from the point F on the object 24 to the ranging apparatus 21B. Note that the symbol indicating the vector (→) will be omitted above V FB and V AB in this specification. |V| represents the absolute value of the vector V.

[0248] The signal processing section 53 converts the distance C AB into a phase difference T AB using the speed of light. Thereafter, a command for shifting the phase of the light source 2 of the ranging apparatus 21A by the phase difference amount T d is issued by wireless communication, so that the phase difference between the ranging apparatuses 21A and 21B can be compensated for.

[0249] Thus, the light reception control processing is completed.

[0250] According to the light emission control processing in the light emission side distance measuring apparatus 21 and the light reception control processing in the light reception side distance measuring apparatus 21, the light reception side distance measuring apparatus 21 can perform frequency synchronization with the synchronization clock signal generated by the light emission side distance measuring apparatus 21 by performing synchronization processing on the transmission signal transmitted by the light emission side through wireless communication. Subsequently, time synchronization is achieved, and a reference clock signal serving as a reference clock for the light emission timing and the light reception timing is generated.

[0251] That is, the light emission timing of the light emission side distance measuring apparatus 21 and the light reception timing of the light reception side distance measuring apparatus 21 are each controlled based on one clock source 41 of the light emission side.

[0252] Thus, even when the light emission side distance measuring apparatus 21 and the light reception side distance measuring apparatus 21 are located at a distant position, respectively, frequency synchronization between the laser emission part 47 (light source) and the ToF sensor 52 can be achieved. Accordingly, it is possible to reduce the error of the distance measurement result.

[0253] In a case where there are a plurality of distance measuring apparatuses 21 that emit pulsed light, and three or more distance measuring apparatuses 21 are simultaneously operated in any one of the operation modes of (1) the time division mode, (2) the modulation frequency mode, and (3) the light emission pattern mode, as shown in FIG. 6, the distance measuring apparatuses 21 each generate a synchronization clock signal or a reference clock signal in synchronization with the master clock of the clock source 41 of any one of the distance measuring apparatuses 21. Figures 11 to 13

[0254] It is to be noted that, in a case where the operation mode is set to the modulation frequency mode or the light emission pattern mode, frequency synchronization is necessary, and time synchronization is unnecessary.

[0255] Further, according to the above-described light reception control processing, it is possible to detect the phase difference between the light emission side distance measuring apparatus 21 and the light reception side distance measuring apparatus 21, and to compensate for the phase difference.

[0256] In the distance measuring system 11, a state in which frequency synchronization and phase synchronization of light emission pulses from a plurality of distance measuring apparatuses 21 each having a light emission function are achieved is constructed. Thus, for example, the distance measuring unit 3 of the distance measuring apparatus 21E installed on the mobile carrier vehicle 22 can measure the distance of a freely selected object 24 based on the irradiation light emitted from the light sources 2 of the plurality of distance measuring apparatuses. The method in which the distance measuring unit 3 of the distance measuring apparatus 21E installed on the carrier vehicle 22 measures the distance of the freely selected object 24 will be described below.

[0257] ​The light emission side distance measuring apparatus 21 and the light reception side distance measuring apparatus 21 can transmit and receive light source information through wireless communication. Examples of information that can be transmitted and received as light source information include a unique ID, a light emission time, light emission start time information, a repetition frequency, a modulation frequency, a component length, a position and an attitude of the distance measuring apparatus, a frame length, a wavelength of a light source, a light emission pattern, light source calibration data, an operation mode, and the like, as described previously. However, it is not necessary to transmit and receive all types of information. Some parts of the information can be pre-stored in the memory 49 as fixed data.

[0258] In a case where separate distance measuring apparatuses 21 are provided on the light emission side and the light reception side, the light reception side distance measuring apparatus 21 as a separate apparatus can have difficulty in reserving light source calibration data in advance. In this case, the light source calibration data is transmitted from the light emission side to the light reception side as a part of the light source information, so that the light reception side can accurately detect a phase difference.

[0259] <5. Modification of the First Embodiment>

[0260] In the above-described first embodiment, the plurality of distance measuring apparatuses 21 that constitute the distance measuring system 11 are configured to achieve synchronization of the synchronization clock signal and the reference clock signal with the clock source 41 of any distance measuring apparatus 21.

[0261] However, a clock source of an apparatus other than the plurality of distance measuring apparatuses 21 that constitute the distance measuring system 11 can be used as a clock source with which the distance measuring apparatuses 21 achieve synchronization.

[0262] For example, a signal of a global navigation satellite system (GNSS) such as a GPS (Global Positioning System), a GLONASS (Global Navigation Satellite System), Galileo, or a QZSS (Quasi-Zenith Satellite System) (hereinafter referred to as a GNSS signal) has a function of outputting a high-precision clock signal and high-precision time information. The plurality of distance measuring apparatuses 21 that constitute the distance measuring system 11 can each be configured to receive a GNSS signal and achieve synchronization with the same clock signal and time information acquired from the GNSS signal.

[0263] Figure 19 is a block diagram of a distance measuring apparatus 21' that achieves time synchronization based on a GNSS signal.

[0264] Figure 19 corresponds to Figure 10 Parts of corresponding elements in

[0265] Figure 19Two ranging devices 21' are shown, each of which has a light emission function and a light reception function. Thereafter, a ranging device 21'-1 as one of the two ranging devices 21' is disposed on the light emission side, and a ranging device 21'-2 as the other of the two ranging devices 21' is disposed on the light reception side. The ranging devices 21' each having the same configuration operate as a light emission side and a light reception side according to, for example, predetermined setting information (light source information).

[0266] The ranging device 21' includes a GNSS receiving section 81, a time information setting section 82, a reference clock generating section 44, an operation mode setting section 45, a laser driver 46, a laser emitting section 47, a light source information acquiring section 48, a memory 49, a time synchronization section 50, a timing control section 51, a ToF sensor 52, and a signal processing section 53.

[0267] In other words, the ranging device 21' is obtained by modifying the ranging device 21 in Figure 10 to include the GNSS receiving section 81 and the time information setting section 82 in place of the clock source 41, the communication module 42, and the antenna 43.

[0268] The GNSS receiving section 81 receives a GNSS signal from a GNSS satellite 83, and supplies the GNSS signal to the time information setting section 82 and the reference clock generating section 44.

[0269] The time information setting section 82 acquires high-precision time information from the GNSS signal supplied from the GNSS receiving section 81, and performs time setting according to a standard time.

[0270] The reference clock generating section 44 extracts a clock signal from the GNSS signal, generates a reference clock signal serving as a light emission reference for the laser emitting section 47 based on this clock signal, and supplies this reference clock signal to the operation mode setting section 45.

[0271] The memory 49 stores light source information, and supplies the light source information to the light source information acquiring section 48 as needed. The light source information stored in the memory 49 includes information necessary for the corresponding ranging device 21' to operate as a light emission side or a light reception side. For example, data on the operation mode, a light emission pattern, light source calibration data, and the like of the light emission side ranging device 21' is pre-stored in the memory 49 of the light reception side ranging device 21'-2.

[0272] Since the distance measuring device 21' has the above configuration, it is possible to achieve synchronization of the light emission timing and the light reception timing using a clock source of another device (the GNSS satellite 83) that is separate from the light emission side distance measuring device 21' and the light reception side distance measuring device 21'. That is, it is possible to achieve frequency synchronization between the laser emission section 47 of the light emission side and the ToF sensor 52 of the light reception side. Accordingly, it is possible to reduce the error of the distance measurement result.

[0273] In the above modification, the GNSS signal is received, synchronization of time and clock is achieved based on the received GNSS signal, and since the transmission and reception are preset, data transmission and reception regarding the light source information and the like are not additionally performed. However, data regarding the light source information and the like can be transmitted and received by another wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark) and the like.

[0274] Note that, in addition to the above GNSS signal, a wireless signal of UWB (Ultra Wide Band) communication can be used as a signal that outputs a high-precision clock signal and high-precision time information. In this case, the GNSS reception section 81 of the distance measuring device 21' functions as a UWB reception section 81 that receives a wireless signal of UWB (Ultra Wide Band) communication.

[0275] Further, in Figure 19 , the light emission side distance measuring device 21' and the light reception side distance measuring device 21' have the same configuration, and each distance measuring device 21' can be set to the light emission side or the light reception side. However, in the case where the distance measuring device 21' is limited to the light emission side or the light reception side, unnecessary parts thereof can be omitted, as shown in Figure 14 and Figure 15 .

[0276] <6. Second embodiment of the distance measuring system>

[0277] Figure 20 A configuration example of a distance measuring system according to a second embodiment of the present disclosure is shown.

[0278] The distance measuring system 11 according to the second embodiment in Figure 20 includes a plurality of unmanned aerial vehicles 101. Each unmanned aerial vehicle 101 is a mobile flying body equipped with a plurality of rotors. Each unmanned aerial vehicle 101 includes a distance measuring device including a light source 2 and a distance measuring unit 3.

[0279] For ease of explanation, it is assumed that the distance measuring system 11 includes two unmanned aerial vehicles 101A and 101B, as shown in Figure 20As shown, although the distance measuring system 11 can include three or more unmanned aerial vehicles 101. The unmanned aerial vehicles 101A and 101B have the same configuration, and each has a different unique ID to identify the respective unmanned aerial vehicle body. For example, the unique ID of the unmanned aerial vehicle 101A is "0" (ID0), and the unique ID of the unmanned aerial vehicle 101B is "1" (ID1).

[0280] The unmanned aerial vehicle 101A applies irradiation light to the unmanned aerial vehicle 101B at a predetermined modulation frequency, and further receives irradiation light applied from the unmanned aerial vehicle 101B. In addition, the unmanned aerial vehicle 101B applies irradiation light to the unmanned aerial vehicle 101A at a predetermined modulation frequency, and further receives irradiation light applied from the unmanned aerial vehicle 101A.

[0281] Similar to the modification of the first embodiment, based on the GNSS signal provided from the GNSS satellite 102, the synchronization clock signal and the reference clock signal for the unmanned aerial vehicles 101A and 101B are synchronized with each other.

[0282] The unmanned aerial vehicles 101A and 101B report to each other the light propagation time or distance measured by each of the unmanned aerial vehicles 101A and 101B. Therefore, the distance to the opposing unmanned aerial vehicle (hereinafter referred to as the relative distance) can be accurately measured.

[0283] The principle of calculating the relative distance will be described with reference to Figure 21 and Figure 23 .

[0284] First, as Figure 21 shown, the unmanned aerial vehicles 101A and 101B each include a light source 2 and a distance measuring unit 3.

[0285] The light source 2 of the unmanned aerial vehicle 101A applies irradiation light to the unmanned aerial vehicle 101B at a predetermined modulation frequency. In addition, the distance measuring unit 3 of the unmanned aerial vehicle 101A receives irradiation light applied from the light source 2 of the unmanned aerial vehicle 101B.

[0286] The light source 2 of the unmanned aerial vehicle 101B applies irradiation light to the unmanned aerial vehicle 101A at a predetermined modulation frequency. In addition, the distance measuring unit 3 of the unmanned aerial vehicle 101B receives irradiation light applied from the light source 2 of the unmanned aerial vehicle 101A.

[0287] When the relative distance between the unmanned aerial vehicles 101A and 101B is defined as d, the light propagation time Δt d is represented by Expression (8).

[0288] [Mathematical Calculation 8]

[0289]

[0290] In Expression (8), c represents the speed of light.

[0291] Figure 22 is a diagram schematically depicting pulsed light applied by the drones 101A and 101B in the indirect ToF method.

[0292] The drones 101A and 101B share high-precision time information based on GNSS signals provided from the GNSS satellites 102, so that the irradiation light emission timing can be matched. However, it is difficult to match the timing of the emission pulses.

[0293] Specifically, even in the case where the light emission timing of the drones 101A and 101B is matched, the drone 101B in some cases regards the light emission pulse from the drone 101A as being delayed Δt offset , as shown in FIG. 8. Figure 22

[0294] That is, the light propagation time Δt 1→2 between the drones 101 observed by the ranging unit 3 of the drone 101B is represented by the following expression (9) because the light propagation time is affected by the offset Δt offset corresponding to the light emission timing deviation.

[0295] Δt 1→2 = Δt d + Δt offset ...(9)

[0296] On the other hand, the drone 101A regards the light emission pulse from the drone 101B as being advanced Δt offset . Therefore, the light propagation time Δt 2→1 between the drones 101 observed by the ranging unit 3 of the drone 101A is represented by the following expression (10).

[0297] Δt 2→1 = Δt d - Δt offset ...(10)

[0298] After that, the drones 101A and 101B mutually report the propagation times Δt measured by each of the drones 101A and 101B, so that the accurate light propagation time Δt offset which has been removed the influence of the offset Δt d can be obtained by the following expression (11). Then, the accurate relative distance d can be obtained by expression (12) which is a modification of expression (8).

[0299] [mathematical calculation 9]

[0300]

[0301] d = c · Δt​d ...(12)

[0302] Note that the unmanned aerial vehicles 101A and 101B can report the propagation time Δt to each other, or can report the distance d obtained by multiplying the propagation time Δt by the speed of light c to each other, and can calculate the average of the distances d. Also in the latter case, the accurate relative distance d can be obtained.

[0303] Figure 23 is a diagram schematically depicting the pulsed light applied from the unmanned aerial vehicles 101A and 101B in the direct ToF method.

[0304] Also in the direct ToF method, the expression can be defined in the same manner as the indirect ToF method explained in Figure 22

[0305] That is, as shown in Figure 23 , the unmanned aerial vehicle 101B regards the light emission pulse from the unmanned aerial vehicle 101A as a light emission pulse delayed by the propagation time Δt offset observed by the ranging unit 3 of the unmanned aerial vehicle 101B between the unmanned aerial vehicles 101 can be expressed by Expression (9). 1→2

[0306] On the other hand, the propagation time Δt 2→1 of light between the unmanned aerial vehicles 101 observed by the ranging unit 3 of the unmanned aerial vehicle 101A can be expressed by Expression (10).

[0307] Therefore, the accurate propagation time Δt d of light, which has been removed from the influence of the offset Δt offset , can be obtained by Expression (11). The accurate relative distance d can be obtained by Expression (12).

[0308] <Unmanned aerial vehicle block diagram>

[0309] Figure 24 is a block diagram of the ranging apparatus integrated in the unmanned aerial vehicle 101.

[0310] Figure 24 The portions corresponding to the ranging apparatus 21' shown in Figure 19 are also denoted by the same reference numerals, and the explanation thereof is appropriately omitted.

[0311] The ranging apparatus of the unmanned aerial vehicle 101 has a configuration obtained by modifying the configuration of the ranging apparatus 21' in Figure 19 to further include the IMU 121, the position / posture calculation section 122, the communication section 123, and the inter-light-source distance calculation section 124.

[0312] ​​In the second embodiment, the light source information acquisition section 48 acquires light source information (e.g., an operation mode) stored in the storage 49, and provides this light source information to the operation mode setting section 45, the timing control section 51, the signal processing section 53, and the like, as appropriate.

[0313] The IMU 121 is configured to detect an angle (or an angular velocity) and an acceleration in three-axis directions of the drone 101, and provide a signal indicating a detection result to the position / attitude calculation section 122.

[0314] The position / attitude calculation section 122 detects a position and an attitude of the drone 101 based on a detection signal provided from the IMU 121 and position information provided from the GNSS reception section 81, and provides the position and the attitude to the communication section 123 and the inter-light-source distance calculation section 124. The position / attitude calculation section 122 can detect one of the position and the attitude of the drone 101.

[0315] Note that the IMU 121 is not an essential part, and thus can be omitted. In this case, the position / attitude calculation section 122 detects the position and the attitude of the drone 101 based on the position information provided from the GNSS reception section 81. Further, a different type of sensor (e.g., a magnetic sensor or a pressure sensor) can be provided instead of the IMU 121, so as to detect the position and the attitude of the drone 101 based on a detection signal obtained by the sensor. Alternatively, both the IMU 121 and the magnetic sensor can be provided, and the like.

[0316] The signal processing section 53 includes a depth image generation section 141 and an external modulation light source detection section 142.

[0317] The depth image generation section 141 generates a depth image in which a distance d to the opposing drone 101, i.e., a distance d calculated for each pixel, is stored as a pixel value of a pixel. The generated depth image is provided to the communication section 123 and the inter-light-source distance calculation section 124.

[0318] The external modulation light source detection section 142 detects a position (light source position) of the light source 2 (laser emission section 47) of the opposing drone 101 that emits light.

[0319] In the indirect ToF method, the external modulation light source detection section 142 detects the position of the light source using a confidence image in which a confidence value indicating a light intensity is stored as a pixel value of each pixel.

[0320] Figure 25 Examples of a confidence image obtained when the drone 101A observes the drone 101B, and an example of a confidence image obtained when the drone 101B observes the drone 101A are shown.

[0321] The external modulated light source detection section 142 detects the position of the pixel whose pixel value (confidence value) is equal to or greater than a threshold value decided in advance and is the highest pixel in the confidence image, as the light source position.

[0322] In contrast, in the direct ToF method, the external modulated light source detection section 142 detects the position of the pixel having a peak of the generated histogram that is equal to or greater than a threshold value decided in advance and is the highest, as the light source position.

[0323] The drone 101A and the drone 101B can emit light in a time-division manner, thereby performing light emission operation and light reception operation, respectively. Alternatively, in an environment where there is no object around the drone 101A and the drone 101B, for example, in a case where the background other than the drone 101 is the sky, the drone 101A and the drone 101B can simultaneously emit light and receive light.

[0324] After that, the signal processing section 53 supplies the light source position detected by the external modulated light source detection section 142 and the distance dl detected at the light source position to the communication section 123 and the inter-light source distance calculation section 124.

[0325] The communication section 123 communicates with the partner drone 101 through wireless communication (for example, WiFi (registered trademark), Bluetooth (registered trademark)) or mobile body communication called 4G or 5G line in general, for example. The communication section 123 transmits the light source position of the partner drone 101 obtained by the signal processing section 53 and the distance dl from the partner drone 101 and the approximate self-position detected by the position / posture calculation section 122 to the partner drone 101 through wireless communication. Further, the communication section 123 receives the light source position and the distance d2 detected by the partner drone 101 and supplies the light source position and the distance d2 to the inter-light source distance calculation section 124.

[0326] The inter-light source distance calculation section 124 acquires the light source position, the distance d2, and the approximate position received from the partner drone 101 through wireless communication from the communication section 123. Further, the inter-light source distance calculation section 124 acquires the distance dl from the partner drone 101 detected by self-measurement and the light source position of the partner drone from the signal processing section 53. By using the distance dl obtained by self-measurement and the distance d2 measured by the partner drone 101, the inter-light source distance calculation section 124 calculates the accurate relative distance d that is removed from the influence of the above-described offset At. offset Further, the inter-light source distance calculation section 124 uses the approximate self-position and the self-posture detected by the position / posture calculation section 122 to improve the accuracy of the relative distance d.

[0327] Figure 26is a diagram for explaining a process of improving the accuracy of the relative distance d using the approximate self attitude detected by the position / attitude calculation section 122.

[0328] Based on the principle of the pinhole camera, the direction of the opponent drone 101 is obtained from the light source position detected in the confidence image. In addition, the self-measured distance d1 from the opponent drone and the distance d2 transmitted from the opponent drone 101 are used to obtain the accurate relative distance d.

[0329] In this case, due to the arbitrariness of the rotation around the line-of-sight direction, the position and attitude are not uniquely determined, as shown in A of Figure 26

[0330] Therefore, using the detection result obtained by the IMU 121, the direction of gravity on the ToF sensor 52 can be recognized, as shown in B of Figure 26

[0331] Since the attitude (tilt) of the drone 101 detected by the IMU 121 is used instead of the high-precision distance d obtained by the drone 101A and the drone 101B reporting each other's distance d, the accuracy of the relative position and attitude is improved.

[0332] Figure 27 is a diagram for explaining a process of improving the accuracy of the relative distance d using the approximate self position detected by the position / attitude calculation section 122 and the approximate position of the opponent drone 101 transmitted from the opponent drone 101.

[0333] In the indirect ToF method, the measurable distance d is limited by the modulation frequency. For example, in the case where the modulation frequency is 20 Mhz, 1 phase period corresponds to about 15 m. As shown in Figure 27 the phase difference detected for a relative distance d of 15 m is equal to the phase difference detected for a relative distance of 30 m. Therefore, it is not possible to distinguish between a distance of 15 m and a distance of 30 m.

[0334] ​​As a countermeasure to this, the approximate own position of the drone 101 detected by the position / posture calculation section 122 and the approximate position of the partner drone 101 transmitted from the partner drone 101 can be used to determine whether the relative distance d is 15 m or 30 m. That is, by using the rough own position detected by the position / posture calculation section 122, the accuracy of the relative distance d can be increased.

[0335] Note that, to determine whether the relative distance d is 15 m or 30 m, a method of setting a plurality of modulation frequencies and determining the final distance in the common distance obtained from the measurement results at the respective modulation frequencies can be employed.

[0336] <Processing flow of relative distance measurement processing>

[0337] Next, the processing for measuring the relative distance between the drones 101 will be explained with reference to the flowchart in Figure 28 This processing is started, for example, after the drone 101 starts flying.

[0338] First, in step S41, the GNSS reception section 81 receives a GNSS signal from the GNSS satellite 83, and supplies the GNSS signal to the time information setting section 82 and the reference clock generation section 44. The time information setting section 82 acquires high-precision time information from the GNSS signal supplied from the GNSS reception section 81, and performs time setting in accordance with a standard time. The reference clock generation section 44 extracts a clock signal from the GNSS signal, generates a reference clock signal on the basis of the clock signal, and supplies the reference clock signal to the operation mode setting section 45. Further, the GNSS reception section 81 supplies position information acquired from the GNSS signal to the position / posture calculation section 122.

[0339] In step S42, the position / posture calculation section 122 detects the position and posture of the drone 101 on the basis of the detection signal supplied from the IMU 121 and the position information supplied from the GNSS reception section 81, and supplies the position and posture to the communication section 123 and the inter-light-source distance calculation section 124. In the case where the IMU 121 is omitted, the position / posture calculation section 122 detects the position and posture of the drone 101 on the basis of only the position information supplied from the GNSS reception section 81.

[0340] In step S43, the timing control section 51 generates start timing control information on the basis of the high-precision time information supplied from the time setting section 82 and the like, and supplies the start timing control information to the laser driver 46 and the ToF sensor 52.

[0341] In step S44, the operation mode setting section 45 generates a light emission pulse signal corresponding to the operation mode supplied from the light source information acquisition section 48 using the reference clock signal supplied from the reference clock generation section 44, and supplies the light emission pulse signal to the laser driver 46 and the ToF sensor 52.

[0342] In step S45, the laser driver 46 generates a drive signal at a predetermined timing based on the start timing control information and supplies the drive signal to the laser emission section 47.

[0343] In step S46, the laser emission section 47 repeats the turn-on and turn-off of the light emission based on the drive signal supplied from the laser driver 46 at a predetermined cycle.

[0344] In step S47, the ToF sensor 52 receives the irradiation light applied from the partner drone 101 based on the start timing control information supplied from the timing control section 51 and the reference clock signal supplied from the operation mode setting section 45, generates image frames of 0-degree phase, 90-degree phase, 180-degree phase, and 270-degree phase in order, and supplies the image frames to the signal processing section 53.

[0345] In step S48, the depth image generation section 141 of the signal processing section 53 generates a depth image in which the distance d1 to the partner drone 101 calculated in each pixel is stored as a pixel value of the pixel, and supplies the depth image to the communication section 123 and the inter-light-source distance calculation section 124.

[0346] In step S49, the external modulation light source detection section 142 detects the position (light source position) of the light source 2 (laser emission section 47) of the partner drone 101 that is emitting light, and supplies the detected position to the communication section 123 and the inter-light-source distance calculation section 124.

[0347] In step S50, the communication section 123 transmits the distance d1 to the partner drone 101 obtained by the signal processing section 53, and the light source position of the partner drone 101 and the approximate own position detected by the position / posture calculation section 122 to the partner drone 101 by wireless communication. Further, the communication section 123 receives the light source position and the distance d2 detected by the partner drone 101, and the approximate position of the partner drone 101, and supplies the light source position, the distance d2, and the approximate position to the inter-light-source distance calculation section 124.

[0348] In step S51, the inter-light-source distance calculation section 124 calculates the accurate relative distance d with respect to the partner drone 101. Specifically, the inter-light-source distance calculation section 124 identifies the gravity direction using the coarse self attitude detected by the position / attitude calculation section 122, and obtains the relative position and attitude. Further, the inter-light-source distance calculation section 124 acquires the light source position, the distance d2, and the approximate position received from the partner drone 101 by wireless communication from the communication section 123. Further, using the distance dl from the partner drone 101 detected by self detection and the distance d2 transmitted from the partner drone 101, the inter-light-source distance calculation section 124 calculates the accurate relative distance d, from which the influence of the offset At is removed. Further, the inter-light-source distance calculation section 124 improves the accuracy of the relative distance d using the coarse position obtained by self detection and the coarse position detected by the partner drone 101. offset

[0349] Up to this point, the relative distance measurement processing is completed. The drones 101A and 101B execute the above-described relative distance measurement processing in parallel.

[0350] According to the above-described relative distance measurement processing, the distances dl (or d2) detected by self detection are reported to each other, and thus the accurate distance d is calculated.

[0351] In the second embodiment, the frequency synchronization between the drones 101A and 101B is achieved by implementing synchronization based on the received wireless signal, as in the first embodiment. However, the method of achieving frequency synchronization using the reflected light (main irradiation light) of the master drone 101 disclosed in PTL 1 can be employed.

[0352] <7. Third embodiment of distance measuring system>

[0353] Figure 29 A configuration example of a distance measuring system according to a third embodiment of the present disclosure is shown.

[0354] The distance measuring system 11 according to the third embodiment in the example in Figure 29 includes a plurality of distance measuring devices 201. In Figure 29 , the distance measuring system 11 includes five distance measuring devices 201 as distance measuring devices 201A to 201E. However, the number of distance measuring devices 201 is not limited to five, and any number of distance measuring devices 201 can be provided.

[0355] In the third embodiment, the four distance measuring devices 201A to 201D each include the light source 2 and the distance measuring unit 3, and are fixed to the upper side (for example, the ceiling). On the other hand, the distance measuring device 201E includes only the distance measuring unit 3, and is provided at a position on, for example, the floor at which the distance measuring device 201E can receive the irradiation light applied from the four distance measuring devices 201A to 201D or the reflected light thereof.​

[0356] The configurations of the distance measuring devices 201A to 201D each including the light source 2 and the distance measuring unit 3 are similar to that of the first embodiment shown in FIG. 1. Figure 10 The configuration of the distance measuring device 201E including only the distance measuring unit 3 is similar to that of the first embodiment shown in FIG. 1. Thus, the explanation of these configurations is omitted. Figure 15 The configuration of the distance measuring device 201E including only the distance measuring unit 3 is similar to that of the first embodiment shown in FIG. 1. Thus, the explanation of these configurations is omitted.

[0357] In the third embodiment, the distance measuring device 201E including only the distance measuring unit 3 is described by the process of calculating the distance to the distance object 202 by receiving the irradiation light applied from the plurality of (four) distance measuring devices 201A to 201D fixed to the ceiling or the like.

[0358] Figure 30 is a flowchart for explaining the distance calculation process in the distance measuring system 11 according to the third embodiment.

[0359] First, in step S61, the distance measuring devices 201A to 201E each establish a connection for wireless communication. Then, the distance measuring devices 201A to 201E exchange time stamps with each other to achieve high-precision time synchronization.

[0360] In step S62, the distance measuring devices 201A to 201E each read out the light source information from the memory 49 of the corresponding distance measuring device 201, and transmit the light source information to the other distance measuring devices 201 through wireless communication. Thus, the light source information about the distance measuring devices 201A to 201E is shared.

[0361] In step S63, the distance measuring devices 201A to 201E each generate a reference clock signal in accordance with a synchronization clock signal generated by detecting a transmission signal from the master distance measuring device 201 (e.g., the distance measuring device 201A), and generate a light emission pulse signal corresponding to the operation mode.

[0362] In step S64, the distance measuring devices 201A to 201D fixed to the ceiling each emit light and receive light based on the respective light emission pulse signals. Specifically, the light source 2 (laser emission section 47) of the distance measuring devices 201A to 201D emits light in accordance with the light emission pulse signal, and the distance measuring unit 3 (ToF sensor 52) receives reflected light in accordance with the light emission pulse signal, and sequentially generates image frames of 0-degree phase, 90-degree phase, 180-degree phase, and 270-degree phase.

[0363] In step S65, the distance measuring devices 201A to 201D fixed to the ceiling each calculate the position and attitude of the distance measuring devices 201A to 201D themselves. In other words, the distance measuring devices 201A to 201D each calculate the relative position and attitude with respect to the other distance measuring devices 201.

[0364] The processing in step S65 will be described. Each of the ranging devices 201A to 201D is a depth camera that measures the distance d of an object. Thus, with the ranging devices 201A to 201D, the three-dimensional position of an object can be obtained in the camera coordinate system.

[0365] A method of obtaining the relative position and pose of a known three-dimensional point and a camera by observing the three-dimensional point with a camera is commonly called PNP (Perspective-n-Point). By using PNP, each of the ranging devices 201A to 201D calculates the relative position and pose of the ranging device 201 itself with respect to the other ranging devices 201.

[0366] For example, by receiving reflected light returned from the known object 202, the ranging device 201A acquires coordinate values of feature points on the object 202. Similarly, by receiving reflected light returned from the known object 202, the ranging device 201B also acquires coordinate values of feature points on the object 202. With the coordinate values of the feature points obtained by the ranging devices 201A and 201B and the coordinate values of the known three-dimensional points, the PNP problem is solved. Thus, the relative position and pose of the ranging devices 201A and 201B are calculated.

[0367] Alternatively, an optimization problem is solved so that the difference between the shapes of the surrounding environments of the ranging devices 201A and 201B becomes the smallest, and thus the relative pose of the ranging devices 201A and 201B can be obtained. This method is called ICP (Iterative Closest Point). For example, ICP is described in "DIGITAL IMAGE PROCESSING [Revised Edition], Computer Graphic Arts Society" and Szymon. R and Marc. L, "Efficient Variants of the ICP Algorithm", Proceedings Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 145-152.

[0368] In the above-described manner, the ranging devices 201A to 201D each calculate the relative position and pose. As a result, the relative relationship in the positions and poses of all the ranging devices 201A to 201D is obtained.

[0369] In step S66, the ranging devices 201A to 201D fixed to the ceiling each detect the phase difference of the irradiation light, and perform phase matching.

[0370] As previously described inFigure 22 As explained in the second embodiment depicted, even when frequency synchronization is achieved, the timing of the light emission pulses applied from the ranging devices 201 slightly deviate from each other. Therefore, it is necessary to calibrate the phase difference.

[0371] Specifically, the ranging devices 201A and 201B each apply illuminating light to the object 202 physically located at the same position, and receive reflected light, as shown in Figure 31

[0372] When the deviation (phase difference) of the light emission pulses applied from the ranging devices 201A and 201B is defined as Δt offset_12 , the time taken for the light emitted by the ranging device 201A to be received by the ranging device 201B is obtained by the following expression (13).

[0373] Δt 1→2 = Δt d + Δt offset_12 ...(13)

[0374] On the other hand, the time taken for the light emitted by the ranging device 201B to be received by the ranging device 201A is obtained by the following expression (14).

[0375] Δt 2→1 = Δt d - Δt offset_12 ...(14)

[0376] Therefore, the deviation (phase difference) Δt offset_12 of the light emission pulses applied from the ranging devices 201A and 201B can be obtained by the following expression (15).

[0377] Δt offset_12 = (Δt 1→2 - Δt 2→1 ) / 2...(15)

[0378] In the above-described manner, the phase difference in each of the light sources 2 of the ranging devices 201A to 201D is detected, and the illuminating light phase of the ranging devices 201A to 201D is adjusted so as not to produce a phase difference. Note that the calculation of the phase difference can be performed in a later stage of signal processing, rather than adjusting the illuminating light phase.

[0379] ​In step S67, the TOF sensor 52 of the distance measuring device 201E including only the distance measuring unit 3 receives reflected light generated by the irradiation light emitted by each of the distance measuring devices 201A to 201D and reflected by the object 202, wherein the distance measuring device is fixed to the ceiling. Due to step S66, the frequency and phase of the irradiation light emitted by the distance measuring devices 201A to 201D are synchronized. In addition, since the light source information can be shared, it is possible to identify the distance measuring device from which the received reflected light is emitted among the distance measuring devices 201A to 201D.

[0380] In step S68, the signal processing section 53 of the distance measuring device 201E calculates the distance to the object 202 based on the phase difference of the reflected light received from each of the distance measuring devices 201A to 201D. The method for calculating the distance to the object 202 will be described later.

[0381] In the above distance calculation processing, when the light source information on the irradiation light applied from the distance measuring devices 201A to 201D fixed to the ceiling is already known and when the phase synchronization and the frequency synchronization are achieved, the phase difference of the irradiation light applied from the distance measuring devices 201A to 201D is detected, and further the position and the attitude of the distance measuring device 201E can be calculated. This is similar to the case where the own position is calculated based on the difference in the GNSS signal, for example, the distance measuring devices 201A to 201D are regarded as GNSS satellites.

[0382] In the third embodiment, the frequency synchronization between the drones 101A and 101B is achieved by achieving synchronization based on the received wireless signal, as in the first embodiment. However, the method of achieving the frequency synchronization using the reflected light (primary irradiation light) of the primary distance measuring device 201 as disclosed in PTL 1 can be employed.

[0383] <8. Distance calculation method involving reception of reflected light from multiple light sources>

[0384] In the above first embodiment, for example, the distance measuring unit 3 of the distance measuring device 21E receives reflected light generated by the irradiation light emitted from the light source 2 of the distance measuring device 21A and reflected by the object 24, and reflected light generated by the irradiation light emitted from the light source 2 of the distance measuring device 21B and reflected by the object 24, and thereby the distance to the object 202 can be measured.

[0385] In the above third embodiment, for example, the distance measuring unit 3 of the distance measuring device 201E receives reflected light generated by the irradiation light emitted from the light source 2 of the distance measuring device 201B and reflected by the object 202, and reflected light generated by the irradiation light emitted from the light source 2 of the distance measuring device 201D and reflected by the object 202, and thereby the distance to the object 202 can be measured.

[0386] Hereinafter, a method of calculating a distance to a predetermined object by receiving reflected light of irradiation light applied from a plurality of light sources 2 and reflected by the predetermined object will be described.

[0387] For the purpose of explanation, a system apparatus including three light sources a to c, an object 301, and a distance measuring unit 302 is assumed as shown in Figure 32 Point D is a reflection position on the object 301 for pulsed light applied from the light sources a to c. Point O is an origin of a camera coordinate system which is a coordinate system of the distance measuring unit 302.

[0388] For example, the light sources a to c correspond to the light sources 2 of the distance measuring devices 21A, 21B, and 21D in the first embodiment, and for example, correspond to the light sources 2 of the distance measuring devices 201A, 201B, and 201D in the third embodiment. The object 301 corresponds to the object 24 in the first embodiment, and corresponds to the object 202 in the third embodiment. The distance measuring unit 302 corresponds to the distance measuring unit 3 of the distance measuring device 21E in the first embodiment, and corresponds to the distance measuring unit 3 of the distance measuring device 201E in the third embodiment.

[0389] As shown in Figure 32 The three light sources a to c are respectively located in positions E1, E2, E3. The relative positions of the light sources a to c in a light source coordinate system are already known, but the positions where the three light sources a to c are located are unknown. The relative position of the distance measuring unit 302 with respect to the light sources a to c is unknown. The distance measuring unit 302 can establish frequency synchronization based on a wireless signal transmitted from the light sources a to c, but a phase deviation still exists.

[0390] The distance measuring unit 302 calculates a distance (from the light sources a to c to the distance measuring unit 302) with use of reflected light generated based on irradiation light emitted from the light sources a to c and reflected by the object 301. The distance from the light sources a to c to the distance measuring unit 302 is represented by the following expression (16).

[0391] [mathematical calculation 10]

[0392]

[0393]

[0394]

[0395] In expression (16), u D = (u D , v D ) represents a pixel position onto which the point D on the object 301 is projected, Obs(u D , E1) represents a pixel position uD Obs(u, E1) represents the distance observed at pixel position u of the pixel array of the distance measuring unit 302 when the light source a at position E1 emits light. D Obs(u, E2) represents the distance observed at pixel position u of the pixel array of the distance measuring unit 302 when the light source b at position E2 emits light. D Obs(u, E3) represents the distance observed at pixel position u of the pixel array of the distance measuring unit 302 when the light source c at position E3 emits light. D |OD| represents the magnitude (distance) of the vector connecting the distance measuring unit 302 to the object 301, and |DE D | represents the magnitude (distance) of the vector connecting the position E k (k = 1, 2, or 3) to the reflection position on the object 301, and C represents an offset term corresponding to the distance measurement error caused by the phase difference between the clocks of the light sources a to c and the distance measuring unit 302. k (k = 1, 2, or 3) to the reflection position on the object 301, and C represents an offset term corresponding to the distance measurement error caused by the phase difference between the clocks of the light sources a to c and the distance measuring unit 302.

[0396] Note that the vector symbol (arrow symbol (→)) in the above expression (16) represents a vector starting from the origin of a certain coordinate system (e.g., a light source coordinate system). In the description of the expression in this specification, the vector symbol is omitted. Note that the vector symbol is omitted in the description in this specification in the following.

[0397] Thus, the distance measuring unit 302 calculates three calculation expressions from the three light sources a to c to the distance measuring unit 302. The expressions include

[0398] (a) distance data |OD| between the distance measuring unit 302 and the object 301,

[0399] (b) distance data |DE k | between the object 301 and each of the light sources a to c, and

[0400] (c) an offset amount corresponding to the distance measurement error generated based on the phase difference between the clock of the distance measuring unit 302 (sensor clock) and the clocks of the light sources a to c (light source clocks).

[0401] Next, using the above distances Obs(u D ,E1), distance (u D ,E2), and distance Obs(u D ,E3), the distance measuring unit 302 generates simultaneous equations in expression (17) including a calculation expression for the distance difference, as shown in Figure 33 .

[0402] [Mathematical Calculation 11]

[0403]

[0404]

[0405]

[0406] By solving the simultaneous equations of expression (17), the ranging unit 302 obtains a plurality of object positions L Dk (k = 1, 2, 3, or 4), and the corresponding pixel positions u D Since the positions of the light sources a to c are unknown from the camera coordinate system, expression (17) cannot be solved directly in terms of the camera coordinate system. However, since the positions of the light sources a to c in the light source coordinate system are already known, expression (17) can be solved in terms of the light source coordinate system.

[0407] When the simultaneous equations of expression (17) are solved in terms of the light source coordinate system, a plurality of object positions L DK and the corresponding pixel positions u D in the light source coordinate system can be obtained. Note that the plurality of object positions L DK include, for example, the respective positions E1, E2, and E3 of the light sources a to c and the surface positions on the object 301.

[0408] Note that, in general, if the correspondence between a plurality of points in a certain coordinate system and the pixel positions onto which the points are projected by a camera are known, the relative position and attitude between the coordinate system and the camera coordinate system can be obtained. That is, when a problem generally called a PnP problem is solved, the relative relationship between the light source coordinate system and the camera coordinate system can be derived, and the object positions in the light source coordinate system can be converted into object positions in the camera.

[0409] As a result of this processing, the object positions L Dk in the camera coordinate system and the corresponding pixel positions u D can be obtained. From the photographed pixel positions of the light sources a to c, the respective positions E1, E2, and E3 of the light sources a to c and the surface positions on the object 301 can be obtained in terms of the camera coordinate system. Since the surface positions on the object 301 are obtained in terms of the camera coordinate system, the distance to the object 301 can be calculated. That is, the distance from the ranging unit 302 to the object 301 can be calculated.

[0410] <9.Configuration of TOF sensor>

[0411] The configuration of the TOF sensor 52 will be described.

[0412] Figure 34 An example of the configuration of the light-receiving portion of the TOF sensor 52 is shown.

[0413] The TOF sensor 52 includes a pixel array section 232 in which pixels 231 that generate charges in accordance with light reception amounts and output signals corresponding to the charges are two-dimensionally arranged in a line direction and a row direction in a matrix form. The TOF sensor 52 also includes a drive control circuit 233 provided in a peripheral region of the pixel array section 232.

[0414] The drive control circuit 233 outputs control signals (for example, a distribution signal DIMIX, a selection signal ADDRESS DECODE, a reset signal RST, and the like, which will be described later) for controlling the driving of the pixels 231 based on a light emission pulse signal or the like.

[0415] Each pixel 231 includes a photodiode 251 and a first tap 252A and a second tap 252B that detect charges having undergone photoelectric conversion at the photodiode 251. In each pixel 231, the charges generated by one photodiode 251 are distributed to the first tap 252A or the second tap 252B. Thereafter, of the charges generated by the photodiode 251, the charges distributed to the first tap 252A are output as a detection signal A from a signal line 253A, and the charges distributed to the second tap 252B are output as a detection signal B from a signal line 253B.

[0416] The first tap 52A includes a transfer transistor 241A, an FD (floating diffusion) section 242A, a selection transistor 243A, and a reset transistor 244A. In a similar manner, the second tap 252B includes a transfer transistor 241B, an FD section 242B, a selection transistor 243B, and a reset transistor 244B.

[0417] The operation of each pixel 231 will be described.

[0418] As Figure 2 shown, irradiation light that has undergone modulation such that the on and off of irradiation is repeated at a frequency of 1 / f is output from the light source 2, and its reflected light is received by the photodiode 251 with a delay time AT corresponding to the distance from the object. In addition, the on and off of the transfer transistor 241A is controlled with the distribution signal DIMIX_A, and the on / off of the transfer transistor 241B is controlled with the distribution signal DIMIX_B. At the execution timing of the 0-degree phase, the distribution signal DIMIX_A is a signal of the same phase as the phase of the irradiation light (that is, the 0-degree phase), and the distribution signal DIMIX_B is a signal of the phase that is the inverse of the distribution signal DIMIX_A (that is, the 180-degree phase). At the execution timing of the 90-degree phase, the distribution signal DIMIX_A has a phase that is shifted by 90 degrees from the phase of the irradiation light, and the distribution signal DIMIX_B has a phase that is the inverse of the distribution signal DIMIX_A (that is, the 270-degree phase).

[0419] Thus, in Figure 34 when the transfer transistor 241A is turned on, the electric charge generated by the reflected light reflection at the photodiode 251 is transferred to the FD section 242A in accordance with the distribution signal DIMIX_A, and when the transfer transistor 241B is turned on, the electric charge is transferred to the FD section 242B in accordance with the distribution signal DIMIX_B. As a result, during a predetermined period in which the application of the irradiation light is cyclically performed at a frequency of 1 / f, the electric charge transferred via the transfer transistor 241A is sequentially accumulated in the FD section 242A, and the electric charge transferred via the transfer transistor 241B is sequentially accumulated in the FD section 242B.

[0420] After that, when the selection transistor 243A is turned on in accordance with the selection signal ADDRESS DECODE_A after the accumulation period of the accumulated electric charge, the electric charge accumulated in the FD section 242A is read out through the signal line 253A, and a detection signal A corresponding to the amount of the read electric charge is output from the pixel array section 232. In a similar manner, when the selection transistor 243B is turned on in accordance with the selection signal ADDRESS DECODE_B, the electric charge accumulated in the FD section 242B is read out through the signal line 253B, and a detection signal B corresponding to the amount of the read electric charge is output from the pixel array section 232. Further, when the reset transistor 244A is turned on in accordance with the reset signal RST_A, the electric charge accumulated in the FD section 242A is discharged, and when the reset transistor 244B is turned on in accordance with the reset signal RST_B, the electric charge accumulated in the FD section 242B is discharged.

[0421] In the above-described manner, in each pixel 231, the electric charge generated by the reflected light received by the photodiode 251 is distributed to the first tap 252A or the second tap 252B in accordance with the delay time ΔT, and the detection signal A and the detection signal B are output. The detection signal A and the detection signal B at the execution timing of the 0-degree phase correspond to the electric charge Q0 at the 0-degree phase and the electric charge Q Figure 3 indicated in the middle of the above-described 180 . The detection signal A and the detection signal B at the execution timing of the 90-degree phase correspond to the electric charge Q Figure 3 at the 90-degree phase and the electric charge Q 90 at the 270-degree phase in the middle of the above-described 270 .

[0422] <10. Application to mobile bodies>

[0423] The technology according to the present disclosure (prior art) can be applied to various products. For example, the technology according to the present disclosure can be implemented as a device mounted on any type of mobile body, such as a car, an electric car, a hybrid car, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, and a robot.

[0424] Figure 35 is a block diagram depicting an example of a schematic configuration of a vehicle control system that is an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied.

[0425] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a vehicle body system control unit 12020, an outside information detecting unit 12030, an inside information detecting unit 12040, and an integrated control unit 12050. Further, a microcomputer 12051, a sound / image output section 12052, and a vehicle-mounted network interface (I / F) 12053 are shown as a functional configuration of the integrated control unit 12050. Figure 35

[0426] The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle in accordance with various programs. For example, the drive system control unit 12010 functions as a drive force control device for generating a drive force of the vehicle, such as an internal combustion engine, a drive motor, or the like, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a brake device for generating a brake force of the vehicle, or the like.

[0427] The vehicle body system control unit 12020 controls the operation of various devices provided to the vehicle body in accordance with various programs. For example, the vehicle body system control unit 12020 functions as a control device of a keyless entry system, a smart key system, a power window device, or various lamps, such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device can be input to the vehicle body system control unit 12020 as a substitute for a key or a signal of various switches. The vehicle body system control unit 12020 receives these input radio waves or signals, and controls a door lock device, a power window device, a lamp, or the like of the vehicle.

[0428] The outside information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000. For example, the outside information detecting unit 12030 is connected with an imaging section 12031. The outside information detecting unit 12030 causes the imaging section 12031 to image an image of the outside of the vehicle, and receives the imaged image. On the basis of the received image, the outside information detecting unit 12030 can perform a process of detecting an object (such as a person, a vehicle, an obstacle, a sign, a road surface mark, or the like), or a process of detecting a distance thereto.

[0429] ​The imaging section 12031 is an optical sensor that receives light, and outputs an electric signal corresponding to the light quantity of the received light. The imaging section 12031 can output the electric signal as an image, or can output the electric signal as information on a measured distance. Further, the light received by the imaging section 12031 can be visible light, or can be non-visible light (for example, infrared rays, etc.).

[0430] The in-vehicle information detection unit 12040 detects information on the inside of the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver state detection section 12041 that detects the state of the driver. The driver state detection section 12041 includes, for example, a camera that images the driver. On the basis of detection information input from the driver state detection section 12041, the in-vehicle information detection unit 12040 can calculate the degree of fatigue of the driver or the degree of attention of the driver, or can determine whether the driver is drowsy.

[0431] The microcomputer 12051 calculates a control target value of a driving force generation device, a steering mechanism, or a braking device on the basis of information obtained by the outside-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040, regarding the inside or outside of the vehicle, and outputs a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control for realizing functions of an advanced driver assistance system (ADAS) including collision avoidance or mitigation of the vehicle, follow-up driving based on a following distance, vehicle speed maintenance driving, vehicle collision warning, warning of vehicle departure from a lane, and the like.

[0432] Further, the microcomputer 12051 can perform cooperative control for automatic driving, further causing the vehicle to travel automatically without depending on the operation of the driver by controlling the driving force generation device, the steering mechanism, the braking device, or the like on the basis of information obtained by the outside-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040, regarding the outside of the vehicle or the inside of the vehicle.

[0433] Further, the microcomputer 12051 can output a control command to the body system control unit 12020 on the basis of information obtained by the outside-vehicle information detection unit 12030, regarding the outside of the vehicle. For example, the microcomputer 12051 can perform cooperative control to prevent glare by controlling the headlamp, so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030.

[0434] The sound / image output section 12052 transmits an output signal of at least one of sound and image to an output device capable of visually or aurally notifying information to an occupant of the vehicle or outside of the vehicle. In Figure 35In the example in FIG. 12A, the audio speaker 12061, the display portion 12062, and the instrument panel 12063 are illustrated as output devices. The display portion 12062 may, for example, include at least one of a vehicle-mounted display and a head-up display.

[0435] Figure 36 FIG. 12B is a diagram depicting an example of mounting positions of the imaging portions 12031.

[0436] In Figure 36 The imaging portions 12031 include imaging portions 12101, 12102, 12103, 12104, and 12105.

[0437] The imaging portions 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions of a front nose, side mirrors, a rear bumper, and a rear door of the vehicle 12100, and an inner windshield. The imaging portion 12101 provided to the front nose and the imaging portion 12105 provided to an upper portion of the inner windshield mainly obtain images of a front portion of the vehicle 12100. The imaging portions 12102 and 12103 provided to the side mirrors mainly obtain images of side portions of the vehicle 12100. The imaging portion 12104 provided to the rear bumper or the rear door mainly obtains images of a rear portion of the vehicle 12100. The imaging portion 12105 provided to the upper portion of the inner windshield is mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, and the like.

[0438] Incidentally, Figure 36 Examples of imaging ranges of the imaging portions 12101 to 12104 are depicted. An imaging range 12111 represents an imaging range of the imaging portion 12101 provided to the front nose. Imaging ranges 12112 and 12113 represent imaging ranges of the imaging portions 12102 and 12103 provided to the side mirrors, respectively. An imaging range 12114 represents an imaging range of the imaging portion 12104 provided to the rear bumper or the rear door. For example, an aerial view of the vehicle 12100 is obtained by, for example, superimposing image data imaged by the imaging portions 12101 to 12104.

[0439] At least one of the imaging portions 12101 to 12104 can have a function of acquiring distance information. For example, at least one of the imaging portions 12101 to 12104 can be a stereo camera constituted by a plurality of imaging elements, or can be an imaging element having pixels for phase difference detection.

[0440] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the time change (relative speed with respect to the vehicle 12100) in the distance based on the distance information obtained from the imaging sections 12101 to 12104, thereby extracting, as the preceding vehicle, the three-dimensional object existing closest on the travel path of the vehicle 12100 and traveling at substantially the same speed as the vehicle 12100 of the predetermined speed (for example, equal to or greater than 0 km / hour). Further, the microcomputer 12051 can set a following distance with respect to the preceding vehicle in advance, and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), and the like. Thus, cooperative control for automatic driving can be performed to cause the vehicle to automatically travel without depending on the operation of the driver or the like.

[0441] For example, the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of two-wheeled vehicles, standard-size vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging sections 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data to automatically avoid obstacles. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that can be visually identified by the driver of the vehicle 12100 and obstacles that are difficult for the driver of the vehicle 12100 to visually identify. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than a set value and thus a collision is possible, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062, and performs forced deceleration or avoidance steering by the drive system control unit 12010. Thus, the microcomputer 12051 can assist driving to avoid a collision.

[0442] At least one of the imaging sections 12101 to 12104 can be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the imaged image of the imaging section 12101 to 12104. For example, such recognition of a pedestrian is performed by, for example, extracting feature points in the imaged image of the imaging section 12101 to 12104 that is an infrared camera, and determining whether it is a pedestrian by performing a pattern matching process on a series of feature points that represent the outline of the object. When the microcomputer 12051 determines that a pedestrian is present in the imaged image of the imaging section 12101 to 12104 and thus recognizes the pedestrian, the sound / image output section 12052 controls the display section 12062 so as to display a square outline for emphasis so as to be superimposed on the recognized pedestrian. The sound / image output section 12052 can also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at the expected position.

[0443] One example of a vehicle control system according to the technology of the present disclosure has been explained so far. The technology according to the present disclosure is applicable to the above-described configuration of the imaging section 12031. Specifically, the above-described distance measuring device 21 can be used as the imaging section 12031. When the technology according to the present disclosure is applied to the imaging section 12031, distance information can be acquired. In addition, the acquired distance information can be used to enhance the safety of the driver and the vehicle.

[0444] Embodiments of the present technology are not limited to the above-described type. Various changes can be made within the spirit of the present technology.

[0445] The aspects of the present technology explained in this specification can be implemented independently as long as there is no inconsistency. It goes without saying that freely selected aspects of the present technology can be implemented in combination. For example, part or all of the present technology explained in any one embodiment can be implemented in combination with the present technology explained in another embodiment. In addition, part or all of the freely selected present technology can be implemented in combination with another technology that has not been explained so far.

[0446] In addition, for example, a configuration explained as one device (or one processing section) can be divided into a plurality of devices (or a plurality of processing sections). Conversely, a configuration explained above as a plurality of devices (or a plurality of processing sections) can be integrated into one device (or one processing section). Furthermore, it goes without saying that a configuration other than the above-described configuration can be added to the configuration of each device (or each processing section). Furthermore, part of the configuration including a certain device (or a certain processing section) can be included in the configuration of another device (or another processing section) as long as the configuration or operation of the entire system does not change.

[0447] Furthermore, the term "system" in this specification refers to a group of a plurality of components (devices, modules (components), and the like), regardless of whether all the components are included in the same housing or not. Thus, a group of a plurality of devices housed in separate housings and connected through a network is a system, and a single device having a plurality of modules housed in a single housing is also a system.

[0448] Note that the effects described in this specification are merely illustrative and non-limiting. An effect other than those described in this specification can be provided.

[0449] Note that the present technology can have the following configurations.

[0450] (1) A ranging device including:

[0451] a generation section that generates a signal synchronized with a synchronization clock signal of a separate device by performing synchronization processing on a transmission signal transmitted from the separate device through wireless communication;

[0452] a reference clock generation section that generates a reference clock signal to be used as a reference of light emission from a light source based on the synchronization clock signal; and

[0453] a ToF sensor that receives reflected light generated from irradiation light applied from the light source and reflected by an object based on the reference clock signal.

[0454] (2) The ranging device according to (1), in which

[0455] the reception section receives light source information about the irradiation light applied from the light source through the wireless communication.

[0456] (3) The ranging device according to (2), in which

[0457] the light source information includes calibration data about the light source.

[0458] (4) The ranging device according to (2) or (3), in which

[0459] the light source information includes an operation mode.

[0460] (5) The ranging device according to (4), in which

[0461] the operation mode includes information indicating any one of a time division mode, a modulation frequency mode, or a light emission pattern mode.

[0462] (6) The ranging device according to any one of (2) to (5), in which

[0463] The light source information includes any of a light emission time, a light emission start time information, a modulation frequency, a light source wavelength, or a light emission pattern.

[0464] (7) The distance measuring device according to any one of (1) to (6), wherein

[0465] The receiving section receives a transmission signal transmitted from the separate device through wireless communication via a network.

[0466] (8) The distance measuring device according to any one of (1) to (6), wherein

[0467] The receiving section receives a GNSS signal as the transmission signal.

[0468] (9) The distance measuring device according to any one of (1) to (8), further comprising:

[0469] A signal processing section calculates a distance to the separate device based on a detection result obtained by the TOF sensor.

[0470] (10) The distance measuring device according to (9), further comprising:

[0471] A communication section defines a distance to the separate device as a first distance and transmits the first distance to the separate device, and further receives a distance calculated by the separate device as a second distance.

[0472] (11) The distance measuring device according to (10), further comprising:

[0473] A distance calculation section calculates a relative distance with respect to the separate device based on the first distance and the second distance.

[0474] (12) The distance measuring device according to any one of (1) to (11), wherein

[0475] The TOF sensor receives reflected light generated from irradiation light applied from a plurality of the light sources and reflected by the object.

[0476] (13) The distance measuring device according to (12), further comprising:

[0477] A signal processing section calculates a distance to the object based on a phase difference of the irradiation light applied from a plurality of the light sources.

[0478] (14) A distance measuring device comprising:

[0479] A communication section transmits a transmission signal through wireless communication;

[0480] a reference clock generation section that generates a reference clock signal to be used as a reference for light emission from a light source, based on a synchronization clock signal that is synchronized with the transmission signal; and

[0481] a light source that applies irradiation light based on the reference clock signal.

[0482] (15) A ranging device, comprising:

[0483] a communication section that transmits and receives a transmission signal through wireless communication;

[0484] a reference clock generation section that generates a reference clock signal to be used as a reference for light emission from a light source, based on a synchronization clock signal that is synchronized with the transmission signal;

[0485] a light source that applies irradiation based on the reference clock signal; and

[0486] a ToF sensor that receives reflected light produced from irradiation light applied from a separate device and reflected by an object, based on the reference clock signal.

[0487] (16) A light reception method for a ranging device, the method comprising:

[0488] by the ranging device,

[0489] generating a signal that is synchronized with a separate synchronization clock signal by performing synchronization on a transmission signal transmitted from the separate device through wireless communication;

[0490] generating a reference clock signal to be used as a reference for light emission from a light source, based on the synchronization clock signal; and

[0491] receiving reflected light produced from irradiation light applied from the light source and reflected by an object, based on the reference clock signal.

[0492] [List of Reference Signs]

[0493] 1: Ranging system

[0494] 2: Light source

[0495] 3: Ranging unit

[0496] 11: Ranging system

[0497] 21 (21', 21A to 21E): Ranging device

[0498] 23: Network

[0499] 41: Clock source

[0500] 42: communication module

[0501] 44: reference clock generation section

[0502] 45: operation mode setting section

[0503] 47: laser emission section

[0504] 48: light source information acquisition section

[0505] 49: memory

[0506] 50: time synchronization section

[0507] 51: timing control section

[0508] 52: ToF sensor

[0509] 53: signal processing section

[0510] 61: synchronized clock generation section

[0511] 62: signal transmission / reception section

[0512] 81: GNSS reception section

[0513] 82: time information setting section

[0514] 101 (101A, 101B): drone

[0515] 122: position / attitude calculation section

[0516] 123: communication section

[0517] 124: inter-light source distance calculation section

[0518] 141: depth image generation section

[0519] 142: external modulated light source detection section

Claims

1. A distance measuring device comprising: a reception section that generates a signal synchronized with a synchronization clock signal of a separate device by performing synchronization processing on a transmission signal transmitted from the separate device through wireless communication; a reference clock generation section that generates a reference clock signal to be used as a reference for light emission of a light source based on the synchronization clock signal; and a ToF sensor that receives reflected light generated from irradiation light applied from the light source and reflected by an object based on the reference clock signal, wherein the reception section receives light source information on the irradiation light applied from the light source through the wireless communication, the distance measuring device further comprising: a signal processing section that calculates a distance to the separate device based on a detection result obtained by the ToF sensor; a communication section that defines the distance to the separate device as a first distance, and transmits the first distance to the separate device, and further receives a distance calculated by the separate device as a second distance; a distance calculation section that calculates a relative distance to the separate device based on the first distance and the second distance.

2. The distance measuring device according to claim 1, wherein the light source information includes calibration data on the light source.

3. The distance measuring device according to claim 1, wherein the light source information includes an operation mode.

4. The distance measuring device according to claim 3, wherein the operation mode includes information indicating any one of a time division mode, a modulation frequency mode, a light emission pattern mode.

5. The distance measuring device according to claim 1, wherein the light source information includes any one of a light emission time, a light emission start time information, a modulation frequency, a light source wavelength, a light emission pattern.

6. The distance measuring device according to claim 1, wherein the reception section receives a transmission signal transmitted from the separate device through wireless communication via a network.

7. The distance measuring device according to claim 1, wherein the reception section receives a GNSS signal as the transmission signal.

8. The distance measuring device according to claim 1, wherein the ToF sensor receives reflected light generated from irradiation light applied from a plurality of the light sources and reflected by the object.

9. The distance measuring device according to claim 8, further comprising: a signal processing section that calculates a distance to the object based on a phase difference of irradiation light applied from a plurality of the light sources.

10. A distance measuring device comprising: a communication section that transmits and receives a transmission signal through wireless communication; a reference clock generation section that generates a reference clock signal to be used as a reference for light emission from a light source based on a synchronization clock signal synchronized with the transmission signal; a light source that applies irradiation light based on the reference clock signal; and a ToF sensor that receives reflected light generated from irradiation light applied from a separate device and reflected by an object based on the reference clock signal, ​ The communication section is configured to transmit, through the wireless communication, light source information about the irradiation light applied from the light source, The ranging device further includes: a signal processing section that calculates a distance to the separate device based on a detection result obtained by the ToF sensor; wherein the communication section defines the distance to the separate device as a first distance, and transmits the first distance to the separate device, and further, the communication section receives a distance calculated by the separate device as a second distance, The ranging device further includes: a distance calculation section that calculates a relative distance to the separate device based on the first distance and the second distance.

Images