Measurement apparatus, control program for measurement system, control method for measurement system, and recording medium

Abstract

To improve a measuring device such as LiDAR.SOLUTION: A measurement device includes a light projector that emits light, a first light receiver that receives light, and a controller, in which the controller causes the light projector to execute a light projecting operation of emitting light, causes the first light receiver to execute a first light receiving operation of receiving reflected light that is emitted light from the light projector, reflected by a measurement target, and returned, acquires first three dimensional data, and further outputs first output data. Acquiring first two dimensional data by causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light projector to perform the light projecting operation; Acquiring second two dimensional data by causing a second optical receiver disposed at a position different from a position of the first optical receiver to execute a third light receiving operation of receiving external light; Second three dimensional data is generated based on the first two dimensional data and the second two dimensional data, and second output data is output.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The technology disclosed in this specification relates to a measurement device, a control program for a measurement system, a control method for a measurement system, and a recording medium. [Background technology]

[0002] With the advancement of autonomous driving (AD) systems and advanced driver assistance systems (ADAS), research and development of LiDAR (light detection and ranging) is underway as one of the measurement devices used to grasp the surrounding environment and estimate the vehicle's own position while driving. LiDAR includes a projector that projects laser light onto a measurement target, a photoreceiver that receives the light reflected from the measurement target, and a controller. The controller causes the projector to perform a light projecting operation of emitting light, causes the photoreceiver to perform a light receiving operation of receiving the light reflected from the measurement target, and outputs three-dimensional data acquired by the light receiving operation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2018-535402 Summary of the Invention [Problem to be solved by the invention]

[0004] The present inventors have discovered that there is room for further utilization of measurement devices such as LiDAR.

[0005] This specification discloses a technique that can solve the above-mentioned problems. [Means for solving the problem]

[0006] The technology disclosed in this specification can be realized, for example, in the following forms.

[0007] (1) A measurement device disclosed in this specification includes a light projector that emits light, a first light receiver that receives light, and a controller. The controller causes the light projector to perform a light projecting operation that emits light and causes the first light receiver to perform a first light receiving operation that receives reflected light from the light projector that is reflected off a measurement object. The controller also causes the light projector not to perform the light projecting operation but to perform a second light receiving operation that causes the first light receiver to receive external light. The controller outputs first output data based on first three-dimensional data acquired by the first light receiving operation and outputs second output data based on first two-dimensional data acquired by the second light receiving operation. This measurement device can output first output data based on first three-dimensional data of the measurement object by performing the light projecting operation and the first light receiving operation. Also, it can output second output data based on first two-dimensional data of the measurement object by performing the second light receiving operation.

[0008] (2) The above measuring device may further include an acquisition unit that executes a third light receiving operation to cause a second light receiver disposed at a different position from the first light receiver to receive external light, and acquires second two-dimensional data acquired by the third light receiving operation, wherein the controller acquires the second two-dimensional data from the second light receiver via the acquisition unit, generates second three-dimensional data based on the first two-dimensional data from the first light receiver and the second two-dimensional data from the second light receiver, and outputs the second output data based on the second three-dimensional data. With this configuration, the second output data based on the second three-dimensional data of the measurement object can be output without executing the light projecting operation.

[0009] (3) In the above measurement device, the light projector may emit light in a first wavelength band, the first light receiver may receive light in the first wavelength band and light in a second wavelength band different from the first wavelength band, and the second light receiver may receive light in the second wavelength band. With this configuration, first output data based on first three-dimensional data of the object to be measured can be output by performing a light projecting operation and a first light receiving operation using light in the first wavelength band, and second output data based on second three-dimensional data of the object to be measured can be output by performing a second light receiving operation and a third light receiving operation using light in the second wavelength band.

[0010] (4) In the above measuring device, the controller may be configured so that, when a first condition is satisfied, the controller causes the transmitter to execute the light-projecting operation and the first receiver to execute the first light-receiving operation, and, when the first condition is not satisfied, the controller does not cause the transmitter to execute the light-projecting operation but causes the first receiver to execute the second light-receiving operation. With this configuration, the controller can measure the object to be measured with minimal power by executing the light-projecting operation and the first light-receiving operation when the condition is satisfied and executing the second light-receiving operation when the condition is not satisfied.

[0011] The technology disclosed in this specification can be realized in various forms, for example, in the form of a measuring device, a second photodetector, a control method, a control program, or a computer-readable recording medium on which the control program is recorded. [Brief explanation of the drawings]

[0012] [Figure 1] Schematic diagram of an entire vehicle equipped with a measurement system according to the first embodiment [Figure 2] FIG. 1 is a block diagram showing a schematic configuration of a measurement system according to a first embodiment; [Figure 3] 1 is an explanatory diagram of the principle of a distance measurement method using a first light receiver and an imaging device; [Figure 4] FIG. 1 is an explanatory diagram of the parallax between a first two-dimensional image and a second two-dimensional image; [Figure 5] 1 is a flowchart showing the flow of measurement processing when the first condition is satisfied. [Figure 6] 10 is a flowchart showing the flow of measurement processing when the first condition is not satisfied. [Figure 7] Flowchart showing the flow of first output data generation processing [Figure 8] 10 is a flowchart showing the flow of a second output data generation process. [Figure 9] 10 is a flowchart showing the flow of a second two-dimensional data generation process. [Figure 10] 10 is a flowchart showing the flow of a third three-dimensional data generation process in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] A. First embodiment: The first embodiment will be described with reference to Fig. 1 to Fig. 9. Fig. 1 shows X, Y, and Z axes that are orthogonal to each other and are used to specify directions. The positive X-axis direction is the forward direction, and the negative X-axis direction is the backward direction. The positive Y-axis direction is the left direction, and the negative Y-axis direction is the right direction. The positive Z-axis direction is the upward direction, and the negative Z-axis direction is the downward direction.

[0014] FIG. 1 is a perspective view of a vehicle 1 equipped with a measuring device 10 and an imaging device 40 (hereinafter, sometimes collectively referred to as a "measurement system 50") according to the first embodiment. The vehicle 1 is equipped with a pair of headlamps 70. The pair of headlamps 70 are respectively arranged on both the left and right sides of the front end of the vehicle 1. Each headlamp 70 is equipped with a low beam lamp unit 61, a high beam lamp unit 62, an imaging device 40, and a measuring device 10. The low beam lamp unit 61, the high beam lamp unit 62, the imaging device 40, and the measuring device 10 are lined up in this order from the center of the vehicle toward the side.

[0015] A-1. Configuration of the measuring device 10: The measuring device 10 of the first embodiment is a FLASH-type LiDAR. The measuring device 10 is mounted on, for example, a vehicle equipped with an AD (automated drive) or an ADAS (advanced driver assistance system). For example, the measuring device 10 assists in detecting objects such as people and other vehicles while the vehicle is traveling, and provides various types of information to other devices and users that are useful for ensuring the safety of the driver of the vehicle and those around the vehicle, and for reducing damage to surrounding objects while the vehicle is being driven.

[0016] As shown in FIGS. 1 and 2, the measurement device 10 includes a light projector 100, a first light receiver 300, an information processing device 600, and a first communication interface (I / F) 800.

[0017] (Floodlight) The projector 100 includes a light projection optical system 110, a light source unit 120, a current source 130, and a first control circuit board 210.

[0018] The light source unit 120 has a plurality of light-emitting elements (not shown) or a plurality of light-emitting element arrays (for example, light-emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). Each light-emitting element is, for example, a laser diode, a surface-emitting type laser light-emitting element (for example, a VCSEL (Vertical Cavity Surface Emitting Laser), hereinafter referred to as a "surface-emitting element"), a surface-emitting element array (for example, a VCSEL array) in which a plurality of surface-emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (semiconductor substrate, ceramic substrate, etc.), etc.

[0019] The light projection optical system 110 adjusts the light distribution of the output light Lout by applying optical effects (such as refraction, scattering, and diffraction) to light (e.g., a light beam (laser light)) emitted by each light-emitting element of the light source unit 120 (hereinafter referred to as "output light Lout"). The light projection optical system 110 is configured using optical components such as various lenses, such as collimating lenses, and reflectors (mirrors). The light projection optical system 110 is disposed on the optical path of the output light Lout output from each light-emitting element of the light source unit 120. In other words, the light projection optical system 110 is disposed so as to face each light-emitting element in the optical axis direction of each light-emitting element. The light projection optical system 110 is disposed at a distance from each light-emitting element in the optical axis direction.

[0020] The current source 130 supplies current to each light-emitting element that constitutes the light source unit 120. The current source 130 supplies, for example, to each light-emitting element, a periodic pulse wave current for turning on and off the current flowing through each light-emitting element.

[0021] The first control circuit board 210 is a circuit board on which electronic components and the like are mounted for controlling the light emission of each light-emitting element of the light source section 120. The first control circuit board 210 is communicably connected to the current source 130, a TOF measuring device 340 of the first light receiver 300 (described later), and an information processing device 600.

[0022] (First receiver) As shown in FIG. 2, the first light receiver 300 includes a first light receiving optical system 310, a first light receiving unit 320, and a TOF measurement device 340.

[0023] The first light receiving optical system 310 is an optical system for causing the first light receiving unit 320 to receive incident light. The first light receiving optical system 310 may be, for example, various lenses such as a condenser lens, various filters such as a wavelength filter, or a reflecting mirror. The first light receiving optical system 310 is disposed on the optical path of incident light input to each light receiving element of the first light receiving unit 320. In other words, the first light receiving optical system 310 is disposed so as to face each light receiving element in the optical axis direction of each light receiving element.

[0024] The first light receiving unit 320 includes a plurality of light receiving elements. Each light receiving element is, for example, a photodiode, an avalanche photodiode, or a single-photon avalanche diode (SPAD). The first light receiving unit 320 photoelectrically converts the incident light from the first light receiving optical system 310 to generate a light receiving signal with a current level or voltage level corresponding to the intensity of the incident light.

[0025] The TOF measurement device 340 has, for example, a time measurement IC (integrated circuit) equipped with a TDC (time-to-digital converter) circuit. The TOF measurement device 340 is communicably connected between the first control circuit board 210 and the information processing device 600. Details of the TOF measurement device 340 will be described later.

[0026] (information processing devices, etc.) The information processing device 600 has a processor. The processor may be, for example, a CPU (central processing unit), an MPU (micro processing unit), an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a DSP (digital signal processor), or the like. The information processing device 600 is communicatively connected to the first control circuit board 210, the TOF measurement device 340, and the first communication I / F 800. The information processing device 600 is an example of a controller. Details of the information processing device 600 will be described later.

[0027] The first communication I / F 800 is communicatively connected to the information processing device 600, a second communication I / F 470 of the imaging device 40 (described later), and an external device 90. The first communication I / F 800 transmits information processed by the information processing device 600 to the external device 90. The first communication I / F 800 also receives information obtained by the imaging device 40 and transmits it to the information processing device 600. The first communication I / F 800 is an example of an acquisition unit. Communication between the first communication I / F 800 and the second communication I / F 470 or the external device 90 may be wired or wireless.

[0028] The external device 90 may be, for example, a device that creates an environmental map using a point cloud, or a device that performs self-position estimation (SLAM: Simultaneous Localization and Mapping) using a scan matching algorithm such as NDT (Normal Distributions Transform) or ICP (Iterative Closest Point).

[0029] A-2. Configuration of imaging device 40: 2, the imaging device 40 includes a second light receiving optical system 410, a second light receiving unit 420, a two-dimensional data generator 450, a second control circuit board 460, and a second communication I / F 470. The imaging direction of the imaging device 40, the light projector 100 of the measuring device 10, and the first light receiver 300 of the measuring device 10 is the same (the positive X-axis direction in the case of FIG. 1). The imaging device 40 is an example of a second light receiver.

[0030] The second light receiving optical system 410 is an optical system for causing the second light receiving unit 420 to receive incident light. The second light receiving optical system 410 may be, for example, various lenses such as a condenser lens, various filters such as a wavelength filter, or a reflecting mirror. The second light receiving optical system 410 is disposed on the optical path of the incident light input to each light receiving element of the second light receiving unit 420. In other words, the second light receiving optical system 410 is disposed so as to face each light receiving element in the optical axis direction of each light receiving element.

[0031] The second light receiving unit 420 includes a light receiving element. The light receiving element is, for example, a photodiode, an avalanche photodiode, or a single-photon avalanche diode (SPAD). The second light receiving unit 420 also includes an image sensor. Examples of the image sensor include a CCD (Change Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The second light receiving unit 420 photoelectrically converts the incident light from the second light receiving optical system 410 to generate a light receiving signal with a current level or voltage level corresponding to the incident light.

[0032] The two-dimensional data generator 450 includes, for example, an image processor, a DRAM, etc. The two-dimensional data generator 450 is communicably connected between the second light receiving unit 420 and the second control circuit board 460. The two-dimensional data generator 450 receives the light receiving signal output from the second light receiving unit 420 and generates image data.

[0033] A-3. Lighting period etc.: (Light projection operation) The information processing device 600 causes the projector 100 to repeatedly execute a light-projection operation at predetermined intervals. The light-projection operation is an operation in which the first control circuit board 210 controls the current source 130 to sequentially cause the multiple light-emitting elements of the light source unit 120 to emit light. Hereinafter, a period during which the projector 100 executes a light-projection operation will be referred to as a "light-projection period," and a period during which the projector 100 does not execute a light-projection operation (for example, a period from the end of one light-projection operation to the start of the next light-projection operation) will be referred to as a "non-light-projection period." Furthermore, a signal indicating the timing at which each light-emitting element emits the output light Lout (hereinafter referred to as "light-projection timing") will be referred to as a "light-projection timing signal."

[0034] The emitted light Lout is light in a first wavelength band, which is a certain wavelength band. Specifically, the first wavelength band is infrared light. The first wavelength band is, for example, 780 nm to 2000 nm, or 850 nm to 950 nm.

[0035] (First light receiving operation) The first light receiving unit 320 outputs a light receiving signal (hereinafter referred to as a "first light receiving signal") corresponding to the amount of light received by each of the multiple light receiving elements during a first light receiving period (hereinafter referred to as a "first light receiving operation"). The first light receiving period is a period corresponding to the light projection period, during which each light receiving element can receive reflected light Lre that is returned by reflecting output light Lout emitted by each light emitting element during the light projection operation off the measurement object W (see FIG. 2). Furthermore, a signal indicating the timing at which each light receiving element receives reflected light Lre (hereinafter referred to as a "light receiving timing") is referred to as a "light receiving timing signal."

[0036] (Second light receiving operation) The first light receiving unit 320 outputs a light receiving signal (hereinafter referred to as a "second light receiving signal") corresponding to the amount of light received by each of the plurality of light receiving elements during the second light receiving period (hereinafter referred to as a "second light receiving operation"). The second light receiving period is a period corresponding to the non-light emitting period, during which each light receiving element cannot receive reflected light Lre that is returned by reflecting the emitted light Lout emitted by each light emitting element during the light emitting operation onto the measurement object W.

[0037] The light received by the first light receiving unit 320 during the second light receiving period (hereinafter referred to as "first ambient light Ln1") is light in a second wavelength band, which is a certain wavelength band. Specifically, the second wavelength band is visible light. Specifically, the second wavelength band is 360 nm to 830 nm, or 380 nm to 780 nm. The first ambient light Ln1 includes light from sunlight, street lights, etc. (hereinafter referred to as "natural light"), and light that is reflected back from the measurement object W from the natural light.

[0038] (Generation of signals corresponding to TOF) The TOF measuring device 340 receives a light projection timing signal from the first control circuit board 210 during the light projection period, receives a first light reception signal and a light reception timing signal from the first light receiving unit 320 during the first light reception period, and receives a second light reception signal from the first light receiving unit 320 during the second light reception period.

[0039] The TOF measurement device 340 calculates the difference between the light projection timing and the light reception timing, i.e., the time of flight (TOF) of the laser light, based on the light projection timing signal and the light reception timing signal. The TOF measurement device 340 outputs a signal corresponding to the calculated TOF, the first light reception signal, and the second light reception signal to the information processing device 600.

[0040] (Third light receiving operation) The second light receiving unit 420 of the imaging device 40 outputs a light receiving signal (hereinafter referred to as a "third light receiving signal") corresponding to the amount of light received by each of the multiple light receiving elements (hereinafter referred to as a "third light receiving operation"). Each light receiving element of the second light receiving unit 420 receives external light (hereinafter referred to as "second ambient light Ln2"). The second ambient light Ln2 is light of a second wavelength band, which is the same fixed wavelength band as the first ambient light Ln1. After the third light receiving operation, the second light receiving unit 420 inputs the third light receiving signal to the two-dimensional data generator 450.

[0041] The two-dimensional data generator 450 generates second two-dimensional data D22 based on the third received light signal. The second two-dimensional data D22 is transmitted to the information processing device 600 of the measuring device 10 via the second control circuit board 460 and the second communication I / F 470.

[0042] A-4. Measurement process of the first embodiment: 5 to 9 are flowcharts showing the flow of measurement processing by the measurement system 50. When the measurement system 50 starts the measurement processing as shown in Fig. 5, the information processing device 600 (an example of a controller) determines whether a first condition is satisfied (S110).

[0043] The first condition is as follows: Note that the following is an example. The moving speed of the measurement system 50 (car 1) is above the standard. The distance between the measurement system 50 (car 1) and the measurement target W is equal to or greater than the standard. The brightness of the environment around the measurement system 50 (car 1) is below the standard. The battery charge level in car 1 is above the standard.

[0044] When the information processing device 600 determines that the first condition is satisfied (S110: Yes), the information processing device 600 activates the projector 100 and causes the projector 100 to start a light projection operation (S120). The information processing device 600 generates first three-dimensional data D31 (S30), as shown in FIG.

[0045] The information processing device 600 causes each light-emitting element of the light source unit 120 to perform a light projection operation via the first control circuit board 210 (S310). The first light-receiving unit 320 performs a first light-receiving operation (S320). The TOF measurement device 340 generates a signal corresponding to the TOF from the light-projection timing signal and the light-receiving timing signal (S360). The information processing device 600 receives the signal corresponding to the TOF and the first light-receiving signal, and generates first three-dimensional data D31 based on these (S370). The first three-dimensional data D31 may be, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, point cloud information, etc.

[0046] As shown in FIG. 5, the information processing device 600 generates first output data D41 from the first three-dimensional data D31 (S130). The information processing device 600 transmits the first output data D41 to the first communication I / F 800 (S140). If the information processing device 600 determines that it has received an instruction to end measurement (S150: Yes), it ends this measurement process. If the information processing device 600 determines that it has not received an instruction to end measurement, it proceeds to S160. The information processing device 600 determines whether the first condition is satisfied (S160). If the information processing device 600 determines that the first condition is satisfied (S160: Yes), it returns to S30. If the information processing device 600 determines that the first condition is not satisfied (S160: No), it stops the projector 100 and causes the projector 100 to stop projecting (S170). Then, it proceeds to S40 in FIG. 6.

[0047] As shown in Fig. 5, when the measurement system 50 starts the measurement process, if it determines that the first condition is not satisfied (S110: No), the information processing device 600 proceeds to S40 in Fig. 6. The information processing device 600 generates second three-dimensional data D32 (S40), as shown in Figs. 8 and 9.

[0048] The first light receiving section 320 of the measurement device 10 performs a second light receiving operation (S420) as shown in Fig. 8. The information processing device 600 generates first two-dimensional data D21 from the second light receiving signal (S430).

[0049] 9, the second light receiving section 420 of the imaging device 40 performs a third light receiving operation (S520). The two-dimensional data generator 450 generates second two-dimensional data D22 from the third light receiving signal (S530). The second two-dimensional data D22 is transmitted from the two-dimensional data generator 450 to the second communication I / F 470 via the second control circuit board 460 (S540). The second two-dimensional data D22 transmitted to the second communication I / F 470 is transmitted to the first communication I / F 800 of the measuring device 10 (S550).

[0050] 8, the information processing device 600 acquires the second two-dimensional data D22 from the second communication I / F 470 via the first communication I / F 800 (S450). The information processing device 600 generates the second three-dimensional data D32 from the first two-dimensional data D21 and the second two-dimensional data D22 (S470).

[0051] Information about the distance C2 (depth) between the measurement system 50 and the measurement target W is calculated from the first two-dimensional data D21 and the second two-dimensional data D22 according to the principle of triangulation, as shown in Figures 3 and 4. This generates second three-dimensional data D32.

[0052] As shown in FIG. 3 , the first light receiver 300 of the measurement device 10 and the imaging device 40, which is the second light receiver, have the same imaging direction with respect to the measurement object W. The first light receiver 300 receives first ambient light Ln1 at point A on the measurement object W at a first light receiving unit 320 via a first light receiving optical system 310. The intersection of a line connecting point A and a center F310 of the first light receiving optical system 310 with a first light receiving surface R320 of the first light receiving unit 320 is a first focal point P1 for point A. Meanwhile, the imaging device 40 receives second ambient light Ln2 at point A on the measurement object W at a second light receiving unit 420 via a second light receiving optical system 410. The second focal point Q for point A is the point where a straight line connecting point A and the center F410 of the second light receiving optical system 410 intersects with the second light receiving surface R420 of the second light receiving section 420.

[0053] FIG. 4 is a conceptual diagram of a first two-dimensional image I21 of the first two-dimensional data D21 of the measuring device 10 and a second two-dimensional image I22 of the second two-dimensional data D22 of the imaging device 40. The measurement object W in the first two-dimensional image I21 is imaged to the right of the measurement object W in the second two-dimensional image I22. That is, the first focus P1 and the second focus Q are imaged at different positions. The first focus P1 and the second focus Q have a difference in length of parallax B1. When parallax B1 is shown in FIG. 3, the first focus P1 corresponds to a corresponding point P2 on the second light-receiving surface R420 of the second light-receiving unit. Specifically, the corresponding point P2 is the intersection of a line Ln12 drawn through the center F410 of the second light-receiving optical system 410 and parallel to the first ambient light Ln1 with the second light-receiving surface R420.

[0054] Here, by triangulation, the following relationship holds among the parallax B1, the inter-camera distance B2 between the center F310 and the center F410, the focal length C1 between the center F310 and the first light receiving surface R320 (or the center F410 and the second light receiving surface R420), and the distance C2 between point A and the measuring device 10 (or the imaging device 40): <Formula> C2=C1×B2 / B1 As described above, the measurement device 10 and the imaging device 40 can measure the distance C2 (depth) between the measurement system 50 and the measurement object W. According to this theory, the second three-dimensional data D32 is created by calculating the distance C2 (depth) between the measurement system 50 and the measurement object W for each pixel from the parallax B1.

[0055] As shown in FIG. 6, the information processing device 600 generates second output data D42 from the second three-dimensional data D32 (S230). The information processing device 600 transmits the second output data D42 to the first communication I / F 800 (S240). If the information processing device 600 determines that it has received an instruction to end measurement (S250: Yes), it ends this measurement process. If the information processing device 600 determines that it has not received an instruction to end measurement, it proceeds to S260. The information processing device 600 determines whether the first condition is satisfied (S260). If the information processing device 600 determines that the first condition is not satisfied (S260: No), it returns to S40. If the information processing device 600 determines that the first condition is satisfied (S260: Yes), it proceeds to S120 of FIG. 5.

[0056] A-5. Advantages of the first embodiment: According to the first embodiment, the measurement device 10 includes a projector 100 that emits light, a first light receiver 300 that receives the light, and an information processing device 600 that serves as a controller. The information processing device 600 causes the projector 100 to perform a light projecting operation in which it emits light, and causes the first light receiver 300 to perform a first light receiving operation in which it receives reflected light Lre that is emitted light Lout from the projector 100 and reflected by the measurement target W and returns. The information processing device 600 does not cause the projector 100 to perform the light projecting operation, but instead causes the first light receiver 300 to perform a second light receiving operation in which it receives first ambient light Ln1. The information processing device 600 outputs first output data D41 based on first three-dimensional data D31 acquired by the first light receiving operation, and outputs second output data D42 based on first two-dimensional data D21 acquired by the second light receiving operation. According to the measuring device 10, by performing a light projection operation and a first light receiving operation, it is possible to output first output data D41 based on first three-dimensional data D31 of the object to be measured W, and by performing a second light receiving operation, it is possible to output second output data D42 based on first two-dimensional data D21 of the object to be measured W.

[0057] The measuring device 10 of the first embodiment includes a first communication I / F 800 as an acquisition unit that executes a third light receiving operation to cause the imaging device 40, which is a second light receiver disposed at a position different from the first light receiver 300, to receive second ambient light Ln2, and acquires second two-dimensional data D22 acquired by the third light receiving operation. The information processing device 600 acquires the second two-dimensional data D22 from the imaging device 40 via the first communication I / F 800. The information processing device 600 may be configured to generate second three-dimensional data D32 based on the first two-dimensional data D21 from the first light receiver 300 and the second two-dimensional data D22 from the imaging device 40, and output second output data D42 based on the second three-dimensional data D32. This configuration allows the second output data D42 based on the second three-dimensional data D32 of the measurement object W to be output without executing a light projection operation.

[0058] In the measuring device 10 of the first embodiment, the light projector 100 may emit light in a first wavelength band, the first light receiver 300 may receive light in the first wavelength band and light in a second wavelength band different from the first wavelength band, and the imaging device 40 may receive light in the second wavelength band. With this configuration, by performing a light projecting operation and a first light receiving operation using light in the first wavelength band, it is possible to output first output data D41 based on first three-dimensional data D31 of the measurement object W. Furthermore, by performing a second light receiving operation and a third light receiving operation using light in the second wavelength band, it is possible to output second output data D42 based on second three-dimensional data D32 of the measurement object W.

[0059] In the measuring device 10 of the first embodiment, the information processing device 600 may be configured to cause the projector 100 to perform a light projecting operation and the first receiver 300 to perform a first light receiving operation when a first condition is satisfied, and to cause the first receiver 300 to perform a second light receiving operation without causing the projector 100 to perform a light projecting operation when the first condition is not satisfied. According to this configuration, the light projecting operation and the first light receiving operation are performed when the first condition is satisfied, and the second light receiving operation is performed when the first condition is not satisfied, thereby making it possible to measure the measurement object W with minimal power.

[0060] The first three-dimensional data D31 is three-dimensional data generated based on TOF data generated by the LiDAR of the measurement device 10 using light in a first wavelength band, which is infrared light. The second three-dimensional data D32 is three-dimensional data generated based on the principle of triangulation by generating first two-dimensional data D21 and second two-dimensional data D22 using light in a second wavelength band, which is visible light, using the stereo camera function of the measurement device 10 and the imaging device 40. In other words, the first three-dimensional data D31 and the second three-dimensional data D32 differ in that the wavelength bands of light used to measure the distance between the measurement system 50 (car 1) and the measurement target W are different, and the principles by which the three-dimensional data are generated are different.

[0061] The vehicle 1 requires more power than ever before due to the installation of an autonomous driving system and an advanced driver assistance system, as well as the electrification of the vehicle 1 itself. Meanwhile, the LiDAR that generates the first three-dimensional data D31 consumes a lot of power. Specifically, the LiDAR consumes a lot of power when emitting the infrared light, the TOF measurement device 340, performing calculations, and generating the first three-dimensional data D31 in the information processing device 600. The measurement system 50 of the first embodiment can acquire three-dimensional information about the surroundings of the vehicle 1 while reducing power consumption by selectively using the LiDAR of the measurement device 10 and image data from the measurement device 10 and the image capture device 40.

[0062] For example, as a first condition, if the moving speed of the measurement system 50 (car 1) is equal to or greater than a reference speed, information from a greater distance is required. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that can measure distances from a greater distance. On the other hand, if the speed is less than the reference speed, the car 1 does not require information from a greater distance. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0063] For example, as a first condition, if the distance between the measurement system 50 (car 1) and the measurement target W is equal to or greater than a reference value, information from a greater distance is required. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that can measure distances from a greater distance. On the other hand, if the distance is less than the reference value, the car 1 does not require information from a greater distance. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0064] For example, as a first condition, if the brightness of the environment around the measurement system 50 (car 1) is below a standard, it becomes difficult for the measurement system 50 to obtain information about the environment around the car 1 using visible light. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that performs measurements using infrared rays. On the other hand, if the brightness is above the standard, the measurement system 50 can sufficiently obtain information about the environment around the car 1 using visible light. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0065] For example, as a first condition, if the charge level of the battery installed in the car 1 is equal to or higher than a reference level, the battery of the car 1 has a margin of charge. Therefore, the measurement system 50 obtains environmental information of the car 1 using the LiDAR function. On the other hand, if the charge level is less than the reference level, in order to reduce the consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information of the car 1 using the stereo camera function, which consumes less power.

[0066] B. Second embodiment: The second embodiment will be described with reference to Figures 1, 2, 8, and 10. Note that the second embodiment is the same as the measurement system 50, which includes the measurement device 10 and the imaging device 40, in terms of the configuration, the light projection operation, each light receiving operation, and generation of a signal for TOF of the measurement system 50, the generation of first three-dimensional data D31 (S30) shown in Figure 8, and the generation of second three-dimensional data D32 (S40) shown in Figure 9. Therefore, in the second embodiment, descriptions of these will be omitted.

[0067] B-1. Measurement process of the second embodiment: The measurement system 50 synthesizes the first three-dimensional data D31 and the second three-dimensional data D32 to generate new third three-dimensional data D33.

[0068] When the measurement system 50 is activated, the measurement system 50 starts each light receiving operation. The measurement system 50 starts the measurement process as shown in FIG. 10. The information processing device 600 generates first three-dimensional data D31 (S30). The information processing device 600 generates second three-dimensional data D32 (S40). The information processing device 600 generates third three-dimensional data D33 from the first three-dimensional data D31 and the second three-dimensional data D32 (S610). The information processing device 600 generates third output data D43 from the third three-dimensional data D33 (S630). The third output data D43 is transmitted to the first communication I / F 800 (S640). When the information processing device 600 determines that it has received an instruction to end the measurement (S650: Yes), it ends the measurement process. If it is not determined that an instruction to end the measurement has been received (S650: No), the process returns to S30.

[0069] B-2. Advantages of the second embodiment: The first three-dimensional data D31 is generated based on TOF data generated by the LiDAR of the measuring device 10 using light in a first wavelength band, which is infrared light. The measuring device 10 can generate the first three-dimensional data D31, for example, even at night, by using infrared light as the emitted light Lout. However, because the LiDAR of the measuring device 10 uses infrared light as the emitted light Lout, there is room for improvement in generating three-dimensional data for measurement objects that absorb, transmit, or diffusely reflect infrared light, such as black or transparent objects. The measuring system 50 of the second embodiment generates the first three-dimensional data D31 using the LiDAR of the measuring device 10, and also generates second three-dimensional data D32 using light in a second wavelength band (visible light) that is not infrared and a method other than the LiDAR (a stereo camera function). By using two types of three-dimensional data, the measuring system 50 of the second embodiment can generate more accurate three-dimensional data. Specifically, the first three-dimensional data D31 and the second three-dimensional data D32 are compared, and more appropriate data is selected for each section (pixel, etc.), and new third three-dimensional data D33 is generated by combining the selected data.

[0070] C. Variations: The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified into various forms without departing from the spirit thereof, for example, the following modifications are also possible.

[0071] The configurations of the measuring device 10 and the imaging device 40 in the above embodiment are merely examples, and various modifications are possible. (1) In the above embodiment, the measurement systems 50 are disposed at both left and right ends of the front end of the vehicle 1 and measure the measurement target W in front (positive direction of the X-axis), but the position and measurement direction of the measurement systems can be changed. Specifically, the measurement systems 50 may be disposed at the rear end of the vehicle 1. (2) In the above embodiment, one measurement system 50 is disposed in the vehicle 1, but multiple measurement systems 50 may be disposed in the vehicle 1. Furthermore, the arrangement of the measurement system 50 and each lamp unit can be changed as appropriate. (3) In the above embodiment, the imaging device 40 includes the two-dimensional data generator 450, the second control circuit board 460, and the second communication I / F 470, but these are not essential. As an example, it is also possible to transmit the third light receiving signal to the information processing device 600 of the measurement device 10 to generate second two-dimensional data D22. (4) In the above embodiment, the measuring device 10 and the imaging device 40 are incorporated into the headlamp 70, but they do not have to be incorporated into the headlamp 70. For example, the imaging device 40 may be disposed in a position similar to the second imaging device 45 above the body of the vehicle 1 (on the positive Z-axis side) in FIG. 1 . (5) In the above embodiment, the imaging device 40 may be an existing imaging device that is installed in the vehicle 1. Specifically, an imaging device used in an adaptive driving beam (ABD) headlamp may be used. (6) In the above embodiment, the measuring device 10 is a flash type LiDAR, but it may be a scanning type. Also, the light source unit 120 may have only one light emitting element or light emitting element array. (7) In the above embodiment, the light receiving element of the first light receiving unit 320 of the measurement device 10 is a light receiving element that is compatible with all wavelengths of light, but is not limited to such a light receiving element. Specifically, it is also possible to use multiple types of light receiving elements that receive only specific wavelengths by passing the incident light through a filter that selects the wavelength of light before receiving it. (8) In the above embodiment, the distance between the center F310 of the first light receiving optical system 310 of the measuring device 10 and the first light receiving surface R320 of the first light receiving unit 320 is theoretically the same as the distance between the center F410 of the second light receiving optical system 410 of the imaging device 40 and the second light receiving surface R420 of the second light receiving unit 420, but the respective focal lengths may be adjusted by correcting the first two-dimensional data D21 and the second two-dimensional data D22. (9) In the above embodiment, the imaging directions of the imaging device 40, the projector 100 of the measuring device 10, and the first light receiver 300 of the measuring device 10 are the same. Even if the imaging directions are slightly misaligned, the imaging directions may be aligned by correcting the generated first two-dimensional data D21 and second two-dimensional data D22. (10) In the above embodiment, the second output data D42 is generated from the second three-dimensional data D32, but it may also be generated from the second two-dimensional data D22. (11) In the first embodiment, the selection between generating the first three-dimensional data D31 and generating the second three-dimensional data D32 is performed automatically based on the first condition, but it can also be performed manually. (12) In the first embodiment, when the first condition is satisfied, the third light receiving operation is not stopped, but the third light receiving operation may be stopped. (13) In the second embodiment, the measurement system 50 generates the third three-dimensional data D33 from the first three-dimensional data D31 and the second three-dimensional data D32, and then generates the third output data D43. However, this order is not limited to this. For example, the measurement system 50 may generate the first output data D41 from the first three-dimensional data D31, generate the second output data D42 from the second three-dimensional data D32, and generate the third output data D43 from the first output data D41 and the second output data D42. (14) In the above embodiment, the first embodiment and the second embodiment may be combined. For example, when the first condition of the first embodiment is satisfied, the measurement system 50 executes the second embodiment. Furthermore, the measurement system 50 can set a second condition and selectively use the first three-dimensional data D31, the second three-dimensional data D32, and the third three-dimensional data D33. [Explanation of symbols]

[0072] 1: Vehicle 10: Measuring device 40: Imaging device 45: Second imaging device 50: Measuring system 61: Low beam lamp unit 62: High beam lamp unit 70: Headlamp 90: External device 100: Light projector 110: Light projecting optical system 120: Light source unit 130: Current source 210: First control circuit board 300: First light receiver 310: First light receiving optical system 320: First light receiving unit 340: TOF measuring device 410: Second light receiving optical system 420: Second light receiving unit 450: 2D data generator 460: Second control circuit board 470: Second communication interface (I / F) 600: Information processing device 800: First communication interface (I / F) A: Point B1: Parallax B2: Distance between cameras C1: Focal length C2: Distance D21: First 2D data D22: Second 2D data D31: First 3D data D32: Second 3D data D33: Third 3D data D41: First output data D42: Second output data D43: Third output data F310: Center F410: Center I21: First 2D image I22: Second 2D image Lin: Incident light Ln1: First ambient light Ln2: Second ambient light Lout: Emitted light Lre: Reflected light P1: First focal point P2: Corresponding point Q: Second focal point R320: First light-receiving surface R420: Second light-receiving surface W: Measurement object

Claims

1. A measuring device comprising: A floodlight that emits light; a first light receiver that receives light; a controller; The controller causing the light projector to perform a light projecting operation of emitting light, and causing the first light receiver to perform a first light receiving operation of receiving reflected light that is the light emitted from the light projector and reflected by a measurement object and returns; causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light transmitter to perform the light projecting operation; outputting first output data based on the first three-dimensional data acquired by the first light receiving operation; outputting second output data based on the first two-dimensional data acquired by the second light receiving operation; Measuring equipment.

2. 2. The measuring device according to claim 1, an acquisition unit that causes a second light receiver disposed at a position different from the first light receiver to perform a third light receiving operation in which the second light receiver receives external light, and acquires second two-dimensional data acquired by the third light receiving operation; The controller acquiring the second two-dimensional data from the second light receiver via the acquisition unit; generating second three-dimensional data based on the first two-dimensional data from the first light receiver and the second two-dimensional data from the second light receiver; outputting the second output data based on the second three-dimensional data; Measuring equipment.

3. 3. The measuring device according to claim 2, the light projector emits light in a first wavelength band, the first optical receiver receives light in the first wavelength band and light in a second wavelength band different from the first wavelength band; the second optical receiver receives light in the second wavelength band; Measuring equipment.

4. The measuring device according to claim 2 or 3, The controller causing the light projector to perform the light projecting operation and the first light receiver to perform the first light receiving operation when a first condition is satisfied; If the first condition is not satisfied, the light projector is not caused to perform the light projecting operation, and the first light receiver is caused to perform the second light receiving operation. Measuring equipment.

5. A computer included in a measurement system including a measurement device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light and is disposed at a position different from the first light receiver, causing the light projector to perform a light projecting operation of emitting light, and causing the first light receiver to perform a first light receiving operation of receiving reflected light that is the light emitted from the light projector and reflected by a measurement object and returns; causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light transmitter to perform the light projecting operation; causing the second light receiver to perform a third light receiving operation of receiving external light; outputting first output data based on the first three-dimensional data acquired by the first light receiving operation; generating second three-dimensional data based on the first two-dimensional data acquired by the second light receiving operation and the second two-dimensional data acquired by the third light receiving operation; outputting second output data based on the second three-dimensional data; Measurement system control program.

6. A control method for a measurement system including a measurement device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light and is disposed at a position different from that of the first light receiver, the method comprising: causing the light projector to perform a light projecting operation of emitting light, and causing the first light receiver to perform a first light receiving operation of receiving reflected light that is the light emitted from the light projector and reflected by a measurement object and returns; causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light transmitter to perform the light projecting operation; causing the second light receiver to perform a third light receiving operation of receiving external light; outputting first output data based on the first three-dimensional data acquired by the first light receiving operation; generating second three-dimensional data based on the first two-dimensional data acquired by the second light receiving operation and the second two-dimensional data acquired by the third light receiving operation; outputting second output data based on the second three-dimensional data; How to control the measurement system.

7. A computer-readable recording medium having recorded thereon a computer program for controlling a measurement system including a measurement device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light and is disposed at a position different from that of the first light receiver, causing the light projector to perform a light projecting operation of emitting light, and causing the first light receiver to perform a first light receiving operation of receiving reflected light that is the light emitted from the light projector and reflected by a measurement object and returns; causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light transmitter to perform the light projecting operation; causing the second light receiver to perform a third light receiving operation of receiving external light; outputting first output data based on the first three-dimensional data acquired by the first light receiving operation; generating second three-dimensional data based on the first two-dimensional data acquired by the second light receiving operation and the second two-dimensional data acquired by the third light receiving operation; outputting second output data based on the second three-dimensional data; A computer-readable recording medium having a computer program recorded thereon that causes the measuring device to execute the above steps.

Images