Measurement device, control program for measurement system, control method for measurement system, and recording medium

Abstract

The purpose of the present invention is to improve a measurement device, such as LiDAR. This measurement device includes a light projector that emits light, a first light receiver that receives light, and a controller. The controller acquires first three-dimensional data by causing the light projector to execute a light projection action of emitting light and causing the first light receiver to execute a first light reception action of receiving reflected light that is the light emitted from the light projector, reflected off an object being measured, and returning, and further outputs first output data. The controller acquires first two-dimensional data by causing the first light receiver to execute a second light reception action of receiving external light, without causing the light projector to execute the light projection action. The controller acquires second two-dimensional data by causing a second light receiver disposed at a position different from the first light receiver to execute a third light reception action of receiving external light. The controller generates second three-dimensional data on the basis of the first two-dimensional data and the second two-dimensional data, and further outputs second output data.

Description

Measurement device, measurement system control program, measurement system control method, and recording medium

[0001] The technology disclosed in this specification relates to a measurement device, a control program for a measurement system, a control method for a measurement system, and a recording medium.

[0002] With the advancement of autonomous driving (AD) systems and advanced driver assistance systems (ADAS), research and development of light detection and ranging (LiDAR) is underway as one of the measurement devices used to grasp the surrounding environment and estimate the vehicle's position while the vehicle is traveling. LiDAR includes a light projector that projects laser light onto a measurement target, a light receiver that receives light reflected from the measurement target, and a controller. The controller causes the light projector to perform a light projecting operation of emitting light, causes the light receiver to perform a light receiving operation of receiving light reflected from the measurement target, and outputs three-dimensional data acquired by the light receiving operation (see, for example, Patent Document 1).

[0003] Special table 2018-535402 publication

[0004] The present inventors have discovered that there is room for further utilization of measurement devices such as LiDAR.

[0005] This specification discloses a technique that can solve the above-mentioned problems.

[0006] The technology disclosed in this specification can be realized, for example, in the following forms.

[0007] (1) A measurement device disclosed in this specification includes a light projector that emits light, a first light receiver that receives light, and a controller. The controller causes the light projector to perform a light projecting operation that emits light and causes the first light receiver to perform a first light receiving operation that receives light that is emitted from the light projector and reflected back from a measurement object. The controller also causes the first light receiver to perform a second light receiving operation that receives external light without causing the light projector to perform the light projecting operation. The controller outputs first output data based on first three-dimensional data acquired by the first light receiving operation and outputs second output data based on first two-dimensional data acquired by the second light receiving operation. According to this measurement device, by performing the light projecting operation and the first light receiving operation, first output data based on first three-dimensional data of the measurement object can be output. Furthermore, by performing the second light receiving operation, second output data based on first two-dimensional data of the measurement object can be output.

[0008] (2) The measuring device may further include an acquisition unit that executes a third light receiving operation to cause a second light receiver disposed at a different position from the first light receiver to receive external light, and acquires second two-dimensional data acquired by the third light receiving operation, wherein the controller acquires the second two-dimensional data from the second light receiver via the acquisition unit, generates second three-dimensional data based on the first two-dimensional data from the first light receiver and the second two-dimensional data from the second light receiver, and outputs the second output data based on the second three-dimensional data. With this configuration, the second output data based on the second three-dimensional data of the measurement object can be output without executing the light projecting operation.

[0009] (3) In the above measurement device, the light projector may emit light in a first wavelength band, the first light receiver may receive light in the first wavelength band and light in a second wavelength band different from the first wavelength band, and the second light receiver may receive light in the second wavelength band. With this configuration, first output data based on first three-dimensional data of the object to be measured can be output by performing a light projecting operation and a first light receiving operation using light in the first wavelength band, and second output data based on second three-dimensional data of the object to be measured can be output by performing a second light receiving operation and a third light receiving operation using light in the second wavelength band.

[0010] (4) In the above-described measuring device, the controller may be configured to cause the transmitter to perform the light-projecting operation and the first receiver to perform the first light-receiving operation when a first condition is satisfied, and to cause the first receiver to perform the second light-receiving operation without causing the transmitter to perform the light-projecting operation when the first condition is not satisfied. With this configuration, the controller can measure the object to be measured with minimal power by performing the light-projecting operation and the first light-receiving operation when the condition is satisfied and performing the second light-receiving operation when the condition is not satisfied.

[0011] The technology disclosed in this specification can be realized in various forms, for example, in the form of a measuring device, a second photodetector, a control method, a control program, or a computer-readable recording medium on which the control program is recorded.

[0012] Schematic diagram of an entire vehicle equipped with a measurement system according to the first embodiment; Block diagram showing the configuration of the measurement system according to the first embodiment; Diagram explaining the principle of a distance measurement method using a first light receiver and an imaging device; Diagram explaining the parallax between a first two-dimensional image and a second two-dimensional image; Flowchart showing the flow of measurement processing when a first condition is satisfied; Flowchart showing the flow of measurement processing when a first condition is not satisfied; Flowchart showing the flow of first output data generation processing; Flowchart showing the flow of second output data generation processing; Flowchart showing the flow of second two-dimensional data generation processing; Flowchart showing the flow of third three-dimensional data generation processing in the second embodiment.

[0013] A. First Embodiment: The first embodiment will be described with reference to Fig. 1 to Fig. 9. Fig. 1 shows mutually orthogonal X, Y and Z axes for specifying directions. The positive X-axis direction is the forward direction, and the negative X-axis direction is the backward direction. The positive Y-axis direction is the left direction, and the negative Y-axis direction is the right direction. Furthermore, the positive Z-axis direction is the upward direction, and the negative Z-axis direction is the downward direction.

[0014] 1 is a perspective view of a vehicle 1 equipped with a measuring device 10 and an imaging device 40 (hereinafter, sometimes collectively referred to as a "measurement system 50") according to the first embodiment. The vehicle 1 is equipped with a pair of headlamps 70. The pair of headlamps 70 are respectively disposed on both the left and right sides of the front end of the vehicle 1. Each headlamp 70 is equipped with a low beam lamp unit 61, a high beam lamp unit 62, an imaging device 40, and a measuring device 10. The low beam lamp unit 61, the high beam lamp unit 62, the imaging device 40, and the measuring device 10 are arranged in this order from the center of the vehicle toward the side.

[0015] A-1. Configuration of the measuring device 10: The measuring device 10 of the first embodiment is a FLASH-type LiDAR. The measuring device 10 is mounted, for example, on a vehicle equipped with an AD (automated driving system) or an ADAS (advanced driver assistance system). The measuring device 10 assists in the detection of objects such as people and other vehicles while the vehicle is traveling, and provides various information to other devices and users that is useful for ensuring the safety of the vehicle driver and those around the vehicle and for reducing damage to surrounding objects while the vehicle is being driven.

[0016] As shown in FIGS. 1 and 2, the measurement device 10 includes a light projector 100, a first light receiver 300, an information processing device 600, and a first communication interface (I / F) 800.

[0017] (Light Projector) The light projector 100 includes a light projection optical system 110 , a light source unit 120 , a current source 130 , and a first control circuit board 210 .

[0018] The light source unit 120 has a plurality of light-emitting elements (not shown) or a plurality of light-emitting element arrays (for example, light-emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). Each light-emitting element is, for example, a laser diode, a surface-emitting type laser light-emitting element (for example, a VCSEL (Vertical Cavity Surface Emitting Laser), hereinafter referred to as a "surface-emitting element"), a surface-emitting element array (for example, a VCSEL array) in which a plurality of surface-emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (such as a semiconductor substrate or a ceramic substrate), or the like.

[0019] The projection optical system 110 adjusts the light distribution of the output light Lout by applying optical effects (such as refraction, scattering, and diffraction) to light (e.g., a light beam (laser light)) emitted by each light-emitting element of the light source unit 120 (hereinafter referred to as "output light Lout"). The projection optical system 110 is configured using optical components such as various lenses, such as collimating lenses, and reflectors (mirrors). The projection optical system 110 is disposed on the optical path of the output light Lout output from each light-emitting element of the light source unit 120. In other words, the projection optical system 110 is disposed so as to face each light-emitting element in the optical axis direction of each light-emitting element. The projection optical system 110 is disposed at a distance from each light-emitting element in the optical axis direction.

[0020] The current source 130 supplies current to each light-emitting element that constitutes the light source unit 120. The current source 130 supplies, for example, to each light-emitting element, a periodic pulse wave current for turning on and off the current flowing through each light-emitting element.

[0021] The first control circuit board 210 is a circuit board on which electronic components and the like are mounted for controlling the light emission of each light-emitting element of the light source unit 120. The first control circuit board 210 is communicably connected to the current source 130, a TOF measuring device 340 of the first light receiver 300 (described later), and the information processing device 600.

[0022] (First Photoreceiver) As shown in FIG. 2, the first photoreceiver 300 includes a first photoreception optical system 310, a first photoreceptor 320, and a TOF measuring device 340.

[0023] The first light receiving optical system 310 is an optical system for causing the first light receiving unit 320 to receive incident light. The first light receiving optical system 310 may be, for example, various lenses such as a condenser lens, various filters such as a wavelength filter, or a reflecting mirror. The first light receiving optical system 310 is disposed on the optical path of incident light input to each light receiving element of the first light receiving unit 320. In other words, the first light receiving optical system 310 is disposed so as to face each light receiving element in the optical axis direction of each light receiving element.

[0024] The first light receiving unit 320 includes a plurality of light receiving elements. Each light receiving element is, for example, a photodiode, an avalanche photodiode, or a single-photon avalanche diode (SPAD). The first light receiving unit 320 photoelectrically converts the incident light from the first light receiving optical system 310 to generate a light receiving signal having a current level or a voltage level corresponding to the intensity of the incident light.

[0025] The TOF measuring device 340 has, for example, a time measurement integrated circuit (IC) equipped with a time-to-digital converter (TDC) circuit. The TOF measuring device 340 is communicably connected between the first control circuit board 210 and the information processing device 600. Details of the TOF measuring device 340 will be described later.

[0026] (Information Processing Device, etc.) The information processing device 600 has a processor. The processor may be, for example, a central processing unit (CPU), a micro processing unit (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), etc. The information processing device 600 is communicatively connected to the first control circuit board 210, the TOF measuring device 340, and the first communication I / F 800. The information processing device 600 is an example of a controller. Details of the information processing device 600 will be described later.

[0027] The first communication I / F 800 is communicatively connected to the information processing device 600, a second communication I / F 470 of the imaging device 40 (described later), and an external device 90. The first communication I / F 800 transmits information processed by the information processing device 600 to the external device 90. The first communication I / F 800 also receives information obtained by the imaging device 40 and transmits it to the information processing device 600. The first communication I / F 800 is an example of an acquisition unit. Communication between the first communication I / F 800 and the second communication I / F 470 or the external device 90 may be wired or wireless.

[0028] The external device 90 may be, for example, a device that creates an environmental map using a point cloud, or a device that performs self-location estimation (SLAM: Simultaneous Localization and Mapping) using a scan matching algorithm such as NDT (Normal Distributions Transform) or ICP (Iterative Closest Point).

[0029] A-2. Configuration of imaging device 40: As shown in Fig. 2, imaging device 40 includes a second light-receiving optical system 410, a second light-receiving unit 420, a two-dimensional data generator 450, a second control circuit board 460, and a second communication I / F 470. The imaging direction of imaging device 40, projector 100 of measuring device 10, and first light-receiver 300 of measuring device 10 is the same (positive X-axis direction in Fig. 1). Note that imaging device 40 is an example of a second light-receiver.

[0030] The second light receiving optical system 410 is an optical system for causing the second light receiving unit 420 to receive incident light. The second light receiving optical system 410 may be, for example, various lenses such as a condenser lens, various filters such as a wavelength filter, or a reflecting mirror. The second light receiving optical system 410 is disposed on the optical path of the incident light input to each light receiving element of the second light receiving unit 420. In other words, the second light receiving optical system 410 is disposed so as to face each light receiving element in the optical axis direction of each light receiving element.

[0031] The second light receiving unit 420 includes a light receiving element. The light receiving element is, for example, a photodiode, an avalanche photodiode, or a single-photon avalanche diode (SPAD). The second light receiving unit 420 also includes an image sensor. Examples of the image sensor include a CCD (Change Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The second light receiving unit 420 photoelectrically converts the incident light from the second light receiving optical system 410 to generate a light receiving signal with a current level or voltage level corresponding to the incident light.

[0032] The two-dimensional data generator 450 includes, for example, an image processor, a DRAM, etc. The two-dimensional data generator 450 is communicably connected between the second light receiving unit 420 and the second control circuit board 460. The two-dimensional data generator 450 receives the light receiving signal output from the second light receiving unit 420 and generates image data.

[0033] A-3. Light Projection Period, etc.: (Light Projection Operation) The information processing device 600 causes the projector 100 to repeatedly execute a light projection operation at predetermined intervals. The light projection operation is an operation in which the first control circuit board 210 controls the current source 130 to sequentially cause the multiple light-emitting elements of the light source unit 120 to emit light. Hereinafter, a period in which the projector 100 executes a light projection operation will be referred to as a "light projection period," and a period in which the projector 100 does not execute a light projection operation (for example, a period from the end of one light projection operation to the start of the next light projection operation) will be referred to as a "non-light projection period." Furthermore, a signal indicating the timing at which each light-emitting element emits the output light Lout (hereinafter referred to as "light projection timing") will be referred to as a "light projection timing signal."

[0034] The output light Lout is light in a first wavelength band, which is a certain wavelength band. Specifically, the first wavelength band is infrared light. The first wavelength band is, for example, 780 nm to 2000 nm, or 850 nm to 950 nm.

[0035] (First Light Receiving Operation) The first light receiving unit 320 outputs a light receiving signal (hereinafter referred to as the "first light receiving signal") corresponding to the amount of light received by each of the multiple light receiving elements during a first light receiving period (hereinafter referred to as the "first light receiving operation"). The first light receiving period is a period corresponding to the light projection period, during which each light receiving element can receive reflected light Lre that is returned by reflecting output light Lout emitted by each light emitting element during the light projection operation off the measurement target W (see FIG. 2). Furthermore, a signal indicating the timing at which each light receiving element receives reflected light Lre (hereinafter referred to as the "light receiving timing") is referred to as the "light receiving timing signal."

[0036] (Second Light Receiving Operation) The first light receiving unit 320 outputs a light receiving signal (hereinafter referred to as a "second light receiving signal") corresponding to the amount of light received by each of the plurality of light receiving elements during a second light receiving period (hereinafter referred to as a "second light receiving operation"). The second light receiving period is a period corresponding to the non-light projecting period, during which each light receiving element cannot receive reflected light Lre that is returned by reflecting output light Lout emitted by each light emitting element during the light projecting operation onto the measurement target W.

[0037] The light received by the first light receiving unit 320 during the second light receiving period (hereinafter referred to as "first ambient light Ln1") is light in a second wavelength band, which is a certain wavelength band. Specifically, the second wavelength band is visible light. Specifically, the second wavelength band is 360 nm to 830 nm, or 380 nm to 780 nm. The first ambient light Ln1 includes light from sunlight, street lights, etc. (hereinafter referred to as "natural light"), and light that is reflected back from the measurement target W.

[0038] (Generation of signal corresponding to TOF) The TOF measuring instrument 340 receives a light-projection timing signal from the first control circuit board 210 during the light-projection period, receives a first light-receiving signal and a light-receiving timing signal from the first light-receiving unit 320 during the first light-receiving period, and receives a second light-receiving signal from the first light-receiving unit 320 during the second light-receiving period.

[0039] The TOF measuring device 340 calculates the difference between the light projection timing and the light reception timing, i.e., the time of flight (TOF) of the laser light, based on the light projection timing signal and the light reception timing signal. The TOF measuring device 340 outputs a signal corresponding to the calculated TOF, the first light reception signal, and the second light reception signal to the information processing device 600.

[0040] (Third Light Receiving Operation) The second light receiving unit 420 of the imaging device 40 outputs a light receiving signal (hereinafter referred to as the "third light receiving signal") corresponding to the amount of light received by each of the multiple light receiving elements (hereinafter referred to as the "third light receiving operation"). Each light receiving element of the second light receiving unit 420 receives external light (hereinafter referred to as the "second ambient light Ln2"). The second ambient light Ln2 is light in a second wavelength band, which is the same fixed wavelength band as the first ambient light Ln1. After the third light receiving operation, the second light receiving unit 420 inputs the third light receiving signal to the two-dimensional data generator 450.

[0041] The two-dimensional data generator 450 generates second two-dimensional data D22 based on the third light-receiving signal. The second two-dimensional data D22 is transmitted to the information processing device 600 of the measuring device 10 via the second control circuit board 460 and the second communication I / F 470.

[0042] A-4. Measurement Process of First Embodiment: Figures 5 to 9 are flowcharts showing the flow of the measurement process by the measurement system 50. As shown in Figure 5, when the measurement system 50 starts the measurement process, the information processing device 600 (an example of a controller) determines whether a first condition is satisfied (S110).

[0043] The first condition is as follows. Note that the following is an example. The moving speed of the measurement system 50 (car 1) is equal to or greater than a standard. The distance between the measurement system 50 (car 1) and the measurement target W is equal to or greater than a standard. The brightness of the environment around the measurement system 50 (car 1) is less than a standard. The charge level of the battery installed in car 1 is equal to or greater than a standard.

[0044] When the information processing device 600 determines that the first condition is satisfied (S110: Yes), the information processing device 600 activates the projector 100 and causes the projector 100 to start a light projection operation (S120). The information processing device 600 generates first three-dimensional data D31 as shown in FIG. 7 (S30).

[0045] The information processing device 600 causes each light-emitting element of the light source unit 120 to perform a light projection operation via the first control circuit board 210 (S310). The first light-receiving unit 320 performs a first light-receiving operation (S320). The TOF measurement device 340 generates a signal corresponding to the TOF from the light-projection timing signal and the light-receiving timing signal (S360). The information processing device 600 receives the signal corresponding to the TOF and the first light-receiving signal, and generates first three-dimensional data D31 based on these (S370). The first three-dimensional data D31 may be, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, point cloud information, etc.

[0046] As shown in FIG. 5 , the information processing device 600 generates first output data D41 from the first three-dimensional data D31 (S130). The information processing device 600 transmits the first output data D41 to the first communication I / F 800 (S140). If the information processing device 600 determines that it has received an instruction to end measurement (S150: Yes), it terminates the measurement process. If the information processing device 600 determines that it has not received an instruction to end measurement, it proceeds to S160. The information processing device 600 determines whether the first condition is satisfied (S160: Yes). If the information processing device 600 determines that the first condition is satisfied (S160: Yes), it returns to S30. If the information processing device 600 determines that the first condition is not satisfied (S160: No), it stops the projector 100 and causes the projector 100 to stop projecting light (S170). Then, it proceeds to S40 of FIG. 6 .

[0047] As shown in Fig. 5, if the measurement system 50 determines that the first condition is not satisfied when starting the measurement process (S110: No), the information processing device 600 proceeds to S40 in Fig. 6. The information processing device 600 generates second three-dimensional data D32 (S40), as shown in Figs. 8 and 9.

[0048] The first light receiving unit 320 of the measurement device 10 performs a second light receiving operation (S420) as shown in Fig. 8. The information processing device 600 generates first two-dimensional data D21 from the second light receiving signal (S430).

[0049] As shown in FIG. 9 , the second light receiving unit 420 of the imaging device 40 performs a third light receiving operation (S520). The two-dimensional data generator 450 generates second two-dimensional data D22 from the third light receiving signal (S530). The second two-dimensional data D22 is transmitted from the two-dimensional data generator 450 to the second communication I / F 470 via the second control circuit board 460 (S540). The second two-dimensional data D22 transmitted to the second communication I / F 470 is then transmitted to the first communication I / F 800 of the measurement device 10 (S550).

[0050] 8, the information processing device 600 acquires the second two-dimensional data D22 from the second communication I / F 470 via the first communication I / F 800 (S450). The information processing device 600 generates the second three-dimensional data D32 from the first two-dimensional data D21 and the second two-dimensional data D22 (S470).

[0051] 3 and 4, information on the distance C2 (depth) between the measurement system 50 and the measurement target W is calculated from the first two-dimensional data D21 and the second two-dimensional data D22 according to the principle of triangulation, thereby generating second three-dimensional data D32.

[0052] As shown in FIG. 3 , the first light receiver 300 of the measurement device 10 and the second light receiver, the imaging device 40, have the same imaging direction with respect to the measurement object W. The first light receiver 300 receives first ambient light Ln1 at point A on the measurement object W via a first light receiving optical system 310 at a first light receiving unit 320. The intersection of a line connecting point A and a center F310 of the first light receiving optical system 310 with a first light receiving surface R320 of the first light receiving unit 320 is a first focal point P1 for point A. Meanwhile, the imaging device 40 receives second ambient light Ln2 at point A on the measurement object W via a second light receiving optical system 410 at a second light receiving unit 420. The second focal point Q for point A is the intersection of a straight line connecting point A and the center F410 of the second light receiving optical system 410 with the second light receiving surface R420 of the second light receiving unit 420.

[0053] FIG. 4 is a conceptual diagram of a first two-dimensional image I21 of the first two-dimensional data D21 of the measuring device 10 and a second two-dimensional image I22 of the second two-dimensional data D22 of the imaging device 40. The measurement object W in the first two-dimensional image I21 is imaged to the right of the measurement object W in the second two-dimensional image I22. That is, the first focus P1 and the second focus Q are imaged at offset positions. The first focus P1 and the second focus Q have a difference in length of parallax B1. When parallax B1 is shown in FIG. 3, the first focus P1 corresponds to a corresponding point P2 on the second light receiving surface R420 of the second light receiving unit. Specifically, the corresponding point P2 is the intersection of a line Ln12 drawn through the center F410 of the second light receiving optical system 410 and parallel to the first ambient light Ln1 with the second light receiving surface R420.

[0054] Here, by triangulation, the following relationship holds among the parallax B1, the inter-camera distance B2 between the centers F310 and F410, the focal length C1 between the center F310 and the first light-receiving surface R320 (or the center F410 and the second light-receiving surface R420), and the distance C2 between point A and the measurement device 10 (or the image capture device 40): <Formula> C2 = C1 × B2 / B1 As described above, the measurement device 10 and the image capture device 40 can measure the distance C2 (depth) between the measurement system 50 and the measurement object W. According to this theory, the second three-dimensional data D32 is created by calculating the distance C2 (depth) between the measurement system 50 and the measurement object W from the parallax B1 for each pixel.

[0055] As shown in FIG. 6 , the information processing device 600 generates second output data D42 from the second three-dimensional data D32 (S230). The information processing device 600 transmits the second output data D42 to the first communication I / F 800 (S240). If the information processing device 600 determines that it has received an instruction to end measurement (S250: Yes), it ends the measurement process. If the information processing device 600 determines that it has not received an instruction to end measurement, it proceeds to S260. The information processing device 600 determines whether the first condition is satisfied (S260). If the information processing device 600 determines that the first condition is not satisfied (S260: No), it returns to S40. If the information processing device 600 determines that the first condition is satisfied (S260: Yes), it proceeds to S120 of FIG. 5 .

[0056] A-5. Advantages of the First Embodiment: According to the first embodiment, the measurement device 10 includes a projector 100 that emits light, a first light receiver 300 that receives the light, and an information processing device 600 that serves as a controller. The information processing device 600 causes the projector 100 to perform a light projecting operation in which it emits light, and causes the first light receiver 300 to perform a first light receiving operation in which it receives reflected light Lre that is emitted light Lout from the projector 100 and reflected by the measurement target W and returns. The information processing device 600 causes the first light receiver 300 to perform a second light receiving operation in which it receives first ambient light Ln1, without causing the projector 100 to perform the light projecting operation. The information processing device 600 outputs first output data D41 based on the first three-dimensional data D31 acquired by the first light receiving operation, and outputs second output data D42 based on the first two-dimensional data D21 acquired by the second light receiving operation. According to the measuring device 10, by performing a light projecting operation and a first light receiving operation, it is possible to output the first output data D41 based on the first three-dimensional data D31 of the measurement object W, and by performing a second light receiving operation, it is possible to output the second output data D42 based on the first two-dimensional data D21 of the measurement object W.

[0057] The measuring apparatus 10 of the first embodiment may include a first communication I / F 800 as an acquisition unit that executes a third light receiving operation to cause the imaging device 40, which is a second light receiver disposed at a position different from the first light receiver 300, to receive second ambient light Ln2, and acquires second two-dimensional data D22 acquired by the third light receiving operation. The information processing apparatus 600 may also be configured to generate second three-dimensional data D32 based on the first two-dimensional data D21 from the first light receiver 300 and the second two-dimensional data D22 from the imaging device 40, and output second output data D42 based on the second three-dimensional data D32. This configuration allows the second output data D42 based on the second three-dimensional data D32 of the measurement object W to be output without executing a light projection operation.

[0058] In the measuring apparatus 10 of the first embodiment, the light projector 100 may emit light in a first wavelength band, the first light receiver 300 may receive light in the first wavelength band and light in a second wavelength band different from the first wavelength band, and the imaging device 40 may receive light in the second wavelength band. According to this configuration, by performing a light projecting operation and a first light receiving operation using light in the first wavelength band, it is possible to output first output data D41 based on first three-dimensional data D31 of the measurement object W. Furthermore, by performing a second light receiving operation and a third light receiving operation using light in the second wavelength band, it is possible to output second output data D42 based on second three-dimensional data D32 of the measurement object W.

[0059] In the measuring device 10 of the first embodiment, the information processing device 600 may be configured to cause the transmitter 100 to perform a light-projecting operation and the first receiver 300 to perform a first light-receiving operation when the first condition is satisfied, and to cause the first receiver 300 to perform a second light-receiving operation without causing the transmitter 100 to perform a light-projecting operation when the first condition is not satisfied. According to this configuration, the light-projecting operation and the first light-receiving operation are performed when the first condition is satisfied, and the second light-receiving operation is performed when the first condition is not satisfied, thereby making it possible to measure the measurement object W with minimal power.

[0060] The first three-dimensional data D31 is three-dimensional data generated based on TOF data generated by the LiDAR of the measurement device 10 using light in a first wavelength band, which is infrared light. The second three-dimensional data D32 is three-dimensional data generated based on the principle of triangulation using the stereo camera function of the measurement device 10 and the imaging device 40 to generate first two-dimensional data D21 and second two-dimensional data D22 using light in a second wavelength band, which is visible light. In other words, the first three-dimensional data D31 and the second three-dimensional data D32 differ in that the wavelength bands of light used to measure the distance between the measurement system 50 (car 1) and the measurement target W are different, and the principles by which the three-dimensional data are generated are different.

[0061] The vehicle 1 requires more power than ever before due to the installation of an autonomous driving system and an advanced driver assistance system, as well as the electrification of the vehicle 1 itself. Meanwhile, the LiDAR that generates the first three-dimensional data D31 requires a lot of power consumption. Specifically, the LiDAR consumes a lot of power when emitting the infrared emitted light Lout, performing calculations in the TOF measuring device 340, and generating the first three-dimensional data D31 in the information processing device 600. The measurement system 50 of the first embodiment can acquire three-dimensional information about the surroundings of the vehicle 1 while reducing power consumption by selectively using the LiDAR of the measurement device 10 and image data from the measurement device 10 and the imaging device 40.

[0062] For example, as a first condition, if the moving speed of the measurement system 50 (car 1) is equal to or greater than a reference speed, information from a greater distance is required. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that can measure distances from a greater distance. On the other hand, if the speed is less than the reference speed, the car 1 does not require information from a greater distance. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0063] For example, as a first condition, if the distance between the measurement system 50 (car 1) and the measurement target W is equal to or greater than a reference value, information from a greater distance is required. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that can measure distances from a greater distance. On the other hand, if the distance is less than the reference value, the car 1 does not require information from a greater distance. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0064] For example, as a first condition, if the brightness of the environment surrounding the measurement system 50 (car 1) is below a standard, it becomes difficult for the measurement system 50 to obtain information about the environment around the car 1 using visible light. Therefore, the measurement system 50 obtains environmental information about the car 1 using a LiDAR function that performs measurements using infrared rays. On the other hand, if the brightness is above the standard, the measurement system 50 can sufficiently obtain information about the environment around the car 1 using visible light. Therefore, in order to reduce consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using a stereo camera function.

[0065] For example, as a first condition, if the charge level of the battery installed in the car 1 is equal to or higher than a reference level, the battery in the car 1 has a remaining charge level. Therefore, the measurement system 50 obtains environmental information about the car 1 using the LiDAR function. On the other hand, if the charge level is less than the reference level, in order to reduce the consumption of the battery installed in the car 1, the measurement system 50 obtains environmental information about the car 1 using the stereo camera function, which consumes less power.

[0066] B. Second Embodiment: The second embodiment will be described with reference to Figures 1, 2, 8, and 10. Note that the second embodiment is the same as the second embodiment in terms of the configuration of the measurement device 10 and the image capture device 40 that constitute the measurement system 50, the light projection operation, each light receiving operation, and generation of a signal for TOF of the measurement system 50, the generation of first three-dimensional data D31 (S30) shown in Figure 8, and the generation of second three-dimensional data D32 (S40) shown in Figure 9. Therefore, description of these will be omitted in the second embodiment.

[0067] B-1. Measurement Processing of Second Embodiment: The measurement system 50 synthesizes the first three-dimensional data D31 and the second three-dimensional data D32 to generate new third three-dimensional data D33.

[0068] When the measurement system 50 is activated, the measurement system 50 starts each light receiving operation. The measurement system 50 starts the measurement process as shown in FIG. 10 . The information processing device 600 generates first three-dimensional data D31 (S30). The information processing device 600 generates second three-dimensional data D32 (S40). The information processing device 600 generates third three-dimensional data D33 from the first three-dimensional data D31 and the second three-dimensional data D32 (S610). The information processing device 600 generates third output data D43 from the third three-dimensional data D33 (S630). The third output data D43 is transmitted to the first communication I / F 800 (S640). If the information processing device 600 determines that it has received an instruction to end the measurement (S650: Yes), it ends the measurement process. If it does not determine that it has received an instruction to end the measurement (S650: No), it returns to S30.

[0069] B-2. Effect of the Second Embodiment: The first three-dimensional data D31 is three-dimensional data generated based on TOF data generated by the LiDAR of the measuring device 10 using light in a first wavelength band, which is infrared. The measuring device 10 can generate the first three-dimensional data D31, for example, even at night, by using infrared light as the emitted light Lout. However, because the LiDAR of the measuring device 10 uses infrared light as the emitted light Lout, there is room for improvement in generating three-dimensional data for measurement objects that absorb, transmit, or diffusely reflect infrared light, such as black objects or transparent objects. The measurement system 50 of the second embodiment generates the first three-dimensional data D31 using the LiDAR of the measuring device 10, and also generates second three-dimensional data D32 using light in a second wavelength band (visible light) that is not infrared and a method other than LiDAR (stereo camera function). Using two types of three-dimensional data, the measurement system 50 of the second embodiment can generate more accurate three-dimensional data. Specifically, the first three-dimensional data D31 and the second three-dimensional data D32 are compared, more accurate data is selected for each section (pixel, etc.), and new third three-dimensional data D33 is generated by combining the selected data.

[0070] C. Modifications: The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0071] The configuration of the measuring device 10 and the imaging device 40 in the above embodiment is merely an example and can be modified in various ways. (1) In the above embodiment, the measurement systems 50 are disposed at both left and right ends of the front end of the vehicle 1 and measure the measurement target W forward (positive direction of the X-axis). However, the position and measurement direction can be changed. Specifically, the measurement system 50 may be disposed at the rear end of the vehicle 1. (2) In the above embodiment, one measurement system 50 is disposed on the vehicle 1. However, multiple measurement systems 50 may be disposed on the vehicle 1. Furthermore, the positions of the measurement system 50 and each lamp unit can be changed as appropriate. (3) In the above embodiment, the imaging device 40 includes a 2D data generator 450, a second control circuit board 460, and a second communication I / F 470, but these are not required. As an example, it is also possible to transmit a third light receiving signal to the information processing device 600 of the measuring device 10 and generate second 2D data D22. (4) In the above embodiment, the measuring device 10 and the imaging device 40 are incorporated into the headlamp 70, but they do not have to be incorporated into the headlamp 70. For example, the imaging device 40 may be disposed in a position similar to the second imaging device 45 above the vehicle body (positive Z-axis direction) of the vehicle 1 in FIG. 1 . (5) In the above embodiment, the imaging device 40 may be an existing imaging device incorporated into the vehicle 1. Specifically, an imaging device used in an adaptive driving beam (ABD) headlamp may be utilized. (6) In the above embodiment, the measuring device 10 is a flash-type LiDAR, but may also be a scanning-type LiDAR. Furthermore, the light source unit 120 may include only one light-emitting element or light-emitting element array. (7) In the above embodiment, the light-receiving element of the first light-receiving unit 320 of the measuring device 10 is a light-receiving element that is compatible with all wavelengths of light, but is not limited to such a light-receiving element. Specifically, by passing the incident light through a filter that selects the wavelength of the light before receiving it, it is possible to use a plurality of types of light receiving elements that receive only light of specific wavelengths.(8) In the above embodiment, the distance between the center F310 of the first light receiving optical system 310 of the measuring device 10 and the first light receiving surface R320 of the first light receiving unit 320 and the distance between the center F410 of the second light receiving optical system 410 of the imaging device 40 and the second light receiving surface R420 of the second light receiving unit 420 are theoretically the same, but the respective focal lengths may be aligned by correcting the first two-dimensional data D21 and the second two-dimensional data D22. (9) In the above embodiment, the imaging direction of the imaging device 40, the light projector 100 of the measuring device 10, and the first light receiver 300 of the measuring device 10 is the same. Even if the imaging directions are slightly misaligned, the imaging directions may be aligned by correcting the generated first two-dimensional data D21 and second two-dimensional data D22. (10) In the above embodiment, the second output data D42 is generated from the second three-dimensional data D32, but may be generated from the second two-dimensional data D22. (11) In the above first embodiment, the selection between generating the first three-dimensional data D31 and the second three-dimensional data D32 is performed automatically based on the first condition, but may also be performed manually. (12) In the above first embodiment, the third light receiving operation is not stopped when the first condition is satisfied, but the third light receiving operation may be stopped. (13) In the above second embodiment, the measurement system 50 generates the third three-dimensional data D33 from the first three-dimensional data D31 and the second three-dimensional data D32, and then generates the third output data D43, but this order is not limited to this. For example, the measurement system 50 may generate first output data D41 from the first three-dimensional data D31, generate second output data D42 from the second three-dimensional data D32, and generate third output data D43 from the first output data D41 and the second output data D42. (14) In the above embodiments, the first and second embodiments may be combined. For example, when the first condition of the first embodiment is satisfied, the measurement system 50 executes the second embodiment. Furthermore, the measurement system 50 may set a second condition and selectively use the first three-dimensional data D31, the second three-dimensional data D32, and the third three-dimensional data D33.

[0072] This international application claims priority based on Japanese Patent Application No. 2024-131898, filed on August 8, 2024, the entire contents of which are incorporated herein by reference.

[0073] The above descriptions of specific embodiments of the present invention have been presented for purposes of illustration. They are not intended to be exhaustive or to limit the invention to the precise forms described. Numerous modifications and variations will be apparent to those skilled in the art in light of the above description.

[0074] 1: Vehicle 10: Measuring device 40: Imaging device 45: Second imaging device 50: Measuring system 61: Low beam lamp unit 62: High beam lamp unit 70: Headlamp 90: External device 100: Light projector 110: Light projecting optical system 120: Light source unit 130: Current source 210: First control circuit board 300: First light receiver 310: First light receiving optical system 320: First light receiving unit 340: TOF measuring device 410: Second light receiving optical system 420: Second light receiving unit 450: Two-dimensional data generator 460: Second control circuit board 470: Second communication interface (I / F) 600: Information processing device 800: First communication interface (I / F) A: Point B1: Parallax B2: Distance between cameras C1: Focal length C2: Distance D21: First 2D data D22: Second 2D data D31: First 3D data D32: Second 3D data D33: Third 3D data D41: First output data D42: Second output data D43: Third output data F310: Center F410: Center I21: First 2D image I22: Second 2D image Lin: Incident light Ln1: First ambient light Ln2: Second ambient light Lout: Emitted light Lre: Reflected light P1: First focal point P2: Corresponding point Q: Second focal point R320: First light-receiving surface R420: Second light-receiving surface W: Measurement object

Claims

1. A measuring device comprising: a light projector that emits light; a first light receiver that receives light; and a controller, wherein the controller: causes the light projector to perform a light projecting operation of emitting light; causes the first light receiver to perform a first light receiving operation of receiving reflected light that is emitted from the light projector and reflected off an object to be measured; causes the first light receiver to perform a second light receiving operation of receiving external light without causing the light projector to perform the light projecting operation; outputs first output data based on first three-dimensional data acquired by the first light receiving operation; and outputs second output data based on first two-dimensional data acquired by the second light receiving operation.

2. A measuring device as claimed in claim 1, comprising an acquisition unit that causes a second light receiver, located at a different position from the first light receiver, to execute a third light receiving operation in which external light is received, and acquires second two-dimensional data acquired by the third light receiving operation, wherein the controller: acquires the second two-dimensional data from the second light receiver via the acquisition unit; generates second three-dimensional data based on the first two-dimensional data from the first light receiver and the second two-dimensional data from the second light receiver; and outputs second output data based on the second three-dimensional data.

3. A measuring device according to claim 2, wherein the light projector emits light in a first wavelength band, the first light receiver receives light in the first wavelength band and light in a second wavelength band different from the first wavelength band, and the second light receiver receives light in the second wavelength band.

4. A measuring device as claimed in claim 2 or claim 3, wherein the controller, when a first condition is satisfied, causes the light projector to execute the light projecting operation and causes the first light receiver to execute the first light receiving operation, and when the first condition is not satisfied, causes the light projector not to execute the light projecting operation and causes the first light receiver to execute the second light receiving operation.

5. A measurement system including a measuring device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light and is positioned differently from the first light receiver, includes the following steps: cause the light projector to perform a light projecting operation of emitting light, cause the first light receiver to perform a first light receiving operation of receiving reflected light that is emitted from the light projector and reflected back from the object to be measured, cause the first light receiver to perform a second light receiving operation of receiving external light without causing the light projector to perform the light projecting operation, cause the second light receiver to perform a third light receiving operation of receiving external light, output first output data based on first three-dimensional data acquired by the first light receiving operation, generate second three-dimensional data based on first two-dimensional data acquired by the second light receiving operation and second two-dimensional data acquired by the third light receiving operation, a control program for the measurement system that outputs second output data based on the second three-dimensional data.

6. A control method for a measurement system including a measurement device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light and is located at a position different from the first light receiver, the method comprising: causing the light projector to perform a light projecting operation of emitting light, causing the first light receiver to perform a first light receiving operation of receiving reflected light that is emitted from the light projector and reflected back from an object to be measured, causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light projector to perform the light projecting operation, causing the second light receiver to perform a third light receiving operation of receiving external light, outputting first output data based on first three-dimensional data acquired by the first light receiving operation, generating second three-dimensional data based on first two-dimensional data acquired by the second light receiving operation and second two-dimensional data acquired by the third light receiving operation, outputting second output data based on the second three-dimensional data.

7. A computer-readable recording medium having recorded thereon a computer program for controlling a measurement system including a measurement device having a light projector that emits light, a first light receiver that receives light, and a controller, and a second light receiver that receives light that is positioned differently from the first light receiver, the computer program comprising: causing the light projector to perform a light projecting operation of emitting light; causing the first light receiver to perform a first light receiving operation of receiving reflected light that is emitted from the light projector and reflected back from an object to be measured; causing the first light receiver to perform a second light receiving operation of receiving external light without causing the light projector to perform the light projecting operation; causing the second light receiver to perform a third light receiving operation of receiving external light; and outputting first output data based on first three-dimensional data acquired by the first light receiving operation. A computer-readable recording medium having a computer program recorded thereon that causes the measuring device to execute the following: generate second three-dimensional data based on first two-dimensional data acquired by the second light receiving operation and second two-dimensional data acquired by the third light receiving operation; and output second output data based on the second three-dimensional data.

Images