Light projection device, light projection-and-reception apparatus, and distance measurement system

Abstract

A light projection-and-reception apparatus includes: multiple light projection units to project pattern light onto an object, each of the multiple light projection units including: multiple light sources to emit light; and multiple optical systems to project the pattern light obtained from the light emitted from the multiple light sources to the object, the multiple optical systems respectively having multiple optical axes arranged non-parallel to each other to form an angle; and a light receiver to receive reflection light reflected from the object, to which the light is projected from the multiple light projection units.

Description

[DESCRIPTION][Title of Invention]LIGHT PROJECTION DEVICE, LIGHT PROJECTION- AND-RECEPTION APPARATUS, AND DISTANCE MEASUREMENT SYSTEM[Technical Field]

[0001] The present disclosure relates to a light projection device, a light projection-and-reception apparatus, and a distance measurement system.[Background Art]

[0002] A light projection device that projects pattern light (e.g., a dot pattern) to an object to measure the distance to the object by a time of flight (ToF) method is known. For example, PTL 1 describes a specific configuration of a light projection device of this type.

[0003] In the ToF camera system described in PTL 1, to distinguish direct incident light beams from indirect incident light beams, a pattern forming unit discretely changes modulation light emitted from an irradiation unit to a scene and emits the modulation light to elementary areas of the scene with different incident intensities.[Citation List][Patent Literature]

[0004] [PTL 1]Japanese Translation of PCT International Application Publication No. JP-T-2017-517737 [Summary of Invention][Technical Problem]

[0005] While distance information is acquired using pattern light in PTL 1, there is room for improvement in view of increasing the spatial resolution of a distance image using the pattern light.

[0006] The present disclosure is made in view of the above-described circumstances, and an object of the present disclosure is to provide a light projection device, a light projection-and-reception apparatus, and a distance measurement system that can increase the spatial resolution of a distance image using pattern light.[Solution to Problem]

[0007] According to an embodiment of the present disclosure, a light projection-and-reception apparatus includes: multiple light projection units to project pattern light onto an object, each of the multiple light projection units including: multiple light sources to emit light; andmultiple optical systems to project the pattern light obtained from the light emitted from the multiple light sources to the object, the multiple optical systems respectively having multiple optical axes arranged non-parallel to each other to form an angle; and a light receiver to receive reflection light reflected from the object, to which the light is projected from the multiple light projection units.According to another embodiment of the present disclosure, a distance measurement system includes the above-described light projection-and-reception apparatus; and a distance calculator that calculates a distance to the object based on an output as a result of reception of the light by the light receiver.According to further another embodiment of the present disclosure, a light projection device includes multiple light projection units. Each of the light projection units includes a light source that emits a light beam; and an optical system that projects pattern light obtained from the light beam emitted from the light source to an object. The multiple light projection units are disposed non-parallel to each other to have optical axes forming an angle.[Advantageous Effects of Invention]

[0008] With the embodiments of the present disclosure, the light projection device, the light projection-and-reception apparatus, and the distance measurement system that can increase the spatial resolution of a distance image using pattern light are provided.[Brief Description of Drawings]

[0009] A more complete appreciation of embodiments of the present disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings.[FIG. 1]FIG. l is a block diagram illustrating a configuration of a distance measurement system according to an embodiment of the present disclosure.[FIG. 2]FIG. 2 is a block diagram illustrating a configuration of a light projection-and-reception unit according to the embodiment of the present disclosure.[FIG. 3]FIG. 3 is a diagram illustrating an example of ToF imaging by a light projection-and-reception apparatus according to the embodiment of the present disclosure.[FIG. 4]FIG. 4 is a schematic configuration diagram illustrating an implementation example of a light projection device.[FIG. 5]FIG. 5 is a schematic configuration diagram illustrating an implementation example of the light projection-and-reception apparatus.[FIG. 6A]FIG. 6A is a diagram illustrating the relationship between the optical axes of light projection units and dot-pattern light emitted to an object according to the embodiment of the present disclosure.[FIG. 6B]FIG. 6B is a diagram illustrating the relationship between the optical axes of light projection units and dot-pattern light emitted to an object according to another embodiment of the present disclosure.[FIG. 6C]FIG. 6C is a diagram illustrating the arrangement relationship between light projection units according to another embodiment of the present disclosure.[FIG. 7 A]FIG. 7A is a view illustrating the relationship between the optical axes of the light projection units and dot-pattern light emitted to an object according to the embodiment of the present disclosure.[FIG. 7B]FIG. 7B is a view illustrating the relationship between the optical axes of light projection units and dot-pattern light emitted to an object according to another embodiment of the present disclosure.[FIG. 8]FIG. 8 is a diagram illustrating the relationship between projected dot-pattern light and the distance to an object according to the embodiment of the present disclosure.[FIG. 9]FIG. 9 is a view illustrating a holding mechanism according to the embodiment of the present disclosure.[FIG. 10]FIG. 10 is a view illustrating a holding mechanism according to a first modification of the present disclosure.[FIG. 11 A]FIG. 11 A is a diagram illustrating a change in dot-pattern light when an optical-axis adjustment mechanism according to the first modification of the present disclosure changes the relative angle between a pair of optical axes.[FIG. 11B]FIG. 1 IB is a diagram illustrating dot-pattern light emitted to an object at a long distance according to another embodiment of the present disclosure.[FIG. 12 A]FIG. 12A is a view illustrating an optical -axis adjustment mechanism according to a second modification of the present disclosure.[FIG. 12B]FIG. 12B is a diagram illustrating the optical-axis adjustment mechanism according to the second modification of the present disclosure.[FIG. 12C]FIG. 12C is a diagram illustrating dot-pattern light emitted to an object at a long distance according to another embodiment of the present disclosure.[FIG. 13 A]FIG. 13 A is a view illustrating the optical -axis adjustment mechanism according to the second modification of the present disclosure.[FIG. 13B]FIG. 13B is a diagram illustrating the optical-axis adjustment mechanism according to the second modification of the present disclosure.[FIG. 14]FIG. 14 is a block diagram illustrating a configuration of a distance measurement system according to a third modification of the present disclosure.[FIG. 15]FIG. 15 is a flowchart of processing executed according to the third modification of the present disclosure.[FIG. 16]FIG. 16 is a flowchart of processing executed according to a fourth modification of the present disclosure.[FIG. 17]FIG. 17 is a view illustrating another arrangement example of the optical elements of the light proj ection-and-reception apparatus.[FIG. 18]FIG. 18 is a block diagram illustrating a configuration of a three-dimensional shape generation system according to another embodiment of the present disclosure.[FIG. 19]FIG. 19 is a diagram illustrating an example in which a distance measurement system according to another embodiment of the present disclosure is applied to a portable information terminal.[FIG. 20]FIG. 20 is a diagram illustrating an example in which a distance measurement system according to another embodiment of the present disclosure is applied to an autonomous moving system of a moving body.[FIG. 21]FIG. 21 is a schematic configuration diagram illustrating an implementation example according to another embodiment of the present disclosure.[FIG. 22]FIG. 22 is a view illustrating an optical-axis adjustment mechanism according to another embodiment of the present disclosure.[FIG. 23]FIG. 23 is a view illustrating an example in which a distance measurement system according to another embodiment of the present disclosure is applied to a portable information terminal. The accompanying drawings are intended to depict embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted. Also, identical or similar reference numerals designate identical or similar components throughout the several views. [Description of Embodiments]

[0010] In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.Referring now to the drawings, embodiments of the present disclosure are described below.As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.Embodiments of the present disclosure will be described below with reference to the drawings. In the description, like reference signs denote like elements, and redundant description may be simplified or omitted as appropriate.

[0011] FIG. l is a block diagram illustrating a configuration of a distance measurement system 1 according to an embodiment of the present disclosure.

[0012] As illustrated in FIG. 1, the distance measurement system 1 includes a light project! on-and- reception apparatus 2 and a computing device 3.

[0013] The distance measurement system 1 according to the present embodiment measures the distance from the light projection-and-reception apparatus 2 to an object (referred to as an "object OB") by a ToF method.In the ToF method, distance measurement light (e.g., a laser beam) having a wavelength (e.g., 940 nm) different from the wavelength of visible light is emitted to the object OB. The distance to each portion (i.e., each irradiation position) of the object OB is calculated based on the time difference between the emission timing and the reception timing of the laser beam at each irradiation position.

[0014] In the present embodiment, a diffractive optical element is used in a light projection optical system that projects the laser beam. When pattern light such as a dot pattern is projected to the object OB using the diffractive optical element, point group data with high brightness and high density can be acquired. Hereinafter, the diffractive optical element is referred to as a "diffractive optical element (DOE)". Pattern light is a collection of light having discrete light intensities in a space. The shape of light emitted to a plane may be any one of a circular shape, a polygonal shape, and a stripe shape.

[0015] The light projection-and-reception apparatus 2 includes a light projection-and-reception unit 10 and a red-green-blue (RGB) light receiver 20. The "light projection-and-reception apparatus" may be also referred to as, for example, a "light projection apparatus", an "imaging apparatus", and a "distance measurement apparatus".

[0016] The light projection-and-reception apparatus 2 includes, for example, a rechargeable battery. In other words, the light projection-and-reception apparatus 2 is driven by a battery. The light projection-and-reception apparatus 2 may be driven by a commercial power supply.

[0017] The light projection-and-reception unit 10 includes multiple light projection units 100 A and 100B, a ToF light receiver 120, and a controller 140. Hereinafter, the light projection unit 100 A and the light projection unit 100B may be collectively referred to as a "light projection unit 100".

[0018] The light projection-and-reception unit 10 may include three or more light projection units instead of the two light projection units 100 A and 100B. Since the two light projection units 100 of the light projection unit 100 A and the light projection unit 100B are included in the present embodiment, the two light projection units 100 may be referred to as a "pair of light projection units 100".

[0019] FIG. 2 is a block diagram of the light projection-and-reception unit 10.

[0020] The light projection unit 100 (i.e., each of the light projection units 100A and 100B) includes a light source 102, a first lens group 104, a DOE 106, and a second lens group 108. The light projection unit 100 projects the pattern light to the object OB.

[0021] Any reference to elements using expressions such as "first", "second", and so forth used in the present disclosure does not generally limit the quantity or order of those elements. These expressions are used for convenience to distinguish between two or more elements. Thus, reference to first and second elements does not mean, for example, that two elements are employed and that the first element precedes the second element.

[0022] The controller 140 controls the light projection unit 100 and the ToF light receiver 120. For example, the controller 140 includes, as a circuit configuration, a central processing unit (CPU), a light source drive circuit, an imaging signal processing circuit, an input / output circuit, and a memory.

[0023] The controller 140 is, for example, a single processor or a multiprocessor, and includes at least one processor. In the case of the configuration including multiple processors, the controller 140 may be packaged as a single device, or may be physically separated into multiple devices in the light projection-and-reception unit 10.

[0024] The light source 102 is an example of a light source that emits at least one light beam. The light source 102 is, for example, a laser diode (LD) that emits a laser beam. The light source 102 emits the laser beam at a timing controlled by the controller 140.

[0025] The first lens group 104, the DOE 106, and the second lens group 108 are examples of components in an optical system that projects pattern light obtained from the laser beam emitted from the light source 102 to an object.

[0026] The first lens group 104 is an example of a first lens group. The laser beam emitted from the light source 102 goes in the first lens group 104.

[0027] The DOE 106 is an example of an optical element that obtains pattern light (forms pattern light) in which multiple emission light are regularly arranged from the laser beam going in the DOE 106 through the first lens group 104. The DOE 106 emits pattern light (hereinafter referred to as "dot-pattern light") in which multiple dots are regularly arranged (e.g., in a lattice shape). Hereinafter, the dots are an example of each emission light included in the pattern light.

[0028] The second lens group 108 is an example of a second lens group. The dot-pattern light emitted from the DOE 106 goes in the second lens group 108. The second lens group 108 projects the dot-pattern light to the object OB.

[0029] FIG. 3 is a diagram illustrating an example of ToF imaging performed by the light projection- and-reception apparatus 2. In FIG. 3, for the convenience of simple description of ToF imaging, one light projection unit 100 (one of the light projection units 100A and 100B) is illustrated, and the illustration of the RGB light receiver 20 is omitted.

[0030] As illustrated in FIG. 3, the light projection-and-reception apparatus 2 projects the dot-pattern light to the object OB.To project this type of pattern light, for example, it is conceivable to configure the light source 102 with a vertical cavity surface emitting laser (VCSEL) including multiple light emitters arranged in a pattern, to form a pattern in a dot shape using a diffraction phenomenon of a fine structure of the DOE 106, or to apply both. Accordingly, the dot-pattern light with high brightness and high density can be projected to the object OB, and the distance measurement accuracy is increased.

[0031] Specifically, the case where the diffraction pattern of the DOE 106 is formed in a dot pattern will be described below. In this case, the light source 102 is configured by a point light source such as a LD. The laser beam emitted from the light source 102 goes in the DOE 106 through the first lens group 104.

[0032] When the laser beam goes in the DOE 106, conjugate points having the number corresponding to the diffraction order (Oth order, ±lst order, ±2nd order, ...) of the DOE 106 occur, and the dot-pattern light corresponding to the diffraction order is emitted to infinity.

[0033] When the VCSEL and the DOE 106 are used in combination, the dot-pattern light can be emitted in a wide range.

[0034] The dot-pattern light emitted to the object OB is reflected or scattered by the object OB. The ToF light receiver 120 receives light directly reflected from the object OB (hereinafter referred to as "direct reflection light").

[0035] The ToF light receiver 120 is an example of a light receiver that receives the reflection light from the object OB to which the dot-pattern light is projected. As illustrated in FIG. 2, the ToF light receiver 120 includes an optical system 122 and a ToF sensor 124.

[0036] The optical system 122 includes, for example, an aperture, an imaging optical system, and a filter. The direct reflection light reflected from the object OB to which the dot-pattern light is emitted passes through the optical system 122 and is received by the ToF sensor 124.

[0037] The ToF sensor 124 is an image sensor such as a complementary metal oxide semiconductor (CMOS) image sensor, and photoelectrically converts the sum of exposure amounts in multiple exposure periods having a predetermined phase difference for emission light and outputs the result as light reception data to the controller 140. The light reception data output by the ToF sensor 124 is input into the computing device 3 via the controller 140.

[0038] A computing unit 31 of the computing device 3 is implemented by a command from the CPU of the computing device 3, and calculates the distance to each portion (i.e., each irradiation position) of the object OB based on the sum of the exposure amounts in the exposure periods input from the ToF sensor 124. A single photon avalanche diode (SPAD) or the like may be used as the ToF sensor 124, and the computing unit 31 may calculate the distance based on the time difference between the emission timing of the laser beam (the emission timing of the light source 102) and the reception timing of the laser beam (the input timing from the ToF sensor 124) at each irradiation position.

[0039] In other words, the computing unit 31 of the computing device 3 is an example of a distance calculator that calculates the distance to the object OB based on an output as a result of reception of the light by the ToF light receiver 120.

[0040] The RGB light receiver 20 includes, for example, an aperture, an imaging optical system, a filter, and an image sensor. The image sensor is, for example, a CMOS image sensor and includes an RGB color filter.

[0041] The image sensor may be replaced with another type of image sensor such as a charge coupled device (CCD) image sensor. The image sensor may include a complementary color filter having a checkered pattern.

[0042] The image sensor is driven under the control of the controller 140 and receives visible light (i.e., natural light) on the light reception surface. The image sensor accumulates the electric charges corresponding to the light intensity at each pixel of the light reception surface on which an optical image is formed, and outputs the electric charges at a timing synchronized with, for example, the ToF imaging. The controller 140 outputs RGB image data based on each pixel data to the computing device 3.

[0043] In the example of the configuration illustrated in FIG. 1, the light projection-and-reception apparatus 2 includes the light projection-and-reception unit 10. As illustrated in FIG. 2, the light projection-and-reception unit 10 may be configured as a single device independent of the light projection-and-reception apparatus 2.

[0044] The light projection unit 100 (i.e., a light projection device) may be configured as a single device independent of the light projection-and-reception unit 10. FIG. 4 is a block diagram of a light projection device 10a configured as a single device.

[0045] As illustrated in FIG. 4, the light projection device 10a includes the light projection units 100A and 100B and the controller 140. In other words, in the example of this configuration,the light projection device 10a has a configuration excluding the ToF light receiver 120 from the light projection-and-reception unit 10 illustrated in FIG. 2.

[0046] The light projection device 10a may not include the controller 140. In other words, the light projection device 10a may be an optical device including the light projection units 100 A and 100B each including optical components including the light source 102, the first lens group 104, the DOE 106, and the second lens group 108. In this case, for example, a signal sending component such as a cable is connected to a terminal disposed in the light projection device 10a. As a result, the light source 102 and the controller 140 connected to another end of the cable are connected to each other via the signal sending component.

[0047] In the example of the configuration illustrated in FIG. 1, the light projection-and-reception apparatus 2 includes the RGB light receiver 20. The light projection-and-reception apparatus 2 may not include the RGB light receiver 20.

[0048] The computing device 3 is, for example, a terminal device such as a personal computer (PC) or a server located on a cloud. The light projection-and-reception apparatus 2 and the computing device 3 can communicate with each other through wired communication or wireless communication via a transmitter-and-receiver disposed in each of the light projection-and-reception apparatus 2 and the computing device 3. Data may be transmitted (output) from the light projection-and-reception apparatus 2 to the computing device 3 via a network. The transmitter-and-receiver may be configured by an interface circuit for a portable storage medium such as a secure digital (SD) card or a PC.

[0049] In another embodiment, the light projection-and-reception apparatus 2 may include the computing unit 31. In this case, the computing unit 31 may be configured as, for example, a processor included in the controller 140. The computing unit 31 may be configured as a processor independent of the controller 140.

[0050] In still another embodiment, the light projection-and-reception apparatus 2 may include the entire computing device 3.

[0051] As described above, the configuration of each component of the distance measurement system 1 has a degree of freedom in design, and the design can be changed in various ways.

[0052] FIG. 5 is a schematic configuration diagram illustrating an implementation example of the light projection-and-reception apparatus 2. In the implementation example illustrated in FIG. 5, the light projection-and-reception apparatus 2 is configured as an apparatus that can take a full-spherical panoramic image. The full-spherical panoramic image is a panoramic imageobtained by taking an image of the range of the full sphere by obtaining an image within a solid angle of 4K steradians. The light projection-and-reception apparatus 2 illustrated in FIG. 5 is driven by a battery.

[0053] The light projection-and-reception apparatus 2 illustrated in FIG. 5 includes a pair of light projection units 100 A and 100B on one side portion, and a pair of light projection units 100 A and 100B on another side portion of the apparatus. The light projection-and-reception apparatus 2 includes a wide-angle ToF light receiver 120 for obtaining hemispherical distance images in an upper portion of the apparatus.

[0054] The light projection-and-reception apparatus 2 includes a pair of wide-angle RGB light receivers 20 for obtaining hemispherical color images on side portions of the apparatus. In FIG. 5, one of the pair of RGB light receivers 20 is located on the front surface of the apparatus. Another one of the pair of RGB light receivers 20 is located on the back surface of the apparatus. Since the RGB light receiver 20 on the back surface of the apparatus is invisible in FIG. 5, the RGB light receiver 20 does not appear in the drawing.

[0055] The pair of RGB light receivers 20 take images of an object (e.g., the object OB) around the light projection-and-reception apparatus 2. Accordingly, a pair of hemispherical images are obtained. The controller 140 combines the pair of hemispherical images to generate, for example, a full-spherical panoramic image expressed in Mercator projection.

[0056] The ToF light receiver 120 receives the direct reflection light from the object OB irradiated with the dot-pattern light emitted from the light projection units 100 A and 100B. The computing device 3 calculates the distance to each portion (i.e., each irradiation position) of the object OB based on the time difference between the emission timing and the reception timing of the laser beam at each irradiation position. From the calculation result, distance information on the full-spherical range corresponding to the full -spherical panoramic image is obtained. In other words, a three-dimensional point group that is an aggregate of coordinate points in a three-dimensional space can be acquired. Color information (e.g., RGB values of each coordinate point) may be added to each coordinate point of the point group.

[0057] As illustrated in FIG. 5, the light projection unit 100A and the light projection unit 100B are arranged non-parallel to each other so that the optical axis AXA of the light projection unit 100A and the optical axis AXB of the light projection unit 100B form an angle (relative angle 0). The reason for the arrangement will be described in detail.

[0058] FIG. 6A and FIG. 7A are a diagram and a view illustrating the relationship between the optical axes AXA and AXB of the light projection units 100 A and 100B and the dot-patternlight emitted to the object OB. In the subsequent drawings illustrating the dot-pattern light (except for FIG. 7A), the dot-pattern light projected from the light projection unit 100A is indicated by dotted lines and white circles. The dot-pattern light projected from the light projection unit 100B is indicated by one-dot chain lines and black circles. In FIG. 7A, the object OB is illustrated in black, and dots emitted on the object OB are illustrated in white.

[0059] In FIG. 7A, dot-pattern light DPA, dot-pattern light DPB, and dot-pattern light DPC are emitted on the object OB. The dot-pattern light DPA indicates the dot-pattern light emitted from the light projection unit 100 A on the object OB. The dot-pattern light DPB indicates the dot-pattern light emitted from the light projection unit 100B on the object OB. Dots DA are individual dots (white circles in FIG. 6A) of the dot-pattern light DPA. Dots DB are individual dots (black circles in FIG. 6 A) of the dot-pattern light DPB.

[0060] In FIG. 7A, the dot-pattern light DPC indicates dot-pattern light emitted from a light projection unit according to a comparative example on the object OB.

[0061] As illustrated in FIG. 6 A, the light projection unit 100 A and the light projection unit 100B are disposed so that the optical axis AXA and the optical axis AXB form a relative angle 0. Hence the dot-pattern light emitted from the light projection unit 100 A and the dot-pattern light emitted from the light projection unit 100B are emitted at positions shifted from each other on the object OB. Specifically, the dots DA (an example of first dots) and the dots DB (an example of second dots) are emitted so as to be alternately and periodically arranged on the object OB.

[0062] To increase the spatial resolution of the distance image with the pattern light, it is conceivable to increase the density of dots. However, when the density of dots is increased, adjacent dots are emitted so as to overlap each other, for example, on an object OB at a short distance as in the dot-pattern light DPC of the comparative example of FIG. 7 A. To perform processing of removing the influence of indirect light beams as disclosed in PTL 1, an area in which light does not go in between dots is used. When adjacent dots are emitted so as to overlap each other, appropriate distance information cannot be obtained.

[0063] Thus, the light projection-and-reception apparatus 2 according to the present embodiment sequentially emits the dot-pattern light DPA and the dot-pattern light DPB. The light projection-and-reception apparatus 2 sequentially receives the direct reflection light from the object OB irradiated with the dot-pattern light DPA and the direct reflection light from the object OB irradiated with the dot-pattern light DPB using the ToF light receiver 120, and outputs the light reception data to the computing device 3. The computing unit 31 of the computing device 3 sequentially calculates distance data A corresponding to the dot-patternlight DPA and distance data B corresponding to the dot-pattern light DPB, and merges the calculated distance data A and distance data B.

[0064] Depending on the distance of the object OB, the dot-pattern light DPA and the dot-pattern light DPB do not overlap each other. Thus, the light projection-and-reception apparatus 2 may simultaneously emit the dot-pattern light DPA and the dot-pattern light DPB.

[0065] For example, the irradiation positions of the dots of the dot-pattern light DPA are shifted from the irradiation positions of the dots of the dot-pattern light DPB by a half pitch. Thus, the distance information between the dots DA of the dot-pattern light DPA is interpolated with the distance information corresponding to the dots DB of the dot-pattern light DPB located between the dots DA. In other words, the distance information between the dots DB is interpolated with the distance information corresponding to the dots DA located between the dots DB.

[0066] As described above, with the present embodiment, while dots are emitted so as not to overlap each other even on an object OB at a short distance, a distance image with high spatial resolution equivalent to the spatial resolution in the case where high-density dot-pattern light is emitted on the object OB can be obtained.

[0067] With the present embodiment, the spatial resolution of the distance image with the dot-pattern light can be increased.

[0068] The arrangement of the dot-pattern light DPA and the dot-pattern light DPB is not limited to the square lattice form. As illustrated in FIG. 7B, the dot-pattern light DPA and the dotpattern light DPB may be arranged in another form such as a hexagonal lattice form. The light projection unit 100A and the light projection unit 100B may be disposed to be shifted in the left-right direction so that the dot-pattern light DPA and the dot-pattern light DPB are shifted in the lateral direction in the drawing.

[0069] FIG. 8 is a diagram illustrating the relationship between the dot-pattern light DPA (dotted lines) and the dot-pattern light DPB (one-dot chain lines), and the distance to the object OB. In FIG. 8, the distance (i.e., an irradiation distance) to the object OB from the light projection- and-reception apparatus 2 includes distances Nlm to N5m. The distances Nlm, N2m, and N3m are less than 1 m. The distances N4m and N5m exceed 1 m.

[0070] As illustrated in FIG. 8, in a short distance range, the positional relationship between the dots DA and the dots DB may change, or the dots DA and the dots DB may be emitted so as to overlap each other depending on the irradiation distance. Specifically, while the emitted dotsDA, DB, DA, ... are arranged in order from the top at the distance Nlm, the dots DB, DA,DB, ... are arranged in order from the top at the distance N3m. The dots DA and the dots DB are emitted so as to substantially overlap each other at the distance N2m.

[0071] At a short distance, the position difference between the emission spots (the position difference D between the light projection unit 100A and the light projection unit 100B) is not negligible with respect to the irradiation distance. Since the optical axis AXA and the optical axis AXB form the relative angle 0 and the position difference D between both emission spots has a non- negligible magnitude, the positional relationship between the dots DA and the dots DB may change, or the dots DA and the dots DB may be emitted so as to overlap each other in the short distance range depending on the irradiation distance.

[0072] In contrast, at a long distance, the position difference D between the emission spots is very small with respect to the irradiation distance. At the long distance, the position difference D between the emission spots is substantially negligible, and emission can be regarded as emission from substantially the same spot. Thus, the positional relationship between the dots DA and the dots DB is constant.

[0073] In other words, in a long distance range, the positional relationship between the dots DA and the dots DB is constant regardless of the irradiation distance. Specifically, the emitted dots DB, DA, DB, ... are arranged in order from the top at any one of the distances N4m and N5m.

[0074] However, as the distance increases, the interval between the dots increases, and the spatial resolution with the dot-pattern light decreases. To obtain high spatial resolution at the distance at which the object OB is located, it is desirable to appropriately set the relative angle 9 between the optical axis AXA and the optical axis AXB and the position difference D between the light projection unit 100 A and the light projection unit 100B.

[0075] For example, at distances less than 1 m from the light projection-and-reception apparatus 2, information that is not used, such as information on the photographer holding the light projection-and-reception apparatus 2 or a securing tool (e.g., a tripod) of the light projection- and-reception apparatus 2, is often included. Hence it is less likely to increase the spatial resolution in this distance range.

[0076] Thus, in the example of FIG. 8, to increase the spatial resolution in the distance range of 1 m or more, the relative angle 9 and the position difference D are set so that the positional relationship between the dots DA and the dots DB is constant while the dots DA and the dots DB are emitted so as not to overlap each other.

[0077] The light projection unit 100A and the light projection unit 100B are disposed so that the optical axis AXA and the optical axis AXB intersect with each other in an emission side (direction) of light in the example of FIG. 6A. The” emission side of the light” is the right side of the light projection unit 100 A and the light projection unit 100B in FIG. 6 A. Alternatively, for example, as illustrated in FIG. 6B, the light projection unit 100 A and the light projection unit 100B may be disposed in orientations in which the optical axis AXA and the optical axis AXB do not intersect with each other in the emission side (direction) of light. In this case, an area in which the dot pattern is sparse is likely to be generated at an end of the irradiation range of the light projection unit 100 A and the light projection unit 100B; however, a wide irradiation range can be ensured.

[0078] In the example of FIG. 6A and FIG. 6B, the light projection unit 100A and the light projection unit 100B are disposed side by side in the vertical direction in the drawing, and the light projection unit 100A and the light projection unit 100B are disposed to be inclined with respect to the horizontal direction perpendicular to the vertical direction so that the optical axis AXA and the optical axis AXB form the relative angle 0. Alternatively, for example, as illustrated in FIG. 6C, one of the light projection unit 100 A and the light projection unit 100B may be disposed in a manner that the optical axis is parallel to the horizontal direction and another one of the light projection unit 100 A and the light projection unit 100B may be disposed in a manner that the optical axis is inclined with respect to the horizontal direction so that the optical axis AXA and the optical axis AXB form the relative angle 9. With such a configuration, for example, processing of a component such as a housing that holds the lens barrel is simplified, and an advantage such as a reduction in cost is obtained. In the example of FIG. 6C, the optical axis AXA and the optical axis AXB intersect with each other. As another example, the optical axis AXA and the optical axis AXB may not intersect with each other as illustrated in FIG. 6B. When the optical axis AXA and the optical axis AXB are inclined as illustrated in FIGs. 6A and 6B, the inclination of the optical axis AXA with respect to the horizontal direction and the inclination of the optical axis AXB with respect to the horizontal direction may be different from each other.

[0079] FIG. 9 is a view illustrating a holding mechanism of the light projection unit 100A and the light projection unit 100B. As illustrated in FIG. 9, the light projection unit 100 A including the light source 102, the first lens group 104, the DOE 106, and the second lens group 108 is held by a lens barrel 160A. Similarly, the light projection unit 100B including the light source 102, the first lens group 104, the DOE 106, and the second lens group 108 is held by a lens barrel 160B.

[0080] FIG. 9 illustrates a portion of a housing 180 of the light projection-and-reception apparatus 2. As illustrated in FIG. 9, the lens barrels 160A and 160B are fastened and secured to the housing 180 with screws 170.

[0081] The lens barrels 160A and 160B, the screws 170, and the housing 180 define the relative angle 0 and the position difference D so that the positional relationship between the dots DA and the dots DB is constant in the distance range of 1 m or more while the dots DA and the dots DB are emitted so as not to overlap each other.

[0082] As described above, the lens barrels 160A and 160B, the screws 170, and the housing 180 serve as a setting unit that sets (defines) the relative angle 9 formed by the optical axis AXA and the optical axis AXB and the position difference D between the multiple light projection units 100 so that the dot-pattern light projected from the multiple light projection units 100 are emitted so as not to overlap each other on an object OB separated from the light projection-and-reception apparatus 2 by 1 m (an example of a first distance) or more.

[0083] In other words, the dots DA (an example of first dots) projected from the light projection unit 100 A (one of a pair of light projection units 100, an example of one of multiple light projection units) and the dots DB (an example of second dots) projected from the light projection unit 100B (another one of the pair of light projection units 100, an example of another one of the multiple light projection units) are emitted so as to at least partially overlap each other on an object OB located at a second distance (e.g., the distance N2m of FIG. 8) shorter than 1 m (the example of the first distance) from the light projection-and-reception apparatus 2. The dots DA and the dots DB are emitted so that the dots DA and the dots DB do not overlap each other and the dots DA and the dots DB are alternately and periodically arranged on an object OB located at a third distance (e.g., the distance N4m of FIG. 8) longer than Im (the example of the first distance) from the light projection-and-reception apparatus 2.

[0084] The exemplary embodiments of the present disclosure have been described above. While the embodiments of the present disclosure have been described above, various modifications may be made within the scope of the technical idea of the present disclosure without being limited to the above-described configurations. For example, the embodiments of the present application also include contents obtained by appropriately combining the embodiments explicitly described in the description or the obvious embodiments.

[0085] FIG. 10 is a view illustrating an optical-axis adjustment mechanism according to a first modification of the present disclosure. FIG. 10 is a view similar to FIG. 9 and illustrates aholding mechanism of the light projection unit 100A and the light projection unit 100B according to the first modification.

[0086] The optical-axis adjustment mechanism according to the first modification can adjust the relative angle 0.

[0087] Specifically, a coil spring 172 is disposed between each of the lens barrels 160A and 160B and the housing 180. A screw 170 is inserted through the coil spring 172. The screw 170 fastens each of the lens barrels 160A and 160B to the housing 180. The coil spring 172 is compressed between each of the lens barrels 160 A and 160B and the housing 180. The total length (the length in the axial direction) of the coil spring 172 changes depending on the degree of fastening of the screw 170.

[0088] The screw 170 and the coil spring 172 are included in an adjustment screw 174. A pair of adjustment screws 174 are provided for each of the lens barrels 160A and 160B. A user can change the posture of the light projection unit 100 with respect to the housing 180 (i.e., the orientation of the optical axis) by individually adjusting the degree of fastening of the screw 170 of each adjustment screw 174. By changing the difference in posture between the light projection unit 100A and the light projection unit 100B, the relative angle 9 is changed.

[0089] The coil spring 172 may be replaced with a component having another form such as a shim spacer. In other words, the configuration of the optical -axis adjustment mechanism has a degree of freedom in design, and the design can be changed in various ways.

[0090] FIG. 11 A is a diagram illustrating a change in dot-pattern light DPA (dotted lines) and dotpattern light DPB (one-dot chain lines) when the optical-axis adjustment mechanism according to the first modification changes the relative angle 9.

[0091] When the relative angle 9 is adjusted to the angle illustrated in an example EXI of FIG. 11 A, the irradiation positions of the dots DA and the dots DB on an object OB at a long distance have substantially equal pitches. Thus, the spatial resolution of the distance image can be increased for the object OB at the long distance.

[0092] When the relative angle 9 is adjusted to the angle illustrated in an example EX2 of FIG. 11 A, the irradiation positions of the dots DA and the dots DB on an object OB at a short distance have substantially equal pitches. Thus, the spatial resolution of the distance image can be increased for the object OB at the short distance.

[0093] As described above, in the first modification, the user operates the optical-axis adjustment mechanism that is an example of an adjuster to adjust the relative angle 9, thereby increasing the spatial resolution of the distance image for a target object OB.

[0094] In the example EX2 of FIG. 11 A, the dot DA and the dot DB are close to each other at the long distance. When dot light has a divergence angle, for example, as illustrated in FIG. 11B, the diameter of the dot increases as the distance increases. Thus, when the diameters of the dot DA and the dot DB are larger than the distance between the dot DA and the dot DB at the long distance, the light projection areas of the dot DA and the dot DB have an interference area I in which the light projection areas of the dot DA and the dot DB interfere with each other. In this case, when the dot-pattern light DPA and the dot-pattern light DPB are simultaneously projected, the intensity of projection light increases in the interference area I. Using this phenomenon, it is possible to increase the SN ratio at a long distance where a decrease in SN ratio due to a decrease in light intensity has been a disadvantage.

[0095] For example, the light projection-and-reception apparatus 2 may have a long distance mode in which the dot-pattern light DPA and the dot-pattern light DPB are simultaneously projected after the relative angle 9 is adjusted so that the diameters of the dot DA and the dot DB are larger than the distance between the dot DA and the dot DB at a target long distance.

[0096] In the above-described embodiment, the optical axis AXA and the optical axis AXB form the relative angle 9 in the vertical direction in the drawing, in other words, the optical axis AXA and the optical axis AXB are inclined toward the vertical direction from the horizontal direction in the drawing and are parallel to the plane defined by the vertical direction and the horizontal direction in the drawing. Alternatively, for example, as illustrated in FIG. 21, the optical axis AXA and the optical axis AXB may form a relative angle 9 in the horizontal direction in the drawing. Still alternatively, the optical axis AXA and the optical axis AXB may form a relative angle 9 in a two-degree-of-freedom system in the vertical direction and the horizontal direction in the drawing. In other words, the optical axis AXA and the optical axis AXB may be inclined toward the vertical direction from the horizontal direction and also non-parallel to the plane defined by the vertical direction and the horizontal direction. Even with these configurations, an advantageous effect similar to that of the above-described embodiment is obtained. The vertical direction is an example of a first axis, and the horizontal direction is an example of a second axis.

[9997] FIG. 12A, FIG. 12B, FIG. 13A, and FIG. 13B are views and diagrams illustrating an optical- axis adjustment mechanism according to a second modification of the present disclosure. In these drawings, for convenience of simplifying the description, the optical axis AXA and the optical axis AXB are illustrated so as to be parallel to each other.

[0098] The optical-axis adjustment mechanism according to the second modification can adjust the position difference D between the light projection unit 100 A and the light projection unit 100B.

[0099] Specifically, in the second modification, a pair of long holes 182A and 182B are formed in the housing 180. The long hole 182A and the long hole 182B have shapes extending in the same direction. The light projection units 100 A and 100B are supported by the housing 180 so as to be slidable in the longitudinal directions of the long holes 182A and 182B, respectively.

[0100] The light projection units 100 A and 100B slide in the longitudinal directions of the long holes 182A and 182B, respectively, by a driving unit such as a motor or a manual operation by the user. Accordingly, the position difference D between the light projection units 100 A and 100B decreases (see FIG. 12 A) or increases (see FIG. 13 A).

[0101] When the position difference D is decreased, for example, as illustrated in FIG. 12B, the irradiation positions of the dots DA and the dots DB on an object OB at a short distance have substantially equal pitches. Thus, the spatial resolution of the distance image can be increased for the object OB at the short distance.

[0102] When the diameters of the dot DA and the dot DB are larger than the position difference D at a certain distance or more, for example, as in the example of FIG. 11B, the light projection areas of the dot DA and the dot DB have an interference area I in which the light projection areas of the dot DA and the dot DB interfere with each other (see FIG. 12C). In this case, when the dot-pattern light DPA and the dot-pattern light DPB are simultaneously projected, the intensity of projection light increases in the interference area I. Using this phenomenon, it is possible to increase the SN ratio at a long distance where a decrease in SN ratio due to a decrease in light intensity has been a disadvantage.

[0103] For example, the light projection-and-reception apparatus 2 may have a long distance mode in which the dot-pattern light DPA and the dot-pattern light DPB are simultaneously projected after the position difference D is decreased so that the diameters of the dot DA and the dot DB are larger than the position difference D at a target long distance.

[0104] When the position difference D is increased, for example, as illustrated in FIG. 13B, the irradiation positions of the dots DA and the dots DB on the object OB at a long distance have substantially equal pitches. Thus, the spatial resolution of the distance image can be increased for objects OB at a long distance and a short distance.

[0105] Scales 184 indicating irradiation distances are engraved on sides of the long holes 182A and 182B. In the case of the manual operation, the user adjusts the position difference D while visually checking the scales 184, thereby increasing the spatial resolution of the distance image for the object OB at a target distance.

[0106] In other words, the user operates the optical -axis adjustment mechanism that is an example of the adjuster to adjust the position difference D between the light projection unit 100A and the light projection unit 100B, thereby increasing the spatial resolution of the distance image for a target object OB.

[0107] In the above-described embodiment, the light projection unit 100 A and the light projection unit 100B are disposed with the position difference D in the vertical direction in the drawing. Alternatively, for example, as illustrated in FIG. 22, an adjustment mechanism that can adjust the relative positions of the light projection unit 100 A and the light projection unit 100B in the horizontal direction in the drawing may be employed. Still alternatively, an adjustment mechanism of a two-degree-of-freedom system that can adjust the relative positions of the light projection unit 100A and the light projection unit 100B in the vertical direction and the horizontal direction in the drawing may be employed. Even with these configurations, an advantageous effect similar to that of the above-described embodiment is obtained.

[0108] The optical-axis adjustment mechanism according to the first modification and the optical- axis adjustment mechanism according to the second modification may be combined as appropriate. In other words, the light projection-and-reception apparatus 2 can adjust both the relative angle 0 and the position difference D.

[0109] FIG. 14 is a block diagram illustrating a configuration of a distance measurement system 1 according to a third modification of the present disclosure.

[0110] As illustrated in FIG. 14, in the third modification, a controller 140 includes an adjuster 141. The adjuster 141 controls a driving unit (a motor or the like) that drives the optical-axis adjustment mechanism (see FIG. 10) according to the first modification or the optical -axis adjustment mechanism (see FIG. 12A and FIG. 13 A) according to the second modification to adjust at least one of the relative angle 9 and the position difference D.

[0111] FIG. 15 is a flowchart of processing executed by the distance measurement system 1 according to the third modification of the present disclosure.

[0112] The order of steps in the flowchart described in the embodiment of the present disclosure may be changed as long as there is no contradiction. The steps of the flowchart described in theembodiment of the present disclosure may be executed concurrently or in parallel as long as there is no contradiction. For example, the embodiment of the present disclosure presents processing of various steps using an exemplary order. However, the order is not limited to the presented order.

[0113] As illustrated in FIG. 15, the computing device 3 waits for a distance selection operation performed by the user (step S101).

[0114] For example, the computing device 3 displays a list in which multiple distances are listed on a screen. The user can select any distance from the distance list displayed on the screen.

[0115] When the user performs the distance selection operation (step S 101 : YES), the computing device 3 (computing unit 31) sets the distance from the light projection-and-reception apparatus 2 in accordance with the operation (step SI 02).

[0116] In other words, the computing unit 31 operates as a distance setting unit that sets the distance from the light projection-and-reception apparatus 2 in accordance with an operation input from the user to the computing device 3.

[0117] The computing device 3 acquires the relative angle 0 or the position difference D, or both of the relative angle 9 and the position difference D (step SI 03).

[0118] Specifically, the computing device 3 geometrically calculates the relative angle 9 and / or the position difference D from the distance set in step SI 02 to acquire the relative angle 9 and / or the position difference D. The computing device 3 may acquire the relative angle 9 and / or the position difference D corresponding to the distance set in step SI 02 from a table stored in advance.

[0119] The computing device 3 transmits the relative angle 9 and / or the position difference D acquired in step S103 to the controller 140.

[0120] The controller 140 (adjuster 141) controls the optical -axis adjustment mechanism to adjust the posture and the position of each light projection unit 100 so as to meet the relative angle 9 and / or the position difference D received from the computing device 3 (step SI 04).

[0121] Specifically, the adjuster 141 adjusts at least one of the relative angle 9 and the position difference D so that the dot DA and the dot DB (an example of pattern light projected from the pair of light projection units) are emitted so as not to overlap each other on an object OB located at the distance set by the computing unit 31. Accordingly, for example, the dots DAand the dots DB are emitted so as not to overlap each other on the object OB located at the distance. Thus, the spatial resolution of the distance image can be increased for an object OB at a target distance.

[0122] The method of acquiring the relative angle 9 and the position difference D is not limited to the method described in the third modification. For example, the computing unit 31 projects the dot-pattern light DPA to acquire distance information, and detects a distance at which many objects OB are present based on the acquired distance information. The computing unit 31 determines the relative angle 9 and the position difference D so that the dots DA and the dots DB do not overlap each other at the detected distance.

[9123] In this case, the light projection-and-reception apparatus 2 controls the optical-axis adjustment mechanism so as to meet the determined relative angle 9 and position difference D and then projects the dot-pattern light DPB.

[9124] In this example, the adjuster 141 acquires information on reflection light corresponding to pattern light (e.g., dot-pattern light DPA) projected from one of the pair of light projection units 1 9 using the ToF light receiver 129, determines the position and the orientation of the optical axis of another one of the pair of light projection units 199 (e.g., light projection unit 1 9B) based on the acquired information on the reflection light, and changes the position and the orientation of the optical axis of the other one of the pair of light projection units 1 9 to the determined position and orientation of the optical axis to adjust the relative angle 9 and the position difference D.

[9125] FIG. 16 is a flowchart of processing executed by a distance measurement system 1 according to a fourth modification of the present disclosure.

[9126] In the fourth modification, for example, the relative angle 9 is temporarily and gradually changed, the degree of overlap between the dot-pattern light DPA and the dot-pattern light DPB on the object OB is detected in each change state, and the relative angle 9 is determined based on each detected degree of overlap.

[9127] In the fourth modification, the light projection-and-reception apparatus 2 drives the light projection units 199 A and 199B by the driving unit such as the motor to reset the relative angle 9 to an initial angle (e.g., an angle serving as the origin) (step S291).

[9128] The light projection-and-reception apparatus 2 projects the dot-pattern light DPA to acquire information on reflection light from the ToF light receiver 129, and projects the dot-pattern light DPB to acquire information on reflection light from the ToF light receiver 129 (stepS202). The light projection-and-reception apparatus 2 transmits the information to the computing device 3.

[0129] The computing device 3 calculates an overlap ratio (degree of overlap) between the dotpattern light DPA and the dot-pattern light DPB based on the information on each reflection light received from the light projection-and-reception apparatus 2 (step S203).

[0130] The overlap ratio is, for example, the sum of values (values obtained by dividing the area in which the dots DA and the dots DB overlap each other on the object OB by the area in which the dots DA and the dots DB do not overlap each other on the same object OB) calculated for each object OB at each distance.

[0131] The light projection-and-reception apparatus 2 drives the light projection units 100 A and 100B by the driving unit to minutely vary the relative angle 0 (step S204).

[0132] The processing of steps S202 to S204 are repeatedly executed until the relative angle 9 reaches the upper limit angle.

[0133] When the relative angle 9 reaches the upper limit angle (step S205: YES), the computing device 3 determines the relative angle 9 at which the overlap ratio between the dot-pattern light DPA and the dot-pattern light DPB is the minimum (step S206).

[0134] The computing device 3 presents the relative angle 9 at which the overlap ratio is determined to be the minimum in step S206 to the user, for example, by displaying the relative angle 9 on the screen (step S207).

[0135] The computing device 3 may present multiple relative angles 9 (e.g., relative angles 9 corresponding to the four lowest overlap ratios) to the user. The user can select one relative angle 9 from the multiple relative angles 9.

[0136] When a user operation of permitting (or selecting) the presented relative angle 9 is performed (step S208: YES), the light projection-and-reception apparatus 2 controls the optical -axis adjustment mechanism to adjust the relative angle 9 to the presented relative angle 9 (step S209).

[0137] When a user operation of not permitting (or not selecting) the presented relative angle 9 is performed (step S208: NO), the light projection-and-reception apparatus 2 controls the optical-axis adjustment mechanism to adjust the relative angle 9 to a predetermined relative angle 9 (step S210).

[0138] While the relative angle 9 is adjusted in the above description of the fourth modification, the position difference D may be adjusted instead of or in addition to the relative angle 9.

[0139] With the fourth modification, even when objects OB are located at various distances, ToF imaging with appropriate dot-pattern light can be performed, and the spatial resolution of the distance image can be increased for the objects OB at various distances.

[0140] As described above, in the fourth modification, the controller 140 (adjuster 141) temporarily and gradually changes at least one of the relative angle 9 and the position difference D, detects the degree of overlap on the object OB between the dot-pattern light DPA and the dotpattern light DPB (an example of pattern light projected from the pair of light projection units 100) in each change state, and determines at least one of the relative angle 9 and the position difference D based on the detected degree of overlap.

[0141] The light projection-and-reception apparatus 2 according to the above-described embodiment includes the light projection unit 100A and the light projection unit 100B on each of both sides of the apparatus as illustrated in FIG. 5 to obtain the angle of view over the entire circumference of the apparatus. However, the configuration of the light projection-and- reception apparatus 2 according to the present disclosure is not limited to this configuration.

[9142] FIG. 17 is a view illustrating another arrangement example of the optical elements of the light projection-and-reception apparatus 2.

[9143] The light projection-and-reception apparatus 2 illustrated in FIG. 17 includes the light projection units 1 9 A and 199B and the ToF light receiver 129 on one surface of the light projection-and-reception apparatus 2.

[9144] The light projection-and-reception apparatus 2 further includes a tool 299 such as a tripod that functions as a support, and a rotator 229 that is provided on an upper stage of the tool 299 and that is a rotary table rotated by, for example, an electric motor. In this case, the light projection-and-reception apparatus 2 acquires a distance image of the entire circumference while changing the imaging direction by rotating the apparatus itself by the rotator 229.

[9145] A rotation controller of the light projection-and-reception apparatus 2 has a predetermined rotation pattern when the rotation controller controls the rotator 229. For example, the rotation controller of the light projection-and-reception apparatus 2 controls the light projection-and-reception operation to the entire circumference of the apparatus in multiple times with a rotation pattern including (1) rotating by a predetermined distance, (2) stoppingthe rotation and projecting and receiving light, (3) rotating by a predetermined distance, and (4) stopping the rotation and projecting and receiving light for imaging.

[0146] FIG. 18 is a block diagram illustrating a configuration of a three-dimensional shape generation system 4 according to another embodiment of the present disclosure.

[0147] As illustrated in FIG. 18, the three-dimensional shape generation system 4 includes the light projection-and-reception apparatus 2 and a computing device 3 A. The three-dimensional shape generation system 4 is obtained by adding a generation processing unit 32 and a storage unit 33 to the computing device 3 of the distance measurement system 1 of FIG. 1.

[0148] The computing device 3 A includes the generation processing unit 32 that generates three- dimensional shape information based on an acquired three-dimensional point group, in addition to the computing unit 31. The three-dimensional shape information is information indicating a three-dimensional shape of an object that can be handled by a computer or the like.

[0149] The information indicating the three-dimensional shape is information with which it is possible to geometrically specify the three-dimensional shape. For example, when the information expresses a sphere, the center coordinates and the radius of the sphere correspond to the information indicating the three-dimensional shape, or when the information expresses a polyhedron (polygon), the coordinate points of the vertices of the polyhedron correspond to the information indicating the three-dimensional shape. The three-dimensional shape information may include information related to the color or material of the object in addition to the information indicating the three-dimensional shape of the object.

[0150] The storage unit 33 of the computing device 3 A includes a setting information management database (DB) 301, a storage processing management DB 302, a point group management DB 303, and a three-dimensional shape management DB 304.

[0151] The setting information management DB 301 stores and manages various information. The storage processing management DB 302 stores and manages various processing programs for generating a three-dimensional shape. The point group management DB 303 stores and manages three-dimensional point group information acquired by the distance measurement system. The three-dimensional shape management DB 304 stores and manages three- dimensional shape information.

[0152] The setting information management DB 301 includes a setting information management table. The setting information management table is a table for managing three-dimensionalpoint group data for generating a three-dimensional shape, and the execution order and the processing mode of generation processing (i.e., three-dimensional shape generation processing) for generating a three-dimensional shape. With the setting information management table, the file name of the three-dimensional point group data, and the execution order and the processing mode of the three-dimensional shape generation processing are managed in association with each other.

[0153] The three-dimensional shape generation processing includes, for example, registration processing, noise removal processing, segmentation processing, and modeling processing.

[0154] The registration processing is processing of converting multiple three-dimensional point groups into one unified three-dimensional point group. The noise removal processing is processing of removing an unnecessary point group from the three-dimensional point group.

[0155] The segmentation processing is processing of labeling a specific point group in the three- dimensional point group so as to distinguish the specific point group from other point groups, and differently labels each of multiple specific point groups so as to distinguish the multiple specific point groups from each other.

[0156] The segmentation processing may be executed together with clustering processing of grouping point groups at close distances among the labeled point groups.

[0157] The modeling processing is processing of comparing a specific point group in the three- dimensional point group with a three-dimensional model shape and replaces the specific point group with the model shape. The three-dimensional model shape is a model of, for example, a pattern or a template that is used for generating three-dimensional shape information from the three-dimensional point group.

[0158] The processing mode includes manual processing of executing part or the whole of three- dimensional shape information generation processing based on an operation input from the user without executing storage processing, automatic processing of executing part or the whole of the three-dimensional shape information generation processing based on storage processing stored in advance without depending on an operation input, and mixed processing of executing part or the whole of the three-dimensional shape information generation processing by mixing the manual processing and the automatic processing.

[0159] The generation processing unit 32 receives an input operation from the user. The input operation includes, for example, an operation of setting point group setting information indicating a three-dimensional point group to be processed and an operation of settingprocessing setting information indicating the execution order and the processing mode of each processing in the three-dimensional shape information generation processing.

[0160] The generation processing unit 32 executes the three-dimensional shape information generation processing set by the input operation.

[0161] Specifically, the generation processing unit 32 searches the point group management DB 303 using the point group setting information as a search key to read three-dimensional point group data associated with the point group setting information. The generation processing unit 32 also searches the storage processing management DB 302 using the processing mode of the generation processing in the processing setting information as a search key to read a processing program associated with the processing setting information.

[0162] The generation processing unit 32 generates three-dimensional shape information based on the three-dimensional point group data and the processing program read from the storage unit 33, and the execution order and the processing mode of the generation processing in the processing setting information.

[0163] When the processing mode includes the manual processing and the mixed processing, the generation processing unit 32 generates an operation screen that receives an operation input for generating the three-dimensional shape information, displays the operation screen on a display unit (e.g., a display), and receives a predetermined input operation from the user on the displayed operation screen.The input operation includes an operation input that executes processing in which the manual processing mode is selected in the three-dimensional shape information generation processing. The generation processing unit 32 generates three-dimensional shape information based on the operation input information by the input operation.

[0164] The generation processing unit 32 converts the generated three-dimensional shape information into, for example, a computer aided design (CAD) format, and stores the converted three-dimensional shape information in the three-dimensional shape management DB 304, an external recording media, or the like.

[0165] In the three-dimensional shape generation system according to the present embodiment, the three-dimensional shape information can be generated based on the point group acquired by the distance measurement system. Accordingly, for example, three-dimensional modeling of an existing structure in the field of architecture, construction, civil engineering, or the like is facilitated, and the present embodiment can be used for building information modeling (BIM) or construction information modeling (CIM).

[0166] The generation processing unit 32 is not limited to be included in the computing device 3 A including the computing unit 31 A, and may be included in, for example, another terminal device connected to the computing device 3 A by wired communication or wireless communication or a server located on a cloud. Alternatively, the three-dimensional shape generation processing may be executed by multiple devices (e.g., the computing device 3A and a server connected to the computing device 3 A).

[0167] Application examples in which the distance measurement system 1 is used in various detection systems will be described referring to FIGs. 19 and 20. Each of the detection systems in these application examples includes functional blocks described later in addition to the distance measurement system 1. In FIGs. 19 and 20, functional blocks such as a determination unit included in the detection system are illustrated outside the detection system for the convenience of drawing. The various detection systems illustrated in FIGs. 19 and 20 each include a controller that receives information input from the distance measurement system 1 and controls the detection system based on the information from the distance measurement system 1.

[0168] FIG. 19 illustrates an example of a shape measurement system as the detection system, and an application example in which the distance measurement system 1 is used for user authentication of an electronic device.

[0169] A portable information terminal 60X that is an electronic device has a user authentication function. The authentication function may be implemented by dedicated hardware, or may be implemented by a central processing unit (CPU) that controls the portable information terminal 60X executing a program stored in a read only memory (ROM) or the like.

[0170] When the user is authenticated, the light source device of the distance measurement system 1 mounted on the portable information terminal 60X projects light toward a user 6 IX who uses the portable information terminal 60X.

[0171] The light reception element of the distance measurement system 1 receives the light reflected from the user 6 IX and the surrounding area. An image processing unit 62X generates image data from the received light (performs imaging). A determination unit 63X determines the degree of coincidence between image information obtained by taking the image of the user 61X by the distance measurement system 1 and previously registered user information, and determines whether the user is a registered user.

[0172] Specifically, the shape (e.g., contour or unevenness) of the face, ears, or head of the user 61X can be measured and used as the user information.

[0173] In the application example of FIG. 19, the detection of the user 61X by the distance measurement system 1 can provide an advantageous effect similar to that of the distance measurement system 1 (an increase in detection accuracy). In particular, since the information on the user 6 IX can be detected in a wide range by projecting light from the light projection device in a wide angle, the amount of information for recognizing the user is increased as compared with the case where the detection range is narrow, and thus the recognition accuracy can be increased.

[0174] The case where the shape of the face, ears, or head of the user 6 IX is recognized using the portable information terminal 60X will be considered. In this case, the user 61X as a measurement object is close to the portable information terminal 60X in terms of distance. In particular, since measurement errors due to disparity are large in measurement at a short distance, it is desirable to reduce the measurement errors due to disparity by disposing the ToF light receiver 120 between the light projection unit 100 A and the light projection unit 100B. Hence, in the example of the configuration of FIG. 23, to make the measurement errors due to disparity be less likely to occur, the light projection unit 100A and the light projection unit 100B are disposed close to the ToF light receiver 120 at equal distances.

[0175] While FIG. 19 illustrates the example in which the distance measurement system 1 is mounted on the portable information terminal 60X, the user authentication using the distance measurement system 1 may be used for a desktop PC, an office appliance (OA) such as a printer, a security system for a building, or the like.

[0176] In terms of function, the present disclosure is not limited to the personal authentication function, and can be used for scanning of a three-dimensional object such as a face. In this case, the distance measurement system 1 that projects light in a wide angle can also implement high-precision scanning.

[0177] FIG. 20 illustrates an application example in which the distance measurement system 1 is used in an autonomous moving system in a moving body that is an example of the detection system.

[0178] In the application example of FIG. 20, the distance measurement system 1 is used for sensing an object outside a moving body 70X. The moving body 70X is an autonomous moving body that can automatically move while recognizing a surrounding situation.

[0179] The distance measurement system 1 is mounted on the moving body 70X. The distance measurement system 1 emits light in the moving direction of the moving body 70X and toward the surrounding area. In a room 71X that is a moving area for the moving body 70X, a desk 72X is installed in the moving direction of the moving body 70X.

[0180] The light, which is included in the light projected from the light source device of the distance measurement system 1 mounted on the moving body 70X and which is reflected from the desk 72X and the surrounding area, is received by the light reception element of the distance measurement system 1, and photoelectrically converted into an electric signal. The electric signal is sent to a signal processing unit 73X.

[0181] The signal processing unit 73X calculates information on the layout of the room 71X, such as the distance to the desk 72X, the position of the desk 72X, and the surroundings other than the desk 72X, based on the electric signal sent from the light reception element.

[0182] A determination unit 74X determines the moving path and the moving speed of the moving body 70X based on the calculated information. An operation controller 75X controls the movement of the moving body 70X (e.g., the operation of a motor as a driving source) based on the determination result of the determination unit 74X.

[0183] In the application example of FIG. 20, an advantageous effect similar to that of the distance measurement system 1 (an increase in detection accuracy) can be provided for the detection of the layout in the room 71X by the distance measurement system 1. In particular, since the information on the room 7 IX can be detected in a wide range by projecting the light from the light projection device in a wide angle, a large amount of information can be obtained as compared with the case where the detection range is narrow. As a result, the accuracy of the autonomous movement of the moving body 70X can be increased.

[0184] While FIG. 20 illustrates the example in which the distance measurement system 1 is mounted on the autonomous moving body 70X that moves in the room 71X, the distance measurement system 1 may be applied to an autonomous moving vehicle (i.e., a self-driving vehicle) that moves outdoors.

[0185] Alternatively, the distance measurement system 1 can be applied to a driving assistance system in a moving body such as an automobile that a driver drives rather than the autonomous moving body. In this case, the surroundings of the moving body is detected using the distance measurement system 1, and the driver's driving can be assisted depending on the detected surroundings.

[0186] Still alternatively, the distance measurement system 1 may be applied to an article inspection system in, for example, a factory. Specifically, the state of each article is determined by the determination unit of the article inspection system based on the information acquired by the distance measurement system 1.

[0187] Yet alternatively, the distance measurement system 1 may be applied to operation control of a movable device.

[0188] A multi -joint arm as the movable device includes multiple arms coupled by joints that are bendable and includes a hand portion at the tip of the multi -joint arm. The multi -joint arm is used in, for example, an assembly line of a factory, and grips an object with the hand portion when the object is inspected, conveyed, or assembled.

[0189] The distance measurement system 1 detects an object and the surrounding area. The determination unit of the movable device determines various information related to the object, such as the distance to the object, the shape of the object, the position of the object, and the positional relationship between multiple objects when the multiple objects are present, based on the information acquired by the distance measurement system 1. Accordingly, the drive controller controls the multi -joint arm operation based on the determination result of the determination unit.

[0190] Yet alternatively, the distance measurement system 1 may be applied to a driving assistance system for a moving body such as an automobile.

[0191] The distance measurement system 1 mounted in the automobile detects the driver who drives the automobile and the surrounding area. The determination unit of the driving assistance system determines information such as the face (facial expression) and the posture of the driver based on the information acquired by the distance measurement system 1. The controller appropriately assists driving depending on the driver's situation based on the determination result by the determination unit.

[0192] The shape measurement system, the moving body, the article inspection system, the movable device, and the driving assistance system are all examples of the detection system. With the distance measurement system 1 according to any one of the embodiments, the spatial resolution of the distance image can be increased. Accordingly, the detection system to which the distance measurement system 1 is applied can perform detection with high accuracy.

[0193] Aspects of the present disclosure are as follows.[Aspect 1]According to Aspect 1, a light projection-and-reception apparatus includes multiple light projection units; and a light receiver that receives reflection light from an object to which light are projected from the multiple light projection units. Each of the light projection units includes a light source that emits light; and an optical system that projects pattern light obtained from the light emitted from the light source to an object. The optical systems of the multiple light projection units are disposed non-parallel to each other to have optical axes forming an angle.[Aspect 2]According to Aspect 2, the light projection-and-reception apparatus of Aspect 1 includes a setting unit that sets (defines) a relative angle formed by the optical axes and a position difference between the multiple light projection units so that the pattern light projected from the multiple light projection units are emitted so as not to overlap each other on an object located at a first distance or more from the light projection units. [Aspect 3]According to Aspect 3, in the light projection-and-reception apparatus of Aspect 2, the pattern light projected from the multiple light projection units include pattern light in which multiple emission lights are regularly arranged. First emission light projected from one light projection unit of the multiple light projection units and second emission light projected from another light projection unit of the multiple light projection units are emitted so as to at least partially overlap each other on an object located at a second distance shorter than the first distance from the light projection units. The light projection units emit the first emission light and the second emission light so that the first emission light and the second emission light do not overlap each other and the first emission light and the second emission light are alternately and periodically arranged on an object located at a third distance longer than the first distance from the light projection units.[Aspect 4]According to Aspect 4, the light projection-and-reception apparatus of any one of Aspect 1 to Aspect 3 includes an adjuster that adjusts at least one of a relative angle formed by the optical axes and a position difference between the multiple light projection units.[Aspect 5]According to Aspect 5, the light projection-and-reception apparatus of Aspect 4 further includes a distance setting unit that sets a distance from the light projection units in accordance with an operation input. The adjuster adjusts at least one of the relative angle and the position difference so that the pattern light projected from the multiple light projection units are emitted so as not to overlap each other on an object located at the distance set by the distance setting unit.Thus, a distance setting unit sets a distance from the multiple light projection units. The adjuster adjusts at least one of: the relative angle; and the position difference, to shift positions of the pattern light, projected from the multiple light projection units, on the objectso that the pattern light projected from the multiple light projection units does not overlap, and the object is disposed away from the multiple light projection units at the distance set by the distance setting unit.[Aspect 6]According to Aspect 6, in the light projection-and-reception apparatus of Aspect 4 or Aspect 5, the adjuster changes at least one of the relative angle and the position difference, and in a state in which the at least one of the relative angle and the position difference is changed, detects a degree of overlap on the object of the pattern light projected from the multiple light projection units.[Aspect 7]According to Aspect 7, in the light projection-and-reception apparatus of Aspect 6, the adjuster determines at least one of the relative angle and the position difference based on the detected degree of overlap.[Aspect 8]According to Aspect 8, in the light projection-and-reception apparatus of any one of Aspect 4 to Aspect 7, the adjuster acquires, from the light receiver, information on reflection light corresponding to pattern light projected from one light projection unit of the multiple light projection units, determines a position or an orientation of the optical axis of another light projection unit of the multiple light projection units based on the acquired information on the reflection light, and changes the position or the orientation of the optical axis of said another light projection unit of the multiple light projection units to the determined position or the determined orientation of the optical axis to adjust at least one of the relative angle and the position difference.[Aspect 9]According to Aspect 9, in the light projection-and-reception apparatus of any one of Aspect 1 to Aspect 8, the multiple optical systems of the multiple light projection units are disposed so that the optical axes intersect with each other in an emission side (direction) of light.Thus, the multiple optical systems project light having the multiple optical axes intersecting with each other in an emission direction of light.[Aspect 10]According to Aspect 10, in the light projection-and-reception apparatus of any one of Aspect 1 to Aspect 9, the multiple light projection units are disposed side by side along a first axis. The optical axes of the optical systems of the multiple light projection units are inclined toward the first axis from a second axis perpendicular to the first axis and non-parallel to a plane defined by the first axis and the second axis.[Aspect 11]According to Aspect 11, a distance measurement system includes the light projection-and- reception apparatus of Aspect 1 to Aspect 10; and a distance calculator that calculates adistance to the object based on an output as a result of reception of the light by the light receiver.[Aspect 12]According to Aspect 12, a light projection device includes multiple light projection units. Each of the light projection units includes a light source that emits a light beam; and an optical system that projects pattern light obtained from the light beam emitted from the light source to an object. The multiple light projection units are disposed non-parallel to each other to have optical axes forming an angle.

[0194] The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and / or features of different illustrative embodiments may be combined with each other and / or substituted for each other within the scope of the present invention. Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.

[0195] The present invention can be implemented in any convenient form, for example using dedicated hardware, or a mixture of dedicated hardware and software. The present invention may be implemented as computer software implemented by one or more networked processing apparatuses. The processing apparatuses include any suitably programmed apparatuses such as a general purpose computer, a personal digital assistant, a Wireless Application Protocol (WAP) or third-generation (3G)-compliant mobile telephone, and so on. Since the present invention can be implemented as software, each and every aspect of the present invention thus encompasses computer software implementable on a programmable device. The computer software can be provided to the programmable device using any conventional carrier medium (carrier means). The carrier medium includes a transient carrier medium such as an electrical, optical, microwave, acoustic or radio frequency signal carrying the computer code. An example of such a transient medium is a Transmission Control Protocol / Intemet Protocol (TCP / IP) signal carrying computer code over an IP network, such as the Internet. The carrier medium may also include a storage medium for storing processor readable code such as a floppy disk, a hard disk, a compact disc read-only memory (CD- ROM), a magnetic tape device, or a solid state memory device.

[0196] The functionality of the elements disclosed herein may be implemented using circuitry or processing circuitry which includes general purpose processors, special purpose processors, integrated circuits, application- specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or combinations thereof which are configured or programmed, using one or more programs stored in one or more memories, to perform the disclosed functionality. Processors are considered processing circuitry or circuitry as they include transistors andother circuitry therein. In the disclosure, the circuitry, units, or means are hardware that carry out or are programmed to perform the recited functionality. The hardware may be any hardware disclosed herein which is programmed or configured to carry out the recited functionality.There is a memory that stores a computer program which includes computer instructions. These computer instructions provide the logic and routines that enable the hardware (e.g., processing circuitry or circuitry) to perform the method disclosed herein. This computer program can be implemented in known formats as a computer-readable storage medium, a computer program product, a memory device, a record medium such as a CD-ROM or DVD, and / or the memory of an FPGA or ASIC.

[0197] This patent application is based on and claims priority to Japanese Patent Application No. 2023-207060, filed on December 7, 2023, and No. 2024-154946, filed on September 9, 2024, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.[Reference Signs List]

[0198] 1 distance measurement system2 light projection-and-reception apparatus3 computing device10 light projection-and-reception unit20 RGB light receiver31 computing unit100 A light projection unit100B light projection unit102 light source104 first lens group106 DOE108 second lens group120 ToF light receiver122 optical system124 ToF sensor140 controller141 adjuster160 A lens barrel160B lens barrel170 screw172 coil spring174 adjustment screwhousingAlong holeB long hole scale

Claims

[CLAIMS]1. A light proj ection-and-reception apparatus comprising: multiple light projection units to project pattern light onto an object, each of the multiple light projection units including: multiple light sources to emit light; and multiple optical systems to project the pattern light obtained from the light emitted from the multiple light sources to the object, the multiple optical systems respectively having multiple optical axes arranged non-parallel to each other to form an angle; and a light receiver to receive reflection light reflected from the object, to which the light is projected from the multiple light projection units.

2. The light proj ection-and-reception apparatus according to claim 1, comprising: a setting unit to set: a relative angle formed by the multiple optical axes; and a position difference between the multiple light projection units, to project the pattern light from the multiple light projection units on an object at positions shifted from each other so that the pattern light projected from the multiple light projection units does not overlap, and the object disposed at a first distance or more from the light projection units.

3. The light proj ection-and-reception apparatus according to claim 2, wherein the multiple light projection units each project the pattern light having multiple emission lights regularly arranged, the multiple light projection units including: a first light projection unit to emit first emission light of the multiple emission lights on the object disposed at a first distance from the light projection units; and a second light projection unit to emit second emission light of the multiple emission lights, the second emission light partially overlapping with the first emission light on the object disposed at a second distance shorter than the first distance from the light projection units, and the first light projection unit and the second light projection units respectively emit: the first emission light and the second emission light on the object at positions shifted from each other so that the first emission light and the second emission light do not overlap at the object disposed at a third distance longer than the first distance from the multiple light projection units; and the first emission light and the second emission light alternately and periodically arranged on the object at the third distance.

4. The light projection-and-reception apparatus according to claim 1, further comprising: an adjuster to adjust at least one of: a relative angle formed by the multiple optical axes; and a position difference between the multiple light projection units.

5. The light projection-and-reception apparatus according to claim 4, further comprising: a distance setting unit to set a distance from the multiple light projection units, wherein the adjuster adjusts at least one of: the relative angle; and the position difference, to shift positions of the pattern light, projected from the multiple light projection units, on the object so that the pattern light projected from the multiple light projection units does not overlap, and the object is disposed away from the multiple light projection units at the distance set by the distance setting unit.

6. The light projection-and-reception apparatus according to claim 4, wherein the adjuster changes at least one of: the relative angle; and the position difference, to detect a degree of overlap of the pattern light, projected from the multiple light projection units, on the object, when the adjuster changes the at least one of the relative angle and the position difference.

7. The light projection-and-reception apparatus according to claim 6, wherein the adjuster determines at least one of: the relative angle; and the position difference, based on the detected degree of overlap.

8. The light projection-and-reception apparatus according to claim 4, wherein the multiple light projection units including: a first light projection unit to emit first pattern light on the object; and a second light projection unit to emit second pattern light on the object, the adjuster:acquires, from the light receiver, information on a reflection light, of the first pattern light, reflected from the object; determines a position or an orientation of an optical axis of the multiple optical axes of the second light projection unit based on the information on the reflection light; and changes the position or the orientation of the optical axis of the second light projection unit to the position or the orientation of the optical axis, determined based on the information, to adjust at least one of the relative angle and the position difference.

9. The light projection-and-reception apparatus according to any one of claims 1 to 8, wherein the multiple optical systems project light having the multiple optical axes intersecting with each other in an emission side of light.

10. The light projection-and-reception apparatus according to any one of claims 1 to 8, wherein the multiple light projection units are arrayed along a first axis, the multiple optical systems respectively have the multiple optical axes, and each of the multiple optical axes is: inclined with respect to a second axis perpendicular to the first axis; and non-parallel to a plane defined by the first axis and the second axis.

11. A distance measurement system comprising: the light projection-and-reception apparatus according to any one of claims 1 to 8; and a distance calculator to calculate a distance to the object based on an output of the light received by the light receiver.

12. A light projection device comprising: multiple light projection units, each of the multiple light projection units including: a light source to emit a light beam; and an optical system to project pattern light, obtained from the light beam emitted from the light source, to an obj ect, wherein the multiple light projection units respectively have multiple optical axes arranged non-parallel to each other to form an angle.

Images