Light projection device, light projection and reception device, and distance measuring system

Abstract

To improve the spatial resolution of a distance image using pattern light.SOLUTION: A light projection and reception device comprises a plurality of light projection units, and a light receiving unit that receives rays of reflected light from an object on which rays of light are projected from the plurality of light projection units. Each of the light projection units includes a light source that emits light, and an optical system that projects, on the object, pattern light obtained from the light emitted from the light source. The optical systems of the plurality of light projection units are arranged not in parallel to each other so that their optical axes form angles.SELECTED DRAWING: Figure 6A

Description

Technical Field

[0001] The present disclosure relates to a light projecting device, a light projecting and receiving device, and a distance measuring system.

Background Art

[0002] In order to measure the distance to an object by the ToF (Time of Flight) method, a light projecting device that projects pattern light (for example, a dot pattern) onto the object is known. For example, Patent Document 1 describes a specific configuration of this type of light projecting device.

[0003] In the TOF camera system described in Patent Document 1, in order to distinguish direct incident light rays from indirect incident light rays, the modulated light irradiated from the irradiation unit to the scene is discretely changed by the pattern forming means, and the basic regions of the scene are irradiated with different incident intensities.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, distance information is obtained by pattern light, but there is room for improvement in terms of improving the spatial resolution of the distance image by pattern light.

[0006] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a light projecting device, a light projecting and receiving device, and a distance measuring system capable of improving the spatial resolution of a distance image by pattern light.

Means for Solving the Problems

[0007] A light transmitting and receiving device according to an embodiment of the present disclosure includes a plurality of light transmitting units and a light receiving unit that receives reflected light from an object irradiated with light from the plurality of light transmitting units. Each light transmitting unit includes a light source that emits light and an optical system that projects pattern light obtained from the light emitted from the light source onto the object. The optical systems of each of the plurality of light transmitting units are arranged non-parallel to each other such that their optical axes form an angle.

Advantages of the Invention

[0008] According to an embodiment of the present disclosure, there are provided a light transmitting device, a light transmitting and receiving device, and a distance measuring system capable of improving the spatial resolution of a distance image using pattern light.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 7A

Figure 7B

Figure 8

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12A

Figure 12B

Figure 12C

Figure 13A

Figure 13B

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the following description, common or corresponding elements are denoted by the same or similar reference numerals, and overlapping descriptions are appropriately simplified or omitted.

[0011] FIG. 1 is a block diagram showing the configuration of a distance measurement system 1 according to an embodiment of the present disclosure.

[0012] As shown in FIG. 1, the distance measurement system 1 includes a light transmitting and receiving device 2 and an arithmetic device 3.

[0013] The distance measurement system 1 according to the present embodiment is a system that measures the distance from the light transmitting and receiving device 2 to an object (denoted as “target object OB”) by the ToF (Time of Flight) method. In the ToF method, distance measurement light (for example, laser light) having a wavelength different from that of visible light (for example, 940 nm) is irradiated onto the target object OB. Based on the time difference between the emission timing and the reception timing of the laser light at each irradiation position, the distance to each part (each irradiation position) of the target object OB is calculated.

[0014] In this embodiment, a diffractive optical element is used in a light projecting optical system that emits laser light. By projecting pattern light such as dots onto an object OB using the diffractive optical element, high-brightness and high-density point cloud data can be acquired. Hereinafter, the diffractive optical element is referred to as "DOE (Diffractive Optical Element)". Here, the pattern light is a collection of light with discrete light intensity in space, and the shape of the light when irradiated on a plane may be any of circular, polygonal, striped, etc.

[0015] The light projecting and receiving device 2 includes a light projecting and receiving unit 10 and an RGB light receiving unit 20. The "light projecting and receiving device" may be rephrased as, for example, "light projecting device", "imaging device", "distance measuring device".

[0016] The light projecting and receiving device 2 is equipped with, for example, a rechargeable battery. That is, the light projecting and receiving device 2 is driven by the battery. The light projecting and receiving device 2 may also be drivable by a commercial power supply.

[0017] The light projecting and receiving unit 10 includes a plurality of light projecting units 100A, 100B, a ToF light receiving unit 120, and a control unit 140. Hereinafter, the light projecting unit 100A and the light projecting unit 100B may be collectively referred to as "light projecting unit 100".

[0018] The light projecting and receiving unit 10 is not limited to two light projecting units 100A and 100B, and may include three or more light projecting units. In this embodiment, since it includes two light projecting units 100, namely the light projecting unit 100A and the light projecting unit 100B, these two light projecting units 100 may be referred to as "a pair of light projecting units 100".

[0019] FIG. 2 is a block diagram of the light projecting and receiving unit 10 alone.

[0020] The light projecting unit 100 (that is, each of the light projecting units 100A and 100B) includes a light source 102, a first lens group 104, a DOE 106, and a second lens group 108. The light projecting unit 100 projects pattern light onto the object OB.

[0021] Note that any reference to an element using designations such as "first", "second", etc. used in the present disclosure does not generally limit the quantity or order of those elements. These designations are used for convenience to distinguish between two or more elements. Therefore, a reference to a first and a second element does not mean, for example, that only two elements are employed, or that the first element must precede the second element.

[0022] The control unit 140 controls the light projecting unit 100 and the ToF light receiving unit 120. Specifically, as a circuit configuration, the control unit 140 includes a CPU (Central Processing Unit), a light source drive circuit, an imaging signal processing circuit, an input / output circuit, a memory, and the like.

[0023] The control unit 140 is, for example, a single processor or a multi-processor and includes at least one processor. When configured to include a plurality of processors, the control unit 140 may be packaged as a single device, or may be composed of a plurality of physically separated devices within the light transmitting and receiving unit 10.

[0024] The light source 102 is an example of a light source that emits at least one light. The light source 102 is, for example, an LD (Laser Diode) that emits laser light. The light source 102 emits laser light at a timing controlled by the control unit 140.

[0025] The first lens group 104, the DOE 106, and the second lens group 108 are an example of an optical system that projects pattern light obtained from the laser light emitted from the light source 102 onto an object.

[0026] The first lens group 104 is an example of a first lens group. The laser light emitted from the light source 102 is incident on the first lens group 104.

[0027] DOE106 is an example of an optical element that obtains (forms) patterned light in which a plurality of emitted lights are regularly arranged from the laser light incident through the first lens group 104. Here, DOE106 emits patterned light in which a plurality of dots are regularly arranged (for example, in a lattice pattern) (hereinafter referred to as "dot patterned light"). Hereinafter, a dot is an example of each emitted light constituting the patterned light.

[0028] The second lens group 108 is an example of the second lens group. Dot patterned light emitted from DOE106 is incident on the second lens group 108. The second lens group 108 projects this dot patterned light onto the object OB.

[0029] FIG. 3 is a diagram for explaining an example of ToF imaging by the light transmitting and receiving device 2. In FIG. 3, for the sake of simplicity in explaining ToF imaging, only one light projecting unit 100 (only one of the light projecting units 100A and 100B) is shown, and the illustration of the RGB light receiving unit 20 is omitted.

[0030] As shown in FIG. 3, the light transmitting and receiving device 2 projects dot patterned light onto the object OB. In order to project this type of patterned light, for example, the light source 102 may be configured by a VCSEL (Vertical Cavity Surface Emitting Laser) having a plurality of light emitting units arranged in a pattern, or the pattern may be formed in a dot shape by the diffraction phenomenon of the fine structure of DOE106, or both may be applied. Thereby, high-brightness and high-density dot patterned light can be projected onto the object OB, and the ranging accuracy is improved.

[0031] An explanation will be supplemented regarding the case where the diffraction pattern of DOE106 is formed in a dot pattern. In this case, the light source 102 is configured by a point light source such as an LD, for example. The laser light emitted from the light source 102 is incident on DOE106 through the first lens group 104.

[0032] When the laser light is incident on the DOE 106, a corresponding number of conjugate points are generated according to the diffraction orders (0th, ±1st, ±2nd, ···) of the DOE 106, and dot pattern light corresponding to the diffraction orders is irradiated to infinity.

[0033] In addition, when the VCSEL and the DOE 106 are used in combination, the dot pattern light can be irradiated over a wider range.

[0034] The dot pattern light irradiated on the object OB is reflected or scattered by the object OB. The ToF light receiving unit 120 receives the light directly reflected by the object OB (hereinafter referred to as "direct reflected light").

[0035] The ToF light receiving unit 120 is an example of a light receiving unit that receives the reflected light from the object OB irradiated with the dot pattern light. As shown in FIG. 2, the ToF light receiving unit 120 includes an optical system 122 and a ToF sensor 124.

[0036] The optical system 122 includes, for example, an aperture, an imaging optical system, and a filter. The direct reflected light from the object OB irradiated with the dot pattern light passes through the optical system 122 and is received by the ToF sensor 124.

[0037] The ToF sensor 124 is an imaging device such as a CMOS image sensor, and photoelectrically converts the sum of the exposure amounts in a plurality of exposure periods having a predetermined phase difference with respect to the irradiated light and outputs the result to the control unit 140. The light reception data from the ToF sensor 124 is input to the arithmetic unit 3 via the control unit 140.

[0038] The arithmetic unit 31 of the arithmetic device 3 is realized by an instruction from the CPU of the arithmetic device 3, and calculates the distance to each part (each irradiation position) of the object OB based on the sum of the exposure amounts in each exposure period input from the ToF sensor 124. Note that, when using a SPAD or the like as the ToF sensor 124, the arithmetic unit 31 may calculate the distance based on the time difference between the emission timing of the laser light (the emission timing of the light source 102) and the reception timing (the input timing from the ToF sensor 124) at each irradiation position.

[0039] That is, the arithmetic unit 31 of the arithmetic device 3 is an example of a distance calculation unit that calculates the distance to the object OB based on the output from the light reception by the ToF light reception unit 120.

[0040] The RGB light reception unit 20 includes, for example, an aperture, an imaging optical system, a filter, and an image sensor. The image sensor is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor and is provided with an RGB color filter.

[0041] The image sensor may be replaced with other types of imaging devices such as a CCD (Charge Coupled Device) image sensor. The image sensor may be configured to include a complementary color checkerboard filter.

[0042] The image sensor is driven under the control of the control unit 140 and receives visible light (natural light) on the light reception surface. The image sensor accumulates the optical image formed on each pixel on the light reception surface as electric charges corresponding to the amount of light and outputs the charges, for example, at a timing synchronized with the ToF imaging. The control unit 140 outputs RGB image data based on each pixel data to the arithmetic device 3.

[0043] In the configuration example shown in FIG. 1, the light transmitting and receiving unit 10 is included in the light transmitting and receiving device 2. The light transmitting and receiving unit 10 may be configured as a single device independent from the light transmitting and receiving device 2 as shown in FIG. 2.

[0044] The light projecting unit 100 (in other words, the light projecting device) may be configured as a single device independent from the light transmitting and receiving unit 10. FIG. 4 is a block diagram of the light projecting device 10a configured as a single device.

[0045] As shown in FIG. 4, the light projecting device 10a includes a light projecting units 100A and 100B and a control unit 140. That is, in this configuration example, the light projecting device 10a has a configuration in which the ToF light reception unit 120 is omitted from the light transmitting and receiving unit 10 shown in FIG. 2.

[0046] The light projecting device 10a may be configured not to include the control unit 140. That is, the light projecting device 10a may be an optical device including a light projecting unit 100A and 100B in which optical components such as a light source 102, a first lens group 104, a diffractive optical element 106, and a second lens group 108 are arranged. In this case, for example, signal transmission means such as a cable is connected to a terminal provided in the light projecting device 10a. Thereby, the light source 102 and the control unit 140 connected to the other end of the cable are connected via the signal transmission means.

[0047] In the configuration example shown in FIG. 1, the light projecting and receiving device 2 includes an RGB light receiving unit 20. The light projecting and receiving device 2 may be configured not to include the RGB light receiving unit 20.

[0048] The arithmetic unit 3 is, for example, a terminal device such as a PC (Personal Computer) or a server arranged on the cloud. The light projecting and receiving device 2 and the arithmetic unit 3 can communicate with each other by wired communication or wireless communication through their respective transmitting and receiving units. Data may be transmitted (output) from the light projecting and receiving device 2 to the arithmetic unit 3 via a network, or the transmitting and receiving unit may be configured by an interface circuit with a portable storage medium such as an SD card or a personal computer.

[0049] In another embodiment, the arithmetic unit 31 may be configured to be included in the light projecting and receiving device 2. In this case, the arithmetic unit 31 may be configured as, for example, a processor included in the control unit 140. The arithmetic unit 31 may be configured as a processor independent of the control unit 140.

[0050] In yet another embodiment, the light projecting and receiving device 2 may be configured to include the entire arithmetic unit 3.

[0051] Thus, there is freedom in the aspects of each part of the distance measurement system 1, and various design changes are possible.

[0052] FIG. 5 is a schematic configuration diagram showing an implementation example of the light transmitting and receiving device 2. In the implementation example shown in FIG. 5, the light transmitting and receiving device 2 is configured as a device capable of capturing an omnidirectional panoramic image. The omnidirectional panoramic image is a panoramic image obtained by obtaining an image within a solid angle of 4π radians and capturing the entire spherical range. The light transmitting and receiving device 2 shown in FIG. 5 is driven by a battery.

[0053] The light transmitting and receiving device 2 shown in FIG. 5 is provided with a pair of light projecting units 100A and 100B on each side of the device. The light transmitting and receiving device 2 is provided with a wide-angle ToF light receiving unit 120 for obtaining a hemispherical distance image at the upper part of the device.

[0054] Also, the light transmitting and receiving device 2 is provided with a pair of wide-angle RGB light receiving units 20 for obtaining a hemispherical color image on each side of the device. In FIG. 5, one of the pair of RGB light receiving units 20 is located in front of the device. The other of the pair of RGB light receiving units 20 is located on the back of the device. Since the RGB light receiving unit 20 on the back of the device is not visible in FIG. 5, it does not appear in the drawing.

[0055] Each of the pair of RGB light receiving units 20 images an object (for example, the target object OB) around the light transmitting and receiving device 2. Thereby, a pair of hemispherical images are obtained. The control unit 140 synthesizes the pair of hemispherical images to generate, for example, an omnidirectional panoramic image expressed in the Mercator projection method.

[0056] The direct reflected light from the target object OB irradiated with the dot pattern light emitted from each of the light projecting units 100A and 100B is received by the ToF light receiving unit 120. The arithmetic unit 3 calculates the distance to each part (each irradiation position) of the target object OB based on the time difference between the emission timing and the reception timing of the laser light at each irradiation position. From this calculation result, distance information in the omnidirectional range corresponding to the omnidirectional panoramic image is obtained. That is, a three-dimensional point cloud, which is a set of coordinate points in a three-dimensional space, can be acquired. Color information (for example, the RGB values of each coordinate point) may be added to each coordinate point of the point cloud.

[0057] As shown in FIG. 5, the light projecting units 100A and 100B are arranged non-parallel to each other such that the optical axes AXA and AXB form an angle (relative angle θ). The reason for this will be described in detail.

[0058] FIGS. 6A and 7A are diagrams for explaining the relationship between the respective optical axes AXA and AXB of the light projecting units 100A and 100B and the dot pattern light irradiated onto the object OB. In the subsequent figures (except FIG. 7A) showing the dot pattern light, the dot pattern light projected from the light projecting unit 100A is indicated by a dotted line and a white circle. The dot pattern light projected from the light projecting unit 100B is indicated by a dashed-dotted line and a black circle. In FIG. 7A, the object OB is shown in black, and the dots irradiated onto the object OB are shown in white.

[0059] In FIG. 7A, the reference signs DPA, DPB, and DPC indicate the dot pattern light irradiated onto the object OB. The dot pattern light DPA indicates the dot pattern light irradiated from the light projecting unit 100A onto the object OB. The dot pattern light DPB indicates the dot pattern light irradiated from the light projecting unit 100B onto the object OB. The individual dots of the dot pattern light DPA (white circles in FIG. 6A) are indicated by the reference sign DA. The individual dots of the dot pattern light DPB (black circles in FIG. 6A) are indicated by the reference sign DB.

[0060] Also, in FIG. 7A, the dot pattern light DPC indicates the dot pattern light irradiated from the light projecting unit according to the comparative example onto the object OB.

[0061] As shown in FIG. 6A, due to the arrangement of the light projecting units 100A and 100B such that the optical axes AXA and AXB form a relative angle θ, the dot pattern lights projected from the respective light projecting units 100 are irradiated onto the object OB at positions shifted from each other. Specifically, on the object OB, the dots DA (an example of the first dot) and the dots DB (an example of the second dot) are irradiated so as to be alternately and periodically arranged.

[0062] Here, in order to improve the spatial resolution of the distance image by the pattern light, it is conceivable to increase the dot density. However, when the dot density is increased, for example, on a nearby object OB, adjacent dots are irradiated overlappingly like the dot pattern light DPC of the comparative example in FIG. 7A. Here, in order to perform a process of removing the influence of indirect light as disclosed in Patent Document 1, a region where light does not enter between dots is required. Therefore, when adjacent dots are irradiated overlappingly, appropriate distance information cannot be measured.

[0063] Therefore, the light projecting and receiving device 2 according to the present embodiment projects the dot pattern lights DPA and DPB in order. The light projecting and receiving device 2 sequentially receives the direct reflected light from the object OB irradiated with the dot pattern lights DPA and DPB by the ToF light receiving unit 120, and outputs the received data to the arithmetic unit 3. The arithmetic unit 31 of the arithmetic unit 3 sequentially calculates the distance data A corresponding to the dot pattern light DPA and the distance data B corresponding to the dot pattern light DPB, and merges the calculated distance data A and the distance data B.

[0064] Depending on the distance of the object OB, the dot pattern light DPA and the dot pattern light DPB do not overlap. Therefore, the light projecting and receiving device 2 may project the dot pattern light DPA and the dot pattern light DPB simultaneously.

[0065] The dot pattern light DPA and the dot pattern light DPB are, for example, shifted by a half pitch from each other in the dot irradiation position. Therefore, the distance information between the dots DA of the dot pattern light DPA is interpolated with the distance information corresponding to the dots DB of the dot pattern light DPB located between the dots DA. In other words, the distance information between the dots DB is interpolated with the distance information corresponding to the dots DA located between the dots DB.

[0066] As described above, according to the present embodiment, even on a nearby object OB, a high-spatial-resolution distance image equivalent to the case where a high-density dot pattern light is irradiated on the object OB can be obtained while the dots are not irradiated overlappingly.

[0067] According to this embodiment, the spatial resolution of the distance image by the dot pattern light can be improved.

[0068] The arrays of the dot pattern lights DPA and DPB are not limited to a square lattice. As shown in FIG. 7B, the dot pattern lights DPA and DPB may be arranged in other forms such as a hexagonal lattice. The light projecting units 100A and 100B may be arranged with a lateral shift so that the dot pattern light DPA and the dot pattern light DPB are shifted horizontally in the figure.

[0069] FIG. 8 is a diagram showing the relationship between the distances of the dot pattern light DPA (dotted line) and the dot pattern light DPB (dashed-dotted line) and the object OB. In FIG. 8, N1m to N5m indicate the distance of the object OB from the light projecting and receiving device 2 (alternatively, the irradiation distance). N1m, N2m, and N3m are less than 1 m. N4m and N5m are greater than 1 m.

[0070] As shown in FIG. 8, in the short-distance region, depending on the irradiation distance, the positional relationship between the dot DA and the dot DB changes, or the dot DA and the dot DB are irradiated overlappingly. Specifically, at the distance N1m, the arrangement of the irradiated dots is dot DA, DB, DA... in order from the top, while at the distance N3m, this arrangement is dot DB, DA, DB... in order from the top. Also, at the distance N2m, the dot DA and the dot DB are irradiated almost overlappingly.

[0071] At short distances, the positional difference between the light emission points (the positional difference D between the light projecting units 100A and 100B) is of a magnitude that cannot be ignored with respect to the irradiation distance. Since the optical axes AXA and AXB form a relative angle θ and the positional difference D between both light emission points is of a magnitude that cannot be ignored, in the short-distance region, depending on the irradiation distance, the positional relationship between the dot DA and the dot DB changes, or the dot DA and the dot DB are irradiated overlappingly.

[0072] On the other hand, at a long distance, the positional difference D of the light-emitting points is extremely small with respect to the irradiation distance. At a long distance, the positional difference D of the light-emitting points can be substantially ignored and can be regarded as radiation from almost the same point. Therefore, the positional relationship between dot DA and dot DB becomes constant.

[0073] That is, in the long-distance region, regardless of the irradiation distance, the positional relationship between dot DA and dot DB is constant. Specifically, at both distance N4m and distance N5m, the arrangement of the irradiated dots is, in order from the top, dot DB, DA, DB...

[0074] However, the farther the distance, the wider the interval between the dots, and the lower the spatial resolution by the dot pattern light. In order to obtain a high spatial resolution at the distance where the object OB is located, it is desirable to appropriately set the relative angle θ between the optical axes AXA and AXB and the positional difference D between the light projecting unit 100A and the light projecting unit 100B.

[0075] Exemplarily, at a distance of less than 1 m from the light projecting and receiving device 2, information that is not originally important, such as a photographer holding the light projecting and receiving device 2 or a fixing means (e.g., a tripod) of the light projecting and receiving device 2, is often included. Therefore, in this distance region, there is little need to improve the spatial resolution.

[0076] Therefore, in the example of FIG. 8, in the distance region of 1 m or more, in order to improve the spatial resolution, the relative angle θ and the positional difference D are set so that dot DA and dot DB are irradiated without overlapping and the positional relationship between dot DA and dot DB becomes constant.

[0077] In the example of FIG. 6A, the light projecting unit 100A and the light projecting unit 100B are arranged so that the optical axes AXA and AXB intersect in the light emission direction. However, for example, as shown in FIG. 6B, the light projecting unit 100A and the light projecting unit 100B may be arranged in a direction in which the optical axes AXA and AXB do not intersect in the light emission direction. In this case, a region where the dot pattern becomes sparse at the end of the irradiation range by the light projecting unit 100A and the light projecting unit 100B is likely to occur, while this irradiation range can be secured widely.

[0078] In the examples of FIGS. 6A and 6B, the light projecting units 100A and 100B are arranged side by side in the vertical direction in the drawing, and each of the light projecting units 100A and 100B is inclined with respect to the horizontal direction perpendicular to the vertical direction, so that the optical axes AXA and AXB form a relative angle θ. However, for example, as shown in FIG. 6C, one of the light projecting units 100A and 100B may be arranged such that the optical axis is parallel to the horizontal direction, and the other of the light projecting units 100A and 100B may be arranged such that the optical axis is inclined with respect to the horizontal direction, so that the optical axes AXA and AXB form a relative angle θ. By adopting such a configuration, for example, the processing of components such as the housing that holds the lens barrel becomes simple, and advantages such as cost reduction can be obtained. In the example of FIG. 6C, a configuration in which the optical axes AXA and AXB intersect is adopted, but in another example, a configuration in which the optical axes AXA and AXB do not intersect as in FIG. 6B may be adopted. Further, when the optical axes AXA and AXB are inclined as in FIGS. 6A and 6B, the inclination of the optical axis AXA with respect to the horizontal direction and the inclination of the optical axis AXB with respect to the horizontal direction may be made different.

[0079] FIG. 9 is a diagram showing a holding mechanism for the light projecting units 100A and 100B. As shown in FIG. 9, the light projecting unit 100A including the light source 102, the first lens group 104, the DOE 106, and the second lens group 108 is held by the lens barrel 160A. Similarly, the light projecting unit 100B including the light source 102, the first lens group 104, the DOE 106, and the second lens group 108 is held by the lens barrel 160B.

[0080] A part of the housing 180 of the light projecting and receiving device 2 is shown in FIG. 9. As shown in FIG. 9, the lens barrels 160A and 160B are fastened and fixed to the housing 180 with screws 170.

[0081] The relative angle θ and the positional difference D are defined by the lens barrels 160A and 160B, the screws 170, and the housing 180 so that the positional relationship between the dots DA and DB is constant without the dots DA and DB overlapping and being irradiated in a distance region of 1 m or more.

[0082] In this way, the lens barrels 160A and 160B, the screw 170, and the housing 180 serve as a defining part that defines the relative angle θ formed by the optical axes AXA and AXB with each other and the positional difference D of the plurality of light projecting units 100 so that the dot pattern light projected from each of the plurality of light projecting units 100 does not overlap and irradiate each other on the object OB that is 1 m (an example of the first distance) or more away from the light projecting and receiving device 2.

[0083] Incidentally, the dot DA (an example of the first dot) projected from the light projecting unit 100A (one of the pair of light projecting units 100 and an example of one of the plurality of light projecting units) and the dot DB (the second dot) projected from the light projecting unit 100B (the other of the pair of light projecting units 100 and an example of the other of the plurality of light projecting units) are irradiated such that at least a part of them overlaps each other on the object OB at a second distance (for example, N2m in FIG. 8) shorter than 1 m (an example of the first distance) from the light projecting and receiving device 2. Further, the dots DA and DB are irradiated on the object OB at a third distance (for example, N4m in FIG. 8) longer than 1 m (an example of the first distance) such that the dots DA and DB do not overlap and the dots DA and DB are arranged alternately and periodically.

[0084] The above is the description of the exemplary embodiments of the present disclosure. The embodiments of the present disclosure are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present disclosure. For example, the content obtained by appropriately combining the embodiments explicitly exemplified in the specification or obvious embodiments is also included in the embodiments of the present application.

[0085] FIG. 10 is a diagram showing an optical axis adjustment mechanism according to Modification 1 of the present disclosure. FIG. 10 is the same as FIG. 9 and shows the holding mechanisms of the light projecting unit 100A and the light projecting unit 100B according to this Modification 1.

[0086] In the optical axis adjustment mechanism according to this Modification 1, the relative angle θ can be adjusted.

[0087] Specifically, a coil spring 172 is disposed between each lens barrel 160A, 160B and the housing 180. The lens barrels 160A, 160B and the housing 180 are fastened by a screw 170 inserted through the coil spring 172. The total length (axial length) of the coil spring 172 compressed between the lens barrels 160A, 160B and the housing 180 changes according to the tightening degree of the screw 170.

[0088] The adjustment screw 174 composed of the screw 170 and the coil spring 172 is provided in a pair for each of the lens barrels 160A, 160B. By individually adjusting the tightening degree of the screw 170 of each adjustment screw 174, the user can change the posture of the light projecting unit 100 with respect to the housing 180 (in other words, the direction of the optical axis). By changing the posture difference between the light projecting unit 100A and the light projecting unit 100B, the relative angle θ changes.

[0089] The coil spring 172 may be replaced with other forms of components such as shim spacers. That is, there is freedom in the form of the optical axis adjustment mechanism, and various design changes are possible.

[0090] FIG. 11A is a diagram showing changes in the dot pattern light DPA (dotted line) and the dot pattern light DPB (dash-dotted line) when the relative angle θ is changed by the optical axis adjustment mechanism according to the first modification.

[0091] When the relative angle θ is adjusted to the angle shown in the example EX1 of FIG. 11A, the irradiation positions of the dots DA and DB on the distant object OB become almost equal pitch. Therefore, the spatial resolution of the distance image can be improved for the distant object OB.

[0092] When the relative angle θ is adjusted to the angle shown in the example EX2 of FIG. 11A, the irradiation positions of the dots DA and DB on the near object OB become almost equal pitch. Therefore, the spatial resolution of the distance image can be improved for the near object OB.

[0093] Thus, in the first modified example, the user can improve the spatial resolution of the distance image with respect to the target object OB by operating the optical axis adjustment mechanism, which is an example of the adjustment unit, to adjust the relative angle θ.

[0094] In the example EX2 of FIG. 11A, dots DA and DB are close to each other at a long distance. Considering the case where the dot light has a divergence angle, for example, as shown in FIG. 11B, the diameter of the dot expands as the distance increases. Therefore, at a long distance, when the diameters of dots DA and DB are larger than the distance between dots DA and DB, the light projection areas of dots DA and DB have an interference area I that interferes with each other. In this case, when dot pattern light DPA and dot pattern light DPB are simultaneously projected, the projection intensity increases in the interference area I. By utilizing this, it is possible to achieve an improvement in the signal-to-noise ratio (SN ratio) at a long distance where the decrease in the SN ratio due to the decrease in the light amount was a problem.

[0095] For example, a mode for long distances may be mounted on the light transmitting and receiving device 2 in which the relative angle θ is adjusted so that the diameters of dots DA and DB are larger than the distance between dots DA and DB at the target long distance, and then dot pattern light DPA and dot pattern light DPB are simultaneously projected.

[0096] In the above embodiments, the configuration in which the optical axis AXA and the optical axis AXB form a relative angle θ in the vertical direction in the figure, that is, the optical axis AXA and the optical axis AXB are inclined from the horizontal direction to the vertical direction in the figure and are parallel to the plane defined by the vertical direction and the horizontal direction has been given. However, for example, as shown in FIG. 21, a configuration in which the optical axis AXA and the optical axis AXB form a relative angle θ in the horizontal direction in the figure may be adopted. Further, a configuration in which the optical axis AXA and the optical axis AXB form a relative angle θ in a two-degree-of-freedom system of the vertical direction and the horizontal direction in the figure, that is, the optical axis AXA and the optical axis AXB are inclined from the horizontal direction to the vertical direction and are also non-parallel to the plane defined by the vertical direction and the horizontal direction may be adopted. Even when these configurations are adopted, the same effects as those of the above embodiments can be obtained. The vertical direction is an example of the first axis, and the horizontal direction is an example of the second axis.

[0097] Figures 12A, 12B, 13A, and 13B are diagrams showing the optical axis adjustment mechanism according to Modification 2 of the present disclosure. In these figures, for the sake of simplicity of explanation, the optical axis AXA and the optical axis AXB are shown to be parallel.

[0098] In the optical axis adjustment mechanism according to Modification 2, the positional difference D between the light projecting units 100A and 100B can be adjusted.

[0099] Specifically, in Modification 2, a pair of long holes 182A and 182B are formed in the housing 180. The long holes 182A and 182B have a shape extending in the same direction. The light projecting units 100A and 100B are supported by the housing 180 so as to be slidable along the longitudinal directions of the long holes 182A and 182B, respectively.

[0100] The light projecting units 100A and 100B slide along the longitudinal directions of the long holes 182A and 182B, respectively, by driving means such as a motor or a manual operation by a user. As a result, the positional difference D between the light projecting units 100A and 100B becomes smaller (see Fig. 12A) or larger (see Fig. 13A).

[0101] When the positional difference D is reduced, for example, as shown in Fig. 12B, the irradiation positions of the dots DA and DB on the short-distance object OB become almost equal pitch. Therefore, the spatial resolution of the distance image can be improved for the short-distance object OB.

[0102] Here, when the diameters of the dots DA and DB are larger than the positional difference D at a certain distance or more, for example, as in the example of Fig. 11B, the light projection areas of the dots DA and DB have an interference area I that interferes with each other (see Fig. 12C). In this case, when the dot pattern light DPA and the dot pattern light DPB are projected simultaneously, the light projection intensity increases in the interference area I. By utilizing this, it is possible to achieve an improvement in the SNR at a long distance where a decrease in the SNR due to a decrease in the light amount has been a problem.

[0103] For example, a long-distance mode may be mounted on the light transmitting and receiving device 2, in which the dot pattern lights DPA and DPB are simultaneously projected after bringing the positional difference D closer so that the diameters of the dots DA and DB become larger than the positional difference D at a target long distance.

[0104] When the positional difference D is increased, as shown in FIG. 13B for example, the irradiation positions of the dots DA and DB on the long-distance object OB become substantially equidistant. Therefore, the spatial resolution of the distance image can be improved for the object OB at both long and short distances.

[0105] On the sides of the slots 182A and 182B, a scale 184 indicating the irradiation distance is engraved. In the case of manual operation, the user can improve the spatial resolution of the distance image for the object OB at the target distance by adjusting the positional difference D while visually recognizing the scale 184.

[0106] That is, the user can improve the spatial resolution of the distance image for the target object OB by operating the optical axis adjustment mechanism, which is an example of the adjustment unit, to adjust the positional difference D between the light projecting unit 100A and the light projecting unit 100B.

[0107] In the above embodiments, the configuration in which the light projecting unit 100A and the light projecting unit 100B are arranged with the positional difference D in the vertical direction in the drawing has been given. However, for example, as shown in FIG. 22, a configuration having an adjustment mechanism capable of adjusting the relative positions of the light projecting unit 100A and the light projecting unit 100B in the horizontal direction in the drawing may be adopted. Also, a configuration having a two-degree-of-freedom adjustment mechanism capable of adjusting the relative positions of the light projecting unit 100A and the light projecting unit 100B in both the vertical and horizontal directions in the drawing may be adopted. Even when these configurations are adopted, the same effects as those of the above embodiments can be obtained.

[0108] The optical axis adjustment mechanism of Modification 1 and the optical axis adjustment mechanism of Modification 2 may be appropriately combined. That is, in the light transmitting and receiving device 2, a configuration capable of adjusting both the relative angle θ and the positional difference D may be adopted.

[0109] FIG. 14 is a block diagram showing the configuration of the distance measurement system 1 according to Modification 3 of the present disclosure.

[0110] As shown in FIG. 14, in this Modification 3, the control unit 140 includes an adjustment unit 141. The adjustment unit 141 controls means (such as a motor) for driving the optical axis adjustment mechanism of Modification 1 (see FIG. 10) or the optical axis adjustment mechanism of Modification 2 (see FIGS. 12A and 13A) to adjust at least one of the relative angle θ and the position difference D.

[0111] FIG. 15 is a flowchart showing the processing executed by the distance measurement system 1 according to Modification 3 of the present disclosure.

[0112] Note that each step of the flowchart shown in the present disclosure may be rearranged within a non - conflicting range. Also, each step of the flowchart shown in the present disclosure may be executed in parallel or in series within a non - conflicting range. For example, in the present disclosure, although the processing of various steps is presented using an exemplary order, it is not limited to this presented order.

[0113] As shown in FIG. 15, the arithmetic unit 3 waits for a distance selection operation by the user (step S101).

[0114] Exemplarily, the arithmetic unit 3 displays a list listing a plurality of distances on the screen. The user can select an arbitrary distance from the distance list displayed on the screen.

[0115] When a distance selection operation by the user is performed (step S101: YES), the arithmetic unit 3 (arithmetic section 31) of the arithmetic unit 3 sets the distance from the light transmitting and receiving device 2 according to this operation (step S102).

[0116] That is, the arithmetic section 31 operates as a distance setting unit that sets the distance from the light transmitting and receiving device 2 according to the operation input from the user to the arithmetic unit 3.

[0117] The arithmetic unit 3 acquires one or both of the relative angle θ and the position difference D (step S103).

[0118] Specifically, the arithmetic unit 3 acquires the relative angle θ and the position difference D by geometrically calculating from the distance set in step S102. The arithmetic unit 3 may acquire the relative angle θ and the position difference D corresponding to the distance set in step S102 from a previously stored table.

[0119] The arithmetic unit 3 transmits the relative angle θ and the position difference D acquired in step S103 to the control unit 140.

[0120] The control unit 140 (adjustment unit 141) controls the optical axis adjustment mechanism so that the relative angle θ and the position difference D received from the arithmetic unit 3 are obtained, and adjusts the posture and position of each light projecting unit 100 (step S104).

[0121] Specifically, the adjustment unit 141 adjusts at least one of the relative angle θ and the position difference D so that the dot DA and the dot DB (an example of the pattern light projected from each of the pair of light projecting units) are not irradiated overlapping each other on the object OB at the distance set in the arithmetic unit 31. As a result, for example, on the object OB at the said distance, the dot DA and the dot DB are irradiated without overlapping. Therefore, it is possible to improve the spatial resolution of the distance image with respect to the object OB at the target distance.

[0122] The method for acquiring the relative angle θ and the position difference D is not limited to that shown in Modification 3. For example, the arithmetic unit 31 projects the dot pattern light DPA to acquire distance information, and detects the distance at which many objects OB exist from the acquired distance information. The arithmetic unit 31 determines the relative angle θ and the position difference D so that the dot DA and the dot DB do not overlap at the detected distance.

[0123] In this case, after controlling the optical axis adjustment mechanism so that the determined relative angle θ and position difference D are obtained, the light projecting and receiving device 2 projects the dot pattern light DPB.

[0124] In this example, the adjustment unit 141 acquires information on the reflected light corresponding to the pattern light (e.g., dot pattern light DPA) projected from one of the pair of light projecting units 100 from the ToF light receiving unit 120, determines the position and the direction of the optical axis of the other of the pair of light projecting units 100 (e.g., light projecting unit 100B) based on the acquired reflected light information, and adjusts the relative angle θ and the positional difference D by changing the position and the direction of the optical axis of the other of the pair of light projecting units 100 to the determined position and the direction of the optical axis.

[0125] FIG. 16 shows a flowchart of the process executed in the distance measurement system 1 according to Modification Example 4 of the present disclosure.

[0126] In this Modification Example 4, for example, the relative angle θ is tentatively changed, and in each changed state, the degree of overlap of the dot pattern light DPA and the dot pattern light DPB on the object OB is detected, and the relative angle θ is determined based on each detected degree of overlap.

[0127] In this Modification Example 4, the light projecting and receiving device 2 drives the light projecting units 100A and 100B by driving means such as a motor to reset the relative angle θ to an initial angle (e.g., an angle serving as the origin) (step S201).

[0128] The light projecting and receiving device 2 projects the dot pattern light DPA and acquires information on the reflected light from the ToF light receiving unit 120, and also projects the dot pattern light DPB and acquires information on the reflected light from the ToF light receiving unit 120 (step S202). The light projecting and receiving device 2 transmits this information to the arithmetic unit 3.

[0129] The arithmetic unit 3 calculates the overlapping rate (degree of overlap) of the dot pattern light DPA and the dot pattern light DPB based on the information on each reflected light received from the light projecting and receiving device 2 (step S203).

[0130] This overlapping rate is, for example, the total of the values calculated for each object OB at each distance (the value obtained by dividing the area where dots DA and DB overlap on the object OB by the area where dots DA and DB do not overlap on the same object OB).

[0131] The light transmitting and receiving device 2 drives the light projecting units 100A and 100B by driving means to slightly vary the relative angle θ (step S204).

[0132] The processes of steps S202 to S204 are repeatedly executed until the relative angle θ reaches the upper limit angle.

[0133] When the relative angle θ reaches the upper limit angle (step S205: YES), the arithmetic unit 3 determines the relative angle θ at which the overlapping rate between the dot pattern light DPA and the dot pattern light DPB becomes minimum (step S206).

[0134] The arithmetic unit 3 presents to the user the relative angle θ determined in step S206 to have the minimum overlapping rate, for example, by displaying it on the screen (step S207).

[0135] The arithmetic unit 3 may present a plurality of relative angles θ (for example, the relative angles θ corresponding to the lower four overlapping rates) to the user. The user can select one relative angle θ from among the plurality of relative angles θ.

[0136] When a user operation to permit (or select) the presented relative angle θ is performed (step S208: YES), the light transmitting and receiving device 2 controls the optical axis adjustment mechanism to adjust to the presented relative angle θ (step S209).

[0137] When a user operation not to permit (or not to select) the presented relative angle θ is performed (step S208: NO), the light transmitting and receiving device 2 controls the optical axis adjustment mechanism to adjust to a default relative angle θ (step S210).

[0138] In the above description of this modification 4, only the relative angle θ is adjusted, but instead of or in addition to the relative angle θ, the position difference D may be adjusted.

[0139] According to the fourth modification example, even when the object OB is located at various distances, ToF imaging can be performed with appropriate dot pattern light, and the spatial resolution of the distance image can be improved for objects OB at various distances.

[0140] As described above, in the fourth modification example, the control unit 140 (adjustment unit 141) temporarily changes at least one of the relative angle θ and the positional difference D, and in each change state, the overlapping degree of the dot pattern light DPA and the dot pattern light DPB (an example of the mutual pattern lights projected from each of the pair of light projecting units 100) on the object OB is detected, and at least one of the relative angle θ and the positional difference D is determined based on each detected overlapping degree.

[0141] In order to obtain an angle of view over the entire circumference of the device, the light transmitting and receiving device 2 of the above embodiment includes a light projecting unit 100A and a light projecting unit 100B on both sides of the device as shown in FIG. 5, but the configuration of the light transmitting and receiving device 2 of the present disclosure is not limited thereto.

[0142] FIG. 17 is a diagram showing another arrangement example of each optical element of the light transmitting and receiving device 2.

[0143] The light transmitting and receiving device 2 shown in FIG. 17 includes light projecting units 100A and 100B and a ToF light receiving unit 120 on one side of the light transmitting and receiving device 2.

[0144] This light transmitting and receiving device 2 further includes a device 200 such as a tripod that functions as a support unit, and a rotating means 220 that is a rotating table rotated by, for example, an electric motor or the like on the upper stage of the device 200. In this case, the light transmitting and receiving device 2 rotates the device itself by the rotating means 220 to change the imaging direction and acquires distance images of the entire circumference.

[0145] When the rotation control unit of the light transmitting and receiving device 2 controls the rotation of the rotating means 220, it has a predetermined rotation pattern. For example, when the rotation control unit of the light transmitting and receiving device 2 performs light transmission and reception around the entire device in multiple times, it controls according to a rotation pattern of (1) rotating by a predetermined distance, (2) stopping the rotation to perform light transmission and reception, (3) rotating by a predetermined distance, and (4) stopping the rotation to perform light transmission and reception imaging.

[0146] FIG. 18 is a block diagram showing the configuration of a three-dimensional shape generation system 4 according to another embodiment of the present disclosure.

[0147] As shown in FIG. 18, the three-dimensional shape generation system 4 includes a light transmitting and receiving device 2 and an arithmetic device 3A. The three-dimensional shape generation system 4 is obtained by adding a generation processing unit 32 and a storage unit 33 to the arithmetic device 3 of the distance measurement system 1 in FIG. 1.

[0148] In addition to the arithmetic unit 31, the arithmetic device 3A includes a generation processing unit 32 that generates three-dimensional shape information based on the acquired three-dimensional point cloud. The three-dimensional shape information is information indicating the three-dimensional shape of an object that can be handled by a computer or the like.

[0149] The information indicating the three-dimensional shape is information that can geometrically specify the three-dimensional shape. For example, when representing a sphere, the coordinates of the center and the radius, and when representing a polyhedron (polygon), the coordinate points of each vertex of the polyhedron respectively correspond to the information indicating the three-dimensional shape. In addition to the information indicating the three-dimensional shape of the object, information related to the color and material of the object may be added to the three-dimensional shape information.

[0150] In the storage unit 33 of the arithmetic device 3A, a setting information management DB301, a storage processing management DB302, a point cloud management DB303, and a three-dimensional shape management DB304 are constructed.

[0151] The setting information management DB301 stores and manages various types of information. The memory processing management DB302 stores and manages various processing programs for generating a three-dimensional shape. The point cloud management DB303 stores and manages the three-dimensional point cloud information acquired by the distance measurement system. The three-dimensional shape management DB304 stores and manages the three-dimensional shape information.

[0152] The setting information management DB301 is composed of a setting information management table. The setting information management table is a table for managing the three-dimensional point cloud data for generating a three-dimensional shape, and the execution order and processing mode of the generation process (three-dimensional shape generation process) for generating a three-dimensional shape. In this setting information management table, the file name of the three-dimensional point cloud data, and the execution order and processing mode of the three-dimensional shape generation process are managed in an associated manner.

[0153] The three-dimensional shape generation process includes, for example, a registration process, a noise removal process, a segmentation process, and a modeling process.

[0154] The registration process is a process of converting a plurality of three-dimensional point clouds into one integrated three-dimensional point cloud. The noise removal process is a process of removing unnecessary point clouds from the three-dimensional point cloud.

[0155] The segmentation process is a process of labeling specific point clouds in the three-dimensional point cloud so that they can be distinguished from other point clouds, and by labeling each of the plurality of specific point clouds with a different label, each of the plurality of specific point clouds can be made mutually distinguishable.

[0156] The segmentation process may be executed in combination with a clustering process of grouping point clouds with close distances among the labeled point clouds.

[0157] The modeling process is a process of collating a specific point cloud in the three-dimensional point cloud with the three-dimensional model shape and replacing the specific point cloud with the model shape. Note that the three-dimensional model shape is a model such as a template, a prototype used to generate three-dimensional shape information from the three-dimensional point cloud.

[0158] The processing mode includes a manual process that executes part or all of the three-dimensional shape information generation process based on the user's operation input without executing the storage process, an automatic process that executes part or all of the three-dimensional shape information generation process based on the storage process stored in advance without relying on the operation input, and a mixed process that mixes the manual process and the automatic process to execute part or all of the three-dimensional shape information generation process.

[0159] The generation processing unit 32 receives an input operation from the user. The input operation includes, for example, an operation of setting point cloud setting information indicating the three-dimensional point cloud for which processing is to be performed, and an operation of setting processing setting information indicating the execution order and processing mode of each process in the three-dimensional shape information generation process.

[0160] The generation processing unit 32 executes the three-dimensional shape information generation process set by the input operation.

[0161] Specifically, the generation processing unit 32 reads out the three-dimensional point cloud data associated with the point cloud setting information by searching the point cloud management DB303 using the point cloud setting information as a search key. Also, the processing program associated with the processing setting information is read out by searching the storage process management DB302 using the processing mode of the generation process in the processing setting information as a search key.

[0162] The generation processing unit 32 generates three-dimensional shape information based on the three-dimensional point cloud data read from the storage unit 33, the processing program, and the execution order and processing mode of the generation process in the processing setting information.

[0163] When the processing mode includes manual processing and mixed processing, the generation processing unit 32 generates an operation screen for receiving an operation input for generating three-dimensional shape information, displays it on a display unit (such as a display), and receives a predetermined input operation of the user with respect to the displayed operation screen. This input operation includes an operation input for executing the processing in which the manual processing mode is selected in the three-dimensional shape information generation processing. The generation processing unit 32 generates three-dimensional shape information based on the operation input information by the operation input.

[0164] The generation processing unit 32 converts the generated three-dimensional shape information into, for example, CAD (Computer Aided Design) format, and stores the converted three-dimensional shape information in the three-dimensional shape management DB304 or an external recording medium or the like.

[0165] According to the three-dimensional shape generation system according to this embodiment, three-dimensional shape information can be generated based on the point cloud acquired by the distance measurement system. Thereby, for example, three-dimensional modeling of existing buildings in fields such as architecture, construction, and civil engineering becomes easy, and it can be used for BIM (Building Information Modeling) / CIM (Construction Information Modeling).

[0166] Note that the generation processing unit 32 is not limited to being provided in the arithmetic device 3A including the arithmetic unit 31A. For example, it may be provided in another terminal device connected to the arithmetic device 3A by wired communication or wireless communication, or a server arranged on the cloud. Further, the three-dimensional shape generation processing may be executed by a plurality of devices (for example, the arithmetic device 3A and a server connected to the arithmetic device 3A).

[0167] An application example in which the distance measurement system 1 is used in various detection systems will be described with reference to FIGS. 19 to 20. The detection systems in these application examples have respective functional blocks described later in addition to the distance measurement system 1. In FIGS. 19 to 20, functional blocks such as a determination unit included in the detection system are described outside the detection system for convenience of drawing. The various detection systems shown in FIGS. 19 to 20 have a control unit that receives information from the distance measurement system 1 and controls the various detection systems based on the information from the distance measurement system 1.

[0168] FIG. 19 is an example of a shape measurement system as a detection system, and shows an application example in which the distance measurement system 1 is used for user authentication of an electronic device.

[0169] The portable information terminal 60X, which is an electronic device, has a user authentication function. The authentication function may be realized by dedicated hardware, or may be realized by a CPU (Central Processing Unit) that controls the portable information terminal 60X executing a program such as a ROM (Read Only Memory).

[0170] When performing user authentication, light is projected from the light source device of the distance measurement system 1 mounted on the portable information terminal 60X toward the user 61X who uses the portable information terminal 60X.

[0171] The light reflected by the user 61X and its surroundings is received by the light receiving element of the distance measurement system 1, and image data is generated (imaging is performed) by the image processing unit 62X. The determination unit 63X determines the degree of coincidence between the image information of the user 61X imaged by the distance measurement system 1 and the pre-registered user information, and determines whether the user is a registered user.

[0172] Specifically, the shape (contour and unevenness) of the face, ears, head, etc. of the user 61X can be measured and used as user information.

[0173] In the application example of FIG. 19, regarding the detection of the user 61X by the distance measurement system 1, the same effect as that of the distance measurement system 1 (improvement in detection accuracy) can be obtained. In particular, since light is projected from the light projecting device at a wide angle to detect information of the user 61X in a wide range, compared with the case where the detection range is narrow, the amount of information for recognizing the user increases, and improvement in recognition accuracy can be realized.

[0174] Consider the case of recognizing the shapes of the face, ears, head, etc. of the user 61X using the portable information terminal 60X. In this case, the user 61X to be measured is close to the portable information terminal 60X in terms of distance. Particularly in short-distance measurement, since the measurement error due to parallax is large, it is desirable to arrange the ToF light receiving unit 120 between the light projecting unit 100A and the light projecting unit 100B to suppress the measurement error due to parallax. Therefore, in the configuration example of FIG. 23, in order to make it difficult to generate a measurement error due to parallax, the light projecting unit 100A and the light projecting unit 100B are arranged close to the ToF light receiving unit 120 at equal distances respectively.

[0175] FIG. 19 shows an example in which the distance measurement system 1 is mounted on the portable information terminal 60X, but it is also possible to use the user authentication using the distance measurement system 1 for stationary personal computers, OA devices such as printers, and building security systems.

[0176] In terms of function, it is possible to use it not only for the personal authentication function but also for scanning three-dimensional shapes such as the face. Also in this case, by mounting the distance measurement system 1 capable of projecting light at a wide angle, high-precision scanning can be realized.

[0177] FIG. 20 shows an application example in which the distance measurement system 1 is used in an autonomous driving system in a moving body which is an example of a detection system.

[0178] In the application example of FIG. 20, the distance measurement system 1 is used for sensing an object outside the moving body 70X. The moving body 70X is an autonomous driving type moving body capable of automatically traveling while recognizing the external situation.

[0179] The mobile body 70X is equipped with the distance measurement system 1, and the distance measurement system 1 irradiates light toward the traveling direction of the mobile body 70X and its surrounding area. In the room 71X which is the moving area of the mobile body 70X, a desk 72X is installed in the traveling direction of the mobile body 70X.

[0180] Among the light projected from the light source device of the distance measurement system 1 mounted on the mobile body 70X, the light reflected by the desk 72X and its surroundings is received by the light receiving element of the distance measurement system 1, and the electrical signal obtained by photoelectric conversion is sent to the signal processing unit 73X.

[0181] Based on the electrical signal sent from the light receiving element and the like, the signal processing unit 73X calculates information regarding the layout of the room 71X, such as the distance to the desk 72X, the position of the desk 72X, and the surrounding situation other than the desk 72X.

[0182] Based on this calculated information, the determination unit 74X determines the movement path, movement speed, etc. of the mobile body 70X, and based on the determination result of the determination unit 74X, the driving control unit 75X controls the traveling of the mobile body 70X (such as the operation of the motor which is the drive source).

[0183] In the application example of FIG. 20, regarding the layout detection of the room 71X by the distance measurement system 1, the same effect (improvement of detection accuracy) as that of the distance measurement system 1 can be obtained. In particular, since light is projected from the light projecting device at a wide angle to detect information of the room 71X in a wide range, a larger amount of information can be obtained compared to the case where the detection range is narrow, and the accuracy improvement of the autonomous driving of the mobile body 70X can be realized.

[0184] FIG. 20 shows an example in which the distance measurement system 1 is mounted on the autonomous driving mobile body 70X traveling in the room 71X, but it can also be applied to an autonomous driving vehicle (so-called self-driving vehicle) traveling outdoors.

[0185] In addition, it can also be applied to a driving support system in a mobile body such as an automobile driven by a driver, rather than an autonomous driving type. In this case, the distance measurement system 1 can be used to detect the surrounding situation of the mobile body, and the driving of the driver can be supported according to the detected surrounding situation.

[0186] In addition to the above, the distance measurement system 1 may be applied to an article inspection system in a factory or the like. Specifically, based on the information acquired by the distance measurement system 1, the determination unit of the article inspection system determines the state of each article.

[0187] Also, the distance measurement system 1 may be applied to the operation control of a movable device.

[0188] The articulated arm as a movable device has a plurality of arms connected by bendable joints and is provided with a hand portion at its tip. The articulated arm is used, for example, on an assembly line in a factory, and grips an object with the hand portion during inspection, conveyance, and assembly of the object.

[0189] The distance measurement system 1 detects an object and its surrounding area, and the determination unit of the movable device determines various information about the object, such as the distance to the object, the shape of the object, the position of the object, and the positional relationship between a plurality of objects if they exist, based on the information acquired by the distance measurement system 1. Then, based on the determination result of the determination unit, the drive control unit controls the operation of the articulated arm.

[0190] Also, the distance measurement system 1 may be applied to a driving support system in a moving body such as an automobile.

[0191] The distance measurement system 1 mounted inside the automobile detects the driver driving the automobile and the surrounding area thereof. The determination unit of the driving support system determines information such as the face (expression) and posture of the driver based on the information acquired by the distance measurement system 1. Then, based on the determination result of the determination unit, the control unit performs appropriate driving support according to the situation of the driver.

[0192] The shape measurement system, the moving body, the article inspection system, the movable device, and the driving support system are all examples of detection systems. In the distance measurement system 1 of the present embodiment, the spatial resolution of the distance image can be improved. Therefore, the detection system to which the distance measurement system 1 is applied can perform highly accurate detection.

[0193] The invention described in the claims of the present application at the time of filing is appended below. [Appendix 1] A plurality of light projecting units, a light receiving unit that receives reflected light from an object on which light from the plurality of light projecting units is projected, and comprising each of the light projecting units includes a light source that emits light and an optical system that projects pattern light obtained from the light emitted from the light source onto an object, the optical systems of each of the plurality of light projecting units are arranged non-parallel to each other such that their optical axes form an angle, a light projecting and receiving device. [Appendix 2] including a defining unit that defines a relative angle formed by the optical axes of the plurality of light projecting units and a positional difference between the plurality of light projecting units so that the pattern light projected from each of the plurality of light projecting units does not overlap and irradiate each other on an object located at a position of a first distance or more from the light projecting unit, the light projecting and receiving device according to Appendix 1. [Appendix 3] the pattern light projected from each of the plurality of light projecting units is pattern light in which a plurality of emitted lights are regularly arranged, a first emitted light projected from one of the plurality of light projecting units and a second emitted light projected from another one of the plurality of light projecting units are at least partially overlapped and irradiated on an object located at a second distance shorter than the first distance from the light projecting unit, on an object located at a third distance longer than the first distance from the light projecting device, the first emitted light and the second emitted light do not overlap and are irradiated so that the first emitted light and the second emitted light are alternately and periodically arranged, the light projecting and receiving device according to Appendix 2. [Appendix 4] including an adjusting unit that adjusts at least one of the relative angle formed by the optical axes of the plurality of light projecting units and the positional difference between the plurality of light projecting units, the light projecting and receiving device according to any one of Appendices 1 to 3. [Appendix 5] further comprising a distance setting unit that sets the distance from the light projecting device according to an operation input, the adjusting unit adjusts at least one of the relative angle and the positional difference so that the pattern lights projected from each of the plurality of light projecting units do not overlap and irradiate each other on an object at the distance set by the distance setting unit, The light projecting and receiving device according to Supplementary Note 4. [Supplementary Note 6] The adjusting unit, changes at least one of the relative angle and the positional difference, and in the changed state, detects the degree of overlap of each pattern light projected from each of the plurality of light projecting units on the object, The light projecting and receiving device according to Supplementary Note 4 or Supplementary Note 5. [Supplementary Note 7] The adjusting unit, determines at least one of the relative angle and the positional difference based on each detected degree of overlap, The light projecting and receiving device according to Supplementary Note 6. [Supplementary Note 8] The adjusting unit, acquires information on reflected light corresponding to the pattern light projected from one of the plurality of light projecting units from the light receiving unit, determines the position or the direction of the optical axis of another one of the plurality of light projecting units based on the acquired information on the reflected light, adjusts at least one of the relative angle and the positional difference by changing the position or the direction of the optical axis of another one of the plurality of light projecting units to the determined position or the direction of the optical axis, The light projecting and receiving device according to any one of Supplementary Notes 4 to 7. [Supplementary Note 9] the optical systems of each of the plurality of light projecting units are arranged such that their optical axes intersect in the light emission direction, The light projecting and receiving device according to any one of Supplementary Notes 1 to 8. [Supplementary Note 10] the plurality of light projecting units are arranged side by side on a first axis, The optical axis of the optical system of each of the plurality of light projecting units inclines from a second axis perpendicular to the first axis toward the first axis and is non-parallel to the plane defined by the first axis and the second axis. The light transmitting and receiving device according to any one of Appendices 1 to 9. [Appendix 11] The light transmitting and receiving device according to any one of Appendices 1 to 10, and a distance calculation unit that calculates the distance to the object based on the output from the light reception by the light receiving unit. A distance measurement system. [Appendix 12] including a plurality of light projecting units, each of the light projecting units includes a light source that emits at least one light, and an optical system that projects pattern light obtained from the light emitted from the light source onto an object, the plurality of light projecting units are arranged non-parallel to each other such that their optical axes form an angle. A light projecting device.

Explanation of Signs

[0194] 1: Distance measurement system 2: Light transmitting and receiving device 3: Arithmetic unit 10: Light transmitting and receiving unit 20: RGB light receiving unit 31: Arithmetic unit 100A: Light projecting unit 100B: Light projecting unit 102: Light source 104: First lens group 106: Diffractive optical element 108: Second lens group 120: ToF light receiving unit 122: Optical system 124: ToF sensor 140: Control unit 141: Adjusting unit 160A: Lens barrel 160B: Lens barrel 170: Screw 172: Coil spring 174: Adjusting screw 180: Housing 182A: Long hole 182B: Long hole 184: Scale

Claims

1. A plurality of light projecting units; a light receiving unit that receives reflected light from an object onto which light from the plurality of light projecting units is projected; Equipped with Each of the light projecting units includes a light source that emits light, and an optical system that projects a pattern of light obtained from the light emitted from the light source onto an object; The optical systems of the plurality of light projecting units are arranged non-parallel to each other so that their optical axes form an angle with each other. Light emitting / receiving device.

2. a defining unit that defines a relative angle between the optical axes of the light projecting units and a position difference between the light projecting units so that the pattern lights projected from the light projecting units are not overlapped with each other on an object located at a position equal to or greater than a first distance from the light projecting units, The light projecting and receiving device according to claim 1.

3. the pattern light projected by each of the plurality of light projecting units is a pattern light in which a plurality of emitted lights are regularly arranged, a first outgoing light projected from one of the plurality of light-projecting units and a second outgoing light projected from another of the plurality of light-projecting units are irradiated on an object located at a second distance from the light-projecting unit that is shorter than the first distance, with at least a portion of the first outgoing light overlapping with each other; The first emitted light and the second emitted light are irradiated on an object located at a third distance from the light projecting device that is longer than the first distance such that the first emitted light and the second emitted light do not overlap and are alternately and periodically arranged. The light projecting and receiving device according to claim 2.

4. and an adjustment unit that adjusts at least one of a relative angle between the optical axes and a position difference between the plurality of light projecting units. The light projecting and receiving device according to claim 1.

5. a distance setting unit that sets a distance from the light projecting device in accordance with an operation input, the adjustment unit adjusts at least one of the relative angle and the position difference so that the pattern lights projected from each of the plurality of light projecting units are not irradiated in an overlapping manner on the target object at the distance set by the distance setting unit.

5. The light projecting and receiving device according to claim 4.

6. The adjustment unit is changing at least one of the relative angle and the position difference, and detecting an overlapping degree of each of the pattern lights projected from each of the plurality of light projecting units on the target object in a state in which the relative angle and the position difference are changed; 5. The light projecting and receiving device according to claim 4.

7. The adjustment unit is determining at least one of the relative angle and the position difference based on each of the detected overlapping degrees; 7. The light projecting and receiving device according to claim 6.

8. The adjustment unit is acquiring, from the light receiving unit, information on reflected light corresponding to a pattern of light projected from one of the plurality of light projecting units; determining a position or an orientation of an optical axis of another one of the plurality of light projectors based on the acquired information of the reflected light; changing a position or an orientation of an optical axis of another one of the plurality of light-projecting units to the determined position or orientation of the optical axis, thereby adjusting at least one of the relative angle and the position difference; 5. The light projecting and receiving device according to claim 4.

9. The optical systems of the plurality of light projecting units are arranged such that their optical axes intersect in the light emission direction. The light projecting and receiving device according to any one of claims 1 to 8.

10. The plurality of light projecting units are arranged in line along a first axis, an optical axis of each of the optical systems of the plurality of light projecting units is inclined from a second axis perpendicular to the first axis toward the first axis and is non-parallel to a plane defined by the first axis and the second axis; The light projecting and receiving device according to any one of claims 1 to 8.

11. A light projecting and receiving device according to any one of claims 1 to 8, A distance calculation unit that calculates a distance to the object based on an output of the light receiving unit. Ranging system.

12. A light emitting device including a plurality of light emitting units; Each of the light projecting units includes a light source that emits at least one light, and an optical system that projects a pattern light obtained from the light emitted from the light source onto an object, The plurality of light projecting units are arranged non-parallel to each other so that their optical axes form an angle with each other. Floodlight.

Images