Light projecting device, light projecting / receiving device, and ranging system

Abstract

To provide a light projecting device, a light projecting / receiving device, and a ranging system that reduce adverse effects caused by reflection of 0th-order light on a background.SOLUTION: A light projecting device includes a light source that emits irradiation light and a diffractive optical element that converts the irradiation light into patterned light. The light source and the diffractive optical element are configured so that the patterned light includes, at mutually different positions, a plurality of light components of ±1st-order light or higher, a first light component of the plurality of light components overlaps but a second light component of the plurality of light components does not overlap a first region within the region in which the 0th-order light is located, and the first light component does not overlap but the second light component overlaps a second region different from the first region within the region in which the 0th-order light is located. Alternatively, the light source and the diffractive optical element are configured such that the patterned light includes, at mutually different positions, a plurality of light components of ±1st-order light or higher, and includes a region in which light components of the ±1st-order light among the plurality of light components overlap but light components of orders higher than ±1st-order among the plurality of light components do not overlap the region in which the 0th-order light is located.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a light projecting device, a light projecting and receiving device, and a distance measuring system.

Background Art

[0002] Patent Document 1 describes a pattern projector having a laser light source and a diffractive optical element. The laser light source has a plurality of laser light emitting units, and the plurality of laser light emitting units are arranged. The diffractive optical element converts the irradiation light from each laser light emitting unit into an isolated light pattern composed of a plurality of discrete isolated lights. The isolated light pattern includes multiplexed isolated lights in which the irradiation lights from the respective laser light emitting units overlap. When the multiplexed isolated lights are viewed from the irradiation surface, the expected angles of the plurality of irradiation lights forming the multiplexed isolated lights are different from each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the intensive research of the present inventor, the prior art including the pattern projector of Patent Document 1 has room for improvement from the viewpoint of reducing the possibility that the zero-order light among each order existing in the isolated light pattern, that is, the zero-order light, the first-order light, the second-order light, ···, is reflected into the background and has an adverse effect.

[0005] The present invention has been made in view of the above circumstances, and one of the objects is to provide a light projecting device, a light projecting and receiving device, and a distance measuring system capable of reducing the possibility that the zero-order light is reflected into the background and has an adverse effect.

Means for Solving the Problems

[0006] In one aspect, the light projecting device of the present embodiment includes a light source that emits irradiation light, and a diffractive optical element that converts the irradiation light into pattern light. The light source and the diffractive optical element are configured such that the pattern light includes a plurality of light components of ± first-order light or higher at different positions, and a first light component among the plurality of light components overlaps with a first region of a region where the zero-order light is located, and a second light component among the plurality of light components does not overlap (resulting in different positions from each other), and the first light component does not overlap (resulting in different positions from each other) with a second region different from the first region of the region where the zero-order light is located, and the second light component overlaps.

[0007] In another aspect, the light projecting device of the present embodiment includes a light source that emits irradiation light, and a diffractive optical element that converts the irradiation light into pattern light. The light source and the diffractive optical element are configured such that the pattern light includes a plurality of light components of ± first-order light or higher at different positions, and in a region where the zero-order light is located, there is a region where the light components of ± first-order light among the plurality of light components overlap and light components of an order higher than ± first-order among the plurality of light components do not overlap (resulting in different positions from each other).

Advantages of the Invention

[0008] According to the present invention, it is possible to provide a light projecting device, a light projecting and receiving device, and a distance measurement system that can reduce the risk of the zero-order light being reflected into the background and causing adverse effects.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Best Mode for Carrying Out the Invention

[0010] Hereinafter, with reference to the drawings, the light projecting device, the light projecting and receiving device, and the distance measuring system of the present embodiment will be described in detail. In the following description, for common or corresponding elements, the same or similar reference numerals are given, and duplicate descriptions are appropriately simplified or omitted as necessary.

[0011] <Overall Configuration of Distance Measuring System 1> FIG. 1 is a block diagram showing an example of the configuration of the distance measuring system 1.

[0012] The distance measuring system 1 includes a light projecting and receiving device 2 and an arithmetic device 3.

[0013] The distance measuring system 1 according to the present embodiment is a system that measures the distance from the light projecting and receiving device 2 to an object (referred to as "target object OB") by the ToF (Time of Flight) method. In the ToF method, distance measuring light having a wavelength different from that of visible light (for example, laser light) is irradiated onto the target object OB. Based on the time difference between the emission timing and the reception timing of the laser light at each irradiation position, the distance to each part (each irradiation position) of the target object OB is calculated.

[0014] In the present embodiment, a diffractive optical element is used in the light projecting optical system that emits laser light. By projecting pattern light such as dots onto the target object OB using the diffractive optical element, high-intensity and high-density point group data can be obtained. Hereinafter, the diffractive optical element is referred to as "DOE (Diffractive Optical Element)".

[0015] The light projecting and receiving device 2 includes a light projecting and receiving unit 10 and an RGB light receiving unit 20. The "light projecting and receiving device" may be referred to as a "light projecting device", an "imaging device", or a "distance measuring device".

[0016] The light projecting and receiving device 2 is equipped with, for example, a rechargeable battery. That is, the light projecting and receiving device 2 is driven by the battery. The light projecting and receiving device 2 may also be drivable by a commercial power supply.

[0017] The light transmitting and receiving unit 10 includes a ToF light transmitting unit (light transmitting device) 100, a ToF light receiving unit (light receiving unit) 120, and a control unit 140.

[0018] FIG. 2 is a block diagram showing an example of the configuration of the light transmitting and receiving unit 10.

[0019] The ToF light transmitting unit 100 projects pattern light onto the object OB. As shown in FIG. 2, the ToF light transmitting unit 100 includes a light source 102, a first lens group 104, a DOE 106, and a second lens group 108.

[0020] The control unit 140 controls the ToF light transmitting unit 100 and the ToF light receiving unit 120. Specifically, as a circuit configuration, the control unit 140 includes a CPU (Central Processing Unit), a light source driving circuit, an imaging signal processing circuit, an input / output circuit, a memory, and the like.

[0021] The control unit 140 is, for example, a single processor or a multi-processor and includes at least one processor. When configured to include a plurality of processors, the control unit 140 may be packaged as a single device, or may be composed of a plurality of physically separated devices within the light transmitting and receiving unit 10.

[0022] The light source 102 is an example of a light source that emits at least one light, that is, emits irradiation light. The light source 102 is, for example, an LD (Laser Diode) that emits laser light. The light source 102 emits laser light at a timing controlled by the control unit 140.

[0023] The first lens group 104, the DOE 106, and the second lens group 108 are an example of an optical system that projects pattern light obtained from the laser light emitted from the light source 102 onto the object OB. Note that the configuration of the optical system (light transmitting optical system) is not limited to that illustrated in FIG. 2, and various design changes are possible. For example, three or more lens groups including a third lens group may be provided.

[0024] The first lens group 104 is an example of the first lens group. Laser light emitted from the light source 102 is incident on the first lens group 104.

[0025] The DOE 106 is an example of an optical element that obtains pattern light from the laser light (irradiation light) incident through the first lens group 104 (converts the irradiation light into pattern light, forms pattern light from the irradiation light). The DOE 106 emits, for example, pattern light in which a plurality of dots are arranged in a grid pattern (hereinafter referred to as "dot pattern light").

[0026] In the present embodiment, there are features (devices) in the structures (arrangement properties) of the light source 102 and the DOE 106, and the details thereof will be described later.

[0027] The second lens group 108 is an example of the second lens group. Dot pattern light emitted from the DOE 106 is incident on the second lens group 108. The second lens group 108 projects this dot pattern light onto the object OB.

[0028] FIG. 3 is a diagram for explaining an example of ToF imaging by the light transmitting and receiving device 2. In FIG. 3, for convenience, the illustration of the RGB light receiving unit 20 is omitted.

[0029] As shown in FIG. 3, the light transmitting and receiving device 2 projects dot pattern light (depicted by thick arrows in FIG. 3) onto the object OB. In order to project this type of pattern light, for example, the light source 102 may be configured by a VCSEL (Vertical Cavity Surface Emitting Laser) having a plurality of pattern-arranged light emitting portions, or the pattern may be formed in a dot shape by the diffraction phenomenon of the fine structure of the DOE 106, or both may be applied. Thereby, high-brightness and high-density dot pattern light can be projected onto the object OB, and the ranging accuracy is improved. In the present embodiment, as an example, it is assumed that both configuring the light source 102 by a VCSEL having a plurality of pattern-arranged light emitting portions and forming the pattern in a dot shape by the diffraction phenomenon of the fine structure of the DOE 106 are applied.

[0030] Supplement the description of the case where the diffraction pattern of DOE106 is formed in a dot pattern. Hereinafter, the case where the light source 102 is composed of a point light source such as an LD, for example, or the case where attention is paid to one of a plurality of light sources such as a VCSEL will be exemplified and described. The laser light emitted from the light source 102 is incident on the DOE106 through the first lens group 104.

[0031] When the laser light is incident on the DOE106, a conjugate point corresponding to the diffraction order (0th order, ±1st order, ±2nd order, ···) of the DOE106 is generated, and dot pattern light corresponding to the diffraction order is irradiated to infinity. When a VCSEL light source and a DOE are used in combination, the dot pattern light can be irradiated over a wider range.

[0032] The dot pattern light irradiated on the object OB is reflected or scattered by the object OB. The ToF light receiving unit 120 receives the light directly reflected by the object OB (hereinafter referred to as "direct reflected light"). (The direct reflected light is drawn as a thin arrow in FIG. 3).

[0033] The ToF light receiving unit 120 is an example of a light receiving unit that receives the reflected light from the object OB irradiated with the dot pattern light. As shown in FIG. 2, the ToF light receiving unit 120 includes an optical system 122 and a ToF sensor 124.

[0034] The optical system 122 includes, for example, an aperture, an imaging optical system, and a filter. The direct reflected light from the object OB irradiated with the dot pattern light passes through the optical system 122 and is received by the ToF sensor 124.

[0035] The ToF sensor 124 is an imaging element such as a CMOS image sensor, for example, and photoelectrically converts the sum of the exposure amounts in a plurality of exposure periods having a predetermined phase difference with respect to the irradiated light and outputs the result to the control unit 140. The light reception data by the ToF sensor 124 is input to the arithmetic unit 3 via the control unit 140.

[0036] The arithmetic unit 31 of the arithmetic device 3 is realized by an instruction from the CPU of the arithmetic device 3, and calculates the distance to each part (each irradiation position) of the object OB based on the sum of the exposure amounts of each exposure period input from the ToF sensor 124. Note that, using an SPAD or the like as the ToF sensor 124, the arithmetic unit 31 may calculate the distance based on the time difference between the emission timing of the laser light (the light emission timing of the light source 102) and the light reception timing (the input timing from the ToF sensor 124) at each irradiation position.

[0037] That is, the arithmetic unit 31 of the arithmetic device 3 is an example of a distance calculation unit that calculates the distance to the object OB based on the output from the light reception by the ToF light reception unit 120.

[0038] The RGB light reception unit 20 includes, for example, an aperture, an imaging optical system, a filter, and an image sensor. The image sensor is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor and includes an RGB color filter.

[0039] The image sensor may be replaced with another type of imaging device such as a CCD (Charge Coupled Device) image sensor. The image sensor may be configured to include a complementary color checkerboard filter.

[0040] The image sensor is driven under the control of the control unit 140 and receives visible light (natural light) on the light reception surface. The image sensor accumulates the optical image formed at each pixel on the light reception surface as electric charges according to the amount of light, and outputs the accumulated charges at a timing synchronized with, for example, ToF imaging. The control unit 140 outputs RGB image data based on each pixel data to the arithmetic device 3.

[0041] Note that the light transmitting / receiving device 2 may be configured as a device capable of capturing an omnidirectional panoramic image. The omnidirectional panoramic image is a panoramic image obtained by capturing an image within a solid angle of 4π radians and covering the entire celestial sphere. In this case, the light transmitting / receiving device 2 includes a pair of wide-angle ToF light projecting units 100 and a pair of ToF light receiving units 120 for forming hemisphere images, respectively. Similarly, a pair of RGB light receiving units 20 is also provided. The pair of RGB light receiving units 20 image objects (e.g., the target object OB) around the light transmitting / receiving device 2, respectively. Thereby, a pair of hemisphere images is obtained. The control unit 140 synthesizes the pair of hemisphere images to generate, for example, an omnidirectional panoramic image expressed in the Mercator projection method. The direct reflected light from the target object OB irradiated with the dot pattern light emitted from each of the pair of ToF light projecting units 100 is received by the corresponding ToF light receiving unit 120. The arithmetic unit 3 calculates the distance to each part (each irradiation position) of the target object OB based on the time difference between the emission timing and the reception timing of the laser light at each irradiation position. By combining the distance information corresponding to each of the pair of ToF light receiving units 120, distance information for the entire celestial sphere range corresponding to the omnidirectional panoramic image can be obtained. That is, a three-dimensional point cloud, which is a set of coordinate points in a three-dimensional space, can be acquired. Color information (e.g., the RGB values of each coordinate point) may be added to each coordinate point of the point cloud.

[0042] Note that in the configuration example shown in FIG. 1, the light transmitting / receiving unit 10 is included in the light transmitting / receiving device 2, but the light transmitting / receiving unit 10 may be configured as an independent single device as shown in FIG. 2.

[0043] The ToF light projecting unit 100 (in other words, the light projecting device) may be configured as an independent single device separate from the light transmitting / receiving unit 10.

[0044] The arithmetic unit 3 is, for example, a terminal device such as a PC (Personal Computer) or a server arranged on the cloud. The light transmitting / receiving device 2 and the arithmetic unit 3 can communicate with each other by wired communication or wireless communication through their respective transmitting / receiving units. Data may be transmitted (output) from the light transmitting / receiving device 2 to the arithmetic unit 3 via a network, or the transmitting / receiving unit may be configured by an interface circuit with a portable storage medium such as an SD card or a personal computer. In another embodiment, the arithmetic unit 31 may be included in the light transmitting / receiving device 2. In this case, the arithmetic unit 31 may be configured as, for example, a processor included in the control unit 140. The arithmetic unit 31 may be configured as a processor independent of the control unit 140.

[0045] As described above, there is freedom in the mode of each part of the distance measurement system 1, and various design changes are possible.

[0046] FIG. 4 is a diagram showing an example of the light reception timing of reflected light in the ToF sensor. Under the control of the control unit 140, dot pattern light is projected by the ToF light projecting unit 100 at a predetermined light emission timing. Further, direct reflected light from the object OB is received by the ToF light receiving unit 120 at a light reception timing having a phase difference (time difference) corresponding to the distance to the object OB from the predetermined light emission timing. Based on the phase difference (time difference) between the light emission timing by the ToF light projecting unit 100 and the light reception timing by the ToF light receiving unit 120, the distance to the object OB is detected.

[0047] <Conventional Technical Problem> As described above, as one method of photographing distance information from a photographing device to a subject, there is known a method called the ToF method, in which light rays having a wavelength different from that of visible light are irradiated, and based on the time difference between the light emission timing and the light reception timing of the laser light at each scanning position, the distance to each part (each scanning position) of the detection target object is calculated.

[0048] The light projection optical system that emits ranging light can obtain a high-brightness and high-density point cloud by spatially imaging a surface-emitting laser (VCSEL) with a large number of light-emitting points and projecting a large number of dot pattern lights. At this time, by freely designing the diffraction pattern of the diffraction optical element (DOE), it can be divided into a two-dimensional beam matrix (M×N) or a one-dimensional beam array (1×N or M×1), and the number of dot patterns projected into space can be increased or the density can be increased.

[0049] However, according to the intensive research of the present inventor, in the expansion of the dot pattern using the conventional DOE, when one or both (at least one) of the division numbers M and N by the diffraction optical element are designed to be even numbers, a small amount of zero-order light generated from the DOE is reflected into the background, deteriorating the ranging accuracy in ToF imaging. Therefore, there is a problem that the degree of freedom in design is reduced.

[0050] Also, for example, in the configuration of a head-mounted display (HMD), by increasing the density of the dot pattern imaged by the light source and the micro lens array (MLA), the uniformity of the light intensity incident on the viewer's pupil is increased, the flickering and rainbow phenomenon of the display image are reduced, or high-definition display is enabled. For this purpose, a beam splitter composed of a polarization grating, a diffraction grating, a hologram, etc. is used, and the virtual images of the dot patterns are irradiated twice with the first-order light and the -first-order light so that the diffraction patterns do not overlap.

[0051] However, when the beam is split into the first-order light and the -first-order light by the beam splitter, a small amount of zero-order light that travels straight without diffraction is generated. Depending on the design of the optical element, the zero-order light can be reduced, but it is difficult to completely remove it, and the zero-order light is irradiated as noise to the background between the dot patterns. In ToF imaging, the background plays an important role in detecting the scattered light after the dot pattern irradiates the object. Therefore, the presence of noise due to the zero-order light will affect the ranging accuracy.

[0052] <Technical idea of the present invention> In the present embodiment, the above problems are regarded as important technical issues, and an idea is conceived to optimize the arrangement of the light source 102 and the DOE 106 in order to reduce the risk that the zero-order light is reflected into the background and has an adverse effect. For example, as a design of the DOE, when the dot pattern is divided into an even number like a beam splitter so that the zero-order light is not used, even if the zero-order light is generated, by superimposing it on the dot patterns of the first-order light and the -first-order light, it is possible to prevent unnecessary pattern light from being projected onto the background. As a result, the background can be secured and the adverse effect on distance measurement can be removed. Also, while making the positions of the first-order light and the -first-order light, which are components other than the zero-order light, different (including the case where a part overlaps (not excluded)), at least a part of a plurality of light components of ±first-order or higher with different positions (for example, a part of the light component of the +first-order light and a part of the light component of the -first-order light) overlap at the position of the zero-order light, so that the projection range of the dot pattern can be expanded.

[0053] The ToF light projecting unit (light projecting device) 100 of the present embodiment includes a light source 102 that emits irradiation light and a DOE 106 that converts the irradiation light into pattern light. In this specification, the irradiation light may be simply read as light, laser light, or a beam. Also, in this specification, the pattern light may be read as dot pattern light, distance measurement light, or laser light.

[0054] The light source 102 and the DOE 106 are configured such that the pattern light includes a plurality of optical components of ± first-order light or higher at different positions, and among the regions where the zero-order light is located, a first optical component among the plurality of optical components overlaps with a first region and a second optical component among the plurality of optical components does not overlap (resulting in different positions from each other), and the first optical component does not overlap (resulting in different positions from each other) with a second region different from the first region among the regions where the zero-order light is located and the second optical component overlaps. More specifically, half of the region where the zero-order light is located can overlap with the + first-order light (the first optical component) as the "first region", and the remaining half of the region where the zero-order light is located can overlap with the - first-order light (the second optical component) as the "second region". Note that this includes (without exclusion) cases where there are other regions in the region where the zero-order light is located that overlap with the + first-order light (the first optical component) and the - first-order light (the second optical component).

[0055] The light source 102 and the DOE 106 are configured such that the pattern light includes a plurality of optical components of ± first-order light or higher at different positions, and in the region where the zero-order light is located, there is a region where the optical components of the ± first-order light among the plurality of optical components overlap and the optical components of an order higher than ± first-order among the plurality of optical components do not overlap (resulting in different positions from each other). Thereby, among the plurality of optical components of ± first-order light or higher, the optical components of the ± first-order light that overlap with the region where the zero-order light is located and the optical components of an order higher than ± first-order outside the region where the zero-order light is located can be clearly defined (distinguished). Note that this includes (without exclusion) cases where there are other regions in the region where the zero-order light is located that overlap with the optical components of an order higher than ± first-order among the plurality of optical components.

[0056] The light source 102 and the DOE 106 are configured such that for the patterned light, light components other than the zero-order light included in the patterned light (for example, a plurality of light components of ± first-order light or higher) are at different positions from each other (including the case where some overlap (not excluded)), and at least a part of a plurality of light components of ± first-order light or higher at different positions included in the patterned light overlaps at the position of the zero-order light, whereby the array property is optimally set. For example, the light source 102 and the DOE 106 are configured such that at least a part of the region where the zero-order light is located, a part of the + first-order light component and a part of the - first-order light component included in the plurality of light components of ± first-order light or higher at different positions described above overlap.

[0057] The basic structure of the array property of the light source 102 and the DOE 106 will be described with reference to FIGS. 5A and 5B. FIG. 5A is a diagram showing an example of the basic structure of the array property of the light source 102, and FIG. 5B is a diagram showing an example of the basic structure of the array property of the DOE 106. Further, FIGS. 5A and 5B are diagrams showing the relationship between the DOE matrix and the VCSEL matrix.

[0058] As shown in FIG. 5A, the light source 102 has a matrix structure with light source periodicity (regularity). When one set of light source periodicity is taken as a basic period, the basic period is defined by a rows (i.e., the number of divisions in the vertical direction is a) and b columns (i.e., the number of divisions in the horizontal direction is b). That is, the light source 102 emits a plurality of irradiation lights with the number of divisions in the vertical direction being a and the number of divisions in the horizontal direction being b as irradiation light. Here, both the number of divisions a in the vertical direction and the number of divisions b in the horizontal direction are positive integers, including the case of 1 in addition to the case of 2 or more. When the number of divisions a in the vertical direction is 1, it means not to divide in the vertical direction, and when the number of divisions b in the horizontal direction is 1, it means not to divide in the horizontal direction. Therefore, for example, the light source 102 may be an array-shaped light source of 1×b (a positive integer of 2 or more) or a (a positive integer of 2 or more)×1. FIG. 5A exemplarily depicts the case where the light source 102 emits a plurality of irradiation lights with the number of divisions in the vertical direction being 3 and the number of divisions in the horizontal direction being 5 as irradiation light. In FIG. 5A, although two types of division patterns with different phases are alternately positioned in both the vertical and horizontal directions, the number of divisions in the vertical direction is defined as 3 and the number of divisions in the horizontal direction is defined as 5 in a form that includes these two types of division patterns.

[0059] As shown in FIG. 5B, the DOE 106 divides each of the plurality of irradiation lights emitted from the light source 102 into a matrix shape centered on the optical axis by M rows (i.e., the number of divisions in the vertical direction is M) and N columns (i.e., the number of divisions in the horizontal direction is N) by diffraction action. That is, the DOE 106 converts the plurality of irradiation lights from the light source 102 into pattern light by dividing each of them with the number of divisions in the vertical direction being M and the number of divisions in the horizontal direction being N. Here, both the number of divisions M in the vertical direction and the number of divisions N in the horizontal direction are positive integers, including the case of 1 in addition to the case of 2 or more. When the number of divisions M in the vertical direction is 1, it means not to divide in the vertical direction, and when the number of divisions N in the horizontal direction is 1, it means not to divide in the horizontal direction. Therefore, for example, the DOE 106 may perform an array-shaped division process of 1×N (a positive integer of 2 or more) or M (a positive integer of 2 or more)×1. FIG. 5B exemplarily depicts the case where the DOE 106 converts the plurality of irradiation lights from the light source 102 into pattern light by dividing each of them with the number of divisions in the vertical direction being 5 and the number of divisions in the horizontal direction being 5.

[0060] In this way, the division pattern of the two-dimensional DOE 106 forms a matrix that spreads vertically and horizontally, and the division pattern can be expressed as M×N. Also, regarding the VCSEL light source 102, the number of regularities can be expressed as a×b depending on how many basic periods, which are the smallest units of periodicity, exist in the vertical and horizontal directions. The patterned light formed by the VCSEL light source 102 and the two-dimensional DOE 106 is an aggregate of a plurality of dot-patterned lights divided into M×a in the vertical direction and N×b in the horizontal direction.

[0061] In this embodiment, when the DOE 106 converts (divides) each of the plurality of irradiation lights emitted from the light source 102 into patterned light, that is, when projecting the patterned light onto the space without gaps, in order to reduce the risk that the zero-order light is reflected into the background and causes an adverse effect and to improve the measurement accuracy (distance measurement accuracy), the array properties of the light source 102 and the DOE 106 are classified and defined as follows.

[0062] <<Type 1>> Both the number of divisions M in the vertical direction and the number of divisions N in the horizontal direction by the DOE 106 are odd. In this case, it does not matter whether the number of divisions a in the vertical direction and the number of divisions b in the horizontal direction by the light source 102 are odd or even. That is, if "-" is defined as either odd or even (either is fine), when (M, N) = (odd, odd), (a, b) = (-, -) holds.

[0063] <<Type 2>> The number of divisions M in the vertical direction by the DOE 106 is odd, the number of divisions N in the horizontal direction by the DOE 106 is even, and the number of divisions b in the horizontal direction by the light source 102 is even. In this case, it does not matter whether the number of divisions a in the vertical direction by the light source 102 is odd or even. That is, when (M, N) = (odd, even), (a, b) = (-, even) holds.

[0064] <<Type 3>> The number of vertical divisions M by DOE106 is even, the number of horizontal divisions N by DOE106 is odd, and the number of vertical divisions a by the light source 102 is even. In this case, it doesn't matter whether the number of horizontal divisions b by the light source 102 is odd or even. That is, when (M, N) = (even, odd), (a, b) = (even, -) holds.

[0065] <<Type 4>> The number of vertical divisions M and the number of horizontal divisions N by DOE106 are both even, and the number of vertical divisions a and the number of horizontal divisions b by the light source 102 are both odd. That is, when (M, N) = (even, even), (a, b) = (odd, odd) holds.

[0066] <<Type 5>> The number of vertical divisions M and the number of horizontal divisions N by DOE106 are both even, and the number of vertical divisions a and the number of horizontal divisions b by the light source 102 are both even. That is, when (M, N) = (even, even), (a, b) = (even, even) holds.

[0067] Figure 6 is a table defining the combinations of the number of vertical divisions M and the number of horizontal divisions N by DOE106, and the number of vertical divisions a and the number of horizontal divisions b by the light source 102. As shown in Figure 6, it is defined that when (M, N) = (odd, odd), (a, b) = (-, -) holds, when (M, N) = (odd, even), (a, b) = (-, even) holds, when (M, N) = (even, odd), (a, b) = (even, -) holds, and when (M, N) = (even, even), (a, b) = (odd, odd) or (a, b) = (even, even) holds.

[0068] By setting the number of vertical divisions M and the number of horizontal divisions N by the DOE 106, and the number of vertical divisions a and the number of horizontal divisions b by the light source 102 so as to satisfy the combinations defined in the table of FIG. 6, light components other than the zero-order light included in the pattern light (for example, light components of ± first order or higher) are at different positions from each other (including the case where some overlap (not excluded)), and at the position of the zero-order light, at least a part of a plurality of light components of ± first order or higher at different positions included in the pattern light (for example, a part of the light component of + first order and a part of the light component of - first order) overlap. Therefore, it is possible to reduce the risk that the zero-order light is reflected into the background and has an adverse effect, and improve the measurement accuracy (distance measurement accuracy).

[0069] Here, setting the number of vertical divisions a and the number of horizontal divisions b by the light source 102 may mean creating (manufacturing) the entire irradiation region (maximum irradiation region) of the light source 102 so as to satisfy the combinations defined in the table of FIG. 6 on the premise of using the entire irradiation region of the light source 102. Alternatively, it may mean selecting and determining a part of the irradiation region of the light source 102 so as to satisfy the combinations defined in the table of FIG. 6 among the entire irradiation region (maximum irradiation region) of the light source 102 on the premise of using a part of the irradiation region of the light source 102.

[0070] Also, setting the number of vertical divisions M and the number of horizontal divisions N by the DOE 106 may mean creating (manufacturing) the entire effective optical region of the DOE 106 so as to satisfy the combinations defined in the table of FIG. 6 on the premise of using the entire effective optical region of the DOE 106. Alternatively, it may mean selecting and determining a part of the effective optical region of the DOE 106 so as to satisfy the combinations defined in the table of FIG. 6 among the entire effective optical region of the DOE 106 on the premise of using a part of the effective optical region of the DOE 106.

[0071] When projecting the pattern of a light source having a basic period into space, in order to project the pattern light with a higher angular resolution and without gaps using a DOE, each divided light source image may be arranged without gaps.

[0072] When designing the number of DOE divisions to be even, only odd diffraction orders such as ±1st order, ±3rd order, etc. are used. At this time, the 0th order light that is not originally used is projected between the -1st order light and the 1st order light. This means that the 0th order light, the 1st order light, and the 0th order light and the -1st order light are projected in a state where they are shifted by half of the light source width, and the pattern of the 0th order light is projected onto the background.

[0073] To solve this problem, in this embodiment, the shifted and projected 0th order light is superimposed on the original pattern, that is, a part of the pattern other than the 0th order light.

[0074] First, consider the case where the DOE division number is an array of (M, N) = (1, even) for simplicity. At this time, if b in the light source matrix (a, b) is even, the shift amount = half of the light source width, and b / 2 = an integer. At this time, the phase difference disappears, and the 0th order light is projected by being superimposed on the -1st order light and the 1st order light.

[0075] Subsequently, consider the case where the DOE division number is a matrix of (M, N). When (M, N) = (even, even), as described above, to eliminate the phase difference, it is only necessary to absorb the phase difference in each direction. Therefore, the light source matrix may be (a, b) = (even, even). Furthermore, when the light source matrix is (a, b) = (odd, odd), the 0th order light is also superimposed, so the condition is satisfied.

[0076] This is because a phase difference occurs in the shift amount of the 0th order light and the ±1st order light, where half of the light source width = a / 2 = an integer + 0.5 and b / 2 = an integer + 0.5. However, the phase of the light source matrix with a two-dimensional structure shifts by half in both the vertical and horizontal directions, so that the total phase difference becomes an integer, and as a result, the 0th order light and the ±1st order light are superimposed.

[0077] Subsequently, when (M, N) = (odd, even), based on the same principle as when (M, N) = (1, even), if b in the light source matrix (a, b) is even, the shift amount = half of the light source width becomes b / 2 = integer, resulting in no phase difference, and the 0th-order light is superimposed on the -1st-order and 1st-order lights and projected. Conversely, when (M, N) = (even, odd), if a in the light source matrix (a, b) is even, the shift amount = half of the light source width becomes a / 2 = integer, resulting in no phase difference, and the 0th-order light is superimposed on the -1st-order and 1st-order lights and projected.

[0078] Finally, when (M, N) = (odd, odd), since it is assumed that the 0th-order light is used for each of the two axes, regardless of the structure of the light source matrix (regardless of whether a and b are odd or even), the 0th-order light does not pose a problem.

[0079] <Specific Examples and Comparative Examples> Figure 7 is a diagram showing an example of light beam splitting by diffraction of a DOE when the number of DOE divisions is odd. When diffraction by the DOE occurs, the light beam bends according to the diffraction formula dsinθ = nλ (d: diffraction grating constant, θ: diffraction angle, n: diffraction order, λ: wavelength) for each diffraction order. When the number of divisions is odd, the configuration uses the 0th-order light and the ±1st-order, ±2nd-order lights, ···. Depending on the design of the DOE, as shown in Figure 7, the configuration can be such that diffraction orders of ±3 and higher are not used. In this case, the light beam is split into five.

[0080] Figure 8 is a diagram showing an example of light beam splitting by diffraction of a DOE when the number of DOE divisions is even. When it is desired to make the number of divisions of the DOE even, this can be achieved by configuring not to use the even-order light beams including the 0th-order light. By configuring to emit only the ±1st-order and ±3rd-order light beams without using the 0th-order light as shown in Figure 8, the number of divisions can be made four. However, it is difficult to completely block the 0th-order light at this time, and it is reasonable to consider that a small amount of 0th-order light is generated. The 0th-order light is related to the manufacturing tolerance that causes a very small part of the incident light not to be affected by the diffraction pattern, and it is not realistic to completely block it.

[0081] FIG. 9 is a diagram showing an example of a splitting pattern of a single laser beam when using a DOE having a two-dimensional diffraction grating constant. The single laser beam is emitted, for example, by an LD. By designing the DOE to have a diffraction grating constant in the X-axis direction and the Y-axis direction, a two-dimensional DOE can be obtained. By using such a DOE, it becomes possible to split the light beam emitted from the LD in two-dimensional directions and irradiate more points in space uniformly. Such an optical element is useful in 3D sensing.

[0082] FIG. 10 is a diagram showing an example of a splitting pattern of a plurality of laser beams when using a DOE having a two-dimensional diffraction grating constant. The plurality of laser beams are emitted, for example, by a VCSEL. By using a surface-emitting laser VCSEL having a light source with a large number of light spots as the light source, the light source pattern can be irradiated in space. At this time, when using a two-dimensional DOE, it becomes possible to replicate the projection pattern uniformly in space. Further, by adjusting the magnification and the diffraction grating constant so that the light source images of each order are arranged without gaps, the projection pattern can be replicated without gaps. As a result, it becomes possible to perform 3D sensing over a wide range with high precision. Also, in the case of Indirect ToF, generally called the iToF method of imaging, when the laser irradiated toward the object is reflected and returned to the sensor, the distance is calculated using the phase difference between the irradiated light and the reflected light according to the distance of the object. At this time, by performing correction using the background where the pattern light does not hit, the ranging accuracy can be further improved. Therefore, it is important that no noise enters the background.

[0083] FIG. 11 is a diagram for explaining the relationship between the projection pattern and the zero-order light when both the number of vertical divisions M and the number of horizontal divisions N by the DOE 106 are odd, that is, in the case of (M, N) = (odd, odd). Here, M = 3 and N = 3 are illustrated. FIG. 11 corresponds to the <<Type 1>> of the present embodiment described above. When the division matrix of the DOE 106 is odd for both axes, a design using zero-order light as shown in FIG. 7 is adopted for both axes. In this case, since the zero-order light is not projected onto the background, it does not pose a problem. Also, here, the case where both the number of vertical divisions a and the number of horizontal divisions b by the light source 102 are 4 is illustrated, but it does not matter whether the division numbers a and b are odd or even.

[0084] FIG. 12 is a diagram for explaining the relationship between the projection pattern and the zero-order light when the number of vertical divisions M by the DOE 106 is odd and the number of horizontal divisions N is even, that is, in the case of (M, N) = (odd, even). Here, M = 3 and N = 2 are illustrated. Also, the division numbers by the light source 102 are illustrated as a = 4 and b = 3. FIG. 12 is a comparative example outside the scope of the present embodiment, showing a case where the adverse effect of the zero-order light cannot be reduced. In FIG. 12, M is odd, 3, and a design using zero-order light is adopted, so the zero-order light is not projected onto the background regardless of the pattern in the coaxial direction of the light source. Therefore, a can be either even or odd. However, N is even, 2, and although a design not using zero-order light is adopted, b is odd and does not fall under the <<Type 2>> of the present embodiment described above. For this reason, when the light source pattern is projected into the space without gaps, the +1st-order light and the -1st-order light are projected adjacent to each other without gaps, so the zero-order light is projected in the middle. That is, the zero-order light is shifted by half of the light source width and projected together with the +1st-order light and the -1st-order light. That is, the shift amount is b / 2, and since b is odd (3), it is projected in a state where the phase is shifted by half. It can actually be confirmed in FIG. 12 that the zero-order light is projected in a state where the phase is shifted by half (surrounded by a dashed-dotted line frame). In this case, the zero-order light pattern is projected onto the background, which is disadvantageous in the calibration using the background in ToF imaging.

[0085] FIG. 13 is a diagram for explaining the relationship between the projection pattern and the zero-order light when the number of vertical divisions M by DOE106 is odd and the number of horizontal divisions N is even, that is, in the case of (M, N) = (odd, even). Here, M = 3 and N = 2 are exemplified. Also, the number of divisions by the light source 102 is exemplified as a = 4 and b = 4. FIG. 13 corresponds to the <<Type 2>> of the present embodiment described above. In FIG. 13, N is 2 which is even, and it is a design that does not use the zero-order light, but b is 4 which is even, and it conforms to the <<Type 2>> of the present embodiment described above. For this reason, the shift amount b / 2 becomes an integer multiple of the phase, and the zero-order light and the first-order light and the zero-order light and the -first-order light are superimposed and projected (enclosed by a two-dot chain line frame). Thus, when the division matrix of the DOE is (M, N) = (odd, even), by setting the light source matrix to (a, b) = (-, even), it is possible to avoid the zero-order light being projected onto the background.

[0086] By the same logic as in FIG. 13, when the number of vertical divisions M by DOE106 is even and the number of horizontal divisions N is odd, that is, in the case of (M, N) = (even, odd), by setting the light source matrix to (a, b) = (even, -), it is possible to avoid the zero-order light being projected onto the background. This corresponds to the <<Type 3>> of the present embodiment described above.

[0087] FIG. 14 is a diagram for explaining the relationship between the projection pattern and the zero-order light when the number of vertical divisions M by the DOE 106 is even and the number of horizontal divisions N is even, that is, in the case of (M, N) = (even, even). Here, M = 2 and N = 2 are illustrated. Also, the number of divisions by the light source 102 is illustrated as a = 4 and b = 3. FIG. 14 is a comparative example outside the scope of the present embodiment, showing a case where the adverse effect of the zero-order light cannot be reduced. In FIG. 14, since the number of divisions M and N of the DOE 106 are both even numbers 2 and it is a design that does not use the zero-order light, the zero-order light that is not originally used is projected between the first-order light and the -first-order light. First, regarding the vertical projection, the zero-order light is projected with a shift of half the width of the light source. Since a is an even number (4), the shift amount a / ₂ is an integer, and there is no phase shift at this stage. However, since b is an odd number (3), regarding the horizontal projection, the shift amount b / ₂ of the zero-order light is not an integer and is projected in a state of half-phase shift (enclosed by a dashed-dotted line frame). As a result, in the space, the zero-order light, the first-order light, and the zero-order light and the -first-order light are projected in a state of half-phase shift, so that the zero-order light is projected onto the background.

[0088] FIG. 15 is a diagram for explaining the relationship between the projection pattern and the zero-order light when the number of vertical divisions M by the DOE 106 is even and the number of horizontal divisions N is even, that is, in the case of (M, N) = (even, even). Here, M = 2 and N = 2 are illustrated. Also, the number of divisions by the light source 102 is illustrated as a = 4 and b = 4. FIG. 15 corresponds to the <<Type 5>> of the present embodiment described above. In FIG. 15, M and N are even numbers 2 and it is a design that does not use the zero-order light, but a and b are even numbers 4, which conforms to the <<Type 5>> of the present embodiment described above. For this reason, the shift amounts a / ₂ and b / ₂ of the zero-order light are integers on both axes, no phase shift occurs, and the zero-order light, the first-order light, and the zero-order light and the -first-order light are projected in an overlapping manner (enclosed by a two-dot chain line frame). Thus, when the division matrix of the DOE is (M, N) = (even, even), by setting the light source matrix to (a, b) = (even, even), it is possible to avoid the zero-order light being projected onto the background.

[0089] FIG. 16 is a diagram for explaining the relationship between the projection pattern and the zero-order light when the number of vertical divisions M by DOE106 is even and the number of horizontal divisions N is even, that is, in the case of (M, N) = (even, even). Here, M = 2 and N = 2 are illustrated. Also, the number of divisions by the light source 102 is illustrated as a = 3 and b = 3. FIG. 16 corresponds to <<Type 4>> of the present embodiment described above. In FIG. 16, M and N are 2 which are even, and it is a design that does not use zero-order light, but a and b are 3 which are odd, and it applies to <<Type 4>> of the present embodiment described above. In this case, the shift amounts a / 2 and b / 2 of the zero-order light are projected in a state where they are half-phase shifted in both axes, but since the sum of these shift amounts is an integer, the zero-order light and the first-order light and the zero-order light and the -first-order light are projected in an overlapping manner (enclosed by a two-dot chain line frame). Thus, when the division matrix of the DOE is (M, N) = (even, even), by setting the light source matrix to (a, b) = (odd, odd), it is possible to avoid the zero-order light being projected onto the background.

[0090] As described above, the light projection device of the present embodiment includes a light source that emits irradiation light, and a diffractive optical element that converts the irradiation light into pattern light. The light source and the diffractive optical element are configured such that the pattern light includes a plurality of light components of ± first-order light or higher at different positions, and a first light component among the plurality of light components overlaps in a first region of the region where the zero-order light is located and a second light component among the plurality of light components does not overlap (becomes different positions), and the first light component does not overlap (becomes different positions) in a second region different from the first region of the region where the zero-order light is located and the second light component overlaps. Additionally / alternatively, the light source and the diffractive optical element are configured such that the pattern light includes a plurality of light components of ± first-order light or higher at different positions, and in the region where the zero-order light is located, there is a region where the light components of ± first-order light among the plurality of light components overlap and light components of a higher order than ± first-order among the plurality of light components do not overlap (become different positions). Thereby, it is possible to reduce the risk that the zero-order light is reflected into the background and has an adverse effect. As a result, it is possible to obtain pattern light with a higher contrast.

[0091] For example, focusing on the technical problem of reducing the adverse effects of the zero-order light when the number of divisions of a DOE (Diffractive Optical Element) as a diffractive optical element is even both vertically and horizontally, and as a configuration for solving this technical problem, in addition to the case where the periodicity required for the light source is even both vertically and horizontally, adopting the case where it can be odd both vertically and horizontally is one of the major features of this embodiment. As a prior art document, there is something known to suppress the zero-order light by the design of the DOE itself, but reducing the adverse effects of the zero-order light by controlling the periodicity of the light source according to the configuration of the DOE (so to speak, the combination technique of the DOE and the light source configuration) as in this embodiment is one of the major features of this embodiment.

[0092] To know the number of divisions of the DOE, it is necessary to obtain the overall illumination angle of view and the angle of view per diffraction order. However, when the pattern light is irradiated without gaps, it is difficult to obtain the angle of view per diffraction order (because the boundary line is unknown). However, if the DOE can be removed from the optical system, only the zero-order light can be obtained, and the angle of view per diffraction order can be obtained. By dividing the overall angle of view by the angle of view per diffraction order, the diffraction order can be known. Also, since the light source pattern can be known when the DOE is removed, it is possible to grasp that the configuration requirements of the light projection device of this embodiment are satisfied in combination with the diffraction order.

[0093] The light projection device of this embodiment can be applied to a pattern projection device for use in a device that performs scanning by pattern projection, in addition to a distance measurement device using the ToF (Time of Flight) method.

[0094] The light projection device of this embodiment can also capture its constituent elements as follows. More specifically, the light projection device includes a plurality of light-emitting points that each emit a light beam, and has a light source that generates a plurality of light beams by the plurality of light-emitting points. Further, the light projection device has a diffractive optical element that, by diffraction, divides each of the plurality of light beams into a plurality in at least one axial direction and emits patterned light. The light-emitting points of the light source include a two-dimensional array in which the basic periods are regularly arranged. And the diffractive optical element emits the patterned light of the first or higher order to different positions from each other, and the irradiation pattern of the zero order to a position overlapping a part of the patterned light of the first or higher order, respectively.

[0095] The light projection device of this embodiment can also capture its constituent elements as follows. More specifically, the light projection device is a system that irradiates the surroundings with patterned light or an image, and has a light source that generates a plurality of beams, a lens group that forms an image of the beams, and a diffractive optical element that divides the beams into two or more in at least one axial direction by diffraction and emits them. The light spots of the light source consist of a two-dimensional array in which the basic periods are regularly arranged, and the light from the light source is replicated by the diffractive optical element to irradiate the patterned light. And the irradiation patterns of orders other than the zero order are irradiated without overlapping each other (to different positions from each other), and the irradiation pattern of the zero order is irradiated so as to overlap a part of the irradiation patterns other than the zero order.

[0096] The number of divisions of the diffractive optical element is even or odd for each of the two axes, and the two-dimensional array of the light source may have an even number, an odd number, or an even number, an even number of basic periods with respect to the two axes in the same direction as the diffractive optical element. Conversely, the number of divisions of the diffractive optical element is odd or even for each of the two axes, and the two-dimensional array of the light source may have an odd number, an even number, or an even number, an even number of basic periods with respect to the two axes in the same direction as the diffractive optical element. That is, when (M, N) = (even, odd), (a, b) may be (even, -), and when (M, N) = (odd, even), (a, b) may be (-, even).

[0097] The number of divisions of the diffractive optical element is even for each of the two axes, and the two-dimensional array of light sources may have an even number, an even number or an odd number, an odd number of basic periods with respect to the two axes in the same direction as the diffractive optical element. That is, when (M, N) = (even, even), (a, b) may be (even, even) or (odd, odd).

[0098] This embodiment may be directed to a system that projects patterned light. This system may include a light source that generates patterned light in which a plurality of dot lights are arranged, and a plurality of light receiving elements corresponding to a plurality of pixels. Further, the system may include a light receiving sensor that receives, for each pixel, light reflected by an object from the light irradiated by the irradiation unit, and a distance measuring unit that measures the distance to the reflecting object by measuring the time from light emission to light reception. Further, the system may be a distance measuring device that performs correction using the background.

[0099] <Application Example> FIG. 17 is a diagram showing another arrangement example of each optical element of the light projecting and receiving device 2. The light projecting and receiving device 2 shown in FIG. 17 includes a ToF light projecting unit 100 and a ToF light receiving unit 120 on one side of the light projecting and receiving device 2. Then, an instrument 200 such as a tripod that functions as a support unit and a rotating means 220 that is a rotating table rotated by, for example, an electric motor or the like may be provided above the instrument 200. In this case, the light projecting and receiving device 2 rotates the device itself by the rotating means 220 to change the imaging direction and acquires a distance image of the entire circumference.

[0100] The rotation control unit of the light projecting and receiving device 2 has a predetermined rotation pattern when controlling the rotation of the rotating means 220. For example, the rotation control unit of the light projecting and receiving device 2 controls in a rotation pattern of (1) rotating by a predetermined distance, (2) stopping the rotation and performing light projecting and receiving, (3) rotating by a predetermined distance, and (4) stopping the rotation and performing light projecting and receiving imaging so as to perform light projecting and receiving around the entire circumference of the device in a plurality of times.

[0101] FIG. 18 is a block diagram showing the configuration of the three-dimensional shape generation system 4.

[0102] As shown in FIG. 18, the three-dimensional shape generation system 4 includes a light projecting and receiving device 2 and an arithmetic unit 3A. The three-dimensional shape generation system 4 is obtained by adding a generation processing unit 32 to the arithmetic unit 3A of the distance measurement system 1 in FIG. 1.

[0103] In addition to the arithmetic unit 31, the arithmetic unit 3A includes a generation processing unit 32 that generates three-dimensional shape information based on the acquired three-dimensional point cloud. The three-dimensional shape information is information indicating the three-dimensional shape of an object that can be handled by a computer or the like. The information indicating the three-dimensional shape is information that can geometrically specify the three-dimensional shape. For example, when representing a sphere, the coordinates of the center and the radius, and when representing a polyhedron (polygon), the coordinate points of each vertex of the polyhedron respectively correspond to the information indicating the three-dimensional shape. In the three-dimensional shape information, in addition to the information indicating the three-dimensional shape of the object, information related to the color and material of the object may be added.

[0104] In the storage unit 33 of the arithmetic unit 3A, a setting information management DB 301, a storage processing management DB 302, a point cloud management DB 303, and a three-dimensional shape management DB 304 are constructed.

[0105] The setting information management DB 301 stores and manages various information. The storage processing management DB 302 stores and manages various processing programs for generating a three-dimensional shape. The point cloud management DB 303 stores and manages the three-dimensional point cloud information acquired by the distance measurement system. The three-dimensional shape management DB 304 stores and manages the three-dimensional shape information.

[0106] The setting information management DB 301 is constituted by a setting information management table. The setting information management table is a table for managing the three-dimensional point cloud data for generating a three-dimensional shape, and the execution order and processing mode of the generation process (three-dimensional shape generation process) for generating a three-dimensional shape. In this setting information management table, the file name of the three-dimensional point cloud data, and the execution order and processing mode of the three-dimensional shape generation process are associated and managed.

[0107] The three-dimensional shape generation process includes, for example, registration processing, noise removal processing, segmentation processing, and modeling processing, etc.

[0108] The registration processing is a process of converting a plurality of three-dimensional point clouds into one integrated three-dimensional point cloud. The noise removal processing is a process of removing unnecessary point clouds from the three-dimensional point cloud.

[0109] The segmentation processing is a process of labeling specific point clouds in the three-dimensional point cloud so that they can be distinguished from other point clouds. By labeling each of the plurality of specific point clouds differently, each of the plurality of specific point clouds can be made mutually distinguishable. Further, the segmentation processing may be executed in combination with clustering processing for grouping point clouds with close distances among the labeled point clouds.

[0110] The modeling processing is a process of collating specific point clouds in the three-dimensional point cloud with a three-dimensional model shape and replacing the specific point clouds with the model shape. Note that the three-dimensional model shape is a model such as a prototype or a template used to generate three-dimensional shape information from the three-dimensional point cloud.

[0111] The processing mode includes a manual process that executes part or all of the three-dimensional shape information generation process based on a user's operation input without executing a storage process, an automatic process that executes part or all of the three-dimensional shape information generation process based on a storage process stored in advance without being based on an operation input, and a mixed process that mixes the manual process and the automatic process to execute part or all of the three-dimensional shape information generation process.

[0112] The generation processing unit 32 receives an input operation from the user. The input operation includes, for example, an operation of setting point cloud setting information indicating the three-dimensional point cloud for which processing is to be performed, and an operation of setting processing setting information indicating the execution order and processing mode of each of the processes in the three-dimensional shape information generation process.

[0113] Next, the generation processing unit 32 executes a three-dimensional shape information generation process set by an input operation. Specifically, the generation processing unit 32 reads out three-dimensional point cloud data associated with the point cloud setting information by searching the point cloud management DB303 using the point cloud setting information as a search key. Also, the generation processing unit 32 reads out a processing program associated with the processing setting information by searching the storage processing management DB302 using the processing mode of the generation process in the processing setting information as a search key. Next, the generation processing unit 32 generates three-dimensional shape information based on the three-dimensional point cloud data read from the storage unit 33, the processing program, and the execution order and processing mode of the generation process in the processing setting information.

[0114] When the processing mode includes manual processing and mixed processing, the generation processing unit 32 generates an operation screen for receiving an operation input for generating three-dimensional shape information, displays it on a display unit (such as a display), and receives a predetermined input operation of the user with respect to the displayed operation screen. This input operation includes an operation input for executing the process in which the manual processing mode is selected in the three-dimensional shape information generation process. The generation processing unit 32 generates three-dimensional shape information based on the operation input information by the operation input.

[0115] The generation processing unit 32 converts the generated three-dimensional shape information into a CAD format or the like, and stores the converted three-dimensional shape information in the three-dimensional shape management DB304 or an external recording medium or the like.

[0116] According to the three-dimensional shape generation system according to the present embodiment, three-dimensional shape information can be generated based on the point cloud acquired by the distance measurement system. Thereby, for example, three-dimensional modeling of existing buildings in fields such as architecture, construction, and civil engineering becomes easy, and it can be used for BIM (Building Information Modeling) / CIM (Construction Information Modeling).

[0117] Note that the generation processing unit 32 is not limited to being provided in the arithmetic device 3A including the arithmetic unit 31A. For example, it may be provided in another terminal device connected to the arithmetic device 3A by wired communication or wireless communication, or in a server arranged on the cloud. Further, the three-dimensional shape generation processing may be executed by a plurality of devices (for example, the arithmetic device 3A and a server connected to the arithmetic device 3A).

[0118] Application examples of using the distance measurement system 1 in various detection systems will be described with reference to FIGS. 19 to 20. The detection systems in these application examples have respective function blocks described later in addition to the distance measurement system 1. In FIGS. 19 to 20, function blocks such as a determination unit included in the detection system are described outside the detection system for convenience of drawing. Each of the various detection systems shown in FIGS. 19 to 20 has a control unit that receives information from the distance measurement system 1 and controls the various detection systems based on the information from the distance measurement system 1.

[0119] FIG. 19 is an example of a shape measurement system as a detection system, and shows an application example in which the distance measurement system 1 is used for user authentication of an electronic device. The portable information terminal 60X, which is an electronic device, has a user authentication function. The authentication function may be realized by dedicated hardware, or may be realized by a CPU (Central Processing Unit) that controls the portable information terminal 60X executing a program such as a ROM (Read Only Memory).

[0120] When authenticating a user, light is projected from the light source device of the distance measurement system 1 mounted on the mobile information terminal 60X toward the user 61X who uses the mobile information terminal 60X. The light reflected by the user 61X and its surroundings is received by the light receiving element of the distance measurement system 1, and the image processing unit 62X generates image data (performs imaging). The determination unit 63X determines the degree of coincidence between the image information of the user 61X captured by the distance measurement system 1 and the pre-registered user information, and determines whether the user is a registered user. Specifically, the shapes (contours and irregularities) of the face, ears, head, etc. of the user 61X can be measured and used as user information.

[0121] In the application example of FIG. 19, regarding the detection of the user 61X by the distance measurement system 1, the same effect (improvement in detection accuracy) as that of the distance measurement system 1 can be obtained. In particular, since light is projected from the light projection device at a wide angle to detect the information of the user 61X in a wide range, the amount of information for recognizing the user is increased compared to the case where the detection range is narrow, and the recognition accuracy can be improved.

[0122] FIG. 19 shows an example in which the distance measurement system 1 is mounted on the mobile information terminal 60X. However, it is also possible to use the user authentication using the distance measurement system 1 for stationary personal computers, OA devices such as printers, building security systems, etc. Further, in terms of function, it is not limited to the personal authentication function, but can also be used for scanning three-dimensional shapes such as faces. Also in this case, by mounting the distance measurement system 1 capable of projecting light at a wide angle, high-precision scanning can be realized.

[0123] FIG. 20 shows an application example in which the distance measurement system 1 is used in an autonomous driving system in a moving body which is an example of a detection system. In the application example of FIG. 20, the distance measurement system 1 is used for sensing an object outside the moving body 70X. The moving body 70X is an autonomous driving type moving body that can automatically travel while recognizing the external situation.

[0124] The mobile body 70X is equipped with the distance measurement system 1, and the distance measurement system 1 irradiates light toward the traveling direction of the mobile body 70X and its peripheral area. In the room 71X which is the movement area of the mobile body 70X, a desk 72X is installed in the traveling direction of the mobile body 70X. Among the light projected from the light source device of the distance measurement system 1 mounted on the mobile body 70X, the light reflected by the desk 72X and its surroundings is received by the light receiving element of the distance measurement system 1, and the photoelectrically converted electrical signal is sent to the signal processing unit 73X. Based on the electrical signal sent from the light receiving element and the like, the signal processing unit 73X calculates information regarding the layout of the room 71X, such as the distance to the desk 72X, the position of the desk 72X, and the peripheral situation other than the desk 72X. Based on this calculated information, the determination unit 74X determines the movement path and movement speed of the mobile body 70X, and based on the determination result of the determination unit 74X, the driving control unit 75X controls the traveling of the mobile body 70X (such as the operation of the motor which is the drive source).

[0125] In the application example of FIG. 20, regarding the layout detection of the room 71X by the distance measurement system 1, the same effect (improvement of detection accuracy) as that of the distance measurement system 1 can be obtained. In particular, since light is projected from the light projecting device at a wide angle to detect information on the room 71X in a wide range, a larger amount of information can be obtained compared to the case where the detection range is narrow, and the accuracy improvement of the autonomous driving of the mobile body 70X can be realized.

[0126] FIG. 20 shows an example in which the distance measurement system 1 is mounted on the autonomous driving mobile body 70X traveling in the room 71X, but it can also be applied to an autonomous driving vehicle (so-called self-driving vehicle) traveling outdoors. Further, it can also be applied to a driving support system in a mobile body such as an automobile driven by a driver, rather than an autonomous driving type. In this case, the distance measurement system 1 can be used to detect the peripheral situation of the mobile body, and the driving of the driver can be supported according to the detected peripheral situation.

[0127] In addition to the above, the distance measurement system 1 may be applied to an article inspection system in a factory or the like. Specifically, based on the information obtained by the distance measurement system 1, the determination unit of the article inspection system determines the state of each article.

[0128] Further, the distance measurement system 1 may be applied to the operation control of a movable device. A multi-joint arm as a movable device has a plurality of arms connected by bendable joints and is provided with a hand portion at its tip. The multi-joint arm is used, for example, in an assembly line of a factory, etc., and grips an object with the hand portion during inspection, conveyance, and assembly of the object. The distance measurement system 1 detects the object and its surrounding area, and a determination unit of the movable device determines various pieces of information regarding the object, such as the distance to the object, the shape of the object, the position of the object, and the positional relationship between a plurality of objects when they exist, based on the information acquired by the distance measurement system 1. Then, based on the determination result of the determination unit, a drive control unit controls the multi-joint arm operation.

[0129] Further, the distance measurement system 1 may be applied to a driving support system in a moving body such as an automobile. The distance measurement system 1 mounted inside the automobile detects the driver driving the automobile and the surrounding area thereof, and a determination unit of the driving support system determines information such as the face (expression) and posture of the driver based on the information acquired by the distance measurement system 1. Then, based on the determination result of the determination unit, the control unit performs appropriate driving support according to the situation of the driver.

[0130] The shape measurement system, the moving body, the article inspection system, the movable device, and the driving support system are all examples of the detection system. In the distance measurement system 1 of the present embodiment, since it is possible to reduce the risk that the 0th-order light is reflected into the background and has an adverse effect, and to achieve high contrast, the detection system to which the distance measurement system 1 is applied can perform detection over a wide range with high accuracy.

[0131] Hereinafter, the invention described in the claims of the present application at the time of initial filing is appended. [Appendix 1] A light source that emits irradiation light, A diffractive optical element that converts the irradiation light into patterned light, and the light source and the diffractive optical element are The pattern light includes a plurality of light components of ± first-order light or higher at different positions from each other, and a first light component among the plurality of light components overlaps with a first region among regions where the zero-order light is located, and a second light component among the plurality of light components does not overlap, and the first light component does not overlap with a second region different from the first region among regions where the zero-order light is located, and the second light component overlaps, and is configured as such. A light projecting device characterized by the above. [Appendix 2] A light source that emits irradiation light, A diffractive optical element that converts the irradiation light into pattern light, and has The light source and the diffractive optical element are configured such that the pattern light includes a plurality of light components of ± first-order light or higher at different positions from each other, and in a region where the zero-order light is located, there is a region where the light components of ± first-order light among the plurality of light components overlap and light components of an order higher than ± first-order among the plurality of light components do not overlap. A light projecting device characterized by the above. [Appendix 3] The light source and the diffractive optical element are configured such that the plurality of light components include a light component of + first-order light and a light component of - first-order light, and a part of the light component of + first-order light and a part of the light component of - first-order light overlap with at least a part of the region where the zero-order light is located. The light projecting device according to Appendix 1 or Appendix 2, characterized by the above. [Appendix 4] The light source emits a plurality of irradiation lights with the number of divisions in the vertical direction being a (a is a positive integer) and the number of divisions in the horizontal direction being b (b is a positive integer) as the irradiation light, The diffractive optical element converts each of the plurality of irradiation lights into the pattern light by dividing them with the number of divisions in the vertical direction being M (M is a positive integer) and the number of divisions in the horizontal direction being N (N is a positive integer). The light projecting device according to any one of Appendices 1 to 3, characterized by the above. [Appendix 5] Both the number of divisions M in the vertical direction and the number of divisions N in the horizontal direction by the diffractive optical element are odd numbers. The light projecting device according to appended note 4, characterized in that... [Appended note 6] The number of vertical divisions M by the diffractive optical element is odd, the number of horizontal divisions N by the diffractive optical element is even, and the number of horizontal divisions b by the light source is even. The light projecting device according to appended note 4, characterized in that... [Appended note 7] The number of vertical divisions M by the diffractive optical element is even, the number of horizontal divisions N by the diffractive optical element is odd, and the number of vertical divisions a by the light source is even. The light projecting device according to appended note 4, characterized in that... [Appended note 8] Both the number of vertical divisions M and the number of horizontal divisions N by the diffractive optical element are even, and both the number of vertical divisions a and the number of horizontal divisions b by the light source are odd. The light projecting device according to appended note 4, characterized in that... [Appended note 9] Both the number of vertical divisions M and the number of horizontal divisions N by the diffractive optical element are even, and both the number of vertical divisions a and the number of horizontal divisions b by the light source are even. The light projecting device according to appended note 4, characterized in that... [Appended note 10] The light projecting device according to any one of appended notes 1 to 9, A light receiving unit that receives reflected light from an object on which the pattern light is projected, A control unit that controls the light source and the light receiving unit, A light transmitting and receiving device, characterized by comprising... [Appended note 11] The light transmitting and receiving device according to appended note 10, A distance calculation unit that calculates the distance to the object based on the output from the light receiving unit, A distance measuring system, characterized by comprising...

Explanation of reference numerals

[0132] 1: Distance measuring system 2: Light transmitting and receiving device 3: Arithmetic unit 31: Calculation unit (distance calculation unit) 10: Light transmitting and receiving unit 20: RGB light receiving unit 100: ToF light projecting unit (light projecting device) 102: Light source 104: First lens group 106: Diffractive optical element (DOE: Diffractive Optical Element) 108: Second lens group 120: ToF light receiving unit (light receiving unit) 122: Optical system 124: ToF sensor 140: Control unit

Claims

1. A light source that emits irradiation light, A diffractive optical element that converts the irradiation light into patterned light, having, The light source and the diffractive optical element, The patterned light includes a plurality of light components of ± first-order light or higher at different positions, and a first light component among the plurality of light components overlaps a first region among regions where the zero-order light is located, and a second light component among the plurality of light components does not overlap, and the first light component does not overlap and the second light component overlaps a second region different from the first region among regions where the zero-order light is located, A light projecting device characterized by this.

2. A light source that emits irradiation light, A diffractive optical element that converts the irradiation light into patterned light, having, The light source and the diffractive optical element, The patterned light includes a plurality of light components of ± first-order light or higher at different positions, and in a region where the zero-order light is located, a region where the light components of ± first-order light among the plurality of light components overlap and light components of an order higher than ± first-order among the plurality of light components do not overlap is included, A light projecting device characterized by this.

3. The light source and the diffractive optical element, The plurality of light components include a light component of + first-order light and a light component of - first-order light, and a part of the light component of + first-order light and a part of the light component of - first-order light overlap at least a part of the region where the zero-order light is located, The light projecting device according to claim 1 or claim 2, characterized by this.

4. The light source emits a plurality of irradiation lights with the number of divisions in the vertical direction being a (a is a positive integer) and the number of divisions in the horizontal direction being b (b is a positive integer) as the irradiation light, The diffractive optical element converts each of the plurality of irradiation lights into the patterned light by dividing them with the number of divisions in the vertical direction being M (M is a positive integer) and the number of divisions in the horizontal direction being N (N is a positive integer), The light projecting device according to claim 1 or claim 2, characterized by this.

5. Both the number of divisions M in the vertical direction and the number of divisions N in the horizontal direction by the diffractive optical element are odd numbers, The light projecting device according to claim 4, characterized by this.

6. The number of divisions M in the vertical direction by the diffractive optical element is an odd number, the number of divisions N in the horizontal direction by the diffractive optical element is an even number, and the number of divisions b in the horizontal direction by the light source is an even number, The light projecting device according to claim 4, characterized by this.

7. The number of vertical divisions M by the diffractive optical element is even, the number of horizontal divisions N by the diffractive optical element is odd, and the number of vertical divisions a by the light source is even. The light projecting device according to claim 4, characterized in that.

8. Both the number of vertical divisions M and the number of horizontal divisions N by the diffractive optical element are even, and both the number of vertical divisions a and the number of horizontal divisions b by the light source are odd. The light projecting device according to claim 4, characterized in that.

9. Both the number of vertical divisions M and the number of horizontal divisions N by the diffractive optical element are even, and both the number of vertical divisions a and the number of horizontal divisions b by the light source are even. The light projecting device according to claim 4, characterized in that.

10. The light projecting device according to any one of claims 1 to 9, A light receiving unit that receives reflected light from an object onto which the pattern light is projected, A control unit that controls the light source and the light receiving unit, A light transmitting and receiving device, characterized by comprising.

11. The light transmitting and receiving device according to claim 10, A distance calculation unit that calculates the distance to the object based on the output from the light reception by the light receiving unit, A distance measurement system, characterized by comprising.

Images