Distance measurement device, distance measurement method, and distance measurement program

Abstract

A projection unit 30 projects pattern light 50 within a range in which the visual field of a first imaging unit 10 and the visual field of a second imaging unit 20 overlap each other. A measurement unit 404 measures the distance to a measurement surface to which the pattern light 50 has been projected, on the basis of parallax between a first image acquired by the first imaging unit 10 and a second image acquired by the second imaging unit 20. The pattern light 50 is formed of a plurality of light regions different in hue and a plurality of light regions having the same hue but different in luminance, has a plurality of wide-range light regions 51 distributed in a prescribed pattern, and has, in each of the wide-range light regions 51, narrow-range light regions 52 distributed in a prescribed pattern.

Description

Distance measurement device, distance measurement method, and distance measurement program

[0001] The technology disclosed herein relates to distance measurement technology.

[0002] Patent Literature 1 discloses a three-dimensional measurement system. In this system, an imaging unit includes a first imaging unit and a second imaging unit that are spaced apart from each other. A first calculation unit calculates the parallax of a first feature point by using at least one of the images of the object captured by the first imaging unit and the second imaging unit and distance information obtained by a three-dimensional measurement method other than the stereo camera method. A second calculation unit calculates the parallax of a second feature point by using both the images of the object captured by the first imaging unit and the second imaging unit and the stereo camera method. The second calculation unit then identifies the three-dimensional shape of the object from the parallax of the first feature point and the parallax of the second feature point.

[0003] Japanese Patent Application Laid-Open No. 2021-192064

[0004] In a system such as that described in Patent Document 1, if the measurement surface to be measured for distance includes a plain surface without texture, it is difficult to perform stereo matching (corresponding point search) on such a plain surface, which makes it difficult to accurately measure the distance to the measurement surface.

[0005] The technology disclosed herein relates to a distance measurement device, which comprises a first imaging unit and a second imaging unit arranged side by side so that their respective fields of view overlap, a projection unit that projects pattern light into the area where the fields of view of the first imaging unit and the second imaging unit overlap, and a measurement unit that measures the distance to a measurement surface onto which the pattern light is projected based on the parallax between a first image obtained by the first imaging unit and a second image obtained by the second imaging unit, and the pattern light is pattern light that consists of a plurality of light regions with different hues and a plurality of light regions with the same hue but different luminance, and where a plurality of wide-range light regions are distributed in a predetermined pattern, and in each of the plurality of wide-range light regions, a plurality of narrow-range light regions are distributed in a predetermined pattern.

[0006] The technology disclosed herein relates to a distance measurement method performed using a first imaging unit and a second imaging unit arranged side by side so that their fields of view overlap, and a projection unit that projects pattern light into the area where the fields of view of the first imaging unit and the second imaging unit overlap. This distance measurement method includes a projection step of projecting the pattern light from the projection unit, an acquisition step of acquiring a first image obtained by the first imaging unit and a second image obtained by the second imaging unit, and a measurement step of measuring the distance to a measurement surface onto which the pattern light is projected based on the parallax between the first image and the second image. The pattern light is pattern light that consists of a plurality of light regions with different hues and a plurality of light regions with the same hue but different brightness, and where a plurality of wide-range light regions are distributed in a predetermined pattern, and in each of the plurality of wide-range light regions, a plurality of narrow-range light regions are distributed in a predetermined pattern.

[0007] The technology disclosed herein relates to a distance measurement program that causes a computer to execute the distance measurement method.

[0008] According to the technology disclosed herein, even if the measurement surface includes a plain surface, the distance to the measurement surface can be accurately measured.

[0009] FIG. 1 is a schematic diagram illustrating a schematic configuration of a distance measurement device of an embodiment. FIG. 2 is a block diagram illustrating a configuration of a distance measurement device of an embodiment. FIG. 3 is a block diagram illustrating a functional configuration of a control unit. FIG. 4 is a schematic diagram for explaining a first search process. FIG. 5 is a schematic diagram for explaining a second search process. FIG. 6 is a schematic diagram illustrating projected pattern light. FIG. 7 is a schematic diagram illustrating a configuration of a filter. FIG. 8 is a graph for explaining output characteristics of a light source and transmittance characteristics of a filter region in an embodiment. FIG. 9 is a flowchart illustrating a brightness adjustment process. FIG. 10 is a flowchart illustrating a distance measurement process. FIG. 11 is a block diagram illustrating a configuration of a distance measurement device of a first modified example of an embodiment. FIG. 12 is a graph for explaining output characteristics of a light source and transmittance characteristics of a filter region in a first modified example of an embodiment. FIG. 13 is a schematic diagram illustrating a part of pattern light in a second modified example of an embodiment. FIG. 14 is a schematic diagram illustrating a part of a filter in a second modified example of an embodiment.

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and their description will not be repeated.

[0011] 1 and 2 illustrate the configuration of a distance measurement device 1 according to an embodiment. The distance measurement device 1 includes a first imaging unit 10, a second imaging unit 20, a projection unit 30, a control unit 40, a storage unit 41, and a communication interface 42. The distance measurement device 1 measures a distance D0 to a measurement surface. In this example, the measurement surface is the surface of an object OB (the surface facing the first imaging unit 10 and the second imaging unit 20).

[0012] In the following description, the direction perpendicular to the X-axis direction will be referred to as the "Y-axis direction," and the direction perpendicular to both the X-axis direction and the Y-axis direction will be referred to as the "Z-axis direction."

[0013] [Imaging Unit] The first imaging unit 10 images the range of a first field of view 10a. The second imaging unit 20 images the range of a second field of view 20a. The first imaging unit 10 and the second imaging unit 20 are arranged side by side so that their respective fields of view overlap. In this example, the first field of view 10a and the second field of view 20a are oriented in the Z-axis direction, and the first imaging unit 10 and the second imaging unit 20 are aligned in the X-axis direction.

[0014] The measurement surface (in this example, the surface of the object OB) is included in the overlapping range of the first field of view 10a of the first imaging unit 10 and the second field of view 20a of the second imaging unit 20. The first imaging unit 10 and the second imaging unit 20 are so-called stereo cameras, and simultaneously capture images of the field of view (imaged range) from different viewpoints.

[0015] The first imaging unit 10 captures an image of the first field of view 10a at predetermined time intervals to obtain a first image P10. The first imaging unit 10 includes a first imaging lens 11 and a first imaging element 12.

[0016] The first imaging lens 11 focuses light from the field of view 10a of the first imaging unit 10 onto the first imaging surface 12a of the first imaging element 12. The first imaging lens 11 may be a single lens having a predetermined focal length, or may be a combination of multiple lenses.

[0017] The first imaging element 12 converts light incident on the first imaging surface 12a into an electrical signal, thereby obtaining a first image P10. The first imaging element 12 may be a monochrome image sensor. For example, the first imaging element 12 may be a CMOS image sensor, a CCD image sensor, or the like.

[0018] The second imaging unit 20 acquires a second image P20 by capturing an image of the range of a second field of view 20a at predetermined time intervals. The configuration of the second imaging unit 20 is similar to that of the first imaging unit 10. The second imaging unit 20 has a second imaging lens 21 and a second imaging element 22. The configurations of the second imaging lens 21 and the second imaging element 22 are similar to the configurations of the first imaging lens 11 and the first imaging element 12, and the second imaging lens 21 has the same focal length as the first imaging lens 11. The second imaging element 22 has a second imaging surface 22a.

[0019] The size of the second image P20 is the same as the size of the first image P10. The size of the pixels of the second image P20 is the same as the size of the pixels of the first image P10. The number of pixels included in the second image P20 is the same as the number of pixels included in the first image P10.

[0020] The imaging direction of the first imaging unit 10 may be slightly tilted from the Z-axis direction toward the second imaging unit 20 (direction toward the second imaging unit 20). The imaging direction of the second imaging unit 20 may be slightly tilted from the Z-axis direction toward the first imaging unit 10 (direction toward the first imaging unit 10). The positions of the first imaging unit 10 and the second imaging unit 20 in the Z-axis direction and the Y-axis direction are the same.

[0021] [Projection Unit] The projection unit 30 projects pattern light 50 into the area where the first field of view 10a of the first imaging unit 10 and the second field of view 20a of the second imaging unit 20 overlap. In this example, the projection direction of the pattern light 50 by the projection unit 30 is the Z-axis direction. The pattern light 50 is projected onto the surface (an example of a measurement surface) of the object OB included in the area where the first field of view 10a of the first imaging unit 10 and the second field of view 20a of the second imaging unit 20 overlap. The pattern light 50 will be described in detail later.

[0022] The projection unit 30 includes a light source 31 , an optical system 32 , a filter 33 , a projection lens 34 , and a light source driver 35 .

[0023] The light source 31 emits light used to generate the pattern light 50. In this example, the light source 31 includes first to third light sources 311 to 313. For example, the first light source 311 emits light in a wavelength band corresponding to "red" (590 to 640 nm). The second light source 312 emits light in a wavelength band corresponding to "green" (490 to 550 nm). The third light source 313 emits light in a wavelength band corresponding to "blue" (430 to 490 nm). The first to third light sources 311 to 313 may be light-emitting diodes or other types of light sources such as semiconductor lasers.

[0024] The optical system 32 guides the light emitted from the light source 31 to the filter 33. In this example, the optical system 32 has first to third collimator lenses 321 to 323, a first dichroic mirror 324, and a second dichroic mirror 325.

[0025] The first to third collimator lenses 321 to 323 convert the light emitted from the first to third light sources 311 to 313 into approximately parallel light, respectively. The first dichroic mirror 324 transmits the light incident from the first collimator lens 321 and reflects the light incident from the second collimator lens 322. The second dichroic mirror 325 transmits the light incident from the first dichroic mirror 324 and reflects the light incident from the third collimator lens 323. With this configuration, the light emitted from each of the first to third light sources 311 to 313 is integrated and guided to the filter 33.

[0026] The filter 33 generates the pattern light 50. The configuration of the filter 33 will be described in detail later.

[0027] The projection lens 34 projects the pattern light 50 generated by the filter 33. The projection lens 34 may be a single lens or a combination of multiple lenses.

[0028] The light source driving unit 35 drives the light sources 31 (in this example, the first to third light sources 311 to 313) in response to control by the control unit 40. Specifically, based on the drive current value set by the control unit 40, the light source driving unit 35 drives the light sources 31 so as to emit light with a brightness according to the drive current value.

[0029] [Control Unit] The control unit 40 performs various processes. Specifically, the control unit 40 acquires information and data from each unit of the distance measurement device 1, and performs various processes based on that information and data. The processes performed by the control unit 40 will be described in detail later.

[0030] For example, the control unit 40 includes a processor and a memory (storage medium) that stores a program for operating the processor. The processor executes the program to realize various functions of the control unit 40. In other words, the control unit 40 has various functional blocks that realize various functions. The control unit 40 is an example of a computer. The program is an example of a distance measurement program.

[0031] The control unit 40 and the communication interface 42 may be configured by a semiconductor integrated circuit such as an FPGA (Field Programmable Gate Array), or may be configured by other semiconductor integrated circuits such as a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), or an ASIC (Application Specific Integrated Circuit).

[0032] [Storage Unit] The storage unit 41 stores various types of information and data. In this example, the storage unit 41 stores a first image P10 obtained by the first imaging unit 10, a second image P20 obtained by the second imaging unit 20, etc.

[0033] 3 illustrates an example of the functional configuration of the control unit 40. In this example, the control unit 40 includes a first imaging processing unit 401, a second imaging processing unit 402, a projection control unit 403, and a measurement unit 404.

[0034] <Image Capture Processing Unit> The first image capture processing unit 401 controls the first image capture element 12 of the first image capture unit 10. The first image capture processing unit 401 also performs preprocessing such as brightness correction and camera correction on the first image P10 (pixel signal) obtained by the first image capture element 12 of the first image capture unit 10. In this example, the first image P10 processed by the first image capture processing unit 401 is stored in the storage unit 41.

[0035] The second imaging processing unit 402 controls the second imaging element 22 of the second imaging unit 20. In addition, the second imaging processing unit 402 performs preprocessing such as brightness correction and camera correction on the second image P20 (pixel signal) obtained by the second imaging element 22 of the second imaging unit 20. In this example, the second image P20 processed by the second imaging processing unit 402 is stored in the storage unit 41.

[0036] <Projection Control Unit> The projection control unit 403 controls the projection unit 30. Specifically, the projection control unit 403 controls the projection unit 30 to cause the projection unit 30 to project the pattern light 50.

[0037] In this example, the projection control unit 403 performs a brightness adjustment process. In the brightness adjustment process, the projection control unit 403 adjusts the brightness of the light emitted from the light source 31 (in this example, each of the first to third light sources 311 to 313) so that the brightness of the light emitted from the light source 31 does not become saturated. The brightness adjustment process will be described in detail later.

[0038] <Measurement Unit> The measurement unit 404 measures a distance D0 to a measurement surface onto which the pattern light 50 is projected, based on the parallax between a first image P10 obtained by the first imaging unit 10 and a second image P20 obtained by the second imaging unit 20. In this example, the measurement unit 404 has a first search unit 411, a second search unit 412, and a distance derivation unit 413.

[0039] <<First Search Unit (First Search Process)>> The first search unit 411 performs the first search process. In the first search process, the first search unit 411 sequentially selects wide range reference blocks B11 from the first image P10, and searches the second image P20 for wide range corresponding blocks B21 that correspond to the wide range reference blocks B11. Specifically, in the first search process, the first search unit 411 performs the following processes.

[0040] As shown in FIG. 4 , the first search unit 411 sequentially selects wide-range reference blocks B11 from the first image P10 by shifting the pixel range for selecting the wide-range reference blocks B11 in the first image P10 by a predetermined amount (first reference shift amount). The wide-range reference blocks B11 are pixel blocks (groups of multiple pixels) that serve as reference points for the corresponding point search in the first search process. In the example of FIG. 4 , the wide-range reference blocks B11 include 36 pixels arranged in a matrix of 6 rows and 6 columns. The first search unit 411 then performs the following process on each of the wide-range reference blocks B11 sequentially selected from the first image P10.

[0041] The first search unit 411 sequentially selects wide range reference blocks BR1 from the second image P20 by shifting a pixel range in the second image P20 by a predetermined amount (first reference shift amount) for selecting a wide range reference block BR1 to be compared with the wide range reference block B11. The wide range reference block BR1 is a pixel block that is a candidate for the wide range corresponding block B21 that corresponds to the wide range reference block B11. The shape and size of the wide range reference block BR1 are the same as those of the wide range reference block B11.

[0042] The first search unit 411 derives the similarity between each of the wide range reference blocks BR1 selected in order from the second image P20 and the wide range reference block B11. Then, the first search unit 411 determines, among the wide range reference blocks BR1 selected in order from the second image P20, the "wide range reference block BR1 having the greatest similarity to the wide range reference block B11" as the wide range corresponding block B21 corresponding to the wide range reference block B11.

[0043] Note that the similarity can be derived using a well-known similarity derivation process (similarity calculation method), such as ZNCC (Zero Means Normalized Cross-Correlation), NCC (Normalized Cross Correlation), SSD (Sum of Squared Difference), and SAD (Sum of Absolute Difference).

[0044] As described above, by performing the first search process on the first image P10 and the second image P20, multiple combinations of wide-range reference blocks B11 and wide-range corresponding blocks B21 (hereinafter referred to as "wide-range block combinations") are obtained.

[0045] In this example, the first search unit 411 sequentially selects wide range reference blocks BR1 within a search range R20 that starts from the same position in the second image P20 as the "position of the wide range reference block B11 in the first image P10" and extends in a direction (left and right in this example) corresponding to the "separation direction (X-axis direction) between the first imaging unit 10 and the second imaging unit 20." The extension direction of the search range R20 is set to the direction in which the pixel block located at the above-mentioned starting point deviates from the above-mentioned starting point due to parallax.

[0046] The starting point of the search range R20 is not limited to the same position (reference position) in the second image P20 as the "position of the wide-range reference block B11 in the first image P10." For example, the starting point of the search range R20 may be set to a position in the second image P20 that is shifted a predetermined amount (e.g., several blocks) to the right (the direction of shift due to parallax) from the reference position.

[0047] <<Second Search Unit (Second Search Process)>> The second search unit 412 performs the second search process. In the second search process, the second search unit 412 sequentially selects narrow range reference blocks B12 from the wide range reference blocks B11, and searches for the narrow range corresponding block B22 corresponding to the narrow range reference block B12 from the wide range corresponding blocks B21 corresponding to the wide range reference block B11. Specifically, in the second search process, the second search unit 412 performs the following process.

[0048] The second search unit 412 performs the following processing on each of the wide range reference blocks B11 selected in order from the first image P10 by the first search unit 411 and the wide range corresponding block B21 corresponding to that wide range reference block B11.

[0049] As shown in FIG. 5 , the second search unit 412 sequentially selects narrow-range reference blocks B12 from the wide-range reference block B11 by shifting the pixel range for selecting the narrow-range reference block B12 in the wide-range reference block B11 by a predetermined amount (second reference shift amount). The narrow-range reference block B12 is a pixel block that serves as a reference point for searching for corresponding points in the second search process. The narrow-range reference block B12 is smaller than the wide-range reference block B11. In the example of FIG. 5 , the narrow-range reference block B12 includes nine pixels arranged in a matrix of three rows and three columns. The second search unit 412 then performs the following process on each narrow-range reference block B12 sequentially selected from the wide-range reference block B11.

[0050] The second search unit 412 sequentially selects narrow-range reference blocks BR2 from the wide-range corresponding blocks B21 by shifting the pixel range for selecting the narrow-range reference block BR2 to be compared with the narrow-range reference block B12 by a predetermined amount (second reference shift amount) in the wide-range corresponding block B21 corresponding to the wide-range reference block B11. The narrow-range reference blocks BR2 are pixel blocks that are candidates for the narrow-range corresponding block B22 corresponding to the narrow-range reference block B12.

[0051] The second search unit 412 derives the similarity between each of the narrow-range reference blocks BR2 selected in order from the wide-range correspondence blocks B21 and the narrow-range reference block B12. Then, the second search unit 412 determines the narrow-range reference block BR2 that has the greatest similarity to the narrow-range reference block B12, among the narrow-range reference blocks BR2 selected in order from the wide-range correspondence blocks B21, as the narrow-range correspondence block B22 corresponding to the narrow-range reference block B12.

[0052] As described above, by performing a second search process on each of multiple wide-range block combinations (combinations of wide-range reference block B11 and wide-range corresponding block B21), multiple combinations of narrow-range reference block B12 and narrow-range corresponding block B22 (hereinafter referred to as ``narrow-range block combinations'') are obtained.

[0053] <<Distance Derivation Unit (Distance Derivation Process)>> The distance derivation unit 413 performs distance derivation processing. In the distance derivation processing, the distance derivation unit 413 derives the distance to the measurement surface in the narrow range reference block B12 based on the parallax between the narrow range reference block B12 and the narrow range corresponding block B22. Specifically, the distance derivation unit 413 performs the following processing in the distance derivation processing.

[0054] The distance derivation unit 413 performs the following process on each of the narrow-range reference blocks B12 selected in order from within the first image P10 by the second search unit 412.

[0055] The distance derivation unit 413 selects a narrow-range corresponding block B22 (narrow-range corresponding block B22 detected by the second search unit 412) corresponding to the narrow-range reference block B12 from the second image P20, and derives the difference in position (pixel shift amount) between the narrow-range reference block B12 and the narrow-range corresponding block B22.

[0056] Then, the distance derivation unit 413 derives the distance D0 in the narrow range reference block B12 (distance D0 to the measurement surface) based on the derived position difference. Specifically, the distance derivation unit 413 derives the distance D0 to the measurement surface by triangulation from the derived position difference (pixel shift amount), the separation distance between the first imaging unit 10 and the second imaging unit 20, and the focal length of the first imaging lens 11. Note that the focal length of the second imaging lens 21 is equal to the focal length of the first imaging lens 11.

[0057] The above processing obtains the distance D0 (distance D0 to the measurement surface) for each of the narrow-range reference blocks B12 selected in order from within the first image P10 by the second search unit 412. The distance derivation unit 413 outputs distance information indicating the distance D0 for each of the narrow-range reference blocks B12 selected in order from within the first image P10. For example, the distance derivation unit 413 transmits the distance information to an external device via the communication interface 42.

[0058] <<Search Accuracy>> In this example, the accuracy of the search by the first search unit 411 is lower than the accuracy of the search by the second search unit 412 .

[0059] Specifically, the first reference movement amount, which is the amount of movement of the pixel range for selecting the wide-range reference block B11, is greater than the second reference movement amount, which is the amount of movement of the pixel range for selecting the narrow-range reference block B12. Specifically, the first reference movement amount is set to n pixels (n is an integer of 2 or greater), and the second reference movement amount is set to m pixels (m is an integer of 1 or greater and smaller than n). Preferably, the second reference movement amount is 1 pixel.

[0060] Furthermore, the first reference movement amount, which is the amount of movement of the pixel range for selecting the wide-range reference block BR1, is greater than the second reference movement amount, which is the amount of movement of the pixel range for selecting the narrow-range reference block BR2. Specifically, the first reference movement amount is set to j pixels (j is an integer of 2 or greater), and the second reference movement amount is set to k pixels (k is an integer of 1 or greater and smaller than j). It is preferable that the second reference movement amount is 1 pixel.

[0061] [Pattern Light] Next, the pattern light 50 will be described with reference to Fig. 6. Fig. 6 illustrates the pattern light 50 projected onto the measurement surface.

[0062] <Wide Range Light Region> The pattern light 50 includes a plurality of wide range light regions 51. In the example of Fig. 6, the plurality of wide range light regions 51 are arranged in a matrix. Specifically, in the pattern light 50, 35 wide range light regions 51 are arranged in a matrix of 5 rows and 7 columns.

[0063] The multiple wide range light regions 51 are classified into multiple types based on hue. In other words, the multiple wide range light regions 51 include multiple (two or more) wide range light regions 51 with different hues. In the pattern light 50, the multiple types of wide range light regions 51 (multiple wide range light regions 51 with different hues) are distributed in a predetermined hue pattern. In this example, the hue pattern is a random pattern.

[0064] 6, the plurality of wide range light regions 51 are classified into four types of wide range light regions 51 (specifically, first to fourth wide range light regions 511 to 514). The first to fourth wide range light regions 511 to 514 have different hues (in other words, different wavelength bands of light).

[0065] The hue of the first wide range light region 511 is "red." The hue of the second wide range light region 512 is "orange." The hue of the third wide range light region 513 is "green." The hue of the fourth wide range light region 514 is "blue." In other words, the wavelength band of light in the first wide range light region 511 is a wavelength band corresponding to red (e.g., 640 to 770 nm). The wavelength band of light in the second wide range light region 512 is a wavelength band corresponding to orange (e.g., 590 to 640 nm). The wavelength band of light in the third wide range light region 513 is a wavelength band corresponding to green (e.g., 490 to 550 nm). The wavelength band of light in the fourth wide range light region 514 is a wavelength band corresponding to blue (e.g., 430 to 490 nm).

[0066] 6, each wide-range light region 51 is labeled with a letter (R, O, G, B) indicating its hue. "R" indicates red, "O" indicates orange, "G" indicates green, and "B" indicates blue. For example, the hue of a wide-range light region 51 labeled with "R" is "red."

[0067] <Narrow-Range Light Region> Each of the wide-range light regions 51 includes a plurality of narrow-range light regions 52. In the example of Fig. 6, the narrow-range light regions 52 are arranged in a matrix. Specifically, in each of the 35 wide-range light regions 51, nine narrow-range light regions 52 are arranged in a matrix of three rows and three columns.

[0068] The multiple narrow-range light regions 52 are classified into multiple types based on hue and luminance. In other words, the multiple narrow-range light regions 52 included in each of the multiple wide-range light regions 51 include multiple (two or more) narrow-range light regions 52 that have the same hue but different luminance. In each of the multiple wide-range light regions 51, the multiple types of narrow-range light regions 52 (multiple narrow-range light regions 52 that have the same hue but different luminance) are distributed in a predetermined luminance pattern. In this example, the luminance pattern is a random pattern.

[0069] 6, the narrow-range light regions 52 included in each of the wide-range light regions 51 are classified into four types of narrow-range light regions 52 (specifically, first to fourth narrow-range light regions 521 to 524). The first to fourth narrow-range light regions 521 to 524 have different brightness levels.

[0070] The brightness of the first narrow-range light region 521 is "Level 1." The brightness of the second narrow-range light region 522 is "Level 2." The brightness of the third narrow-range light region 523 is "Level 3." The brightness of the fourth narrow-range light region 524 is "Level 4." Note that the brightness gradually increases as the brightness level moves from "Level 1" to "Level 4."

[0071] 6, each narrow-range light region 52 is labeled with a letter (R, O, G, B) indicating the hue and a number (1, 2, 3, 4) indicating the brightness level. For example, the narrow-range light region 52 labeled "R1" has a hue of "red" and a brightness of "Level 1."

[0072] [Relationship between Pattern Light and Image] Pattern light 50 is projected onto a measurement surface (in this example, the surface of the object OB), and the first imaging unit 10 captures an image of the range of the first field of view 10a that includes the measurement surface onto which the pattern light 50 is projected, thereby obtaining a first image P10 that includes the pattern light 50 projected onto the measurement surface. Furthermore, imaging by the second imaging unit 20 is performed (simultaneously) with imaging by the first imaging unit 10, thereby obtaining a second image P20 that includes the pattern light 50 projected onto the measurement surface.

[0073] In this example, when the distance D0 to the measurement surface is a reference distance (e.g., the intermediate distance of the ranging range), the shape of the narrow-range light region 52 (dot light) in the pattern light 50 included in the first image P10 corresponds to the shape of one pixel in the first image P10 (e.g., the same shape as the shape of one pixel). The ranging range is the range of distances that can be measured by the distance measurement device 1. The size of the narrow-range light region 52 (dot light) corresponds to the size of one pixel in the first image P10 (e.g., the same size as the size of one pixel).

[0074] The shape and size of the narrow-range light region 52 in the first image P10 are not limited to the above-described shape and size. For example, the shape of the narrow-range light region 52 in the first image P10 may be different from the shape (rectangle) of one pixel in the first image P10. Furthermore, the size of the narrow-range light region 52 in the first image P10 may correspond to the size of two or more (preferably two to four) pixels in the first image P10. In other words, one narrow-range light region 52 in the first image P10 may be included in two or more pixels in the first image P10. Alternatively, the size of the narrow-range light region 52 in the first image P10 may be smaller than the size of one pixel in the first image P10.

[0075] In this example, when the distance D0 to the measurement surface is the reference distance, the shape of the wide-range light region 51 (a collection of dot lights) in the pattern light included in the first image P10 corresponds to the shape of the narrow-range reference block B12 (for example, the same shape as the narrow-range reference block B12). Also, the size of the wide-range light region 51 (a collection of dot lights) corresponds to the size of the narrow-range reference block B12 (for example, the same size as the narrow-range reference block B12).

[0076] The shape and size of the wide-range light region 51 in the first image P10 are not limited to the above. For example, the shape of the wide-range light region 51 in the first image P10 may be different from the shape (rectangular) of the narrow-range reference block B12. Furthermore, the size of the wide-range light region 51 in the first image P10 may be larger or smaller than the size of the narrow-range reference block B12.

[0077] Also, in this example, the hue pattern (distribution pattern of multiple types of wide-range light regions 51) in the pattern light 50 contained in the first image P10 is set so that multiple types of wide-range light regions 51 (multiple wide-range light regions 51 with different hues) are included in each of the wide-range reference blocks B11 selected in sequence from the first image P10 in the first search process, under the condition that the distance D0 to the measurement surface is the reference distance.

[0078] In addition, under the above conditions, it is preferable that the hue pattern in the pattern light 50 contained in the first image P10 is set so that the distribution patterns of the multiple types of wide-range light regions 51 contained in each of the wide-range reference blocks B11 selected in order from the first image P10 in the first search process are different from each other.

[0079] Also, in this example, the brightness pattern (distribution pattern of multiple types of narrow-range light regions 52) in each of the multiple wide-range light regions 51 included in the first image P10 is set so that, under the condition that the distance D0 to the measurement surface is the reference distance, each of the narrow-range reference blocks B12 selected in order from the wide-range reference blocks B11 in the second search process includes multiple types of narrow-range light regions 52 (multiple narrow-range light regions 52 having the same hue but different brightness).

[0080] In addition, it is preferable that the brightness pattern in each of the multiple wide-range light regions 51 included in the first image P10 is set so that, under the above conditions, the distribution patterns of the multiple types of narrow-range light regions 52 included in each of the narrow-range reference blocks B12 selected in order from the wide-range reference blocks B11 in the second search process are different from one another.

[0081] [Filter] Next, the configuration of the filter 33 will be described with reference to Fig. 7. Fig. 7 illustrates the filter 33 as viewed from the light incident surface side. In this example, the filter 33 is a transmission filter.

[0082] <Wide Range Filter Regions> The filter 33 includes a plurality of wide range filter regions 61. The plurality of wide range filter regions 61 correspond to the plurality of wide range light regions 51, respectively, and generate the corresponding wide range light regions 51. In the example of Fig. 7, the plurality of wide range filter regions 61 are arranged in a matrix. Specifically, in the filter 33, 35 wide range filter regions 61 are arranged in a matrix of 5 rows and 7 columns.

[0083] The plurality of wide range filter regions 61 are classified into a plurality of types according to the hue of the wide range light region 51 to be generated. In other words, the plurality of wide range filter regions 61 include a plurality (two or more) of wide range filter regions 61 having different hues of the wide range light region 51 to be generated.

[0084] In the pattern light 50, multiple types of wide-range filter regions 61 (multiple wide-range filter regions 61 with different hues of the wide-range light region 51 to be generated) correspond to multiple types of wide-range light regions 51, respectively, and are distributed in the same pattern as the hue pattern.

[0085] Each of the plurality of wide range filter regions 61 extracts light in a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61 from the light emitted from the light source 31 .

[0086] In this example, each of the multiple wide range filter regions 61 transmits light emitted from the light source 31 in a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61. Specifically, the transmission wavelength band (the wavelength band of light that can be transmitted) of each of the multiple wide range filter regions 61 is set to a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61. Each of the multiple wide range filter regions 61 has high transmittance for the transmission wavelength band (i.e., the wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61) and low transmittance for other wavelength bands.

[0087] 7, the plurality of wide range filter regions 61 are classified into four types of wide range filter regions 61 (specifically, first to fourth wide range filter regions 611 to 614) corresponding respectively to the four types of wide range light regions 51. The first to fourth wide range filter regions 611 to 614 have different transmission wavelength bands.

[0088] The transmission wavelength band of the first wide range filter region 611 is set to a wavelength band corresponding to "red", which is the hue of the first wide range light region 511. The transmission wavelength band of the second wide range filter region 612 is set to a wavelength band corresponding to "orange", which is the hue of the second wide range light region 512. The transmission wavelength band of the third wide range filter region 613 is set to a wavelength band corresponding to "green", which is the hue of the third wide range light region 513. The transmission wavelength band of the fourth wide range filter region 614 is set to a wavelength band corresponding to "blue", which is the hue of the fourth wide range light region 514.

[0089] In the example of FIG. 7 , each wide range filter region 61 is marked with a letter (r, o, g, b) indicating the hue corresponding to the transmission wavelength band of that wide range filter region 61. "r" indicates that the transmission wavelength band is a wavelength band corresponding to "red." "o" indicates that the transmission wavelength band is a wavelength band corresponding to "orange." "g" indicates that the transmission wavelength band is a wavelength band corresponding to "green." "b" indicates that the transmission wavelength band is a wavelength band corresponding to "blue." For example, the wide range filter region 61 marked with "r" has its transmission wavelength band set to a wavelength band corresponding to "red."

[0090] <Narrow Range Filter Region> Each of the multiple wide range filter regions 61 includes multiple narrow range filter regions 62. The multiple narrow range filter regions 62 correspond to the multiple narrow range light regions 52, respectively, and generate the corresponding narrow range light regions 52. In the example of Fig. 7, the multiple narrow range filter regions 62 are arranged in a matrix. Specifically, in each of the 35 wide range filter regions 61, nine narrow range filter regions 62 are arranged in a matrix of 3 rows and 3 columns.

[0091] The plurality of narrow-range filter regions 62 included in each of the plurality of wide-range filter regions 61 are classified into a plurality of types according to the luminance of the narrow-range light region 52 to be generated. In other words, the plurality of narrow-range filter regions 62 included in each of the plurality of wide-range filter regions 61 include a plurality (two or more) of narrow-range filter regions 62 having the same hue but different luminance of the narrow-range light region 52 to be generated.

[0092] In each of the multiple wide-range filter regions 61, multiple types of narrow-range filter regions 62 (multiple narrow-range filter regions 62 having the same hue but different brightness of the narrow-range light region 52 to be generated) correspond to multiple types of narrow-range light regions 52 included in the wide-range light region 51 corresponding to that wide-range filter region 61, and are distributed in the same pattern as the brightness pattern in that wide-range light region 51.

[0093] Each of the multiple narrow range filter regions 62 extracts light of a wavelength band corresponding to the hue of the wide range light region 51 corresponding to the wide range filter region 61 including that narrow range filter region 62 from the light emitted from the light source 31, in an amount corresponding to the brightness of the narrow range light region 52 corresponding to that narrow range filter region 62.

[0094] In this example, each of the multiple narrow-range filter regions 62 transmits light emitted from the light source 31 in a wavelength band corresponding to the hue of the wide-range light region 51 corresponding to the wide-range filter region 61 that includes that narrow-range filter region 62, at a transmittance corresponding to the brightness of the narrow-range light region 52 corresponding to that narrow-range filter region 62.

[0095] Specifically, the transmission wavelength band of each of the plurality of narrow-range filter regions 62 is set to a wavelength band corresponding to the hue of the wide-range light region 51 corresponding to the wide-range filter region 61 that includes that narrow-range filter region 62. The transmittance for the transmission wavelength band in each of the plurality of narrow-range filter regions 62 is set to a transmittance corresponding to the luminance of the narrow-range light region 52 corresponding to that narrow-range filter region 62. The higher the luminance of the narrow-range light region 52, the higher the transmittance for the transmission wavelength band of the narrow-range filter region 62. Note that the transmittance for wavelength bands other than the transmission wavelength band in each of the plurality of narrow-range filter regions 62 is lower than the transmittance for the transmission wavelength band.

[0096] 7, the plurality of narrow-range filter regions 62 included in each of the plurality of wide-range filter regions 61 are classified into four types of narrow-range filter regions 62 (specifically, first to fourth narrow-range filter regions 621 to 624) corresponding respectively to the four types of narrow-range light regions 52. The first to fourth narrow-range filter regions 621 to 624 have different levels of transmittance for the transmission wavelength band.

[0097] The transmittance level for the transmission wavelength band of the first narrow-range filter region 621 is set to "Level 1" corresponding to the luminance level (Level 1) of the first narrow-range light region 521 corresponding to the first narrow-range filter region 621. The transmittance level for the transmission wavelength band of the second narrow-range filter region 622 is set to "Level 2" corresponding to the luminance level (Level 2) of the second narrow-range light region 522 corresponding to the second narrow-range filter region 622. The transmittance level for the transmission wavelength band of the third narrow-range filter region 623 is set to "Level 3" corresponding to the luminance level (Level 3) of the third narrow-range light region 523 corresponding to the third narrow-range filter region 623. The transmittance level for the transmission wavelength band of the fourth narrow-range filter region 624 is set to "Level 4" corresponding to the luminance level (Level 4) of the fourth narrow-range light region 524 corresponding to the fourth narrow-range filter region 624. Note that the transmittance gradually increases as the transmittance level increases from "Level 1" to "Level 4."

[0098] 7 , each narrow-range filter region 62 is assigned a letter (r, o, g, b) indicating a hue corresponding to the transmission wavelength band of the narrow-range filter region 62, and a number (1, 2, 3, 4) indicating a transmittance level for that transmission wavelength band. For example, the narrow-range filter region 62 assigned "r1" has a transmission wavelength band set to a wavelength band corresponding to "red," and the transmittance for that transmission wavelength band is set to "Level 1."

[0099] 8A illustrates the output (spectral output) of each of the first light source 311, the second light source 312, and the third light source 313. The first light source 311 emits light having a center wavelength of approximately 610 nm and an emission bandwidth of approximately 80 nm. The second light source 312 emits light having a center wavelength of approximately 520 nm and an emission bandwidth of approximately 150 nm. The third light source 313 emits light having a center wavelength of approximately 470 nm and an emission bandwidth of approximately 100 nm.

[0100] As shown in FIG. 8(a), in this example, the maximum outputs of the first to third light sources 311 to 313 can be considered to be the same.

[0101] <Transmittance Characteristics of Narrow-Range Filter Region> Fig. 8(b) illustrates the transmittance characteristics of the first narrow-range filter region 621. In Fig. 8(b), "r1" indicates the transmittance characteristics of the first narrow-range filter region 621 in which the transmission wavelength band is set to a wavelength band corresponding to "red." "o1" indicates the transmittance characteristics of the first narrow-range filter region 621 in which the transmission wavelength band is set to a wavelength band corresponding to "orange." "g1" indicates the transmittance characteristics of the first narrow-range filter region 621 in which the transmission wavelength band is set to a wavelength band corresponding to "green." "b1" indicates the transmittance characteristics of the first narrow-range filter region 621 in which the transmission wavelength band is set to a wavelength band corresponding to "blue."

[0102] 8(b), in this example, the light transmittance (transmittance for the transmission wavelength band) in the first narrow-range filter region 621 is set to level 1 (approximately 0.3 times the maximum transmittance in the example of FIG. 8(b)) for all hues. As a result, the luminance level of the first narrow-range light region 521 generated by the first narrow-range filter region 621 is the same level (level 1) for all hues.

[0103] 8(c) illustrates the transmittance characteristics of the second narrow-range filter region 622. In FIG. 8(c), "r2" indicates the transmittance characteristics of the second narrow-range filter region 622 in which the transmission wavelength band is set to a wavelength band corresponding to "red." "o2" indicates the transmittance characteristics of the second narrow-range filter region 622 in which the transmission wavelength band is set to a wavelength band corresponding to "orange." "g2" indicates the transmittance characteristics of the second narrow-range filter region 622 in which the transmission wavelength band is set to a wavelength band corresponding to "green." "b2" indicates the transmittance characteristics of the second narrow-range filter region 622 in which the transmission wavelength band is set to a wavelength band corresponding to "blue."

[0104] 8(c), in this example, the light transmittance (transmittance for the transmission wavelength band) in the second narrow-range filter region 622 is set to level 2 (approximately 0.5 times the maximum transmittance in the example of FIG. 8(c)) for all hues. As a result, the luminance level of the second narrow-range light region 522 generated by the second narrow-range filter region 622 is the same level (level 2) for all hues.

[0105] 8(d) illustrates the transmittance characteristics of the third narrow-range filter region 623. In FIG. 8(d), "r3" indicates the transmittance characteristics of the third narrow-range filter region 623 in which the transmission wavelength band is set to a wavelength band corresponding to "red." "o3" indicates the transmittance characteristics of the third narrow-range filter region 623 in which the transmission wavelength band is set to a wavelength band corresponding to "orange." "g3" indicates the transmittance characteristics of the third narrow-range filter region 623 in which the transmission wavelength band is set to a wavelength band corresponding to "green." "b3" indicates the transmittance characteristics of the third narrow-range filter region 623 in which the transmission wavelength band is set to a wavelength band corresponding to "blue."

[0106] 8(d), in this example, the light transmittance (transmittance for the transmission wavelength band) in the third narrow-range filter region 623 is set to level 3 (approximately 0.7 times the maximum transmittance in the example of FIG. 8(d)) for all hues. As a result, the luminance level of the third narrow-range light region 523 generated by the third narrow-range filter region 623 is the same level (level 3) for all hues.

[0107] 8( e) illustrates the transmittance characteristics of the fourth narrow-range filter region 624. In FIG. 8( e), "r4" indicates the transmittance characteristics of the fourth narrow-range filter region 624 in which the transmission wavelength band is set to a wavelength band corresponding to "red." "o4" indicates the transmittance characteristics of the fourth narrow-range filter region 624 in which the transmission wavelength band is set to a wavelength band corresponding to "orange." "g4" indicates the transmittance characteristics of the fourth narrow-range filter region 624 in which the transmission wavelength band is set to a wavelength band corresponding to "green." "b4" indicates the transmittance characteristics of the fourth narrow-range filter region 624 in which the transmission wavelength band is set to a wavelength band corresponding to "blue."

[0108] 8( e), in this example, the light transmittance (transmittance for the transmission wavelength band) in the fourth narrow-range filter region 624 is set to level 4 (similar to the maximum transmittance in the example of FIG. 8( e)) for all hues. As a result, the luminance level of the fourth narrow-range light region 524 generated by the fourth narrow-range filter region 624 is the same level (level 4) for all hues.

[0109] [Brightness Adjustment Processing] Next, the brightness adjustment processing by the projection control unit 403 will be described with reference to Fig. 9. For example, the brightness adjustment processing is performed before the start of the distance measurement processing.

[0110] <Step S1> The projection control unit 403 sets the drive current value of the light source 31 (specifically, each of the first to third light sources 311 to 313) to an initial value. The initial drive current value is set so that the maximum luminance of light emitted from the light source 31 falls appropriately within the "range of gradations (e.g., 0 to 255) that defines the luminance in the control unit 40" when the reflectance of the measurement surface is a predetermined value (an expected standard reflectance). For example, the initial drive current value of the light source 31 is set so that the maximum luminance of the light source 31 is slightly smaller than the maximum gradation in the above-mentioned range of gradations (e.g., approximately 80 to 90% of the maximum gradation).

[0111] <Step S2> Next, the projection control unit 403 selects a light source 31 to be processed from the unprocessed light sources 31 among the first to third light sources 311 to 313, and causes the light source driving unit 35 to drive the selected light source 31. The unprocessed light source 31 is a light source 31 for which the processing of steps S2 to S4 has not been performed after step S1 or step S7.

[0112] Specifically, the projection control unit 403 transmits to the light source drive unit 35 a command to drive the light source 31 selected as the processing target, and a drive current value set for the light source 31. The light source drive unit 35 drives the light source 31 selected by the projection control unit 403 based on the drive current value transmitted from the projection control unit 403. As a result, light is projected from the light source 31 selected by the projection control unit 403 onto the measurement surface (the surface of the object OB in this example).

[0113] <Step S3> Next, the projection control unit 403 causes the first imaging unit 10 to capture an image while light is being projected onto the measurement surface from the light source 31 selected in step S2. This results in a first image P10 including the light projected onto the measurement surface from the light source 31.

[0114] <Step S4> Next, the projection control unit 403 obtains the maximum luminance of a pixel from the first image P10 obtained in step S3. This maximum luminance of a pixel corresponds to the maximum luminance of the light projected onto the measurement surface from the light source 31 selected in step S2.

[0115] <Step S5> Next, the projection control unit 403 determines whether or not there are any unprocessed light sources 31 remaining among the first to third light sources 311 to 313. If there are any unprocessed light sources 31 remaining, the processing of step S2 is performed, and if not, the processing of step S6 is performed.

[0116] <Step S6> Next, the projection control unit 403 determines whether the maximum brightness of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is appropriate. If the maximum brightness of the light emitted from the light source 31 is appropriate, the brightness adjustment process ends. If not, the process of step S7 is performed.

[0117] In this example, the projection control unit 403 determines whether or not there is an appropriate balance between the maximum luminances of the first to third light sources 311 to 313. The projection control unit 403 determines that there is an appropriate balance between the maximum luminances of the first to third light sources 311 to 313 when the maximum luminances of the first to third light sources 311 to 313 are considered to be the same (for example, when the difference between the maximum luminances of the first to third light sources 311 to 313 is within an allowable range).

[0118] In this example, the projection control unit 403 determines whether the maximum luminance of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is saturated. Specifically, the projection control unit 403 determines that the maximum luminance of the light source 31 is saturated when the maximum luminance of the light source 31 has reached the "maximum gradation in the range of gradations that defines the luminance in the control unit 40."

[0119] <Step S7> If the maximum brightness of the light source 31 (specifically, each of the first to third light sources 311 to 313) is not appropriate, the projection control unit 403 resets the drive current value of the light source 31 so that the maximum brightness of the light source 31 becomes appropriate. Next, the process of step S2 is performed.

[0120] For example, in this example, if the balance of the maximum brightness of each of the first to third light sources 311 to 313 is not appropriate, the projection control unit 403 resets the drive current value of each of the first to third light sources 311 to 313 so that the balance of the maximum brightness of each of the first to third light sources 311 to 313 is appropriate.

[0121] Specifically, the projection control unit 403 selects the highest maximum luminance as the "reference luminance" from the maximum luminances of the first to third light sources 311 to 313. Next, the projection control unit 403 selects a "light source 31 whose maximum luminance is lower than the reference luminance" from the first to third light sources 311 to 313, and increases the drive current value set for the selected light source 31.

[0122] Also, in this example, if the maximum brightness of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is saturated, the projection control unit 403 resets the drive current value of the light source 31 so that the brightness emitted from the light source 31 does not become saturated.

[0123] Specifically, the projection control unit 403 reduces the drive current value set for the light source 31 whose maximum luminance is saturated among the first to third light sources 311 to 313. For example, the projection control unit 403 corrects the drive current value set for the light source 31 so that it is lower by a predetermined gradation than the drive current value derived from the "relationship between the luminance of light emitted from the light source 31 and the drive current value" and the "maximum gradation in the range of gradations that defines the luminance in the control unit 40."

[0124] [Distance Measurement Processing] Next, the distance measurement processing will be described with reference to Fig. 10. The distance measurement processing is an example of a distance measurement method. For example, when the distance measurement device 1 is started, the control unit 40 performs the following processing.

[0125] <Step S10> First, the control unit 40 (projection control unit 403) controls the projection unit 30 so that the pattern light 50 is projected into the area where the first field of view 10a of the first imaging unit 10 and the second field of view 20a of the second imaging unit 20 overlap.

[0126] <Step S11> Next, the control unit 40 acquires the first image P10 acquired by the first imaging unit 10 and the second image P20 acquired by the second imaging unit 20. In this example, the control unit 40 selects the first image P10 and the second image P20 to be processed from the first images P10 and the second images P20 stored in the storage unit 41, and acquires the selected first image P10 and second image P20.

[0127] <Step S12> Next, the control unit 40 (first search unit 411) performs a first search process on the first image P10 and the second image P20 obtained in step S11, thereby obtaining multiple wide-range block combinations (combinations of wide-range reference blocks B11 and wide-range corresponding blocks B21).

[0128] <Step S13> Next, the control unit 40 (second search unit 412) performs a second search process on the wide-range block combination (combination of the wide-range reference block B11 and the wide-range corresponding block B21) obtained in step S12, thereby obtaining multiple narrow-range block combinations (combinations of the narrow-range reference block B12 and the narrow-range corresponding block B22).

[0129] <Step S14> Next, the control unit 40 (distance derivation unit 413) performs distance derivation processing based on the narrow-range block combinations (combinations of narrow-range reference blocks B12 and narrow-range corresponding blocks B22) obtained in step S13, thereby obtaining distance information (distance information indicating the distance D0 for each of the narrow-range reference blocks B12 selected in order from the first image P10).

[0130] <Step S15> Next, the control unit 40 determines whether or not to continue the distance measurement process. If the distance measurement process is to be continued, the process of step S11 is performed. If not, the distance measurement process ends.

[0131] Effect of the embodiment As described above, in the distance measurement device 1 of the embodiment, the projection unit 30 projects pattern light 50 into a range where the first field of view 10a of the first imaging unit 10 and the second field of view 20a of the second imaging unit 20 overlap. The pattern light 50 is pattern light in which a plurality of wide-range light regions 51 with different hues are distributed in a predetermined hue pattern, and in each of the plurality of wide-range light regions 51, a plurality of narrow-range light regions 52 with the same hue but different luminance are distributed in a predetermined luminance pattern.

[0132] In other words, the pattern light 50 is composed of a plurality of light regions with different hues and a plurality of light regions with the same hue but different brightness, and is a pattern light in which a plurality of wide-range light regions 51 are distributed in a predetermined pattern, and a plurality of narrow-range light regions 52 are distributed in a predetermined pattern in each of the plurality of wide-range light regions 51. The plurality of wide-range light regions 51 include a plurality of light regions with different hues and are distributed in a predetermined hue pattern. The plurality of narrow-range light regions 52 include a plurality of light regions with the same hue but different brightness and are distributed in a predetermined brightness pattern.

[0133] In the above configuration, a unique pattern (texture) can be formed on the measurement surface by projecting the unique pattern light 50 onto the measurement surface. This allows accurate stereo matching (corresponding point search) even when the measurement surface includes a plain surface (e.g., a flat, monochromatic surface). This allows accurate measurement of the distance D0 to the measurement surface.

[0134] Depending on the measurement surface, the light absorption rate may be high or the light reflectance may be low in a specific wavelength band. Therefore, if the pattern light 50 is composed of light in a single wavelength band, it will be difficult to form a unique pattern on the measurement surface if the wavelength band of the pattern light 50 falls within the specific wavelength band.

[0135] On the other hand, in the distance measurement device 1 of the embodiment, a plurality of wide range light regions 51 with different hues (wavelength bands) are distributed in a predetermined hue pattern in the pattern light 50. As a result, even if the wavelength band corresponding to the hue of any of the plurality of wide range light regions 51 is included in the above-mentioned specific wavelength band (a wavelength band where the light absorption rate is high or the light reflectance is low), the remaining wide range light regions 51 are projected onto the measurement surface (the surface of the object OB in this example), so that a unique pattern can be formed on the measurement surface.

[0136] Furthermore, in the distance measuring device 1 of the embodiment, the projection unit 30 has a filter 33 for generating pattern light 50. The filter 33 includes a plurality of wide range filter regions 61 that correspond to the plurality of wide range light regions 51, respectively, and are distributed in the same pattern as the hue pattern. Each of the plurality of wide range filter regions 61 includes a plurality of narrow range filter regions 62 that correspond to a plurality of narrow range light regions 52 included in the wide range light region 51 that corresponds to that wide range filter region 61 among the plurality of wide range light regions 51, and are distributed in the same pattern as the luminance pattern.

[0137] In other words, the filter 33 is composed of a plurality of filter regions for generating a plurality of light regions with different hues and a plurality of filter regions for generating a plurality of light regions with the same hue but different brightness, and includes a plurality of wide range filter regions 61 that correspond to the plurality of wide range light regions 51, respectively, and are distributed in the same pattern as a predetermined pattern (the predetermined pattern of the wide range light regions 51). Each of the plurality of wide range filter regions 61 includes a plurality of narrow range filter regions 62 that correspond to a plurality of narrow range light regions 52 included in the wide range light region 51 that corresponds to that wide range filter region 61 among the plurality of wide range light regions 51, and are distributed in the same pattern as a predetermined pattern (the predetermined pattern of the narrow range light regions 52).

[0138] The above configuration makes it possible to easily generate patterned light 50 having a desired pattern. Furthermore, unlike a diffractive optical element, there is no variation in diffraction efficiency (variation in brightness gradation) due to manufacturing errors or assembly errors, so patterned light 50 having a desired pattern can be stably generated.

[0139] In the distance measurement device 1 of the embodiment, the projection unit 30 includes a light source 31 and an optical system 32 that guides light emitted from the light source 31 to the filter 33. With this configuration, the filter 33 can be easily irradiated with light for generating the pattern light 50.

[0140] Furthermore, in the distance measurement device 1 of the embodiment, each of the plurality of wide range filter regions 61 transmits light of a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61, out of the light emitted from the light source 31. Each of the plurality of narrow range filter regions 62 transmits light of a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61, out of the light emitted from the light source 31, at a transmittance corresponding to the luminance of the narrow range light region 52 corresponding to that narrow range filter region 62.

[0141] In other words, in filter 33, each of the plurality of filter regions for generating a plurality of light regions with different hues extracts light in a wavelength band corresponding to a plurality of light regions with different hues from the light emitted from light source 31. Each of the plurality of filter regions for generating a plurality of light regions with the same hue but different luminance extracts light in a wavelength band corresponding to a plurality of light regions with the same hue but different luminance from the light emitted from light source 31 in an amount corresponding to the luminance.

[0142] In the above configuration, each of the plurality of wide range filter regions 61 can selectively extract light in a wavelength band corresponding to the hue of the wide range light region 51 corresponding to that wide range filter region 61. Each of the plurality of narrow range filter regions 62 can selectively extract light in a wavelength band corresponding to the hue of the wide range light region 51 corresponding to the wide range filter region 61 including that narrow range filter region 62, in an amount corresponding to the luminance of the narrow range light region 52 corresponding to that narrow range filter region 62. This makes it possible to efficiently generate pattern light 50.

[0143] Furthermore, in the distance measurement device 1 of the embodiment, the projection control unit 403 adjusts the brightness of the light emitted from the light source 31 so that the brightness of the light emitted from the light source 31 does not become saturated. With this configuration, the brightness of the light emitted from the light source 31 can be appropriately set.

[0144] In the distance measurement device 1 of the embodiment, the first search unit 411 sequentially selects wide range reference blocks B11 from the first image P10, and searches the second image P20 for wide range corresponding blocks B21 corresponding to the wide range reference blocks B11. The second search unit 412 sequentially selects narrow range reference blocks B12 from the wide range reference blocks B11, and searches for narrow range corresponding blocks B22 corresponding to the narrow range reference blocks B12 from the wide range corresponding blocks B21 corresponding to the wide range reference blocks B11. The distance derivation unit 413 derives the distance D0 to the measurement surface in the narrow range reference block B12 based on the difference in position between the narrow range reference block B12 and the narrow range corresponding block B22.

[0145] In the above configuration, the second search unit 412 can perform a search (relatively fine search) on pixel blocks detected by a search (relatively coarse search) by the first search unit 411. This reduces the time required for the corresponding point search (specifically, the search for the narrow range corresponding block B22) compared to when only the search by the second search unit 412 is performed (specifically, when the narrow range reference blocks B12 are selected in order from the first image P10 and the narrow range corresponding block B22 corresponding to the narrow range reference block B12 is searched for in the second image P20). This allows the distance measurement device 1 to measure the distance D0 at higher speed.

[0146] In the distance measurement device 1 of the embodiment, the first reference movement amount, which is the amount of movement of the pixel range for selecting the wide-range reference block B11, is greater than the second reference movement amount, which is the amount of movement of the pixel range for selecting the narrow-range reference block B12. This configuration reduces the time required to select the wide-range reference block B11, thereby speeding up the first search process (search by the first search unit 411). This allows the distance measurement device 1 to measure the distance D0 more quickly.

[0147] Furthermore, in the distance measurement device 1 of the embodiment, the first reference movement amount, which is the amount of movement of the pixel range for selecting the wide-range reference block BR1, is greater than the second reference movement amount, which is the amount of movement of the pixel range for selecting the narrow-range reference block BR2. This configuration can shorten the time required to select the wide-range reference block BR1, thereby speeding up the first search process (search by the first search unit 411). This allows the distance measurement device 1 to measure the distance D0 at high speed.

[0148] 11 illustrates the configuration of the distance measurement device 1 of the first modification of the embodiment. The distance measurement device 1 of the first modification of the embodiment differs from the distance measurement device 1 of the embodiment in the configuration of the projection unit 30. The other configurations and processes of the distance measurement device 1 of the first modification of the embodiment are the same as those of the distance measurement device 1 of the embodiment.

[0149] In the first modification of the embodiment, the projection unit 30 has a single light source 31. For example, the light source 31 is a white laser diode. The optical system 32 has a collimator lens 326. The collimator lens 326 converts the light emitted from the light source 31 into parallel light. The other configurations of the projection unit 30 in the first modification of the embodiment are similar to the configurations of the projection unit 30 in the embodiment.

[0150] 12A illustrates the output (spectral output) of the light source 31 in Modification 1 of the embodiment. The output of light output from the single light source 31 changes in response to a change in wavelength.

[0151] Specifically, as the wavelength of light increases from 430 nm to 470 nm, the light output gradually increases from the minimum level (zero) to the maximum level, and as the wavelength of light increases from 470 nm to 510 nm, the light output gradually decreases from the maximum level to "about 0.2 times the maximum level." Then, as the wavelength of light increases from 510 nm to 580 nm, the light output gradually increases from "about 0.2 times the maximum level" to "about 0.4 times the maximum level," and as the wavelength of light increases from 580 nm, the light output gradually decreases from "about 0.4 times the maximum level" to the minimum level.

[0152] 12(b) to 12(e) illustrate examples of the transmittance characteristics of first to fourth narrow-range filter regions 621 to 624 in Modification 1 of the embodiment. As shown in FIGS. 12(b) to 12(e), when compared for each hue, it can be considered that the transmittance levels for the transmission wavelength bands of first to fourth narrow-range filter regions 621 to 624 are set to "Levels 1 to 4," respectively, for each hue.

[0153] 12(b), the transmittance for the transmission wavelength band of the first narrow-range filter region 621 is set for each hue in accordance with the output characteristics of the single light source 31 (changes in output associated with changes in the wavelength of light emitted from the single light source 31). In the example of Fig. 12(b), the transmittance (transmittance for the transmission wavelength band) of the first narrow-range filter region 621 whose transmission wavelength band is set to a wavelength band corresponding to "red" is higher than the transmittance (transmittance for the transmission wavelength band) of the first narrow-range filter region 621 whose transmission wavelength band is set to a wavelength band corresponding to "other hues."

[0154] In this way, by setting the transmittance for the transmission wavelength band of the first narrow-range filter region 621 for each hue in accordance with the output characteristics of the single light source 31, the luminance level of the first narrow-range light region 521 generated by the first narrow-range filter region 621 can be set to the same level (level 1) for all hues. The same can be said for the second to fourth narrow-range filter regions 622 to 624.

[0155] (Second Modification of the Embodiment) The distance measurement device 1 of the second modification of the embodiment differs from the distance measurement device 1 of the embodiment in the configurations of the pattern light 50 and the filter 33. The other configurations and processes of the distance measurement device 1 of the second modification of the embodiment are the same as those of the distance measurement device 1 of the embodiment.

[0156] 13 illustrates a portion of the pattern light 50 according to the second modification of the embodiment. In the pattern light 50 according to the second modification of the embodiment, the shape of the wide-range light region 51 is different from the shape (rectangular) of the narrow-range reference block B12. The arrangement (distribution pattern) of the wide-range light region 51 in the pattern light 50 is similar to the arrangement of the wide-range light region 51 in the pattern light 50 according to the embodiment (see FIG. 6). The configuration (shape) and arrangement (distribution pattern) of the narrow-range light region 52 included in each of the plurality of wide-range light regions 51 are similar to the configuration and arrangement of the narrow-range light region 52 in the pattern light 50 according to the embodiment (see FIG. 6).

[0157] 14 illustrates a portion of the filter 33 according to the second modification of the embodiment. In the filter 33 according to the second modification of the embodiment, the shape of the wide range filter region 61 corresponds to the shape of the wide range light region 51 shown in FIG. 13 and is different from the shape (rectangular) of the narrow range reference block B12. The arrangement (distribution pattern) of the wide range filter region 61 in the filter 33 is the same as the arrangement of the wide range filter region 61 in the filter according to the embodiment (see FIG. 7). The configuration (shape) and arrangement (distribution pattern) of the narrow range filter region 62 included in each of the plurality of wide range filter regions 61 are the same as the configuration and arrangement of the narrow range filter region 62 in the filter 33 according to the embodiment (see FIG. 7).

[0158] (Third Modification of the Embodiment) The distance measurement device 1 of the third modification of the embodiment differs from the distance measurement device 1 of the embodiment in the first search process performed by the control unit 40 (first search unit 411).

[0159] In the third modification of the embodiment, the first search unit 411 performs a reduction process on the wide range reference block B11. The reduction process is a process for reducing the amount of data. The first search unit 411 also performs a reduction process on the wide range reference block BR1. The first search unit 411 then derives the similarity between the wide range reference block B11 and the wide range reference block BR1 based on the wide range reference block B11 and the wide range reference block BR1 that have been subjected to the reduction process. Examples of the reduction process include thinning and binning.

[0160] [Effects of Variation 3 of Embodiment] As described above, in the distance measurement device 1 of Variation 3 of embodiment, the first search unit 411 derives the similarity between the wide-range reference block B11 and the wide-range reference block BR1 based on the wide-range reference block B11 and the wide-range reference block BR1 that have been subjected to the reduction process.

[0161] This configuration can shorten the time required to derive the similarity between the wide-range base block B11 and the wide-range reference block BR1, thereby speeding up the first search process (search by the first search unit 411), thereby speeding up the measurement of the distance D0 by the distance measurement device 1.

[0162] (Application Example of Distance Measurement Device) The distance measurement device 1 described above is installed, for example, on an end effector (e.g., a gripper, not shown) of a robot arm performing work operations in a factory. In this case, the control unit 40 of the distance measurement device 1 receives a distance acquisition instruction from a robot controller (not shown) via the communication interface 42 during the work process of the robot arm. In response to this instruction, the control unit 40 (measurement unit 404) measures the distance between the position of the end effector and the surface of the object OB to be worked on, and transmits the measurement result (distance information) to the robot controller via the communication interface. The robot controller performs feedback control of the operation of the end effector based on the distance information received from the distance measurement device 1. Note that when the distance measurement device 1 is installed on an end effector, it is desirable that the distance measurement device 1 be small and lightweight.

[0163] (Other Embodiments) In the above description, the wide-range reference block B11 is a pixel block including 36 pixels arranged in a matrix of 6 rows and 6 columns, but this is not limiting. The wide-range reference block B11 may have other shapes and sizes. The same can be said for the wide-range reference block BR1, the narrow-range reference block B12, and the narrow-range reference block BR2.

[0164] In the above description, the case where the number of imaging units is two (the case where the first imaging unit 10 and the second imaging unit 20 are provided) has been given as an example, but the present invention is not limited to this. The distance measurement device 1 may be provided with three or more imaging units. In this case, these imaging units are arranged so that their respective fields of view overlap, and the pattern light 50 is projected into the range where these fields of view overlap.

[0165] In the above description, the case where the wide range light regions 51 included in the pattern light 50 are four types has been described as an example, but this is not limiting. The wide range light regions 51 may be two, three, five or more types. The same can be said for the types of wide range filter regions 61 included in the filter 33.

[0166] In the above description, an example has been given in which there are four types of narrow-range light regions 52 included in each of the plurality of wide-range light regions 51 in the pattern light 50, but this is not limiting. There may be two, three, five or more types of narrow-range light regions 52. The same can be said for the types of narrow-range filter regions 62 included in each of the plurality of wide-range filter regions 61 in the filter 33.

[0167] In the above description, the filter 33 is a transmission filter, but the present invention is not limited to this. For example, the filter 33 may be a reflection filter.

[0168] In the above description, the plurality of narrow-range light regions 52 included in the pattern light 50 may include narrow-range light regions 52 with zero brightness (no-light dots). The plurality of narrow-range filter regions 62 included in the filter 33 may include narrow-range filter regions 62 that block light without transmitting it (narrow-range filter regions 62 for generating no-light dots).

[0169] In the above description, the distance measurement device 1 is installed on the end effector of a robot arm, but the present invention is not limited to this. For example, the distance measurement device 1 may be applied to other systems that perform predetermined control based on the distance D0 to a measurement surface (e.g., the surface of an object OB).

[0170] Furthermore, the configuration of the distance measurement device 1 is not limited to the configuration described above. For example, the first imaging element 12 and the second imaging element 22 may be a photosensor array in which a plurality of photosensors are arranged in a matrix.

[0171] In the above description, the components of the distance measurement device 1 may be arranged together as a single device, or may be distributed across multiple devices (e.g., multiple devices communicating via a communication network such as the Internet). The control unit 40 may be realized by a single processor, or multiple processors. The control unit 40 may also be realized by multiple arithmetic processing devices (e.g., multiple arithmetic processing devices communicating via a communication network such as the Internet).

[0172] Furthermore, the above-described embodiments and modifications may be combined as appropriate. The above-described embodiments and modifications are essentially preferred examples and are not intended to limit the scope of the technology disclosed herein, its applications, or its uses.

[0173] As described above, the technology disclosed herein is useful as a distance measurement technology.

[0174] REFERENCE SIGNS LIST 1 Distance measurement device 10 First imaging unit 20 Second imaging unit 30 Projection unit 31 Light source 32 Optical system 33 Filter 40 Control unit 401 First imaging processing unit 402 Second imaging processing unit 403 Projection control unit 404 Measurement unit 411 First search unit 412 Second search unit 413 Distance derivation unit 50 Pattern light 51 Wide range light region 52 Narrow range light region 61 Wide range filter region 62 Narrow range filter region

Claims

1. a first imaging unit and a second imaging unit arranged side by side such that their respective fields of view overlap; a projection unit that projects pattern light into a range where a field of view of the first imaging unit and a field of view of the second imaging unit overlap; a measurement unit that measures a distance to a measurement surface onto which the pattern light is projected based on a parallax between a first image obtained by the first imaging unit and a second image obtained by the second imaging unit, The pattern light is composed of a plurality of light regions having different hues and a plurality of light regions having the same hue but different luminance, and the plurality of wide-range light regions are distributed in a predetermined pattern, and in each of the plurality of wide-range light regions, a plurality of narrow-range light regions are distributed in a predetermined pattern. Distance measuring device.

2. 2. The distance measuring device according to claim 1, the projection unit has a filter for generating the pattern light, the filter includes a plurality of filter regions for generating a plurality of light regions having different hues, and a plurality of filter regions for generating a plurality of light regions having the same hue but different luminance, and includes a plurality of wide range filter regions that correspond to the plurality of wide range light regions, respectively, and are distributed in the same pattern as the predetermined pattern; Each of the wide range filter regions includes a plurality of narrow range filter regions that correspond to the narrow range light regions included in the wide range light region corresponding to the wide range filter region among the wide range light regions and are distributed in the same pattern as the predetermined pattern. Distance measuring device.

3. 3. The distance measuring device according to claim 2, The projection unit includes: A light source; an optical system for directing the light emitted from the light source to the filter; Distance measuring device.

4. 4. The distance measuring device according to claim 3, Each of the plurality of filter regions for generating the plurality of light regions with different hues extracts light having a wavelength band corresponding to the plurality of light regions with different hues from the light emitted from the light source, Each of the plurality of filter regions for generating the plurality of light regions having the same hue and different luminance extracts light having a wavelength band corresponding to the plurality of light regions having the same hue and different luminance from the light emitted from the light source in an amount corresponding to the luminance. Distance measuring device.

5. 5. The distance measuring device according to claim 4, a projection control unit that adjusts the brightness of the light emitted from the light source so that the brightness of the light emitted from the light source does not become saturated; Distance measuring device.

6. 2. The distance measuring device according to claim 1, The plurality of wide range light regions include a plurality of light regions having different hues, and are distributed in a predetermined hue pattern; The plurality of narrow-range light regions includes a plurality of light regions having the same hue and different luminance, and are distributed in a predetermined luminance pattern. Distance measuring device.

7. In the distance measuring device according to any one of claims 1 to 6, The measurement unit is a first search unit that sequentially selects wide range reference blocks from the first image and searches for wide range corresponding blocks corresponding to the wide range reference blocks from the second image; a second search unit that sequentially selects narrow range reference blocks from the wide range reference blocks and searches for a narrow range corresponding block corresponding to the narrow range reference block from among the wide range corresponding blocks corresponding to the wide range reference block; a distance deriving unit that derives a distance to the measurement surface in the narrow range reference block based on a difference in position between the narrow range reference block and the narrow range corresponding block. Distance measuring device.

8. 8. The distance measuring device according to claim 7, the first search unit sequentially selects the wide range template blocks from the first image by shifting a pixel range for selecting the wide range template blocks in the first image by a first reference shift amount; the second search unit sequentially selects the narrow range template blocks from the wide range template blocks by shifting a pixel range for selecting the narrow range template blocks by a second reference shift amount in the wide range template blocks; The first reference movement amount is greater than the second reference movement amount. Distance measuring device.

9. 8. The distance measuring device according to claim 7, the first search unit sequentially selects the wide range reference blocks from the second image by shifting a pixel range for selecting a wide range reference block that is a candidate for the wide range corresponding block by a first reference shift amount in the second image; the second search unit sequentially selects the narrow range reference block from the wide range corresponding blocks by shifting a pixel range for selecting a narrow range reference block that is a candidate for the narrow range corresponding block by a second reference shift amount in the wide range corresponding block; The first reference movement amount is greater than the second reference movement amount. Distance measuring device.

10. 8. The distance measuring device according to claim 7, The first search unit is Selecting wide range reference blocks from the second image in order as candidates for the wide range corresponding block corresponding to the wide range reference block, and determining the wide range reference block having the greatest similarity to the wide range reference block among the wide range reference blocks selected in order from the second image as the wide range corresponding block; A reduction process is performed on the wide range reference block and the wide range reference block to reduce the amount of data, and a similarity between the wide range reference block and the wide range reference block is derived based on the wide range reference block and the wide range reference block that have been subjected to the reduction process. Distance measuring device.

11. A distance measurement method using a first imaging unit and a second imaging unit arranged side by side such that their respective fields of view overlap, and a projection unit that projects pattern light into an area where the fields of view of the first imaging unit and the fields of view of the second imaging unit overlap, a projection step of projecting the pattern light from the projection unit; an acquisition step of acquiring a first image acquired by the first imaging unit and a second image acquired by the second imaging unit; a measuring step of measuring a distance to a measurement surface onto which the pattern light is projected, based on a parallax between the first image and the second image; The pattern light is composed of a plurality of light regions having different hues and a plurality of light regions having the same hue but different luminance, and the plurality of wide-range light regions are distributed in a predetermined pattern, and in each of the plurality of wide-range light regions, a plurality of narrow-range light regions are distributed in a predetermined pattern. Distance measurement method.

12. A distance measuring program that causes a computer to execute the distance measuring method according to claim 11.