Distance measurement device, distance measurement method, and distance measurement program

Abstract

The projection unit (30) projects pattern light (50) into a range in which the field of view of the first imaging unit (10) and the field of view of the second imaging unit (20) overlap. The measurement unit (404) measures the distance to the measurement surface on which the pattern light (50) has been projected on the basis of the parallax between the first image obtained by the first imaging unit (10) and the second image obtained by the second imaging unit (20). The patterned light (50) includes a plurality of light regions having different hues and a plurality of light regions having the same hue but different brightness, in which a plurality of large-range light regions (51) are distributed in a prescribed pattern, and in each of the plurality of large-range light regions (51), a small-range light region (52) is distributed in a prescribed pattern.

Description

Technical Field

[0001] The technology disclosed here relates to a distance measurement technology. Background Art

[0002] Patent document 1 discloses a three-dimensional measurement system. In the system, a camera unit has a first camera unit and a second camera unit that are separated from each other. A first calculation unit uses at least one of the images of the object captured by the first camera unit and the second camera unit, and uses distance information of a three-dimensional measurement method different from a stereo camera method to calculate the parallax of a first feature point. A second calculation unit uses the images of the object captured by the first camera unit and the second camera unit, and uses a stereo camera method to calculate the parallax of a second feature point. Then, the second calculation unit determines the three-dimensional shape of the object based on the parallax of the first feature point and the parallax of the second feature point.

[0003] Patent Document 1: Japanese Patent Publication No. 2021-192064 Summary of the invention

[0004] 1. Technical problem to be solved by the invention

[0005] In the system of Patent Document 1, when the measurement surface to be measured includes a plain surface without texture, it is difficult to perform stereo matching (corresponding point search) on such a plain surface. Therefore, it is difficult to accurately measure the distance to the measurement surface.

[0006] 1. Technical solutions for solving technical problems 1

[0007] The technology disclosed herein relates to a distance measuring device, which includes a first camera unit and a second camera unit, a projection unit and a measuring unit, wherein the first camera unit and the second camera unit are arranged so that their fields of view overlap with each other, the projection unit projects pattern light to a range where the field of view of the first camera unit overlaps with the field of view of the second camera unit, and the measuring unit measures the distance to a measurement surface onto which the pattern light has been projected based on a parallax between a first image obtained by the first camera unit and a second image obtained by the second camera unit, the pattern light being pattern light including a plurality of light regions with different hues and a plurality of light regions with the same hue but different brightness, wherein a plurality of large-range light regions are distributed in a prescribed pattern, and in each of the plurality of large-range light regions, a plurality of small-range light regions are distributed in a prescribed pattern.

[0008] The technology disclosed herein relates to a distance measurement method for measuring using a first camera unit, a second camera unit, and a projection unit, wherein the first camera unit and the second camera unit are arranged so that their fields of view overlap with each other, and the projection unit projects pattern light to a range where the field of view of the first camera unit overlaps with the field of view of the second camera unit. The distance measurement method includes: a projection step of projecting the pattern light from the projection unit, an acquisition step of acquiring a first image obtained by the first camera unit and a second image obtained by the second camera unit, and a measurement step of measuring the distance to a measurement surface onto which the pattern light has been projected based on a parallax between the first image and the second image, wherein the pattern light is a pattern light including a plurality of light areas with different hues and a plurality of light areas with the same hue but different brightness, wherein a plurality of large-range light areas are distributed in a prescribed pattern, and in each of the plurality of large-range light areas, a plurality of small-range light areas are distributed in a prescribed pattern.

[0009] The technology disclosed here relates to a distance measurement program that causes a computer to execute the above-mentioned distance measurement method.

[0010] Effect of the invention

[0011] According to the technology disclosed herein, even when the measurement surface includes a solid-color surface, the distance to the measurement surface can be accurately measured. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram showing a schematic structure of a distance measuring device in an embodiment;

[0013] Figure 2 is a block diagram showing the structure of a distance measuring device in an embodiment;

[0014] Figure 3 is a block diagram showing the functional structure of the control unit;

[0015] Figure 4 is a simplified diagram for explaining the first search process;

[0016] Figure 5 is a simplified diagram for explaining the second search process;

[0017] Figure 6 is a simplified diagram showing the projected pattern light;

[0018] Figure 7 is a simplified diagram showing the structure of a filter;

[0019] Figure 8 is a graph for explaining the output characteristics of the light source and the transmittance characteristics of the filter region in the embodiment;

[0020] Fig. 9 is a flow chart showing a brightness adjustment process;

[0021] Fig.10 is a flow chart showing a distance measurement process;

[0022] Fig.11 is a block diagram showing the structure of a distance measuring device in a first variation of the embodiment;

[0023] Fig.12 is a graph for explaining the output characteristics of the light source and the transmittance characteristics of the filter region in Modification 1 of the embodiment;

[0024] Fig.13 is a schematic diagram showing a portion of pattern light in Modification 2 of the embodiment;

[0025] Fig.14 This is a schematic diagram showing a part of the optical filter in Modification 2 of the embodiment. DETAILED DESCRIPTION

[0026] The following is a detailed description of the embodiments with reference to the accompanying drawings. It should be noted that the same or corresponding parts are represented by the same symbols in the accompanying drawings, and the description thereof will not be repeated.

[0027] (Distance measuring device)

[0028] Figure 1 and Figure 2 The structure of the distance measuring device 1 of the embodiment is shown. The distance measuring device 1 includes a first imaging unit 10, a second imaging unit 20, a projection unit 30, a control unit 40, a storage unit 41, and a communication interface unit 42. The distance measuring device 1 measures the distance D0 to the measurement surface. In this example, the measurement surface is the surface of the object 0B (the surface opposite to the first imaging unit 10 and the second imaging unit 20).

[0029] In the following description, a direction perpendicular to the X-axis direction is referred to as a “Y-axis direction”, and a direction perpendicular to both the X-axis direction and the Y-axis direction is referred to as a “Z-axis direction”.

[0030] 〔Camera Department〕

[0031] The first camera unit 10 captures the range of the first field of view 10a. The second camera unit 20 captures the range of the second field of view 20a. The first camera unit 10 and the second camera unit 20 are arranged so that their fields of view overlap. In this example, the first field of view 10a and the second field of view 20a are oriented in the Z-axis direction, and the direction in which the first camera unit 10 and the second camera unit 20 are arranged is the X-axis direction.

[0032] The range where the first field of view 10a of the first camera unit 10 overlaps with the second field of view 20a of the second camera unit 20 includes the measurement surface (the surface of the object 0B in this example). The first camera unit 10 and the second camera unit 20 are so-called stereo cameras, and they simultaneously capture the range of the field of view (the captured range) from different viewpoints.

[0033] The first imaging unit 10 acquires a first image P10 by imaging the range of the first field of view 10 a at predetermined intervals. The first imaging unit 10 includes a first imaging lens 11 and a first imaging element 12 .

[0034] The first imaging lens 11 focuses light from the field of view 10a of the first imaging unit 10 on a first imaging surface 12a of the first imaging element 12. The first imaging lens 11 may be a single lens having a predetermined focal length or a combination of a plurality of lenses.

[0035] The first imaging element 12 converts the light given to the first imaging surface 12a into an electrical signal. In this way, a first image P10 can be obtained. The first imaging element 12 can be a black and white image sensor. For example, the first imaging element 12 can be a CMOS image sensor, a CCD image sensor, etc.

[0036] The second imaging unit 20 acquires the second image P20 by capturing the range of the second field of view 20a at predetermined intervals. The structure of the second imaging unit 20 is the same as that of the first imaging unit 10. The second imaging unit 20 includes a second imaging lens 21 and a second imaging element 22. The structures of the second imaging lens 21 and the second imaging element 22 are the same as those of the first imaging lens 11 and the first imaging element 12, and the second imaging lens 21 has the same focal length as that of the first imaging lens 11. The second imaging element 22 has a second imaging surface 22a.

[0037] The size of the second image P20 is equal to that of the first image P10. The size of the pixels of the second image P20 is equal to that of the first image P10. The number of pixels included in the second image P20 is equal to the number of pixels included in the first image P10.

[0038] It should be noted that the shooting direction of the first camera unit 10 may be slightly tilted from the Z-axis direction toward the direction of the second camera unit 20 (towards the second camera unit 20). The shooting direction of the second camera unit 20 may be slightly tilted from the Z-axis direction toward the direction of the first camera unit 10 (towards the first camera unit 10). The position of the first camera unit 10 in the Z-axis direction and the position of the first camera unit 10 in the Y-axis direction are the same as the position of the second camera unit 20 in the Z-axis direction and the position of the second camera unit 20 in the Y-axis direction.

[0039] 〔Projection Department〕

[0040] The projection unit 30 projects the pattern light 50 to the range where the first field of view 10a of the first imaging unit 10 overlaps with the second field of view 20a of the second imaging unit 20. In this example, the projection direction of the pattern light 50 projected by the projection unit 30 is the Z-axis direction. The pattern light 50 is projected onto the surface (an example of a measurement surface) of the object 0B included in the range where the first field of view 10a of the first imaging unit 10 overlaps with the second field of view 20a of the second imaging unit 20. It should be noted that the pattern light 50 will be described in detail later.

[0041] The projection unit 30 includes a light source 31 , an optical system 32 , a filter 33 , a projection lens 34 , and a light source driving unit 35 .

[0042] The light source 31 emits light for generating the pattern light 50. In this example, the light source 31 includes first to third light sources 311 to 313. For example, the first light source 311 emits light in a wavelength range (590 to 640 nm) corresponding to "red". The second light source 312 emits light in a wavelength range (490 to 550 nm) corresponding to "green". The third light source 313 emits light in a wavelength range (430 to 490 nm) corresponding to "blue". The first to third light sources 311 to 313 can be light emitting diodes or other types of light sources such as semiconductor lasers.

[0043] The optical system 32 guides the light emitted from the light source 31 toward the filter 33. In this example, the optical system 32 includes first to third collimator lenses 321 to 323, a first dichroic mirror 324, and a second dichroic mirror 325.

[0044] The first to third collimating lenses 321 to 323 convert the light emitted from the first to third light sources 311 to 313 into approximately parallel light, respectively. The first dichroic mirror 324 transmits the light incident from the first collimating lens 321 and reflects the light incident from the second collimating lens 322. The second dichroic mirror 325 transmits the light incident from the first dichroic mirror 324 and reflects the light incident from the third collimating lens 323. With such a structure, the light emitted from each of the first to third light sources 311 to 313 is combined and guided toward the filter 33.

[0045] The filter 33 generates the pattern light 50. The structure of the filter 33 will be described in detail later.

[0046] The projection lens 34 projects the pattern light 50 generated by the filter 33. The projection lens 34 may be a single lens or a combination of a plurality of lenses.

[0047] The light source driving unit 35 drives the light source 31 (the first to third light sources 311 to 313 in this example) in response to the control performed by the control unit 40. Specifically, the light source driving unit 35 drives the light source 31 according to the driving current value set by the control unit 40 so as to emit light with brightness corresponding to the driving current value.

[0048] 〔Control Department〕

[0049] The control unit 40 performs various processes. Specifically, the control unit 40 obtains information and data from various components of the distance measurement device 1, and performs various processes based on the information and data. The processes performed by the control unit 40 will be described in detail later.

[0050] For example, the control unit 40 includes a processor and a memory (storage medium) storing a program for operating the processor. The various functions of the control unit 40 can be realized by executing the program by the processor. In other words, the control unit 40 has various functional blocks for realizing various functions. It should be noted that the control unit 40 is an example of a computer. The program is an example of a distance measurement program.

[0051] It should be noted that the control unit 40 and the communication interface unit 42 may be formed by a semiconductor integrated circuit, and the semiconductor integrated circuit may be formed by an FPGA (Field Programmable Gate Array). Alternatively, the control unit 40 and the communication interface unit 42 may also be formed by other semiconductor integrated circuits, such as a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), or an ASIC (Application Specific Integrated Circuit).

[0052] 〔Storage Department〕

[0053] The storage unit 41 stores various information and data. In this example, the storage unit 41 stores a first image P10 obtained by the first imaging unit 10 and a second image P20 obtained by the second imaging unit 20.

[0054] [Functional structure of the control unit]

[0055] Figure 3 2 shows the functional configuration of the control unit 40 . In this example, the control unit 40 includes a first image processing unit 401 , a second image processing unit 402 , a projection control unit 403 , and a measurement unit 404 .

[0056] <Image Processing Unit)

[0057] The first imaging processing unit 401 controls the first imaging element 12 of the first imaging unit 10. In addition, the first imaging processing unit 401 performs pre-processing such as brightness correction and camera calibration on the first image P10 (pixel signal) obtained by the first imaging element 12 of the first imaging unit 10. In this example, the first image P10 processed by the first imaging processing unit 401 is stored in the storage unit 41.

[0058] The second image processing unit 402 controls the second image sensor 22 of the second image sensor 20. In addition, the second image processing unit 402 performs pre-processing such as brightness correction and camera calibration on the second image P20 (pixel signal) obtained by the second image sensor 22 of the second image sensor 20. In this example, the second image P20 processed by the second image processing unit 402 is stored in the storage unit 41.

[0059] <Projection Control Unit)

[0060] The projection control unit 403 controls the projection unit 30 . Specifically, the projection control unit 403 controls the projection unit 30 to cause the projection unit 30 to project the pattern light 50 .

[0061] In this example, the projection control unit 403 performs a brightness adjustment process. In the brightness adjustment process, the projection control unit 403 adjusts the brightness of the light emitted from the light source 31 so that the brightness of the light emitted from the light source 31 (in this example, each of the first to third light sources 311 to 313) is not saturated. The brightness adjustment process will be described in detail later.

[0062] <Measurement Department>

[0063] The measuring unit 404 measures the distance D0 to the measurement surface onto which the pattern light 50 is projected based on the parallax between the first image P10 obtained by the first imaging unit 10 and the second image P20 obtained by the second imaging unit 20. In this example, the measuring unit 404 includes a first search unit 411, a second search unit 412, and a distance derivation unit 413.

[0064] 《First Search Section (First Search Process)》

[0065] The first search unit 411 performs a first search process. In the first search process, the first search unit 411 sequentially selects a large-range reference block B11 from the first image P10, and searches for a large-range corresponding block B21 corresponding to the large-range reference block B11 from the second image P20. Specifically, the first search unit 411 performs the following process in the first search process.

[0066] like Figure 4As shown, the first search unit 411 sequentially selects the large-scale reference block B11 from the first image P10 by moving the pixel range used to select the large-scale reference block B11 in the first image P10 by a predetermined amount (first reference movement amount) each time. The large-scale reference block B11 is a pixel block (a collection of multiple pixels) that becomes a reference point for the corresponding point search in the first search process. Figure 4 In the example of , the large-area reference block B11 includes 36 pixels arranged in a matrix of 6 rows and 6 columns. The first search unit 411 performs the following processing on each of the large-area reference blocks B11 sequentially selected from the first image P10.

[0067] The first search unit 411 sequentially selects the large range reference block BR1 from the second image P20 by moving the pixel range of the large range reference block BR1 used to select the large range reference block BR1 for comparison with the large range reference block B11 in the second image P20 by a predetermined amount (first reference movement amount) at a time. The large range reference block BR1 is a pixel block that is a candidate for the large range corresponding block B21 corresponding to the large range reference block B11. The shape of the large range reference block BR1 is the same as that of the large range reference block B11, and the size of the large range reference block BR1 is equal to that of the large range reference block B11.

[0068] The first search unit 411 derives the similarity between each of the large range reference blocks BR1 selected sequentially from the second image P20 and the large range reference block B11. Then, the first search unit 411 determines “the large range reference block BR1 having the greatest similarity to the large range reference block B11” among the large range reference blocks BR1 selected sequentially from the second image P20 as the large range corresponding block B21 corresponding to the large range reference block B11.

[0069] It should be noted that in the process of deriving the similarity, a known similarity derivation process (similarity calculation method) can be adopted. Examples of similarity calculation methods include ZNCC (Zero means Normalized Cross-Correlation), NCC (Normalized Cross Correlation), SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), etc.

[0070] As described above, by performing the first search process on both the first image P10 and the second image P20, a plurality of combinations of the large-range reference blocks B11 and the large-range corresponding blocks B21 (hereinafter referred to as "large-range block combinations") can be obtained.

[0071] It should be noted that, in this example, the first search unit 411 sequentially selects the large-scale reference block BR1 in the second image P20 and in the following search range R20, which starts from the same position as the "position of the large-scale reference block B11 in the first image P10" and extends in a direction (in this example, the left-right direction) corresponding to the "direction in which the first camera unit 10 and the second camera unit 20 are separated from each other (X-axis direction)". The direction in which the search range R20 extends is set to the direction in which the pixel block located at the above starting point deviates from the above starting point due to parallax.

[0072] The starting point of the search range R20 is not limited to the same position (reference position) as the "position of the large-range reference block B11 in the first image P10" in the second image P20. For example, the starting point of the search range R20 may be set to a position in the second image P20 that is offset from the reference position to the right (in the direction offset due to parallax) by a predetermined amount (e.g., several blocks).

[0073] Second search unit (second search process)

[0074] The second search unit 412 performs a second search process. In the second search process, the second search unit 412 sequentially selects the small-range reference block B12 from the large-range reference block B11, and searches for the small-range corresponding block B22 corresponding to the small-range reference block B12 from the large-range corresponding block B21 corresponding to the large-range reference block B11. Specifically, the second search unit 412 performs the following process in the second search process.

[0075] The second search section 412 performs the following processing on each of the large range reference blocks B11 sequentially selected from the first image P10 by the first search section 411 and the large range corresponding block B21 corresponding to the large range reference block B11 .

[0076] like Figure 5 As shown, the second search unit 412 sequentially selects small-range reference blocks B12 from the large-range reference block B11 by moving the pixel range for selecting the small-range reference block B12 in the large-range reference block B11 by a predetermined amount (second reference movement amount) each time. The small-range reference block B12 is a pixel block that becomes a reference point for the corresponding point search in the second search process. The small-range reference block B12 is smaller than the large-range reference block B11. Figure 5 In the example of , the small range reference block B12 includes 9 pixels arranged in a matrix of 3 rows and 3 columns. Then, the second search unit 412 performs the following processing on each of the small range reference blocks B12 sequentially selected from the large range reference block B11.

[0077] The second search unit 412 sequentially selects small range reference blocks BR2 from the large range corresponding block B21 by moving the pixel range for selecting the small range reference block BR2 for comparison with the small range reference block B12 in the large range corresponding block B21 corresponding to the large range reference block B11 by a predetermined amount (second reference movement amount). The small range reference block BR2 is a pixel block that is a candidate for the small range corresponding block B22 corresponding to the small range reference block B12.

[0078] The second search unit 412 derives the similarity between each of the small range reference blocks BR2 selected sequentially from the large range corresponding block B21 and the small range reference block B12. Then, the second search unit 412 determines "the small range reference block BR2 having the greatest similarity to the small range reference block B12" among the small range reference blocks BR2 selected sequentially from the large range corresponding block B21 as the small range corresponding block B22 corresponding to the small range reference block B12.

[0079] As described above, by performing a second search process on each of multiple large-range block combinations (combinations of large-range reference blocks B11 and large-range corresponding blocks B21), multiple groups of combinations of small-range reference blocks B12 and small-range corresponding blocks B22 (hereinafter referred to as "small-range block combinations") can be obtained.

[0080] 《Distance Derivation Department (Distance Derivation Processing)》

[0081] The distance derivation unit 413 performs a distance derivation process. In the distance derivation process, the distance derivation unit 413 derivates the distance to the measurement surface corresponding to the small range reference block B12 based on the parallax between the small range reference block B12 and the small range corresponding block B22. Specifically, the distance derivation unit 413 performs the following process in the distance derivation process.

[0082] The distance derivation section 413 performs the following processing on each of the small-range reference blocks B12 sequentially selected from the first image P10 by the second search section 412 .

[0083] The distance derivation unit 413 selects a small range corresponding block B22 (the small range corresponding block B22 detected by the second search unit 412) corresponding to the small range reference block B12 from the second image P20, and derives the position difference (pixel offset) between the small range reference block B12 and the small range corresponding block B22.

[0084] Then, the distance derivation unit 413 derivates the distance D0 (the distance D0 to the measurement surface) corresponding to the small-range reference block B12 based on the derived position difference. Specifically, the distance derivation unit 413 uses triangulation to derive the distance D0 to the measurement surface from the derived position difference (pixel offset), the spacing distance between the first camera unit 10 and the second camera unit 20, and the focal length of the first camera lens 11. It should be noted that the focal length of the second camera lens 21 is equal to the focal length of the first camera lens 11.

[0085] Through the above processing, the distance D0 (the distance D0 to the measurement surface) corresponding to each of the small range reference blocks B12 sequentially selected from the first image P10 by the second search unit 412 can be obtained. The distance derivation unit 413 outputs distance information indicating the distance D0 corresponding to each of the small range reference blocks B12 sequentially selected from the first image P10. For example, the distance derivation unit 413 sends the distance information to an external device via the communication interface unit 42.

[0086] Search Accuracy

[0087] It should be noted that, in this example, the search accuracy of the first search unit 411 is lower than the search accuracy of the second search unit 412 .

[0088] Specifically, the movement amount of the pixel range for selecting the large-range reference block B11, i.e., the first reference movement amount, is greater than the movement amount of the pixel range for selecting the small-range reference block B12, i.e., the second reference movement amount. Specifically, the first reference movement amount is set to n pixels (n is an integer greater than 2), and the second reference movement amount is set to m pixels (m is an integer greater than 1 and less than n). It should be noted that the second reference movement amount is preferably one pixel.

[0089] In addition, the first reference movement amount, which is the movement amount of the pixel range used to select the large-range reference block BR1, is greater than the second reference movement amount, which is the movement amount of the pixel range used to select the small-range reference block BR2. Specifically, the first reference movement amount is set to j pixels (j is an integer greater than 2), and the second reference movement amount is set to k pixels (k is an integer greater than 1 and less than j). It should be noted that the second reference movement amount is preferably one pixel.

[0090] 〔Pattern Light〕

[0091] Next, refer to Figure 6 The pattern light 50 will be described. Figure 6 Pattern light 50 is shown which has been projected onto the measurement surface.

[0092] <Large light area>

[0093] The pattern light 50 includes a plurality of large-scale light regions 51. Figure 6 In the example of , a plurality of wide-range light regions 51 are arranged in a matrix. Specifically, in the pattern light 50 , 35 wide-range light regions 51 are arranged in a matrix of 5 rows and 7 columns.

[0094] The plurality of wide-range light regions 51 are divided into a plurality of types according to hue. In other words, the plurality of wide-range light regions 51 include a plurality of (two or more) wide-range light regions 51 with different hues. In the patterned light 50, the plurality of wide-range light regions 51 (a plurality of wide-range light regions 51 with different hues) are distributed in a predetermined hue pattern. In this example, the hue pattern is a random pattern.

[0095] exist Figure 6 In the example of , the plurality of wide-range light regions 51 are divided into four types of wide-range light regions 51 (specifically, first to fourth wide-range light regions 511 to 514). The hues (in other words, wavelength ranges of light) of the first to fourth wide-range light regions 511 to 514 are different from each other.

[0096] The hue of the first large-range light area 511 is "red". The hue of the second large-range light area 512 is "orange". The hue of the third large-range light area 513 is "green". The hue of the fourth large-range light area 514 is "blue". In other words, the wavelength range of the light in the first large-range light area 511 is a wavelength range corresponding to "red" (for example, 640 to 770 nm). The wavelength range of the light in the second large-range light area 512 is a wavelength range corresponding to "orange" (for example, 590 to 640 nm). The wavelength range of the light in the third large-range light area 513 is a wavelength range corresponding to "green" (for example, 490 to 550 nm). The wavelength range of the light in the fourth large-range light area 514 is a wavelength range corresponding to "blue" (for example, 430 to 490 nm).

[0097] It should be noted that in Figure 6 In the example of , a letter (R, 0, G, B) indicating the hue is marked for each of the wide-range light regions 51. "R" indicates "red", "0" indicates "orange", "G" indicates "green", and "B" indicates "blue". For example, the hue of the wide-range light region 51 marked with "R" is "red".

[0098] <Small light area>

[0099] Each of the plurality of large-range light regions 51 includes a plurality of small-range light regions 52. Figure 6In the example of , a plurality of small-range light regions 52 are arranged in a matrix. Specifically, in each of the 35 large-range light regions 51, 9 small-range light regions 52 are arranged in a matrix of 3 rows and 3 columns.

[0100] The plurality of small-range light regions 52 are divided into a plurality of types according to hue and brightness. In other words, the plurality of small-range light regions 52 included in each of the plurality of large-range light regions 51 include a plurality of (two or more) small-range light regions 52 having the same hue but different brightness. In each of the plurality of large-range light regions 51, a plurality of small-range light regions 52 (a plurality of small-range light regions 52 having the same hue but different brightness) are distributed in a predetermined brightness pattern. In this example, the brightness pattern is a random pattern.

[0101] exist Figure 6 In the example of , the multiple small-range light regions 52 included in each of the multiple large-range light regions 51 are divided into four types of small-range light regions 52 (specifically, the first small-range light region to the fourth small-range light region 521 to 524). The brightness of the first small-range light region to the fourth small-range light region 521 to 524 is different from each other.

[0102] The brightness of the first small range light area 521 is "level 1". The brightness of the second small range light area 522 is "level 2". The brightness of the third small range light area 523 is "level 3". The brightness of the fourth small range light area 524 is "level 4". It should be noted that as the brightness level goes from "level 1" to "level 4", the brightness gradually increases.

[0103] It should be noted that in Figure 6 In the example of FIG. 5 , a letter indicating the hue (R, 0, G, B) and a number indicating the brightness level (1, 2, 3, 4) are marked for each of the small-range light regions 52. For example, the hue of the small-range light region 52 marked with “R1” is “red” and the brightness is “level 1”.

[0104] 〔Relationship between pattern light and image〕

[0105] The pattern light 50 is projected onto the measurement surface (in this example, the surface of the object 0B), and the first camera unit 10 captures the range of the first field of view 10a including the measurement surface onto which the pattern light 50 is projected, thereby obtaining a first image P10 including the pattern light 50 projected onto the measurement surface. In addition, the second camera unit 20 captures an image while the first camera unit 10 captures an image, thereby obtaining a second image P20 including the pattern light 50 projected onto the measurement surface.

[0106] In this example, when the distance D0 to the measurement surface is the reference distance (for example, the middle distance of the distance measurement range), in the pattern light 50 included in the first image P10, the shape of the small-range light region 52 (spot light) is a shape corresponding to the shape of one pixel of the first image P10 (for example, the same shape as the shape of one pixel). It should be noted that the distance measurement range is a distance range that can be measured by the distance measurement device 1. In addition, the size of the small-range light region 52 (spot light) is a size corresponding to the size of one pixel of the first image P10 (for example, the same size as the size of one pixel).

[0107] It should be noted that the shape and size of the small-range light area 52 in the first image P10 are not limited to the above-mentioned shape and size. For example, the shape of the small-range light area 52 in the first image P10 may be a shape different from the shape (rectangle) of one pixel of the first image P10. In addition, the size of the small-range light area 52 in the first image P10 may be a size corresponding to the size of two or more (preferably 2 to 4) pixels in the first image P10. In other words, in the first image P10, one small-range light area 52 may also be included in more than two pixels of the first image P10. Alternatively, the size of the small-range light area 52 in the first image P10 may also be a size smaller than the size of one pixel of the first image P10.

[0108] In this example, when the distance D0 to the measurement surface is the reference distance, in the pattern light included in the first image P10, the shape of the wide-range light region 51 (the collection of point lights) is a shape corresponding to the shape of the small-range reference block B12 (for example, the same shape as the small-range reference block B12). In addition, the size of the wide-range light region 51 (the collection of point lights) is a size corresponding to the size of the small-range reference block B12 (for example, a size equal to the size of the small-range reference block B12).

[0109] It should be noted that the shape and size of the wide-range light region 51 in the first image P10 are not limited to the above-mentioned shape and size. For example, the shape of the wide-range light region 51 in the first image P10 may be a shape different from the shape (rectangular) of the small-range reference block B12. In addition, the size of the wide-range light region 51 in the first image P10 may be a size larger than the size of the small-range reference block B12, or a size smaller than the size of the small-range reference block B12.

[0110] In addition, in this example, the hue pattern of the pattern light 50 contained in the first image P10 (the distribution pattern of multiple large-range light areas 51) is set as follows: under the condition that the distance D0 to the measuring surface is the reference distance, each of the large-range reference blocks B11 selected sequentially from the first image P10 in the first search process includes multiple large-range light areas 51 (multiple large-range light areas 51 with different hues).

[0111] It should be noted that, preferably, the hue pattern of the pattern light 50 contained in the first image P10 is set as follows: under the above conditions, the distribution patterns of the multiple wide-range light areas 51 contained in each of the wide-range reference blocks B11 selected sequentially from the first image P10 in the first search process are different from each other.

[0112] In addition, in this example, the brightness pattern of each of the multiple large-range light areas 51 contained in the first image P10 (the distribution pattern of the multiple small-range light areas 52) is set as follows: under the condition that the distance D0 to the measuring surface is the reference distance, each of the small-range reference blocks B12 selected in sequence from the large-range reference block B11 in the second search process contains multiple small-range light areas 52 (multiple small-range light areas 52 with the same hue but different brightness).

[0113] It should be noted that, preferably, the brightness pattern of each of the multiple large-range light areas 51 contained in the first image P10 is set as follows: under the above conditions, the distribution patterns of the multiple small-range light areas 52 contained in each of the small-range reference blocks B12 selected sequentially from the large-range reference block B11 in the second search process are different from each other.

[0114] 〔Filter〕

[0115] Next, refer to Figure 7 The structure of the filter 33 will be described. Figure 7 The optical filter 33 is shown as viewed from the light incident surface side. In this example, the optical filter 33 is a transmission type filter.

[0116] <Wide filter area>

[0117] The filter 33 includes a plurality of wide-range filter regions 61. The plurality of wide-range filter regions 61 correspond to the plurality of wide-range light regions 51, respectively, and generate corresponding wide-range light regions 51. Figure 7 In the example of , a plurality of wide-range filter regions 61 are arranged in a matrix. Specifically, in the filter 33, 35 wide-range filter regions 61 are arranged in a matrix of 5 rows and 7 columns.

[0118] The plurality of wide range filter regions 61 are divided into a plurality of types according to the hue of the wide range light region 51 to be generated. In other words, the plurality of wide range filter regions 61 include a plurality (two or more) of wide range filter regions 61 having different hues of the wide range light region 51 to be generated.

[0119] In the pattern light 50 , multiple types of wide range filter regions 61 (multiple wide range filter regions 61 with different hues to be generated for the wide range light regions 51 ) respectively correspond to the multiple types of wide range light regions 51 and are distributed in the same pattern as the hue pattern.

[0120] Each of the plurality of wide range filter regions 61 extracts light of a wavelength range corresponding to the color of the wide range light region 51 corresponding to the wide range filter region 61 , from light emitted from the light source 31 .

[0121] In this example, each of the plurality of wide range filter regions 61 transmits light of a wavelength range corresponding to the color of the wide range light region 51 corresponding to the wide range filter region 61, among the light emitted from the light source 31. Specifically, the transmission wavelength range (wavelength range of light that can be transmitted) of each of the plurality of wide range filter regions 61 is set to the wavelength range corresponding to the color of the wide range light region 51 corresponding to the wide range filter region 61. Each of the plurality of wide range filter regions 61 has a high transmittance for light of the transmission wavelength range (that is, the wavelength range corresponding to the color of the wide range light region 51 corresponding to the wide range filter region 61), and has a low transmittance for light of other wavelength ranges.

[0122] exist Figure 7 In the example of , the plurality of wide-range filter regions 61 are divided into four types of wide-range filter regions 61 (specifically, the first wide-range filter region to the fourth wide-range filter region 611 to 614), and the four types of wide-range filter regions 61 respectively correspond to the four types of wide-range light regions 51. The transmission wavelength ranges of the first wide-range filter region to the fourth wide-range filter region 611 to 614 are different from each other.

[0123] The transmission wavelength range of the first wide range filter area 611 is set to a wavelength range corresponding to the hue of the first wide range light area 511, that is, "red". The transmission wavelength range of the second wide range filter area 612 is set to a wavelength range corresponding to the hue of the second wide range light area 512, that is, "orange". The transmission wavelength range of the third wide range filter area 613 is set to a wavelength range corresponding to the hue of the third wide range light area 513, that is, "green". The transmission wavelength range of the fourth wide range filter area 614 is set to a wavelength range corresponding to the hue of the fourth wide range light area 514, that is, "blue".

[0124] It should be noted that in Figure 7 In the example of , each of the wide-range filter areas 61 is marked with a letter (r, o, g, b) indicating the hue corresponding to the transmission wavelength range of the wide-range filter area 61. "r" indicates that the transmission wavelength range is a wavelength range corresponding to "red". "o" indicates that the transmission wavelength range is a wavelength range corresponding to "orange". "g" indicates that the transmission wavelength range is a wavelength range corresponding to "green". "b" indicates that the transmission wavelength range is a wavelength range corresponding to "blue". For example, the transmission wavelength range of the wide-range filter area 61 marked with "r" is set to a wavelength range corresponding to "red".

[0125] <Small filter area>

[0126] Each of the plurality of large-range filter regions 61 includes a plurality of small-range filter regions 62. The plurality of small-range filter regions 62 correspond to the plurality of small-range light regions 52, respectively, and generate corresponding small-range light regions 52. Figure 7 In the example of , a plurality of small-range filter regions 62 are arranged in a matrix. Specifically, in each of the 35 large-range filter regions 61, 9 small-range filter regions 62 are arranged in a matrix of 3 rows and 3 columns.

[0127] The multiple small-range filter areas 62 included in each of the multiple large-range filter areas 61 are divided into multiple types according to the brightness of the small-range light area 52 to be generated. In other words, the multiple small-range filter areas 62 included in each of the multiple large-range filter areas 61 include multiple (two or more) small-range filter areas 62 having the same hue as the small-range light area 52 to be generated but different brightness.

[0128] In each of the multiple large-range filter areas 61, a plurality of small-range filter areas 62 (a plurality of small-range light areas 62 having the same hue but different brightness as the small-range light area 52 to be generated) respectively correspond to the plurality of small-range light areas 52, and the plurality of small-range light areas 52 are included in the large-range light area 51 corresponding to the large-range filter area 61, and the plurality of small-range filter areas 62 are distributed in a pattern identical to the brightness pattern of the large-range light area 51.

[0129] Each of the multiple small-range filter areas 62 extracts light in a wavelength range corresponding to the color of the large-range light area 51 corresponding to the large-range filter area 61 including the small-range filter area 62 from the light emitted from the light source 31, and the extraction amount corresponds to the brightness of the small-range light area 52 corresponding to the small-range filter area 62.

[0130] In this example, each of the multiple small-range filter areas 62 allows light in a wavelength range corresponding to the color of the large-range light area 51 in the light emitted from the light source 31 to pass through at a transmittance corresponding to the brightness of the small-range light area 52 corresponding to the small-range filter area 62, and the large-range light area 51 corresponds to the large-range filter area 61 including the small-range filter area 62.

[0131] Specifically, the transmission wavelength range of each of the plurality of small-range filter areas 62 is set to a wavelength range corresponding to the color of the large-range light area 51, and the large-range light area 51 corresponds to the large-range filter area 61 including the small-range filter area 62. The transmittance of each of the plurality of small-range filter areas 62 with respect to light in the transmission wavelength range is set to a transmittance corresponding to the brightness of the small-range light area 52 corresponding to the small-range filter area 62. The higher the brightness of the small-range light area 52, the higher the transmittance of the small-range filter area 62 with respect to light in the transmission wavelength range. It should be noted that the transmittance of each of the plurality of small-range filter areas 62 with respect to light in wavelength ranges other than the transmission wavelength range is lower than the transmittance with respect to light in the transmission wavelength range.

[0132] exist Figure 7In the example of , the plurality of small-range filter regions 62 included in each of the plurality of large-range filter regions 61 are divided into four types of small-range filter regions 62 (specifically, first to fourth small-range filter regions 621 to 624), and the four types of small-range filter regions 62 respectively correspond to the four types of small-range light regions 52. The first to fourth small-range filter regions 621 to 624 have different levels of transmittance with respect to light in the transmission wavelength range.

[0133] The level of transmittance of the first small range filter area 621 with respect to the light of the transmission wavelength range is set to “level 1” corresponding to the brightness level (level 1) of the first small range light area 521 corresponding to the first small range filter area 621. The level of transmittance of the second small range filter area 622 with respect to the light of the transmission wavelength range is set to “level 2” corresponding to the brightness level (level 2) of the second small range light area 522 corresponding to the second small range filter area 622. The level of transmittance of the third small range filter area 623 with respect to the light of the transmission wavelength range is set to “level 3” corresponding to the brightness level (level 3) of the third small range light area 523 corresponding to the third small range filter area 623. The level of transmittance of the fourth small range filter area 624 with respect to the light of the transmission wavelength range is set to “level 4” corresponding to the brightness level (level 4) of the fourth small range light area 524 corresponding to the fourth small range filter area 624. It should be noted that, as the level of transmittance increases from “level 1” to “level 4”, the transmittance gradually increases.

[0134] It should be noted that in Figure 7 In the example of , each of the small-range filter regions 62 is labeled with a letter (r, o, g, b) indicating a hue corresponding to the transmission wavelength range of the small-range filter region 62 and a number (1, 2, 3, 4) indicating the level of transmittance with respect to the transmission wavelength range. For example, with respect to the small-range filter region 62 labeled "r1", the transmission wavelength range is set to a wavelength range corresponding to "red", and the transmittance with respect to the transmission wavelength range is set to "level 1".

[0135] (Output characteristics of light source>

[0136] Figure 8(a) shows the output (spectral output) of each of the first light source 311, the second light source 312, and the third light source 313. The first light source 311 emits light having a central wavelength of about 610 nm and an emission band width of about 80 nm. The second light source 312 emits light having a central wavelength of about 520 nm and an emission band width of about 150 nm. The third light source 313 emits light having a central wavelength of about 470 nm and an emission band width of about 100 nm.

[0137] like Figure 8 As shown in (a), in this example, the maximum outputs of the first to third light sources 311 to 313 can be considered to be the same.

[0138] <Transmittance characteristics of small filter areas>

[0139] Figure 8 (b) shows the transmittance characteristics of the first small range filter area 621. Figure 8 In (b), "r1" represents the transmittance characteristic of the first small range filter area 621 whose transmission wavelength range is set to the wavelength range corresponding to "red". "o1" represents the transmittance characteristic of the first small range filter area 621 whose transmission wavelength range is set to the wavelength range corresponding to "orange". "g1" represents the transmittance characteristic of the first small range filter area 621 whose transmission wavelength range is set to the wavelength range corresponding to "green". "b1" represents the transmittance characteristic of the first small range filter area 621 whose transmission wavelength range is set to the wavelength range corresponding to "blue".

[0140] like Figure 8 (b) shows an example in which the transmittance of light in the first narrow filter region 621 (the transmittance of light in the transmission wavelength range) is set to level 1 (in any hue). Figure 8 In the example of (b), it is about 0.3 times the maximum transmittance.) In this way, the brightness level of the first narrow light region 521 generated by the first narrow filter region 621 becomes the same level (level 1) in any hue.

[0141] Figure 8 (c) shows the transmittance characteristics of the second small range filter area 622. Figure 8In (c), "r2" represents the transmittance characteristic of the second small range filter area 622 whose transmission wavelength range is set to the wavelength range corresponding to "red". "o2" represents the transmittance characteristic of the second small range filter area 622 whose transmission wavelength range is set to the wavelength range corresponding to "orange". "g2" represents the transmittance characteristic of the second small range filter area 622 whose transmission wavelength range is set to the wavelength range corresponding to "green". "b2" represents the transmittance characteristic of the second small range filter area 622 whose transmission wavelength range is set to the wavelength range corresponding to "blue".

[0142] like Figure 8 (c) In this example, the transmittance of light in the second narrow filter region 622 (the transmittance of light in the transmission wavelength range) is set to level 2 (in any hue). Figure 8 In the example of (c), it is about 0.5 times the maximum transmittance.) In this way, the brightness level of the second small range light area 522 generated by the second small range filter area 622 becomes the same level (level 2) in any hue.

[0143] Figure 8 (d) shows the transmittance characteristics of the third small range filter area 623. Figure 8 In (d), "r3" represents the transmittance characteristic of the third small range filter area 623 whose transmission wavelength range is set to the wavelength range corresponding to "red". "o3" represents the transmittance characteristic of the third small range filter area 623 whose transmission wavelength range is set to the wavelength range corresponding to "orange". "g3" represents the transmittance characteristic of the third small range filter area 623 whose transmission wavelength range is set to the wavelength range corresponding to "green". "b3" represents the transmittance characteristic of the third small range filter area 623 whose transmission wavelength range is set to the wavelength range corresponding to "blue".

[0144] like Figure 8 As shown in (d), in this example, the transmittance of light of the third small range filter area 623 (the transmittance of light in the transmission wavelength range) is set to level 3 (in any hue). Figure 8 In the example of (d), it is about 0.7 times the maximum transmittance.) In this way, the brightness level of the third small range light area 523 generated by the third small range filter area 623 becomes the same level (level 3) in any hue.

[0145] Figure 8 (e) shows the transmittance characteristics of the fourth small range filter area 624. Figure 8In (e), "r4" represents the transmittance characteristic of the fourth small range filter area 624 whose transmission wavelength range is set to the wavelength range corresponding to "red". "o4" represents the transmittance characteristic of the fourth small range filter area 624 whose transmission wavelength range is set to the wavelength range corresponding to "orange". "g4" represents the transmittance characteristic of the fourth small range filter area 624 whose transmission wavelength range is set to the wavelength range corresponding to "green". "b4" represents the transmittance characteristic of the fourth small range filter area 624 whose transmission wavelength range is set to the wavelength range corresponding to "blue".

[0146] like Figure 8 As shown in (e), in this example, the transmittance of light of the fourth small range filter area 624 (the transmittance of light in the transmission wavelength range) is set to level 4 ( Figure 8 In the example of (e), it is substantially equal to the maximum transmittance.) In this way, the brightness level of the fourth small range light region 524 generated by the fourth small range filter region 624 becomes the same level (level 4) in any hue.

[0147] 〔Brightness adjustment processing〕

[0148] Next, refer to Fig. 9 The following describes the brightness adjustment process performed by the projection control unit 403. For example, the brightness adjustment process is performed before the distance measurement process is started.

[0149] <Step S1>

[0150] The projection control unit 403 sets the driving current value of the light source 31 (specifically, each of the first to third light sources 311 to 313) to an initial value. The initial value of the driving current value is set so that, when the reflectivity of the measurement surface is a specified value (the assumed standard reflectivity), the maximum brightness of the light emitted from the light source 31 appropriately falls within the "grayscale range (e.g., 0 to 255) of the brightness specified by the control unit 40". For example, the initial value of the driving current value of the light source 31 is set so that the maximum brightness of the light source 31 is slightly smaller than the maximum grayscale level in the above-mentioned grayscale level range (e.g., about 80 to 90% of the maximum grayscale level).

[0151] <Step S2>

[0152] Next, the projection control unit 403 selects a light source 31 to be processed from the unprocessed light sources 31 among the first to third light sources 311 to 313, and causes the light source driving unit 35 to drive the selected light source 31. The unprocessed light source 31 is a light source 31 that has not been processed in steps S2 to S4 after step S1 or step S7.

[0153] Specifically, the projection control unit 403 sends to the light source driving unit 35 an instruction to instruct the light source driving unit 35 to drive the light source 31 selected as the processing object, and the driving current value set for the light source 31. The light source driving unit 35 drives the light source 31 selected by the projection control unit 403 according to the driving current value sent from the projection control unit 403. In this way, light is projected from the light source 31 selected by the projection control unit 403 onto the measurement surface (in this example, the surface of the object 0B).

[0154] <Step S3>

[0155] Next, the projection control unit 403 controls the first imaging unit 10 to capture images while the light source 31 selected in step S2 is projecting light onto the measurement surface. In this way, a first image P10 including light projected from the light source 31 onto the measurement surface can be obtained.

[0156] <Step S4>

[0157] Next, the projection control unit 403 obtains the maximum brightness of a pixel from the first image P10 obtained in step S3. The maximum brightness of the pixel corresponds to the maximum brightness of the light projected onto the measurement surface from the light source 31 selected in step S2.

[0158] <Step S5>

[0159] Next, the projection control unit 403 determines whether there is an unprocessed light source 31 among the first to third light sources 311 to 313. If there is an unprocessed light source 31, the process of step S2 is performed; otherwise, the process of step S6 is performed.

[0160] <Step S6>

[0161] Next, the projection control unit 403 determines whether the maximum brightness of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is appropriate. If the maximum brightness of the light emitted from the light source 31 is appropriate, the brightness adjustment process is terminated; otherwise, the process of step S7 is performed.

[0162] In this example, the projection control unit 403 determines whether the balance of the maximum brightness of each of the first to third light sources 311 to 313 is appropriate. When the maximum brightness of each of the first to third light sources 311 to 313 is considered to be the same (for example, when the difference in the maximum brightness of each of the first to third light sources 311 to 313 is within the allowable range), the projection control unit 403 determines that the balance of the maximum brightness of each of the first to third light sources 311 to 313 is appropriate.

[0163] In addition, in this example, the projection control unit 403 determines whether the maximum brightness of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is saturated. Specifically, the projection control unit 403 determines that the maximum brightness of the light source 31 is saturated when the maximum brightness of the light source 31 reaches the "maximum grayscale level in the grayscale level range of the brightness specified in the control unit 40".

[0164] <Step S7>

[0165] When the maximum brightness of the light source 31 (specifically, each of the first to third light sources 311 to 313 ) is inappropriate, the projection control unit 403 resets the driving current value of the light source 31 so that the maximum brightness of the light source 31 is appropriate. Next, the process of step S2 is performed.

[0166] For example, in this example, when the balance of the maximum brightness of each of the first to third light sources 311 to 313 is not appropriate, the projection control unit 403 resets the driving current value of each of the first to third light sources 311 to 313 so that the balance of the maximum brightness of each of the first to third light sources 311 to 313 is appropriate.

[0167] Specifically, the projection control unit 403 selects the largest maximum brightness from the maximum brightness of each of the first to third light sources 311 to 313 as the "reference brightness". Next, the projection control unit 403 selects a "light source 31 whose maximum brightness is lower than the reference brightness" from the first to third light sources 311 to 313, and increases the drive current value set for the selected light source 31.

[0168] In addition, in this example, when the maximum brightness of the light emitted from the light source 31 (specifically, each of the first to third light sources 311 to 313) is saturated, the projection control unit 403 resets the driving current value of the light source 31 so that the brightness emitted from the light source 31 is unsaturated.

[0169] Specifically, the projection control unit 403 reduces the driving current value set for the light source 31 whose maximum brightness is saturated among the first to third light sources 311 to 313. For example, the projection control unit 403 corrects the driving current value set for the light source 31 so that the corrected driving current value is lower by a specified grayscale level than the driving current value derived from "the relationship between the brightness of the light emitted from the light source 31 and the driving current value" and "the maximum grayscale level in the grayscale level range of the brightness specified in the control unit 40".

[0170] 〔Distance measurement processing〕

[0171] Next, refer to Fig.10The distance measurement process is described below. This distance measurement process is an example of a distance measurement method. For example, when the distance measurement device 1 is started, the control unit 40 performs the following process.

[0172] <Step S10>

[0173] First, the control unit 40 (projection control unit 403 ) controls the projection unit 30 to project the pattern light 50 into a range where the first field of view 10 a of the first imaging unit 10 and the second field of view 20 a of the second imaging unit 20 overlap.

[0174] <Step S11>

[0175] Next, the control unit 40 acquires the first image P10 acquired by the first camera unit 10 and the second image P20 acquired by the second camera unit 20. In this example, the control unit 40 selects the first image P10 and the second image P20 to be processed from the first image P10 and the second image P20 stored in the storage unit 41, and acquires the selected first image P10 and the second image P20.

[0176] <Step S12>

[0177] Next, the control unit 40 (first search unit 411) performs a first search process on the first image P10 and the second image P20 obtained in step S11. In this way, a plurality of large-range block combinations (combinations of large-range reference blocks B11 and large-range corresponding blocks B21) can be obtained.

[0178] <Step S13>

[0179] Next, the control unit 40 (second search unit 412) performs a second search process on the large range block combination (combination of the large range reference block B11 and the large range corresponding block B21) obtained in step S12. In this way, a plurality of small range block combinations (combinations of the small range reference block B12 and the small range corresponding block B22) can be obtained.

[0180] <Step S14>

[0181] Next, the control unit 40 (distance derivation unit 413) performs distance derivation processing based on the small range block combination (combination of the small range reference block B12 and the small range corresponding block B22) obtained in step S13. In this way, distance information (distance information indicating the distance D0 corresponding to each of the small range reference blocks B12 selected sequentially from the first image P10) can be obtained.

[0182] <Step S15>

[0183] Next, the control unit 40 determines whether to continue the distance measurement process. If the distance measurement process is to be continued, the process of step S11 is performed; otherwise, the distance measurement process is terminated.

[0184] [Effects of implementation methods]

[0185] As described above, in the distance measuring device 1 of the embodiment, the projection unit 30 projects the pattern light 50 to the range where the first field of view 10a of the first imaging unit 10 overlaps with the second field of view 20a of the second imaging unit 20. The pattern light 50 is a pattern light in which a plurality of large-range light regions 51 having different hues are distributed in a prescribed hue pattern, and in each of the plurality of large-range light regions 51, a small-range light region 52 having the same hue but different brightness is distributed in a prescribed brightness pattern.

[0186] In other words, the pattern light 50 is a pattern light including a plurality of light regions with different hues and a plurality of light regions with the same hue but different brightness, in which a plurality of large-range light regions 51 are distributed in a prescribed pattern, and in each of the plurality of large-range light regions 51, a plurality of small-range light regions 52 are distributed in a prescribed pattern. The plurality of large-range light regions 51 include a plurality of light regions with different hues and are distributed in a prescribed hue pattern. The plurality of small-range light regions 52 include a plurality of light regions with the same hue but different brightness and are distributed in a prescribed brightness pattern.

[0187] In the above structure, by projecting the unique pattern light 50 onto the measuring surface, a unique pattern (texture) can be formed on the measuring surface. In this way, even if the measuring surface includes a plain surface (for example, a flat single-color surface), stereo matching (corresponding point search) can be performed with good accuracy. In this way, the distance D0 to the measuring surface can be accurately measured.

[0188] It should be noted that, depending on the measurement surface, the following situation may sometimes occur: within a specific wavelength range, the light absorptivity becomes higher or the light reflectivity becomes lower. Therefore, assuming that the pattern light 50 is composed of light within a single wavelength range, it is difficult to form a unique pattern on the measurement surface when the wavelength range of the pattern light 50 is included in the above-mentioned specific wavelength range.

[0189] On the other hand, in the distance measuring device 1 of the present embodiment, a plurality of wide-range light regions 51 having different hues (wavelength ranges) are distributed in a predetermined hue pattern in the pattern light 50. Thus, even if the wavelength range corresponding to the color of any wide-range light region 51 among the plurality of wide-range light regions 51 is included in the above-mentioned specific wavelength range (a wavelength range in which the light absorptivity becomes high or the light reflectivity becomes low), a unique pattern can be formed on the measurement surface (the surface of the object 0B in this example) by projecting the remaining wide-range light regions 51 onto the measurement surface.

[0190] In addition, in the distance measuring device 1 of the present embodiment, the projection unit 30 has a filter 33 for generating the pattern light 50. The filter 33 includes a plurality of wide-range filter areas 61, each corresponding to the plurality of wide-range light areas 51, and the plurality of wide-range filter areas 61 are distributed in the same pattern as the hue pattern of the plurality of wide-range light areas 51. Each of the plurality of wide-range filter areas 61 includes a plurality of small-range filter areas 62, each corresponding to the plurality of small-range light areas 52 included in the wide-range light area 51 corresponding to the wide-range filter area 61 among the plurality of wide-range light areas 51, and distributed in the same pattern as the brightness pattern.

[0191] In other words, the optical filter 33 includes a plurality of wide-range filter areas 61, the plurality of wide-range filter areas 61 including a plurality of filter areas for generating a plurality of light areas with different hues and a plurality of filter areas for generating a plurality of light areas with the same hue but different brightness, the plurality of wide-range filter areas 61 respectively corresponding to the plurality of wide-range light areas 51, and distributed in a pattern identical to the prescribed pattern (the prescribed pattern of the wide-range light areas 51). Each of the plurality of wide-range filter areas 61 includes a plurality of small-range filter areas 62, the plurality of small-range filter areas 62 respectively corresponding to the plurality of small-range light areas 52 included in the wide-range light area 51 corresponding to the wide-range filter area 61 among the plurality of wide-range light areas 51, and distributed in a pattern identical to the prescribed pattern (the prescribed pattern of the small-range light areas 52).

[0192] With the above structure, it is possible to easily generate pattern light 50 having a desired pattern. In addition, since there is no deviation in diffraction efficiency (deviation in brightness grayscale) due to manufacturing error or assembly error as in a diffractive optical element, it is possible to stably generate pattern light 50 having a desired pattern.

[0193] In the distance measuring device 1 of the embodiment, the projection unit 30 includes the light source 31 and the optical system 32 for guiding the light emitted from the light source 31 toward the filter 33. With such a configuration, the light for generating the pattern light 50 can be easily irradiated onto the filter 33.

[0194] In addition, in the distance measurement device 1 of the embodiment, each of the plurality of wide range filter areas 61 transmits light of a wavelength range corresponding to the color of the wide range light area 51 corresponding to the wide range filter area 61, among the light emitted from the light source 31. Each of the plurality of small range filter areas 62 transmits light of a wavelength range corresponding to the color of the wide range light area 51, among the light emitted from the light source 31, at a transmittance corresponding to the brightness of the small range light area 52 corresponding to the small range filter area 62, and the wide range light area 51 corresponds to the wide range filter area 61 including the small range filter area 62.

[0195] In other words, in the filter 33, each of the plurality of filter regions for generating a plurality of light regions with different hues extracts light of a wavelength range corresponding to the plurality of light regions with different hues from the light emitted from the light source 31. Each of the plurality of filter regions for generating a plurality of light regions with the same hue but different brightness extracts light of a wavelength range corresponding to the plurality of light regions with the same hue but different brightness from the light emitted from the light source 31, and the amount of extraction corresponds to the brightness.

[0196] Under the above structure, each of the plurality of large-range filter areas 61 can selectively extract light of a wavelength range corresponding to the color of the large-range light area 51 corresponding to the large-range filter area 61. Each of the plurality of small-range filter areas 62 can selectively extract light of a wavelength range corresponding to the color of the large-range light area 51 corresponding to the large-range filter area 61 including the small-range filter area 62, and the extraction amount corresponds to the brightness of the small-range light area 52 corresponding to the small-range filter area 62. In this way, the pattern light 50 can be efficiently generated.

[0197] In the distance measurement device 1 of the embodiment, the projection control unit 403 adjusts the brightness of the light emitted from the light source 31 so that the brightness of the light emitted from the light source 31 is not saturated. With such a configuration, the brightness of the light emitted from the light source 31 can be appropriately set.

[0198] In addition, in the distance measurement device 1 of the embodiment, the first search unit 411 sequentially selects the large-range reference block B11 from the first image P10, and searches for the large-range corresponding block B21 corresponding to the large-range reference block B11 from the second image P20. The second search unit 412 sequentially selects the small-range reference block B12 from the large-range reference block B11, and searches for the small-range corresponding block B22 corresponding to the small-range reference block B12 from the large-range corresponding block B21 corresponding to the large-range reference block B11. The distance derivation unit 413 derivates the distance D0 to the measurement surface corresponding to the small-range reference block B12 based on the position difference between the small-range reference block B12 and the small-range corresponding block B22.

[0199] With the above configuration, the second search unit 412 can search (finer search) the pixel block detected by the search (coarser search) of the first search unit 411. In this way, compared with the case where only the second search unit 412 performs the search (specifically, sequentially selecting the small range reference block B12 from the first image P10 and searching the small range corresponding block B22 corresponding to the small range reference block B12 from the second image P20), the time required for the corresponding point search (specifically, the search for the small range corresponding block B22) can be shortened. As a result, the distance measurement device 1 can achieve high-speed measurement of the distance D0.

[0200] In addition, in the distance measurement device 1 of the embodiment, the first reference movement amount, which is the movement amount of the pixel range for selecting the large-range reference block B11, is greater than the second reference movement amount, which is the movement amount of the pixel range for selecting the small-range reference block B12. With such a structure, the time required for selecting the large-range reference block B11 can be shortened, thereby making it possible to speed up the first search process (the search performed by the first search unit 411). In this way, the distance measurement device 1 can achieve speedy measurement of the distance D0.

[0201] In addition, in the distance measurement device 1 of the embodiment, the movement amount of the pixel range for selecting the large-range reference block BR1, i.e., the first reference movement amount, is greater than the movement amount of the pixel range for selecting the small-range reference block BR2, i.e., the second reference movement amount. With such a structure, the time required for selecting the large-range reference block BR1 can be shortened, thereby making it possible to speed up the first search process (the search performed by the first search unit 411). In this way, the distance measurement device 1 can achieve high-speed measurement of the distance D0.

[0202] (Variation 1 of the embodiment)

[0203] Fig.11The structure of the distance measuring device 1 of the modification example 1 of the embodiment is shown. The distance measuring device 1 of the modification example 1 of the embodiment is different from the distance measuring device 1 of the embodiment in that the structure of the projection unit 30 is different. The other structures and processes of the distance measuring device 1 of the modification example 1 of the embodiment are the same as those of the distance measuring device 1 of the embodiment.

[0204] In the first modification of the embodiment, the projection unit 30 has a single light source 31. For example, the light source 31 is a white laser diode. The optical system 32 has a collimator lens 326. The collimator lens 326 converts the light emitted from the light source 31 into parallel light. The other structures of the projection unit 30 of the first modification of the embodiment are the same as those of the projection unit 30 of the embodiment.

[0205] <Output characteristics of light source>

[0206] Fig.12 (a) shows the output (spectral output) of the light source 31 according to Modification 1 of the embodiment. The output of light output from a single light source 31 changes according to a change in wavelength.

[0207] Specifically, as the wavelength of light increases from 430nm to 470nm, the light output gradually increases from the minimum level (zero) to the maximum level, and as the wavelength of light increases from 470nm to 510nm, the light output gradually decreases from the maximum level to "about 0.2 times the maximum level". Furthermore, as the wavelength of light increases from 510nm to 580nm, the light output gradually increases from "about 0.2 times the maximum level" to "about 0.4 times the maximum level", and as the wavelength of light increases from 580nm, the light output gradually decreases from "about 0.4 times the maximum level" toward the minimum level.

[0208] <Transmittance characteristics of small filter area)

[0209] Fig.12 (b)~ Fig.12 (e) shows the transmittance characteristics of the first to fourth small range filter regions 621 to 624 of Modification 1 of the embodiment. Fig.12 (b)~ Fig.12 As shown in (e), when comparing by hue, it can be considered that the transmittance levels of the first to fourth small filter regions 621 to 624 with respect to light in the transmission wavelength range at any hue are set to "levels 1 to 4", respectively.

[0210] In addition, if Fig.12As shown in (b), based on the output characteristics of the single light source 31 (the output changes with the wavelength of the light emitted from the single light source 31), the transmittance of the first small range filter area 621 with respect to the light of the transmission wavelength range is set for each hue. Fig.12 In the example of (b), the transmittance (transmittance of light relative to the transmitted wavelength range) of the first small range filter area 621 whose transmitted wavelength range is set to the wavelength range corresponding to "red" is higher than the transmittance (transmittance of light relative to the transmitted wavelength range) of the first small range filter area 621 whose transmitted wavelength range is set to the wavelength range corresponding to "other hues".

[0211] In this way, according to the output characteristics of the single light source 31, the transmittance of the first small range filter area 621 relative to the light of the transmission wavelength range is set according to the hue, so that the brightness level of the first small range light area 521 generated by the first small range filter area 621 can be the same level (level 1) at any hue. It should be noted that the same can be said for the second small range filter area to the fourth small range filter area 622 to 624.

[0212] (Variation 2 of the embodiment)

[0213] The distance measurement device 1 of the second modification of the embodiment differs from the distance measurement device 1 of the embodiment in the structures of the pattern light 50 and the filter 33. The other structures and processes of the distance measurement device 1 of the second modification of the embodiment are the same as those of the distance measurement device 1 of the embodiment.

[0214] Fig.13 FIG. 2 shows a portion of the pattern light 50 of the second variant of the embodiment. In the pattern light 50 of the second variant of the embodiment, the shape of the wide-range light region 51 is different from the shape (rectangular) of the small-range reference block B12. It should be noted that the arrangement (distribution pattern) of the wide-range light region 51 in the pattern light 50 is different from the pattern light 50 of the embodiment (see FIG. 2 ). Figure 6 ) is the same as the arrangement of the large-range light region 51 in the embodiment. In addition, the structure (shape) and arrangement (distribution pattern) of the small-range light region 52 included in each of the plurality of large-range light regions 51 are the same as those of the pattern light 50 (see Figure 6 ) has the same structure and arrangement as the small light area 52.

[0215] Fig.14 FIG. 2 shows a portion of the optical filter 33 according to the second modification of the embodiment. In the optical filter 33 according to the second modification of the embodiment, the shape of the wide-range optical filter region 61 is similar to Fig.13The shape of the wide-range light region 51 shown is also different from the shape (rectangular) of the small-range reference block B12. It should be noted that the arrangement (distribution pattern) of the wide-range filter region 61 in the filter 33 is different from the filter of the embodiment (see Figure 7 ) is arranged in the same manner as the large-range filter regions 61 in the embodiment. In addition, the structure (shape) and arrangement (distribution pattern) of the small-range filter regions 62 included in each of the plurality of large-range filter regions 61 are the same as those of the filter 33 of the embodiment (see Figure 7 ) has the same structure and arrangement as the small-range filter area 62.

[0216] (Variation 3 of the embodiment)

[0217] The distance measurement device 1 according to the third modification of the embodiment is different from the distance measurement device 1 according to the embodiment in that the first search process performed by the control unit 40 (first search unit 411 ) is different.

[0218] In the third variant of the embodiment, the first search unit 411 performs a reduction process on the large-scale reference block B11. The reduction process is a process for reducing the amount of data. In addition, the first search unit 411 performs a reduction process on the large-scale reference block BR1. Then, the first search unit 411 derives the similarity between the large-scale reference block B11 and the large-scale reference block BR1 based on the large-scale reference block B11 that has been subjected to the reduction process and the large-scale reference block BR1. It should be noted that as examples of the reduction process, thinning processing, merging processing, etc. can be listed.

[0219] [Effects of Modification 3 of Implementation Method]

[0220] As described above, in the distance measurement device 1 according to the third variation of the embodiment, the first search unit 411 derives the similarity between the large range reference block B11 and the large range reference block BR1 based on the reduced large range reference block B11 and the large range reference block BR1.

[0221] This configuration can shorten the time required to derive the similarity between the large-range base block B11 and the large-range reference block BR1, thereby speeding up the first search process (the search performed by the first search unit 411). This allows the distance measurement device 1 to measure the distance D0 at a faster speed.

[0222] (Application example of distance measuring device)

[0223] The above distance measuring device 1 is, for example, arranged on the end effector (e.g., a clamping part, etc., not shown in the figure) of a robot arm that performs operating actions in a factory. In this case, the control unit 40 of the distance measuring device 1 receives an instruction to obtain the distance from a robot controller (not shown in the figure) via the communication interface unit 42 during the operating process of the robot arm. In response to the instruction, the control unit 40 (measuring unit 404) measures the distance between the position of the end effector and the surface of the object OB as the operating object, and sends the measurement result (distance information) to the robot controller via the communication interface unit. The robot controller performs feedback control on the action of the end effector based on the distance information received from the distance measuring device 1. It should be noted that when the distance measuring device 1 is arranged on the end effector, it is preferred that the distance measuring device 1 is small in size and light in weight.

[0224] (Other embodiments)

[0225] In the above description, the case where the large range reference block B11 is a pixel block including 36 pixels arranged in a matrix of 6 rows and 6 columns is taken as an example, but the present invention is not limited to this. The shape and size of the large range reference block B11 may also be other shapes and sizes. It can be said that the same is true for the large range reference block BR1, the small range reference block B12, and the small range reference block BR2.

[0226] In addition, in the above description, the example cited is a case where the number of camera units is two (including the first camera unit 10 and the second camera unit 20), but it is not limited to this. The distance measurement device 1 may include three or more camera units. In this case, these camera units are arranged so that their fields of view overlap with each other, and the pattern light 50 is projected into the range where these fields of view overlap.

[0227] In the above description, the example cited is a case where the types of the wide-range light region 51 included in the pattern light 50 are four, but the present invention is not limited thereto. The types of the wide-range light region 51 may be two, three, or five or more. It can be said that the same is true for the types of the wide-range filter region 61 included in the filter 33.

[0228] In addition, in the above description, the example cited is a case where the types of the small-range light regions 52 included in each of the plurality of large-range light regions 51 in the pattern light 50 are four, but the present invention is not limited thereto. The types of the small-range light regions 52 may be two, three, or five or more. It can be said that the same is true for the types of the small-range filter regions 62 included in each of the plurality of large-range filter regions 61 in the filter 33.

[0229] In the above description, the example given is the case where the filter 33 is a transmission filter, but the present invention is not limited thereto. For example, the filter 33 may be a reflection filter.

[0230] In the above description, the multiple small-range light regions 52 included in the pattern light 50 may include a small-range light region 52 with zero brightness (no light spot). The multiple small-range filter regions 62 included in the filter 33 may also include a small-range filter region 62 that blocks light and does not allow it to pass through (a small-range filter region 62 for generating a no light spot).

[0231] In addition, in the above description, the example given is a case where the distance measuring device 1 is set on the end effector of the robot arm, but it is not limited to this. For example, the distance measuring device 1 can also be applied to other systems that perform predetermined control based on the distance D0 to the measurement surface (for example, the surface of the object OB).

[0232] In addition, the structure of the distance measuring device 1 is not limited to the structure described above. For example, the first imaging element 12 and the second imaging element 22 may be a photoelectric sensor array in which a plurality of photoelectric sensors are arranged in a matrix.

[0233] In the above description, the components of the distance measuring device 1 may be arranged as a single device or may be arranged as a plurality of devices (e.g., a plurality of devices communicating via a communication network such as the Internet). The control unit 40 may be implemented by a single processor or by a plurality of processors. In addition, the control unit 40 may be implemented by a plurality of operation processing devices (e.g., a plurality of operation processing devices communicating via a communication network such as the Internet).

[0234] In addition, the above embodiments and modifications may be appropriately combined and implemented. The above embodiments and modifications are essentially preferred examples, and are not intended to limit the technology disclosed herein, its application objects, or its scope of use.

[0235] -Industrial Applicability-

[0236] In summary, the technology disclosed here is very useful as a distance measurement technology.

[0237] -Explanation of symbols-

[0238] 1 Distance measurement device

[0239] 10 First Camera Department

[0240] 20 Second Camera Department

[0241] 30 Projection Department

[0242] 31 Light Source

[0243] 32 Optical system

[0244] 33 Filters

[0245] 40 Control Department

[0246] 401 First Camera Processing Unit

[0247] 402 Second Camera Processing Unit

[0248] 403 Projection Control Department

[0249] 404 Measurement Department

[0250] 411 First Search Department

[0251] 412 Second Search Department

[0252] 413 Distance Derivation Department

[0253] 50 pattern light

[0254] 51 Large light area

[0255] 52 Small light area

[0256] 61 Large filter area

[0257] 62 Small filter area.

Claims

1. A distance measuring device, characterized in that: The distance measuring device includes a first camera unit, a second camera unit, a projection unit, and a measuring unit. The first camera unit and the second camera unit are arranged so that their fields of view overlap. The projection unit projects pattern light to a range where the field of view of the first camera unit and the field of view of the second camera unit overlap. The measuring unit measures a distance to a measurement surface onto which the pattern light is projected based on a parallax between a first image obtained by the first imaging unit and a second image obtained by the second imaging unit. The pattern light is a pattern light including multiple light areas with different hues and multiple light areas with the same hue but different brightness. In the pattern light, multiple large-range light areas are distributed in a prescribed pattern, and in each of the multiple large-range light areas, multiple small-range light areas are distributed in a prescribed pattern.

2. The distance measuring device according to claim 1, characterized in that: The projection unit has a filter for generating the pattern light. The filter includes a plurality of large-range filter areas, the plurality of large-range filter areas include a plurality of filter areas for generating the plurality of light areas with different hues, and a plurality of filter areas for generating the plurality of light areas with the same hue but different brightness, the plurality of large-range filter areas respectively correspond to the plurality of large-range light areas, and are distributed in a pattern identical to the prescribed pattern, Each of the multiple wide-range filter areas includes multiple small-range filter areas, and the multiple small-range filter areas respectively correspond to the multiple small-range light areas included in the wide-range light area corresponding to the wide-range filter area among the multiple wide-range light areas, and are distributed in a pattern identical to the prescribed pattern.

3. The distance measuring device according to claim 2, characterized in that: The projection unit includes a light source and an optical system that guides light emitted from the light source toward the filter.

4. The distance measuring device according to claim 3, characterized in that: Each of the plurality of filter regions for generating the plurality of light regions with different hues extracts light of a wavelength range corresponding to the plurality of light regions with different hues from the light emitted from the light source, Each of the multiple filter areas used to generate the multiple light areas with the same hue but different brightness extracts light in a wavelength range corresponding to the multiple light areas with the same hue but different brightness from the light emitted from the light source, and the extracted amount corresponds to the brightness.

5. The distance measuring device according to claim 4, characterized in that: The distance measuring device includes a projection control section that adjusts the brightness of the light emitted from the light source so that the brightness of the light emitted from the light source is not saturated.

6. The distance measuring device according to any one of claims 1 to 5, characterized in that: The plurality of wide-range light regions include the plurality of light regions with different hues and are distributed in a predetermined hue pattern. The plurality of small-range light regions include the plurality of light regions having the same hue but different brightness, and are distributed in a predetermined brightness pattern.

7. The distance measuring device according to any one of claims 1 to 6, characterized in that: The measuring unit includes a first searching unit, a second searching unit and a distance derivation unit. The first search unit sequentially selects a large-range reference block from the first image, and searches the second image for a large-range corresponding block corresponding to the large-range reference block. The second search unit sequentially selects small-range reference blocks from the large-range reference blocks, searches for small-range corresponding blocks corresponding to the small-range reference blocks from the large-range corresponding blocks corresponding to the large-range reference blocks, The distance deriving unit derives the distance to the measurement surface corresponding to the small range reference block according to the position difference between the small range reference block and the small range corresponding block.

8. The distance measuring device according to claim 7, characterized in that: The first search unit sequentially selects the large-range reference blocks from the first image by moving a pixel range used to select the large-range reference blocks in the first image by a first reference movement amount. The second search section sequentially selects the small-range reference blocks from the large-range reference block by moving the pixel range used to select the small-range reference block in the large-range reference block by a second reference movement amount at a time. The first reference movement amount is greater than the second reference movement amount.

9. The distance measuring device according to claim 7, characterized in that: The first search unit sequentially selects the large-range reference blocks from the second image by moving a pixel range of a large-range reference block used to select a candidate large-range corresponding block in the second image by a first reference movement amount. The second search unit sequentially selects the small range reference blocks from the large range corresponding block by moving the pixel range of the small range reference block used to select the candidate small range corresponding block in the large range corresponding block by a second reference movement amount. The first reference movement amount is greater than the second reference movement amount.

10. The distance measuring device according to claim 7, characterized in that: The first search unit is configured as follows: sequentially selecting from the second image a large-scale reference block that is a candidate for the large-scale corresponding block corresponding to the large-scale reference block, and determining the large-scale reference block having the greatest similarity with the large-scale reference block among the large-scale reference blocks sequentially selected from the second image as the large-scale corresponding block, A reduction process for reducing the amount of data is performed on the large-range reference block and the large-range reference block, and a similarity between the large-range reference block and the large-range reference block is derived based on the large-range reference block and the large-range reference block that have been subjected to the reduction process.

11. A distance measurement method, which uses a first camera unit, a second camera unit, and a projection unit to perform measurement, wherein the first camera unit and the second camera unit are arranged so that their fields of view overlap, and the projection unit projects pattern light to a range where the fields of view of the first camera unit and the fields of view of the second camera unit overlap, characterized in that: The distance measurement method comprises: a step of projecting the pattern light from the projection unit, a step of acquiring a first image obtained by the first camera unit and a second image obtained by the second camera unit, and a measuring step of measuring a distance to a measuring surface onto which the pattern light has been projected based on a parallax between the first image and the second image, The pattern light is a pattern light including multiple light areas with different hues and multiple light areas with the same hue but different brightness. In the pattern light, multiple large-range light areas are distributed in a prescribed pattern, and in each of the multiple large-range light areas, multiple small-range light areas are distributed in a prescribed pattern.

12. A distance measurement program, characterized in that: The distance measurement program is used to cause a computer to execute the distance measurement method according to claim 11.

Images