Three-dimensional measurement system, three-dimensional measurement device, and three-dimensional measurement method

Abstract

To provide a three-dimensional measurement system that takes precise three-dimensional measurements of an object by photographing it with a moving camera.SOLUTION: The three-dimensional measurement system uses a pattern projection device consisting of first and second projectors 2-1, 2-2 that project patterns 4-1, 4-2 onto an object 1, and a 3D scanner 3 that is provided between the object and the pattern projection device and captures an image of the object while moving. The three-dimensional measurement system includes: a correction function generation unit that generates a correction function F1 that is multiplied by a first pattern, and a correction function F2 that is multiplied by a second pattern input to the second projector so that a pattern captured within a field angle of the 3D scanner is aligned with a reference pattern; and a projection image correction unit that corrects the first pattern and the second pattern. The three-dimensional measurement system projects the corrected first pattern onto the object using a first projector, and the corrected second pattern onto the object using the second projector, and projects images of the first pattern and the second pattern superimposed on each other onto the object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to three-dimensional measurement technology. [Background technology]

[0002] In plant construction, building construction, vehicle manufacturing, and other projects, it is important to perform highly accurate on-site alignment in order to reflect construction drawings on the construction site.

[0003] Therefore, an effective method is to use 3D measurement, which involves scanning an object while moving a visual sensor (hereinafter sometimes referred to as a 3D scanner) such as an optical camera, infrared camera, ToF (Time Of Flight) sensor, or LiDAR (Light Detection And Ranging), successively estimating the amount of movement of the 3D scanner, and ultimately generating a 3D model of the 3D object.In addition, in recent years, photogrammetry technology, which uses a computer to analyze and combine photos taken from various angles, has proven effective in capturing images of an object from various directions.

[0004] However, if there are no distinctive patterns or shapes within the field of view during scanning, it is not possible to extract distinctive points (hereinafter referred to as feature points or landmarks), making it impossible to estimate the amount of movement of the 3D scanner, and the scan may fail. In particular, during plant construction, there are often objects with uniform surfaces such as pipes, and walls and floors that are covered with uniform protective sheets, making the walls and floors often featureless. As a result, there is a high possibility of the scan failing.

[0005] In Patent Document 1, three-dimensional measurement is performed by projecting a pattern onto the object using a pattern projection means that projects a slit pattern consisting of multiple slit lights and a stripe pattern consisting of one or more stripe-shaped lights onto the surface of the object to be measured.

[0006] Patent Document 2 introduces a method of projecting patterns onto an object from two directions using projectors and then photographing the object.

[0007] Patent Document 3 discloses a control device that identifies a shadow area and corrects the projection light in order to easily eliminate the shadow of an object when projecting an image using multiple projectors. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-003409 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-242183 [Patent Document 3] WO 2019 / 009100 A1 Summary of the Invention [Problem to be solved by the invention]

[0009] If the object being measured does not have a distinctive pattern or shape, it is not possible to extract feature points. However, by projecting a pattern onto the object, it is possible to add obtainable feature points. Furthermore, by moving a 3D scanner (hereinafter sometimes referred to as a camera) close to the object and performing three-dimensional measurements, it is possible to model a wide area with high precision. When the distance between the 3D scanner and the object is reduced, the pattern projected by the projector (hereinafter sometimes referred to as a projector) may be blocked by the 3D scanner (or even by the person or equipment operating the 3D scanner). In such cases, the pattern is distorted by the shadow of the camera, resulting in mismatches or a reduction in the number of feature points, which reduces the accuracy of the three-dimensional measurement.

[0010] In Patent Document 1, a pattern is projected onto an object to perform three-dimensional measurement, but the range of the object that can be measured is limited because the camera is fixed. For example, to measure piping inside a plant during construction, it is necessary to move the 3D scanner and capture images of any range.

[0011] Patent Document 2 introduces a method of projecting a pattern onto an object from two directions using a projector and then photographing the object, but does not address the issue that the camera may block the image projected from the projector when photographing from various directions.

[0012] In Patent Document 3, the projection light is corrected to eliminate shadows on the assumption that the positioning information representing the positioning of the projector and the target object is known. However, since the shape of the target object is unknown in 3D measurement, this method cannot be used in 3D measurement use cases.

[0013] Therefore, an object of the present invention is to enable precise three-dimensional measurement while minimizing the effect of camera movement when photographing an object with an arbitrarily moving camera to perform three-dimensional measurement, even if the object has few feature points. [Means for solving the problem]

[0014] A preferred aspect of the present invention is a three-dimensional measurement system that uses a pattern projection device consisting of first and second projectors with different optical axes that project a pattern onto an object, and a 3D scanner that is provided between the object and the pattern projection device and captures an image of the object while moving, the three-dimensional measurement system including: a correction function generation unit that generates a correction function F1 that multiplies a first pattern input to the first projector and a correction function F2 that multiplies a second pattern input to the second projector so that the pattern captured within the angle of view of the 3D scanner is aligned with a separately prepared reference pattern; and a projection image correction unit that corrects the first pattern using the correction function F1 and corrects the second pattern using the correction function F2, and projects the corrected first pattern onto the object using the first projector, projects the corrected second pattern onto the object using the second projector, and projects images of the first pattern and the second pattern onto the object in a superimposed manner.

[0015] Another preferred aspect of the present invention is a three-dimensional measuring device that inputs a pattern to a pattern projection device consisting of first and second projectors with different optical axes that project a pattern onto an object, in order to perform three-dimensional measurement with a 3D scanner that is provided between an object and a pattern projection device and captures an image of the object while moving. The three-dimensional measuring device includes: a correction function generation unit that generates a correction function F1 that multiplies a first pattern input to the first projector and a correction function F2 that multiplies a second pattern input to the second projector so that the pattern captured within the angle of view of the 3D scanner is aligned with a separately prepared reference pattern; and a projection image correction unit that corrects the first pattern using the correction function F1 and corrects the second pattern using the correction function F2. The corrected first pattern is projected onto the object by the first projector, and the corrected second pattern is projected onto the object by the second projector. This three-dimensional measuring device projects images of the first pattern and the second pattern onto the object in a superimposed manner.

[0016] Another preferred aspect of the present invention is a three-dimensional measurement method that includes: a first step of preparing a 3D scanner; a second step of determining the positional relationship between a first projector and a second projector, which have different optical axes from each other, and an object; a third step of defining a reference pattern on the object at a predetermined position within the angle of view of the 3D scanner; a fourth step of projecting a first correction pattern with the first projector and determining a correction function F1 that matches the reference pattern within the angle of view of the 3D scanner; a fifth step of projecting a second correction pattern with the second projector and determining a correction function F2 that matches the reference pattern within the angle of view of the 3D scanner; a sixth step of projecting a feature point pattern converted by the first correction function F1 with the first projector and projecting a feature point pattern converted by the second correction function F2 with the second projector; and a seventh step of imaging the object and performing three-dimensional measurement while moving the 3D scanner between the object and the pattern irradiation device. [Effects of the Invention]

[0017] According to the present invention, when an object is photographed with a camera that can be moved arbitrarily to perform three-dimensional measurement, even if the object has few feature points, precise three-dimensional measurement is possible while minimizing the effect of camera movement. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of a three-dimensional measurement system according to a first embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing the actions of a photographer using the three-dimensional measurement system according to the first embodiment. [Figure 3] FIG. 1 is a flow diagram of a three-dimensional measurement system according to a first embodiment. [Figure 4A] FIG. 4 is a conceptual diagram of the first projector according to the first embodiment before image correction. [Figure 4B] FIG. 10 is an image diagram of the first projector according to the first embodiment after image correction. [Figure 5A] FIG. 10 is an image diagram of the second projector according to the first embodiment before image correction. [Figure 5B] FIG. 10 is an image diagram of the second projector according to the first embodiment after image correction. [Figure 6] FIG. 4 is an image diagram of point correspondence for obtaining a correction function according to the first embodiment. [Figure 7] 5A and 5B are projection image diagrams of a projection image after image correction according to the first embodiment. [Figure 8] 10A and 10B are conceptual diagrams of deviation patterns when corrected images of two different colors are projected according to the second embodiment. [Figure 9A] 10 is a grid pattern as a corrected image according to the first embodiment. [Figure 9B] 10 is a pattern in which squares are arranged at equal intervals as a correction image according to the first embodiment. [Figure 9C] 10 is a pattern in which circles or dots are arranged at equal intervals as a correction image according to the first embodiment. [Figure 10] FIG. 10 is a schematic diagram of a three-dimensional measurement system according to a fourth embodiment. [Figure 11]FIG. 10 is an image diagram of generating a first projector mask in the three-dimensional measurement system according to the fourth embodiment. [Figure 12] FIG. 11 is an image diagram of point correspondence for obtaining a correction function according to the fourth embodiment. [Figure 13] FIG. 10 is an image diagram of generation of a second projector mask in the three-dimensional measurement system according to the fourth embodiment. [Figure 14] FIG. 10 is a screen image diagram of a CAD system according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments are illustrative for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0020] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0021] In the configurations of the embodiments described below, the same parts or parts having similar functions are denoted by the same reference numerals in different drawings, and redundant explanations may be omitted.

[0022] When there are multiple elements having the same or similar functions, they may be described using the same reference numeral with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted.

[0023] The designations "first," "second," "third," etc. in this specification are used to identify components and do not necessarily limit the number, order, or content. Furthermore, they do not prevent a component identified by a certain number from also serving the function of a component identified by another number.

[0024] All publications, patents, and patent applications cited herein are incorporated by reference in their entirety.

[0025] As used herein, elements referred to in the singular are intended to include the plural unless the context clearly indicates otherwise.

[0026] A three-dimensional measurement system according to a representative embodiment uses a pattern projection device consisting of at least first and second projectors that project onto the object, and a 3D scanner that is disposed between the object and the pattern projection device and captures the object while moving. The system also includes a correction function generation unit that generates a correction function F1 that is multiplied by the projection pattern of the first projector to match the projection pattern of the first projector to a separately prepared reference pattern within the field of view of the 3D scanner, and a correction function F2 that is multiplied by the projection pattern of the second projector, and a projection image correction unit that corrects the projection pattern using the correction functions F1 and F2.

[0027] The 3D scanner has an image capture function and sends the captured image to the correction function generator. As a first operation, the correction function generator converts the projection pattern input to the projector and defines a reference pattern that serves as a target for aligning the projection pattern within the field of view of the 3D scanner.

[0028] As a second function, the correction function unit generates a correction function F1 to be multiplied by the projection pattern projected from the first projector so that it matches the reference pattern. Similarly, the correction function generation unit generates a correction function F2 to be multiplied by the projection pattern projected from the second projector so that it matches the reference pattern. The reference pattern defined by the correction function generation unit is defined so that it matches the first and second projectors. By matching the patterns projected from both projectors to the same reference pattern, it is possible to project the same projected image from both projectors.

[0029] The projection image correction unit corrects the projection image using the correction functions F1 and F2 calculated by the correction function generation unit, and the same projection pattern is projected onto the object from the first projector and the second projector, and a three-dimensional model is generated by capturing an image with a 3D scanner.

[0030] According to the above configuration, in a system that projects patterns for adding feature points onto an object from at least two projectors, when a camera capturing an image of the object passes in front of the projector, it is possible to prevent the pattern of the projected image on the object from being changed by the shadow of the camera. Even when the camera is brought close to the object and images are captured at any position, it is possible to project the same image onto the object from multiple projectors. This makes it easy to associate feature points between multiple images captured by the cameras, enabling highly accurate three-dimensional measurement. [Example]

[0031] A representative first embodiment will be described with reference to FIGS. 1 to 7. FIG. Fig. 1 is a schematic diagram of a three-dimensional measurement system of Example 1. Fig. 1 shows an object 1 to be measured, at least a first projector 2-1 and a second projector 2-2 that serve as a pattern irradiation device, and a 3D scanner 3 that moves between the object 1 and the first projector 2-1 and second projector 2-2.

[0032] A first feature point adding pattern (hereinafter sometimes referred to as a "feature point pattern") 8-1 is input to the first projector 2-1, and a second feature point pattern 8-2 is input to the second projector 2-2. The first feature point pattern 8-1 is projected onto the object 1 by the first projector 2-1 as a first feature point image 4-1. The second feature point pattern 8-2 is projected onto the object 1 by the second projector 2-2 as a second feature point image 4-2.

[0033] The first feature point image 4-1 and the second feature point image 4-2 are projected so as to overlap (match) on the object 1. The first feature point pattern 8-1 and the second feature point pattern 8-2 may basically be the same pattern.

[0034] The 3D scanner 3 captures an image of the object 1 onto which the feature point images 4 projected by the first projector 2-1 and the second projector 2-2 are projected, and performs three-dimensional measurement of the object 1. Even if feature points cannot be sufficiently extracted from the object 1 itself, they can be obtained from the feature point image 4, making it possible to perform highly accurate three-dimensional measurement. Furthermore, even if the 3D scanner 3 blocks the irradiation of one projector and casts a shadow on the feature point image 4 of the object 1, the other projector projects the same image onto the shadowed part, so the feature point image 4 of the shadowed part can be compensated for.

[0035] However, as described above, the feature point images 4-1 and 4-2 projected by the first projector 2-1 and the second projector 2-2 must match on the target object 1. If they do not match, the shadowed parts will differ from the original pattern, which may make it difficult to associate feature points among the multiple captured images.

[0036] Here, the three-dimensional measurement system of the embodiment has a correction function generation unit 9 that generates a correction function F1 to multiply the first feature point pattern 8-1 of the first projector 2-1 and a correction function F2 to multiply the second feature point pattern 8-2 of the second projector 2-2 so that the images 5-1 and 5-2 of the feature point images 4-1 and 4-2 (hereinafter sometimes referred to as "feature point images") captured by the 3D scanner 3 are aligned with a reference pattern 7 separately prepared within the angle of view 6 of the 3D scanner 3, and projection image correction units 10-1 and 10-2 that correct the feature point patterns 8-1 and 8-2 using the correction functions F1 and F2.

[0037] 1, the feature point image 4 (feature point pattern 8) is shown in a grid pattern, but as will be described later, it is desirable to use a feature point pattern 8 that makes it easy for the 3D scanner 3 to acquire feature amounts. Also, in the above, the feature point image 5 is aligned with the reference pattern 7, but instead of the feature point pattern 8, a dedicated correction pattern for aligning with the reference pattern 7 may be used.

[0038] The projection image correction unit 10 may be an integrated unit that sends corrected images independently to the first and second projectors 2, or may be provided separately for the first projector 2-1 and the second projector 2-2. The correction function generation unit 9 may be integrated with the projection image correction unit 10 as a function integrated into a device with a calculation function, such as a personal computer, and may send corrected patterns independently to the first and second projectors 2.

[0039] FIG. 2 shows the movement of a photographer 100 using a 3D scanner 3. The photographer 100 moves between the object 1 and the first and second projectors 2, scanning the object 1 onto which the feature point image 4 is projected with the 3D scanner 3 to perform three-dimensional measurement. While the photographer 100 is depicted as a person in FIG. 2, the photographer is not limited to a human as long as it is mobile and can be equipped with a 3D scanner 3. For example, the photographer 100 may be a robot capable of autonomous movement, such as a humanoid robot or a dog-like robot, an AGV (Automated Guided Vehicle), a vehicle, a dolly, a cart, a drone, or the like. Publicly known technology may be used for three-dimensional measurement after scanning the object 1.

[0040] The operation of the systems in FIGS. 1 and 2 will be described according to the procedure shown in FIG. 3. As shown in FIGS. 1 and 2, the first projector 2-1 and the second projector 2-2 have different optical axes. In this embodiment, in order to match the feature point images 4 projected from the first projector 2-1 and the second projector 2-2, a correction pattern 11 dedicated to correction is input in addition to the feature point pattern 8. An example in which a grid pattern is used as the correction pattern 11 is shown, but the pattern shape is not limited to this.

[0041] FIG. 4A is a conceptual diagram of the first projector 2-1 before image correction. FIG. 4B is a conceptual diagram of the first projector 2-1 after image correction. FIG. 5A is a conceptual diagram of the second projector 2-2 before image correction. FIG. 5B is a conceptual diagram of the image of the second projector 2-2 after image correction.

[0042] As shown in FIG. 4A, a correction pattern 11-1 is input to a projection image correction unit 10-1 and a correction function generation unit 9. A correction pattern (sometimes referred to as a "correction image 401-1") projected by a projector 2-1 based on the correction pattern 11-1 is captured by a 3D scanner 3. Meanwhile, the correction function generation unit 9 generates a reference pattern 402-1 based on the correction pattern 11-1. The correction pattern 11-1 may be used as is as the reference pattern 402-1, or may be modified as appropriate.

[0043] Before correcting the correction pattern 11-1, the image of the correction image 401-1 (sometimes referred to as the "correction image 403-1") captured by the 3D scanner 3 is misaligned with the defined reference pattern 402-1. By superimposing the correction image 403-1 on the reference pattern 402-1, a correction function F1 for the first projector 2-1 is generated. An example of the generated correction function will be described below.

[0044] In FIG. 3, first, the positions of the object 1, the optical axis of the first projector 2-1, and the optical axis of the second projector 2-2 are determined (S301).

[0045] The correction pattern 11 is input to the projected image correction unit 10 and the correction function generation unit 9. The correction function generation unit 9 defines and stores a reference pattern 402 based on the input correction pattern 11 (S302).

[0046] The correction pattern 11-1 is projected by the first projector 2-1 and captured by the 3D scanner 3 to obtain a correction image 403-1, and a correction function F1 is calculated that indicates a transformation formula that matches the correction image 403-1 with the reference pattern 402 within the scanner's angle of view 6 (S303).

[0047] The correction function generated by the correction function generation unit 9 can be expressed in various formats, and any expression can be used. For example, it can be expressed as a two-dimensional linear transformation. If the coordinates of a pixel before transformation are (u, v) and the coordinates after transformation are (x, y), the two-dimensional linear transformation can be expressed by the following equation (1).

[0048]

number

[0049] Figure 6 is an image diagram of point correspondences for determining the correction function. For example, the correction function F can be analytically determined from the relationships between the feature points (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the reference pattern 402 defined within the 3D scanner field of view 6 shown in Figure 6 and the feature points (u1, v1), (u2, v2), (u3, v3), and (u4, v4) of the correction image 403, which are combinations of (x1, y1) and (u1, v1), (x2, y2) and (u2, v2), (x3, y3) and (u3, v3), and (x4, y4) and (u4, v4). As a result, the correction function F1 can be determined.

[0050] In Fig. 4A, the correction function F1 obtained by such means is input to the projected image correction unit 10-1, multiplied by the input correction pattern 11-1, and projected by the first projector 2-1. As a result, as shown in Fig. 4B, the reference pattern 402-1 and the correction image 403-1 overlap and match, and the pattern is positioned at the desired position on the object 101.

[0051] As shown in Fig. 5A, the second projector 2-2, correction function generator 9, and projected image corrector 10-2 can similarly calculate a correction function F2 from correction pattern 11-2, and as shown in Fig. 5B, reference pattern 402-2 and correction image 403-2 overlap and match (S304). It is desirable to use the same pattern for correction pattern 11-1 and correction pattern 11-2. Furthermore, since it is desirable that reference pattern 402-1 and reference pattern 402-2 are the same, one of them may be used in common by the two projected image correctors.

[0052] As described above, the correction pattern 11 can be corrected using the reference pattern 402 as a reference. After the correction is complete, any feature point patterns 8-1 and 8-2 are corrected using the correction functions F1 and F2, so that the feature point images 4-1 and 4-2 projected by the first projector 2-1 and the second projector 2-2 overlap and match, as shown in Fig. 1. Thereafter, the 3D scanner 3 scans the object 1 onto which the feature point image 4 is projected, and three-dimensional measurement is performed (S305).

[0053] As the measurement progresses and the 3D scanner 3 moves so that the feature point image 4 falls outside the scanner's angle of view, the positional relationship between the object 1 and the optical axis of the first projector 2-1 and the optical axis of the second projector 2-2 is changed so that the feature point image falls within the scanner's angle of view, and the projected image is corrected in the same way (S306). Alternatively, multiple pairs of the first projector 2-1 and the second projector 2-2 whose projected images have been corrected using a similar procedure may be prepared, and the range over which the feature point image 4 is projected may be expanded.

[0054] By extracting feature points from multiple images obtained by scanning using the method shown in Figure 3 and matching the feature points between the images, it is possible to perform 3D measurement of any range of an object. The method of extracting feature points from multiple images and generating a 3D model itself can be based on conventional technology, but in this embodiment, the number of feature points that can be extracted can be increased by projecting a feature point pattern. Furthermore, by scanning the object closely, high-resolution images can be obtained and differences between images caused by camera shadows can be reduced, enabling high-precision 3D measurement.

[0055] Figure 7 is a projection image diagram of the feature point image 4 after image correction. When feature point patterns 8-1 and 8-2 are input to the projection image correction unit 10 and corrected with correction functions F1 and F2, the feature point images 4-1 and 4-2 projected from the first and second projectors 2 overlap and match on the object 1. In this state, even if the 3D scanner 3 moves in front of the projector 2 to scan the object 1, the effect of the shadow of the 3D scanner can be minimized and the feature pattern can be projected onto the object.

[0056] Although the correction pattern 11 may be used as the feature point image 4, it is preferable to use a feature point pattern such as hiragana or the alphabet, which allows many feature points to be acquired when scanned. That is, in this embodiment, when performing three-dimensional measurement using a 3D scanner, the correction pattern 11 is switched to irradiate the target object 1 with a feature point pattern 8 other than the correction pattern 11.

[0057] In the above embodiment, two projectors are used, but the processing is similar even if three or more projectors are used, and the influence of shadows can be further reduced.

[0058] According to this embodiment, the same pattern can be projected onto an object from at least two projectors, and even if the camera passes in front of the projector, the pattern is prevented from being changed by a shadow, thereby enabling a highly accurate three-dimensional measurement system. [Example]

[0059] The second embodiment is a method for checking and correcting misalignment when the correction pattern 11 is projected by the first and second projectors 2 in an overlapping manner in the first embodiment.

[0060] 8, the correction patterns 11 projected by the first projector 2-1 and the second projector 2-2 are projected in different colors. The area formed by only the first correction pattern 11-1 by the first projector 2-1 is a first color projection pattern 801, the area formed by only the second correction pattern 11-2 by the second projector 2-2 is a second color projection pattern 802, and the area where both overlap is a mixed color projection pattern 803.

[0061] If there is a misalignment, there will be three areas: a first color projection pattern 801 of the first projector, a second color projection pattern 802 of the second projector, and a mixed color projection pattern 803 which is a mixed color of the first color projection pattern 801 and the second color projection pattern 802.

[0062] For example, if the projection pattern color of the first projector is blue and the first color projection pattern 801 is blue, and the projection pattern color of the second projector is red and the second color projection pattern 802 is red, the color of the mixed color projection pattern 803, which is the mixed color, will be magenta.

[0063] Furthermore, if two complementary colors are selected as the two different colors, and the first color projection pattern 801 of the first projector is the first complementary color and the second color projection pattern 802 of the second projector is the second complementary color, the mixed color projection pattern 803 where the complementary colors overlap will be white.

[0064] For example, if the first color projection pattern 801 of the first projector is magenta and its complementary color, green, is used as the second color projection pattern 802 of the second projector, the resulting mixed color is white, resulting in a mixed color projection pattern 803.

[0065] As described above, by changing the colors of the patterns projected by the first and second projectors, the misalignment becomes more apparent visually or chromatically. Once the misalignment is detected, it can be corrected by optimizing either or both of the correction functions F1 and F2 in the same manner as in the first embodiment. [Example]

[0066] 9A to 9C, a description will be given of a third embodiment. In the third embodiment, a specific example of the correction pattern 11 used in the first embodiment will be illustrated.

[0067] FIG. 9A is a grid pattern also shown in the first embodiment, which is considered to be an effective correction pattern that makes it easy to estimate feature points and eliminates arbitrariness in the pattern.

[0068] FIG. 9B is a pattern in which the same type of figures such as polygons are arranged at equal intervals, and is considered to be an effective correction pattern when analytically determining a correction function due to the large number of characteristic points.

[0069] FIG. 9C is a pattern in which black circles or dots are arranged at equal intervals, and is considered to be a correction pattern that makes it easier to estimate the direction of deformation distortion, although it reduces arbitrariness. [Example]

[0070] Example 4 will be described with reference to Figures 10 to 13. This example is an example of a case where a three-dimensional object having a protrusion on the target object 1 is targeted. When a protrusion is present, the patterns projected by the two projectors will differ due to the influence of the shadow of the protrusion.

[0071] 10, if there is a protrusion 1001 on the object 1, a shadow 1002 of the protrusion may affect the feature point image 4. A shadow 1002-1 appears in the feature point image 4-1 of the first projector 2-1, but the shadow 1002-1 does not appear in the feature point image 4-2 of the second projector 2-2. A shadow 1002-2 appears in the feature point image 4-2 of the second projector 2-2, but the shadow 1002-2 does not appear in the feature point image 4-1 of the first projector 2-1.

[0072] In the above state, the feature point image 4-1 of the first projector 2-1 and the feature point image 4-2 of the second projector 2-2 are different, so in this embodiment, the image mask unit 1004 masks the shadow part of the feature point pattern 8 so that the projected images of both projectors match.

[0073] As shown in FIG. 11, a reference pattern 402 is defined within the 3D scanner angle of view 6, and a correction function F1' is obtained.

[0074] 12 is an image diagram of point correspondence for obtaining a correction function according to Example 4. For example, F1' can be obtained by analytically solving Equation (1) from the relationship between the feature points (x5, y5), (x6, y6), (x7, y7), and (x8, y8) of the reference pattern 402 defined within the 3D scanner field of view 6 shown in FIG. 12 and the feature points (u5, v5), (u6, v6), (u7, v7), and (u8, v8) of the correction image 403, which are combinations of (x5, y5) and (u5, v5), (x6, y6) and (u6, v6), (x7, y7) and (u7, v7), and (x8, y8) and (u8, v8).

[0075] Here, the correction pattern 11 is input to the projection image correction unit 10, multiplied by the correction function F1', and projected onto the object 1 by the first projector 2-1. The mask generation unit 1003 generates and stores a first shadow mask pattern 1103-1 corresponding to the image 1102-1 of the shadow 1002-1 created by the first projector 2-1.

[0076] As shown in Fig. 13, a correction function F2' can be obtained in the same manner as the correction function F1'. Here, when the correction pattern 11-2 is input to the projection image correction unit 10-2 and multiplied by the correction function F2' and projected onto the object 1 by the second projector 2-2, the mask generation unit 1003 generates and stores a second shadow mask pattern 1103-2 for the image 1102-2 of the shadow 1002-2 created by the second projector 2-2.

[0077] Using these, the second shadow mask pattern 1103-2 is input to the projection image correction unit 10-1, the input correction pattern 11-1 and the shadow mask pattern 1103-2 are combined and converted using the correction function F1', and the first projector 2-1 projects a pattern that masks the shadow area of ​​the second projector 2-2.

[0078] Similarly, the first shadow mask pattern 1103-1 is input to the projection image correction unit 10-2, and the input correction pattern 11-2 and the masked portion are combined and input to the projection image correction unit 10-2, where they are converted using the correction function F2'.The second projector 2-2 then projects a pattern that masks the shadow portion of the first projector 2-1, thereby projecting the same pattern that masks the shadow 1002 in Figure 10.

[0079] In this state, when the feature point pattern 8 is input to the projection image correction unit 10 for correction and masking, the feature point images 4-1 and 4-2 projected from the first and second projectors 2 overlap and match on the object 1, and the shadow of the object 1 is masked to prevent its influence. [Example]

[0080] Example 5 will be described with reference to Fig. 14. In Example 5, in addition to the three-dimensional measurement system shown in Example 1, the measured three-dimensional measurement data is imported into a CAD (Computer Aided Design) system to estimate the processing dimensions of the extension pipe, etc. According to Example 5, the dimensions of the pipe processing can be clarified using data on the on-site piping, making it possible to process the pipe in a factory and shortening the construction period.

[0081] 14 is a schematic diagram of a computer screen in which three-dimensional measurement data of on-site piping is loaded into a CAD system according to this embodiment. In Example 5, during the piping installation process in a plant, the final connecting piping section is modeled using a three-dimensional measurement system and imported into CAD. This allows the machining dimensions of the final connecting piping to be designed on CAD, allowing precision machining in the factory.

[0082] Piping model 1401 and piping model 1402 are the result of three-dimensionally measuring on-site piping in Example 1 and modeling the results, which have been imported into a CAD system. At intersection 1405 of center lines 1403 and 1404 of piping model 1401 and piping model 1402, angle 1406 formed by the center lines can be determined, and lengths 1407 and 1408 from intersection 1405 to the end faces of the respective pipes can be estimated. By processing and manufacturing the pipes in a factory based on these processing parameters, the construction period can be shortened.

[0083] According to the above embodiment, efficient three-dimensional measurement can be realized, which reduces energy consumption, reduces carbon emissions, prevents global warming, and contributes to the realization of a sustainable society. [Explanation of symbols]

[0084] 1: Object 2: Projector 3: 3D scanner 5: Feature point image 6: Angle of view 7: Reference pattern 8: Minutiae pattern 9: Correction function generator 10: Projection image correction unit 11: Correction pattern

Claims

1. A three-dimensional measurement system using a pattern irradiation device consisting of first and second projectors having different optical axes that irradiate a pattern onto an object, and a 3D scanner that is provided between the object and the pattern irradiation device and captures an image of the object while moving, A correction function generation unit that generates a correction function F1 to be multiplied by a first pattern input to the first projector so as to align the pattern imaged within the angle of view of the 3D scanner with a separately prepared reference pattern, and a correction function F2 to be multiplied by a second pattern input to the second projector; a projection image correcting unit that corrects the first pattern using the correction function F1 and corrects the second pattern using the correction function F2; projecting the corrected first pattern onto the object using the first projector, projecting the corrected second pattern onto the object using the second projector, and projecting images of the first pattern and the second pattern onto the object in a superimposed manner; Three-dimensional measurement system.

2. In order to generate the correction function F1 and the correction function F2, a dedicated correction pattern is used as the pattern to be imaged within the angle of view of the 3D scanner, During three-dimensional measurement by the 3D scanner, a pattern other than the correction pattern is switched to be irradiated onto the object. The three-dimensional measurement system according to claim 1.

3. a color of the image projected from the first projector and a color of the image projected from the second projector that are different from each other are used to generate the correction function F1 and the correction function F2; The three-dimensional measurement system according to claim 1.

4. In order to generate the correction function F1 and the correction function F2, colors that are complementary to each other are used as the color of the image projected from the first projector and the color of the image projected from the second projector. The three-dimensional measurement system according to claim 3.

5. The correction pattern includes a repeating pattern of the same figure. The three-dimensional measurement system according to claim 2.

6. The repeating pattern is a grid pattern. The three-dimensional measurement system according to claim 5.

7. The repeating pattern is a pattern in which the same type of figure is arranged at equal intervals. The three-dimensional measurement system according to claim 5.

8. The repeating pattern is a pattern in which black circles or dots are arranged at equal intervals. The three-dimensional measurement system according to claim 5.

9. the correction function generating unit generates the reference pattern using at least one of the first pattern and the second pattern. The three-dimensional measurement system according to claim 1.

10. As the other pattern, at least one of hiragana and alphabet is used. The three-dimensional measurement system according to claim 2.

11. a mask generating unit that generates a mask that masks a shadow cast by the object on the pattern that is projected onto the object, the projection image correcting unit masks the first pattern and the second pattern with the mask; The three-dimensional measurement system according to claim 1.

12. A three-dimensional measurement apparatus for inputting a pattern into a pattern irradiation device including first and second projectors having different optical axes that irradiate a pattern onto an object in order to perform three-dimensional measurement with a 3D scanner that is provided between the object and the pattern irradiation device and captures an image of the object while moving, A correction function generation unit that generates a correction function F1 to be multiplied by a first pattern input to the first projector so as to align the pattern imaged within the angle of view of the 3D scanner with a separately prepared reference pattern, and a correction function F2 to be multiplied by a second pattern input to the second projector; a projection image correcting unit that corrects the first pattern using the correction function F1 and corrects the second pattern using the correction function F2; projecting the corrected first pattern onto the object using the first projector, projecting the corrected second pattern onto the object using the second projector, and projecting images of the first pattern and the second pattern onto the object in a superimposed manner; Three-dimensional measuring device.

13. The first step is to prepare the 3D scanner. a second step of determining a positional relationship between the first projector and the second projector, which have different optical axes from each other, and the object; a third step of defining a reference pattern on the object at a predetermined position within the field of view of the 3D scanner; A fourth step of projecting a first correction pattern using the first projector and obtaining a correction function F1 that matches a reference pattern within the angle of view of the 3D scanner; A fifth step of projecting a second correction pattern using the second projector and obtaining a correction function F2 that matches a reference pattern within the angle of view of the 3D scanner; a sixth step of projecting the feature point pattern transformed by the correction function F1 using the first projector and projecting the feature point pattern transformed by the correction function F2 using the second projector; a seventh step of capturing an image of the object and performing three-dimensional measurement while moving the 3D scanner between the object and the first projector and the second projector; A three-dimensional measurement method for performing

14. the first correction pattern and the second correction pattern are the same pattern and include a repeated pattern of the same figure; The three-dimensional measurement method according to claim 13.

15. the feature point pattern is different from the first and second correction patterns; The three-dimensional measurement method according to claim 13.

Images