Three-dimensional measurement method, three-dimensional measurement device, and three-dimensional measurement system

Abstract

To provide a three-dimensional measurement system that can perform three-dimensional measurements while moving a 3D scanner, while reducing the influence of pattern obstruction, in three-dimensional measurements in which an object with few visual features is photographed while a pattern is projected onto it.MEANS FOR SOLVING THE PROBLEM: A preferred aspect of the present invention is a three-dimensional measurement system having: a pattern projection device including a first projector that projects a first pattern onto an object and a second projector that projects a second pattern onto the object; a 3D scanner that is provided between the object and the pattern projection device and captures images of the object while moving to generate projected images; and a three-dimensional model generation device that uses the projected image of the second projector when the 3D scanner is located between the first projector and the object, and uses the projected image of the first projector when the 3D scanner is located between the second projector and the object to output a three-dimensional model.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to three-dimensional measurement technology. [Background technology]

[0002] In plant construction and maintenance, building construction and maintenance, vehicle manufacturing, and other fields, the use of 3D measuring machines is expected to be used to conduct on-site surveys before work begins and to draft work plans and drawings. For example, in piping installation, design drawings for the piping are prepared before construction, but there is a significant discrepancy between the piping installed on-site and the design drawings. This requires measuring the on-site situation using a 3D measuring machine, revising the construction drawings based on the measurement results, and implementing construction procedures that reflect the results in the construction. This requires high-precision 3D measurement of piping.

[0003] When taking 3D measurements of structures with complex, intricate equipment, such as piping, it is necessary to measure the object from various directions, so a highly mobile handheld 3D scanner (hereinafter referred to as a 3D scanner or camera) is effective.Handheld 3D scanners use the SfM (Structure from Motion) method to stitch together images (RGB images, depth images, infrared images, point clouds, etc.) taken continuously while moving to reconstruct the 3D shape.

[0004] In SfM's internal processing, feature points are extracted from each image and similar feature points in each image are matched to estimate the camera position when the image was taken, and the 3D shape of the object is reconstructed based on this result. However, for objects with few visual features, such as pipes, it may not be possible to extract feature points from the image, and the internal processing of 3D measurement may fail. In such cases, it is effective to project a feature pattern onto the object using a projector while taking the image, thereby adding visual features and helping to ensure successful 3D measurement.

[0005] However, when the 3D scanner moves between the projector and the object, the 3D scanner itself may block the projection. If the pattern is blocked, there is a high possibility that 3D measurement will fail for objects with few visual features, such as pipes. A common countermeasure to block occlusion is to project the pattern using multiple projectors to cancel out the shadows caused by occlusion.

[0006] In this regard, Patent Document 1 discloses an image processing device that projects vertical and horizontal patterns from two projectors, respectively, to measure the shape of a moving object at a high density and a high frame rate, and reconstructs the three-dimensional shape based on the projected patterns.

[0007] Patent Document 2 discloses a control device that identifies a shadow area and corrects the projection light in order to easily eliminate the shadow of an object when projecting an image using multiple projectors. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-242183 [Patent Document 2] WO 2019 / 009100 A1 Summary of the Invention [Problem to be solved by the invention]

[0009] With conventional technology, when taking 3D measurements of an object with few visual features while projecting a pattern onto it, it is not possible to perform 3D measurements while moving the 3D scanner without worrying about the pattern being blocked.

[0010] For example, Patent Document 1 describes that using multiple projectors can significantly reduce the amount of occlusion, but does not describe specific countermeasures for when occlusion occurs. Patent Document 2 corrects the projected light to eliminate shadows on the assumption that the layout information representing the layout of the projector and the target object is known, but since the shape of the target object is unknown in the first place, it cannot be used in construction use cases where three-dimensional measurement is desired.

[0011] Based on the above, the object of the present invention is to provide a three-dimensional measurement system that can perform three-dimensional measurements while moving a 3D scanner, reducing the effects of pattern obstruction, in three-dimensional measurements in which a pattern is projected onto an object with few visual features and photographed. [Means for solving the problem]

[0012] A preferred aspect of the present invention is a three-dimensional measurement system having a pattern projection device including a first projector that projects a first pattern onto an object and a second projector that projects a second pattern onto an object; a 3D scanner that is provided between the object and the pattern projection device and that captures images of the object while moving to generate projected images; and a three-dimensional model generation device that uses the projected image of the second projector when the 3D scanner is located between the first projector and the object, and that uses the projected image of the first projector when the 3D scanner is located between the second projector and the object to output a three-dimensional model.

[0013] Another preferred aspect of the present invention is a three-dimensional measuring apparatus that projects a first pattern from a first projector onto an object, projects a second pattern from a second projector onto the object, and generates a three-dimensional model using the projected images from a 3D scanner that captures the object while moving and generates projected images, the three-dimensional measuring apparatus comprising: a three-dimensional model generation device that generates the three-dimensional model using the projected image of the second projector when the 3D scanner is located between the first projector and the object, and using the projected image of the first projector when the 3D scanner is located between the second projector and the object.

[0014] Another preferred aspect of the present invention is a three-dimensional measurement method that executes a first step of projecting a first pattern from a first projector onto an object, a second step of projecting a second pattern from a second projector onto the object, a third step of capturing an image of the object while moving with a 3D scanner to generate a projected image, and a fourth step of performing three-dimensional measurement using the projected image of the second projector if the 3D scanner is between the first projector and the object, or using the projected image of the first projector if the 3D scanner is between the second projector and the object. [Effects of the Invention]

[0015] According to the present invention, in a three-dimensional measurement in which an object with few visual features is photographed while a pattern is projected onto it, the three-dimensional measurement can be performed while moving the 3D scanner while reducing the influence of pattern occlusion. Note that problems, configurations, and effects other than those described above will become clear from the following description of the embodiment of the invention. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 2 is a perspective view showing the actions of a photographer using the three-dimensional measurement system according to the first embodiment. [Figure 2] FIG. 2 is an image diagram showing an example of an image obtained by photographing an object according to the first embodiment. [Figure 3]FIG. 3 is an image diagram showing an example of an image obtained by capturing an object from which a projection pattern is blocked according to the first embodiment. [Figure 4] FIG. 2 is an image diagram showing an example of a generated three-dimensional model according to the first embodiment. [Figure 5] 1 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system according to a first embodiment. [Figure 6] FIG. 3 is a flowchart showing an example of processing during imaging by the three-dimensional measurement system according to the first embodiment. [Figure 7] FIG. 3 is a flowchart showing a processing example when generating a three-dimensional model of the three-dimensional measurement system according to the first embodiment. [Figure 8] FIG. 3 is a flowchart showing an example of a process in which a three-dimensional model generating unit according to the first embodiment generates a three-dimensional model. [Figure 9] 1 is a schematic diagram showing a series of steps from photographing to generating a three-dimensional model according to the first embodiment. [Figure 10A] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10B] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10C] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10D] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10E] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10F] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10G] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 10H] FIG. 2 is a schematic diagram showing the positional relationship between a 3D scanner and a projector according to the first embodiment. [Figure 11] 3 is a schematic diagram showing the inclusion relationship between a projected image and an unobstructed projected image according to the first embodiment. FIG. [Figure 12] FIG. 10 is an image diagram showing an example of an object having a homogeneous pattern and therefore few visual features according to Example 2. [Figure 13]FIG. 10 is an image diagram showing an example in which a pattern is projected onto an object having a uniform appearance and therefore few visual features, according to the second embodiment. [Figure 14] FIG. 10 is an image diagram showing an example of an object with few visual features according to the second embodiment. [Figure 15] FIG. 10 is a graph showing an example of the number of matching pairs obtained from an image of an object with few visual features according to the second embodiment. [Figure 16] FIG. 11 is an image diagram showing an example of an object having few visual features due to a continuous pattern of the same object according to the third embodiment. [Figure 17] FIG. 11 is a graph showing an example of the number of matching pairs obtained from an image of an object having a continuous identical pattern and therefore few visual features, according to the third embodiment. [Figure 18] FIG. 10 is a process flow diagram for classifying a projected pattern in object detection according to the fourth embodiment. [Figure 19] FIG. 11 is a process flow diagram for classifying a projected pattern by color judgment according to the fourth embodiment. [Figure 20] FIG. 10 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having a time synchronization unit according to a fifth embodiment. [Figure 21] FIG. 11 is a flowchart showing an example of processing during imaging by a three-dimensional measurement system having a time synchronization unit according to the fifth embodiment. [Figure 22] FIG. 11 is a flowchart showing an example of a process performed when a model is generated in a three-dimensional measurement system having a time synchronization unit according to the fifth embodiment. [Figure 23] FIG. 20 is a process flow diagram for determining whether a projection pattern is blocked by object detection according to the sixth embodiment. [Figure 24] FIG. 20 is an image diagram showing a first example of dividing a projection pattern into small patterns according to the sixth embodiment. [Figure 25] FIG. 20 is an image diagram showing a second example of dividing a projection pattern into small patterns according to the sixth embodiment. [Figure 26] FIG. 13 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having an occlusion detection camera according to a seventh embodiment. [Figure 27] FIG. 13 is an image diagram showing an example of an occlusion determination image captured by the occlusion determination camera according to the seventh embodiment. [Figure 28] FIG. 13 is a process flow diagram during imaging of a three-dimensional measurement system having an occlusion detection camera according to a seventh embodiment. [Figure 29] FIG. 13 is an explanatory diagram of an additionally installed pipe according to the eighth embodiment. [Figure 30] FIG. 13 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having a processing parameter calculation device according to an eighth embodiment. [Figure 31] FIG. 13 is an explanatory diagram showing a procedure for calculating processing parameters for a pipe to be additionally installed according to the eighth embodiment. [Figure 32] FIG. 13 is a flowchart showing a procedure for calculating processing parameters according to the eighth embodiment. [Figure 33] FIG. 13 is a schematic diagram showing a case where projection is obstructed by the shape of the object itself, according to the ninth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments are illustrative for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0018] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0019] Examples of various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0020] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0021] In the embodiments, there may be cases where processing performed by executing a program is described. Here, a computer executes the program using a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)), and performs processing defined by the program while using storage resources (e.g., memory), interface devices (e.g., communication ports), etc. Therefore, the processor may be the entity that executes the program and performs the processing.

[0022] Similarly, the entity that executes the program and performs the processing may be a controller, device, system, computer, or node having a processor. The entity that executes the program and performs the processing may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0023] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0024] Because the present invention encompasses a wide range of content, an overview of each embodiment will be provided below. First, in Example 1, three-dimensional measurement is described in which two projectors project different patterns onto an object with few visual features, a photographer photographs the object while moving with a 3D scanner, and a three-dimensional model is generated using the images in which the patterns are not occluded. In Example 2, a specific example of an object with few visual features is described. In Example 3, a specific example of an object with a pattern consisting of successive identical patterns is described. In Example 4, a method of classifying projected patterns by image processing of the projected images is described. In Example 5, a method of classifying projected patterns by time synchronization is described. In Example 6, a method of determining whether a pattern is occluded by image processing of the projected images is described. In Example 7, a method of determining whether a pattern is occluded using an occlusion determination camera is described. In Example 8, a method of calculating processing parameters for processing a component from a three-dimensional model measured by a 3D scanner, design information, and component information is described.

[0025] One example of an embodiment is a three-dimensional measurement system having a pattern projection device consisting of a first projector that projects a first pattern onto an object and a second projector that projects a second pattern; a 3D scanner that is provided between the object and the pattern projection device and captures images of the object while moving; and a three-dimensional model generation device that outputs a three-dimensional model using the projected image of the second projector when the 3D scanner is located between the first projector and the object, and using the projected image of the first projector when the 3D scanner is located between the second projector and the object. [Example]

[0026] Example 1 will be described with reference to Figures 1 to 11. Example 1 describes three-dimensional measurement in which different patterns are projected onto an object with few visual features using two projectors, a photographer photographs the object while moving with a 3D scanner, and a three-dimensional model is generated using images in which the patterns are not occluded. According to Example 1, in three-dimensional measurement in which a pattern is projected onto an object with few visual features and photographed, the three-dimensional measurement can be performed while moving the 3D scanner without worrying about occlusion of the pattern.

[0027] FIG. 1 is a diagram showing the operation of a photographer using a three-dimensional measurement system according to a first embodiment of the present invention. In the first embodiment, an object 101 is three-dimensionally measured and a three-dimensional model of the object 101 is generated. A photographer 102 carries a 3D scanner 103 and scans the object 101 while moving. Although the photographer 102 is illustrated as a human in FIG. 1, the photographer 102 is not limited to a human as long as it is mobile and can be equipped with the 3D scanner 103. For example, the photographer 102 may be a humanoid robot, a dog-type robot, a vacuum cleaner-type robot, an AGV (Automated Guided Vehicle), a vehicle, a dolly, a cart, a wagon, a drone, or the like.

[0028] The 3D scanner 103 is a device capable of capturing an image of the object 101 and generating a two-dimensional image or a three-dimensional image (hereinafter, the two-dimensional image and the three-dimensional image may be referred to as an image without distinguishing between them), and possible devices include an optical camera, an infrared camera, a stereo camera, a ToF (Time Of Flight) sensor, and a LiDAR (Light Detection And Ranging).

[0029] The imaging range 104 indicates the range that can be captured with the angle of view of one image captured by the 3D scanner 103. Because the object 101 is generally larger than the imaging range 104, the photographer 102 moves around the object 101 while capturing images, scanning the imaging range 104. A three-dimensional model of the object 101 is generated by combining multiple images obtained during the scan. The details of generating a three-dimensional model from images captured by the 3D scanner 103 will not be described in detail here, as publicly known techniques will be used.

[0030] Here, consider a case where the object 101 has few distinctive patterns or shapes. In generating a three-dimensional model, as will be described later with reference to FIG. 8, the camera movement amount is estimated by matching feature points contained in the image, and the three-dimensional shape is restored. Therefore, if the object 101 has few visually distinctive patterns or shapes, it is not possible to extract feature points, resulting in a failure to estimate the camera movement amount or a deterioration in the accuracy of the camera movement estimation. Therefore, by projecting the first projected pattern 107 or the second projected pattern 108 onto the object 101 using the first projector 105 or the second projector 106, the number of feature points on the object can be increased, improving the accuracy of the camera movement estimation, and ultimately enabling highly accurate three-dimensional measurement.

[0031] The first projection pattern 107 and the second projection pattern 108 may be patterns with different designs or pictures. There is no need to align them and project the same patterns at the same position. It is desirable that the projection locations of the first projection pattern 107 and the second projection pattern 108 do not move while the 3D scanner 103 moves to capture an image of the object. It is desirable that the first projector 105 and the second projector 106 do not move during capture.

[0032] The first projection pattern 107 and the second projection pattern 108 are projected in a time-division manner. In other words, the two types of patterns are not projected simultaneously, but are projected by alternating between them. They may be partially projected simultaneously, but there will be times when only one of the patterns is projected. For example, the first projector 105 and the second projector 106 can be alternately switched between projecting and not projecting at 0.2-second intervals, thereby enabling time-division projection. Hereinafter, time-division may be referred to as time-division.

[0033] The first projector 105 or the second projector 106 may be an LCD (Liquid Crystal Display) projector, a DLP (Digital Light Processing) projector, an LCOS (Liquid Crystal On Silicon) projector, a laser projection mapping, a flexible display, an OHP (Overhead Projector), or the like.

[0034] Here, when the 3D scanner 103 moves, the 3D scanner 103 or the photographer 102 may block the projection. Figure 1 illustrates a situation in which the photographer 102 blocks the projection from the first projector 105, and the first projection pattern 107 (picture of a cat) is not projected onto the blocked portion 109 of the first projection pattern.

[0035] In generating a three-dimensional model, a group of captured images is integrated to reconstruct a three-dimensional shape according to the flow described later in FIG. 8 . It is undesirable for the group of images to include an image in which an occluded portion 109 of the first projection pattern is present, as this will result in inconsistencies in the calculation process for reconstructing the three-dimensional shape from the group of images. In other words, the occluded image becomes a noise image that reduces the accuracy of the three-dimensional shape reconstruction. Therefore, it is desirable to remove images in which the pattern is occluded from the group of captured images before reconstructing the three-dimensional shape. In Example 1, a procedure for determining and removing occluded images and then reconstructing the three-dimensional shape will be described.

[0036] Although FIG. 1 illustrates a situation in which the first projection pattern 107 is blocked by the photographer 102, the same applies to a situation in which the second projection pattern 108 is blocked.

[0037] FIG. 2 is a diagram showing an example of an image of an object according to the first embodiment of the present invention. As an example of the object 101, a cylindrical object is illustrated. The surface of the object 101 has a flat pattern and few visual features. Furthermore, since the shape of the object 101 is cylindrical, it is symmetrical with respect to the central axis (axial symmetry), and even if the periphery of the object 101 is photographed with a camera, it is difficult to distinguish from which angle the photograph was taken. Such an object 101 has few visual features and is difficult to measure in three dimensions.

[0038] 3 is a diagram showing an example of an image captured of an object with a projected pattern blocked according to Example 1 of the present invention. Projecting a first projected pattern 107 onto the object 101 imparts a visual feature to the surface of the object 101. However, it can be seen that a portion of the first projected pattern 107 is blocked, and the pattern is missing from a blocked portion 109 of the first projected pattern.

[0039] 4 is a diagram showing an example of a generated three-dimensional model according to the first embodiment of the present invention. The three-dimensional model 150 is electronic data representing three-dimensional shape information of the object 101. Here, the three-dimensional model 150 does not include surface pattern information (texture), so the first projection pattern 107 is not shown, but the three-dimensional model 150 may include pattern information. Possible formats for the electronic data include OBJ, FBX, GITF, GLB, USD / USDZ, STL, PLY, 3D-CAD, etc.

[0040] FIG. 5 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system according to a first embodiment of the present invention. A pattern projection device 130 including at least a first projector 105 and a second projector 106 projects a pattern onto an object 101. The number of projectors is not limited to two, and may be three or more. A 3D scanner 103 captures an image of the object 101 together with the projected pattern while moving. An image captured by the 3D scanner 103 while the pattern projection device 130 is projecting is hereinafter referred to as a projected image 200. A three-dimensional model generation device 1000 issues a projection command to the pattern projection device 130 and receives the projected image 200 from the 3D scanner 103.

[0041] The three-dimensional model generation device 1000, the 3D scanner 103, and the pattern projection device 130 may be connected by a wired or wireless method. Examples of wired methods include a wired connection method using a wired transmission standard such as USB (trademark), Ethernet (trademark), HDMI (registered trademark), Video Graphics Array (VGA (trademark), and DVI (Digital Visual Interface). Examples of wireless methods include a wireless connection method using a wireless communication standard such as Wi-Fi (trademark), Bluetooth (trademark), and Zigbee (trademark). The three-dimensional model generation device 1000, the 3D scanner 103, and the pattern projection device 130 may also be connected via a network not shown in FIG. 5.

[0042] The three-dimensional model generating device 1000 stores the first projection pattern 107 and the second projection pattern 108 projected by the pattern projection device 130. The first projection pattern 107 and the second projection pattern 108 may be stored in a storage area built into the pattern projection device 130. The location of the data is arbitrary as long as the projection patterns can be transmitted to the pattern projection device 130.

[0043] The following describes the internal processing of the three-dimensional model generating device 1000. The projection image classification unit 1001 classifies the projection image 200 as either an image captured when the first projection pattern 107 is projected (hereinafter referred to as the first projection image 201) or an image captured when the second projection pattern 108 is projected (hereinafter referred to as the second projection image 202).

[0044] The first occlusion determination unit 1002 determines whether the first projection image 201 is an image in which the first projection pattern 107 is projected without being occluded (hereinafter referred to as a first unoccluded projection image 203). The second occlusion determination unit 1003 determines whether the second projection image 202 is an image in which the second projection pattern 108 is projected without being occluded (hereinafter referred to as a second unoccluded projection image 204). The three-dimensional model generation unit 1004 generates a three-dimensional model 150 using the first unoccluded projection image 203 and the second unoccluded projection image 204.

[0045] 6 is a flow diagram showing an example of processing during imaging by the three-dimensional measurement system according to the first embodiment of the present invention. First, the pattern projection device 130 starts projecting the first projection pattern 107 and the second projection pattern 108 onto the object 101 (S601). As described above, the first projection pattern 107 and the second projection pattern 108 are projected in a time-division manner.

[0046] Next, the 3D scanner 103 captures the projected object 101 and stores the projected image 200 in a projected image memory (S602). The projected image memory is a storage area for temporarily saving the projected image, and may be located inside the 3D scanner 103 or inside the three-dimensional model generation device 1000.

[0047] Next, if all of the locations of the object 101 to be measured have been photographed, the photographing ends (S603 yes). If the photographing is not complete, the 3D scanner 103 is moved (S604) and the object 101 is photographed from another position. The object 101 to be measured in three dimensions is generally a solid object including a three-dimensional structure, and therefore is photographed from various angles, such as the front, side, and back.

[0048] Fig. 7 is a flow diagram showing an example of processing when a three-dimensional model is generated by the three-dimensional measurement system according to the first embodiment of the present invention. When a three-dimensional model is generated, the photographing processing shown in Fig. 6 has been completed. As shown in Fig. 6, the projection images 200 captured during photographing are stored in a projection image memory, and processing is performed while reading out the projection images 200 one by one from the projection image memory (S701).

[0049] Steps S2001 and S2002 correspond to the processing performed by the projection image classification unit 1001. Step S2003 corresponds to the processing performed by the first occlusion determination unit 1002. Step S2004 corresponds to the processing performed by the second occlusion determination unit 1003. Step S2005 corresponds to the processing performed by the three-dimensional model generation unit 1004.

[0050] According to this flow, an image captured without the first projection pattern 107 being occluded (first unoccluded projection image 203) is stored in a first unoccluded projection image list (S702), and an image captured without the second projection pattern 108 being occluded (second unoccluded projection image 204) is stored in a second unoccluded projection image list (S703). The first unoccluded projection image list and the second unoccluded projection image list are each a storage area for temporarily saving images.

[0051] It is confirmed whether all the projection images have been read (S704), and then, from the projection images stored in the first unobstructed projection image list and the second unobstructed projection image list, only the images captured without being occluded are extracted from all the projection images 200, and the three-dimensional model generation unit 1004 generates a three-dimensional model (S2005).

[0052] 8 is a flowchart showing an example of processing in which the three-dimensional model generating unit 1004 according to the first embodiment of the present invention generates a three-dimensional model. The internal processing of step S2005 in FIG. 7 will be specifically described with reference to FIG.

[0053] The feature point extraction unit 1201 analyzes the input first unoccluded projection image 203 and second unoccluded projection image 204, and extracts feature points that have a distinctive pattern in the image and can serve as landmarks that are easily distinguishable from other points. Possible feature point extraction algorithms include Harris corner detection, Shi-Tomasi corner detection, SIFT (Scale-Invariant Feature Transformation), SURF (Speeded-Up Robust Features), FAST (Features from Accelerated Segment Test), BRIEF (Binary Robust Independent Elementary Features), ORB (Oriented FAST and Rotated BRIEF), NN (Neural Network), and SuperPoint.

[0054] The feature point matching unit 1202 compares feature points extracted from two images and matches feature points that indicate the same physical location, thereby outputting a feature point matching set. Possible matching algorithms include nearest neighbor search, kd-tree, brute force matching, comparison of Euclidean distances of features calculated for each feature point, SuperGlue, LightGlue, etc.

[0055] The camera movement amount estimation unit 1203 calculates the amount of camera movement when two images were captured using the feature point matching set. Calculating the amount of movement corresponds to finding a 3x3 rotation matrix and a 3x1 translation vector that represent the rotation and translation of the camera, or finding a 4x4 homogeneous transformation matrix that combines the rotation and translation. Possible algorithms for calculating the amount of movement include ICP (Iterative Closest Point) and RANSAC (Random Sample Consensus).

[0056] The three-dimensional shape restoration unit 1204 restores the three-dimensional shape of the object 101 by overlaying images using the camera movement amount, and outputs a three-dimensional model 150. A technical field that performs the functions of the feature point extraction unit 1201, feature point matching unit 1202, camera movement amount estimation unit 1203, and three-dimensional shape restoration unit 1204 all at once is sometimes called three-dimensional restoration, photogrammetry, or SfM (Structure from Motion). Possible three-dimensional restoration algorithms include SLAM (Simultaneous Localization and Mapping) and SfM-MVS (Structure from Motion / Multi View Stereo).

[0057] Although various algorithms have been mentioned, each algorithm is calculated on the premise that feature points are extracted from an image. If the object 101 has few visual features or shapes, the accuracy of each algorithm is likely to deteriorate.

[0058] Since the feature points that can be acquired differ between the first unobstructed projection image 203 and the second unobstructed projection image 204, in one example, part of the processing in the feature point matching unit 1202, the camera movement amount estimation unit 1203, and the three-dimensional shape restoration unit 1204 is performed separately for the first unobstructed projection image 203 and the second unobstructed projection image 204. The three-dimensional models generated separately for the first unobstructed projection image 203 and the second unobstructed projection image 204 are referred to as partial models. The partial models are composited to generate a final three-dimensional model (sometimes referred to as an overall model).

[0059] In this embodiment, the three-dimensional shape restoration unit 1204 performs a synthesis process for the partial models. There are several possible methods for the synthesis process. For example, once two partial models are created, they are matched using the shape information and visual information of the partial models. Specifically, using a GUI (Graphical User Interface), which is a common function of known three-dimensional model software, the user uses a mouse to select pairs of points on the surfaces of the two partial models that indicate "this and this are the same point." Instead of a human performing the matching, it can also be performed automatically using known image matching.

[0060] To facilitate matching the partial models to synthesize an overall model, it is desirable that the partial models are generated so that at least a portion of the two partial models overlap (area 2000 surrounded by a dotted line in Fig. 9). In other words, it is desirable that the three-dimensional model created from the first unobstructed projection image 203 and the three-dimensional model created from the second unobstructed projection image 204 to be synthesized have substantially the same portions.

[0061] Specifically, as shown in Figures 9 and 10A to 10H later, during imaging by the 3D scanner 103, at least one of the two projectors is always projecting without being occluded, a partial model can be generated from the first or second unoccluded projection image that can be projected, and both projectors are not occluded, allowing a three-dimensional model to be generated, and there is a time period during which a partial model can be generated from both the first and second unoccluded projection images. This condition can be achieved without any operational problems by installing the two projectors in appropriate positions. Note that during a time period during which a partial model can be generated from both the first and second unoccluded projection images, both partial models should basically be the same, so in a simple example, one of them can be prioritized and adopted.

[0062] 9 is a schematic diagram showing a series of steps from capturing an image to generating a three-dimensional model according to the first embodiment of the present invention. The 3D scanner 103 captures an image of the object 101 while moving. While the 3D scanner is moving, it may block the projections of the first projector 105 and the second projector 106. The first projector 105 and the second projector 106 project patterns in a time-division manner. Therefore, the projection images 200 captured by the 3D scanner 103 are stored as images in which the first projection pattern 107 and the second projection pattern 108 are alternately projected.

[0063] First, alternately captured projection images 200 are divided and classified into first projection images 201 and second projection images 202. This classification is performed by the projection image classification unit 1001, and corresponds to the processing of steps S2001 and S2002.

[0064] Next, occluded images are determined from among the images and are not used to generate the 3D model. This determination is performed by the first occlusion determination unit 1002 and the second occlusion determination unit 1003, and corresponds to the processing of steps S2003 and S2004. The non-occluded images are stitched together to generate the 3D model 150.

[0065] By using two projectors, even if one projector is occluded, the pattern from the other projector can be projected without being occluded. Taking advantage of this, we extract images captured when at least one pattern is projected without being occluded and generate a 3D model. This makes it possible to eliminate noise images where the pattern is occluded, improving the accuracy of 3D shape reconstruction.

[0066] 10A to 10H are schematic diagrams showing the positional relationship between a 3D scanner and a projector according to Example 1 of the present invention. The 3D scanner 103 photographs an object 101 while moving from the left side to the right side of the drawing.

[0067] In Figure 10A, the first projector 105 is projecting and the 3D scanner 103 is not blocking the projection. In FIG. 10B, the second projector 106 is projecting and the 3D scanner 103 is not blocking the projection. 10C, the first projector 105 is projecting, and the 3D scanner 103 is blocking the projection. The projection image 200 captured at this time is blocked and is excluded from the input for generating the 3D model.

[0068] In FIG. 10D, the second projector 106 is projecting and the 3D scanner 103 is not blocking the projection. In FIG. 10E, the first projector 105 is projecting and the 3D scanner 103 is not blocking the projection. In Figure 10F, the second projector 106 is projecting, and the 3D scanner 103 is blocking the projection. The projection image 200 taken at this time is blocked and is excluded from the input for generating the 3D model.

[0069] In Figure 10G, the first projector 105 is projecting and the 3D scanner 103 is not blocking the projection. In Figure 10H, the second projector 106 is projecting and the 3D scanner 103 is not blocking the projection.

[0070] 11 is a schematic diagram showing the inclusion relationship between a projected image and an unobstructed projected image according to Example 1 of the present invention. The figure shows a projected image 200 obtained when a cylindrical object 101 is photographed by a 3D scanner 103 while a first projected pattern 107 (a picture of a cat) and a second projected pattern 108 (a picture of a dog) are projected onto the object in a time-division manner.

[0071] Among the projected images 200, images captured during projection by the first projector 105 are classified as first projected images 201. The first projected images 201 include a first unobstructed projected image 203 captured when the pattern is not obscured, and a first obscured projected image 205 captured when the pattern is obscured.

[0072] Among the projected images 200, images captured while the second projector 106 is projecting are classified as second projected images 202. The second projected images 202 include a second unobstructed projected image 204 captured when the pattern is not obstructed, and a second obstructed projected image 206 captured when the pattern is obstructed. [Example]

[0073] Example 2 will be described with reference to Figures 12 to 15. In Example 2, a specific example of an object with few visual features will be described.

[0074] FIG. 12 is a diagram showing an example of an object having few visual features due to a uniform-looking pattern according to Example 2 of the present invention. Two projection images 200 captured before and after the 3D scanner is moved laterally are shown in FIG. 12. It can be seen that the two projection images 200 are images with a uniform appearance. The dots in the figure represent feature points extracted by the feature point extraction unit 1201, and the lines in the figure represent pairs of feature points for which matching is achieved by the feature point matching unit 1202. It can be seen that the number of extracted feature points is small because the surface of the object 101 has few visual features. The number of matches achieved is also small. Using such an image is likely to result in a failure of three-dimensional measurement.

[0075] 13 is a diagram showing an example of projecting a pattern onto an object having a homogeneous appearance and therefore few visual features, according to Example 2 of the present invention. A pattern is projected onto the object 101. Compared to the case of FIG. 12, it can be seen that many feature points are extracted from the surface of the object 101, and many matches are obtained.

[0076] FIG. 14 is a diagram showing examples of objects with few visual features according to the second embodiment of the present invention. Wallpapered walls, cabinets, cushioning materials, and pipes are examples of objects with few visual features. There are other materials with few visual features as well. It is difficult to obtain feature points and matching for these materials without pattern projection, but it can be seen that many feature points and matching can be obtained with pattern projection.

[0077] Fig. 15 is a diagram showing an example of the number of matching pairs obtained from an image of an object with few visual features according to Example 2 of the present invention. For the material shown in Fig. 14, it can be seen that the number of matching pairs obtained increases significantly due to the presence of pattern projection. [Example]

[0078] Example 3 will be described with reference to Figures 16 and 17. In Example 3, a specific example of an object having a pattern in which the same pattern is repeated will be described.

[0079] FIG. 16 is a diagram showing an example of an object with few visual features due to a continuous pattern of the same object according to Example 3 of the present invention. FIG. 16 shows two projection images 200 captured before and after the 3D scanner was moved laterally. The object 101 is a tiled wall surface with a pattern of black lines on a white background. Although there are high-contrast pixels at intersections and boundaries due to the black lines, the continuous pattern makes it difficult to capture the correspondence between feature points before and after the 3D scanner was moved. On the other hand, when a pattern was projected, it was found that many matching pairs with feature points were obtained using the pattern, regardless of the material's pattern.

[0080] 17 is a diagram showing an example of the number of matching pairs obtained from an image of an object with few visual features due to a continuous pattern of the same object, according to Example 3 of the present invention. It can be seen that the number of matching pairs obtained for tiles is significantly increased by the presence of pattern projection. [Example]

[0081] A fourth embodiment will be described with reference to Figures 18 and 19. In the fourth embodiment, a method for classifying projected patterns by performing image processing on the projected images will be described.

[0082] Fig. 18 is a process flow diagram of classifying a pattern projected in object detection according to Example 4 of the present invention. This corresponds to the process performed in step S2001 of the flow in Fig. 7. Fig. 18 illustrates the process of loading a first projection pattern (S1801) and processing the first projection pattern, assuming the process of step S2001. However, if the same process is performed for the second projection pattern, the process will be performed in step S2002.

[0083] In step S2006, it is determined whether the first projection pattern 107 is reflected in the loaded projection image 200 by comparing the projection image with the first projection pattern. Possible determination methods include object detection, pattern matching, template matching, CNN (Convolutional Neural Network), Transformer, YOLO (You Only Look Once), SDD (Single Shot MultiBox Detector), and DETR (End-to-End Object Detection with Transformers).

[0084] If the first projection pattern 107 is reflected, the projection image 200 is determined to be an image (first projection image 201) captured while the first projection pattern 107 was being projected (S1802). If the first projection pattern 107 is not reflected, the projection image 200 is determined to not be an image (first projection image 201) captured while the first projection pattern 107 was being projected (S1803).

[0085] This processing flow is effective when the first projected pattern 107 and the second projected pattern 108 are different and patterns that can be clearly classified by a determination method such as object detection are used.

[0086] Fig. 19 is a process flow diagram for classifying a projected pattern by color judgment according to the fourth embodiment of the present invention. This corresponds to the processes performed in steps S2001 and S2002 in the flow of Fig. 7. Fig. 19 shows the first projected pattern assuming the process of step S2001, but if the same process is performed for the second projected pattern, it becomes the process of step S2002.

[0087] In step S2007, a color histogram showing the distribution of each color is generated from the entire image or a portion of the loaded projection image 200. In step S2008, a color histogram showing the distribution of each color of the first projection pattern 107 is generated. In step S2009, the similarity between the color histogram of the projection image and the color histogram of the first projection pattern is calculated, and it is determined whether they are similar.

[0088] Possible similarity measures include the difference between each color, the average value of the difference between each color, the median value of the difference between each color, the maximum value of the difference between each color, the minimum value of the difference between each color, the variance of the difference between each color, the standard deviation of the difference between each color, the effective value of the difference between each color, the root mean square of the difference between each color, etc. Possible methods for determining whether or not something is similar include threshold judgment, threshold judgment taking into account a time series, and machine learning.

[0089] If they are determined to be similar, the projected image 200 is determined to be an image (first projected image 201) captured while the first projected pattern 107 was being projected. This processing flow is effective when patterns that are easy to clearly classify in color are used for the first projected pattern 107 and the second projected pattern 108. For example, it is preferable that the first projected pattern 107 is a pattern made up of red, and the second projected pattern is a projected pattern 108 made up of green. [Example]

[0090] A fifth embodiment will be described with reference to Fig. 20 to Fig. 22. In the fifth embodiment, a method for classifying a projected pattern by time synchronization will be described. According to the fifth embodiment, a three-dimensional model can be generated in real time during shooting.

[0091] FIG. 20 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having a time synchronization unit according to a fifth embodiment of the present invention. The time synchronization unit 230 performs time synchronization between the 3D scanner 103 and the pattern projection device 130. The time synchronization unit 230 can distinguish whether the first projector 105 or the second projector 106 was projecting when the projection image 200 was captured. While the time synchronization unit 230 is illustrated as being inside the 3D scanner 103, it may also be inside the pattern projection device 130 or the three-dimensional model generation device 1000. As a method of time synchronization, for example, time synchronization may be performed based on absolute time, or projection start and end signals may be communicated in real time.

[0092] 21 is a flow diagram showing an example of processing during image capture in a three-dimensional measurement system having a time synchronization unit according to a fifth embodiment of the present invention. Because time synchronization is established between the 3D scanner and the pattern projection device 130, in steps S2010 and S2011, the 3D scanner 103 captures a first projection image 201 and can immediately store it in the first projection image memory. In other words, the processing of step S2001, which is performed during model generation, can be omitted, and the projection image 200 can be classified in real time during capture. Similarly, in steps S2012 and S2013, the 3D scanner 103 captures a second projection image 202 and can immediately store it in the second projection image memory.

[0093] 22 is a flowchart showing an example of processing performed when generating a model in a three-dimensional measurement system having a time synchronization unit 230 according to a fifth embodiment of the present invention. The difference from the flowchart in Fig. 7 is that steps S2001 and S2002 are omitted because the classification of the first projection image 201 and the second projection image 202 has already been completed by time synchronization.

[0094] In the fifth embodiment, if data is stored in the first projected image memory or the second projected image memory, the data processing of FIG. 22 can be performed simultaneously with the imaging flow of FIG. 21. The first projected image is read from the first projected image memory (S2201), and it is determined whether the first projected pattern on the object is occluded (S2003). If not occluded, it is stored in the first unoccluded projected image list (S702). The second projected image is also processed in the same way (S2202, S2004, S703). After that, the flow can be the same as that of FIG. 7.

[0095] In Example 5, since there is no need to classify images through image processing, degradation of accuracy due to the classification accuracy of image processing can be avoided. However, to perform time synchronization, hardware or software modifications are required to enable communication between the 3D scanner and the pattern projection device or the 3D model generation device. Furthermore, since the capture frame rate of the projected images 200 must be slower than the time synchronization period, the capture frame rate may be lower, potentially reducing the number of projected images 200 that can be captured per unit time. The accuracy of the generated 3D model improves as the number of input images increases. However, if the number of images is reduced by performing time synchronization, the accuracy of the generated 3D model may ultimately deteriorate. [Example]

[0096] A sixth embodiment will be described with reference to Fig. 23 to Fig. 25. In the sixth embodiment, a method for determining whether a pattern is occluded by performing image processing on a projected image will be described.

[0097] 23 is a process flow diagram for determining whether a projection pattern is occluded in object detection according to the sixth embodiment of the present invention. This corresponds to the process performed in step S2003. While FIG. 23 illustrates the first projection pattern, assuming the process of step S2003 in the first occlusion determination unit 1002, if the same process is performed for the second projection pattern in the second occlusion determination unit 1003, this becomes the process performed in step S2004.

[0098] The first occlusion determination unit 1002 reads the first projection image (S2301) and reads the first projection pattern (S2302).

[0099] In step S2014, it is determined whether the entire first projection pattern 107 is reflected in the read-out first projection image 201. One possible method for this determination is to divide the first projection pattern 107 into small patterns and perform object detection for each small pattern. If none of the small patterns can be detected in the first projection image, it is determined that the entire first projection pattern is not reflected, and it is determined that the first projection pattern is occluded (S2303). If it is determined that the entire first projection pattern 107 is reflected, it is determined that the read-out first projection image 201 is not occluded (S2304).

[0100] FIG. 24 is a diagram showing a first example of dividing a projection pattern into small patterns according to the sixth embodiment of the present invention. The projection pattern, which is made up of a picture, is divided into four parts, top, bottom, left, and right, to create the small patterns. The division method is not limited to this, and the pattern may be divided into any number of small patterns. Object detection is performed in step S2014 using the small patterns, and if all the small patterns are detected, it can be determined that the object was not occluded. Here, it is also possible to use only some of the small patterns, rather than all of them.

[0101] 25 is a diagram showing a second example of dividing a projection pattern into small patterns according to the sixth embodiment of the present invention. The projection pattern consisting of characters is divided into four parts, top, bottom, left, and right, to create the small patterns. As with the projection pattern consisting of a picture, object detection can be performed in step S2014 using the small patterns. [Example]

[0102] A seventh embodiment will be described with reference to Figures 26 to 28. In the seventh embodiment, a method for determining whether a pattern is occluded by an occlusion determination camera will be described.

[0103] 26 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having an occlusion determination camera according to a seventh embodiment of the present invention. The occlusion determination camera 132 captures an image of the object 101, the 3D scanner 103, the photographer 102, or the pattern projection device 130, and generates an occlusion determination image 207. Using the occlusion determination image 207, it can be determined whether or not a pattern that was occluded when the projection image 200 was captured is now occluded.

[0104] Fig. 27 is a diagram showing an example of an occlusion determination image captured by an occlusion determination camera according to Example 7 of the present invention. The occlusion determination image 207 shown in Fig. 27 was captured when the first projector 105 projected onto the object 101 and the photographer 102 was blocking the projection. In the portion 160 blocking the first projection pattern, the first projection pattern 107 is reflected on the photographer 102, indicating that the pattern is being blocked. By detecting the portion 160 blocking the first projection pattern by image processing, it is possible to determine whether the pattern is being blocked.

[0105] Typically, it is preferable that the entire projector projection area 270 falls within the occlusion determination image 207. Furthermore, it is preferable that the projector projection area 270 covers the shooting range 104 that can be shot with the angle of view per image shot by the 3D scanner 103.

[0106] FIG. 28 is a process flow diagram during image capture by a three-dimensional measurement system having an occlusion detection camera according to a seventh embodiment of the present invention. During image capture, in step S2015, an occlusion detection image 207 is captured by the occlusion detection camera 132. The occlusion detection image 207 may be captured each time the 3D scanner 103 captures one projected image 200, or may be captured each time multiple projected images 200 are captured. A projected image 200 may be captured each time multiple occlusion detection images 207 are captured. In step S2016, it is determined whether the first projected pattern 107 is captured in the projected image and whether the first projected pattern 107 is occluded. Similarly, in step S2017, it is determined whether the second projected pattern 108 is captured in the projected image and whether the second projected pattern 108 is occluded.

[0107] According to the seventh embodiment, the accuracy of the occlusion determination can be improved because the occlusion determination can be performed using the occlusion determination image 207 seen from a third-party viewpoint. However, there are disadvantages in that it is costly to install the additional occlusion determination camera 132 and that the occlusion determination camera needs to be installed in a location where it can capture the part 160 that occludes the first projection pattern. [Example]

[0108] Example 8 will be described with reference to Fig. 29 to Fig. 32. In Example 8, a method for calculating processing parameters for processing a component from a three-dimensional model measured by a 3D scanner, design information, and component information will be described. According to Example 8, it is possible to automatically determine processing parameters for a pipe to be additionally installed based on the construction status of the pipe measured in three dimensions.

[0109] FIG. 29 is an explanatory diagram of an additional pipe to be installed according to Example 8 of the present invention. Three-dimensional measurement is particularly expected in piping construction in the process of connecting multiple existing pipes 2901 with a new pipe 2902. The new pipe 2902 is processed to fit the shape of the existing pipe 2901 that has already been installed, and then attached by welding. If the processing precision of the new pipe 2902 is low, it cannot be installed. In that case, processing will have to be repeated, which will increase the time required to install the pipe. Here, if the shape of the existing pipe can be measured with high precision using three-dimensional measurement and a processing method for the new pipe can be calculated, the time required to install the pipe can be shortened.

[0110] 30 is a block diagram showing an example of the overall configuration of a three-dimensional measurement system having a processing parameter calculation device according to an eighth embodiment of the present invention. A three-dimensional model 150 output from a three-dimensional model generation device 1000 is input to a processing parameter calculation device 1100. Design information 1104 and member information 1105 are also input to the processing parameter calculation device 1100. The three-dimensional model generation device 1000 has a processing shape calculation unit 1101 that calculates a processing shape based on the three-dimensional model 150, the design information 1104, and the member information 1105, and a processing parameter calculation unit 1102 that calculates processing parameters from the processing shape and the member information.

[0111] FIG. 31 is an explanatory diagram showing a procedure in which the processing parameter calculation device 1100 calculates processing parameters for a pipe to be additionally installed according to the eighth embodiment of the present invention. A processing shape calculation unit 1101 calculates a processing shape 1106 from design information 1104, a three-dimensional model 150, and member information 1105. The design information 1104 is a design drawing showing the desired piping route. The three-dimensional model 150 is obtained by three-dimensionally measuring the actual construction situation.

[0112] Here, it can be seen that the three-dimensional model of the existing pipe on the left has a different end face shape from the pipe in the design information 1104. Component information 1105 is information about the unprocessed pipe. The pipe information includes, for example, the length, width, thickness, material, etc. By calculating the difference between the design information 1104 and the three-dimensional model 150, it is possible to calculate a processing shape 1106 for processing the unprocessed pipe in order to create a new pipe.

[0113] The processing parameter calculation unit 1102 calculates processing parameters 1103 required for processing the unprocessed pipe based on the processing shape 1106. The processing parameters 1103 may be, for example, the length by which the pipe is cut in the axial direction, the length by which the pipe is cut in the axial direction, the angle at which the pipe is cut in the normal direction, etc. A processed pipe is obtained by processing the pipe component based on the calculated processing parameters 1103. The processed pipe is created to match the shape of the existing pipe measured in three dimensions, which reduces the time required for pipe installation.

[0114] 32 is a flowchart showing a procedure for calculating processing parameters according to the eighth embodiment of the present invention. The procedure is carried out by the processing parameter calculation device 1100.

[0115] The processing parameter calculation unit 1102 reads the three-dimensional model 150, design information 1104, and component information 1105 (S3201). Next, the model of the new piping is corrected based on the difference between the design information 1104 and the three-dimensional model 150 (S3202), and a processing shape 1106 is calculated based on the difference between the model of the new piping and component information 1105 of the piping component (S3203). The processing parameter calculation unit 1102 calculates processing parameters 1103 to be input to the processing machine so as to process the processing shape 1106 (S3204). [Example]

[0116] FIG. 33 is a schematic diagram illustrating a case where projection is blocked by the shape of the object itself. When a projection pattern is projected onto an object 301 with a complex shape, a portion 302 appears on the surface of the object where the projection is blocked by the object's shape. However, even in this case, the entire projection pattern is projected somewhere. For example, the projection pattern is split and projected onto a portion 303 near the portion on the object where projection is blocked, or onto a portion 304 behind the object. Even if the projection pattern is split, visual characteristics can be imparted to the object, improving the accuracy of three-dimensional measurement. In this embodiment, when projection is blocked by the object's shape as shown in FIG. 33, the projection pattern is not considered to be blocked.

[0117] According to the above-described embodiment, efficient construction or a shortened construction period can be realized, which reduces energy consumption, reduces carbon emissions, prevents global warming, and contributes to the realization of a sustainable society. [Explanation of symbols]

[0118] 101 Object, 102 Photographer, 103 3D scanner, 104 Shooting range, 105 First projector, 106 Second projector, 107 First projection pattern, 108 Second projection pattern, 109 Obscured portion of first projection pattern, 130 Pattern projection device, 200 Projected image, 1000 Three-dimensional model generation device

Claims

1. a pattern projection device including a first projector that projects a first pattern onto an object and a second projector that projects a second pattern; a 3D scanner provided between the object and the pattern projection device, capturing an image of the object while moving and generating a projected image; A three-dimensional measurement system comprising a three-dimensional model generation device that outputs a three-dimensional model using the projected image of the second projector when the 3D scanner is located between the first projector and the object, and using the projected image of the first projector when the 3D scanner is located between the second projector and the object.

2. A projection image classification unit is provided that classifies the projection image captured by the 3D scanner as the projection image of the first projector or the projection image of the second projector. The three-dimensional measurement system according to claim 1.

3. the first pattern and the second pattern are different patterns, The projection image classification unit Classification is performed by comparing the projection image generated by the 3D scanner with at least one of the first pattern and the second pattern. The three-dimensional measurement system according to claim 2.

4. the first pattern and the second pattern are different colors; The projection image classification unit Classifying the object by comparing the projection image generated by the 3D scanner with at least one color of the first pattern and the second pattern. The three-dimensional measurement system according to claim 2.

5. the first pattern and the second pattern are projected in a time-division manner; The projection image classification unit performing classification based on timing of projection of the first pattern and the second pattern; The three-dimensional measurement system according to claim 2.

6. an occlusion determination unit that determines whether the 3D scanner is between at least one of the first projector and the second projector and the object; The three-dimensional measurement system according to claim 1.

7. The occlusion determination unit determining whether the projection image generated by the 3D scanner includes at least one of the first pattern and the second pattern in its entirety; The three-dimensional measurement system according to claim 6.

8. The occlusion determination unit dividing at least one of the first pattern and the second pattern into a plurality of small patterns, and determining whether all of the plurality of small patterns can be detected in the projected image, thereby determining whether at least one of the first pattern and the second pattern is entirely included in the projected image; The three-dimensional measurement system according to claim 7.

9. The occlusion determination unit determining whether the projected image includes the entirety of at least one of the first pattern and the second pattern based on an image captured by an occlusion determination camera that captures the object; The three-dimensional measurement system according to claim 7.

10. Further comprising a processing parameter calculation device, The processing parameter calculation device a machining shape calculation unit that calculates a machining shape of a member from the three-dimensional model, design information, and member information; a processing parameter calculation unit that calculates processing parameters from the member information and the processing shape; The three-dimensional measurement system according to claim 1.

11. A three-dimensional measurement device that projects a first pattern from a first projector onto an object, projects a second pattern from a second projector onto the object, and generates a three-dimensional model using the projected images from a 3D scanner that captures images of the object while moving, a three-dimensional model generation device that generates the three-dimensional model using the projected image of the second projector when the 3D scanner is between the first projector and the object, and using the projected image of the first projector when the 3D scanner is between the second projector and the object.

12. a first step of projecting a first pattern from a first projector onto an object; a second step of projecting a second pattern onto the object from a second projector; a third step of capturing an image of the object while moving it with a 3D scanner to generate a projection image; a fourth step of performing three-dimensional measurement using the projected image of the second projector when the 3D scanner is between the first projector and the object, and using the projected image of the first projector when the 3D scanner is between the second projector and the object; A three-dimensional measurement method for performing

13. a fifth step of determining whether the projected image generated in the third step is a projected image generated by the first projector or the second projector by comparing the projected image with at least one of the shapes and colors of the first pattern and the second pattern; The three-dimensional measurement method according to claim 12.

14. executing the first step and the second step in a time-division manner, and executing a sixth step of determining whether the projected image is a result of the first projector or the second projector based on the execution timing of the first step and the second step; The three-dimensional measurement method according to claim 12.

15. At least one of the first pattern and the second pattern is a pattern that can acquire more feature points than before pattern projection when projecting the pattern onto the object and performing three-dimensional measurement using the projected image captured by the 3D scanner. The three-dimensional measurement method according to claim 12.

Images