Three-dimensional model generation device, three-dimensional model generation method, and three-dimensional model generation program

Abstract

To provide a three-dimensional model generation apparatus, a three-dimensional model generation method, and a three-dimensional model generation program configured to easily generate high-accuracy three-dimensional point cloud data for an object with less texture.SOLUTION: A three-dimensional model generation apparatus for generating three-dimensional point cloud data of an object 20 includes: a projector 10 which projects a cork pattern image CSP in a complementary color of the object 20, to the object 20; a camera 12 which images the object 20 while changing a shooting position PhP, with the cork patten image CSP in the complementary color projected by the projector 10; and a PC 30 which calculates three-dimensional point cloud data of the object 20 on the basis of multi-view imaging data of the object 20 captured by the camera 12.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a three-dimensional model generation device, a three-dimensional model generation method, and a three-dimensional model generation program for generating more accurate three-dimensional point cloud data (three-dimensional model) by projection of a specific pattern.

Background Art

[0002] In recent years, in all industries, the ICT (Information and Communication Technology) of technology has been progressing. Particularly in industries such as civil engineering and construction, various efforts have been made to improve on-site productivity by the proposal of "i-Construction".

[0003] Furthermore, as a further effort, by the recommendation of "Construction DX (Digital Transformation)", not only on-site but also the entire company (all businesses) is strengthening the review of the supply chain.

[0004] For example, in the field of surveying, the on-site utilization of ICT devices and tools that lead to business efficiency is regarded as "Construction DX", and efforts to utilize three-dimensional point cloud data obtained by three-dimensionally modeling an object with high precision (high quality) in the design, construction, and maintenance management of the construction field are spreading.

[0005] Although it is common to use a laser scanner for the delicate three-dimensional modeling of an object, in recent years, three-dimensional modeling of an object by SfM (Structure from Motion) using an image has become popular.

Prior Art Documents

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] However, conventionally, in generating high-quality as-is (current state) models using SfM, for example, only the camera position and orientation and three-dimensional feature point cloud obtained from Bundle adjustment were used to predict the quality of the SfM model generated after Dense Matching, and the optimal shooting position that would allow for improvement of parts where quality degradation is expected was to be calculated using computer assistance.

[0008] In other words, to improve the quality of the as-is model, it was made possible to add captured images by optimizing the shooting position. However, even if a texture image (for example, a random dot marker image) was projected onto the object using a projector, there was a problem that SfM reconstruction of objects with little texture was not sufficient with just the projection of a texture image.

[0009] The present invention has been made in view of the above problems, and aims to provide a three-dimensional model generation device, a three-dimensional model generation method, and a three-dimensional model generation program that can easily generate more accurate three-dimensional point cloud data regardless of the color of the object, even when the object has little texture. [Means for solving the problem]

[0010] Embodiments of the present invention involve applying the complementary color of the object to the object. Cork pattern image A projector that projects the complementary color Cork pattern imageThe gist of the invention is that it comprises a camera that projects an image onto the object and photographs the object while changing the shooting position, and a point cloud data calculation unit that calculates three-dimensional point cloud data of the object based on the photographic data of the object taken by the camera. Furthermore, embodiments of the present invention include a projector that projects a specific pattern of the complementary color of an object onto the object; a camera that photographs the object while changing the shooting position with the specific pattern of the complementary color projected by the projector; and a point cloud data calculation unit that calculates three-dimensional point cloud data of the object based on the shooting data of the object captured by the camera. The gist of the point cloud data calculation unit is that it calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to the multi-viewpoint shooting data acquired from multiple shooting positions. Furthermore, embodiments of the present invention include a projector that projects a cork pattern image of the complementary color of an object onto the object from one or more directions; a camera that photographs the object while changing the shooting position with the cork pattern image of the complementary color projected by the projector; and a point cloud data calculation unit that calculates three-dimensional point cloud data of the object based on the photographic data of the object taken by the camera.

[0011] Another embodiment of the present invention involves projecting the complementary color of an object onto the object using a projector. Cork pattern image A step of projecting the complementary color by the projector Cork pattern image The gist of the invention is that it comprises the steps of: photographing the object while changing the shooting position of the camera while projecting the image onto it; and calculating three-dimensional point cloud data of the object using a point cloud data calculation unit based on the image data of the object photographed by the camera. Furthermore, another embodiment of the present invention comprises the steps of: projecting a specific pattern of the complementary color of an object onto the object using a projector; photographing the object while changing the shooting position of a camera while the specific pattern of the complementary color is projected onto the object using the projector; and calculating three-dimensional point cloud data of the object by a point cloud data calculation unit by applying SfM / MVS processing to multi-viewpoint shooting data of the object photographed by the camera, obtained from multiple shooting positions. Furthermore, another embodiment of the present invention comprises the steps of: projecting a cork pattern image of the complementary color of the object onto the object using a projector from one or more directions; photographing the object while changing the shooting position of a camera while the cork pattern image of the complementary color is projected onto the object using the projector; and calculating three-dimensional point cloud data of the object using a point cloud data calculation unit based on the shooting data of the object captured by the camera.

[0012] A further embodiment of the present invention is a three-dimensional model generation program for realizing a three-dimensional model generation apparatus by computer, wherein the computer projects complementary colors onto the object using a projector. Cork pattern image The gist of this is to function as a point cloud data calculation unit that calculates three-dimensional point cloud data of an object based on the image data of the object, which is captured while changing the shooting position of the camera, with the image projected onto it. Furthermore, yet another embodiment of the present invention is a three-dimensional model generation program for realizing a three-dimensional model generation device using a computer, wherein the computer functions as a point cloud data calculation unit that calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to multi-viewpoint imaging data acquired from multiple imaging positions of the object, which is photographed while changing the imaging position of a camera, with a specific complementary color pattern projected onto the object using a projector. Furthermore, yet another embodiment of the present invention is a three-dimensional model generation program for realizing a three-dimensional model generation apparatus using a computer, wherein the computer functions as a point cloud data calculation unit that calculates three-dimensional point cloud data of the object based on the image data of the object, which is captured while changing the shooting position of the camera, with a complementary color cork pattern image projected onto the object from one or more directions by a projector. [Effects of the Invention]

[0013] As described above, the present invention provides a three-dimensional model generation device, a three-dimensional model generation method, and a three-dimensional model generation program that can easily generate highly accurate three-dimensional point cloud data regardless of the color of the object, even when the object has little texture. [Brief explanation of the drawing]

[0014] [Figure 1] This shows the schematic configuration of the three-dimensional model generation device according to the first embodiment. (a) is a plan view, and (b) is a side view in the direction of line Ib-Ib in (a). [Figure 2] This is a diagram illustrating an image (imaging data) when an object is imaged. [Figure 3] This is a block diagram showing a configuration example of the PC of the three-dimensional model generation device. [Figure 4] This is a flowchart shown to explain the operation during verification of the three-dimensional model generation device. [Figure 5] This is a schematic diagram illustrating a cork pattern image (default image). [Figure 6] This is a flowchart illustrating the case of automatically generating a complementary color cork pattern image. [Figure 7] This illustrates the case of automatically generating a complementary color cork pattern image with a mannequin as the object. (a) shows an example of object imaging / RGB value calculation, (b) shows an example of complementary color calculation, and (c) shows an example of creating a complementary color cork (pattern) image, respectively. [Figure 8] This is a diagram illustrating a pattern of imaging data. [Figure 9] Regarding the calculation of the RMS error, taking the mannequin as an example, (a) shows an example of alignment with the point cloud data to be compared, (b) shows an example of cutting out the evaluation location (cut-out location), and (c) shows an example of surface fitting / RMS error calculation, respectively. [Figure 10] This is a diagram illustrating the result of comparing the RMS error for each color. [Figure 11] Regarding the calculation of the RMS error, taking the Color Cone (registered trademark) as an example, (a) shows an example of alignment with the point cloud data to be compared, (b) shows an example of cutting out the evaluation location (cut-out location), and (c) shows an example of surface fitting / RMS error calculation, respectively. [Figure 12] This is a diagram illustrating the result of comparing the RMS error for the red Color Cone (registered trademark). [Figure 13]The following diagrams illustrate the tie point count using a red mannequin as the object: (a) shows an example of object projection and photography, (b) shows an example of SfM / MVS processing, and (c) shows an example of tie point count comparison. [Figure 14] This flowchart illustrates a case where the number of tie points is being compared. [Figure 15] This diagram illustrates the results of comparing the number of tie points for each color. [Figure 16] This diagram illustrates the automatic generation of a complementary color cork pattern image using a trash can as the target object. (a) shows an example of photographing the target object, (b) shows an example of calculating RGB values, (c) shows an example of calculating complementary colors, and (d) shows an example of creating a complementary color cork (pattern) image. [Figure 17] This is a flowchart for calculating the difference in the projection normal direction between point cloud data obtained by SfM and laser data (point cloud data). [Figure 18] This example shows a comparison of point cloud data obtained by SfM and laser data (point cloud data). (a) is point cloud data obtained by SfM, and (b) is point cloud data obtained by FARO laser. [Figure 19] This is a flowchart for comparing point cloud data obtained by SfM with laser data (point cloud data). [Figure 20] This shows a comparison of cross-sections of point cloud data. (a) is the point cloud data with the viewpoint as the top surface, (b) is a cross-section along line AA' of (a), (c) is a cross-section along line BB' of (a), (d) is a cross-section along line CC' of (a), and (e) is a cross-section along lines DD' and EE' of (a). [Figure 21] This plan view shows a schematic configuration of a three-dimensional model generation apparatus according to the second embodiment, illustrating a case where complementary color cork pattern images are projected from two directions. [Figure 22] This flowchart illustrates a case where the number of tie points is being compared. [Figure 23]This example illustrates the results of SfM processing (after high-density cloud construction). (a) is the point cloud data when the viewpoint is the top surface, (b) is the point cloud data when the viewpoint is in the direction of (A) of (a), (c) is the point cloud data when the viewpoint is in the direction of (B) of (a), (d) is the point cloud data when the viewpoint is in the direction of (C) of (a), and (e) is the point cloud data when the viewpoint is in the direction of (D) of (a). [Figure 24] This figure illustrates the results of comparing point cloud data with and without projection of a complementary color cork pattern image. [Figure 25] This is a schematic diagram showing an example of the application of the three-dimensional model generation apparatus according to the third embodiment. [Figure 26] This is to explain the operation of the three-dimensional model generation device. (a) is an image of the projection surface, and (b) is a captured image. [Figure 27] This is a schematic diagram showing another application example of the three-dimensional model generation apparatus according to the third embodiment. [Figure 28] This is to explain the process of obtaining a heat map of the damaged area. (a) is an image illustrating the application of clay, and (b) is an image illustrating the heat map. [Modes for carrying out the invention]

[0015] The embodiments shown below illustrate devices and methods for realizing the technical concept (structure, arrangement) of the present invention, and the technical concept of the invention is not limited to those described below. The technical concept of the present invention can be modified in various ways within the scope of the claims. It should also be noted that the drawings are schematic, and the configuration of the devices and systems may differ from those in reality.

[0016] In this embodiment of the three-dimensional model generation apparatus, the optimal shooting plan that enables the generation of a high-quality as-is (current state) model using SfM (Structure from Motion) is considered as part of "Construction DX (Digital Transformation)," and is described as projecting a specific pattern of the complementary color of the object (for example, a cork (pattern) image of the complementary color) when the object is photographed.

[0017] Furthermore, this section describes a configuration that allows for the verification of the accuracy of three-dimensional point cloud data generated by projecting a specific pattern of the complementary color of an object. This is achieved by quantifying the quality of the calculated point cloud data as projection clarity, based on factors such as the number of tie points (corresponding points between images acquired during SfM processing) and the RMS (Root Mean Square) error, which indicates the degree of point cloud variability.

[0018] <First Embodiment> Figure 1 shows a schematic configuration of a three-dimensional model generation apparatus according to the first embodiment, where (a) is a plan view and (b) is a side view in the direction along the line Ib-Ib in (a).

[0019] As shown in Figures 1(a) and 1(b), the three-dimensional model generation apparatus of this embodiment comprises, for example, a projector 10 that projects a cork pattern image (specific pattern) CSP of the complementary color of the object 20 onto the object 20 as a texture image from one direction; a camera 12 that photographs the object 20 onto which the complementary color cork pattern image CSP is projected; and a PC (Personal Computer) 30 that generates three-dimensional point cloud data based on the image (captured data) captured by the camera 12 and verifies the accuracy of the generated three-dimensional point cloud data by indexing the clarity of the projection.

[0020] The projector 10 projects a complementary color cork pattern image CSP onto the object 20 from which three-dimensional point cloud data is generated, from a predetermined projection position PrP, for example, at a distance of about 2m from the object 20.

[0021] The camera 12 is configured to repeatedly photograph the object 20 while gradually moving its shooting position PhP horizontally (in one direction) while the complementary color cork pattern image CSP is projected by the projector 10.

[0022] Furthermore, camera 12 performs multiple (multiple viewpoint) shooting positions PhP, for example, where the projection distance from the projection position PrP is about 1m, so that the captured data overlaps by about 80% (multiple overlap shooting).

[0023] Furthermore, camera 12 is equipped with GNSS (Global Navigation Satellite System) functionality and a 9-axis sensor, making it possible to attach various parameters related to the shooting position (PhP) and shooting posture to each image.

[0024] In this embodiment, as the object to be photographed 20, for example, as shown in Figure 2, a traffic cone 20C, a mannequin (head only) 20M, and a trash can 20S are prepared, and three-dimensional point cloud data is generated and the accuracy of the generated three-dimensional point cloud data is verified using at least one of the traffic cone 20C, mannequin 20M, or trash can 20S. In addition, a ground control point 21 for alignment is attached to the object 20. Besides the traffic cone 20C, mannequin 20M, and trash can 20S, polyvinyl chloride pipes (also abbreviated as PVC pipes) can also be used as the object 20.

[0025] Figure 3 illustrates the schematic configuration (functional blocks) of PC30.

[0026] Specifically, as shown in Figure 3, for example, the PC30 includes a control unit 30a, a storage unit 30b, an image input unit 30c, a tie point number comparison unit 30d, an SfM / MVS (Multi-View Stereo) processing unit 30e as a point cloud data calculation unit, a complementary color image generation unit 30f, a PJ drive unit 30g, a surface fitting unit 30h, and an RMS error calculation unit 30i, among others.

[0027] For convenience, the diagrams have been omitted, but PC30 can be a well-known system equipped with a keyboard and mouse for input operations, a display device, or a communication device for communicating with the outside world.

[0028] In the PC30, the control unit 30a controls each part according to various data and programs (calculation processing software) stored in the memory unit 30b, for example, enabling the generation of three-dimensional point cloud data as well as verification of the accuracy of the three-dimensional point cloud data by indexing the clarity of the projection.

[0029] Furthermore, the control unit 30a can be configured to perform tasks such as acquiring laser data (point cloud data) using a 3D laser scanner, such as FARO, or forming a heat map.

[0030] The memory unit 30b stores various data and calculation processing software necessary for generating three-dimensional point cloud data, such as commercially available SfM / MVS software.

[0031] Furthermore, the memory unit 30b may also store, for example, the original cork pattern image (default image) SP for generating a complementary color cork pattern image CSP, and three-dimensional point cloud data generated by projecting the default cork pattern image SP as a texture image, for verification purposes.

[0032] The image input unit 30c takes in, for example, multi-viewpoint image data of the object 20 captured by the camera 12, or images of the object 20 for generating a complementary color cork pattern image CSP.

[0033] The tie point count comparison unit 30d compares the number of corresponding points (tie points) between captured data automatically acquired by a well-known image matching technique. For example, it is possible to compare the number of tie points when a texture image is projected (with projection) and when it is not projected (without projection).

[0034] The SfM / MVS processing unit 30e performs SfM / MVS processing on multi-viewpoint image data of the object 20 using, for example, commercially available SfM / MVS software. This includes automatic acquisition of tie points and the reproduction (restoration) of a three-dimensional model-like shape of the object 20 as point cloud data.

[0035] The complementary color image generation unit 30f automatically generates a complementary color cork pattern image CSP from the default cork pattern image SP by calculating the RGB values ​​of the object 20 from the captured image.

[0036] The PJ drive unit 30g drives the projector 10 to project the complementary color cork pattern image CSP generated by the complementary color image generation unit 30f onto the object 20.

[0037] The surface fitting unit 30h uses the fitting function of predetermined software (e.g., Cloud Compare) for processing point cloud data to perform fitting processes such as plane fitting and curved surface fitting to determine the RMS error of the point cloud data.

[0038] The RMS error calculation unit 30i compares the RMS error with and without projection of a texture image, for example, and makes it possible to verify the accuracy of three-dimensional point cloud data by quantifying the clarity of the projection.

[0039] Here, the three-dimensional model generation device generates three-dimensional point cloud data using, for example, well-known automatic modeling techniques (which are based on roughly the same principles as photogrammetry). Specifically, using commercially available SfM / MVS software, it automatically acquires corresponding points among the multi-viewpoint image data of the object 20, estimates the shooting position (camera position) PhP and orientation of the image data from the corresponding points using the bundle method, and acquires point cloud data of the surface of the object 20 based on the camera position and orientation shown as coordinates on Earth, constructs a triangular mesh that constitutes the surface of the object 20, and then applies texture images to that triangular mesh.

[0040] Therefore, in the PC30 of the three-dimensional model generation device, the tie point comparison unit 30d, the surface fitting unit 30h, and the RMS error calculation unit 30i are not essential components, but are provided for verification by quantifying the clarity of the projection.

[0041] In other words, in a three-dimensional model generation device, for example, when generating three-dimensional point cloud data of an object 20, first, the control unit 30a executes a program stored in the storage unit 30b, which automatically generates a complementary color cork pattern image CSP in the complementary color in the complementary color image generation unit 30f, after which the PJ drive unit 30g is controlled. As a result, the cork pattern image CSP in the complementary color of the object 20 is projected onto the object 20 from the projector 10.

[0042] In this state, multi-viewpoint shooting data of the object 20, obtained by gradually changing the shooting position PhP of the camera 12, is sent to the PC 30 and captured by the image input unit 30c.

[0043] The multi-viewpoint imaging data captured at multiple shooting positions PhP is then used in the SfM / MVS processing unit 30e for purposes such as calculating the number of tie points and acquiring high-density (high-resolution) point cloud data that enables more precise three-dimensional modeling, by, for example, executing commercially available SfM / MVS software.

[0044] Thus, according to the three-dimensional model generation apparatus of this embodiment, by projecting a cork pattern image CSP of the complementary color of the object 20 as a texture image, it becomes possible to easily generate more accurate three-dimensional point cloud data.

[0045] Furthermore, by applying a fitting process to the point cloud data in the surface fitting unit 30h, the accuracy of the generated three-dimensional point cloud data can be verified by determining whether the RMS error can be calculated in the RMS error calculation unit 30i.

[0046] Figure 4 is a flowchart illustrating the procedure for verifying the clarity of projections using an index in a three-dimensional model generation device.

[0047] As shown in Figure 4, first, for example, the camera 12 takes a photograph of the object 20 (step St1), and the control unit 30a acquires the captured image of the object 20 via the image input unit 30c (step St2).

[0048] Next, the default cork pattern image SP pre-stored in the memory unit 30b is read (step St3), and the complementary color image generation unit 30f automatically generates a complementary color cork pattern image CSP of the object 20 based on the acquired captured image of the object 20 (step St4).

[0049] After obtaining a cork pattern image CSP of the complementary color of the object 20 (step St5), the cork pattern image CSP of the complementary color is projected onto the object 20 by the projector 10 via the PJ drive unit 30g (step St6).

[0050] Next, with the complementary color cork pattern image CSP from the projector 10 projected onto the object 20, the camera 12 performs multiple-layer photography of the object 20 (step St7).

[0051] Then, after obtaining multi-viewpoint image data of the object 20 via the image input unit 30c (step St8), SfM / MVS processing is performed in the SfM / MVS processing unit 30e using commercially available SfM / MVS software (step St9). This makes it possible to acquire high-resolution point cloud data even if the object 20 is textureless (step St10).

[0052] Therefore, regardless of the color of the object 20, it becomes easy to generate more accurate three-dimensional point cloud data.

[0053] On the other hand, when evaluating point cloud data, for example, as shown in step St11, the surface fitting unit 30h and the RMS error calculation unit 30i perform the following: cropping of the point cloud data within an arbitrary range (step St11a), surface fitting processing of the cropped point cloud data (step St11b), and calculation of the RMS error from the point cloud data and the results of the surface fitting processing (step St11c).

[0054] Specifically, by calculating the RMS error between the point cloud data of an arbitrary range and the result of surface fitting processing when a cork pattern image CSP of the complementary color is projected onto the object 20, it becomes possible to quantify the clarity of the projection. This makes it possible to verify the accuracy of the obtained three-dimensional point cloud data by quantifying the clarity of the projection.

[0055] Here, referring to Figures 5 to 7(a) to (c), we will further explain the procedure (step St4) for automatically generating a cork pattern image CSP of the object 20 in the complementary color.

[0056] Figure 5 shows an example of the default cork pattern image SP, Figure 6 is a flowchart showing the procedure for automatically generating a complementary color cork pattern image CSP of the object 20 from the default cork pattern image SP, and Figures 7(a) to (c) are schematic diagrams showing the procedure for automatically generating a complementary color cork pattern image CSP using a mannequin (head only) 20M as an example. The explanation will then describe the case where the object 20, which is a mannequin 20M, is a red mannequin 20MR, a green mannequin 20MG, and a yellow mannequin 20MY.

[0057] Specifically, when automatically generating a complementary color cork pattern image CSP for object 20 from the default cork pattern image SP shown in Figure 5, first, as shown in Figure 7(a), for example, images of each mannequin 20MR, 20MG, and 20MY are acquired (step St4a in Figure 6). Then, RGB values ​​are calculated from the acquired images (step St4b in Figure 6). Subsequently, as shown in Figure 7(b), for example, the complementary RGB colors of each mannequin 20MR, 20MG, and 20MY are calculated by complementary color calculation (step St4c in Figure 6). Finally, based on the calculated complementary RGB colors, the complementary color cork pattern images CSP / 20MR, CSP / 20MG, and CSP / 20MY corresponding to the colors of each mannequin 20MR, 20MG, and 20MY are automatically generated (step St4d in Figure 6).

[0058] As a result, if the object 20 is a red mannequin 20MR, the projector 10 will project a cork pattern image CSP / 20MR of the complementary color; if the object 20 is a green mannequin 20MG, the projector 10 will project a cork pattern image CSP / 20MG of the complementary color; and if the object 20 is a yellow mannequin 20MY, the projector 10 will project a cork pattern image CSP / 20MY of the complementary color.

[0059] Next, we will specifically examine the high-precision three-dimensional point cloud data generated by projecting a cork pattern image CSP of the complementary color of object 20.

[0060] Verification 1 The following describes the results of an investigation into whether it is possible to calculate the RMS error in quantifying the clarity of projections, for example, by projecting a complementary color cork pattern image CSP onto a mannequin 20M with a complex shape and then fitting the resulting point cloud data.

[0061] Here, as conditions for verifying the three-dimensional model generation device, the illuminance was set to 12 lux, a Canon EOS R5 (resolution: 7280×5464) was used as the camera 12, and, for example as shown in Figure 1(b), the projection distance from the projection position PrP to the shooting position PhP was kept at approximately 1 m, while the shooting position PhP was moved in one direction in parallel, and the object 20 was captured from multiple viewpoints.

[0062] In addition, three mannequins were prepared as the target objects 20: a red mannequin 20MR, a green mannequin 20MG, and a yellow mannequin 20MY. A BenQ MH550 projector 10 was used (for example, the projection method was DLP, the brightness was 3500lm, the contrast ratio was 20,000:1, and the resolution was 1920×1080).

[0063] Then, for mannequins 20MR, 20MG, and 20MY located approximately 2m from the projection position PrP, images were taken three times for each pattern, as shown in Figure 8, for example: a pattern without projection, a pattern with projection of the default cork pattern image SP, and a pattern with projection of the complementary color cork pattern image CSP.

[0064] Figures 9(a) to 9(c) are shown to further explain the processing in step St11, which involves evaluating point cloud data, in the procedure for verifying the clarity of the projection as described above (see flowchart in Figure 4).

[0065] Specifically, in step St11a, in order to find the average value of the point cloud data (patterns R1, R2, R3) of the red mannequin 20MR, the point cloud data to be compared is aligned, for example as shown in Figure 9(a), and then, for example as shown in Figure 9(b), the point cloud data of an arbitrary range to be evaluated is cropped. Subsequently, in step St11b, for example as shown in Figure 9(c), the surface fitting unit 30h performs surface fitting processing on the cropped point cloud data, and in step St11c, the RMS error calculation unit 30i calculates the RMS error from the point cloud data and the result of the surface fitting processing.

[0066] I will omit the specific details, but the same procedure is followed for the other colored mannequins, 20MG and 20MY.

[0067] Figure 10 shows the results of comparing the RMS errors calculated by the RMS error calculation unit 30i. As is clear from these results, for all colors, the RMS error (average value) was smallest and the accuracy (high density) was highest when the complementary color cork pattern image CSP was projected.

[0068] Specifically, when the complementary color cork pattern images CSP / 20MR, CSP / 20MG, and CSP / 20MY are projected onto the mannequins 20MR, 20MG, and 20MY of each color, the RMS error is minimized. This allows for clearer edges, reduced shape distortion, and improved point cloud density in the point cloud data generated by SfM / MVS processing, enabling the acquisition of higher-resolution point cloud data.

[0069] Furthermore, by cropping the same area of ​​the point cloud data, it is possible to calculate the RMS error using the fitting function. Projecting complementary color texture images allows for the quantification of the clarity of projection using only point cloud data, even for objects with complex shapes such as mannequins 20MR, 20MG, and 20MY. This proves to be effective (optimal) for verifying the accuracy of three-dimensional point cloud data.

[0070] Verification 2 The following describes the results of an investigation into whether it is possible to calculate the RMS error in the indexing of projection clarity by projecting a complementary color cork pattern image CSP onto a red color cone (registered trademark) 20CR, which is a triangular cone 20C, as shown in Figures 11(a) to (c), and then fitting the resulting point cloud data.

[0071] The conditions used during the verification of the 3D model generation device were the same as those used in Verification 1.

[0072] Specifically, in order to determine the average value of the point cloud data (patterns R1, R2, R3) of the color cone (registered trademark) 20CR, the point cloud data to be compared is aligned, for example as shown in Figure 11(a), and then, for example as shown in Figure 11(b), the point cloud data of an arbitrary range to be evaluated is cropped. Subsequently, for example as shown in Figure 11(c), the surface fitting unit 30h performs surface fitting processing on the cropped point cloud data, and the RMS error calculation unit 30i calculates the RMS error from the point cloud data and the result of the surface fitting processing.

[0073] Figure 12 shows the results of comparing the RMS errors calculated by the RMS error calculation unit 30i. As is clear from these results, the RMS error (average value) was smallest and the accuracy (high density) was highest when the complementary color cork pattern image CSP was projected.

[0074] Specifically, when a complementary color cork pattern image CSP / 20CR (not shown) is projected onto a red color cone (registered trademark) 20CR, the RMS error is minimized. This allows for clearer edges, reduced shape distortion, and improved point cloud density in the point cloud data generated by SfM / MVS processing, enabling the acquisition of higher-resolution point cloud data.

[0075] Furthermore, by cropping the same area of ​​the point cloud data, it is possible to calculate the RMS error using the fitting function. Projecting a complementary color texture image allows for the quantification of the clarity of the projection using only point cloud data, even for objects with shapes like the color cone (registered trademark) 20CR. This proves to be effective (optimal) for verifying the accuracy of three-dimensional point cloud data.

[0076] In particular, Verification 1 and Verification 2 show that it is possible to generate three-dimensional point cloud data with sufficient accuracy without being affected by the color of the object 20 and without using a laser scanner or the like.

[0077] Verification 3 The following describes the results of an investigation into whether it is possible to calculate the RMS error in quantifying the clarity of projection by projecting a complementary cork pattern image CSP and a default cork pattern image SP onto a red mannequin 20MR, as shown in Figures 13(a) to (c), and comparing the number of tie points obtained from the respective point cloud data.

[0078] The conditions used during the verification of the 3D model generation device were the same as those used in Verification 1.

[0079] Figure 14 is a flowchart illustrating the procedure for verifying projection clarity by quantifying it.

[0080] First, in order to determine the average value of the tie point count (patterns R1, R2, R3), the image input unit 30c captures, for example, the image data when the default cork pattern image SP is projected onto the red mannequin 20MR, and the image data when the complementary color cork pattern image CSP / 20MR is projected onto the mannequin 20MR (steps St21, St22).

[0081] Next, the SfM / MVS processing unit 30e performs SfM / MVS processing on each of the captured data, and for example, as shown in Figure 13(b), default three-dimensional point cloud data is obtained when the default cork pattern image SP is projected (steps St23, St24), and complementary three-dimensional point cloud data is obtained when the complementary cork pattern image CSP / 20MR is projected (steps St23, St25).

[0082] Then, the tie point number comparison unit 30d compares the number of tie points obtained when generating default three-dimensional point cloud data with the number of tie points obtained when generating complementary three-dimensional point cloud data, for example, as shown in Figure 13(c) (step St26).

[0083] I will omit the specific details, but the same procedure is followed for the other colored mannequins, 20MG and 20MY.

[0084] Figure 15 shows the results of comparing the number of tie points. As is clear from these results, for all colors, the number of tie points (average value) increased when the complementary cork pattern image CSP was projected compared to the default case, indicating high accuracy (high density).

[0085] In other words, by projecting a complementary color cork pattern image CSP, it becomes possible to increase the number of tie points, improving the representation of curved surfaces in the point cloud data, and reducing surface roughness and point dispersion, thereby enabling a more precise reproduction of the object 20 using point cloud data.

[0086] In Figure 15, the large number of tie points without projection is likely due to the inclusion of feature points other than the surrounding objects 20.

[0087] Thus, projecting complementary color texture images allows for an increase in the number of tie points acquired during point cloud data generation. This increase in tie points is effective in estimating the camera's position and orientation during data acquisition, making it ideal for verifying the accuracy of three-dimensional point cloud data.

[0088] Verification 4 The following describes the results of a study that investigated whether projecting a complementary color cork pattern image CSP onto a trash can 20S (object 20) and calculating the difference in the normal direction by overlaying it with laser data (point cloud data) from, for example, a 3D laser scanner, is useful as an index for quantifying the clarity of the projection.

[0089] Figures 16(a) to (d) are schematic diagrams showing the procedure for automatically generating complementary color cork pattern images CSP using trash can 20S as an example, and Figure 17 is a flowchart explaining the procedure for verifying by indexing the clarity of the projection. The explanation will cover the case where the target object 20, trash can 20S, is a yellow trash can 20SY, a green trash can 20SG, and a blue trash can 20SY.

[0090] Here, the conditions for verifying the three-dimensional model generation device were, for example, set to 18 lux to match the conditions for laser data acquisition, using a Canon EOS R5 camera 12 (resolution: 7280 x 5464), maintaining a distance of approximately 1 m from the projection position PrP to the shooting position PhP, and capturing images of the object 20 from multiple viewpoints while translating the shooting position PhP in one direction.

[0091] Furthermore, as the objects 20, for example, as shown in Figure 16(a), a yellow trash can 20SY, a green trash can 20SG, and a blue trash can 20SB were prepared, and a BenQ MH550 projector 10 was used (for example, the projection method is DLP, the brightness is 3500lm, the contrast ratio is 20,000:1, and the resolution is 1920×1080).

[0092] Then, for trash cans 20SY, 20SG, and 20SB located approximately 1m from the projection position PrP, images were taken three times for each pattern: without projection, with the default cork pattern image SP projected, and with the complementary color cork pattern image CSP projected.

[0093] In this way, by reproducing the conditions under which the laser data was acquired in the three-dimensional model generation device, it becomes possible to match the coordinate values ​​(such as scale) between data points, and to superimpose the laser data and point cloud data.

[0094] Specifically, when verifying the indexing of projection clarity using complementary color cork pattern images CSP for trash can 20S, first, as shown in Figure 16(a), for example, captured images of each trash can 20SY, 20SG, and 20SB are acquired (step St51 in Figure 17). Then, as shown in Figure 16(b), for example, RGB values ​​are calculated from the acquired captured images. Subsequently, as shown in Figure 16(c), for example, the complementary RGB values ​​of each trash can 20SY, 20SG, and 20SB are calculated by complementary color calculation (step St52 in Figure 17). Finally, based on the calculated complementary RGB values, complementary color cork pattern images CSP / 20SY, CSP / 20SG, and CSP / 20SB corresponding to the colors of each trash can 20SY, 20SG, and 20SB are automatically generated (step St53 in Figure 17).

[0095] Next, the image data from when the complementary color cork pattern image CSP / 20SY is projected onto the yellow trash can 20SY, the image data from when the complementary color cork pattern image CSP / 20SG is projected onto the green trash can 20SG, and the image data from when the complementary color cork pattern image CSP / 20SB is projected onto the blue trash can 20SB are captured (step St54 in Figure 17).

[0096] Next, the SfM / MVS processing unit 30e applies SfM / MVS processing to each of the captured data (step St55 in Figure 17) to obtain three-dimensional point cloud data when the complementary color cork pattern image CSP is projected (step St56 in Figure 17).

[0097] On the other hand, separate from this three-dimensional point cloud data, laser data (point cloud data) of the object 20 is acquired using a conventional method, for example, a 3D laser scanner (step St57 in Figure 17).

[0098] Next, after alignment of the three-dimensional point cloud data and laser data is performed (step St58 in Figure 17), calculations are performed using the M3C2 method (step St59 in Figure 17) to obtain the difference in the projection normal direction (step St60 in Figure 17).

[0099] Thus, the difference in the normal direction obtained by the M3C2 method is smallest for all colors when a complementary color cork pattern image CSP is projected, resulting in highly accurate three-dimensional point cloud data. Therefore, it seems useful as an indicator, for example, by determining whether the difference in the normal direction is less than 1 cm.

[0100] However, in the case of the yellow trash can 20SB, the number of tie points calculated during point cloud data generation tends to increase, and the relationship that a larger number of tie points necessarily equals a smaller difference does not always hold true. Therefore, when quantifying the clarity of the projection by the difference in the normal direction, it is desirable to evaluate whether the number of tie points has increased (for example, to about 1.2 times or more of the default) and whether the difference in the normal direction is small (for example, to about half of the default).

[0101] Verification 5 The following shows the results of verifying the accuracy of the generated three-dimensional point cloud data by comparing it with point cloud data (laser data) from a 3D laser scanner, for example, as shown in Figures 18(a) and (b), in order to quantify the clarity of the projection.

[0102] Figure 19 is a flowchart illustrating the procedure for verifying projection clarity using an index, and Figures 20(a) to (e) are schematic diagrams illustrating the procedure for verifying projection clarity using an index.

[0103] Here, we investigate to what extent the accuracy of point cloud data generated by SfM processing can be brought closer to the accuracy of point cloud data acquired by a 3D laser scanner by projecting a complementary color cork pattern image CSP onto the object 20.

[0104] The objects used as the target objects 20 were a red traffic cone 20C, a white trash can 20S, a skin-colored mannequin 20M, and a gray PVC pipe 20E.

[0105] As shown in Figure 19, first, with the complementary color cork pattern image CSP projected onto the traffic cone 20C, trash can 20S, mannequin 20M, and PVC pipe 20E by the projector 10, the camera 12 performs multi-layered omnidirectional photography (step St41).

[0106] Then, after obtaining multi-viewpoint image data of the traffic cone 20C, trash can 20S, mannequin 20M, and PVC pipe 20E, SfM / MVS processing is performed in the SfM / MVS processing unit 30e using commercially available SfM / MVS software (step St42). As a result, even if the traffic cone 20C, trash can 20S, mannequin 20M, and PVC pipe 20E are textureless, it is possible to acquire high-resolution point cloud data, for example, as shown in Figure 18(a) (step St43).

[0107] Subsequently, the coordinate values ​​of the point cloud data are transformed (step St44), while at the same time, measurement data is acquired using a 3D laser scanner, for example, as shown in Figure 18(b) (step St45).

[0108] Then, after the coordinate-transformed point cloud data and the measurement data from the 3D laser scanner are superimposed (step St46), a cross-sectional comparison is performed for each of the traffic cone 20C, trash can 20S, mannequin 20M, and PVC pipe 20E (step St47), and the difference calculation in the normal direction (calculation of the mean and standard deviation from the histogram) and the nearest neighbor distance calculation (step St48) as explained in Verification 4 above, and a density comparison is performed (step St49).

[0109] Figures 20(a) to (e) illustrate the results of the cross-sectional comparison in step St47. As is clear from these figures, by projecting a complementary color cork pattern image CSP, the cross-sections of the traffic cone 20C, trash can 20S, mannequin 20M, and PVC pipe 20E can be reproduced with almost the same accuracy as the measurement data from a 3D laser scanner.

[0110] In particular, the density comparison results in step St49 showed that, for example, for the traffic cone 20C, trash can 20S, and mannequin 20M, the point cloud data obtained by projecting the complementary color cork pattern image CSP had a point cloud density with uniformity equivalent to or better than that of the measurement data from a 3D laser scanner.

[0111] <Second Embodiment> Figure 21 shows a schematic configuration of a three-dimensional model generation apparatus according to the second embodiment. Parts common to the three-dimensional model generation apparatus according to the first embodiment are denoted by the same or similar reference numerals, and detailed explanations are omitted.

[0112] As shown in Figure 21, the three-dimensional model generation apparatus of this embodiment comprises, for example, projectors 10a and 10b that project cork pattern images (specific patterns) CSP1 and CSP2 of the complementary color of the object 20 onto the object 20 as texture images from two directions (multi-directions); a camera 12 capable of photographing the object 20 onto which the complementary color cork pattern images CSP1 and CSP2 are projected from all directions (360 degrees); and a PC (Personal Computer) 30 that generates three-dimensional point cloud data based on the images (captured data) captured by the camera 12 and verifies the accuracy of the generated three-dimensional point cloud data by indexing the clarity of the projection.

[0113] In this embodiment, projectors 10a and 10b project two types of complementary cork pattern images CSP1 and CSP2, each with a different cork pattern (tree bark pattern), onto the object 20.

[0114] Furthermore, the projection distance from each projector 10a, 10b to the object 20 is approximately 2m, and the shooting distance to the object 20 by the camera 12 is approximately 1m. The object 20 is repeatedly photographed in all directions while gradually moving multiple (multi-viewpoint) shooting positions PhP.

[0115] Figure 22 is a flowchart illustrating the procedure for verifying projection clarity using an index in a three-dimensional model generation device.

[0116] In this verification of the 3D model generation device, the illuminance was set to 30 lux, a Canon EOS R5 camera (resolution: 7280 x 5464) was used as the camera 12, and the shooting distance from the shooting position PhP to the object 20 was maintained at approximately 1 m. While moving the shooting position PhP in all directions, the object 20 was captured from multiple viewpoints.

[0117] Furthermore, as object 20, a traffic cone, a mannequin (head only), a trash can, and a PVC pipe were prepared, and an EPSON EB-1780 (for example, DLP projection method, brightness 3000lm, contrast ratio 10,000:1, resolution 1280×800) was used for projector 10a, and an EPSON EB-1771 (for example, DLP projection method, brightness 3000lm, contrast ratio 2000:1, resolution 1280×800) was used for projector 10b.

[0118] Then, with each projection distance to object 20 set to approximately 2m, we decided to investigate the effect of projecting complementary colored cork pattern images CSP1 and CSP2 from two directions on the generation of point cloud data.

[0119] As shown in Figure 22, first, with the cork pattern images CSP1 and CSP2 of the complementary color of the object 20 projected onto the object 20 by projectors 10a and 10b, the camera 12 performs multiple-layer photography of the object 20 (step St31).

[0120] Then, after obtaining multi-viewpoint image data of the object 20, SfM / MVS processing is performed in the SfM / MVS processing unit 30e using commercially available SfM / MVS software (step St32). As a result, even if the object 20 is textureless, it is possible to acquire high-resolution point cloud data in each direction, for example, as shown in Figures 23(a) to (e) (step St33).

[0121] Next, the point cloud data is scaled (step St34) and aligned (step St35). In addition, the number of tie points is compared between the point cloud data with projection (where complementary cork pattern images CSP1 and CSP2 are projected) and the point cloud data without projection (step St36).

[0122] Figure 24 shows a comparison of point cloud data with and without projection, where complementary colored cork pattern images CSP1 and CSP2 are projected onto the object 20.

[0123] As is clear from this figure, compared to point cloud data without projection, projection reduces the dispersion of points and clearly represents the shape. In other words, with projection, it is possible to acquire high-precision point cloud data (high-density cloud points) that closely resembles the shape of the object at high density.

[0124] In other words, by projecting cork pattern images CSP1 and CSP2, which are complementary colors with different cork patterns, onto the object 20 from at least two directions, it becomes possible to increase the number of tie points acquired during point cloud data generation, thereby enabling the generation of more accurate three-dimensional point cloud data.

[0125] <Third Embodiment> Application Example 1 Figure 25 shows a schematic configuration of the three-dimensional model generation device according to the third embodiment, where the slope 20HC of road RD is used as the target object. Parts common to the three-dimensional model generation device according to the first embodiment are denoted by the same or similar reference numerals, and detailed explanations are omitted.

[0126] In other words, as shown in Figures 26(a) and (b), the three-dimensional model generation device of this embodiment enables the measurement of the thickness of clay c by projecting a complementary color cork pattern image CSP onto a slope 20HC to which clay c, which simulates mortar, has been applied, and by acquiring point cloud data.

[0127] For the measurement of slope 20HC, the illumination was set to 27 lux (nighttime), and a Canon EOS R5 camera (resolution: 7280 x 5464) was used as camera 12. As shown in Figure 25, for example, the shooting distance from shooting position PhP1 to slope 20HC was kept at approximately 4 m, and the shooting distance from shooting position PhP2 to slope 20HC was kept at approximately 0.5 m. Multiple viewpoints of slope 20HC were captured while shooting positions PhP1 and PhP2 were moved parallel to each other in one direction.

[0128] Furthermore, projector 10 used a BenQ MH550 (for example, DLP projection method, brightness 3500lm, contrast ratio 20,000:1, resolution 1920×1080), and the projection distance to the slope 20HC was set to approximately 4m.

[0129] According to this application example 1, for example, it was found that the actual thickness of clay c can be measured by projecting a complementary color cork pattern image CSP onto the clay in both the case with and without clay, acquiring point cloud data, and then superimposing the cross-sectional and plan views of the data.

[0130] Application Example 2 In this embodiment, the measurement of cracks, damage, etc., using a three-dimensional model generation device is not limited to cases where the slope 20HC is the target object. For example, as shown in Figure 27, it is also possible to use the inner wall (lining) 51a inside the tunnel 51 as the target object.

[0131] Specifically, to measure the inner wall 51a of the tunnel 51, for example, a projector 10 was placed on the sidewalk 53 along the road surface 52, with a projection distance of approximately 6.6m to the inner wall 51a, and while maintaining a shooting distance of approximately 6.6m from the shooting position PhP of the camera 12 to the inner wall 51a, the projection position PrP and shooting position PhP of the projector 10 were moved in one direction along the sidewalk 53, and the inner wall 51a was continuously photographed.

[0132] According to this application example 2, for example, by repeatedly projecting a complementary color cork pattern image CSP of about 300 inches square and generating point cloud data from continuously captured images, it is possible to detect damaged areas such as cracks and scratches on the inner wall 51a.

[0133] Furthermore, the damaged areas of the inner wall 51a can also be identified by aligning past point cloud data with current point cloud data, as shown in Figures 28(a) and (b), and then generating a heat map (HM) of the difference (displacement distribution) between the point cloud data from two different periods.

[0134] As described above, each embodiment makes it possible to easily generate more accurate three-dimensional point cloud data, regardless of the object's color, even when dealing with objects with minimal texture.

[0135] In other words, by projecting a specific pattern of its complementary color onto the object, it becomes possible to increase the density of the point cloud data generated by SfM. Therefore, it is possible to easily acquire more accurate three-dimensional point cloud data without requiring expensive equipment such as laser scanners or complex processing.

[0136] In particular, since the accuracy of point cloud data can be easily quantified in the field as the clarity of the projection, it becomes possible to consistently generate three-dimensional point cloud data using the optimal complementary color pattern.

[0137] Furthermore, the specific patterns using complementary colors are not limited to cork patterns; various irregular images can be used, for example.

[0138] Furthermore, the projector can also be driven separately by a general-purpose PC. [Explanation of symbols]

[0139] 10. Projector 12. Camera (photographing device) 20 Objects 30 PC 30a Control Unit 30d Tie Point Comparison Section (Indexing Section) 30e SfM / MVS Processing Unit (Point Cloud Data Calculation Unit) 30f Complementary Color Image Generation Unit 30i RMS error calculation section (indexing section) CSP complementary color cork pattern image (specific pattern of complementary colors)

Claims

1. A projector that projects a cork pattern image of the complementary color of the object onto the object, A camera that projects the cork pattern image of the complementary color using the projector and photographs the object while changing the shooting position, A point cloud data calculation unit calculates three-dimensional point cloud data of the object based on the image data of the object captured by the aforementioned camera, A three-dimensional model generation apparatus characterized by comprising the following features.

2. The three-dimensional model generation apparatus according to claim 1, characterized in that the projector projects different complementary cork pattern images onto the object from multiple directions.

3. The three-dimensional model generation apparatus according to claim 1, characterized in that the point cloud data calculation unit calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to the multi-viewpoint shooting data acquired from multiple shooting positions.

4. The three-dimensional model generation apparatus according to claim 1, further comprising a complementary color image generation unit that automatically generates a cork pattern image of the complementary color of the object by photographing the object and calculating its RGB values.

5. The three-dimensional model generation apparatus according to claim 1, further comprising an indexing unit that indexes the quality of the three-dimensional point cloud data calculated by the point cloud data calculation unit as projection clarity, based on the number of tie points obtained by the point cloud data calculation unit and the RMS error due to surface fitting processing to the three-dimensional point cloud data calculated by the point cloud data calculation unit.

6. A step of projecting a cork pattern image of the complementary color of the object onto the object using a projector, The process involves projecting the cork pattern image of the complementary color using the projector, and then photographing the object while changing the shooting position of the camera. A step of calculating three-dimensional point cloud data of the object using a point cloud data calculation unit based on the image data of the object captured by the aforementioned camera, A method for generating a three-dimensional model, characterized by comprising the following features.

7. A three-dimensional model generation program for realizing the three-dimensional model generation apparatus described in any one of Claims 1 to 5 using a computer, The aforementioned computer, A point cloud data calculation unit calculates three-dimensional point cloud data of the object based on the image data of the object, which is captured while changing the shooting position of the camera, with a cork pattern image of complementary colors projected onto the object by a projector. A three-dimensional model generation program characterized by its function as such.

8. A projector that projects a specific pattern of the complementary color of the object onto the object, A camera that photographs the object while changing the shooting position, with the projector projecting the specific pattern of the complementary color, A point cloud data calculation unit calculates three-dimensional point cloud data of the object based on the image data of the object captured by the aforementioned camera, Equipped with, The three-dimensional model generation apparatus is characterized in that the point cloud data calculation unit calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to the multi-viewpoint shooting data acquired from multiple shooting positions.

9. The three-dimensional model generation apparatus according to claim 8, characterized in that the specific pattern of the complementary color is a cork pattern image.

10. The three-dimensional model generation apparatus according to claim 8, characterized in that the projector projects specific patterns of different complementary colors onto the object from multiple directions.

11. The three-dimensional model generation apparatus according to claim 8, further comprising a complementary color image generation unit that automatically generates a specific pattern of the complementary color of the object by photographing the object and calculating its RGB values.

12. The three-dimensional model generation apparatus according to claim 8, further comprising an indexing unit that indexes the quality of the three-dimensional point cloud data calculated by the point cloud data calculation unit as projection clarity, based on the number of tie points obtained by the point cloud data calculation unit and the RMS error due to surface fitting processing to the three-dimensional point cloud data calculated by the point cloud data calculation unit.

13. A step of projecting a specific pattern of the complementary color of the object onto the object using a projector, The process involves projecting a specific pattern of the complementary color using the projector, and then photographing the object while changing the shooting position of the camera; The process involves applying SfM / MVS processing to multi-viewpoint imaging data acquired from multiple shooting positions of the object captured by the aforementioned camera, thereby calculating three-dimensional point cloud data of the object using a point cloud data calculation unit. A method for generating a three-dimensional model, characterized by comprising the following features.

14. A three-dimensional model generation program for realizing the three-dimensional model generation apparatus described in any one of claims 8 to 12 using a computer, The aforementioned computer, A point cloud data calculation unit calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to multi-viewpoint image data acquired from multiple shooting positions of the object, which is photographed while a specific pattern of complementary colors is projected onto the object using a projector, and the shooting position of the camera is changed. A three-dimensional model generation program characterized by its function as such.

15. A projector that projects a cork pattern image of the complementary color of the object onto the object from one direction or multiple directions, A camera that projects the cork pattern image of the complementary color using the projector and photographs the object while changing the shooting position, A point cloud data calculation unit calculates three-dimensional point cloud data of the object based on the image data of the object captured by the aforementioned camera, A three-dimensional model generation apparatus characterized by comprising the following features.

16. The three-dimensional model generation apparatus according to claim 15, characterized in that the point cloud data calculation unit calculates three-dimensional point cloud data of the object by applying SfM / MVS processing to the multi-viewpoint shooting data acquired from multiple shooting positions.

17. The three-dimensional model generation apparatus according to claim 15, further comprising a complementary color image generation unit that automatically generates a cork pattern image of the complementary color of the object by photographing the object and calculating its RGB values.

18. The three-dimensional model generation apparatus according to claim 15, further comprising an indexing unit that indexes the quality of the three-dimensional point cloud data calculated by the point cloud data calculation unit as projection clarity, based on the number of tie points obtained by the point cloud data calculation unit and the RMS error due to surface fitting processing to the three-dimensional point cloud data calculated by the point cloud data calculation unit.

19. A step of projecting a cork pattern image of the complementary color of the object onto the object using a projector from one direction or multiple directions, The process involves projecting the cork pattern image of the complementary color using the projector, and then photographing the object while changing the shooting position of the camera. A step of calculating three-dimensional point cloud data of the object using a point cloud data calculation unit based on the image data of the object captured by the aforementioned camera, A method for generating a three-dimensional model, characterized by comprising the following features.

20. A three-dimensional model generation program for realizing the three-dimensional model generation apparatus described in any one of claims 15 to 18 using a computer, The aforementioned computer, A point cloud data calculation unit calculates three-dimensional point cloud data of the object based on the image data of the object, which is captured while changing the shooting position of the camera, with a cork pattern image of complementary colors projected onto the object from one or more directions using a projector. A three-dimensional model generation program characterized by its function as such.

Images