Apparatus, method and program for creating 3D models

Abstract

The purpose of the present disclosure is to enable the creation of a three-dimensional model even for an object for which points are not arranged at equal intervals and only some parts of a point cloud exist.　A device and a method disclosed herein create a three-dimensional model of an object from point cloud data where each point represents a three-dimensional coordinate, superimpose the three-dimensional model on an image in which the object of the three-dimensional model is imaged, compare the three-dimensional model with the object in the image to select point cloud data to be added to the point cloud data constituting the three-dimensional model, and recreate the three-dimensional model of the object by using the point cloud data including the point cloud data to be added.

Description

[Technical Field]

[0001] The present disclosure relates to a technique for creating a three-dimensional model from point cloud data representing three-dimensional coordinates. [Background technology]

[0002] A technology has been developed to create a 3D model of an outdoor structure using an on-board 3D laser scanner (Mobile Mapping System: MMS) (see, for example, Patent Document 1). The technology in Patent Document 1 creates a point cloud and a scan line in a space where no point cloud exists, and then creates a 3D model.

[0003] There is a demand for 3D modeling of cylindrical objects using point cloud data acquired by a fixed 3D laser scanner. However, because an MMS can acquire point clouds while moving along the object, it can acquire point clouds evenly and at fairly regular intervals within the measurement range, whereas a fixed 3D laser scanner will produce a dense point cloud if it is close to the measurement point and a sparse point cloud if it is far away. Therefore, when creating a 3D model using point cloud data acquired by a fixed 3D laser scanner, this characteristic becomes more pronounced depending on the size and shape of the object.

[0004] With conventional technology, points are interpolated to form a scan line up to a certain threshold distance between point clouds, but if the distance between the points is large and they are not considered to be point clouds on the same object, the points between the points cannot be interpolated.As a result, 3D modeling using fixed 3D laser scanners has the problem of making it difficult to create 3D models of thin objects, such as cables next to utility poles. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-156179 Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure aims to enable the creation of a three-dimensional model even for an object in which the inter-point distances are not evenly spaced and only a portion of the point cloud exists. [Means for solving the problem]

[0007] The apparatus and method of the present disclosure comprise: A 3D model of the object is created from point cloud data, where each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; selecting point cloud data to be added to the point cloud data constituting the three-dimensional model by comparing the three-dimensional model with the object in the image; A three-dimensional model of the object is recreated using the point cloud data including the point cloud data to be added. [Effects of the Invention]

[0008] According to the present disclosure, a 3D model of an object can be created without relying on the distance between 3D points. Therefore, the present disclosure makes it possible to create a 3D model even for an object where the inter-point distances are not evenly spaced and only a portion of the point cloud exists. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows an example of point cloud data. [Figure 2] An example of a 3D model of a structure as an object is shown below. [Figure 3] 1 illustrates an example system configuration of the present disclosure. [Figure 4] 1 shows an example of a point cloud stored in a storage medium. [Figure 5] 1 shows an example of an image stored in a storage medium. [Figure 6] An example of the method of this embodiment will be described below. [Figure 7]An example of the three-dimensional model created in step S1 is shown below. [Figure 8] 10 shows an example of superimposing a three-dimensional model image in step S2. [Figure 9] An example of a corrected 3D model is shown below. [Figure 10] A specific example of step S3 will be shown. [Figure 11] An example of the first method for comparing the sizes of objects is shown below. [Figure 12] An example of adding a point cloud that constitutes a 3D model is shown below. [Figure 13] An example of a corrected 3D model is shown below. [Figure 14] An example of a second method for comparing the size of objects is shown below. [Figure 15] A specific example of step S3 will be shown. [Figure 16] An example of processing in which points of different colors are used as endpoints will be described. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0011] The present disclosure relates to an apparatus and method for creating a 3D model of an object from point cloud data representing 3D coordinates acquired by a 3D laser scanner. FIG. 1 shows an example of point cloud data. Point cloud data is data that represents the surface shape of an object, such as a structure, as a collection of points 91, and each point 91 represents the 3D coordinates of the surface of the structure. By forming lines 92 connecting the points 91 in the 3D point cloud data, a 3D model of the structure as an object can be created. For example, as shown in FIG. 2, a 3D utility pole model 111 and a cable model 112 can be created.

[0012] 3 shows an example of a system configuration according to the present disclosure. The system according to the present disclosure includes a fixed 3D laser scanner 1-1 that measures an object 100, a camera 1-2 that captures an image of the object 100, and the device 5 according to the present disclosure. The device 5 according to the present disclosure includes a calculation processing unit 3 and a display unit 4, and may also include a storage medium 2. The device 5 according to the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0013] The system of the present disclosure stores point cloud data acquired by a fixed 3D laser scanner 1-1 and images captured by a camera 1-2 in a storage medium 2. FIG. 4 shows an example of a point cloud stored in the storage medium 2. In this embodiment, points d1 to d25 are stored between point clouds dp1 and dp2 measured on utility poles. FIG. 5 shows an example of an image stored in the storage medium 2. In this embodiment, an image of cables 102-1, 102-2, and 102-3 stretched between utility poles 101-1 and 101-2 is stored.

[0014] The camera 1-2 may be a camera mounted on the fixed 3D laser scanner 1-1, or may be a separately provided camera. Furthermore, it is desirable for the camera 1-2 to capture images at the same position, direction, and angle of view as the fixed 3D laser scanner 1-1 captures the point cloud. This facilitates superimposing the point cloud captured by the fixed 3D laser scanner 1-1 on the image captured by the camera 1-2. However, because the point cloud in the present disclosure has three-dimensional coordinates, the point cloud can be superimposed on the image based on its relative positions as long as there is three-dimensional positional information on the fixed 3D laser scanner 1-1 and the camera 1-2.

[0015] An example of the method of this embodiment is shown in FIG. A method for generating a 3D model of an object from point cloud data acquired by a 3D laser scanner 1-1, comprising: Step S1 in which the calculation processing unit 3 creates a 3D model of the object from the 3D point cloud data; Step S2 in which the calculation processing unit 3 superimposes the created three-dimensional model of the object on an image of the object; Step S3 in which the arithmetic processing unit 3 corrects the three-dimensional model based on a comparison between the three-dimensional model and the superimposed image; It has.

[0016] In step S1, an object is extracted from the point cloud and a 3D model is created (DBSCAN). DBSCAN is a clustering method that regards a point cloud that meets the condition that there are at least a certain number of points within a threshold as a single cluster and creates a cluster. The object is, for example, utility poles 101-1 and 101-2, or cables 102-1, 102-2, and 102-3. An example in which the object is cables 102-1, 102-2, and 102-3 will be described below.

[0017] FIG. 7 shows examples of the three-dimensional models 112-1, 112-2, and 112-3 created in step S1. In step S2, these three-dimensional models 112-1, 112-2, and 112-3 are superimposed on an image as shown in FIG. 8. Then, in step S3, the three-dimensional models 112-1, 112-2, and 112-3 are compared with the cables 102-1, 102-2, and 102-3 in the image, and the three-dimensional models 112-1, 112-2, and 112-3 are corrected as shown in FIG. 9. This allows the present disclosure to calculate facility information (sag, span length, etc.) from the corrected three-dimensional models. The display unit 4 may display the images shown in FIGS. 7 to 9.

[0018] The present disclosure can determine whether a 3D model has been created perfectly by overlaying it with an image in step S2, and can leave existing 3D models as they are in step S3, or add additional models if they are insufficient. This allows the present disclosure to determine the presence or absence of an object even if only a partial point cloud exists for that object. Therefore, the present disclosure can create 3D models of thin-line objects such as suspension wires, optical cables, electric wires, or horizontal support wires. Furthermore, because the present disclosure can create 3D models of thin-line objects, it can detect the status of thin-line target equipment.

[0019] In step S3, the arithmetic processing unit 3 can automatically correct the three-dimensional model by any method. In this embodiment, a form in which points are complemented so as to match the image, and a form in which the model is complemented so as to match the image are exemplified.

[0020] (First embodiment) 10 shows a specific example of step S3. In this embodiment, the calculation processing unit 3 superimposes the created 3D model on the captured image (S2), assigns color information to the point cloud (S311), and compares the size of the 3D model on the image with that of the object in the image (S312). If the object in the image is larger, the points are interpolated to create a 3D model (S313), and the 3D model is stored in the storage medium 2 (S314).

[0021] Here, this embodiment is performed under the prerequisite that the object is oriented horizontally relative to the acquisition range of the 3D laser scanner. In step S312, any method can be used to superimpose the image and the point cloud and compare the size of the object, but the following can be exemplified. First method: Overlay the point cloud and the image, and compare the size determined by the color pixels of the object in the image with the size of the 3D model. Second method: A method in which the shape and size of an object extracted from an image by image analysis is compared with the size and shape of a 3D model created from a point cloud.

[0022] An example of the first method is shown in Figure 11. The processing unit 3 executes the following processes. ·S2: Overlay point cloud and image. ·S311: Add color information to point clouds. S111: Determine how far the point cloud used to create the 3D model extends within the image. S112: The determined color range is compared with the extracted 3D model to see if it is equivalent. S113: The range determined in the image (pixels of the same color) is extracted from the point cloud. S114 to S116: It is determined whether the extracted point cloud is a candidate for a 3D model. S313: A 3D model is created again using the candidate point group, and the shape of the 3D model is corrected. Specifically, the model is created again using the feature point group. S314: Save the final 3D model and exit.

[0023] Specifically, in step S311, the processing unit 3 superimposes the images (S2), and after the superimposition, associates the point clouds with the images and assigns, to each point cloud, color information of the image at the same position on the image. For example, as shown in Fig. 12, the color information of the cable 102-2 is assigned to points d1 to d7, d21, and d22 that overlap with the cable 102-2.

[0024] In this embodiment, in step S111, the processing unit 3 automatically determines, by image analysis, how far pixels of the same color as the extracted color point cloud of the 3D model extend on the image. For example, as shown in Fig. 5, the processing unit 3 determines the x-coordinate of pixel p1 at the left end of cable 102-2 and the x-coordinate of pixel p22 at the right end of cable 102-2 to determine the range of cable 102-2 on the x-axis.

[0025] After determining the range of pixels of the same color on the image in step S112, the processing unit 3 creates a new model from the points within that range, using a point group that is within a predetermined threshold from an extension of the approximation line of the three-dimensional model (S113 to S116 and S313). For example, points d1 to d25 exist in the range on the x-axis of cable 102-2. In this case, the processing unit 3 creates a new three-dimensional model from points d1 to d25 that are within the threshold from an extension of three-dimensional model 112-1 that is superimposed on cable 102-2.

[0026] Here, for example, the thresholds can be set as follows: x<30 mm, Δy<30 mm, Δz<30 mm, where the direction in which the three-dimensional model extends is the x-axis, the depth is the y-axis, and the height is the z-axis, thereby extracting a point cloud that will be used in the three-dimensional model. As a result, d21 and d22 are set as point clouds that constitute the three-dimensional model (S115), as shown in Fig. 12, and a three-dimensional model is created again (S313), as shown in Fig. 13.

[0027] An example of the second method is shown in Figure 14. The processing unit 3 executes the following processes. ·S2: Overlay point cloud and image. ·S311: Add color information to point clouds. · S121: Estimate the shape and size of the object through image analysis. · S122: Compare the object extracted by image analysis with the 3D model extracted from the point cloud to see if the shape and size are equivalent. S123: The object extracted from the image is extracted from the point cloud, thereby extracting a point cloud that is a candidate for a 3D model. S124 to S126: Determine whether the extracted point cloud is a candidate for a 3D model. S313: A 3D model is created again using the candidate point group, and the shape of the 3D model is corrected. Specifically, a 3D model is created again using the feature point group. S314: Save the final 3D model and exit.

[0028] In this embodiment, in step S121, the processing unit 3 automatically extracts an object on the image to be compared with the 3D model by image analysis based on a dictionary that has been trained in advance. For example, the processing unit 3 extracts the cable 102-2 from the image shown in Fig. 8 using image analysis, and reads out the size and shape of the cable 102-2 from the dictionary.

[0029] Then, in step S122, the calculation processing unit 3 compares the size and shape of the three-dimensional model with the size and shape of the object determined by image analysis. For example, the calculation processing unit 3 compares the size and shape of the three-dimensional model 112-1 with the size and shape of the cable 102-2 estimated in step S121.

[0030] If the size and shape of the object estimated by image analysis in step S121 are larger than those of the three-dimensional model 112-1, a three-dimensional model is again created using a point group within the range of the size and shape of the cable 102-2 estimated by the image analysis, and within a pre-specified threshold from the extension of the approximation line of the three-dimensional model (S123 to S126 and S313). The concept of the threshold is the same as in steps S114 to S116.

[0031] (Second embodiment) In this embodiment, the calculation processing unit 3 estimates the shape of the created 3D model and enlarges the 3D model to a certain size according to that shape. When it encounters a point of a different color from the color point cloud used to create the model, it enlarges the 3D model to that point. Assuming that the 3D model is made up of a color point cloud with the same color information assigned, a corrected 3D model can be created by enlarging the 3D model according to the shape of the 3D model.

[0032] FIG. 15 shows a specific example of step S3. In this embodiment, the arithmetic processing unit 3 superimposes the point cloud on the image (S2) and assigns color information to the point cloud (S311). The arithmetic processing unit 3 determines which facility the 3D model is a model of and infers its shape based on the color information assigned to the point cloud (S131). The 3D model is then automatically extended to an arbitrary size and in an arbitrary direction (S132 to S136). The extension of the 3D model in step S134 is performed, for example, by extracting an approximation line of the created 3D model and recreating a model using a point cloud that is within a threshold from the extension of the approximation line. The approximation line can be an approximation curve or a catenary curve.

[0033] Specifically, the calculation processing unit 3 determines whether the size of the three-dimensional model collides with a point group of a different color (S132). If there is no collision in step S132, the three-dimensional model is extended (S135) and the process proceeds to step S132. For example, as shown in FIG. 16, when the three-dimensional model 112-1 is extended, the color information of the point d22 remains as cable. In this case, the process proceeds to step S132.

[0034] On the other hand, if there is a collision in step S132, the processing unit 3 determines whether the point clouds of different colors exceed a density threshold (S133). If the threshold is not exceeded in S133 (No), the 3D model is extended again (S135) and the process returns to step S132.

[0035] If the threshold value is exceeded in S133 (Yes), the calculation processing unit 3 creates a three-dimensional model with point groups of different colors as end points (S134). For example, as shown in Fig. 16, when extending the three-dimensional model 112-1, the color information of the point d26 is that of the utility pole 101-2. In this case, the three-dimensional model 112-1 is created with the point d21 located in front of the point d26 as the end point (S313).

[0036] If the feature point group is not found even after extending the 3D model 112-1 to the set arbitrary size, the calculation processing unit 3 corrects the 3D model to the original size (S136), creates a 3D model (S313), and saves it (S314). When recreating the 3D model in step S31, it may be performed using all point groups that are within a threshold from the approximation line of the 3D model. The threshold is set as in S113 to S116, and is the distance from the approximation line to the point group.

[0037] As described above, in this embodiment, the arithmetic processing unit 3 extends the approximation line of the 3D model, and when it finds a boundary where the color changes and is composed of point cloud densities equal to or greater than a certain value, it determines that this is an end point of the 3D model. Color information such as RGB values ​​is used as a reference to determine whether the color has changed. For example, when the arithmetic processing unit 3 finds a change in color equal to or greater than a pre-specified value, it automatically determines that this is a color change point by using the color point cloud, or by superimposing the point cloud on the image and extracting a point on the image that is on the extension of the approximation line of the 3D model and has a point cloud density equal to or greater than a certain value, and using the color information of the pixels at that point.

[0038] In this embodiment, if a boundary point cloud can be acquired from the fixed 3D laser scanner 1-1 even at a long distance for an object with a distinctive shape, it is possible to create a 3D model with high accuracy. For example, in the case of a cable, a 3D model can be created at a short distance from the fixed 3D laser scanner, and a catenary curve can be estimated. Cables are installed on utility poles and house walls, and in images, the cable is easy to understand because the color of the cable is different from that of the utility pole or house wall, and they are easier to acquire than cable end points. These point clouds can be used as end points to extend the 3D model. This makes it possible to create a highly accurate 3D model.

[0039] In addition, by learning in advance what shape the created 3D model originally was, it becomes possible to accurately extend the 3D model using approximation lines. [Industrial Applicability]

[0040] The present disclosure can be applied to the information and communications industry. [Explanation of symbols]

[0041] 1-1: Fixed 3D laser scanner 1-2: Camera 2:Storage medium 3: Processing unit 4: Equipment 91: point 92: line 100: Object 101-1, 101-2: Electric poles 102-1, 102-2, 102-3: Cable 111: Utility pole model 112: Cable model

Claims

1. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; comparing the size of the object in the image with the three-dimensional model; If the object is larger, the approximation line of the three-dimensional model is extended, and a three-dimensional model of the object is recreated, including the point cloud that is within a threshold from the approximation line.

1. An apparatus comprising: assigning color information of the object in the image to each point superimposed on the object; a color range having the same color information as the point cloud used to create the three-dimensional model is set as the size of the object; The size comparison is performed by comparing the range and shape of the same color range with the three-dimensional model. An apparatus characterized in that

2. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; comparing the size of the object in the image with the three-dimensional model; If the object is larger, the approximation line of the three-dimensional model is extended, and a three-dimensional model of the object is recreated, including the point cloud that is within a threshold from the approximation line.

1. An apparatus comprising: assigning color information of the object in the image to each point superimposed on the object; extracting the object from the image by image analysis and determining the size of the object; The size comparison is performed by comparing the extracted object with the three-dimensional model. An apparatus characterized in that

3. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; comparing the size of the object in the image with the three-dimensional model; If the object is larger, the approximation line of the three-dimensional model is extended, and a three-dimensional model of the object is recreated, including the point cloud that is within a threshold from the approximation line.

1. A method comprising: assigning color information of the object in the image to each point superimposed on the object; a color range having the same color information as the point cloud used to create the three-dimensional model is set as the size of the object; The size comparison is performed by comparing the range and shape of the same color range with the three-dimensional model. A method characterized by:

4. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; comparing the size of the object in the image with the three-dimensional model; If the object is larger, the approximation line of the three-dimensional model is extended, and a three-dimensional model of the object is recreated, including the point cloud that is within a threshold from the approximation line.

1. A method comprising: assigning color information of the object in the image to each point superimposed on the object; extracting the object from the image by image analysis and determining the size of the object; The size comparison is performed by comparing the extracted object with the three-dimensional model. A method characterized by:

5. A program for implementing a computer as the device according to claim 1 or 2.

Images