Apparatus, method and program for creating 3D models

Abstract

The purpose of the present disclosure is to enable creation of a three-dimensional model even for an object for which point-to-point distances are not aligned at equal intervals and for only some parts of which point groups exist.　The present disclosure is a device and a method wherein: a three-dimensional model of an object is created from point group data, each point of which represents three-dimensional coordinates; the three-dimensional model is superimposed on an image of the object of the three-dimensional model; a superimposition image generated by the superimposition is displayed; and when the range of the object is input in the superimposition image, point group data having points positioned in the range of the superimposition image are used to create a three-dimensional model again.

Description

[Technical Field]

[0001] The present disclosure relates to a technique for creating a three-dimensional model from point cloud data representing three-dimensional coordinates. [Background technology]

[0002] A technology has been developed to create a 3D model of an outdoor structure using an on-board 3D laser scanner (Mobile Mapping System: MMS) (see, for example, Patent Document 1). The technology in Patent Document 1 creates a point cloud and a scan line in a space where no point cloud exists, and then creates a 3D model.

[0003] There is a demand for 3D modeling of cylindrical objects using point cloud data acquired by a fixed 3D laser scanner. However, because an MMS can acquire point clouds while moving along the object, it can acquire point clouds evenly and at fairly regular intervals within the measurement range, whereas a fixed 3D laser scanner will produce a dense point cloud if it is close to the measurement point and a sparse point cloud if it is far away. Therefore, when creating a 3D model using point cloud data acquired by a fixed 3D laser scanner, this characteristic becomes more pronounced depending on the size and shape of the object.

[0004] With conventional technology, points are interpolated to form a scan line up to a certain threshold distance between point clouds, but if the distance between the points is large and they are not considered to be point clouds on the same object, the points between the points cannot be interpolated.As a result, 3D modeling using fixed 3D laser scanners has the problem of making it difficult to create 3D models of thin objects, such as cables next to utility poles. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-156179 Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure aims to enable the creation of a three-dimensional model even for an object in which the inter-point distances are not evenly spaced and only a portion of the point cloud exists. [Means for solving the problem]

[0007] The apparatus and method of the present disclosure comprise: A 3D model of the object is created from point cloud data, where each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; Displaying a superimposed image generated by the superimposition; When the range of the object in the superimposed image is input, a three-dimensional model is created again using point cloud data in which points are located within the range of the superimposed image. [Effects of the Invention]

[0008] According to the present disclosure, a 3D model of an object can be created without relying on the distance between 3D points. Therefore, the present disclosure makes it possible to create a 3D model even for an object where the inter-point distances are not evenly spaced and only a portion of the point cloud exists. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows an example of point cloud data. [Figure 2] An example of a 3D model of a structure as an object is shown below. [Figure 3] 1 illustrates an example system configuration of the present disclosure. [Figure 4] 1 shows an example of a point cloud stored in a storage medium. [Figure 5] 1 shows an example of an image stored in a storage medium. [Figure 6] An example of the method of this embodiment will be described below. [Figure 7] An example of the three-dimensional model created in step S1 is shown below. [Figure 8] 1 shows an example of a superimposed image in which a three-dimensional model is superimposed on an image. [Figure 9] An example of the three-dimensional model created in step S3 is shown below. [Figure 10] An example of inputting the range of an object is shown below. [Figure 11] A specific example of step S3 will be shown. [Figure 12] An example of the first method for comparing the sizes of objects is shown below. [Figure 13] An example of adding a point cloud that constitutes a 3D model is shown below. [Figure 14] An example of a corrected 3D model is shown below. [Figure 15] An example of a second method for comparing the size of objects is shown below. [Figure 16] An example of displaying the size of an object is shown. [Figure 17] A specific example of step S3 will be shown. [Figure 18] An example of an extended 3D model is shown. [Figure 19] An example of setting the endpoints of a 3D model is shown below. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0011] The present disclosure relates to an apparatus and method for creating a 3D model of an object from point cloud data representing 3D coordinates acquired by a 3D laser scanner. FIG. 1 shows an example of point cloud data. Point cloud data is data that represents the surface shape of an object, such as a structure, as a collection of points 91, and each point 91 represents the 3D coordinates of the surface of the structure. By forming lines 92 connecting the points 91 in the 3D point cloud data, a 3D model of the structure as an object can be created. For example, as shown in FIG. 2, a 3D utility pole model 111 and a cable model 112 can be created.

[0012] 3 shows an example of a system configuration according to the present disclosure. The system according to the present disclosure includes a fixed 3D laser scanner 1-1 that measures an object 100, a camera 1-2 that captures an image of the object 100, and the device 5 according to the present disclosure. The device 5 according to the present disclosure includes a calculation processing unit 3 and a display unit 4, and may also include a storage medium 2. The device 5 according to the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0013] The system of the present disclosure stores point cloud data acquired by a fixed 3D laser scanner 1-1 and images captured by a camera 1-2 in a storage medium 2. FIG. 4 shows an example of a point cloud stored in the storage medium 2. In this embodiment, points d1 to d25 are stored between point clouds dp1 and dp2 measured on utility poles. FIG. 5 shows an example of an image stored in the storage medium 2. In this embodiment, an image of cables 102-1, 102-2, and 102-3 stretched between utility poles 101-1 and 101-2 is stored.

[0014] The camera 1-2 may be a camera mounted on the fixed 3D laser scanner 1-1, or may be a separately provided camera. Furthermore, it is desirable for the camera 1-2 to capture images at the same position, direction, and angle of view as the fixed 3D laser scanner 1-1 captures the point cloud. This facilitates superimposing the point cloud captured by the fixed 3D laser scanner 1-1 on the image captured by the camera 1-2. However, because the point cloud in the present disclosure has three-dimensional coordinates, the point cloud can be superimposed on the image based on its relative positions as long as there is three-dimensional positional information on the fixed 3D laser scanner 1-1 and the camera 1-2.

[0015] An example of the method of this embodiment is shown in FIG. A method for generating a 3D model of an object from point cloud data acquired by a 3D laser scanner 1-1, comprising: Step S1 in which the calculation processing unit 3 creates a 3D model of the object from the 3D point cloud data; Step S2 in which the calculation processing unit 3 superimposes the created three-dimensional model of the object on an image of the object; Step S3 in which the arithmetic processing unit 3 corrects the three-dimensional model based on a comparison between the three-dimensional model and the superimposed image; It has.

[0016] In step S1, an object is extracted from the point cloud and a 3D model is created (DBSCAN). DBSCAN is a clustering method that regards a point cloud that meets the condition that there are at least a certain number of points within a threshold as a single cluster and creates a cluster. The object is, for example, utility poles 101-1 and 101-2, or cables 102-1, 102-2, and 102-3. An example in which the object is cables 102-1, 102-2, and 102-3 will be described below.

[0017] Fig. 7 shows an example of the three-dimensional models 112-1, 112-2, and 112-3 created in step S1. In step S2, these three-dimensional models 112-1, 112-2, and 112-3 are superimposed on an image as shown in Fig. 8. Then, in step S3, the three-dimensional models 112-1, 112-2, and 112-3 are compared with the cables 102-1, 102-2, and 102-3 in the image, and the three-dimensional models 112-1, 112-2, and 112-3 are corrected as shown in Fig. 9. This allows the present disclosure to calculate facility information (sag, span length, etc.) from the corrected three-dimensional models.

[0018] In the present disclosure, in step S2, the superimposed image generated by the superposition is displayed on the display unit 4. Then, when the user inputs the range of the object in the superimposed image, as with cursors 103-1 and 103-2 shown in Fig. 10, the arithmetic processing unit 3 executes step S3. In this step S3, the arithmetic processing unit 3 recreates a 3D model using point cloud data whose points are located within the range of the superimposed image.

[0019] The present disclosure can determine whether a 3D model has been created perfectly by overlaying it with an image in step S2, and can leave existing 3D models as they are in step S3, or add additional models if they are insufficient. This allows the present disclosure to determine the presence or absence of an object even if only a partial point cloud exists for that object. Therefore, the present disclosure can create 3D models of thin-line objects such as suspension wires, optical cables, electric wires, or horizontal support wires. Furthermore, because the present disclosure can create 3D models of thin-line objects, it can detect the status of thin-line target equipment.

[0020] The method for inputting the range of the object in step S2 is arbitrary. For example, as shown in Fig. 10, the range may be input using the cursor position on the screen, or may be input by dragging.

[0021] In step S3, the three-dimensional model may be corrected by any method. In this embodiment, two methods are exemplified: one in which points are interpolated to match the image, and the other in which the three-dimensional model is interpolated to match the image.

[0022] (First embodiment) 11 shows a specific example of step S3. In this embodiment, the arithmetic processing unit 3 superimposes the created 3D model on the captured image (S2), and displays the superimposed image created by the superimposition on the display unit 4. The arithmetic processing unit 3 acquires the range of the object in the superimposed image, and compares the size of the 3D model on the superimposed image with the size of the object in the image (S311). If the object in the image is larger, the arithmetic processing unit 3 interpolates points to create a 3D model (S312), and stores the 3D model in the storage medium 2 (S313).

[0023] In step S312, any method may be used to superimpose the image and the point cloud and compare the size of the object, but the following may be exemplified. First method: Overlaying the point cloud and the image, and comparing the size of the same color pixel specified in the overlaid image with the size of the 3D model. Second method: Matching the equipment information with a database prepared in advance and comparing the size with the 3D model.

[0024] FIG. 12 shows an example of the first method. The first method determines the extent of the point cloud representing the cable used to create the 3D model of the cable. Specifically, the processing unit 3 superimposes an image (S2), assigns color information of the cable to the point cloud (S111), and obtains the range of the same-colored pixels in the point cloud used to create the 3D model on the image (S112). If the color range and shape differ from those of the 3D model (No in S113), the processing unit 3 extracts the point cloud included in the range specified in S112 (S114 to S117) and creates a 3D model again (S312 and S313).

[0025] Specifically, in step S111, after the superposition, the point clouds are associated with the images, and color information of the image at the same position on the image is assigned to each point cloud. For example, three-dimensional model 112-1 overlaps with cable 102-2. In this case, point clouds d1 to d6 constituting three-dimensional model 112-1 are associated with cable 102-2, and color information of cable 102-2 is assigned to point clouds d1 to d6.

[0026] In step S112, the user manually selects how far on the image the pixels of the same color as the point group d1 to d6 corresponding to the extracted three-dimensional model 112-1 extend, as shown by cursors 103-1 and 103-2 in Fig. 10. This specifies the range on the image where the same color as cable 102-2 extends, and a three-dimensional model is created again from the point groups d1 to d25 within that range, using point groups that are within a pre-specified threshold on an extension of the approximation line of the three-dimensional model (S113 to S117, S312).

[0027] Here, the thresholds can be exemplified by extracting a point cloud likely to be used for the three-dimensional model, with the distances between each point being Δx<30 mm, Δy<30 mm, and Δz<30 mm, where the direction in which the approximation line of the three-dimensional model 112-1 extends is the x-axis, the depth is the y-axis, and the height is the z-axis. As a result, as shown in FIG. 13, d21 and d22 are taken as the point cloud constituting the three-dimensional model (S116), and a new three-dimensional model is created (S312). As a result, the three-dimensional model 112-1 can be corrected, as shown in FIG. 14. In this way, in this embodiment, the coordinates of the point cloud extracted in steps S114 to S117 are also used to correct the three-dimensional model 112-1, making it possible to determine whether the three-dimensional model can be used, even if pixels of the same color are spread over a wide area.

[0028] An example of the second method is shown in Fig. 15. In the second method, the calculation processing unit 3 superimposes a three-dimensional model on an image (S2) and displays the three-dimensional model so that it can be compared with cable information such as sag, span length, and position stored in a database in advance (S121). For example, as shown in Fig. 16, the calculation processing unit 3 displays an area 104 indicating the size of cable 102-2 on the display unit 4. This allows the user to determine the area of ​​cable 102-2 even if the image is unclear.

[0029] 10, the calculation processing unit 3 extracts a point cloud having the same size and shape as the cable information (S122 to S126) and creates a 3D model (S312 and S313). As with the first method, by using the coordinates of the point cloud, it is possible to determine whether pixels of the same color can be used for a 3D model even if they are spread over a wide area.

[0030] In this embodiment, the three-dimensional model 112-1 overlaps with the cable 102-2. In this case, in step S121, when the three-dimensional model 112-1 is selected, the calculation processing unit 3 selects the cable 102-2, which is the object to be compared with the three-dimensional model 112-1, on the superimposed image. Then, a database prepared in advance is searched for a corresponding object based on the position and length of the selected cable 102-2, and information such as its size, shape, position, etc. is retrieved.

[0031] Furthermore, in steps S122 to S126, the calculation processing unit 3 compares the information of the three-dimensional model 112-1 with the information of the cable 102-2 in the database, and if the cable 102-2 in the database is larger or has a different shape, the calculation processing unit 3 may extract point cloud candidates constituting the three-dimensional model 112-1 from the point clouds d1 to d25 based on the information in the database. Then, the calculation processing unit 3 creates a new three-dimensional model from the point clouds d1 to d25 within the target object range using a point cloud that is within a predetermined threshold from an extension of the approximation line of the three-dimensional model. The concept of the threshold is the same as in steps S114 to S117.

[0032] As described above, this embodiment is capable of adding a point cloud to a location where no point cloud exists between the endpoints, so that even if the inter-point distances are not evenly spaced and only a portion of the point cloud exists, it is possible to create a 3D model of the object with high accuracy.

[0033] (Second embodiment) 17 shows a specific example of step S3. In this embodiment, the shape of the created 3D model is estimated. The created 3D model is superimposed on an image (S2), the shape of the 3D model is inferred (S321), an approximation line inferred from the 3D model is displayed on the image (S322), endpoints of the approximation line are selected on the superimposed image (S323), and if a point group exists near the selected location and within a threshold from the approximation line (S324), the point is used as the endpoint and a 3D model is recreated (S325) and saved (S326).

[0034] For example, the arithmetic processing unit 3 superimposes the three-dimensional model 112-1 on the image as shown in Fig. 8 (S2). Then, the arithmetic processing unit 3 extracts an approximation line of the three-dimensional model 112-1, extends the approximation line of the three-dimensional model 112-1 as shown in Fig. 18, and displays it on the display unit 4 (S322). When the arithmetic processing unit 3 acquires the range of the cable 102-2 such as cursors 103-1 and 103-2 shown in Fig. 10, it creates a three-dimensional model again using a cloud of points on the approximation line and in the range of cursors 103-1 and 103-2 (S323 to S327). Here, the approximation line can be an approximation curve or a catenary curve.

[0035] In this embodiment, in step S323, an image in which the approximation line of the three-dimensional model 112-1 intersects with the utility poles 101-1 and 101-2 is displayed on the display unit 4. Therefore, the user can use this intersection to easily select the end points of the three-dimensional model at a glance.

[0036] If a point cloud exists at the selected location within the threshold from the approximation line (Yes in step S324), the point cloud is set as an end point (step S326), and the 3D model is recreated. On the other hand, if a point cloud that can be an end point does not exist (No in step S324), the point cloud that is closest to the end point on the selected approximation line and is within the threshold from the approximation line is set as the end point (step S327). In this embodiment, as shown in FIG. 19, point d1 exists at the location of cursor 103-1 shown in FIG. 10, and point d21 exists at the location of cursor 103-2 shown in FIG. 10. Therefore, the calculation processing unit 3 recreates the 3D model 112-1 with point d1 and point d21 as end points.

[0037] The threshold is set to the distance from the approximation line to the point cloud as in S114 to S117. When recreating the three-dimensional model, all point clouds between the endpoints of the approximation line and within the threshold from the approximation line may be used, or point clouds having the same color information as cable 102-2 may be selectively used.

[0038] For cables, a 3D model can be created at close range using the fixed 3D laser scanner 1-1, and catenary curves can be estimated. Cables are installed on utility poles and house walls, and in images, the cables are easy to understand because they are different colors from the utility poles and house walls, and are easier to obtain than cable endpoints. These point clouds can be used as endpoints to extend the approximation lines of the 3D model. This makes it possible to create a highly accurate 3D model.

[0039] As described above, this embodiment can create a 3D model in which the areas where no point cloud exists between the endpoints are corrected by selecting endpoints and enlarging the model according to the shape of the 3D model.

[0040] In this embodiment, the endpoints can be visually identified by combining them with an image. Also, if a boundary point cloud can be acquired from a target object with a distinctive shape even at a long distance from the fixed 3D laser scanner 1-1, it is possible to create a 3D model with high accuracy. Furthermore, by learning in advance what shape the created 3D model originally was, it becomes possible to accurately extend the 3D model using approximate lines. [Industrial Applicability]

[0041] The present disclosure can be applied to the information and communications industry. [Explanation of symbols]

[0042] 1-1: Fixed 3D laser scanner 1-2: Camera 2:Storage medium 3: Processing unit 4: Display section 5: Equipment 91: point 92: line 100: Object 101-1, 101-2: Electric poles 102-1, 102-2, 102-3: Cable 111: Utility pole model 112: Cable model

Claims

1. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; Displaying a superimposed image generated by the superimposition; When the range of the object in the superimposed image is input, a size comparison is performed between the three-dimensional model and the object in the superimposed image; If the object is larger, the approximation line of the three-dimensional model is extended, and the three-dimensional model is recreated including the point cloud that is within a threshold from the approximation line. Device.

2. assigning color information of the object in the image to each point superimposed on the object; a color range having the same color information as the point cloud used to create the three-dimensional model is set as the size of the object; the size comparison is performed by comparing the range and shape of the same color range with the three-dimensional model; 10. The apparatus of claim 1.

3. acquiring the shape and size of an object in the image by referencing a database storing information about the size of the object; Displaying the acquired shape and size of the object on the superimposed image; the size comparison is a comparison of the shape and size of the object with the three-dimensional model; 10. The apparatus of claim 1.

4. A 3D model of the object is created from point cloud data in which each point represents a 3D coordinate. superimposing the three-dimensional model on an image of the object of the three-dimensional model; Displaying a superimposed image generated by the superimposition; When the range of the object in the superimposed image is input, a size comparison is performed between the three-dimensional model and the object in the superimposed image; If the object is larger, the approximation line of the three-dimensional model is extended, and the three-dimensional model is recreated including the point cloud that is within a threshold from the approximation line. method.

5. A program for implementing a computer as the device according to any one of claims 1 to 3.

Images