Method for establishing static digital twin model of tunnel

Abstract

The invention relates to a tunnel digital twin model establishment method, and belongs to the technical field of tunnel intelligent construction. According to the technical scheme, a tunnel is scanned, and a tunnel image and three-dimensional point cloud data are obtained at the same time; the three-dimensional point cloud of the tunnel is preprocessed, and noise and tunnel irrelevant point cloud are removed; splicing the images shot by the laser scanner and eliminating splicing seams to obtain a panorama, and storing the obtained panorama in an equidistant cylindrical projection mode; performing surface reconstruction by using the denoised point cloud to obtain a seamless high-quality grid model; and fusing the processed three-dimensional point cloud data and two-dimensional image data to obtain a static digital twin tunnel model with high reduction degree and accurate scale. The method has the beneficial effects that a basis is provided for intelligentization, parameterization, automatic construction, management and control and operation and maintenance of the whole life cycle of the tunnel, possibility is provided for digital enabling of tunnel construction, and a digital twinning model building method is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field of tunnel intelligent construction, and in particular relates to a method for establishing a tunnel digital twin model, and in particular relates to a method for establishing a tunnel static digital twin model based on laser scanning data. Background Art

[0002] Digital twins effectively support high-fidelity modeling, virtual reality mapping, and analysis-based decision-making, but their research and application are still in the initial stage. One difficulty that must be faced in the development and implementation of digital twins is the establishment of digital twin virtual models. Existing digital twin virtual models mainly include knowledge and semantic models, geometric models, BIM models, etc. Among them, the geometric model can clearly and intuitively observe the actual construction of the tunnel, and can monitor and manage the tunnel in an immersive way.

[0003] However, there is no mature method for establishing the existing geometric model. Currently, the two commonly used methods are 3D reconstruction methods based on 3D laser scanners and photography combined with motion structure recovery (MVS) algorithms. The internal contour and geometric information of the tunnel can be obtained through laser scanners. However, the 3D point cloud does not have intuitive visualization capabilities and lacks some texture information. The degree of restoration of the tunnel is low and a large amount of effective information is ignored. Therefore, the 3D point cloud cannot be used as a twin model of the tunnel. The method of using camera combined with MVS can obtain the point cloud of the tunnel by matching feature points and mapping, and obtain a 3D model with a high degree of visualization. However, this method has high requirements on the level of manual shooting and the time complexity of model reconstruction is high. The 3D model obtained by this method can only be used for visualization, does not have size information, and has limited effect on tunnel engineering.

[0004] Therefore, it is necessary to develop a method that can simultaneously utilize two data acquisition methods to establish a virtual digital model of the tunnel with high fidelity and high restoration and dimensional information. Summary of the invention

[0005] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a method for establishing a static digital twin model of a tunnel. The method has fast speed, high flexibility, high fidelity, wide application range, and can integrate a static tunnel digital twin model of two modal data.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for establishing a static digital twin model of a tunnel, the specific steps are as follows:

[0008] S1. Scan the tunnel using a 3D laser scanner equipped with a synchronous camera to obtain tunnel images and 3D point cloud data at the same time;

[0009] S2, preprocessing the three-dimensional point cloud of the tunnel to remove noise and tunnel-irrelevant point clouds;

[0010] S3, stitching the images taken by the laser scanner and eliminating the stitching seams to obtain a panoramic image, and storing the obtained panoramic image using an equidistant cylindrical projection method;

[0011] S4, using the denoised point cloud to perform surface reconstruction and obtain a seamless high-quality mesh model;

[0012] S5. The processed three-dimensional point cloud data and two-dimensional image data are fused to obtain a static digital twin tunnel model with high restoration and precise scale.

[0013] Furthermore, the specific content of step S1 is to use a 3D laser scanner equipped with a synchronous photography system to scan the tunnel, and simultaneously obtain 3D point cloud data and image data of the tunnel entity.

[0014] Furthermore, the specific steps of step S2 are: projecting the point cloud onto the cross section of the face, dividing the cross section into grids, normalizing the number of point clouds projected into the grids to the pixel grayscale value of the image, and filtering the point cloud noise by combining data dimensionality reduction with image processing.

[0015] Furthermore, in step S3, the image data obtained in step S1 needs to be preprocessed. The specific operation method is to use the SIFT feature point matching method to obtain the obtained image data, and stitch the two photos with overlapping areas one by one to obtain a complete tunnel stitching panorama. The stitched pictures are used to eliminate the image stitching seams caused by factors such as the camera's aperture and lens distortion by using Poisson fusion or multi-band fusion methods. Furthermore, the spherical coordinates of the stitched panorama are projected into rectangular images for storage (JPG, PNG, etc.), and the projection method is equidistant cylindrical projection.

[0016] Furthermore, step S4 needs to use the processed point cloud obtained in step S1 for surface reconstruction. First, the point cloud is divided into Delaunay tetrahedrons. Delaunay forms closed tetrahedrons between all point clouds. The laser emission center of the laser scanner is regarded as the source point, and the line between the source point and the point cloud is regarded as a "line of sight". Visibility constraints are imposed on the triangular facets of the Delaunay tetrahedron, and a higher penalty is imposed on the facets that the "line of sight" passes through. A penalty mechanism is established to minimize the penalty on the point cloud surface that is to be retained, and the minimum cut is solved by the minimum st cut algorithm to obtain a sealed and leak-free tunnel mesh model.

[0017] Furthermore, the main content of step S5 is to fuse the three-dimensional point cloud data and the two-dimensional image data that have been processed separately. The specific method of fusion is to establish a correspondence between the point cloud and the pixel for the panoramic image and the grid model through equidistant cylindrical projection. Each triangular face of the grid can obtain an image patch that matches it. The mapping information and the grid model information are stored in the three models, and the obtained static tunnel digital twin model is viewed using visualization software.

[0018] Beneficial effects of the present invention:

[0019] Compared with the prior art, the method for establishing a static digital twin model of a tunnel according to the present invention has the following technical characteristics or beneficial effects:

[0020] (1) Efficient fusion of multiple data acquisition methods: This paper proposes an innovative method that can simultaneously utilize two data acquisition methods, a 3D laser scanner and a synchronous camera, to effectively fuse high-precision 3D point cloud data and rich texture image data. This method not only improves the efficiency of building a digital twin model, but also ensures the accuracy and integrity of the model.

[0021] (2) High fidelity and high restoration: By removing noise and irrelevant point clouds, stitching panoramas and eliminating stitching seams, and reconstructing using high-quality mesh models, the present invention can establish a virtual digital model of the tunnel with high fidelity and high restoration. This model can clearly and intuitively reflect the actual construction status of the tunnel, providing managers with an immersive monitoring and management experience.

[0022] (3) Accurate scale information: Different from the 3D model obtained by image stitching alone, the method of the present invention combines 3D point cloud data, so that the established digital twin model not only has a high degree of visualization, but also contains accurate scale information. This is of great significance for the decision-making and management of tunnel engineering.

[0023] (4) High flexibility and wide application range: The method of the present invention is not only applicable to different types of tunnels, but can also be customized and optimized according to different needs. For example, the parameters of point cloud preprocessing can be adjusted according to actual needs, different image stitching algorithms can be selected, etc., to meet the application needs in different scenarios.

[0024] (5) Promote the development of digital twin technology: The proposal and implementation of this invention not only solves the challenges faced by the current digital twin technology in tunnel modeling, but also provides a useful reference and reference for the application of digital twin technology in other fields. It promotes the further development of digital twin technology and makes it possible for safer, more timely and efficient management and construction decisions.

[0025] (6) Significant practical application value: The static digital twin model of the tunnel established by the present invention provides managers with a more flexible and efficient way to remotely explore the tunnel situation. This helps to timely discover and solve problems in tunnel construction and management, improve construction efficiency and quality, and reduce safety risks. At the same time, the model can also provide an important reference for tunnel maintenance, reconstruction and expansion.

[0026] The present invention has made significant progress and innovation in the establishment of static digital twin models of tunnels, and has broad application prospects and important practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0028] Figure 1 It is the technical roadmap of the present invention;

[0029] Figure 2 This is a data collection diagram of Example 1;

[0030] Figure 3 It is a diagram of the equidistant cylindrical projection method of the present invention;

[0031] Figure 4 This is the panorama stitching result image of Example 1;

[0032] Figure 5 It is the Delaunay tetrahedronization flow chart of the present invention;

[0033] Figure 6 This is the point cloud surface reconstruction result diagram in Example 1;

[0034] Figure 7 This is a diagram of the multimodal data fusion method of the present invention;

[0035] Figure 8 This is a display diagram of the static tunnel digital twin model in Example 1. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Figure 1-8The method for establishing a static digital twin model of a tunnel is further explained, and the technical solutions in the embodiments of the present application are clearly and completely described; obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and all other embodiments obtained by ordinary technicians in this field without creative work based on the embodiments in the present application are within the scope of protection of the present application.

[0037] The present invention provides a method for establishing a static digital twin model of a tunnel. The technical route is as follows: Figure 1 As shown, the specific steps include:

[0038] S1, uses a 3D laser scanner equipped with a synchronous camera to scan the tunnel and obtain tunnel images and 3D point cloud data at the same time.

[0039] S2, preprocess the three-dimensional point cloud of the tunnel to remove noise and tunnel-irrelevant point clouds.

[0040] S3, stitching the images taken by the laser scanner and eliminating the stitching seams to obtain a panoramic image, and storing the obtained panoramic image using an equidistant cylindrical projection method.

[0041] S4, uses the denoised point cloud to perform surface reconstruction and obtain a seamless high-quality mesh model.

[0042] S5, the processed three-dimensional point cloud data and two-dimensional image data are fused to obtain a static digital twin tunnel model with high restoration and precise scale.

[0043] Example 1

[0044] S1, use a 3D laser scanner with a synchronous photography system to scan the tunnel. In this embodiment, a Faro Focus 3D laser scanner is used to collect data from the tunnel. The collection diagram of this embodiment is as follows: Figure 2 shown.

[0045] S2, denoising the point cloud data obtained by the scanner. The specific processing method is to project the point cloud onto the tunnel cross section and divide the cross section into grids. The grid coordinate calculation formula after the point cloud is projected is:

[0046] row=up[(xx min +w) / s] (1)

[0047] col=up[(z max -z+w) / s] (2)

[0048] Where row and col represent the number of rows and columns of the grid where the projection is located, up represents rounding up, w represents the width of the blank space around the tunnel, and s represents the size of the divided grid. Each grid on the cross section is regarded as a pixel, and the number of point clouds projected by each grid is used as the grayscale value of the pixel. The threshold is set to filter out pixels with low grayscale values, and the point clouds corresponding to the pixels are cleaned up. This method converts the noise removal problem into a two-dimensional image filtering problem.

[0049] S3, seamlessly stitch the image data captured by the laser scanner. The specific method is to first find the SIFT feature points of each image. The SIFT feature is located at the maximum / minimum value of the difference of the Gaussian function in the scale space. The scale space of the image is defined as the function L(x, y, σ), which is obtained by convolving the variable scale Gaussian function and the input image:

[0050] L(x,y,σ)=G(x,y,σ)*I(x,y) (3)

[0051] Where * represents the convolution operation, I represents the grayscale value of the image pixel, σ is the scale space factor, and:

[0052]

[0053] The scale difference function is:

[0054] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)=L(x,y,kσ)-

[0055] L(x,y,σ) (5)

[0056] Where k represents the size of the Gaussian convolution kernel. The feature points are obtained by finding the local extreme points of the differential function of each image. Then the gradient value near the feature point (usually 16*16 pixels) is calculated. The gradient value represents the partial derivative of the local grayscale change of the pixel point, and has two characteristics: absolute value and direction. The gradient direction is classified into one direction every 10 degrees from (0-360)°, and the gradient direction histogram in the 4*4 pixel block is counted to obtain the gradient descriptor of the local pixel area. At this point, each feature point can get a descriptor, which contains the location information of the feature point, scale space, gradient direction (x, y, σ, θ), etc. This descriptor not only contains the information of the feature point itself, but also contains the surrounding pixels that contribute to the alignment, and the descriptor has a high uniqueness.

[0057] Furthermore, regions with similar or identical features in the images are matched together to splice the two images, so that all overlapping regions of the photos are overlapped to obtain a complete spatial image.

[0058] Furthermore, the image is projected into a 2:1 rectangular image using an equidistant cylindrical projection. Figure 3 As shown, the formula for the reverse projection of an image from a rectangular projection to a spherical coordinate system is

[0059]

[0060] Where λ is the longitude of the position to be projected, is the latitude of the position to be projected, are standard parallels (north-south directions of the equator), is the central parallel line of the image, λ0 is the central meridian of the image, x, y are the horizontal and vertical coordinates of the projection position on the image, and R is the radius of the sphere. The image obtained by the laser scanner can be used to obtain a panoramic view of the tunnel through the above steps. The spliced ​​panoramic view in this embodiment is as follows Figure 4 shown.

[0061] S4, uses the denoised point cloud to perform surface reconstruction and obtain a seamless high-quality mesh model.

[0062] Preferably, the three-dimensional points are Delaunay tetrahedronized. The specific operation steps are as follows: triangulate the boundary of the point set, usually by first constructing a super tetrahedron containing all the points. Insert the points into the initial tetrahedron one by one, and keep the Delaunay condition, delete the tetrahedron where the inserted point is located, and rebuild a new tetrahedron related to the inserted point. The operation method is shown as follows Figure 5 shown.

[0063] Preferably, after the point cloud is tetrahedralized, the tetrahedron inside the convex hull is labeled as the internal tetrahedron, and each face on the convex hull is hypothetically incident on an infinite tetrahedron, and the fourth vertex of the infinite tetrahedron is an auxiliary vertex called an infinite vertex. In this way, each face is incident on exactly two units. The surface to be reconstructed is represented by S, which is the union of a set of directed Delaunay triangles. Define an energy E(S) associated with the surface S:

[0064] E(S)=E vis (S)+λ qual E qual (S) (8)

[0065] E vis (S) represents the sum of constraints imposed by all sight lines passing through the sample point, E qual (S) represents an S triangle that is unlikely to appear on a real surface. qual is a positive constant weight. A penalty is imposed on the triangles between the laser emission point and the point cloud.

[0066] The first regularization term in formula (8) is implemented by the minimum st cut method. The minimum st cut is an important concept in graph theory and is widely used in network flow problems, image segmentation, data mining and other fields. In a flow network, there is a graph in is a vertex set ε is a set of directed edges. Graph partitioning is to remove the edges connecting two sets of vertices so that the two sets of points satisfy This division is called a cut and the capacity of a cut is:

[0067]

[0068] w ij Indicates connection v i , v j The weight or capacity of the edge between two vertices. Two special points are added, the source point s and the sink point t. In addition to being connected to the points associated with them, each vertex is also connected to these two points, and the weights of the connections are s i and t i In this cut, and The capacity of the cut can be broken down as follows:

[0069]

[0070] The solution of the minimum cut problem is equivalent to the solution of the maximum flow problem. Based on this, the above formula is further interpreted as the solution of and The sum of the discretized integrals on S and the two integrals over the inner and outer volumes of S:

[0071]

[0072] Where V out Represents the external volume, V in Represents the internal volume. The Delaunay grid is abstracted into a graph In the Delaunay mesh, each unit (small tetrahedron) is regarded as a node, and each triangular mesh is regarded as an edge connecting two nodes. Unlike the traditional two-dimensional graph, the Delaunay tetrahedral mesh needs to abstract the three-dimensional mesh body into a two-dimensional graph. In order to calculate the sum of discretized integrals, an energy calculation function needs to be set for the triangular face of the Delaunay tetrahedral mesh. This function satisfies the visibility penalty imposed on the triangular face between the point cloud and the laser emission point, corresponding to E in formula (8): vis(S) item. The line connecting the point cloud and the laser emission point is the line of sight of the laser scanner to obtain the point cloud. This energy calculation function is the weight calculation method of the triangular facets intersecting with the line of sight:

[0073]

[0074]

[0075] w(f), w(s), w(t) are mentioned in Eq. (16), which correspond to the weight parameters of different items of the triangular mesh and are the main parameters for calculating the minimum st cut. vis Represents the weight of the current line of sight, dist represents the distance from the patch that intersects the line of sight to the emission point, and σ is a constant, which is generally the median of the edge length of the Delaunay grid.

[0076] Furthermore, simple soft visibility constraints are not sufficient to obtain good reconstruction results, because some elongated Delaunay tetrahedrons may not intersect the line of sight and be mistakenly classified as external. Therefore, a method is adopted to control the quality of triangles: generalizing β-skeleton to 3D. The β-skeleton algorithm is used to select triangles with better shapes by calculating the circumscribed circle radius of adjacent triangles and the position relationship between the center and the edge. The algorithm is generalized to 3D to calculate the angle β,ψ where the circumscribed circle of the tetrahedron sharing a face intersects the plane. In order to make the face with a larger circumscribed hollow sphere obtain a smaller penalty, a penalty term is applied to the orientation weight of each face, which is the second term of formula (8):

[0077] q=1-min{cos(β),cos(ψ)} (15)

[0078] The process of calculating the minimum cut is the process of finding the surface with the minimum energy. The commonly used minimum cut solution is based on the minimum cut-maximum flow theory, including the Ford-Fulkerson algorithm, the Edmonds-Karp algorithm, etc. The minimum cut obtained is the required sealed and leak-free point cloud mesh.

[0079] Preferably, the method also takes into account the contribution of the laser emission point position to the surface reconstruction. Based on the visual constraint, the optimal surface reconstruction is performed and focuses on the observer's perspective, that is, the reconstruction of the inner surface of the tunnel, which can obtain a high-quality reconstructed mesh. The mesh reconstruction effect is as follows Figure 6 shown.

[0080] S5, fuse the processed 3D point cloud data and 2D image data. After the data processing in steps S2-S4, the two data can be properly fused. Equidistant cylindrical projection makes it easier to establish the relationship between point cloud and pixels. In this projection mode, the pixels of the panoramic image are evenly distributed in the horizontal and vertical directions with the angle. Figure 7 Describes the projection principle of the horizontal direction of the equidistant cylindrical projection.

[0081] Preferably, assume that the line connecting the point cloud and the laser emission center has an angle of θ with the X-axis and an angle of The horizontal angle is 2π, and the numerical angle is π. First, we need to calculate the θ of a certain point through the three-dimensional coordinates of the point cloud. Angle, according to Figure 7 The pixel distribution law of the equidistant cylindrical projection can obtain the relationship between angle and pixel, and use point cloud coordinates instead of θ. Then the corresponding calculation formula between point cloud and pixel is:

[0082]

[0083] where x ip ,y ip Represents the coordinates of the pixel corresponding to the i-th point in the point cloud, x i ,y i ,z i represents the world coordinates of the point, W and H represent the resolution width and height of the panorama respectively. One is subtracted at the end of the formula because the pixel coordinates in the computer start from 0.

[0084] Furthermore, through UV unfolding, each vertex of the model is associated with a coordinate point on the map. Through UV mapping, the computer can correctly apply the two-dimensional map image to the surface of the three-dimensional model. Figure 8 shown.

[0085] The foregoing description only discloses and describes exemplary embodiments of the present invention. From the above description and the accompanying drawings and claims, those skilled in the art will easily appreciate that various changes, modifications and variations may be made to the present invention without departing from the spirit and scope of the present invention as defined by the appended claims.

[0086] The beneficial effects of the present invention include: a method for simultaneously utilizing two data acquisition methods to establish a virtual digital model of a tunnel with high fidelity and high restoration and dimension information is proposed. The established model lays a foundation for management personnel to remotely explore tunnel conditions more flexibly, efficiently and in real time, and provides the possibility for safer and more timely management of construction and decision-making, which is of great significance in actual engineering.

[0087] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for establishing a static digital twin model of a tunnel, characterized in that: Here are the steps: S1. Scan the tunnel using a 3D laser scanner equipped with a synchronous camera to obtain tunnel images and 3D point cloud data at the same time; S2, preprocessing the three-dimensional point cloud of the tunnel to remove noise and tunnel-irrelevant point clouds; S3, stitching the images taken by the laser scanner and eliminating the stitching seams to obtain a panoramic image, and storing the obtained panoramic image using an equidistant cylindrical projection method; S4, using the denoised point cloud to perform surface reconstruction and obtain a seamless high-quality mesh model; S5. The processed three-dimensional point cloud data and two-dimensional image data are fused to obtain a static digital twin tunnel model with high restoration and precise scale.

2. The method for establishing a static digital twin model of a tunnel according to claim 1, characterized in that: The specific steps of step S1 are: use a three-dimensional laser scanner equipped with a synchronous photography system to scan the tunnel; and simultaneously obtain three-dimensional point cloud data and image data of the tunnel entity.

3. The method for establishing a static digital twin model of a tunnel according to claim 1, characterized in that: The specific steps of step S2 are: projecting the point cloud onto the cross section of the tunnel face, dividing the cross section into grids, normalizing the number of point clouds projected into the grids to the pixel grayscale value of the image, and filtering the point cloud noise by combining data dimensionality reduction with image processing.

4. The method for establishing a static digital twin model of a tunnel according to claim 1, characterized in that: The specific steps of step S3 are: performing a preprocessing operation on the image data obtained in step S1.

5. The method for establishing a static digital twin model of a tunnel according to claim 4, characterized in that: The specific steps of preprocessing are as follows: use the SIFT feature point matching method to obtain the image data, stitch two photos with overlapping areas one by one to obtain a complete tunnel stitching panorama; use Poisson fusion or multi-band fusion method to eliminate the image stitching seams between pictures caused by the aperture and lens distortion of the camera.

6. The method for establishing a static digital twin model of a tunnel according to claim 5, characterized in that: Furthermore, the spherical coordinates of the stitched panoramic image are projected into a rectangular image for storage, and the projection method is an equidistant cylindrical projection.

7. The method for establishing a static digital twin model of a tunnel according to claim 1, characterized in that: Step S4 uses the processed point cloud obtained in step S1 to perform surface reconstruction; first, the point cloud is divided into Delaunay tetrahedrons, and Delaunay forms closed tetrahedrons between all point clouds; the laser emission center of the laser scanner is regarded as the source point, and the line between the source point and the point cloud is regarded as a "line of sight", and visibility constraints are imposed on the triangular facets of the Delaunay tetrahedron, and a penalty is imposed on the facets passed by the "line of sight"; a penalty mechanism is established to minimize the penalty on the point cloud surface to be retained, and the minimum cut is solved by the minimum st cut algorithm to obtain a sealed and leak-free tunnel mesh model.

8. The method for establishing a static digital twin model of a tunnel according to claim 1, characterized in that: The main steps of step S5 are: fusing the three-dimensional point cloud data and two-dimensional image data that have been processed separately; the specific method of fusion is to establish a correspondence between the point cloud and the pixel for the panoramic image and the grid model through equidistant cylindrical projection; each triangular facet of the grid can obtain an image patch that matches it; storing the mapping information and the grid model information in the three models, and using visualization software to view the resulting static tunnel digital twin model.

Images