A motion recovery structure reconstruction and repair method for tunnels

Abstract

The present invention provides a method for reconstructing and repairing a motion-recovery structure of a tunnel, comprising: calculating a camera intrinsic parameter matrix and a lens distortion coefficient through camera images; preprocessing the first image and extracting sparse feature points; adding a new image, preprocessing, extracting sparse feature points, and matching them with the sparse feature points of the first image; calculating the transformation matrix between images through the result of feature point matching; extracting dense feature points from the new image, and triangulating the spatial coordinates of the dense feature points based on the transformation matrix; generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points; and repairing and improving the three-dimensional point cloud model in the form of a generated surface to obtain a complete entity three-dimensional model. The invention can realize three-dimensional reconstruction of an on-site tunnel through multiple tunnel images, thereby solving the technical problems of slow tunnel modeling, inaccuracy, and inability to complete real-time modeling in the prior art of tunnel three-dimensional visualization technology.

Description

Technical Field

[0001] The present invention relates to a building information model visualization technology, and in particular to a motion recovery structure reconstruction and repair method for a tunnel. Background Art

[0002] 3D visualization technology is a visual information processing technology that simulates and studies actual 3D scenes through visual information such as images and videos, and restores real space in virtual space. With the rapid development of computer technology, especially in image processing technology, the application of 3D visualization technology has become more extensive in the fields of robotics, medicine, architecture, etc.

[0003] The tunnel construction process is complex, the construction site is often closed and dim, and the tunnel is crisscrossed. Conventional media such as text, pictures, and videos are difficult to quickly and intuitively reflect and restore various types of information on the complex construction site under actual circumstances. Personnel need to restore the actual construction scene in their minds through traditional media, which is not conducive to communication between various personnel during construction, and it is difficult to make timely plans and overall grasp of various problems and emergencies at the tunnel construction site.

[0004] When three-dimensional visualization technology is applied in tunnel construction, the tunnel model is often established through building information modeling (BIM) software. However, on the one hand, due to the different shapes of tunnels, it takes a certain amount of working time to establish the model through BIM software, the modeling speed is slow, and the upper and lower limits of modeling restoration accuracy vary greatly. On the other hand, as tunnel construction continues, if the tunnel model is to be updated to reflect the construction status, it is necessary to re-use BIM software for modeling, which makes it difficult to achieve real-time modeling of the tunnel construction site in the actual scene. At the same time, when using motion recovery structure technology for three-dimensional reconstruction, facing complex environmental scenes such as tunnels, mountains, rocks, etc., the generated models are often empty, discontinuous, and incomplete, making them difficult to use in practice. Summary of the invention

[0005] In view of the defects in the prior art, an object of the present invention is to provide a method for reconstructing and repairing a motion recovery structure of a tunnel.

[0006] According to one aspect of the present invention, a method for reconstructing and repairing a motion recovery structure of a tunnel is provided, comprising:

[0007] S1, calculates the camera intrinsic parameter matrix and lens distortion coefficient through camera images;

[0008] S2, preprocessing the first image captured by the camera and extracting sparse feature points;

[0009] S3, adding a new image captured by the camera, performing preprocessing, extracting sparse feature points, and matching them with the sparse feature points of the first image;

[0010] S4, calculating the transformation matrix of the new image relative to the first image through the result of feature point matching;

[0011] S5, extracting dense feature points from the new image, and triangulating the dense feature points based on the transformation matrix to obtain spatial coordinates of the dense feature points;

[0012] S6, generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points;

[0013] S7, repairing and improving the three-dimensional point cloud model in the form of generated surfaces to obtain a complete solid three-dimensional model.

[0014] Preferably, the step S1, calculating the camera intrinsic parameter matrix and the lens distortion coefficient through the camera picture, includes:

[0015] Calculate the camera's intrinsic matrix and lens distortion coefficients using the Camera Calibrator application in MARLAB;

[0016] The internal parameter matrix is ​​a 3×3 matrix, including four data: the focal length of the camera itself in the x and y directions and the offset size of the lens origin in the x and y directions in actual situations;

[0017] The lens distortion coefficient includes a radial distortion coefficient and a tangential distortion coefficient.

[0018] Preferably, the step S2, preprocessing the first image captured by the camera and extracting sparse feature points, includes:

[0019] Eliminating lens distortion of the first image according to the lens distortion coefficient;

[0020] Detect the SURF features of the first image and return an image in the form of SURFPoints;

[0021] Detect features in the image in the form of SURFPoints and eliminate pseudo features at the edge of the image;

[0022] From the first image with distortion and pseudo features eliminated, sparse feature points are extracted through edge detection and corner detection.

[0023] Preferably, the S3, adding a new image captured by the camera, performing preprocessing, extracting sparse feature points, and matching the sparse feature points with the sparse feature points of the first image, includes:

[0024] Eliminating lens distortion of the new image according to the lens distortion coefficient;

[0025] Detect SURF features of a new image and return the image in the form of SURFPoints;

[0026] Detect large-scale features in images in the form of SURFPoints and eliminate pseudo features at the edges of images;

[0027] Extract sparse feature points from the new image with distortion and pseudo features eliminated by edge detection and corner detection;

[0028] Match the sparse feature points extracted from the new image with the sparse feature points extracted from the first image.

[0029] Preferably, the step S4, calculating the transformation matrix of the new image relative to the first image based on the result of feature point matching, includes:

[0030] In the camera coordinate system, calculate the camera pose of the current image relative to the previous image;

[0031] In the spatial coordinate system, transform the camera pose of the previous image into the coordinate system of the first image;

[0032] Obtain the transformation matrix of the newly added image relative to the first image.

[0033] Preferably, the transformation matrix between the newly added image and the first image is related to the camera's extrinsic parameter matrix, which includes a 3×3 rotation matrix and a 3×1 translation matrix. The extrinsic parameter matrix is ​​only related to the position of the camera in the world coordinate system.

[0034] Preferably, the step S5, extracting dense feature points from the new image, triangulating the dense feature points based on the transformation matrix, and obtaining the spatial coordinates of the dense feature points, includes:

[0035] From the new image with distortion and pseudo features eliminated, a set of dense feature points are extracted through edge detection and corner detection;

[0036] Based on the transformation matrix, the pixel coordinates of the group of dense feature points in the camera coordinate system are converted into space coordinates in the world coordinate system.

[0037] Preferably, the step S6, generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points, comprises:

[0038] According to the spatial coordinates of the dense feature points, the dense feature points are imported into the created three-dimensional point cloud;

[0039] Visualize and optimize camera position and orientation;

[0040] Exclude noisy 3D points and display dense 3D point clouds;

[0041] Eliminate reprojection errors in the 3D point cloud and display the final 3D point cloud model.

[0042] Preferably, the step S7, repairing and improving the three-dimensional point cloud model in the form of generating a surface to obtain a complete solid three-dimensional model, includes:

[0043] The coordinates of the obtained three-dimensional point cloud model are read to determine the hollow area in the model;

[0044] Filling the hole area by generating a surface;

[0045] A complete solid three-dimensional model is constructed based on the point cloud in the three-dimensional point cloud model and the surface generated in the hole area.

[0046] Preferably, S3 and S4 are a set of loop steps, and each time a new image is added, S3 and S4 are repeated, and after completion, the next new image is added until all images are processed;

[0047] S5 and S6 are a set of loop steps. Each time a new image is added, S5 and S6 are repeated. After completion, the next new image is added. After all images are processed, a 3D reconstructed model is obtained.

[0048] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0049] 1. The method for reconstructing and repairing the structure from motion for a tunnel in an embodiment of the present invention uses the structure from motion technology (i.e., a method for restoring the structure of a scene by changing the position of a camera) to perform three-dimensional reconstruction of the tunnel, and establishes a three-dimensional point cloud model by detecting image feature points, thereby achieving the establishment of a three-dimensional model of the tunnel in reality. This method can meet the needs of tunnels of various shapes, ensure the accuracy of model restoration, and generally solve the technical problem of a large difference in the upper and lower limits of modeling and restoration accuracy in the prior art.

[0050] 2. The tunnel motion recovery structure reconstruction and repair method in the embodiment of the present invention uses multiple images to construct a three-dimensional model, which improves the modeling speed and can update the three-dimensional model in real time, effectively solving the technical problems in the prior art of slow modeling and difficulty in realizing real-time modeling of the tunnel construction site.

[0051] 3. The tunnel motion recovery structure reconstruction and repair method in the embodiment of the present invention obtains the position of the hole in the three-dimensional point cloud model by reading the spatial coordinates of the three-dimensional points. Then, the point cloud model is repaired in the form of a generated surface. In addition, a solid three-dimensional model is jointly established based on the three-dimensional point cloud model and the surface generated by the hole area. This effectively solves the problem of holes, discontinuity, and incompleteness in the three-dimensional point cloud model in the prior art when facing scenes such as tunnels where feature point extraction is difficult and the environment is complex. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0053] Figure 1 The present invention is a flowchart of a method for motion recovery structure reconstruction and repairing of a tunnel in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0055] In one embodiment of the present invention, a method for reconstructing and repairing a motion recovery structure of a tunnel is provided. Figure 1 , the main steps are as follows:

[0056] S1, calculates the camera intrinsic parameter matrix and lens distortion coefficient through camera images;

[0057] S2, preprocessing the first image captured by the camera and extracting sparse feature points;

[0058] S3, adding the new image, preprocessing, extracting sparse feature points, and matching them with the sparse feature points of the first image;

[0059] S4, calculating the transformation matrix of the new image relative to the first image through the result of feature point matching;

[0060] S5, extracting dense feature points from the new image, triangulating the dense feature points based on the transformation matrix, and obtaining the spatial coordinates of the dense feature points;

[0061] S6, generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points;

[0062] S7, repairing and improving the three-dimensional point cloud model by generating surfaces to obtain a complete solid three-dimensional model.

[0063] The present invention uses motion recovery structure technology to establish a three-dimensional point cloud model by detecting image feature points, thereby realizing the establishment of a three-dimensional model of a tunnel in reality. It can meet the needs of tunnels of various shapes and ensure the accuracy of model restoration. It overall solves the technical problem of the large difference between the upper and lower limits of modeling and restoration accuracy in the prior art.

[0064] The images in the above embodiments are all captured by cameras. For the same scene, there must be at least 3 images containing the scene.

[0065] When switching between scenes, the distance between photos should not be too large, and at least 4 / 5 of the same scenes should be retained in the images under different scenes.

[0066] In a preferred embodiment of the present invention, S1 is implemented to calculate the camera intrinsic parameter matrix and lens distortion coefficient through the camera image. Specifically, the camera intrinsic parameter matrix is ​​a 3×3 matrix, which is composed of four data: the focal length of the camera itself in the x and y directions and the offset size of the lens origin in the x and y directions in actual situations. The intrinsic parameter matrix is ​​only related to the internal parameters of the camera; the lens distortion coefficient is composed of the radial distortion coefficient and the tangential distortion coefficient.

[0067] In this embodiment, after obtaining multiple images, the intrinsic parameter matrix and lens distortion coefficient of the camera can be calculated by the Camera Calibrator application in MARLAB. Of course, in other embodiments, other calculation methods can also be used to obtain the intrinsic parameter matrix and lens distortion coefficient of the camera, which is not limited here.

[0068] In a preferred embodiment of the present invention, S2 is implemented to pre-process the first image and extract sparse feature points. Specifically, it includes:

[0069] S21, create an image set through the imageviewset object and store the first picture in the image set;

[0070] S22, load the image in the image set into the workspace through the readimage function, and eliminate the image distortion through the undistoryImage function;

[0071] S23, detects the SURF features of the image through the detectSURFFeatures function to obtain an image in the form of SURFPoints, detects large-scale features such as edges, corners, color changes, etc. in the high-resolution image through the NumOctaves function, and eliminates pseudo features at the edge of the image through ROI.

[0072] S24, extracting feature point descriptions through the extractFeatures function, and storing the feature point information into the image set.

[0073] In this embodiment, sparseness refers to detecting relatively obvious features in the image, such as edges, corners, and color changes, and the number of such features is generally counted in thousands.

[0074] In a preferred embodiment of the present invention, S3 is implemented to add a new image, perform preprocessing, extract sparse feature points, and perform feature point matching with previously extracted feature points, specifically including:

[0075] S31, adding a new image: after the previous image is processed, adding a new image to the image set;

[0076] S32, processing the image: using the same method as S22 to S23, processing the image to eliminate image distortion and edges;

[0077] S33, extracting a set of feature points of the current image as sparse feature points by edge detection and corner detection methods (extractFeatures function);

[0078] S34, track the set of feature points extracted in S33 through vision.PointTracker, and match the feature points with the feature points extracted from the previous image. The vision.PointTracker function creates a point tracker, records the features of the point, and matches the features of points in different images during matching, and points with the same features are considered to be the same point.

[0079] In a preferred embodiment of the present invention, S4 is implemented to calculate the transformation matrix of the newly added image relative to the first image through the result coordinates of the feature point matching in the camera coordinate system. Specifically,

[0080] The camera pose of the current image relative to the previous image is calculated through the helperEstimateRelativePose function, and converted to the coordinate system of the first image, so as to obtain the transformation matrix of the image in the current workspace relative to the first image.

[0081] The transformation matrix used to transform the coordinates between the newly added image and the first image is related to the camera's extrinsic matrix, which consists of a 3×3 rotation matrix and a 3×1 translation matrix. The extrinsic matrix is ​​only related to the position of the camera in the world coordinate system.

[0082] Furthermore, steps S3 and S4 are a set of loop steps, and all contents are wrapped in a for loop. After each new image is added to the workspace, the steps of image processing, feature point extraction, feature point matching, and transformation matrix solution are executed in sequence to complete a for loop. After completion, the next image is added to the workspace, and the loop ends after all images are processed.

[0083] In a preferred embodiment of the present invention, S5 is implemented to extract dense feature points from the image, and triangulate the feature points to obtain the spatial coordinates of the feature points. The specific process is as follows:

[0084] S51, by setting a smaller MinQuality value, obtaining more dense feature points, the method steps are the same as S3;

[0085] S52, using the transformation matrix calculated in step S4, when the pixel coordinates of each feature point in the current camera coordinate system are known, the dense feature points extracted from each image are converted into spatial coordinates in the world coordinate system.

[0086] The denseness in the above embodiment refers to detecting any features with changes in the image, the number of which is generally counted in hundreds of thousands.

[0087] In a preferred embodiment of the present invention, S6 is implemented to generate a three-dimensional point cloud model according to the coordinates of the feature points. Specifically,

[0088] S61, creating a point cloud and importing three-dimensional points: creating a point cloud through the pointCloud function, importing the spatial coordinates of the feature points calculated in step S5 in the world coordinate system into the point cloud, thereby obtaining a three-dimensional reconstruction model of the actual scene;

[0089] S62, Visualize and optimize camera position and orientation: Use the plotCamera function to visualize the camera position and posture, and refine the camera by adjusting related parameters such as "Size", "Location", "Orientation", and "Opacity";

[0090] S63, eliminate noisy 3D points in the 3D point cloud and display dense 3D point cloud: eliminate reprojection errors in the 3D point cloud by reading the reprojectionErrors number, and visualize the point cloud through the pcshow function to obtain the 3D point cloud model reconstruction of the actual scene.

[0091] Furthermore, steps S5 and S6 are a set of loop steps, and all contents are wrapped in a for loop. After each new image is added to the workspace, the steps of feature point extraction, coordinate system conversion, and importing three-dimensional point cloud are executed in sequence to complete a for loop. After completion, the next image is added to the workspace. After all images are processed, the three-dimensional reconstruction is completed.

[0092] This embodiment uses multiple images to construct a three-dimensional model, which improves the modeling speed and can update the three-dimensional model in real time, effectively solving the technical problems of slow modeling speed and difficulty in realizing real-time modeling of the tunnel construction site in the prior art.

[0093] In a preferred embodiment of the present invention, S7 is implemented to repair and improve the three-dimensional point cloud model by generating a surface. Specifically, in the three-dimensional point cloud model of the tunnel obtained in step S6, there are still model holes caused by the difficulty in extracting feature points. By reading the spatial coordinates of each point in the three-dimensional point cloud model, the spatial area of ​​the three-dimensional point cloud model and the hole area where the point cloud model is not generated are determined, and the hole area is repaired in the form of Plane (surface) generated by the entities.add function, and the points in a certain area in the three-dimensional point cloud model are connected, and finally a complete entity three-dimensional model is jointly constructed.

[0094] The above-mentioned embodiment can realize three-dimensional reconstruction of the on-site tunnel through multiple tunnel images, thereby solving the technical problems of slow and inaccurate tunnel modeling and inability to complete real-time modeling in the prior art three-dimensional tunnel visualization technology.

[0095] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A method for reconstructing and repairing a motion recovery structure of a tunnel, characterized in that: include: S1, calculates the camera intrinsic parameter matrix and lens distortion coefficient through camera images; S2, preprocessing the first image captured by the camera and extracting sparse feature points; S3, adding a new image captured by the camera, performing preprocessing, extracting sparse feature points, and matching them with the sparse feature points of the first image; S4, calculating the transformation matrix of the new image relative to the first image through the result of feature point matching; S5, extracting dense feature points from the new image, and triangulating the dense feature points based on the transformation matrix to obtain spatial coordinates of the dense feature points; S6, generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points; S7, repairing and improving the three-dimensional point cloud model in the form of generated surfaces to obtain a complete solid three-dimensional model; S2, preprocessing the first image captured by the camera and extracting sparse feature points, includes: Eliminating lens distortion of the first image according to the lens distortion coefficient; Detect the SURF features of the first image and return an image in the form of SURFPoints; Detect features in the image in the form of SURFPoints and eliminate pseudo features at the edge of the image; Extract sparse feature points from the first image with distortion and pseudo features eliminated by edge detection and corner detection; S3, adding a new image captured by the camera, performing preprocessing, extracting sparse feature points, and matching them with the sparse feature points of the first image, includes: Eliminating lens distortion of the new image according to the lens distortion coefficient; Detect SURF features of a new image and return the image in the form of SURFPoints; Detect large-scale features in images in the form of SURFPoints and eliminate pseudo features at the edges of images; Extract sparse feature points from the new image with distortion and pseudo features eliminated by edge detection and corner detection; Match the sparse feature points extracted from the new image with the sparse feature points extracted from the first image; The S4, calculating the transformation matrix of the new image relative to the first image through the result of feature point matching, includes: In the camera coordinate system, calculate the camera pose of the current image relative to the previous image; In the spatial coordinate system, transform the camera pose of the previous image into the coordinate system of the first image; Obtain the transformation matrix of the newly added image relative to the first image.

2. A method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: The S1, calculating the camera intrinsic parameter matrix and lens distortion coefficient through the camera image, includes: Calculate the camera's intrinsic matrix and lens distortion coefficients using the Camera Calibrator application in MATLAB; The internal parameter matrix is ​​a 3×3 matrix, including four data: the focal length of the camera itself in the x and y directions and the offset size of the lens origin in the x and y directions in actual situations; The lens distortion coefficient includes a radial distortion coefficient and a tangential distortion coefficient.

3. A method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: The transformation matrix between the newly added image and the first image is related to the camera's extrinsic parameter matrix, which includes a 3×3 rotation matrix and a 3×1 translation matrix. The extrinsic parameter matrix is ​​only related to the position of the camera in the world coordinate system.

4. A method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: The step S5, extracting dense feature points from the new image, triangulating the dense feature points based on the transformation matrix, and obtaining spatial coordinates of the dense feature points, includes: From the new image with distortion and pseudo features eliminated, a set of dense feature points are extracted through edge detection and corner detection; Based on the transformation matrix, the pixel coordinates of the group of dense feature points in the camera coordinate system are converted into space coordinates in the world coordinate system.

5. The method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: The step S6, generating a three-dimensional point cloud model according to the spatial coordinates of the dense feature points, comprises: According to the spatial coordinates of the dense feature points, the dense feature points are imported into the created three-dimensional point cloud; Visualize and optimize camera position and orientation; Exclude noisy 3D points and display dense 3D point clouds; Eliminate reprojection errors in the 3D point cloud and display the final 3D point cloud model.

6. A method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: The step S7, repairing and improving the three-dimensional point cloud model in the form of generating a surface to obtain a complete solid three-dimensional model, includes: The coordinates of the obtained three-dimensional point cloud model are read to determine the hollow area in the model; Filling the hole area by generating a surface; A complete solid three-dimensional model is constructed based on the point cloud in the three-dimensional point cloud model and the surface generated in the hole area.

7. A method for motion recovery structure reconstruction and repair of a tunnel according to claim 1, characterized in that: S3 and S4 are a set of loop steps. Each time a new image is added, S3 and S4 are repeated. After completion, the next new image is added until all images are processed; S5 and S6 are a set of loop steps. Each time a new image is added, S5 and S6 are repeated. After completion, the next new image is added. After all images are processed, a 3D reconstructed model is obtained.

Images