Underground cavity three-dimensional reconstruction method, system, equipment and medium

Abstract

The invention discloses an underground cavity three-dimensional reconstruction method, system and device and a medium, and the method comprises the steps: carrying out the multi-view shooting of an underground cavity through an image collection device, obtaining a multi-view depth image, carrying out the point cloud conversion of the multi-view depth image, and obtaining three-dimensional point cloud data, the method comprises the following steps: acquiring three-dimensional point cloud data of an underground cavity, preprocessing the three-dimensional point cloud data to obtain preprocessed point cloud data, performing feature matching processing on the preprocessed point cloud data to obtain coarse registration point cloud, performing iterative optimization processing on the coarse registration point cloud to obtain fine registration point cloud, and performing error evaluation and fusion processing on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity. According to the embodiment of the invention, a visual underground cavity three-dimensional model can be generated, and the form and the size of the cavity can be visually displayed. The method can be widely applied to the technical field of underground cavity detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underground cavity detection, and in particular to an underground cavity three-dimensional reconstruction method, system, device and medium. BACKGROUND

[0002] Underground cavity detection is a key technology for preventing road collapse disasters and ensuring the safe operation of cities, and has important value for optimizing underground space development and geological disaster prevention and control. In related technologies, underground cavity detection is performed by means of ground penetrating radar, seismic wave exploration and the like. However, these methods can only detect whether an underground cavity exists, and cannot estimate the shape, volume size and the like of the underground cavity, so it is difficult to estimate the grouting amount required for repairing the cavity, which is not conducive to the repair work of the underground cavity in engineering.

[0003] To sum up, the technical problems existing in the related art need to be improved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide an underground cavity three-dimensional reconstruction method, system, device and medium, which can improve the efficiency of underground cavity repair.

[0005] To achieve the above-mentioned purpose, one aspect of the embodiments of the present application provides an underground cavity three-dimensional reconstruction method, which comprises:

[0006] Performing multi-view shooting processing on the underground cavity by means of an image acquisition device to obtain a multi-view depth map;

[0007] Performing point cloud conversion processing on the multi-view depth map to obtain three-dimensional point cloud data;

[0008] Performing preprocessing on the three-dimensional point cloud data to obtain preprocessed point cloud data;

[0009] Performing feature matching processing on the preprocessed point cloud data to obtain coarse registration point cloud;

[0010] Performing iterative optimization processing on the coarse registration point cloud to obtain fine registration point cloud;

[0011] Performing error evaluation and fusion processing on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity.

[0012] In some embodiments, the point cloud conversion processing on the multi-view depth map to obtain three-dimensional point cloud data comprises:

[0013] Performing gray scale format conversion processing on the multi-view depth map to obtain a gray scale image;

[0014] Performing floating point format conversion processing on the gray scale image to obtain floating point data;

[0015] According to the camera parameters of the image acquisition device, the floating point data is subjected to point cloud data generation processing to obtain the three-dimensional point cloud data.

[0016] In some embodiments, the three-dimensional point cloud data is preprocessed to obtain preprocessed point cloud data, including:

[0017] According to the denoising algorithm, the three-dimensional point cloud data is subjected to filtering statistics and outlier removal processing to obtain denoised data.

[0018] According to the downsampling algorithm, the denoised data is subjected to point cloud voxel adjustment processing to obtain the preprocessed point cloud data.

[0019] In some embodiments, the preprocessed point cloud data is subjected to feature matching processing to obtain a coarse registration point cloud, including:

[0020] According to the fast point feature histogram algorithm, the preprocessed point cloud data is subjected to feature extraction processing to obtain point cloud features.

[0021] According to the mutual filtering mechanism, the point cloud features are subjected to global feature matching processing to obtain the coarse registration point cloud.

[0022] In some embodiments, the coarse registration point cloud is subjected to iterative optimization processing to obtain a fine registration point cloud, including:

[0023] The coarse registration point cloud is subjected to matrix calculation processing to obtain a transformation matrix.

[0024] According to the transformation matrix, the coarse registration point cloud is subjected to nearest neighbor search processing to obtain first data.

[0025] According to a preset distance and a normal direction, the first data is subjected to invalid matching filtering processing to obtain second data.

[0026] According to the least squares method, the second data is subjected to linear solution processing, and according to the linear solution result, the transformation matrix is updated, and the nearest neighbor search processing of the coarse registration point cloud according to the transformation matrix is returned until the iteration termination condition is met, to obtain a fine registration point cloud.

[0027] In some embodiments, the second data is subjected to linear solution processing according to the least squares method, including:

[0028] According to the data acquisition sequence, the second data is subjected to classification processing to obtain a source point cloud and a target point cloud.

[0029] According to the transformation matrix, a rotation matrix and a translation vector are obtained.

[0030] The source point cloud is multiplied by the rotation matrix, the multiplication result is added with the translation vector, and the addition result is subtracted from the target point cloud to calculate a distance error result of the source point cloud and the target point cloud.

[0031] The distance error result is minimized to obtain the linear solution result.

[0032] In some embodiments, the error evaluation and fusion processing of the fine registration point cloud to obtain the three-dimensional point cloud model of the underground cavity comprises:

[0033] The fine registration point cloud is down-sampled to obtain down-sampled data.

[0034] The down-sampled data is subjected to outlier removal processing of sparse regions by a radius filtering and secondary statistical filtering algorithm to obtain filtered data.

[0035] The filtered data is subjected to proportion calculation processing of effective matching point pairs to total points to obtain an error evaluation result.

[0036] The filtered data and the fine registration point cloud are subjected to point cloud fusion processing according to the error evaluation result to obtain target point cloud data.

[0037] The target point cloud data is subjected to three-dimensional reconstruction processing to obtain the three-dimensional point cloud model of the underground cavity.

[0038] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a three-dimensional reconstruction system of an underground cavity, which comprises:

[0039] An acquisition module is configured to perform multi-view shooting processing on an underground cavity by an image acquisition device to obtain multi-view depth maps.

[0040] A point cloud conversion module is configured to perform point cloud conversion processing on the multi-view depth maps to obtain three-dimensional point cloud data.

[0041] A preprocessing module is configured to perform preprocessing on the three-dimensional point cloud data to obtain preprocessed point cloud data.

[0042] A feature matching module is configured to perform feature matching processing on the preprocessed point cloud data to obtain coarse registration point cloud.

[0043] An iterative optimization module is configured to perform iterative optimization processing on the coarse registration point cloud to obtain fine registration point cloud.

[0044] A model construction module is configured to perform error evaluation and fusion processing on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity.

[0045] To achieve the above object, another aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0046] To achieve the above object, another aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the method described above when executed by a processor.

[0047] To achieve the above object, another aspect of the embodiments of the present application provides a computer program product, which comprises a computer program, and the computer program implements the method described above when executed by a processor

[0048] The embodiments of the present application at least have the following beneficial effects: the present application provides a method, system, device and medium for three-dimensional reconstruction of underground cavities, which obtains multi-view depth maps by multi-view shooting of underground cavities through an image acquisition device; performs point cloud conversion processing on the multi-view depth maps to obtain three-dimensional point cloud data; performs preprocessing on the three-dimensional point cloud data to obtain preprocessed point cloud data; performs feature matching processing on the preprocessed point cloud data to obtain coarse registration point cloud; performs iterative optimization processing on the coarse registration point cloud to obtain fine registration point cloud; and performs error evaluation and fusion processing on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity. The embodiments of the present application generate a visual three-dimensional model of the underground cavity by processing the collected images, which can intuitively show the shape and volume of the cavity, thereby providing comprehensive and intuitive analysis of the underground cavity for the repair work of the underground cavity. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of a method for three-dimensional reconstruction of underground cavities provided by the embodiments of the present application;

[0050] Figure 2 is a structure diagram of a binocular depth camera support provided by the embodiments of the present application;

[0051] Figure 3 is a three-dimensional point cloud reconstruction schematic diagram for a simulated underground cavity assembly provided by the embodiments of the present application;

[0052] Figure 4 is a structure schematic diagram of a three-dimensional reconstruction system for underground cavities provided by the embodiments of the present application;

[0053] Figure 5 is a hardware structure schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0055] It can be understood that the terms "first", "second" and the like used in the present application can be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0056] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0058] Before the embodiments of the present application are described in detail, first, some nouns and terms involved in the embodiments of the present application are described, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0059] 1) Underground cavity refers to a three-dimensional space structure formed naturally or artificially below the ground surface, which is filled with air or filler. These cavities have important influence in geotechnical engineering, geological exploration and urban safety.

[0060] 2) Point cloud data is a collection of a large number of discrete points in three-dimensional space, each point containing accurate position coordinates (X, Y, Z), and can be attached to multi-dimensional attributes such as color (RGB), reflectivity, normal vector, etc. These data are obtained through technologies such as laser radar (LiDAR), depth camera or multi-view image reconstruction, and can digitally record the surface morphology of an object with millimeter-level precision, and are widely used in three-dimensional modeling, autonomous driving environment perception, industrial detection, geological exploration and other fields.

[0061] In the related art, only underground cavities can be detected by detection technologies such as ground penetrating radar, seismic wave exploration and the like, but the volume of the underground cavity cannot be estimated, so it is difficult to estimate the grouting amount required for repairing the cavity, which is not conducive to the repair work of the underground cavity in engineering.

[0062] Therefore, in the embodiments of the present application, a three-dimensional reconstruction method, system, device and medium for underground cavities are provided. The scheme performs multi-view shooting processing on the underground cavity through an image acquisition device to obtain a multi-view depth map; performs point cloud conversion processing on the multi-view depth map to obtain three-dimensional point cloud data; performs preprocessing on the three-dimensional point cloud data to obtain preprocessed point cloud data; performs feature matching processing on the preprocessed point cloud data to obtain coarse registration point cloud; performs iterative optimization processing on the coarse registration point cloud to obtain fine registration point cloud; and performs error evaluation and fusion processing on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity. The embodiments of the present application generate a visual three-dimensional model of the underground cavity by processing the collected images, which can intuitively display the morphology and volume of the cavity, thereby providing comprehensive and intuitive analysis of the underground cavity for the repair work of the underground cavity.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0064] Figure 1 is an optional flowchart of a three-dimensional reconstruction method for an underground cavity provided by the embodiments of the present application, Figure 1The method in the method can include but is not limited to including steps S101 to S106.

[0065] Step S101, the underground cavity is photographed by the image acquisition device, and a multi-view depth map is obtained;

[0066] Step S102, the multi-view depth map is converted into point cloud data;

[0067] Step S103, the three-dimensional point cloud data is preprocessed to obtain preprocessed point cloud data;

[0068] Step S104, the preprocessed point cloud data is processed by feature matching to obtain coarse registration point cloud;

[0069] Step S105, the coarse registration point cloud is processed by iterative optimization to obtain fine registration point cloud;

[0070] Step S106, the fine registration point cloud is processed by error evaluation and fusion to obtain the three-dimensional point cloud model of the underground cavity.

[0071] The steps S101 to S106 shown in the embodiment of the application, the underground cavity is photographed by the image acquisition device, and a multi-view depth map is obtained. Then the multi-view depth map is converted into point cloud data, and three-dimensional point cloud data is obtained. By removing noise from the three-dimensional point cloud data, preprocessed point cloud data can be obtained. By processing the preprocessed point cloud data, coarse registration point cloud data is obtained. The coarse registration point cloud is iteratively optimized to obtain fine registration point cloud. Finally, the fine registration point cloud is processed by error evaluation and fusion, and a three-dimensional point cloud model of the underground cavity is constructed by combining a three-dimensional reconstruction algorithm.

[0072] The embodiment of the application can intuitively and efficiently establish a three-dimensional model of the underground cavity by simulating the underground cavity and using point cloud technology. It can solve the problem that it is difficult to obtain the shape and size of the underground cavity in dark conditions. The point cloud established can clearly show the internal structural features and collapse trend of the underground cavity, and can be used for three-dimensional reconstruction based on positioning the underground cavity in subsequent practical applications, and can provide visual decision support for road emergency rescue, effectively improving the safety maintenance of urban infrastructure.

[0073] In step S101 of some embodiments, the image acquisition device includes a camera and a camera mounting telescopic rod device. Please refer to Figure 2 , the camera mounting telescopic rod device includes a handle component 201, a sliding locking mechanism 202, a segmented fixed hole 203, and a distal end rotating component 204. The four core components can be coordinated to adjust the camera installation.

[0074] Wherein, the handle component 201 is built-in transmission mechanism, the operator can drive the internal transmission system by pressing the handle, realize the angle adjustment of the remote end rotating component 204, so as to complete the rotation of the camera installed in the remote end rotating component 204 in the vertical plane; The sliding locking mechanism 202 adopts spring pre-tightening structure, in normal state, through the elastic force and any segmented fixed hole 203 on the rod body form locking cooperation, when the operator presses the sliding locking mechanism 202, the locking pin is separated from the fixed hole, the rod body can be freely extended, when the sliding locking mechanism 202 moves to the target fixed hole 203 position and releases the pressing force, the spring drives the locking pin to automatically insert the fixed hole, realize the accurate segmented locking; The design realizes the length adjustment and angle control under the operation of single hand through the organic combination of mechanical linkage and elastic locking, can improve the operation convenience and positioning accuracy of the camera mounting equipment.

[0075] The image acquisition device of the embodiment of the application can further include an underground cavity simulation assembly, which is an opaque PVC plastic bucket placed in a dark environment. The underground cavity simulation assembly includes the PVC plastic bucket and an opaque wood board placed above the bucket mouth. The opaque wood board and the PVC plastic bucket are tightly combined to form a whole. A hole is provided in the middle of the opaque wood board for extending into a camera support on which a binocular depth camera is installed to obtain a depth map of the inside of the PVC plastic bucket. The simulation assembly mainly provides a controllable test environment for a three-dimensional point cloud reconstruction algorithm, so that researchers can verify and adjust the three-dimensional reconstruction algorithm based on simulation data. The simulation assembly simulates the geometric shape and illumination conditions (total darkness) of a real underground cavity, and verifies the data acquisition capability of the binocular depth camera for a depth map in a dark and complex environment. Based on the test data obtained by the simulation assembly, the point cloud registration algorithm can be calibrated in precision, parameters can be adjusted, and a feature point matching strategy can be optimized. After the point cloud registration algorithm is optimized based on the simulation data, the error rate of feature matching is greatly reduced, the internal structure of the cavity is successfully reconstructed, and the volume calculated by the optimized point cloud registration algorithm coincides with the actual volume by more than 97%, which helps to provide the amount of grouting required for repairing the underground cavity and meets the needs of repairing the underground cavity.

[0076] The embodiment of the application can calibrate and adjust the algorithm based on the test data obtained by the simulation component. First, the known geometric parameters (such as the actual size of the PVC plastic bucket) in the simulation data are used as a reference to test the influence of different voxel sizes and feature matching parameters (such as the maximum corresponding point distance and the number of iterative sampling points) on the registration accuracy in the registration module of the code. When it is found that a large number of picture registration scores are lower than a certain threshold (for example, less than the default 0.5) and the volume calculation deviation is too large under the default parameters, the accuracy of the registration is optimized by gradually adjusting the voxel size, or appropriately increasing the feature search radius and the maximum number of neighbors, and by dynamically adjusting the statistical outliers.

[0077] Specifically, the image acquisition device of the embodiment of the application utilizes a small hole to obtain the underground cavity condition under actual conditions without damaging the road surface in a large area. In subsequent tests, the underground cavity image acquisition device can be modified to achieve a free telescopic length, and the appropriate length can be selected according to the burial depth of the underground cavity, which plays an important role in underground cavity detection.

[0078] In step S102 of some embodiments, the multi-view depth map is subjected to point cloud conversion processing to obtain three-dimensional point cloud data, including:

[0079] The multi-view depth map is subjected to gray scale format conversion processing to obtain a gray scale image.

[0080] The gray scale image is subjected to floating point format conversion processing to obtain floating point data.

[0081] The floating point data is subjected to point cloud data generation processing according to the camera parameters of the image acquisition device to obtain the three-dimensional point cloud data.

[0082] In the embodiment of the application, the data format conversion is completed by converting the depth map to a gray scale format and to a floating point type, and the three-dimensional point cloud data is obtained by subjecting the floating point data to point cloud data generation processing according to the camera parameters of the image acquisition device. Each pixel of the depth map stores the depth value (i.e., the distance from the camera) of the corresponding point in the scene. In combination with the camera intrinsic parameters, the pixel coordinates can be converted into 3D points in the camera coordinate system.

[0083] The expression of the camera intrinsic matrix is as follows:

[0084]

[0085] where f x , f y is the focal length, and (c x , c y ) is the optical center coordinate.

[0086] Considering the coordinate transformation, the pixel coordinates (u, v) and the depth value z, are converted into the corresponding 3D point (x, y, z), and the coordinates in the camera coordinate system are:

[0087]

[0088] z = z;

[0089] The embodiment of the present application converts the multi-view depth map into three-dimensional point cloud data by point cloud conversion, which can provide data basis for subsequent point cloud registration and three-dimensional reconstruction.

[0090] In some embodiments, the three-dimensional point cloud data is preprocessed to obtain preprocessed point cloud data, including:

[0091] According to the denoising algorithm, the three-dimensional point cloud data is filtered and statistically processed and outlier points are removed to obtain denoised data;

[0092] According to the downsampling algorithm, the denoised data is adjusted in point cloud voxel to obtain the preprocessed point cloud data.

[0093] In the embodiment of the present application, each point cloud of the three-dimensional point cloud data is removed by statistical filtering and outlier removal to complete the denoising processing to obtain denoised data, and each point cloud of the three-dimensional point cloud data is adjusted in voxel size to complete the downsampling processing to obtain preprocessed data. The denoising algorithm removes outliers by filtering and statistical processing, specifically by calculating the distance of each point in the point cloud to the nearest k neighbors (for example, k = 50). Then, the distances of all points are statistically processed to obtain the average value and standard deviation of the distance distribution of the entire point cloud. According to the statistical information, a threshold is set, which is usually set as an adjustable parameter, i.e. the points whose distance is more than several times the standard deviation are regarded as outliers. The average distance of each point is compared with the threshold, and the points exceeding the threshold are marked as outliers and removed. Since the amount of denoised point cloud data may still be large, in order to reduce the amount of calculation in subsequent processing, downsampling is also needed. Voxel grid downsampling is a commonly used method, and its core idea is to divide the three-dimensional space into multiple equal-sized cubes (voxels), and then replace all points in each voxel with a single point (such as the center of gravity of all points in the voxel) in the voxel. The specific steps are as follows: according to the set voxel size (for example, 0.01 meters), the point cloud space is divided into multiple voxel grids. For each non-empty voxel, all points inside it are aggregated and a representative point (usually the center of gravity or center of these points) is generated. The representative points of all voxels form the downsampled point cloud. Voxel grid downsampling can maintain the shape features of the point cloud while reducing the amount of point cloud data, and the distribution of the downsampled point cloud is uniform.

[0094] In some embodiments, the feature matching processing on the pre-processed point cloud data to obtain a coarse registration point cloud comprises:

[0095] performing feature extraction processing on the pre-processed point cloud data according to a fast point feature histogram algorithm to obtain point cloud features;

[0096] performing global feature matching processing on the point cloud features according to a mutual filtering mechanism to obtain the coarse registration point cloud.

[0097] In the embodiments of the present application, the normal vector of each point in the point cloud is calculated first, and the minimum eigenvalue of the covariance matrix of the neighborhood points is usually used as the normal direction. Then, for each point, three angle features between itself and the neighborhood points are calculated in its neighborhood (radius search or K-neighborhood), which describe the local geometric relationship of the point. Then, these features are binned and counted to form a simplified histogram. For each point, in addition to its own features, the features of its neighborhood points are also aggregated (the weight is inversely proportional to the distance), and finally a three-dimensional feature vector is formed. The feature vector of each point is normalized to obtain the point cloud features. Then, the point cloud features are processed by the mutual filtering mechanism to obtain the coarse registration point cloud. In the matching process, the mutual filtering mechanism is used, the maximum corresponding distance is set to 9 times the size of the voxel, and the edge length checker and distance checker are used to verify the matching consistency. Finally, it is determined whether to apply the transformation matrix according to the registration score, which means that the registration coincidence is higher than a certain threshold, which can be adjusted according to the actual situation, and then the transformation matrix is applied. If the registration coincidence is lower than the threshold, no transformation is made, and finally the coarse registration point cloud is obtained.

[0098] In some embodiments, the iterative optimization processing on the coarse registration point cloud to obtain a fine registration point cloud comprises:

[0099] performing matrix calculation processing on the coarse registration point cloud to obtain a transformation matrix;

[0100] performing nearest neighbor search processing on the coarse registration point cloud according to the transformation matrix to obtain first data;

[0101] performing invalid matching filtering processing on the first data according to a preset distance and a normal direction to obtain second data;

[0102] performing linear solving processing on the second data according to the least square method, and updating the transformation matrix according to the linear solving result, and returning to the nearest neighbor search processing on the coarse registration point cloud according to the transformation matrix until the iteration termination condition is met, to obtain the fine registration point cloud.

[0103] In the embodiments of the present application, based on the obtained coarse registration point cloud, the nearest neighbor point of each point of the source point cloud is searched in the target point cloud based on the initial transformation matrix, and the process of the nearest neighbor search is accelerated by using a KD tree. The KD tree can be automatically constructed by a software tool, and the point with the closest Euclidean distance of each point of the source point cloud is searched in the KD tree of the target point cloud, so as to establish an initial correspondence relationship and obtain first data. The first data is data for establishing an initial correspondence relationship for the coarse registration point cloud. Then, the first data is filtered by a preset distance and a normal direction to obtain second data. Specifically, when filtering the invalid matching point pairs, distance filtering is performed, that is, the matching with the Euclidean distance between the point pairs exceeding the maximum corresponding distance threshold is removed, which can be adjusted according to the actual engineering. In the experiment, it is set to 0.4 times the size of the voxel. Normal filtering is performed, that is, the matching with the normal angle greater than 30 degrees. The first data is filtered by the preset distance and the normal direction. Finally, the second data is linearly solved by the least square method, and the transformation matrix is updated according to the linear solution result, and the nearest neighbor search processing of the coarse registration point cloud based on the transformation matrix is returned until the iteration termination condition is met, and the fine registration point cloud is obtained. The iteration termination condition can include that the change rate of the point cloud data is less than a certain value, the loss change amount is less than a certain value, the maximum iteration number is reached, and the like.

[0104] In some embodiments, the linear solving processing of the second data according to the least square method comprises:

[0105] The second data is classified according to the data acquisition sequence to obtain the source point cloud and the target point cloud;

[0106] A rotation matrix and a translation vector are obtained according to the transformation matrix;

[0107] The source point cloud is multiplied by the rotation matrix, the multiplication result is added to the translation vector, and the addition result is subtracted from the target point cloud, so as to calculate the distance error result of the source point cloud and the target point cloud;

[0108] The linear solution result is obtained by minimizing the distance error result.

[0109] In the embodiments of the present application, the second data is classified and processed according to a data acquisition sequence, wherein the source point cloud P and the target point cloud Q are respectively acquired by a binocular depth camera on a self-made camera support from different angles, specifically, in actual engineering, a small hole is opened above the road cavity by using a drilling machine, and when the camera support with the binocular depth camera is used to detect the cavity below the road from the small hole, two sets of depth images of different angles of the target cavity are acquired by adjusting the camera angle of the support, and P and Q are formed after processing by a depth image to point cloud module; wherein the point cloud formed by processing the first acquired depth image is taken as the target point cloud Q, and the point cloud formed by processing the second acquired depth image is taken as the source point cloud P, and this time sequence division makes P align to Q during registration, which conforms to the physical acquisition sequence of camera shooting. The two sets of point clouds need to have more than about 30% overlapping area in space to ensure that the point cloud registration algorithm can effectively calculate the rigid body transformation. For the given two sets of point clouds, one set is taken as the source point cloud and the other set is taken as the target point cloud Then a rigid body transformation (rotation matrix R and translation vector t) is solved by iteration to minimize the distance error between P and Q after transformation:

[0110]

[0111] Wherein q j is the nearest neighbor point of p i in Q, and p i and q j are three-dimensional coordinate vectors.

[0112] The embodiments of the present application can calculate the precisely registered point cloud by minimizing the distance error result, solving the linear system of point-to-plane distance error by least squares method, and iteratively optimizing the transformation matrix.

[0113] In some embodiments, the error evaluation and fusion processing of the precisely registered point cloud to obtain the three-dimensional point cloud model of the underground cavity comprises:

[0114] The precisely registered point cloud is down-sampled to obtain down-sampled data;

[0115] The down-sampled data is processed by radius filtering and secondary statistical filtering algorithm to remove outliers in sparse areas to obtain filtered data;

[0116] The filtered data is processed to calculate the proportion of effective matching points in the total number of points to obtain an error evaluation result;

[0117] The filtered data and the precisely registered point cloud are fused according to the error evaluation result to obtain target point cloud data;

[0118] Perform three-dimensional reconstruction processing on the target point cloud data to obtain a three-dimensional point cloud model of the underground cavity.

[0119] In the embodiment of the present application, the fine registration point cloud is down-sampled by using smaller voxels to retain details, and down-sampled data is obtained to complete accurate down-sampling. Then, outliers in sparse areas are removed by radius filtering and secondary statistical filtering to obtain filtered data. Then, the coverage of registration is measured by calculating the proportion of effective matching point pairs in the total number of points to realize error evaluation. According to the error evaluation result, the filtered data and the fine registration point cloud are fused, and a three-dimensional model of the underground cavity can be generated based on a three-dimensional reconstruction algorithm. The embodiment of the present application can further obtain the volume of the generated three-dimensional model of the cavity by volume calculation.

[0120] Next, the scheme of the embodiment of the present application will be described and explained in detail in combination with specific application examples:

[0121] The embodiment of the present application is applied to an underground cavity detection scene. A camera support equipped with a binocular depth camera is used to extend into the inside of the underground cavity simulation assembly to obtain depth maps under multiple perspectives. The obtained depth maps are generated into point clouds, and point cloud three-dimensional reconstruction is realized through point cloud registration. Please refer to Figure 3 , Figure 3 is a three-dimensional point cloud reconstruction schematic diagram for an underground cavity simulation assembly provided by the embodiment of the present application. The present application can intuitively and efficiently establish a three-dimensional model of an underground cavity through underground cavity simulation and point cloud technology. The preprocessing module of the embodiment of the present application can perform anti-dark light processing on the depth image, convert the depth map into a 32-bit integer gray mode, and enhance the resolution of depth data in a dark environment. In addition, under global registration, dark environment texture compensation is increased, mutual filtering is enabled, and the feature matching distance is expanded to enhance the robustness of feature matching and reduce the influence of low texture in a dark environment. In terms of noise suppression processing, a loose statistical outlier threshold is used to retain more effective matching feature points. In normal estimation, the neighborhood search radius is expanded to obtain more feature points. In volume calculation, the Poisson reconstruction algorithm is used, and the Poisson reconstruction level is deepened and the smoothing iteration number is increased to compensate for dark light noise. In the global registration of point clouds, through calibration based on the simulation assembly, the mutual filtering mode is enabled to ensure that the matching feature point pairs must be the nearest neighbors of each other, thereby reducing the mismatch caused by missing dark light texture. In the global registration of point clouds, by appropriately increasing the feature search radius and increasing the maximum number of neighbors, sparse feature points in a dark light environment and local feature deficiency in a low texture area are covered.

[0122] Please refer to Figure 4 , the embodiment of the present application further provides a three-dimensional reconstruction system for an underground cavity, which can implement the above method. The system comprises:

[0123] The acquisition module 401 is configured to perform multi-view shooting on the underground cavity by using an image acquisition device to obtain multi-view depth maps.

[0124] The point cloud conversion module 402 is configured to perform point cloud conversion on the multi-view depth maps to obtain three-dimensional point cloud data.

[0125] The preprocessing module 403 is configured to perform preprocessing on the three-dimensional point cloud data to obtain preprocessed point cloud data.

[0126] The feature matching module 404 is configured to perform feature matching on the preprocessed point cloud data to obtain coarse registration point cloud.

[0127] The iterative optimization module 405 is configured to perform iterative optimization on the coarse registration point cloud to obtain fine registration point cloud.

[0128] The model construction module 406 is configured to perform error evaluation and fusion on the fine registration point cloud to obtain a three-dimensional point cloud model of the underground cavity.

[0129] It can be understood that the contents in the above method embodiments are applicable to the present system embodiments, the present system embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0130] The present application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above method. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0131] It can be understood that the contents in the above method embodiments are applicable to the present device embodiments, the present device embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0132] Please refer to Figure 5 , Figure 5 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0133] The processor 501 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the present application.

[0134] The memory 502 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 502 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 502 and are called and executed by the processor 501 to perform the above-mentioned method of the embodiments of the present application.

[0135] The input / output interface 503 is configured to realize information input and output.

[0136] The communication interface 504 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0137] The bus 505 is configured to transmit information between various components (for example, the processor 501, the memory 502, the input / output interface 503, and the communication interface 504) of the device.

[0138] The processor 501, the memory 502, the input / output interface 503, and the communication interface 504 are connected to each other through the bus 505 to realize the communication connection between the device.

[0139] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned method.

[0140] It can be understood that the content in the above-mentioned method embodiments is applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above-mentioned method embodiments, and the beneficial effects achieved are also the same as those of the above-mentioned method embodiments.

[0141] The embodiments of the present application also provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the above-mentioned method.

[0142] It can be understood that the content in the above-mentioned method embodiments is applicable to the present program product embodiments, the functions specifically implemented by the present program product embodiments are the same as those of the above-mentioned method embodiments, and the beneficial effects achieved are also the same as those of the above-mentioned method embodiments.

[0143] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include memory that is remotely disposed relative to the processor, which can be connected to the processor through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The embodiment of the present application provides a kind of underground cavity three-dimensional reconstruction method, system, equipment and medium, the scheme is carried out multi-view shooting processing to underground cavity by image acquisition device, obtains multi-view depth map;Multi-view depth map is carried out point cloud conversion processing, obtains three-dimensional point cloud data;Three-dimensional point cloud data is preprocessed, and preprocessed point cloud data is obtained;Preprocessed point cloud data is carried out feature matching processing, and rough registration point cloud is obtained;Rough registration point cloud is carried out iterative optimization processing, and accurate registration point cloud is obtained;Accurate registration point cloud is carried out error evaluation and fusion processing, and the three-dimensional point cloud model of underground cavity is obtained.The embodiment of the present application generates visual underground cavity three-dimensional model by processing the image collected, can intuitively show the morphology, volume size of cavity, to provide comprehensive, intuitive road underground cavity analysis for underground cavity repair work.

[0145] The embodiment of the present application can accurately reconstruct three-dimensional for underground cavity, by designing the bracket of binocular depth camera and using the method of point cloud technology, in the case of non-large area damage pavement, accurately reconstruct three-dimensional for underground cavity, can more intuitively observe the internal structure of underground cavity, and take corresponding measures, reduce the resource loss of surveying underground cavity.

[0146] The embodiment of the present application can accurately reconstruct three-dimensional for underground cavity, by designing the bracket of binocular depth camera and using the method of point cloud technology, in the case of non-large area damage pavement, accurately reconstruct three-dimensional for underground cavity, can more intuitively observe the internal structure of underground cavity, and take corresponding measures, reduce the resource loss of surveying underground cavity.

[0147] The embodiment of the present application can also solve the problem that the shape and size of the underground cavity are difficult to obtain, by using the designed underground space image acquisition device, the image data can be obtained from the inside of the underground cavity, the three-dimensional reconstruction of the underground cavity is realized efficiently and accurately by using the point cloud technology, and the embodiment of the present application proves that the three-dimensional reconstruction can be well realized by using the point cloud in the dark environment.

[0148] The image acquisition device has the advantages of reliable structure and convenience, is made of stainless steel, is corrosion resistant, light in weight, convenient to carry and operate, and is helpful to quickly carry out operation.

[0149] The application prospect of the embodiment of the present application is wide, which can be applied to the underground cavity of the actual engineering, also applicable to the underground cave, and can also be applied to the internal structure of the object which is difficult for people to know, so as to intuitively know the internal structure and size.

[0150] The embodiments described in the present application are for more clearly illustrating the technical solutions of the present application, and do not constitute a limitation on the technical solutions provided by the present application, and the skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the present application are also applicable to similar technical problems.

[0151] The skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the present application, and can include more or less steps than the figures, or combine certain steps, or different steps.

[0152] The system embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0153] The skilled in the art can understand that all or some steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0154] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above figures are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0155] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.

[0156] In several embodiments provided in the application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between systems or units, which can be electrical, mechanical or other forms.

[0157] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0158] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0159] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0160] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for three-dimensional reconstruction of underground cavities, characterized in that, The method includes the following steps: Multi-view depth maps are obtained by taking images of underground cavities from multiple perspectives using an image acquisition device. The multi-view depth map is processed by point cloud conversion to obtain three-dimensional point cloud data; The three-dimensional point cloud data is preprocessed to obtain preprocessed point cloud data; The preprocessed point cloud data is subjected to feature matching processing to obtain a coarsely registered point cloud; The coarsely registered point cloud is iteratively optimized to obtain the finely registered point cloud; Error assessment and fusion processing are performed on the precise point cloud to obtain a three-dimensional point cloud model of the underground cavity.

2. The method according to claim 1, characterized in that, The point cloud conversion process performed on the multi-view depth map to obtain 3D point cloud data includes: The multi-view depth map is converted to grayscale format to obtain a grayscale image; The grayscale image is converted to a floating-point format to obtain floating-point data; The floating-point data is processed to generate point cloud data based on the camera parameters of the image acquisition device to obtain the three-dimensional point cloud data.

3. The method according to claim 1, characterized in that, The preprocessing of the 3D point cloud data to obtain preprocessed point cloud data includes: The 3D point cloud data is filtered, statistically analyzed, and outlier removal is performed according to the denoising algorithm to obtain denoised data. The denoised data is processed by adjusting the point cloud voxels according to the downsampling algorithm to obtain the preprocessed point cloud data.

4. The method according to claim 1, characterized in that, The step of performing feature matching processing on the preprocessed point cloud data to obtain a coarsely registered point cloud includes: The preprocessed point cloud data is subjected to feature extraction processing using the fast point feature histogram algorithm to obtain point cloud features; The point cloud features are subjected to global feature matching processing based on the mutual filtering mechanism to obtain the coarsely registered point cloud.

5. The method according to claim 1, characterized in that, The iterative optimization process of the coarsely registered point cloud to obtain the finely registered point cloud includes: The coarsely registered point cloud is subjected to matrix calculation processing to obtain the transformation matrix; Based on the transformation matrix, the coarsely registered point cloud is subjected to nearest neighbor search processing to obtain the first data; The first data is filtered for invalid matches based on a preset distance and normal direction to obtain the second data. The second data is linearly solved using the least squares method, and the transformation matrix is ​​updated based on the linear solution result. The process then returns to the nearest neighbor search process performed on the coarsely registered point cloud based on the transformation matrix until the iteration termination condition is met, resulting in the finely registered point cloud.

6. The method according to claim 5, characterized in that, The linear solution process for the second data based on the least squares method includes: The second data is classified and processed according to the data acquisition order to obtain source point cloud and target point cloud; The rotation matrix and translation vector are obtained from the transformation matrix; The source point cloud is multiplied by the rotation matrix, the multiplication result is added to the translation vector, and the addition result is subtracted from the target point cloud to calculate the distance error between the source point cloud and the target point cloud. The linear solution result is obtained by minimizing the distance error result.

7. The method according to any one of claims 1 to 5, characterized in that, The process of error evaluation and fusion processing of the precisely matched point cloud to obtain a three-dimensional point cloud model of the underground cavity includes: The precise point cloud is downsampled to obtain downsampled data; The downsampled data is processed by removing outliers in sparse regions using radius filtering and quadratic statistical filtering algorithms to obtain filtered data. The proportion of effective matching point pairs to the total number of points is calculated in the filtered data to obtain the error evaluation result. Based on the error assessment results, the filtered data and the precisely matched point cloud are fused to obtain the target point cloud data. The target point cloud data is subjected to three-dimensional reconstruction processing to obtain a three-dimensional point cloud model of the underground cavity.

8. A three-dimensional reconstruction system for underground cavities, characterized in that, The system includes: The acquisition module is used to capture and process underground cavities from multiple perspectives using an image acquisition device to obtain multi-view depth maps. The point cloud conversion module is used to perform point cloud conversion processing on the multi-view depth map to obtain three-dimensional point cloud data. The preprocessing module is used to preprocess the three-dimensional point cloud data to obtain preprocessed point cloud data. The feature matching module is used to perform feature matching processing on the preprocessed point cloud data to obtain a coarsely registered point cloud. The iterative optimization module is used to perform iterative optimization processing on the coarsely registered point cloud to obtain the finely registered point cloud. The model building module is used to perform error evaluation and fusion processing on the precise point cloud to obtain a three-dimensional point cloud model of the underground cavity.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images