A pure vision three-dimensional reconstruction and self-adaptive completion method under collapse environment

Abstract

This invention discloses a pure vision-based 3D reconstruction and adaptive completion method in a collapse environment, belonging to the field of computer vision and emergency rescue 3D reconstruction technology. The method completes initial 3D reconstruction by acquiring data through binocular vision, constructs an anisotropic point cloud density distribution field to achieve accurate detection of low-density blind spots, establishes safe and reachable spatial constraints based on an implicit distance field, and optimizes the completion viewpoint by fusing an observability model. Map iteration is completed through local reconstruction and point cloud updates, and a closed-loop completion mechanism is formed using point cloud density as feedback until the reconstructed model meets the integrity requirements. This invention solves the problems of permanent voids in reconstruction caused by viewpoint blind spots, poor consistency between completion results and the real scene, and insufficient safety in the reconstruction process in existing technologies in collapse environments. It can achieve complete and high-precision autonomous reconstruction of 3D models of collapse disaster sites, providing reliable spatial data support for emergency rescue decision-making.

Description

Technical Field

[0001] This invention belongs to the field of computer vision 3D reconstruction technology, specifically relating to 3D environment reconstruction technology in collapse disaster scenarios, and more specifically, to a pure vision 3D reconstruction and adaptive completion method based on point cloud density feedback in a collapse environment. Background Technology

[0002] 3D reconstruction technology is one of the core research directions in the field of computer vision, and it has crucial value in extreme environment applications such as emergency rescue, disaster area assessment, and unmanned exploration. In disaster sites such as collapses and tunnel collapses, rapidly acquiring the complete 3D structure of the disaster area is a core data support for rescue command decisions, route planning, and the deployment of rescue forces.

[0003] However, collapse disaster sites are typically characterized by fragmented structures, narrow spaces, limited viewpoints, and a high risk of secondary collapse, posing a severe challenge to traditional 3D reconstruction technologies. Existing mainstream visual SLAM (Simultaneous Localization and Mapping) systems can achieve good mapping results in normal environments, but in collapse environments, due to a single viewpoint and severe occlusion, the reconstructed model often has large areas of voids, failing to fully represent the geometric structure of the disaster area and seriously affecting subsequent analysis and decision-making.

[0004] Existing point cloud completion techniques can be mainly divided into two categories: one is data-driven completion methods based on deep learning, such as the 3D point cloud completion method based on deep learning and voxels disclosed in patent publication number CN112927359A, and the point cloud completion method based on latent space topological constraints disclosed in CN113205466A. These methods rely on geometric priors in the training data and have poor generalization ability for scenarios with no fixed shape and random structural breakage in collapse environments. They cannot guarantee the consistency between the completion results and the real physical environment, and may even produce geometric errors that deviate significantly from the actual situation, failing to meet the high reliability requirements of emergency rescue. The other category is point cloud encryption methods based on geometric interpolation, such as the high-precision 3D reconstruction method for uncalibrated images disclosed in patent publication number CN104778748A. These methods can only interpolate and encrypt existing point clouds and cannot generate real geometric information for unobserved areas. They are powerless for completely missing areas caused by viewpoint occlusion.

[0005] In summary, existing mainstream 3D reconstruction and point cloud completion methods are all passive data processing or post-processing completion modes. They cannot actively perceive incomplete reconstruction areas, nor can they autonomously plan completion viewpoints. As a result, the reconstruction model always has permanent holes caused by viewpoint blind spots, which cannot meet the core requirements of 3D model integrity, realism, and safety in emergency rescue scenarios for collapse disasters. Summary of the Invention

[0006] To address the problems of the existing technologies, this invention provides a pure visual 3D reconstruction and adaptive completion method in a collapsed environment. This method solves the problems of existing technologies being unable to actively complete the voids in a collapsed environment, having poor accuracy in the completion results, and insufficient safety in the reconstruction process. It enables complete, high-precision, and highly safe autonomous 3D reconstruction of collapsed scenes.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a method for pure visual 3D reconstruction and adaptive completion under collapsed conditions, comprising the following steps: S1. Binocular vision data acquisition and initial 3D reconstruction: Based on the pre-calibrated binocular camera, the first image pair is acquired at the initial position of the collapsed environment; the first image pair is stereo matched to calculate the disparity map, and the depth map is obtained based on the principle of triangulation; according to the depth map and camera intrinsic parameters, the pixels are back-projected into the 3D space to generate an initial 3D point cloud map.

[0008] S2. Anisotropic Point Cloud Density Distribution Modeling: The space of the current 3D point cloud map is divided into a uniform voxel grid. For each non-empty voxel, the 3D covariance matrix of its internal point cloud is calculated. Based on the eigenvalues ​​and eigenvectors of the covariance matrix, an anisotropic density distribution function is constructed. The anisotropic density distribution function is used to describe the expected number of points per unit distance along any direction. Based on the anisotropic density distribution functions of all voxels, a 3D anisotropic density distribution field is generated.

[0009] S3. Low-density region detection and threshold determination: Set an anisotropic density threshold function, traverse all voxels in the three-dimensional anisotropic density distribution field, and if the density value of a voxel in the principal normal direction is lower than the output value of the density threshold function, then mark the voxel as a low-density voxel; cluster spatially adjacent low-density voxels to form a set of low-density regions to be completed.

[0010] S4. Reachability space constraint modeling based on implicit distance field: Obtain the pose of the current stereo camera in the global coordinate system and treat it as a movable point in 3D space; construct an implicit symbolic distance field function based on the current 3D point cloud map, which is used to represent the signed distance from the spatial point to the point cloud; define a reachability space constraint model based on the implicit symbolic distance field function, which is used to limit the movable point of the camera to maintain a distance of not less than a preset safe obstacle avoidance distance from the existing point cloud.

[0011] S5. Adaptive viewpoint planning based on observability model: Within the reachable space constraint model, a set of candidate viewpoints is generated by sampling for each low-density region; for each candidate viewpoint, an observability model based on view coverage is introduced. The observability model comprehensively considers the visibility of the point relative to the viewpoint, the observation angle, and the observation distance, and outputs the observability confidence in the interval [0,1]; construct an information gain optimization objective function, and select the optimal completion viewpoint that maximizes the information gain.

[0012] S6. Local Reconstruction and Point Cloud Update: Control the mobile platform equipped with a binocular camera to move to the optimal completion viewpoint and acquire a second image pair; perform the method of step S1 on the second image pair to complete local 3D reconstruction and obtain a local point cloud map; perform coordinate system unification and stitching fusion between the local point cloud map and the current 3D point cloud map, remove redundant points, and form an updated 3D point cloud map.

[0013] S7. Closed-loop iterative completion mechanism: The updated 3D point cloud map is used as the new current 3D point cloud map, and the process jumps to step S2 and repeats until the set of low-density regions detected in step S3 is empty, thus completing the 3D reconstruction and adaptive completion of the collapsed environment.

[0014] Furthermore, the construction process of the anisotropic density distribution function in step S2 is specifically as follows: input the current point cloud map. voxel division parameters For each voxel If the number of internal points is not less than 3, then calculate its point cloud centroid. Covariance Matrix .

[0015] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues. and the corresponding feature vectors .

[0016] Define the anisotropic density distribution function as: in, For voxels, The number of point clouds inside, where δ is the side length of the voxel mesh. Covariance matrix The false rebellion, Let be an arbitrary direction vector. This function takes a larger value in directions where the point cloud is densely distributed and a smaller value in directions where it is sparse. The technical effect is that it can accurately characterize the density of point cloud distribution in different directions under the collapse environment, providing directional awareness for the subsequent detection of low-density areas and avoiding misjudging unidirectional sparse areas as overall low density due to the lack of viewpoint.

[0017] Furthermore, the density threshold function in step S3 is an adaptive threshold function, and its setting process is as follows: calculate the mean density of all voxels along the principal normal direction. and standard deviation The principal normal direction is the direction of the eigenvector corresponding to the smallest eigenvalue of the covariance matrix.

[0018] The density threshold function is set as follows: in, This is an empirical coefficient. Let be the average density of all voxels along the principal normal direction. Let be the principal normal direction vector of the voxel to be detected. The gravity direction vector is used to take into account the prior knowledge that the surface of an object in a collapsed environment is usually perpendicular to the gravity direction. The technical effect is that the threshold can be adaptively adjusted according to the local direction, which improves the accuracy and robustness of low-density area detection, and is especially suitable for collapsed ruin surfaces with obvious directional textures or structures.

[0019] Furthermore, in step S4, the implicit symbolic distance field function is constructed using a truncated symbolic distance field (TSDF). Specifically, the current point cloud map and its corresponding camera pose are used as input, stored through voxel hashing or an octree data structure, and the global TSDF model is updated online to obtain the implicit symbolic distance field function. The reachable space constraint model is specifically as follows: in This model represents the camera's movable point, representing a preset safe obstacle avoidance distance. It must be located outside the isosurface of the implicit distance field, that is, at least 1 / 3 of the distance from the existing point cloud. The distance.

[0020] The above updates improve the reachability space constraint model. Further specified as: in, The preset safe obstacle avoidance distance, To truncate distances, empty areas that are too far from the reconstructed surface are excluded; the technical advantage is that the TSDF model can compactly represent the geometry of the environment and directly provide distance information to the nearest surface, facilitating efficient collision detection and viewpoint accessibility assessment. The introduction of this prevents the camera from moving into completely unconstrained, unknown areas.

[0021] Furthermore, the construction process of the observability model based on view coverage in step S5 is specifically as follows: Observability model based on view coverage The construction process is as follows: input the points to be observed. and its normal vector Location of candidate viewpoints And camera intrinsic parameters; first determine the point Whether it is located within the camera's field of view and is not occluded by other point clouds; if occluded, the confidence level can be observed. .

[0022] If not obstructed, calculate the observation direction. The observable confidence level is calculated using the following formula: in, To point from the observation point p to the candidate viewpoint ν c The observation direction vector, Let p be the normal vector of the point to be observed. This is the distance attenuation coefficient. The model considers the effects of both the observation angle (the angle between the surface normal and the line of sight) and the observation distance. The technical advantage is that it can quantify the quality of observing the same surface from different viewpoints, so that viewpoint selection not only pursues visibility, but also the high quality of observation, thereby improving the accuracy and completeness of the completed point cloud.

[0023] Furthermore, the information gain optimization objective function in step S5 is: Among them, V candidate R is the set of candidate viewpoints. k For low-density areas that need to be filled in, From viewpoint ν c Observation point The desired increase in local point cloud density is then obtained. For weights based on the observability model; select those that make The candidate viewpoint that is maximized is used as the optimal completion viewpoint. This includes the local point cloud density increment. The estimation method is as follows: based on a pre-trained depth map noise model and a stereo matching uncertainty model, the prediction is made from the viewpoint. Observation point The uncertainty of the position of the reconstructed 3D point is considered, and the expected increase in the density of the voxels around the point is calculated accordingly; specifically, the point... Its uncertainty is projected into voxel space, and a Gaussian weighted distribution is used to... Distributed to neighboring voxels, ultimately resulting in the entire low-density region. The expected density increment field; enabling the information gain to more realistically reflect the actual change in density after reconstruction, avoiding suboptimal viewpoint selection due to ignoring reconstruction noise.

[0024] Furthermore, step S6 also includes a noise filtering step, specifically: calculating the normal vector of each point in the local point cloud map and comparing its consistency with the normal vector of the corresponding point in the global 3D point cloud map; removing points whose normal vector angle is greater than a preset threshold; performing statistical filtering on the remaining points to remove outliers whose average distance from their neighborhood is greater than a preset distance threshold; using voxel grid downsampling to homogenize the point cloud density; and effectively suppressing reconstruction noise introduced by changes in illumination and motion blur through multi-level filtering, ensuring the consistency and purity of the map during the closed-loop update process.

[0025] Furthermore, step S7 also includes a convergence judgment optimization step, specifically: after each iteration, calculate the reduction rate of the total volume of the low-density region between the current iteration and the previous iteration. ;like Less than the preset convergence threshold Even if the set of low-density regions is not empty, the completion process is still considered convergent, and the iteration is terminated. This avoids infinite loops caused by physical inaccessibility, visual dead zones, etc., and provides a reliable algorithm termination criterion for practical engineering applications.

[0026] Furthermore, between steps S4 and S5, there is an environmental mutation detection and response step, specifically: before executing viewpoint planning, visual odometry tracking is performed using images acquired in real time by a binocular camera and the current 3D point cloud map; if tracking is lost or reprojection error exceeds a preset error threshold, an environmental mutation is determined, the adaptive completion process is paused, local map relocalization and updating are triggered, rapid 3D reconstruction is completed using the most recent image pair, and the 3D anisotropic density distribution field and reachable space constraint model are updated. After the system stabilizes, step S2 is re-entered. This enhances the system's robustness and survivability in dynamic and unstable collapse environments, and improves the safety of the completion process.

[0027] Furthermore, the sampling method for the candidate viewpoint set in step S5 is as follows: uniform spiral sampling is performed on the isosurface of the reachable space constraint model, and the isosurface formula is... ,in For implicit sign distance field function, The preset safe obstacle avoidance distance, The preset small offset; sampling density and the area to be completed The volume and complexity are positively correlated; specifically, first calculate The bounding sphere is projected onto the isosurface as the initial point, and then sampling points on the isosurface are generated according to the Fibonacci spiral, so that the sampling points are evenly distributed and cover all possible observation directions. By limiting the sampling to the isosurface at a distance slightly greater than the safety distance from the surface, the safety of the viewpoint is ensured, while the efficiency and coverage of sampling are improved.

[0028] Compared with the prior art, the present invention has the following advantages: 1. This invention constructs an active 3D reconstruction closed-loop architecture based on point cloud density feedback. Unlike the passive post-processing completion mode of existing technologies, it uses point cloud density evaluation results as feedback to drive the cyclic iteration of viewpoint planning, local reconstruction, and map updating. This fundamentally solves the problem of permanent voids in the reconstruction model caused by viewpoint blind spots in the collapsed environment, and realizes autonomous evolution from incomplete acquisition to complete reconstruction, significantly improving the integrity and credibility of the 3D map of the disaster area.

[0029] 2. The anisotropic density distribution modeling and low-density region detection method proposed in this invention constructs a direction-dependent density distribution function through the covariance matrix, which can accurately characterize the density of point clouds in different directions on the surface of collapsed ruins. It overcomes the shortcomings of traditional isotropic density detection methods that cannot distinguish between sparse texture and missing occlusion, and achieves high-precision identification of blind spots, providing accurate target positioning for viewpoint planning.

[0030] 3. This invention constructs an reachability space constraint model based on an implicit distance field, abstracting the camera as a movable point in the implicit field. While ensuring a safe distance from collapsed obstacles, it provides a continuous and efficient solution space description, avoiding the combinatorial explosion problem of discrete grid path search, and effectively avoiding the risk of secondary collapse, thus ensuring the safety of the reconstruction process.

[0031] 4. This invention integrates an observability model to construct a viewpoint optimization objective function, comprehensively considering observation angle, observation distance, and occlusion relationship. It not only ensures the visibility of the area to be completed, but also achieves high-quality reconstruction with the best observation posture. It solves the problems of poor reconstruction accuracy and poor completion effect of traditional methods in complex and narrow spaces, and realizes intelligent autonomous decision-making for the optimal completion viewpoint. Attached Figure Description

[0032] Figure 1 This is the overall flowchart of the present invention.

[0033] Figure 2 This is a flowchart illustrating the anisotropic density distribution modeling and low-density region detection in this invention.

[0034] Figure 3 This is a sub-flowchart of the reachability space constraint modeling based on implicit distance fields in this invention.

[0035] Figure 4 This is a flowchart of the adaptive viewpoint planning and information gain maximization in this invention.

[0036] Figure 5 This is a flowchart of the closed-loop iterative completion and convergence judgment in this invention. Detailed Implementation

[0037] The present invention will be further described below.

[0038] This embodiment provides a pure vision-based 3D reconstruction and adaptive completion method for collapsed environments based on point cloud density feedback. It is implemented on a tracked rescue robot equipped with a pre-calibrated binocular camera with a resolution of 1280×720 and a baseline distance of 120mm. This method is suitable for 3D reconstruction of tunnel collapse disaster sites. Figure 1 As shown, the specific steps are as follows: Step 1: Binocular vision data acquisition and initial 3D reconstruction: Control the rescue robot to reach the safe initial position at the entrance of the collapsed tunnel, acquire the first image pair through the binocular camera, and use the semi-global block matching (SGBM) algorithm to perform stereo matching on the first image pair to calculate the disparity map; based on the calibration intrinsic parameters and baseline distance of the binocular camera, convert the disparity map into a depth map through the principle of triangulation; back-project each pixel of the depth map, combined with the camera intrinsic parameters and the current camera pose, onto the 3D space to generate an initial 3D point cloud map.

[0039] Step 2: Modeling the density distribution of anisotropic point clouds: such as Figure 2 As shown, the initial 3D point cloud map is divided into a uniform voxel grid with a voxel side length of 5cm. All voxels are traversed, and for non-empty voxels with at least 3 internal points, their point cloud centroids are calculated. Covariance Matrix ; where n i This represents the number of point clouds within a voxel. Let be the three-dimensional coordinates of the j-th point within the voxel.

[0040] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues. and the corresponding feature vectors ;in This represents the principal normal direction of the point cloud within the voxel.

[0041] Construct the anisotropic density distribution function: in, For voxels, The number of point clouds inside, where δ is the side length of the voxel mesh. Covariance matrix The false rebellion, Let be an arbitrary direction vector. This function takes larger values ​​in directions where the point cloud distribution is dense and smaller values ​​in sparse directions. The technical advantage is that it can accurately characterize the density of point cloud distribution in different directions under collapse conditions, providing directional awareness for subsequent low-density region detection and avoiding misjudging unidirectional sparse regions as overall low density due to viewpoint limitations. Based on the anisotropic density distribution function of all voxels, a three-dimensional anisotropic density distribution field covering the entire reconstruction space is generated.

[0042] Step 3: Low-density region detection and threshold determination: such as Figure 2 As shown, an adaptive density threshold function is set, and the first step is to calculate the density threshold of all voxels along the principal normal direction. density mean and standard deviation The density threshold function is set as follows: in, This is an empirical coefficient. Let be the average density of all voxels along the principal normal direction. Let be the principal normal direction vector of the voxel to be detected. The gravity direction vector is used to take into account the prior knowledge that the surface of an object in a collapsed environment is usually perpendicular to the gravity direction. The technical effect is that the threshold can be adaptively adjusted according to the local direction, which improves the accuracy and robustness of low-density area detection, and is especially suitable for collapsed ruin surfaces with obvious directional textures or structures.

[0043] Iterate through all voxels in the three-dimensional anisotropic density distribution field. If the density value of a voxel in the principal normal direction is lower than a threshold... If the voxel is identified as a low-density voxel, then a region growing algorithm is used to cluster spatially adjacent low-density voxels to form a set of low-density regions to be completed. .

[0044] Step 4: Modeling reachability space constraints based on implicit distance fields: such as Figure 3 As shown, the pose of the current stereo camera in the global coordinate system is obtained, and the camera optical center is regarded as a movable point in 3D space. Based on the current 3D point cloud map, a truncated symbolic distance field (TSDF) is constructed using voxel hashing to obtain the implicit symbolic distance field function. , Representing a spatial point The signed distance to the nearest point cloud surface; the reachability space constraint model is specifically as follows: in This model represents the camera's movable point, representing a preset safe obstacle avoidance distance. It must be located outside the isosurface of the implicit distance field, that is, at least 1 / 3 of the distance from the existing point cloud. The distance.

[0045] The above updates improve the reachability space constraint model. Further specified as: in, The preset safe obstacle avoidance distance, To truncate distances, empty areas that are too far from the reconstructed surface are excluded; the technical advantage is that the TSDF model can compactly represent the geometry of the environment and directly provide distance information to the nearest surface, facilitating efficient collision detection and viewpoint accessibility assessment. The introduction of this prevents the camera from moving into completely unconstrained, unknown areas.

[0046] Step 5: Environmental Sudden Change Detection and Response: Before executing viewpoint planning, visual odometry tracking is performed using images acquired in real time by the binocular camera and the current 3D point cloud map to calculate the reprojection error of feature points. If tracking is lost, or the reprojection error exceeds the preset error threshold (set to 10 pixels in this embodiment), it is determined that a sudden change such as secondary collapse has occurred in the environment. The current adaptive completion process is paused, and local map relocation and update are triggered in an emergency. The three most recent sets of images are used for rapid 3D reconstruction, and the 3D anisotropic density distribution field and reachable space constraint model are updated. After the visual odometry tracking stabilizes, Step 2 is restarted.

[0047] Step 6: Adaptive viewpoint planning based on the observability model: such as Figure 4 As shown, in the reachable space constraint model Within the scope, for each low-density area ,exist On the isosurface, uniform sampling is performed using a Fibonacci spiral to generate a set of candidate viewpoints. The sampling density is positively correlated with the volume of the region to be completed. In this embodiment, the number of candidate viewpoints in a single low-density region is set to 50-200.

[0048] For each candidate viewpoint Construct an observability model based on view coverage: Input the observation point and its normal vector Location of candidate viewpoints And camera intrinsics; first, by using view frustum clipping and depth occlusion detection, the point is determined. Whether it is located within the camera's field of view and is not occluded by other point clouds; if occluded, the confidence level can be observed. .

[0049] If not obstructed, calculate the observation direction. The observable confidence level is calculated using the following formula: in, To point from the observation point p to the candidate viewpoint ν c The observation direction vector, Let p be the normal vector of the point to be observed. This is the distance attenuation coefficient. The model considers the effects of both the observation angle (the angle between the surface normal and the line of sight) and the observation distance. The technical advantage is that it can quantify the quality of observing the same surface from different viewpoints, so that viewpoint selection not only pursues visibility, but also the high quality of observation, thereby improving the accuracy and completeness of the completed point cloud.

[0050] Construct the information gain optimization objective function: Among them, V candidate R is the set of candidate viewpoints. k For low-density areas that need to be filled in, From viewpoint ν c Observation point The desired increase in local point cloud density is then obtained. For weights based on the observability model; select those that make The candidate viewpoint that is maximized is used as the optimal completion viewpoint. This includes the local point cloud density increment. The estimation method is as follows: based on a pre-trained depth map noise model and a stereo matching uncertainty model, the prediction is made from the viewpoint. Observation point The uncertainty of the position of the reconstructed 3D point is considered, and the expected increase in the density of the voxels around the point is calculated accordingly; specifically, the point... Its uncertainty is projected into voxel space, and a Gaussian weighted distribution is used to... Distributed to neighboring voxels, ultimately resulting in the entire low-density region. The expected density increment field; enabling the information gain to more realistically reflect the actual change in density after reconstruction, avoiding suboptimal viewpoint selection due to ignoring reconstruction noise.

[0051] Step 7: Local Reconstruction and Point Cloud Update: Control the rescue robot to move to the optimal completion viewpoint, acquire the second image pair through the binocular camera, repeat the method in Step 1 to complete the local 3D reconstruction, and obtain the local point cloud map; use the ICP algorithm to register the local point cloud map with the current global 3D point cloud map, complete the coordinate system integration and stitching fusion, and use voxel downsampling to remove redundant points.

[0052] Simultaneously, noise filtering is performed: the normal vector of each point in the local point cloud map is calculated and compared with the normal vector of the corresponding point in the global point cloud map. Points with a normal vector angle greater than 30° are considered noise and removed. Statistical filtering is performed on the remaining points to remove outliers whose average distance from the neighborhood is greater than 2 standard deviations. Finally, 5cm voxel grid downsampling is used to homogenize the point cloud density and obtain the updated 3D point cloud map.

[0053] Step 8: Closed-loop iterative completion and convergence judgment: such as Figure 5 As shown, the updated 3D point cloud map is used as the new current 3D point cloud map. Then, step 2 is executed repeatedly, and the entire process of density modeling, low-density region detection, viewpoint planning, and local reconstruction is repeated.

[0054] After each iteration, a convergence check is performed: calculate the reduction rate of the total volume of the low-density region between the current iteration and the previous iteration. ;like Less than the preset convergence threshold (In this embodiment, it is set to 5%). Even if the set of low-density regions is not empty, the completion process is determined to be converged and the iteration is terminated.

[0055] In the test of the simulated tunnel collapse environment, the method of this embodiment improves the integrity of the reconstructed model by 42% and reduces the geometric error of cavity area completion by 68% compared with the traditional visual SLAM + post-processing completion method. At the same time, it ensures a safe distance between the robot and the collapsed structure throughout the process, which can effectively meet the three-dimensional reconstruction needs of emergency rescue scenarios for collapse disasters.

[0056] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for pure visual 3D reconstruction and adaptive completion in a collapsed environment, characterized in that, Includes the following steps: S1. Based on the pre-calibrated binocular camera, acquire the first image pair at the initial position of the collapsed environment; perform stereo matching to calculate the disparity map of the first image pair, and obtain the depth map based on the principle of triangulation; according to the depth map and camera intrinsic parameters, back-project the pixels to the three-dimensional space to generate an initial three-dimensional point cloud map. S2. Divide the space of the current 3D point cloud map into a uniform voxel grid. For each non-empty voxel, calculate the 3D covariance matrix of the point cloud inside it. Based on the eigenvalues ​​and eigenvectors of the covariance matrix, construct an anisotropic density distribution function, and then generate a 3D anisotropic density distribution field. S3. Set an anisotropic density threshold function, traverse all voxels in the three-dimensional anisotropic density distribution field, and mark the voxel as a low-density voxel if the density value of the voxel in the principal normal direction is lower than the output value of the density threshold function; cluster the spatially adjacent low-density voxels to form a set of low-density regions to be completed. S4. Obtain the current pose of the stereo camera in the global coordinate system and treat it as a movable point in three-dimensional space; An implicit symbolic distance field function is constructed based on the current 3D point cloud map, and a reachable space constraint model is defined based on the implicit symbolic distance field function. S5. Within the reachable space constraint model, sample and generate a set of candidate viewpoints for each low-density region; for each candidate viewpoint, introduce an observability model based on view coverage, construct an information gain optimization objective function, and select the optimal completion viewpoint that maximizes information gain. S6. Control the mobile platform equipped with a binocular camera to move to the optimal completion viewpoint and acquire the second image pair; perform the method of step S1 on the second image pair to complete the local three-dimensional reconstruction and obtain the local point cloud map; perform coordinate system matching and stitching fusion on the local point cloud map and the current three-dimensional point cloud map to form the updated three-dimensional point cloud map. S7. Use the updated 3D point cloud map as the new current 3D point cloud map, jump to step S2 and repeat until the set of low-density regions detected in step S3 is empty, and complete the 3D reconstruction and adaptive completion of the collapsed environment.

2. The method according to claim 1, characterized in that, The construction process of the anisotropic density distribution function in step S2 is as follows: Input the current point cloud map and voxel partitioning parameters; for each voxel, calculate the centroid and covariance matrix of its internal point cloud; perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors; define the anisotropic density distribution function as follows: in, For voxels, The number of point clouds inside, where δ is the side length of the voxel mesh. Covariance matrix The false rebellion, It is a vector with arbitrary direction.

3. The method according to claim 1, characterized in that, The density threshold function in step S3 is an adaptive threshold function, and its setting process is as follows: calculate the mean density of all voxels in the principal normal direction. and standard deviation The principal normal direction is the direction of the eigenvector corresponding to the smallest eigenvalue of the covariance matrix; The density threshold function is set as follows: in, This is an empirical coefficient. Let be the average density of all voxels along the principal normal direction. The direction vector of gravity. Let be the principal normal direction vector of the voxel to be detected.

4. The method according to claim 1, characterized in that, In step S4, the implicit symbolic distance field function is constructed using a truncated symbolic distance field TSDF. Specifically, the current point cloud map and its corresponding camera pose are taken as input, stored using a voxel hash or octree data structure, and the global TSDF model is updated online to obtain the implicit symbolic distance field function. The reachable space constraint model is specifically as follows: in, The preset safe obstacle avoidance distance, To cut off the distance.

5. The method according to claim 1, characterized in that, Step S5: Observability model based on view coverage The construction process is as follows: input the points to be observed. and its normal vector Location of candidate viewpoints And camera intrinsic parameters; first determine the point Whether it is located within the camera's field of view and is not occluded by other point clouds; if occluded, the confidence level can be observed. ; If not obstructed, calculate the observation direction. The observable confidence level is calculated using the following formula: in, To point from the observation point p to the candidate viewpoint ν c The observation direction vector, Let p be the normal vector of the point to be observed. This is the distance attenuation coefficient.

6. The method according to claim 5, characterized in that, The objective function for optimizing information gain in step S5 is: Among them, V candidate R is the set of candidate viewpoints. k For low-density areas that need to be filled in, From viewpoint ν c Observation point The desired increase in local point cloud density is then obtained. For weights based on the observability model; select those that make The candidate viewpoint that is maximized is used as the optimal completion viewpoint.

7. The method according to claim 1, characterized in that, Step S6 also includes a noise filtering step, specifically: calculating the normal vector of each point in the local point cloud map, comparing its consistency with the normal vector of the corresponding point in the global 3D point cloud map, and removing points whose normal vector angle is greater than a preset threshold as noise. Statistical filtering is applied to the remaining points to remove outliers whose average distance from their neighbors is greater than a preset distance threshold; voxel grid downsampling is used to homogenize the point cloud density.

8. The method according to claim 1, characterized in that, Step S7 also includes a convergence judgment optimization step, specifically: after each iteration, calculate the reduction rate of the total volume of the low-density region between the current iteration and the previous iteration. ;like Less than the preset convergence threshold Even if the set of low-density regions is not empty, the completion process is determined to be converged, and the iteration is terminated.

9. The method according to claim 1, characterized in that, Between steps S4 and S5, there is also an environmental change detection and response step, which is as follows: before executing viewpoint planning, visual odometry tracking is performed using images acquired in real time by binocular cameras and the current 3D point cloud map; if tracking is lost or reprojection error exceeds a preset error threshold, it is determined that a sudden change has occurred in the environment, the adaptive completion process is paused, local map relocation and update are triggered, rapid 3D reconstruction is completed using the most recent image pair, the 3D anisotropic density distribution field and reachable space constraint model are updated, and the system re-enters step S2 after it stabilizes.

10. The method according to claim 1, characterized in that, The sampling method for the candidate viewpoint set in step S5 is as follows: uniform spiral sampling is performed on the isosurface of the reachable space constraint model. The isosurface formula is... ,in For implicit sign distance field function, The preset safe obstacle avoidance distance, This is a preset small offset.

Images