Fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation
By employing a graph neural network-based collaborative anchor point optimization mechanism and adaptive control strategy, the problems of long reconstruction time and redundant anchor points in traditional 3DGS are solved, enabling fast and high-quality 3D reconstruction and rendering.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HEFEI UNIV OF TECH
- Filing Date
- 2025-11-25
- Publication Date
- 2026-07-21
AI Technical Summary
Traditional 3DGS reconstruction processes are too time-consuming, especially in complex environments where they may take several hours. Furthermore, existing methods suffer from redundant anchor point generation and pseudo-gradient issues, resulting in slow model convergence.
An anchor point collaboration approach based on graph neural networks is adopted, which uses an anchor point collaborative optimization mechanism for information interaction and feature fusion. Combined with an adaptive anchor point control strategy, the number and distribution of anchor points are dynamically adjusted to optimize Gaussian parameters.
It achieves rapid 3D reconstruction, improves reconstruction quality and rendering effects, reduces redundant anchor points, and enhances model convergence speed and rendering efficiency.
Smart Images
Figure CN121639922B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and computer graphics technology, and in particular relates to a fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation. Background Technology
[0002] With the rapid development of technologies such as augmented reality and digital twins, there is a growing demand for high-fidelity, real-time interactive 3D scene reconstruction technologies. Against this backdrop, 3DGS technology has emerged. However, 3DGS suffers from a lengthy model training phase, which involves transforming multi-view images into 3D reconstructions. Traditional 3DGS reconstruction relies on large-scale stochastic gradient descent and adaptive density control strategies, a process involving iterative optimization of parameters for millions of Gaussian ellipsoids. In complex environments, a complete reconstruction can take hours. Fast scene reconstruction based on 3D Gaussian splashing technology has wide applications across various fields. Accelerating and compressing Gaussian splashing technology can drive the implementation of 3D real-world applications, efficiently completing high-quality 3D scene reconstruction within minutes or even seconds, bringing new possibilities to various industries.
[0003] In the field of autonomous driving, cars need to perceive their surroundings and plan feasible routes. High-speed reconstruction of 3D scenes can provide autonomous driving with realistic road scenes in real time, which helps autonomous driving to better plan driving routes, avoid collisions, and improve driving safety. In the field of augmented reality, navigation systems need to quickly understand the 3D environment in which the user is located. High-speed 3D reconstruction can continuously optimize the reconstructed environment and process new environments as the user moves, providing users with a more immersive experience.
[0004] In recent years, many methods have been proposed to accelerate the reconstruction process of 3DGS. For example, Scaffold-GS reduces the number of Gaussian primitives involved in each rendering, thereby reducing the time required for each training round. However, this algorithm also has some drawbacks: (1) In most cases, the Gaussian primitives generated by the anchor point need to fit the scene together with the Gaussian primitives generated by its neighboring anchor points. Even if the Gaussian primitives generated by two anchor points are similar in properties when projected into the pixel space, the Gaussian parameters controlled by different anchor points are still optimized independently during the Gaussian parameter optimization process. This slows down the convergence speed and makes it easy for pseudo-gradient changes to occur; (2) When the Scaffold-GS algorithm controls the anchor point density, the distribution problem of pseudo-gradient changes in Gaussian primitives causes the density strategy to incorrectly supplement the number of anchor points. In this process, many redundant anchor points will appear, which will occupy some performance during the optimization process and reduce the convergence speed of the model. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a 3DGS three-dimensional reconstruction method accelerated by a graph neural network anchor point collaboration approach. This method can quickly reconstruct three-dimensional scenes with a low model size, while also being computationally simple and easy to implement in engineering.
[0006] To achieve the above objectives, this invention provides a fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation, comprising: Based on the input multi-view images, the camera pose and sparse point cloud of the scene are obtained through the motion recovery structure algorithm, and the sparse point cloud is preprocessed to obtain normalized initial geometric information. Based on the initial geometric information, the three-dimensional space is voxelized, and anchor points are generated at the center of each voxel to obtain the spatial distribution of anchor points. An anchor point topology map is constructed based on the spatial distribution of anchor points. The anchor point topology map is then input into a graph neural network for message passing and feature aggregation to obtain the aggregated features of each anchor point. Based on the aggregated features and the original anchor features, the attribute parameters of Gaussian units are predicted by multilayer perceptron mapping to obtain a three-dimensional Gaussian representation; Multi-view projection and rasterization rendering are performed based on the three-dimensional Gaussian representation to obtain a rendered image. A loss function is calculated based on the rendered image and the input image. At the same time, anchor control signals are generated based on the gradient of Gaussian meta-parameters and scale information. The number and distribution of anchor points are dynamically adjusted according to the anchor point control signal, and the parameters of the graph neural network and multilayer perceptron are optimized by backpropagation. The above steps are repeated until convergence is achieved, thus completing the 3DGS three-dimensional reconstruction.
[0007] Preferably, the process of obtaining normalized initial geometric information includes: SIFT feature points and feature descriptors are extracted from the multi-view images, and sparse point clouds and camera parameters are obtained through feature matching and triangulation. Noise filtering is performed on the sparse point cloud to remove outliers; The filtered sparse point cloud is scaled to obtain a normalized point cloud that is not affected by the actual size of the scene. The spatial range of the 3D reconstruction is determined based on the normalized point cloud, and the initial geometric information is obtained.
[0008] Preferably, the process of obtaining the spatial distribution of anchor points includes: Based on the normalized point cloud and the preset voxel size, the voxel center coordinates are calculated by rounding down and deduplication operations to obtain the anchor point position; The set of basic anchor attributes is obtained by initializing the position attributes, feature vector, scaling factor, and learnable offset for each anchor point. The anchor point spatial distribution is constructed based on the set of basic anchor point attributes.
[0009] Preferably, the process of constructing an anchor point topology map based on the spatial distribution of anchor points includes: Calculate the neighborhood offset coordinates based on the anchor point coordinates and voxel size to obtain a set of candidate neighbor coordinates; Based on all anchor point coordinates stored in the hash table, query the anchor points that exist in the candidate neighbor coordinate set to obtain the actual set of neighbor anchor points; Based on the connection relationship between the neighbor anchor set and the current anchor, construct the anchor topology graph.
[0010] Preferably, the process of obtaining the aggregation features of each anchor point includes: Based on the difference between the features of neighboring anchor points and the features of the current anchor point, feature difference information is obtained; The locations of neighboring anchor points are encoded to obtain location feature information; By concatenating and linearly transforming the feature difference information and the location feature information, the message transmitted from the neighboring anchor point to the current anchor point is obtained. The aggregation feature is obtained by aggregating the messages passed from all neighboring anchors to the current anchor.
[0011] Preferably, the process of aggregating the messages passed from all neighboring anchors to the current anchor to obtain the aggregation feature includes: Calculate the inverse distance weight based on the spatial distance between the neighboring anchor points and the current anchor point, and obtain the weight coefficient of each neighboring anchor point; The weighted sum of messages transmitted by all neighboring anchors is obtained by weighting the messages according to the weighting coefficients. The aggregated features are obtained by processing the weighted message sum using a nonlinear activation function.
[0012] Preferably, the process of obtaining the three-dimensional Gaussian representation includes: Calculate the relative distance and direction vectors based on the current rendering camera position and anchor point position to obtain viewpoint-related information; The aggregated features, the original anchor point features, and the view-related information are input into the multilayer perceptron; The three-dimensional Gaussian representation is obtained by predicting the position, covariance, color, rotation, and opacity parameters of the Gaussian elements based on the output of the multilayer perceptron.
[0013] Preferably, the process of calculating the loss function based on the rendered image and the input image includes: The three-dimensional Gaussian representation is projected onto the target viewpoint, and the RGB information of the rendered image is obtained through rasterization; Calculate L1 loss, structural similarity loss, and volume regularization loss based on the rendered image and the input image; The loss function is obtained by weighting the L1 loss, structural similarity loss, and volume regularization loss.
[0014] Preferably, the process of generating anchor control signals based on Gaussian meta-parameter gradients and scale information includes: The position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element are jointly judged according to a preset threshold. When the position parameter gradient is greater than the position gradient threshold, and the rotation parameter gradient is greater than the rotation gradient threshold or the scaling parameter is greater than the scale threshold, an anchor point supplementation signal is generated. A new anchor point is generated at the center of the corresponding voxel based on the anchor point supplementation signal.
[0015] Preferably, the process of jointly judging the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element according to a preset threshold includes: The position gradient threshold, rotation gradient threshold, and scale threshold are dynamically adjusted according to the level of the multi-resolution voxel mesh to obtain the dynamic threshold corresponding to each level. Based on the dynamic threshold, the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element are jointly determined to generate an anchor point control signal.
[0016] Compared with the prior art, the present invention has the following advantages and technical effects: This invention introduces an anchor point collaborative optimization mechanism based on graph neural networks. Under this collaborative mechanism, anchor points can interact with and fuse features with several surrounding anchor points, eliminating the need for independent optimization of Gaussian meta-parameters. This optimization mechanism improves the model's performance when fitting complex geometry and textures, ultimately enhancing the quality of 3D reconstruction and rendering effects. Furthermore, it enables anchor points to quickly reach consensus with surrounding anchor points when fitting a scene, reducing the number of trial and error attempts and significantly slowing down model convergence.
[0017] This invention employs an improved adaptive anchor point control strategy. Building upon the position gradient as the criterion for determining anchor point additions, it comprehensively considers information from three dimensions: the position gradient, rotation gradient, and scaling parameters of Gaussian units. This strategy refines the conditions for anchor point addition, enabling more precise control over anchor point density and avoiding the generation of numerous redundant anchor points due to misjudgments. This reduces model size and consequently lowers image rendering time during training. Furthermore, the method of this invention has been experimentally validated on both indoor and outdoor open-source datasets, demonstrating its feasibility and ease of implementation, thus possessing practicality. Attached Figure Description
[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the graph neural network anchor point collaboration module according to an embodiment of the present invention. Detailed Implementation
[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0021] like Figure 1 As shown, this embodiment provides a fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation, including: Based on the input multi-view images, the camera pose and sparse point cloud of the scene are obtained through the motion reconstruction structure algorithm, and the sparse point cloud is preprocessed to obtain normalized initial geometric information. Based on the initial geometric information, the three-dimensional space is voxelized, and anchor points are generated at the center of each voxel to obtain the spatial distribution of anchor points. An anchor point topology graph is constructed based on the spatial distribution of anchor points. The anchor point topology graph is then input into a graph neural network for message passing and feature aggregation to obtain the aggregated features of each anchor point. Based on the aggregated features and the original anchor features, the attribute parameters of Gaussian units are predicted by multilayer perceptron mapping to obtain a three-dimensional Gaussian representation. Multi-view projection and rasterization rendering are performed based on the three-dimensional Gaussian representation to obtain the rendered image. The loss function is calculated based on the rendered image and the input image. At the same time, anchor control signals are generated based on the gradient of Gaussian meta-parameters and scale information. The number and distribution of anchor points are dynamically adjusted based on the anchor point control signal, and the parameters of the graph neural network and multilayer perceptron are optimized by backpropagation. The above steps are repeated until convergence is achieved, thus completing the 3DGS three-dimensional reconstruction.
[0022] Furthermore, the process of obtaining normalized initial geometric information includes: SIFT feature points and feature descriptors are extracted from multi-view images, and sparse point clouds and camera parameters are obtained through feature matching and triangulation. Noise filtering is performed on sparse point clouds to remove outliers; The filtered sparse point cloud is scaled to obtain a normalized point cloud that is not affected by the actual size of the scene. The spatial extent of the 3D reconstruction is determined based on the normalized point cloud, and initial geometric information is obtained.
[0023] Furthermore, the process of obtaining the spatial distribution of anchor points includes: Based on the normalized point cloud and the preset voxel size, the voxel center coordinates are calculated by rounding down and deduplication operations to obtain the anchor point position; The set of basic anchor attributes is obtained by initializing the position attributes, feature vector, scaling factor, and learnable offset for each anchor point. Construct the spatial distribution of anchor points based on the set of basic anchor point attributes.
[0024] Furthermore, the process of constructing an anchor point topology map based on the spatial distribution of anchor points includes: Calculate the neighborhood offset coordinates based on the anchor point coordinates and voxel size to obtain a set of candidate neighbor coordinates; Based on all anchor coordinates stored in the hash table, query the anchors that exist in the candidate neighbor coordinate set to obtain the actual set of neighbor anchors; Construct an anchor point topology graph based on the connection relationship between the neighbor anchor point set and the current anchor point.
[0025] Furthermore, the process of obtaining the aggregated features of each anchor point includes: Based on the difference between the features of neighboring anchor points and the features of the current anchor point, feature difference information is obtained; The locations of neighboring anchor points are encoded to obtain location feature information; By concatenating and linearly transforming the feature difference information and the location feature information, the message transmitted from the neighboring anchor point to the current anchor point can be obtained. Aggregate the messages passed from all neighboring anchors to the current anchor to obtain the aggregation feature.
[0026] Furthermore, the process of aggregating the messages passed from all neighboring anchors to the current anchor to obtain the aggregated features includes: Calculate the inverse distance weight based on the spatial distance between the neighboring anchor points and the current anchor point, and obtain the weight coefficient of each neighboring anchor point; The weighted sum of messages transmitted by all neighboring anchors is obtained by weighting the messages according to the weight coefficients. The aggregated features are obtained by processing the weighted message sum using a nonlinear activation function.
[0027] Furthermore, the process of obtaining the three-dimensional Gaussian representation includes: Calculate the relative distance and direction vectors based on the current rendering camera position and anchor point position to obtain viewpoint-related information; The aggregated features, original anchor features, and view-related information are input into the multilayer perceptron; A 3D Gaussian representation is obtained by predicting the position, covariance, color, rotation, and opacity parameters of Gaussian elements based on the output of the multilayer perceptron.
[0028] Furthermore, the process of calculating the loss function based on the rendered image and the input image includes: The three-dimensional Gaussian representation is projected onto the target viewpoint, and the RGB information of the rendered image is obtained through rasterization; Calculate L1 loss, structural similarity loss, and volume regularization loss based on the rendered image and the input image; The loss function is obtained by weighting the L1 loss, structural similarity loss, and volume regularization loss.
[0029] Furthermore, the process of generating anchor control signals based on Gaussian metaparameter gradients and scale information includes: The position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element are jointly judged according to the preset threshold. When the position parameter gradient is greater than the position gradient threshold, and the rotation parameter gradient is greater than the rotation gradient threshold or the scaling parameter is greater than the scale threshold, an anchor point supplementation signal is generated. A new anchor point is generated at the center of the corresponding voxel based on the anchor point supplementation signal.
[0030] Furthermore, the process of jointly judging the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element based on a preset threshold includes: The position gradient threshold, rotation gradient threshold, and scale threshold are dynamically adjusted according to the level of the multi-resolution voxel mesh to obtain the dynamic threshold corresponding to each level. Anchor point control signals are generated by jointly judging the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element based on the dynamic threshold.
[0031] As a preferred implementation, this embodiment provides a fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation, including: First, the input image is processed using the Structure in Motion (SFM) algorithm to recover the camera position for each scene and generate a sparse point cloud. This sparse point cloud, after filtering and scale normalization, is used to determine the spatial extent for reconstruction and the initial geometric prior. A uniform voxel mesh is then constructed based on this point cloud, with an anchor point placed at the center of each voxel. These anchor points serve as surrogate and information aggregation units for subsequent Gaussian unit generation.
[0032] The anchor point map is then sent to the collaboration optimization module, such as... Figure 2As shown, this module uses a graph neural network (GNN) at its core, exchanging feature information between anchor points through neighborhood message passing: each anchor node receives aggregated features from its geometric or topological neighbors (edges are determined based on octagonal adjacency). After several rounds of updates, the node can fuse the perspective, texture, and geometric cues of its neighborhood, enhancing local consistency and the ability to infer unobserved regions. In the Gaussian parameter generation stage, the system concatenates the final node features of each anchor point with its own pixel alignment or view aggregation features, using this as input to a multilayer perceptron (MLP) to predict the parameter set of the corresponding Gaussian units.
[0033] The Gaussian primitives generated from all anchor points are input into the rendering pipeline of the 3D Gaussian splashing algorithm. These Gaussian primitives are projected onto the target viewpoint and rasterized to obtain the viewpoint image. During training, an adaptive anchor point strategy is introduced to dynamically control the number of anchor points.
[0034] Specifically, the method of this implementation includes the following steps: S1: First, the input multi-view images are processed using the Structure on Motion (SFM) algorithm. Then, the camera extrinsic parameters corresponding to each image are estimated. Next, a sparse 3D point cloud of the scene is generated through triangulation using feature point matching. Then, the generated sparse point cloud is subjected to noise filtering to remove outlier point pairs, and scale normalization is performed to ensure that subsequent processing steps are not affected by the actual size of the scene. The normalized point cloud is used to determine the spatial extent of the entire 3D reconstruction, providing preliminary spatial distribution and structural prior information for subsequent steps. S2: After obtaining the normalized point cloud, the 3D space is divided into a voxel grid using the voxelization method. The center position of each voxel is defined as an anchor point. These anchor points not only act as proxies for Gaussian unit generation, but also play a role in gathering information and providing a reference for subsequent feature aggregation and attribute prediction. S3: After establishing the voxel anchors, a local topological relationship graph between the anchors is further constructed. For each anchor, its spatially adjacent anchors are selected as neighbor nodes to form a topological structure. The topological structure graph is fed into the graph neural network as input, and through a multi-layer information transmission and aggregation mechanism, each anchor can fuse its own and its neighbors' feature information to generate an intermediate feature representation rich in local context. S4: The intermediate feature information generated by the anchor points is used together with the feature information of the original anchor points and input into the multilayer perceptron. Through the nonlinear mapping learning of the multilayer perceptron, the input features are transformed into information such as the position, covariance matrix, color, and rotation of Gaussian elements; S5: Project and rasterize the 3D Gaussian model constructed from anchor points from different perspectives, output the optical and depth information of each perspective image after forward rendering, complete the loss function supervision based on the RGB image information of the existing perspective, use L1 loss on the color of the rendered pixels, and combine SSIM and volume regularization loss to optimize the position, covariance and color attributes of each 3D Gaussian. The entire optimization process constitutes the training process of 3D Gaussian splashing, and during the training process, an adaptive anchor point control strategy is established to dynamically add new anchor points to the under-reconstructed region.
[0035] S6: During the training phase, the model continuously optimizes the number and spatial distribution of anchor points, and performs backpropagation optimization using the weight coefficients of the multilayer perceptron. After training, the model can be input with a set of images from different perspectives and camera positions, and can output the 3D information of the scene based on volumetric rendering.
[0036] Furthermore, the motion reconstruction algorithm in step 1 of this embodiment is as follows: For the input multi-frame image sequence, firstly, SIFT feature points are extracted from each frame image; then, feature descriptors are used to describe the local features of each feature point, and the similarity measure of the feature descriptors is used to achieve feature point matching of different images. During the incremental motion reconstruction algorithm reconstruction process, the camera shooting pose, camera intrinsic parameter matrix, and scene sparse point cloud of the input multi-frame images are obtained.
[0037] Furthermore, the method for establishing the voxelized mesh in step 2 of this embodiment is as follows: For each anchor point, four basic properties are introduced: the position of the anchor point. Anchor point characteristics Scaling factor as well as Learnable offset Anchor point location Initial point cloud obtained by voxelization get: in, This indicates the center point of each voxel region, i.e., the location of the anchor point. Indicates voxel size, This indicates a deduplication operation.
[0038] Furthermore, the method for aggregating anchor point features using a graph neural network in step 3 of this embodiment includes the following steps: S301: First, a local topology graph between anchor points is constructed using the K-nearest neighbor algorithm. For each anchor point First, the KNN algorithm is used to establish k nearest neighbor anchor points, thereby establishing an anchor point set. .
[0039] Specifically, considering that each anchor point is obtained through voxelization of the SFM point cloud, the anchor point itself possesses the characteristics of a voxel mesh. Leveraging this, this embodiment obtains the coordinates of the 26 surrounding anchor points by adding the anchor point coordinates to the voxel size offset. However, in some cases, the 26 surrounding positions may not be completely filled. To address this issue, this embodiment stores the coordinates of all anchor points in a hash table. When a neighboring anchor point is needed, the 26 offset coordinates are first calculated, and then the hash table is searched. If found, it means that this neighbor exists and is added to the hash table. If it is in the middle, then it does not exist otherwise. Calculation The formula for the coordinates of neighboring anchor points is as follows: in, Indicates anchor point The j-th neighbor, , In order to represent , In three-dimensional space, v represents the size of the voxel mesh.
[0040] S302: Will The topological graph formed The input is fed into a graph neural network to generate intermediate feature vectors. For the anchor set... One of the anchor points Feature information needs to be transmitted to the anchor point. .
[0041] Therefore, this embodiment uses the following method to transmit feature information: in, The anchor points are represented sequentially. and anchor points Its characteristics. This is used to highlight the feature differences between different anchor points and improve learning ability. This represents the position encoding function used to encode anchor points. The location information is mapped to a higher-dimensional 32-dimensional space, improving the representation capability of anchor point location information. The location encoding function is implemented using hash grid encoding, which can more efficiently represent local spatial features while maintaining low memory consumption. W is a learnable linear parameter.
[0042] In anchor point set Anchor points with different spatial distances should be... Different factors have different effects; therefore, this embodiment performs a weighted summation based on distance when acquiring information sent by neighboring anchor points to distinguish spatial differences. The specific weighting coefficients are: Thus, with anchor point A closer anchor point can have a higher influence weight, while a distance from the anchor point... The influence of more distant anchor points will decrease. All neighboring nodes can be aggregated and sent to the anchor point using the following method. Aggregated information : ; in, Represents aggregated messages The weighting coefficients, This represents a non-linear activation function; the aggregated information can then be obtained. .
[0043] Furthermore, the method for predicting the attributes of Gaussian elements using a multilayer perceptron in step 4 of this embodiment is as follows: Based on the aggregated information obtained in step 3 and anchor points eigenvalues They are fed together into an MLP used to generate Gaussian meta-attributes, for example, in the rotation parameters for generating Gaussian meta-attributes. When calculating, use the following formula: Where k represents the number of Gaussian elements controlled by the anchor point. This represents the coordinates of the current rendering camera. A similar method is used to generate the remaining properties of the Gaussian primitives, but different property parameters will correspond to different MLPs.
[0044] Furthermore, the anchor point generation control strategy and loss function in step 5 of this embodiment are as follows: This embodiment determines whether to generate anchor points at voxel centers by observing changes in the Gaussian gradient. During the optimization of Gaussian primitive parameters, the gradient of the position parameter of the Gaussian primitive is monitored. Rotation parameter gradient Scaling parameters Three conditions are used to determine the generation of anchor points. The specific conditions are as follows: ; in, , The values represent the averaged rotation gradient and scaling parameter, respectively. , This indicates the preset parameter threshold.
[0045] This embodiment quantizes the space into a multi-resolution voxel mesh to allow the addition of new anchor points at different granularity levels. As the voxel size decreases, the threshold for the judgment condition changes accordingly. The specific formula is as follows: ; ; in This indicates the level of voxel size. This represents the threshold value for the corresponding situation.
[0046] The overall loss function is as follows: ; This embodiment uses the structure-of-motion (SOG) method to obtain the camera pose of the input image and the sparse and dense point clouds of the scene as initial geometric information. A uniform voxel grid is constructed in 3D space based on the point cloud, and an anchor point is placed at the center of each voxel. These anchor points carry local contextual features. To enhance local consistency and information transmission, a graph neural network is used to propagate and aggregate features between anchor points, enabling each anchor point to not only rely on its own contextual information but also perceive the semantic and geometric cues of neighboring anchor points. Based on this, a 3D Gaussian splashing approach is adopted as the representation and rendering framework: neural Gaussian units are generated by anchor point proxies, and a conditional decoder maps anchor point features and viewpoint information to Gaussian position information, scale, color, and opacity. Unlike methods that rely solely on individual anchor point features to generate 3D Gaussian units without considering surrounding anchor point information, introducing neighborhood features significantly accelerates the convergence and fitting of Gaussian units. Furthermore, an adaptive anchor point control strategy reduces redundancy and improves rendering and training efficiency. This invention offers high-quality scene rendering, fast speed, and easy engineering implementation.
[0047] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A fast 3DGS three-dimensional reconstruction method based on graph neural network anchor point cooperation, characterized in that, include: Based on the input multi-view images, the camera pose and sparse point cloud of the scene are obtained through the motion recovery structure algorithm, and the sparse point cloud is preprocessed to obtain normalized initial geometric information. Based on the initial geometric information, the three-dimensional space is voxelized, and anchor points are generated at the center of each voxel to obtain the spatial distribution of anchor points. An anchor point topology map is constructed based on the spatial distribution of anchor points. The anchor point topology map is then input into a graph neural network for message passing and feature aggregation to obtain the aggregated features of each anchor point. Based on the aggregated features and the original anchor features, the attribute parameters of Gaussian units are predicted by multilayer perceptron mapping to obtain a three-dimensional Gaussian representation; Multi-view projection and rasterization rendering are performed based on the three-dimensional Gaussian representation to obtain a rendered image. A loss function is calculated based on the rendered image and the input image. At the same time, anchor point control signals are generated based on the position parameter gradient, rotation parameter gradient and scaling parameter of the Gaussian unit. The number and distribution of anchor points are dynamically adjusted according to the anchor point control signal, and the parameters of the graph neural network and multilayer perceptron are optimized by backpropagation. The above steps are repeated until convergence is achieved, thus completing the 3DGS three-dimensional reconstruction.
2. The method according to claim 1, characterized in that, The process of obtaining normalized initial geometric information includes: SIFT feature points and feature descriptors are extracted from the multi-view images, and sparse point clouds and camera parameters are obtained through feature matching and triangulation. Noise filtering is performed on the sparse point cloud to remove outliers; The filtered sparse point cloud is scaled to obtain a normalized point cloud that is not affected by the actual size of the scene. The spatial range of the 3D reconstruction is determined based on the normalized point cloud, and the initial geometric information is obtained.
3. The method according to claim 1, characterized in that, The process of obtaining the spatial distribution of anchor points includes: Based on the normalized point cloud and the preset voxel size, the voxel center coordinates are calculated by rounding down and deduplication operations to obtain the anchor point position; The set of basic anchor attributes is obtained by initializing the position attributes, feature vector, scaling factor, and learnable offset for each anchor point. The anchor point spatial distribution is constructed based on the set of basic anchor point attributes.
4. The method according to claim 1, characterized in that, The process of constructing an anchor point topology map based on the spatial distribution of anchor points includes: Calculate the neighborhood offset coordinates based on the anchor point coordinates and voxel size to obtain a set of candidate neighbor coordinates; Based on all anchor point coordinates stored in the hash table, query the anchor points that exist in the candidate neighbor coordinate set to obtain the actual set of neighbor anchor points; Based on the connection relationship between the neighbor anchor set and the current anchor, construct the anchor topology graph.
5. The method according to claim 1, characterized in that, The process of obtaining the aggregated features of each anchor point includes: Based on the difference between the features of neighboring anchor points and the features of the current anchor point, feature difference information is obtained; The locations of neighboring anchor points are encoded to obtain location feature information; By concatenating and linearly transforming the feature difference information and the location feature information, the message transmitted from the neighboring anchor point to the current anchor point is obtained. The aggregation feature is obtained by aggregating the messages passed from all neighboring anchors to the current anchor.
6. The method according to claim 5, characterized in that, The process of aggregating the messages passed from all neighboring anchors to the current anchor to obtain the aggregation features includes: Calculate the inverse distance weight based on the spatial distance between the neighboring anchor points and the current anchor point, and obtain the weight coefficient of each neighboring anchor point; The weighted sum of messages transmitted by all neighboring anchors is obtained by weighting the messages according to the weighting coefficients. The aggregated features are obtained by processing the weighted message sum using a nonlinear activation function.
7. The method according to claim 1, characterized in that, The process of obtaining a three-dimensional Gaussian representation includes: Calculate the relative distance and direction vectors based on the current rendering camera position and anchor point position to obtain viewpoint-related information; The aggregated features, the original anchor point features, and the view-related information are input into the multilayer perceptron; The three-dimensional Gaussian representation is obtained by predicting the position, covariance, color, rotation, and opacity parameters of the Gaussian elements based on the output of the multilayer perceptron.
8. The method according to claim 1, characterized in that, The process of calculating the loss function based on the rendered image and the input image includes: The three-dimensional Gaussian representation is projected onto the target viewpoint, and the RGB information of the rendered image is obtained through rasterization; Calculate L1 loss, structural similarity loss, and volume regularization loss based on the rendered image and the input image; The loss function is obtained by weighting the L1 loss, structural similarity loss, and volume regularization loss.
9. The method according to claim 1, characterized in that, The process of generating anchor point control signals includes: The position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element are jointly judged according to a preset threshold. When the position parameter gradient is greater than the position gradient threshold, and the rotation parameter gradient is greater than the rotation gradient threshold or the scaling parameter is greater than the scale threshold, an anchor point supplementation signal is generated. A new anchor point is generated at the center of the corresponding voxel based on the anchor point supplementation signal.
10. The method according to claim 9, characterized in that, The process of jointly judging the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element based on a preset threshold includes: The position gradient threshold, rotation gradient threshold, and scale threshold are dynamically adjusted according to the level of the multi-resolution voxel mesh to obtain the dynamic threshold corresponding to each level. Based on the dynamic threshold, the position parameter gradient, rotation parameter gradient, and scaling parameter of the Gaussian element are jointly determined to generate an anchor point control signal.